Machine-Learning Send-Time-Optimization

Introduction

ML STO helps you optimize message send time to the best hour of the day for each person.

ML STO uses machine learning (ML) models based on recipient engagement to determine the optimal delivery time for each recipient. Using ML STO helps you to optimize send time to the best hour of the day for each person in your audience, which should help to improve opens, clicks, and overall engagement.

Note: ML STO is an optional feature that must be enabled in your account. Please speak with your Customer Success Manager for more details.

Get a quick glance on the feature with this video snapshot.

How does ML STO work?

ML STO finds the best hour to email each person based on their email opens and clicks. For example, if a person tends to open and/or click at a particular time each morning, their optimized send time will be just before that observed best time. This leaves time for the email to arrive in their inbox right when they are most likely to engage.

ML STO finds the best hour by using Machine-Learning (ML) models. These models are fully-managed and automatically scheduled to run weekly, as part of the Marigold ML service. All the technical details of loading the data and administering the models are managed "under the hood", freeing you up to focus on the big picture.

STO Hour

Each person has an "STO Hour", which is their best hour of the day to send them an email. Each person is different, so one person might typically engage with your emails at 6am, while another prefers 11pm. This is recalculated each week, to stay current as behavior changes.

Popular STO Hours

Popular STO Hours are the most common STO Hours across all your recipients. For example, most brands show spikes in engagement at time ranges of the day. Popular STO Hours are used when an STO Hour cannot be determined for a specific person. People without enough opens or clicks will be assigned an STO Hour from one of the Popular STO Hours. This is reassigned each week, so eventually an email will reach the inbox at the right time.

Popular STO Hours may also be used for day part sends if a person's STO Hour falls outside the selected day part send time.

Day Part STO Hours

This is an optional feature within ML STO. Contact your Customer Success Manager to have this feature enabled for you.

Day part STO Hours split the day into two periods using a 24-hour clock: STO AM and STO PM:

STO AM hours run from 01:00 to 12:00
STO PM hours run from 13:00 to 24:00

Each person has an STO AM hour and an STO PM hour. These are similar to the STO Hour, but apply to their respective day part instead of the full 24 hours. In most cases, a person's STO Hour is also used for either their STO AM or STO PM hour. The other hour is assigned from one of the Popular STO Hours.

This allows you to use ML STO within a part of the day. You can choose STO AM if you want to send in the morning, or STO PM for the afternoon/evening. Or, you can send one campaign with STO AM and a second with STO PM.

The day parts are configurable, for example, you could limit them to your business hours or typical sending window. Contact your Customer Success Manager to have this changed for you. Changes to the day-parts will take effect on the next weekly ML STO scoring run.

ML Control Group

ML STO maintains a control group, made up of a random selection of your recipients. The control group allows you to measure ML STO results against a base group of people who receive their messages at the default time. The ML Control Group is "static", meaning someone selected for the control group will stay in there, so that results can be compared over time.

Using ML STO

Engage+ provides ML STO for Regular One-off and Date-triggered Email Campaigns.

ML STO is available on the Review tab of the campaign, in your Send Time Optimization options.

In the Review tab of your campaign, select the "ML STO" tab to enable Send Time Optimization.
From the drop-down menu, select “STO Hour” to send at the best hour of the day for each recipient.
Set a "Default Sending Time". Recipients will receive your campaign at the default time if their STO Hour is empty. This includes the ML Control Group.

Note that the drop-down and Default Sending Time may be pre-selected for you by default, but it is the ML STO tab selection that enables/disables ML STO.

Set ML STO for a Day Part

Note: This feature must be turned on for you; please contact your Customer Success Manager for assistance.

When this feature is enabled, two additional choices will appear in the dropdown: STO AM and STO PM.

ML STO runs on a 24-hour clock, generally selecting the best hour out of all 24 to send your messaging. You can also select a day part, which covers either the morning hours (01:00 to 12:00) or the afternoon and evening hours (13:00 to 24:00).

You can send all messages in one day part, or you can schedule two different sends of the same email – one for STO AM and one for STO PM.

To set a send for a day part:

In the Review tab of your campaign, select the "ML STO" tab to enable Send Time Optimization.
Select STO AM or STO PM from the STO Hour drop-down menu.
Set a "Default Sending Time". Recipients will receive your campaign at the default time if their STO Hour is empty. This includes the ML Control Group.

A/B Testing

If desired, you can test the effectiveness of ML STO with an A/B test as follows:

Run two versions of the same campaign, sending one with ML STO and one without.
The audiences should be a random sample a/b split from the main audience.
Compare the engagement metrics for each campaign to see which performed better.

ML STO Tips

Schedule to start early

Set the Send Schedule as early in the day as possible, such as 1am or 2am.

If the campaign is scheduled to send after a person's ML STO best hour, they will receive the message at their best hour on the following day.
Schedule your campaign to send as early in the day as possible. This will ensure each person gets their best hour on the correct day.

Default to a normal sending time

Set the Default Sending Time to a typical sending time such as 8am. Recipients will receive your campaign at the default time if their STO Hour is empty. This includes the ML Control Group.

The ML scores are generated weekly for each email address in your recipient table.
If you need to send a campaign with newly imported recipients then be sure to set a Default Sending Time in case those recipient email addresses are new and haven't yet been scored.
The Default Sending Time is also used for people in the ML Control Group, since they do not have an STO Hour.

Multiple Campaigns per day

If you use ML STO for multiple campaigns per day, they will be sent at the same time. This is because the STO Hour has a single value per person.

If you are sending two campaigns, you can use the optional day-parts for STO AM and STO PM so the two campaigns send at different times. (Be sure to set a different Default Sending Time for each campaign.)
Note that the STO Hour will probably match either STO AM or STO PM, so that does not provide a third time slot.