Following the eec announcement of the Render Rate, some have asked why? Who cares? And what makes the Render Rate better than the other proposed metric—Action Rate—or vice versa?
As outlined in the previous post, How Opens Are Tracked and Reported, there are a large number of scenarios where an open might be tracked. At the most basic level, there are two methods of tracking opens:
1) A unique tracking image inserted into the email was loaded: confirming that the images in the email were rendered.
2) A link in the email was clicked: since obviously a link cannot be clicked if the email was not opened, this method allows us to track some of the people missed by the first method.
Render Rate is calculated using only the first method, while Action Rate is calculated by combining the unique results of both methods.
Advantages of Render Rate
Render Rate has the advantage over Action Rate in two ways:
1) Pre-conversion testing: At the end of the day, most marketers want to look at conversion of some sort. Even so, evaluation of pre-conversion dynamics can be helpful in optimizing a program. Because Render Rate measures only one thing, it is better suited for testing in pre-conversion testing scenarios. If we want to look at the ability of a subject line to get people to look at the email, The Render Rate only measures that one thing. If we use Action Rate to evaluate subject lines we are actually testing two different things: 1) which subject line got more people to view the email with images, and 2) which subject line got more people who didn’t view the email with images to click. Since Action Rate would evaluate success based on two different criterions, it is difficult to determine what the subject line actually does best. A better approach is to use Render Rate to answer the question, “Which subject line got more people to look at my email?” and use Click Through Rate to answer the question, “Which subject line gets more people to click on a link?”
One could take subject line testing a step further by analyzing the results in two stages – first look at which subject line had better reach, and then we can look at which subject line eventually drove more people to the landing page. Then we would optimize the landing page to further increase conversions (e.g. sales) in its own tests. In theory, optimizing each step in the process will increase overall conversions, but there are exceptions, so a final test using conversions from emails delivered is recommended.
Render Rate also improves testing of creative elements in email because it limits both the control and the test groups to only a version of the email where we could be sure that images were seen by all subjects in both groups. When addressing the question, “Which creative is most effective at getting people to click?” conduct the test based on the people who we know saw the creative (as measured by renders) and then look at that segment to determine which creative version got the higher percentage of people to click through. This type of evaluation is impossible using Action Rate since by definition, anyone who clicked also opened, which inaccurately implies 100% effectiveness of the creative.
Limiting to the render rate only means that both the control and the test group see the same thing. Some examples of tests – types of imagery used in pictures, call to action placement (preview pane or below), button vs. text links. This can be particularly important to those that write emails that have few links in them (or even just one) – where the placement of that link, whether it be a feedback method or the call to action itself moves based on image rendering.
2) Ad impressions: Many advertisers sell display advertising in email. Render Rate provides accurate measures for how many of those ads were displayed. Since it limits the value of the metric to specifically image based opens (renders) it gives us the true number of total impressions using the total emails rendered. Keep in mind, for image-based advertisers, there is no value of an email for which no images are rendered—Action Rate would provide an artificially inflated view of performance. We could also use the Render Rate to determine reach and frequency in a specific email. Here the unique number of email renders is the reach and the total number is the frequency. When combined with monetization via ad revenue we can construct a traditional direct marketing RFM model.
Of course, Render Rate has its challenges, which is why a second metric has also been proposed.
Advantages of Action Rate
Action Rate has the advantage over Render Rate in two ways:
1) Less Inaccurate Measure of “Opens”: Render Rate underreports the number of people who seemingly looked at your email more than Action Rate. However, Action Rate still underreports this number. In reality, no reliable method for determining how many subscribers actually looked at your email exists. Both proposed metrics are estimates. Both are low estimates. Action Rate is simply a slightly closer estimate of the number of people who presumably looked at your email.
Under reporting is one of the primary reasons the eec Measurement Accuracy Roundtable recommends a move away from the term Open Rate. Unfortunately, this term has left many marketers with an incorrect understanding of what the metric provides. Worse still, “Open Rates” are calculated different ways by different ESPs—thus the need for standardization.
2) Text-based advertising: Again using the publishing realm as a basis of example, publishers often need to provide reports on text-based advertising. For this purpose, Action Rate is a better standard metric for the reasons listed in the prior section. The Action Rate, which reports, “opens” based on either the rendering of a tracking pixel or a user action (i.e. a click) we can better judge the true number of ad impressions based on action rate. Currently, to our knowledge there is no other industry metric that is defined and standardized that accomplishes this goal.
Use Both to Gain Additional Insights
Finally, using both the Render Rate and the Action Rate together we can learn more about our subscribers and the ways they use emails. For example if you see a regular reader appearing in the list of subscribers that typically views emails with images on and then suddenly drops off the list of those that fit the render rate definition and starts showing up on the action rate only, this may indicate that the subscriber has begun reading their emails with a mobile device. It may also be that they have not added you to their white list, or another set of circumstances may have occurred. The point here, is that using them both we can identify little differences in subscriber behavior that may alert us that it would be worth it to request more feedback from that reader or set off a trigger email to them.
The members of the Measurement Accuracy Roundtable think that if you do adopt our proposed metrics – whether via your ESP or in your own email deployment and reporting in-house system that you will find the Render Rate, the Action Rate and their associated metrics to be superior to current methods—which are inconsistent and sometimes misleading. For those of you that are already defining your open metrics using the Render and Action Rates we congratulate you on being ahead of the times and welcome your support on this initiative and help to spread the metrics throughout the industry so that all email marketers and our stakeholders can have clearer understanding on how our programs work and their effectiveness.
– Luke Glasner, Rodman Publishing & Morgan Stewart, ExactTarget