Copy test

Copy test is the marketing research method used to identify the preferred advertisement by a sample of the target population the ad speaks to.

Every advertisement, be it a video, radio, online, billboard, or other kind of ad, relies on copy intended to convince the audience to take action in favor of the brand.

The copy of an advertisement is its content.

In print advertising, for instance, the copy is made of the headline, the body (text), images, captions, logo, brand name, and every other element an ad shows.

Copy tests are conducted to verify:

• • The impact of an advertisement as a whole is effective
  • How the single parts of an ad contribute to its impact

The measurement of the two aspects is commonly conducted with separate studies, using different methods. This post explains how to conduct copy testing of both kinds, including the formulation of the questions asked to customers.

Copy testing began almost a century ago. During this time many different approaches have been experimented with and a lot of literature has been written. To date, no one single Copy test method can be considered the ‘right’ one. In general, copy testing aims to select the ad execution with the greatest potential to attract the attention of the audience and make them act according to the objective of the campaign.

Copy tests are of two major kinds, depending on the phase of the communication campaign the ad belongs to:

• • Pre-test. This form of copy test is conducted before the campaign is aired. Typically, Copy tests are run at the beginning and end of the creative process and at the end of the production process. Qualitative research is often used at the very first stage. Mockups, animatics, and the like are used thereafter.
  • Post-test. This type of copy test is conducted after the campaign is aired. Of course, it tests the ad used for the communication campaign.

Dozens of agencies offer copy testing of different kinds, some more successful than others.

If you intend to work on copy testing with an agency, be prepared to invest a budget ranging from $50k to $150k or more.

On the other hand, if you would like to try the do-it-yourself (DIY) copy testing approach, read on. This post discusses the key aspects of conducting a professional DIY-Copy test for a fraction of the price charged by marketing research agencies.

Try LogRatio’s fully automated solution for the professional analysis of survey data.
In just a few clicks LogRatio transforms raw survey data into all the survey tables and charts you need,
including a verbal interpretation of the survey results.
It is worth giving LogRatio a try!

What to test

Before testing an ad, we must understand what to test.

Advertising is used to grow the business, and the temptation may be strong to set sales growth as the objective of the campaign. The problem is that sales growth is the end-result of the interaction of many variables, and isolating the effect of advertising is difficult. An exception would be campaigns aimed at generating immediate action, like calls, purchases, inquiries, leads, and the like. In general, however, the effect of advertising on sales tends to extend to the long-term. For this and other reasons, only in rare cases should sales be the overall objective of advertising campaigns.

Copy testing that grows the business

Business growth can be found elsewhere, and there are three main objectives one could consider:

• • Convince new customers to try our product
  • Make existing customers loyal, exclusive
  • Convince existing customers to use more of our product

All three options may lead to increased sales, but sales are not the measured variable.

When the desired customer behavior is chosen, we turn to the process intended to elicit that behavior. Different processes elicit different results. For instance:

• • New users could be won with an advertising campaign aimed at increasing brand awareness among non-users who, as a result, could decide to try our brand. The variable to be measured in this case is the level of awareness among the population the ad targets (Option A in the table that follows).
  • Loyalty could result from a campaign that keeps high levels of brand awareness among existing customers who, when shopping, could remain loyal to their familiar brand (Option B).
  • Increased product use could result from a campaign that informs existing customers of new and appealing product features in an attempt to make them use our brand more frequently and/or in larger quantity (Option C).

Growth can be attained in different ways.

The ultimate function of a communication campaign is to activate the response variable that generates the desired action.

In order to validate the effectiveness of an advertisement, a copy test must measure the response variable linked to the campaign success.

Finding out for our brand which variable causes which reaction in the customer group usually is a trial-and-error process that costs money and takes time. On the other hand, however, this learning process creates so-called reference values of the expected results of a campaign, which can be a great decisional help.

This post explains how to identify and measure response variables that generate action.

 

How to create your own Copy test

Copy testing took wind in the 1950’s with the increase of TV advertising driven by the growing presence of TV sets in private houses. At that time Burke introduced the Day-After-Recall (DAR) to identify commercials with the potential of attracting the interest of the audience. The huge success of the DAR method led to new, less expensive Copy test designs. Among others, the AC-T test and the BUY test made history. Today, a whole new bunch of methods to test the appeal and effect of commercials are being developed. From eye-tracking and facial coding systems, to biometric measurement of emotions, EEG to measure excitement and engagement, and much more.

Like all innovations, the new copy testing approaches need time to become fully validated and broadly (and affordably) applied. The method we have found to be more affordable to DIY copy testing users is (a simplified version of) the BUY test.

The following explains how to measure the effectiveness of an ad as a whole.
Thereafter, we discuss how to measure the contribution of the elements of an ad to its effectiveness.

As with most marketing research studies, there may be filter questions at the beginning and demographic questions at the end of the questionnaire. The latter questions are used to slice the data, and are a fundamental element of the analysis of copy testing results because they help to extract all information in it, for instance, using cross tables like those you can make with LogRatio.

Copy testing the whole advertisement

The concept behind the BUY test postulates that commercials, judged by a sample of their target population, showing high levels of recall, involvement, and persuasion reach high levels of the BUY score. A large BUY score underlines strong potential to attract the interest of the audience and persuade them to take action in favor of the advertised brand.

Strong potential to attract the interest of the audience = (Recall + Involvement + Persuasion)

How to test recall

Recall is what a viewer remembers of an ad, and it can be tested in a variety of ways.

The original BUY test asks about:

• • Evidence of exposure to the ad (awareness)
  • Paying back some content of the ad (memorization)

A positive answer to both recall questions below (including either 2a or 2b) is the minimum requirement to conduct a meaningful test.

After showing each tested ad we could ask, for instance:

How to test involvement

Involvement takes place when we see or hear something interesting that we like.

Positive responses to at least two of the three items that follow signal audience involvement with an ad:

How to test persuasion

The ultimate persuasive effect of an ad is that of making the viewers switch to the advertised product instead of their commonly used brand.

Testing persuasiveness requires at least two exposures to the ad, especially emotional commercials which build response more slowly than rational ads. Testing persuasion could run this way:

• • The audience (the respondent sample) views the test ad (or ads) together with other unrelated ads presented during a half-hour TV show (in the case of print it could be a magazine)
  • Demographic questions are asked, including the preferred brand
  • The respondents view the show a second time

Then we ask:

Questions that predict sales

Determining which variables measure test effectiveness more accurately in a Copy test may take time and several surveys. Previous studies have found the dis-/agreement with certain statements may be predictive of future sales (hence ad effectiveness).

The statements in the next table are measured on dis-/agreement using Likert scales with 5 or 7 answer classes.

The agreement with the statements above can be gathered with a battery of questions where the respondents expresses dis-/agreement on each statement. The results of each statement could be weighted and transformed into comparable indexes.

Keep reading, we are going to explain how.

How to derive an effectiveness index

There are different ways to merge the answers to a survey question into a comparable index.

The next table, for instance, shows the percent results to a liking question concerning an ad as a whole.

The Liking index of Area 1, 26%, was obtained by multiplying each percent value by the respective weight as follows:

[(16.3% * 2) + (51.2% * 1) + … + (9.3% * -2)] / 2.

Such an index favors the comparison between subjects as well as between variables. In this case it refers to respondents from different geographic regions. In Area 1 the ad reached the highest level of appreciation, and the lowest in Area 3.

The same concept applies to compare results from different measures, as shown with the next table. Reading survey results expressed with indexes has the advantage of making evident the differences between respondent groups or other segmentation criteria. But there are also disadvantages, given the fact that averaging values may deliver unrealistic results. Reading the cross tables reduces the risk of interpreting averaged data in the wrong way.

LogRatio creates the cross-tables necessary to calculate comparable indexes applying weights as described above.

It is easy to recognize the advantage of working with indexes. Surveys like Satisfaction studies, Copy test, Product test, and others make much use of indexes in order to facilitate the reading and comparison of survey results. In fact, there are many ways to create Comparable indexes. Common methods used with survey data include weighting scales like the one above and its variations.

In general, an index can be obtained with:

Where:

x = The frequency values (%) of a cross-table

w = Weights

Weights too can take different forms. In the example above, we used +/-2 with a 5-class scale. In other instances, it could be +/-1 or +/-3 or other form, depending on the size of the answer scale.

Alternatively, it could be an all positive weights, like +3,2,1,0, for example to remove the effect of negative answers in an unbalanced scale.

Copy testing the individual parts of an advertisement

When the campaign is running, tracking surveys are commonly used to measure the impact.

A Tracking study is a survey repeated several times, using the same questionnaire and study design, where the respondents rotate at each study wave. Comparing the results of several tracking survey waves shows how the results vary over time. This is a common approach to understand how the campaign affects a brand and how effectively an ad communicates.

In general, tracking the effectiveness of advertisements relies on several variables, such as:

• • Awareness
  • Brand image
  • Brand preference
  • Opinions and beliefs

Tracking studies questionnaires can easily get long and tiring because most communication campaigns cover several media channels, and each channel requires its portion of questions.

You will find an example of a tracking study questionnaire in the next section of this post. The objective of the study should suggest the appropriate questions.

Questionnaires like the following one require extended analysis, including a long list of cross tables. Together with the trends resulting from the repeated administration of the survey, tracking studies represent an extremely valuable monitoring tool for decisional purposes.

 

Questions of a tracking survey, an example

In a tracking survey concerning the advertising effectiveness of campaigns regarding washing machines, the questionnaire covered the following sections:

 

A note of caution

Designing and conducting a Copy test or other sort of survey research is relatively easy. Designing and conducting a Copy test that is also valid and reliable is a different story. Does our Copy test really measure the effectiveness of an advertisement? That is, is it valid? And, is it reliable? In other words, does it measure the same thing every time? Answering these questions helps ensure that the test delivers what it promises.

Copy test validity

The validity of a Copy test depends on whether it measures what the advertisement is trying to achieve. And what an ad is supposed to achieve is determined by its operational objective. Is it sales? Or is it loyalty, or awareness, or increased product usage, or attitude towards the brand, or improved brand image, or is it new customers that the test is meant to achieve? Setting a clear and measurable communication objective is a key requirement to conduct copy tests whose effectiveness can be validated.

High levels of the following two measures have been able to predict the effectiveness of an ad with a certain accuracy:

• • Persuasion has often predicted a sales increase due to an ad in a reliable way. It can be obtained using a differential semantic scale and asking respondents, after viewing the ad, to rate the brand as a whole with a score ranging from poor to excellent. Persuasion is said to be a more stable measure than unaided brand recall.
  • Salience, hence top-of-mind recall, is the number of times a brand is cited first in unaided brand awareness asked for the product category.

When the operational objective of a campaign is clear, it is possible to validate the effectiveness of the test.

Copy test reliability

In order to be reliable, a test must measure the same thing every time it is conducted.

Test reliability is commonly verified with the test-retest approach. Repeating the same test, in the same conditions, at two distinct moments and obtaining results that are compatible and not statistically different confirms the test reliability.

 

Sources

David A. Aaker et al., 11th ed. (2013), Marketing research. John Wiley & Sons, Inc.

Alvin J. Silk (1977), Test-Retest Correlations and the Reliability of Copy Testing. Journal of Marketing Research, Vol. 14, No. 4.

Lyman E. Ostlund, Kevin Clancy (1982), Copy Testing Methods and Measures Favored by  Top Ad Agency and Advertising Executives. Journal of the Academy of Marketing Science, Vol. 10 No. 1.

(1982) Pact: Positioning Advertising Copy Testing. Journal of Advertising