With a Product test we verify that the product meets the expectations of the customer.
Although the core measure of a Product test is the acceptance level (preference) judged by product users, there is not one single way of designing a Product test and there are different ways to measure preference.
Before discussing an example of Product test, let’s briefly mention some important aspects of conducting product testing.
Product tests can be conducted in a lab or in the field.
-
- Product tests conducted in a lab can be with real product category customers, like mothers changing their child’s diaper, or can be done with instrumental analysis, like measuring resistance, stickiness, torsion, tension, etc. This latter approach is common to many products in the early stage of development.
- Product tests conducted in the field imply product usage or knowledge and are typically conducted in the form of a marketing research study.
Product tests that imply product usage can be conducted in a blind or identified manner.
-
- In identified product tests respondents know what brand they are judging.
- In a blind product test setup respondents are not aware of the brand name of the product they judge, in an attempt to reduce the bias due to influencing factors not directly related to the product.
A Product test can be monadic or sequential.
-
- In monadic test design respondents are presented with one product only, and they may be asked to judge it in comparison to a similar product they know or use.
- In sequential test design respondents are presented with two (or more) products and are asked to judge one product in comparison to the other.
Product tests can be one-off or ongoing.
-
- A one-off Product test, aka ad hoc test, is conducted once, and that’s it.
- An ongoing testing program involves several repetitions of the same data gathering process, using the same questionnaire and a comparable sample size. There are two kinds: Tracking and Panel studies. The difference lies in the respondents:
- Tracking studies recruit new respondents at every test wave;
- Panel studies interview the same people at every test wave.
Product tests apply to physical products as well as to immaterial services.
-
- Testing certain immaterial services may end up being complicated because services can be evaluated only after consumption.
- Tests of physical products can refer to the Product strategy in order to identify the characterizing elements of the product technical performance and design, the two areas where most product tests take place.
Try LogRatio’s fully automated solution for the professional analysis of survey data.
In just a few clicks LogRatio transforms raw survey data into all the survey tables and charts you need,
including a verbal interpretation of the survey results.
It is worth giving LogRatio a try!
Structure of product testing
The Product test is a very flexible marketing research tool applicable to very different product kinds, each with its own peculiarities. Therefore, defining a standard product testing layout isn’t easy. In many cases, however, product tests share a common design logic:
-
- Exploration of user habits and product usage
- User grouping
- Investigation of opinions and beliefs after testing the product
- User demographics
The formation of user groups is perhaps the most important part of the test because it requires a deep understanding of the logic governing the product consumption in a market.
Measuring preference with product tests
The fact that product tests are conducted to measure the preference of the customer suggests that we are looking for ways to grow the business. There are three major ways to find business growth:
-
- Convince non-users to buy us
- Switch customers from competitors to us
- Convince our clients to use more product
Each way refers to a specific type of customer. Therefore, when testing products for preference it is advisable to design a test that covers different customer profiles, hence user groups. This way, the test findings may support more business decisions concerning where our product can find room for growth.
Later on, this post explains the grouping process in more detail.
Segmenting usage habits to find growth
The test design is perhaps even more important than the test questions, because good questions in a badly designed test may ultimately be useless, while bad questions in a well-designed test can still supply useful information.
Testing Formula 1 tires
In October 2022 Pirelli conducted a blind, monadic Product test during the free training sessions of the F1 Grand Prix of Austin, Texas. The goal was to test a new tire formulation intended to reduce the difference in performance between soft, medium and hard tires, to level the time gap on a lap.
The blind form (drivers did not know what tire set they were using) was chosen to keep the driver expectations out of the judgment.
The test design split cars into two groups: full gasoline and low gasoline level.
There were four test phases:
-
- Usual driving conditions. All drivers drove the car equipped with the official tires for about one hour in common qualifying conditions.
- Drive test. Each car drove some 130 kilometers with the new hard tires (soft tires were tested during the next Grand Prix in Mexico and the Medium ones in Brazil).
- Personal interview. Each driver was asked about grip and driving feeling in comparison to usual drive conditions (monadic test).
- Telemetric data were analyzed primarily concerning tire temperature.
This kind of test, conducted only in the labs, wouldn’t have been enough to judge the effective performance sensed by the drivers. Some people say because “perception is reality”, meaning that oftentimes products are not what they are, products are what consumers think they are. In fact, the best product formulation is rarely also the market leader.
Testing washing machine detergents
In a Product test conducted among users of detergents for the washing machine, respondents were split into three major groups, according to the answers provided to the test section concerning the exploration of user habits and product usage:
In turn, each group was split into sub-groups.
Users of additives, for instance, were split into several sub-groups, as shown in the image that follows, and each sub-group was asked customized questions. The questions asked with this Product test can be found in the section “Questions for product testing” of this post.
Reading this test design in light of what has been written above about seeking business growth, it is intuitive how each group and sub-group may contribute to finding growth by convincing non-users, switching from competitors, and increasing client usage.
Growing the business should be the primary, hidden objective of every Product test.
Testing food taste
According to an example mentioned in Tull, the Dutch brand Iglo tested the intrinsic quality of frozen food with a panel of 1400 housewives divided into 9 geographic cells, in a monadic, blind test setup.
For each test, the respondents received the product at home, tried it in common usage conditions, and filled out a questionnaire of 5 questions:
-
- Likes and dislikes, open-ended
- Purchase intention, measured on a 7-point scale
- Price receptiveness, judged with the highest and lowest expected price
- Eating habits, relative to the tested product
The interesting thing is that this company assembled a panel of women to test products. This means that the tests were conducted every time with the same respondents, for a maximum of 10 tests a year in this case.
The advantage of using a panel of respondents is that conducting new tests is easier because the respondents already know what to do, they are reached easily, and the risk they misunderstand the instructions lowers with time. On the other hand, designing and administering a panel of respondents requires an ongoing effort to guarantee statistical accuracy.
Be it in the form of a panel, tracking, or ad hoc tests (once only), companies are recommended to subject their new products to the public judgment to not take for granted something consumers may disagree with.
Example of questions for product testing
Continuing with our example about detergents for the washing machine, the following describes the questions asked to the different groups of respondents.
Sources
Donald S. Tull, Del I. Hawkins (1993), Marketing Research Measurement & Method. Macmillian Publishing Company.
Mario van Hamersveld, Cees de Bont (2007), Market Research Handbook. John Wiley & Sons Ltd.
Louise S. Wu (1989), Product testing with consumers for research guidance. American Society for Testing and Materials.