A/B testing of a logo: how to evaluate the effectiveness of variants
Why intuition is not enough
The marketing department often chooses a logo at a meeting — by voting of five people, one of whom clearly lobbies for their own taste. This is not testing, this is a lottery. Research by 597 of logos showed that the most memorable variants share one feature — simplicity: they have 13% higher probability of capturing consumer attention compared to more complex and less memorable competitors (data from Harvard Business Review). Such a pattern is not visible to the naked eye — it is only seen in a sample of people who have not encountered the logo before.
The test is not needed to confirm the designer's or client's correctness, but to find blind spots: what seems obvious to the team but is not perceived by a new person within three seconds of viewing.
Monadic or comparative: what is the difference
This is the first fork question before launching the test. Monadic testing shows each participant only one logo — without comparing it to competitors. Comparative testing shows several variants to the same person at once.
| Parameter | Monadic | Comparative |
|---|---|---|
| What it measures | Perception of the variant 'in a vacuum' | Relative preference among options |
| Accuracy | Higher, without contrast effect | Lower — the winner may be winning due to contrast, not its own qualities |
| Sample size | Requires more participants (one group per option) | Requires less — all options are seen by one group |
| Speed | Slower and more expensive | Faster and cheaper |
| When to use | Final choice between 2-3 strong candidates before launch | Early filtering at the stage when there are many options and some are clearly weak |
A reasonable scheme — first run 5-10 sketches through a comparative test, weed out the weak ones, and then check 2-3 finalists monadically to get a fair, not distorted by contrast, evaluation.
How many people are needed for the test
This is where people most often make mistakes: either they test the logo on ten colleagues and consider the question closed, or, on the contrary, they postpone the decision for months because "a larger sample is needed." The correct answer depends on what you want to get — a direction or proof.
- 20-30 participants — enough for a quick directed conclusion at an early stage, when you simply need to eliminate obviously failing options.
- 69 participants — the minimum sample size for a statistically significant result when comparing two options (90% confidence level).
- 86 participants — the minimum when comparing three options simultaneously.
- 10 variants — the upper limit by methodological standards; more than 3-4 logos in one test leads to participant fatigue and reduced quality of feedback.
Practical conclusion: at the draft concept stage, run quick surveys on 20-30 people and adjust hypotheses. Final decision between two-three variants that will go to the board of directors' presentation should be confirmed with a sample of 69-86 people — otherwise, a difference of 3-5 percentage points means nothing statistically.
What exactly to measure
"Like" is not a metric, it's a feeling without structure. Useful indicators are composed of several planes:
- Recognizability — whether the logo is recognized among similar ones within a few seconds of exposure.
- Memorability (recall) — whether people remember the logo after a day or two, without any hints.
- Differentiation — whether the logo stands out from competitors in the same category or blends into the general background.
- Associations — what words and emotions the logo evokes (reliability, cheapness, modernity, etc.).
- Attractiveness (appeal) — aesthetic evaluation, but always secondary to recognizability.
Separately, color is an important factor. A study published in Acta Graphica on a sample of 56 participants showed that among the factors 'color, shape, font,' only color had a statistically significant impact on perception — red logo variants were preferred over yellow. Shape and font did not produce a significant effect in this experiment. This does not mean that shape is unimportant in general — it means that when testing color, it should be checked in a separate block of questions, not mixed with the overall evaluation of 'like/dislike'.
How to select an audience for a test
A test on random people from a microtask marketplace will yield a nice chart and a useless result. The audience for a logo test should match real brand customers as closely as possible — in age, income, and frequency of contact with the product category. If the brand is premium and the respondents are students, the attractiveness rating will be systematically skewed toward 'bright and youthful'.
It is useful to split the sample into two segments: those who already know the brand (or a similar category), and those who see the logo for the first time. The former will give a reaction close to that of current customers, while the latter will reflect the reaction of a new visitor to the website or a product on the shelf. The difference between segments is often more informative than the average score across the entire sample.
Mistakes that devalue the test results
The most underestimated problem is familiarity bias, the familiarity effect. A new logo almost always receives lower ratings than the existing one, simply because the participant's brain is accustomed to the old version and perceives it as the 'correct' one by default. This distortion regularly leads to teams rejecting objectively better designs merely because they are 'unfamiliar.' If a test compares a new logo to an old one, the result should be interpreted not as a fact that 'the new one is worse,' but with a correction for this effect — often it is more useful to ask not 'which one do you like better,' but 'which one better conveys the idea of reliability/speed/premium quality,' bypassing the direct comparison with the familiar old image.
- Too many options in one survey (more than 4) — participants get tired and start answering formally.
- Mixing monadic and comparative approaches in one questionnaire without separating the data.
- Ignoring the familiarity effect when comparing the new logo to the old one.
- A sample that does not match the brand's actual audience by demographics.
- Making a decision based on a sample of 15-20 people as if it were a statistically significant result, rather than a directional signal.
Logo testing only works when the method matches the task: a quick survey of 20-30 participants is for filtering ideas, a sample of 69 people is for final decision-making, and any interpretation of the results must necessarily take into account that novelty itself reduces ratings, regardless of design quality.
Why intuition is not enough
The marketing department often chooses a logo at a meeting — by voting of five people, one of whom clearly lobbies for their own taste. This is not testing, this is a lottery. Research by 597 of logos showed that the most memorable variants share one feature — simplicity: they have 13% higher probability of capturing consumer attention compared to more complex and less memorable competitors (data from Harvard Business Review). Such a pattern is not visible to the naked eye — it is only seen in a sample of people who have not encountered the logo before.
The test is not needed to confirm the designer's or client's correctness, but to find blind spots: what seems obvious to the team but is not perceived by a new person within three seconds of viewing.
Monadic or comparative: what is the difference
This is the first fork question before launching the test. Monadic testing shows each participant only one logo — without comparing it to competitors. Comparative testing shows several variants to the same person at once.
| Parameter | Monadic | Comparative |
|---|---|---|
| What it measures | Perception of the variant 'in a vacuum' | Relative preference among options |
| Accuracy | Higher, without contrast effect | Lower — the winner may be winning due to contrast, not its own qualities |
| Sample size | Requires more participants (one group per option) | Requires less — all options are seen by one group |
| Speed | Slower and more expensive | Faster and cheaper |
| When to use | Final choice between 2-3 strong candidates before launch | Early filtering at the stage when there are many options and some are clearly weak |
A reasonable scheme — first run 5-10 sketches through a comparative test, weed out the weak ones, and then check 2-3 finalists monadically to get a fair, not distorted by contrast, evaluation.
How many people are needed for the test
This is where people most often make mistakes: either they test the logo on ten colleagues and consider the question closed, or, on the contrary, they postpone the decision for months because "a larger sample is needed." The correct answer depends on what you want to get — a direction or proof.
- 20-30 participants — enough for a quick directed conclusion at an early stage, when you simply need to eliminate obviously failing options.
- 69 participants — the minimum sample size for a statistically significant result when comparing two options (90% confidence level).
- 86 participants — the minimum when comparing three options simultaneously.
- 10 variants — the upper limit by methodological standards; more than 3-4 logos in one test leads to participant fatigue and reduced quality of feedback.
Practical conclusion: at the draft concept stage, run quick surveys on 20-30 people and adjust hypotheses. Final decision between two-three variants that will go to the board of directors' presentation should be confirmed with a sample of 69-86 people — otherwise, a difference of 3-5 percentage points means nothing statistically.
What exactly to measure
"Like" is not a metric, it's a feeling without structure. Useful indicators are composed of several planes:
- Recognizability — whether the logo is recognized among similar ones within a few seconds of exposure.
- Memorability (recall) — whether people remember the logo after a day or two, without any hints.
- Differentiation — whether the logo stands out from competitors in the same category or blends into the general background.
- Associations — what words and emotions the logo evokes (reliability, cheapness, modernity, etc.).
- Attractiveness (appeal) — aesthetic evaluation, but always secondary to recognizability.
Separately, color is an important factor. A study published in Acta Graphica on a sample of 56 participants showed that among the factors 'color, shape, font,' only color had a statistically significant impact on perception — red logo variants were preferred over yellow. Shape and font did not produce a significant effect in this experiment. This does not mean that shape is unimportant in general — it means that when testing color, it should be checked in a separate block of questions, not mixed with the overall evaluation of 'like/dislike'.
How to select an audience for a test
A test on random people from a microtask marketplace will yield a nice chart and a useless result. The audience for a logo test should match real brand customers as closely as possible — in age, income, and frequency of contact with the product category. If the brand is premium and the respondents are students, the attractiveness rating will be systematically skewed toward 'bright and youthful'.
It is useful to split the sample into two segments: those who already know the brand (or a similar category), and those who see the logo for the first time. The former will give a reaction close to that of current customers, while the latter will reflect the reaction of a new visitor to the website or a product on the shelf. The difference between segments is often more informative than the average score across the entire sample.
Mistakes that devalue the test results
The most underestimated problem is familiarity bias, the familiarity effect. A new logo almost always receives lower ratings than the existing one, simply because the participant's brain is accustomed to the old version and perceives it as the 'correct' one by default. This distortion regularly leads to teams rejecting objectively better designs merely because they are 'unfamiliar.' If a test compares a new logo to an old one, the result should be interpreted not as a fact that 'the new one is worse,' but with a correction for this effect — often it is more useful to ask not 'which one do you like better,' but 'which one better conveys the idea of reliability/speed/premium quality,' bypassing the direct comparison with the familiar old image.
- Too many options in one survey (more than 4) — participants get tired and start answering formally.
- Mixing monadic and comparative approaches in one questionnaire without separating the data.
- Ignoring the familiarity effect when comparing the new logo to the old one.
- A sample that does not match the brand's actual audience by demographics.
- Making a decision based on a sample of 15-20 people as if it were a statistically significant result, rather than a directional signal.
Logo testing only works when the method matches the task: a quick survey of 20-30 participants is for filtering ideas, a sample of 69 people is for final decision-making, and any interpretation of the results must necessarily take into account that novelty itself reduces ratings, regardless of design quality.
Recent articles
MegaFon, BMW, VTB and more 240+ companies. See how it looks in the showcase reel:
Our branding agency is approached to order:
Branding agency services
