Key facts
- Split one list into two random halves, change only the subject line, send both at the same times, and compare replies rather than opens.
- Test one change at a time and wait until each half has enough sends to show a real difference.
- Buzzlist runs subject tests inside Buzz Send, and the Scoreboard shows who replied.
- Open counts are unreliable because many mail apps load images on their own, and Buzz Send uses no open tracking.
Why replies and not opens?
Open counts are unreliable because many mail apps load images on their own, and Buzz Send uses no open tracking. Replies are the result you want anyway.
How many sends do I need?
More than most people think. With small lists, a difference of one or two replies is luck. Run the test across a few hundred people before you decide.
What should I test?
- Their company name against a topic.
- A question against a statement.
- Two words against five.
How to run a fair subject test
- Write two subjects
Change one thing only.
- Split at random
Not by region or alphabet.
- Send at the same times
So send time does not skew the result.
- Compare replies
Keep the winner, then test the next idea against it.
When does Buzzlist fit, and when does it not?
It fits when
- You send to a few hundred people or more a month.
It does not fit when
- You send to 20 people a month. Write each subject by hand instead.
Questions people ask next
Can I test the email body too?
Yes, but test one thing at a time so you know what made the difference.
How long should a test run?
Until each half has had all four emails, so late replies count.
Should I use emojis in subjects?
For cold business email, subjects without emojis work better.
Does Buzzlist pick the winner?
Buzzlist runs subject tests in Buzz Send, and the Scoreboard shows what was sent and who replied.