This is another test focused on the videos used in our client’s sales page.
This time, we’re trying to understand how our audience reacts to videos of different lengths. In fact, this experiment gives us valuable insights into user behavior and helps us optimize content structure based on those learnings. For a concise overview of how experiments assign variants and measure outcomes, see PostHog’s experiment documentation.
Quick Takeaways
- We compared a 10-minute video with heavier edits and background music against a 6-minute version with lighter editing.
- To keep the test fair, both groups used the same 2:28 countdown timer, rather than the original 5:40 setting tied to the longer video.
- Over 24 hours, 20,796 users participated, and the 6-minute version had a 100% probability of being the best variant.
- The shorter video communicated the personal-finance-control message more directly, while the longer version provided more editing and a deeper explanation.
Hypothesis
To learn more about our audience, we decided to test different versions of the same video; each with a different length. Our hypothesis was that the test results would provide meaningful insights for future content strategy.
Countdown Timer Note
On this particular sales page, a countdown timer starts as soon as the user opens the page. In the original setup, the timer was set to 5:40, which matched a strategic moment in the 10-minute video. At that point, the rest of the sales page would load if the viewer was still engaged.
However, for the purpose of this experiment, we needed to standardize the experience. Since we couldn’t set separate timers for each test group, we shortened the timer to 2:28 for both the control and test group. This change ensured fairness across versions, though it deviated from the original design.
Test Design
We tested two versions of the video sales letter (VSL):
- Control: A 10-minute video with heavier edits and background music.
- Test: A 6-minute version with lighter editing.
Both versions delivered the same core message: encouraging viewers to take control of their personal finances by joining the community. Importantly, they both moved away from the previous angle of “increased support for travel hacking.” It’s also worth noting that the shorter version communicates this new angle more directly, while the longer version includes more editing and a deeper explanation of the message.
Results
The experiment ran from Friday, Oct 18, 2024, at 6:48 PM UTC, and ended 24 hours later. PostHog’s running time and sample size guidance explains how traffic and effect size shape test duration. During this time, 20,796 users participated. Before relying on the result, our conversion tracking audit explains the checks that make funnel data trustworthy.
The test version (6-minute video) performed significantly better, with a 100% probability of being the best variant. PostHog’s experiment metrics documentation explains how metric configuration and credible intervals are interpreted.
Conclusion
While both videos followed the same structure and messaging angle, the shorter 6-minute version, which focused more clearly on personal finance control, outperformed the longer version. For another case study in which a simpler version won, see our Pre-Lander Quiz Experiment.
This aligns with our previous findings: simplicity, clarity, and a focused message drive better results.
Heavy editing and longer run-time don’t necessarily improve conversion and in this case, they may have diluted the core message. Our Growth Lever Analytics guide connects experiment results to the business decisions that follow.
Simplicity wins. A focused message, delivered clearly and efficiently, can outperform a highly produced version, especially when the audience is early in the funnel.
Read this PostHog article to better understand how to set up A/B/n testing. PostHog’s A/B testing best practices are a useful companion when planning the test.
Also, check out these articles to learn how to set up A/B test on the following platforms. [WordPress, Shopify]
Frequently asked questions
What did the video experiment compare?
It compared a 10-minute video with heavier editing and background music against a six-minute version with lighter editing and a more direct explanation of the same core personal-finance message. Both groups used the same 2:28 page-unlock timer.
How much better did the shorter video perform?
Across 20,796 participants in 24 hours, the six-minute variation converted at 7.36% versus 2.94% for the control. PostHog reported a 100% probability of the shorter variation being the best performer in that experiment.
Does this test prove that shorter videos convert better?
No. Length changed together with editing intensity, background music, pacing, and how directly the message was delivered. The winning package was shorter and clearer, but this test cannot assign the lift to duration alone or predict the ideal length for another offer.
Why did both videos use the same countdown timer?
The rest of the sales page appeared after a timer, and the platform could not assign a separate unlock time to each video. Setting both groups to 2:28 prevented different page availability from becoming an additional between-group difference, although it also changed the original experience built around the longer video.
What may have driven the shorter video’s advantage?
The result may reflect faster delivery, a clearer personal-finance angle, lighter production, different pacing, or an interaction among them. Those are explanations to test next—not findings measured separately by this experiment.
How could a follow-up test isolate video length?
Start from one script, voice track, visual style, music choice, CTA, and page-unlock behavior. Create a shorter cut by removing only preselected material, then confirm that the remaining claims and sequence are unchanged before comparing outcomes.
Which metrics belong in a video sales-letter test?
Use the final opt-in or purchase as the decision metric, then inspect plays, progress milestones, completion, CTA exposure, exits, and revenue as diagnostics. Those measures show whether the shorter version wins because more people start, fewer abandon, or the offer becomes available at a better moment.
How should you choose the right video length?
Use enough time to explain the promise, evidence, objections, and next step without repeating them. Audience awareness, offer complexity, traffic temperature, and viewing context matter more than a universal minute target, so test coherent versions rather than cutting solely to reach a fashionable duration.
