In conversion rate optimization (CRO), we often hear the mantra “simpler is better.” Streamline the form. Remove friction. Cut the clutter. But what happens when simplicity doesn’t deliver the results you expect?
This A/B test found that removing an urgency question increased submissions by 4% but resulted in less qualified leads.
Quick takeaways
- We compared the original application form with a version that removed its urgency question.
- Removing it increased submissions by 4%, but high-income applicants decreased, show-up rates dropped by 4%, and cancellations rose.
- The results suggest that the question acted as a qualifier, so form submissions alone were not enough to judge the test.
Setting the Stage: The Application Funnel
One of our clients runs a high-ticket program. Their funnel is straightforward:
- Webinar: Visitors watch an in-depth session explaining the program.
- Application Form: Interested viewers fill out a form to request a call.
- Booking a Session: Qualified leads book a call with the team.
It’s that application form (the bridge between interest and action) where we focused our experiment. The form contained several questions, including one about urgency: “How urgent are you feeling about getting the tools and support to step into your ‘…’ and build a thriving ‘us against the world’ ‘…’?”
At first glance, it seemed unnecessary. Did we really need to ask this? Was it adding value, or just making the form longer than it needed to be?
The Hypothesis: Would Removing a Question Improve Conversions?
Here’s what we believed:
- Shorter forms convert better. Every CRO expert has said it. It’s backed by plenty of studies.
- Urgency questions might add pressure. Could this question make some users uncomfortable?
So, we hypothesized: Removing the urgency question will increase form submissions.
Our reasoning was simple:
- Fewer fields = lower friction.
- Less perceived pressure = higher comfort = higher conversion.
If we could achieve even a modest increase in applications, we’d fill the pipeline with more leads and hopefully more paying clients.
Test Design
We ran a classic A/B test:
- Control: The original application form (with the urgency question).
- Variant: The same form, minus that one question.
Test size: 4,210 participants
Duration: 5 days
Primary metric: Form submission rate
Secondary metrics: Show-up rates, call cancellations, and demographic quality indicators (like household income).
The Results: A Win That Wasn’t
On the surface, the results were promising. Form submissions increased by 4%. Success, right?
But when we dug deeper, the story changed.
High-income applicants decreased. Fewer people from top income brackets submitted the form in the test group.
For a practical guide to examining experiment results by audience segment, see metric breakdowns in PostHog test results.
Show-up rates dropped by 4%. Even among those who booked calls, fewer actually attended.
Cancellations increased. Both client-initiated and user-initiated cancellations rose in the variant group.
The people filling out the form weren’t the people we wanted. In other words: We got more leads but they were less qualified.
This was the big question: Why would removing one simple field cause such a shift in lead quality?
Our theory:
- The urgency question acted as a qualifier. It may have encouraged only the most committed leads to move forward.
- It created a subtle sense of commitment. When users reflected on how soon they wanted a solution, it made the process feel more serious.
- High-value leads are more willing to answer tough questions. Those in higher income brackets may not mind the extra step if they’re truly interested.
It turns out that this “friction” wasn’t friction at all; it was a filter.
What This Test Taught Us
This experiment reminded us that conversion rate alone isn’t enough. It’s tempting to celebrate higher numbers, but quantity without quality doesn’t grow a business.
Here are our biggest takeaways:
- Measure more than one metric. Always define secondary metrics (lead quality, show-up rates, revenue impact). A higher submission rate is meaningless if revenue doesn’t increase.
- Not all friction is bad. Some questions qualify leads and encourage commitment. Removing them can backfire.
- Dig deeper before declaring a winner. At first glance, the test seemed like a success. But analyzing downstream behaviors told a different story.
Conclusion
This A/B test was humbling. We thought removing a single “unnecessary” question would boost results. Instead, it compromised lead quality and reduced show-up rates.
In CRO, more isn’t always better. But sometimes the question you think is slowing people down is actually helping you find the right people.
So, next time you’re tempted to simplify your forms, ask yourself: What am I really optimizing for: quantity or quality?
Frequently asked questions
What happened when the urgency question was removed?
Across 4,210 participants over five days, removing the question increased form submissions by 4%. The same variant attracted fewer high-income applicants, reduced show-up rates by 4%, and produced more cancellations, so the additional applications did not represent an unqualified business win.
Why might an urgency question improve lead quality?
It can ask applicants to reflect on readiness and reveal how soon they want to act. In this case, the team’s interpretation was that the question functioned as both a qualification signal and a small commitment, but the experiment did not directly test that psychological mechanism.
Why was the 4% increase in submissions not enough to declare a winner?
Form completion was only the first step toward a valuable sales conversation. When attendance, cancellations, and applicant quality moved in the wrong direction, the higher submission rate no longer described the outcome the business actually wanted.
How can you tell useful form friction from unnecessary friction?
Ask what the field contributes after submission. A field is purposeful when its answer informs qualification, routing, sales preparation, or commitment; it is probably waste when it repeats known information, is never used, or adds effort without changing any decision.
Should every high-ticket application include an urgency question?
No. The value depends on the offer, sales process, wording, and whether the answer predicts a meaningful downstream outcome. Test it in the actual funnel, and avoid language that manufactures pressure instead of measuring genuine readiness.
Which metrics should an application-form test monitor?
Track submissions together with qualified-lead rate, booked and attended calls, cancellations, purchases, and revenue when those data are available. Those downstream guardrails show whether a form change creates more opportunities or merely more records for the sales team to process.
Does this experiment prove why lead quality changed?
It provides evidence that removing this one question affected the funnel in this test population, because that was the intended difference between the variants. It does not prove whether reflection, commitment, income mix, or another mechanism caused the downstream pattern, and the article does not report equal certainty for every secondary metric.
How can you simplify a form without losing qualification signals?
Keep the information that changes a real decision, then reduce effort around it. Shorter wording, well-designed answer choices, conditional follow-ups, prefilled known data, and better integration between booking and application tools can make the experience easier without discarding the signal. For a separate test of reducing form steps while preserving the flow, see our form-merge CRO case study.
Need help running a test on WordPress, Shopify, or ClickFunnels? Get started with our platform-specific step-by-step guides: [WordPress, Shopify, ClickFunnels].
