A two-minute post-registration schedule produced a higher reported webinar-join rate than a wait of up to 15 minutes, but the surviving article contains two incompatible lift claims. Its narrative says “nearly 300%,” while its own transcription of the result image gives 1.01% versus 0.70%—about 44% relative lift. Until the original metric and counts are reconciled, neither percentage should be presented as the verified effect.
The operational question
The funnel used StealthSeminar to place registrants into the next available session. Under the default setup, the delay could be anywhere up to 15 minutes. Treatment used a custom JavaScript and API integration to offer a start approximately two minutes after registration. The historical test ran for eight days and intercepted more than 22,000 users.
The intended chain was: assign the experience, find the next session, submit registration with its start time, then record whether the registrant entered the webinar. “Webinar Joined” was the primary metric. The current article does not provide per-arm assignments, registrations or joins, so it cannot establish whether the denominator was every intercepted visitor, every exposed visitor or only completed registrants.
Why the reported lift is unresolved
| Surviving claim | Control | Treatment | Implied comparison |
|---|---|---|---|
| Narrative/caption | Not stated | Not stated | Nearly 300% higher |
| Result-image transcription in article | 0.70% | 1.01% | About 44% relative; +0.31 percentage points |
These statements are not equivalent. A different denominator, metric, time window or screenshot could explain the discrepancy, but the public record does not say which. Choosing 44% because it can be calculated from rounded rates would still be guesswork about whether those rates belong to the promised join metric.
What the experiment can support today
The available record supports a directional operational conclusion: the shorter-schedule experience had the higher displayed join rate in this funnel. It does not support “engagement skyrocketed,” a universal two-minute rule, or the claim that impatience was the measured cause. Losing focus, encountering a technical problem or misunderstanding the schedule are hypotheses.
This test is also different from the waiting-room experiment. Here the intervention changed a minutes-long session schedule after registration. The other test changed a seconds-long entry experience and bundled page elements. One result cannot validate the mechanism of the other.
Implementation checks for a shorter schedule
A safe integration needs explicit handling for time zones, unavailable sessions, API latency, duplicate submissions and a fallback when session lookup fails. Record assignment before the scheduling request, distinguish integration errors from user drop-off, and ensure the same person is linked from exposure through registration and join without publishing personal data.
Measure watch time, key content milestones, completed applications and purchases as separate downstream outcomes. A higher join rate can be valuable, but it does not show that additional attendees watched enough of the presentation or became customers.
Evidence needed before restoring an exact lift
Recover the experiment definition, metric query, time window and assigned-user and join counts for each arm. Reconcile the screenshot with the narrative and state the denominator plainly. Until then, cite the rates only as conflicting historical artifacts and use this page as an integration retrospective rather than a quantified winner story.
For metric design, use the A/B testing metrics guide and PostHog’s current experiment-metrics documentation. Current StealthSeminar API behavior must be checked before reusing any historical implementation detail.
