Featurely Research · Customer behavior study

Featurely predicted real conversion rates without seeing the answers.

We tested it against 99 published results. Thirty-one predictions were nearly exact. The complete record is open below.

Featurely ResearchSeptember 10, 20266 min read

Featurely received only a description of each experience and its audience. It never saw the published result.

We recorded each prediction and then compared it with the published rate. The 99 checks come from 18 sources; some rows are different cuts of the same report. Every result, including the misses, is published below.

The test

99 checks. Featurely never saw the published rate.

What Featurely receivedThe experience and its audience

A description of what people saw and who the experience was built for.

What we checkedThe prediction against the outcome

Featurely’s answer was recorded before we compared it with the published result.

What the results showed

Across all 99 checks

31 of 99within 2 percentage points of the published rate
76 of 99within 10 percentage points
5 of 99more than 20 percentage points away

Difference is measured in percentage points (pp) between Featurely’s predicted rate and the published rate, rounded to one decimal place before counting. The full spread appears below.

One example

A Stripe-verified benchmark reported the trial-to-paid rate at SaaS companies with $500K or more in monthly revenue. Featurely said 20.3% of trials would convert. The published rate was 22.0%.

A near match · trial to paid

Companies at $500K+ monthly revenue

North Metric, Stripe-verified SaaS benchmark. One of 34 trial-to-paid checks.

Featurely20.3%
Published22.0%
Difference1.7pp

Featurely landed 1.7 percentage points under the published rate.

Read the published source

How the 99 results spread

Within 2 pp31
2 to 5 pp25
5 to 10 pp20
10 to 20 pp18
More than 20 pp5
Figure 1. Difference between Featurely’s prediction and the published rate, in percentage points (pp). Within 10 pp: 76 of 99 checks.

Results by customer journey

Average difference from the published rate, in percentage points

Lower is closer.

Email capture28 results · 26 within 10 points
3.1 pp
Trial to paid34 results · 25 within 10 points
6.4 pp
Checkout and purchase16 results · 12 within 10 points
6.7 pp
Landing and signup11 results · 10 within 10 points
7.4 pp
Activation and setup10 results · 3 within 10 points
16.3 pp
Figure 2. Mean absolute difference in percentage points. Counts are rows; several rows come from the same source report.

Strongest on email capture (average gap 3.1 points) and trial to paid. Weakest on activation (average gap 16.3 points, including the 36.5-point miss). Landing-page signup: 10 of 11 within 10 points.

How Featurely gets there

Featurely uses the experience and audience you provide to test an idea before a team builds it.

  1. Read the experienceFeaturely records what each screen asks and what it offers in return.
  2. Describe the audienceThe test captures who the experience is for and how ready they are to act.
  3. Build a simulated crowdFeaturely draws 1,000 people who match that audience.
  4. Let people move through the experienceEach simulated person continues or stops based on what they see and what matters to them.
  5. Show the likely outcomeThe team sees how many people are likely to complete the journey and why others may leave.

The same 99 checks, held fixed

The share of checks within 10 points rose from 27% to 77%.

Every product update was scored on the same 99 checks. Treat 76 of 99 as the current score on this set, not a fresh holdout.

Share of the 99 checks within 10 percentage points, across product updates The share rises overall, with plateaus and a small setback. Milestones show 27%, 37%, 66%, and 77% of the 99 checks within 10 percentage points. Share of checks within 10 points Product updates Start 27% within 10 points 37% within 10 points 66% within 10 points This study: 77% within 10 points, 31% within 2 points Start 27% 37% 66% 77% within 10 points 31% within 2 points This study
Figure 3. Share of the same 99 checks within 10 percentage points of the published rate, after each product update. Same checks throughout, so the line shows the score on this set, not performance on new cases.

The full record stays open

We include the strong results, the ordinary ones, and the misses. The largest miss was 36.5 percentage points. The published study used a stricter definition of “activation” than Featurely understood. That result remains visible below.

What this study shows

  • In this set, 31 of 99 predictions were within 2 percentage points of the published rate and 76 were within 10.
  • The 99 comparisons come from 18 source domains. Some are different cuts of the same report, not separate studies.
  • This study checks predicted conversion rates. It does not prove the exact improvement caused by changing a page.
  • The complete record below lets readers inspect the examples, sources, and misses.

Have a signup, pricing, or product decision to pressure-test? Talk to Featurely. Book a call with Featurely.

The evidence

See all 99 checks

The Prediction Ledger holds all 99 predictions with their published rates, sources, and misses. Rows from the same source report are grouped. Ten rows show at a time; use the filters to narrow.

99 results

No results match those filters.