About certainty of lift

Certainty of lift, available for Brand Lift, Search Lift, and user-based Conversion Lift, helps you assess how reliable your lift results are. It represents the likelihood that the amount of lift was driven by your campaigns and not due to chance.

How it works

Certainty of lift is calculated as 1 - p-value and can sometimes be referred to as the “statistical significance” or the “confidence” of lift results. The p-value shows how likely your lift results would be if the ads weren’t driving real impact on your business goals. A high certainty, corresponding to a low p-value, indicates that results are unlikely to have happened purely by chance. High certainty is a strong indication that your ads generated lift.

Understand levels of certainty

Google tries to collect enough data to detect lift with the highest certainty (90%), but lower certainty results can still be helpful in making advertising decisions. In your Google Ads account, you can find lift results and their associated certainty for all studies with a measurement certainty above 50%. Results below 50% aren’t reported because they wouldn’t be statistically strong enough to be useful and trustworthy.

Note: Certainty of lift isn’t available for all accounts. If you don’t have it in your account, you can only find results with high certainty (>90%).

How to interpret and use results with certainty of lift

The following table gives general guidance on how to interpret lift results with different certainty of lift. Note that the certainty of lift is rounded down by increments of 5% for simplicity. You can use the table for guidance, but it is recommended that you interpret the results based on your business needs and risk tolerance.

Certainty of lift	Interpretation
≥90%	There's a very good chance that your results were caused by your ads and not due to randomness. It would be very unlikely for ineffective ads to generate data this positive by pure chance.
70-90%	There's a good chance that your results were caused by your ads, but there's also a chance that the results were due to randomness. It would be somewhat unusual for ineffective ads to generate data this positive by pure chance.
50-70%	There's a moderate chance that your results were caused by your ads, but there's also a moderate chance that the results were due to randomness. You should use these results directionally as it's possible they happened by chance.
"No lift"	Results with a certainty below 50% are reported as "no lift" in Google Ads. This doesn’t necessarily mean that your ads were ineffective. However, the results of measurement need further confirmation to increase our certainty that any lift observed was not due to chance. We recommend setting up a new test in this case.

It’s important to note that Lift Measurement (such as Brand Lift, Search Lift, and Conversion Lift) aims to detect lift at the highest certainty of 90%. Showing lower certainty results doesn’t degrade the quality of results, but allows you to get data points that would otherwise be unavailable.

Understand low certainty

Low certainty results don’t always mean ads are ineffective. These results can help you get insights that should be further verified. A low certainty of lift means the study wasn't able to detect the lift with high confidence. This can happen either because not enough survey responses were collected (for Brand Lift, for example) or because the lift is low.

The study hasn't collected enough data to detect lift: Low survey counts (below 4100 responses) for brand lift, low query count for search lift, and low conversion counts in conversion lift leads to low measurement power, which means you’re more likely to get a low certainty of lift. Additionally, you will have lower survey or search counts per segment when segmenting your results, for example, by age, gender, or campaigns. You can use remeasurement to increase survey responses, search volume, or conversions.
(Brand Lift only) The data from the study has low lift: When the amount of lift is below 2%, it’s hard to detect with high certainty. However, a low absolute lift isn’t necessarily an indicator of poor performance. Campaigns with low absolute lift can still outperform on cost per lifted user (CPLU).

Compare segment-level lifts with different certainty of lift

It’s likely that different segments, for example different age groups, will have a different certainty of lift. You shouldn’t conclude that the segment with highest certainty of lift is the best performing segment.

First, note that you will need to choose a metric by which to compare segments, depending on your optimization goals. recommended to use absolute lift or CPLU (not lifted users) for Brand Lift, or Relative Lift for Search Lift to compare segments. Second, note that lift performance between different segments or keywords is often similar (high overlap of confidence intervals) which makes it hard to clearly identify the best segments. Note however that the lower the certainty of lift, the higher the chance that the measured performance could be due to noise. If you have multiple segments with similar performance, it can be wise to select the ones with highest certainty to minimize risk. If you’re unsure how to identify the best performing segments, please ask your account representative.

Understand confidence intervals and levels

When referring to the lift of an ad, people usually refer to the lift “point estimate” which is the most likely lift generated by the ad. However, in Google Ads, you can also find a confidence interval for all lift metrics which is an estimated range in which your result could fall.

You can also find a confidence interval, which is an estimated range in which your result could fall. This range is defined by an upper and lower bound which are the highest and lowest values where your lift is likely to actually be. Lift results use 80% 2-sided confidence intervals, which means that there is an 80% chance that the true lift is between the lower bound and upper bound . This also means that you have a 90% chance that the lift is greater than the lower bound.

Example: You may view that your relative lift is 35%, which is the point estimate. You can also see that the confidence interval goes from 30% to 40%, which means that there is an 80% chance that the true lift is between 30% (the lower bound) and 40% (the upper bound). Another way to look at this is that there is a 90% chance that lift is greater than the 30% (the lower bound).

Note that when the certainty of lift is smaller than 90%, the lower bound of the confidence interval will be smaller than 0 because Google can’t guarantee with more than 90% certainty that the lift was positive.

Frequently asked questions

Can I choose my own minimum certainty of lift

No. Results are always shown if their certainty is above 50%. If you want a higher limit (like 80%) discard any results that don’t meet your limit. It’s not possible to set a limit lower than 50%.

How can I increase my certainty of lift?

Certainty of lift depends on the accuracy of the measurement.

To increase certainty of lift for Brand Lift, the following is recommended:

Use remeasurement to increase survey collection.
Measure ad recall or awareness to determine the highest chance of getting lift with high certainty.
Talk to your account manager to select ad campaigns with high lift.

To increase certainty of lift for conversion lift, the following is recommended:

Include all eligible campaigns in the study. Otherwise, people in the control group may still be shown some of your advertising, diluting the results.
Focus on conversion actions that are most directly influenced by your advertising.
- Shallow conversions may be easier to lift than deep ones.
- Conversions with a lot of activity can be noisy and difficult to see lift on.
Increase the size of your study by increasing the study duration up to 56 days.
Start a new study with a higher conversion volume.

To increase certainty of lift for Search Lift, the following is recommended:

Choose keywords closely related to your ad, and avoid using generic keywords.
Use remeasurement

How can I learn more about how certainty of lift is calculated?

There’s always some natural randomness in the amount of data you get, which can fluctuate the results.This is often referred to as "random measurement noise". This random noise may lead to measuring no lift despite the ads creating lift in reality, or positive lift despite the ads not creating lift in reality. The p-value quantifies how likely a measured lift can be due to noise if the ads did not generate lift. If you have a very low p-value, it’s very unlikely that the lift measured was a result of random noise, and it’s certain that the ad campaigns caused lift.

The certainty of lift is calculated as 1-p-value and expressed as a percentage. The higher this number is (and therefore the lower the p-value), the more certain that the ads caused lift.

Example 1: Ad shows 5% absolute lift, p-value = 0.01: This means that there is a 1% chance of seeing 5% lift due to random measurement noise. This leads to a high certainty (99%) that an ad caused lift.

Example 2: Ad shows 5% absolute lift, p-value = 0.35: This means there's a 35% chance of seeing 5% lift due to random measurement noise. This leads to a low certainty (65%) and is not reliable enough evidence that the ad caused lift.

Why would I experience low certainty of lift in segments?

When slicing data by segment, for example, by campaigns, each segment only has a subset of the data. Because the individual segments have fewer data points than the overall study, it’s expected to be harder to detect lift with high certainty.

Note that if one segment has more reach than another, for example, if one campaign has more budget than another one, it may collect more data and is likely to have a higher certainty of lift, even though the lift for that segment could be smaller. Detecting lift is most challenging on segments with the smallest reach.

Where can I find the certainty for each in segments?

In the lift report there is an expandable table below the charts. You can review the table to find all your lift metrics, including the certainty of lift.

Where do I find confidence intervals?

Confidence intervals are included on lift graphs and can be found by hovering over lift results in tables. The graphs allow you to inspect how much uncertainty there is in the measurement. Additionally, when comparing segments this allows you to quickly inspect if the confidence intervals of two segments overlap. The higher the overlap the less certain you can be that one segment is better than another. On all graphs, the confidence intervals are capped at 0 but the exact values can be found by hovering over lift results in tables.

Was this helpful?

How can we improve it?