Run an impact estimate for secure signals (Beta)

Only available in Google Ad Manager 360.

To help publishers understand the revenue impact of their active secure signals, they can run an impact estimate that disables those signals on a small percent of traffic (known as a "holdback").

Jump to a section of the article:

Overview
Set up an impact estimate for secure signals
How impact estimates for secure signals work
Review the results
How to interpret the results

Overview

There are two types of impact estimates for secure signals:

Individual secure signals: This type lets you measure the incremental value of individual signals. Each signal in the experiment has a variant that disables the signal, while leaving all other signals active.
All secure signals: This type lets you measure the overall value of all signals. Use this option to learn the aggregate impact of all active secure signals by running a variant that disables all of them.

Example

Consider two signals (S1 and S2). Together, you get +10% uplift from both S1 and S2, but individually they both give you +8% uplift. The options would work as follows:

Individual secure signals: With this option, you'd learn that (a) you get 10% uplift from S1 and S2 together, and (b) you get 8% uplift from S1 and S2 individually. The baseline is enabled, and we have 3 variants: S1 disabled, S2 disabled, and both S1 and S2 disabled.
All secure signals: With this option, you'd learn that together you get 10% uplift from both S1 and S2. The baseline is both enabled. The variant is both disabled.

As the baseline for secure signals impact estimates has the signal(s) enabled, and the variant has the signal(s) disabled, experiments that show a statistically significant negative number indicate that secure signals in the experiment are driving a positive revenue lift.

Set up an impact estimate for secure signals

You can set up an impact estimate for your active secure signals.

Sign in to Google Ad Manager.
Click Optimization, and then Impact estimates.
On the "Secure signals" card, click New impact estimate.
Next to "Estimate options," select a type of impact estimate to run:
- Individual secure signals
- All secure signals
Next to "Secure signals," click the menu and select up to 5 signals for the impact estimate.
Make your other selections for Demand channel settings, Estimate variants, and Delivery settings. Learn more about setting up impact estimates.
Click Run, and then Confirm.

How impact estimates for secure signals work

Each variant will run on 1% of traffic for the network. You can choose up to 5 signals to measure at a time. If using all 5, the impact estimate will run on a total of 6% of traffic (1% baseline, and 1% for each signal).

The baseline is the variant used to compare the treatment against.

Variant	Type: All secure signals Measures overall value of all signals	Type: Individual secure signals Measures incremental value of a signal
Baseline	All signals	All signals
Treatment	No signals	All signals except A
Treatment	No signals	All signals except B
Treatment	No signals	All signals except C
Treatment	No signals	All signals except D
Treatment	No signals	All signals except E

Note that "All signals" refers to only the signals you enabled. The impact estimate does not enable any signals that you didn't include.

Review the results

You can review your impact estimates, whether ended or still running. The impact estimate results show the impact to revenue, eCPM, and impressions. The results for each variant can be broken down by Operating System category, Browser category, Device category, Demand channel, Demand subchannel, and Bidder.

Sign in to Google Ad Manager.
Click Optimization, and then Impact estimates.
For an overview of your impact estimates, review the table columns:
- Impact estimate: Shows the type of impact estimate you selected
- Estimate period: Shows the time period you selected for the estimate to run.
- Status: Shows whether the estimate is scheduled to run, running, or complete.
Under "Impact estimate," click the name of a secure signals impact estimate.
Review the details. You can make the following selections:
- To view results by a different time frame, next to "View metrics by," click the dropdown and select an option, such as "Weekly."
- Under "Variant," review the results for revenue, eCPM, and impression lift. If there are multiple variants, click the variant you want to review. The breakdown table will update to show data for the selected variant.
- To break down the results, next to "Breakdown," click a dimension, such as "Demand Channel."
- To add a filter, such as for bidders, click Add filter and make your selections.
  Learn more about the filters.
- To view a detailed chart for each row in your breakdown, click .
(Optional) To end a running impact estimate, click End impact estimate, and then Confirm.

How to interpret the results

The results shown in impact estimates consider all non-house impressions served for your network. By default, the results can include guaranteed and non-guaranteed ad server line items, as well as your programmatic demand across all environments. This can help identify potential shifts across your demand channels as a result of using secure signals.

About the filters

The following filters are available: Bidder, Browser category, Device category, and Operating system category.

To limit the scope of your analysis to demand channels, bidders, and environments that receive secure signals only, the "Bidder" and "Browser category" filters are especially useful.

To apply a filter, next to , click Add filter and select a filter.

The Bidder filter

Examine results for the relevant Ad Exchange, Open Bidding, and SDK Bidding demand channel only. Use the Bidder filter to remove Google demand (Google Ads and DV360) from Ad Exchange, as secure signals are not sent to Google buyers. To apply the Bidder filter with these settings:

Select is none of.
Enter "Google Ads" and "DV360," and click Apply.

The Browser category filter

Choose an option:

To limit your results to web inventory:
1. Select is none of.
2. Next to "Not applicable (App)," check the box and click Apply.
To limit your results to app inventory:
1. Select is any of.
2. Next to "Not applicable (App)," check the box and click Apply.
To focus your results on specific browsers, such as Safari or Chrome:
1. Select is any of.
2. Next to one or more browser names, check the box and click Apply.

Other considerations

You can also consider the following details when interpreting results for secure signals impact estimates:

As all non-house impressions are considered, what might be a +10% revenue lift for a demand channel such as Open Bidding in isolation may be less statistically significant in aggregate. Given that Open Bidding is a smaller segment, it might only translate to a +2% overall increase without any breakdowns applied.
Again, as the experiment variants disable secure signals, if you see a negative number in the experiment results, it means the variants without secure signals performed worse than the group with them. In other words, negative results indicate secure signals are driving performance that would otherwise be lost.

What should I do if my impact estimates do not indicate statistical significance?

If your experiment results are not statistically significant, you can try the following:

Focus on relevant demand: As noted, scope your analysis only to the demand sources most affected by secure signals.
Run experiments longer: More data helps reveal smaller, real effects. Experiments can now be run for 90 days.

Note that it is also possible that a secure signal does not lead to a statistically significant impact on performance.

Was this helpful?

How can we improve it?