01
WatchWe turned off our best channel to find out if it was real
A holdout that moved her budget, a scorecard that cost her a senior buyer, and how she told the board growth would look slower.
Dana Reyes ran growth at a top-50 shopping app for six years and has spent fourteen in mobile. Over 45 minutes she walked through three calls that reshaped how her team spent, measured and talked about growth.
1. Switching off the channel attribution loved
The setup. “One network was getting credit for about a third of our installs. Every dashboard said scale it. But our organic installs dipped whenever it spent more, and rose when it paused over a holiday weekend.”
The call. “I asked for a holdout: no spend in two matched regions for four weeks. My CFO loved it, my UA team hated it. I agreed in writing to call it off if purchases in the holdout regions fell more than ten percent.”
What happened. “Purchases barely moved. Most of what that network was credited with would have happened anyway. We moved most of the budget into two channels that held up under the same test, and our cost per incremental purchaser fell by about a third.”
2. Changing what the team was paid on
The setup. “The test was the easy part. My buyers were bonused on installs at a target CPI, so the scorecard still rewarded exactly what we’d just proved was half fiction.”
The call. “We moved the scorecard to incremental purchasers, measured quarterly by holdouts. Two senior buyers pushed back hard. One left. I don’t regret the change, but I regret how fast I made it.”
“If the scorecard rewards the old number, the team will find a way to keep producing it.”
3. Telling the board growth would look slower
The setup. “Cutting fake installs makes the topline chart dip. I knew the board would see the dip before it saw the savings.”
The call. “I went in first, with the holdout results and a two-quarter view: installs down, purchasers flat, cost per purchaser down. No surprises. The board meeting after the dip was the shortest I ever had.”
What she’d do differently
“Run the first test a year earlier. I waited for certainty, and certainty only comes from the test. And tell the team why before the numbers come in, not after.”
01