Is there a way to add Confidence Interval for each point estimate of ROAS?

[bergerita]

I'm trying to explore a way to get Confidence Interval for each point estimate of ROAS or the inverse of it - CPA. In the output, each channel has a bell curve for ROAS. Ideally, we can output CIs for each channel's ROAS. But in the excel output file, there is only mean and total for CPA, but there is no standard error.
Wondering if we can have confidence intervals in the output as well. Or is there any argument for not showing confidence intervals but just point estimates?

[laresbernardo]

Hi @bergerita I believe this answer from another ticket regarding confidence intervals when using regularised regression also applies to this question. If we aren’t providing CI on the predictors due to regularization, we can't provide CI for ROAS since it is based on the transformations coming from there.
CC: @kyletgoldberg

[bergerita]

That answered my question. Thanks!

[gedkitchener-CT]

@bergerita Just want to throw something out there...

I take the following approach to get rough uncertainties on Robyn outputs (but warning: it's not well founded statistically speaking, but at least it's enough to give less technical stakeholders a range to work with)

1. Robyn provides a number of optimal models along the Pareto front which are 'non-dominated'
2. they are all 'correct' from a subjective perspective, depending on your relative importance on decomp.rssd and nrmse
3. to avoid biasing results to a subjectively selected single model, why not treat all of the optimal models as a 'sample of the truth' (this is the part that isn't correct, statistically speaking, but hear me out...)
4. each of the optimal models in the output will state a point estimate for contribution, theta, alpha, etc., and together these point estimates will make some 'empirical' distribution
5. if 100 optimal models are in the output, then by dropping the lowest 5 values and the highest 5 values, we get a noddy 90% confidence interval

Doesn't answer your question, and isn't strictly 'correct'. But like I said, it's good enough to give less technical stakeholders a range/interval to work with.

[bergerita]

Thanks @gedkitchener-CT for relating to my concern about providing point estimates from subjectively selected model(s). That’s very thought provoking. Will take your approach into consideration!