Reported cases does not reflect the setting of accumulate

[sbfnk]

Summary:
Reported cases generated by estimate_infections should reflect the full observation model, but the setting of the accumulate argument is ignored.

Description:
This is probably best fixed within the stan model - perhaps we should do the accumulation step separately from calculating the log-likelihood contributions.

Reproducible Steps:
The plot below highlights the issue, with reports looking to stay substantially below the data that is aggregated weekly.

library("EpiNow2")
#> 
#> Attaching package: 'EpiNow2'
#> The following object is masked from 'package:stats':
#> 
#>     Gamma
library("data.table")

reported_cases <- EpiNow2::example_confirmed
reported_cases_weekly <- data.table::copy(reported_cases)
reported_cases_weekly[, confirm := frollsum(confirm, 7)]
reported_cases_weekly <-
  reported_cases_weekly[seq(7, nrow(reported_cases_weekly), 7)]

est <- epinow(
  reported_cases_weekly,
  generation_time = generation_time_opts(example_generation_time),
  obs = obs_opts(na = "accumulate"),
  verbose = FALSE
)
#> Logging threshold set at INFO for the EpiNow2 logger
#> Writing EpiNow2 logs to the console and: /var/folders/n9/h_419gjj2mg3d208nplgvbg40000gp/T//RtmpbAAIyj/regional-epinow/2020-06-26.log
#> Logging threshold set at INFO for the EpiNow2.epinow logger
#> Writing EpiNow2.epinow logs to the console and: /var/folders/n9/h_419gjj2mg3d208nplgvbg40000gp/T//RtmpbAAIyj/epinow/2020-06-26.log
#> Accumulating modelled values that correspond to NA values in the data by adding them to the next non-NA data point. This means that the first data point is not included in the likelihood but used only to reset modelled observations to zero. If the first data point should be included in the likelihood this can be achieved by adding a data point of arbitrary value before the first data point.
#> WARN [2024-04-23 08:32:29] epinow: There were 13 divergent transitions after warmup. See
#> https://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
#> to find out why this is a problem and how to eliminate them. - 
#> WARN [2024-04-23 08:32:29] epinow: Examine the pairs() plot to diagnose sampling problems
#>  - 
#> WARN [2024-04-23 08:32:30] epinow: Bulk Effective Samples Size (ESS) is too low, indicating posterior means and medians may be unreliable.
#> Running the chains for more iterations may help. See
#> https://mc-stan.org/misc/warnings.html#bulk-ess -

plot(est)

^{Created on 2024-04-23 with reprex v2.1.0}

EpiNow2 Version:
1.4.9000 (current main)

[sbfnk]

This might be worth fixing for 1.5.0 as that introduces the accumulate option.

[sbfnk]

This might be worth fixing for 1.5.0 as that introduces the accumulate option.

Trying to do this turned out to be more complicated than originally thought in #643 thus pushing to a later version.

[sbfnk]

Continuing a discussion in #867 (comment) with the new set up (where missing data are filled with NAs) plots now look like this:

If we're happy with this we can close this issue in favour of a future plotting function that shows predictions on the scale of the data. As far as I'm concerned I'd prefer that to be the default for the top row but can see the continuity argument against.

[seabbs]

I don't mind strongly if its the default or not as long as there are enough options for people to be able to get at both and understand what they are looking at when they do so