Error thrown with variational algorithms

[marcus-waldman]

Versions: R (4.4.0), brms (2.21.0), cmdstandr (0.7.1), Rtools43, Windows 11 Pro

I'm not sure if this is an issue that only I am having or if others are as well, but I get an error thrown if I fit any of the variational algorithms. For example,

 brm(
  count ~ zBase * Trt + (1|patient),
  data = epilepsy, family = poisson(),
  prior = prior(normal(0, 10), class = b) +
    prior(cauchy(0, 2), class = sd), 
  backend = "cmdstanr", 
  algorithm = "laplace", 
  draws = 1
)

results in the following error thrown:

Error in `[.data.frame`(diagnostics[[i]], rstan_diagn_order) : 
  undefined columns selected

Note that the estimation algorithm appeared to proceed normally:

Initial log joint probability = -6966.6 
    Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes  
Error evaluating model log probability: Non-finite gradient. 
      99       -652.23     0.0197727       18.9707           1           1      132    
    Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes  
     199      -650.988     0.0146375        7.2476           1           1      242    
    Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes  
     299      -650.797     0.0126656       2.16591           1           1      350    
    Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes  
     399      -650.687   0.000298552      0.171569      0.5594      0.5594      458    
    Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes  
     400      -650.687   0.000290087     0.0601614           1           1      459    
Optimization terminated normally:  
  Convergence detected: relative gradient magnitude is below tolerance 
Finished in  0.2 seconds.
Calculating Hessian 
Calculating inverse of Cholesky factor 
Generating draws 
iteration: 0 
Finished in  0.1 seconds.

I showed the code setting algorithm = laplace , but the exception is also thrown if I set to algorithm = pathfinder or algorithm = meanfield, as well.

If I use algorithm = sampling, no error is thrown.

All 4 chains finished successfully.
Mean chain execution time: 3.2 seconds.
Total execution time: 13.4 seconds.

 Family: poisson 
  Links: mu = log 
Formula: count ~ zBase * Trt + (1 | patient) 
   Data: epilepsy (Number of observations: 236) 
  Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
         total post-warmup draws = 4000

Multilevel Hyperparameters:
~patient (Number of levels: 59) 
              Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(Intercept)     0.59      0.07     0.47     0.74 1.00      872     1709

Regression Coefficients:
           Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept      1.78      0.12     1.55     2.01 1.00      670     1381
zBase          0.70      0.12     0.46     0.94 1.01      740     1173
Trt1          -0.28      0.16    -0.58     0.03 1.00      732     1410
zBase:Trt1     0.03      0.16    -0.29     0.34 1.00      855     1411

Draws were sampled using sample(hmc). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).

Originally posted by @marcus-waldman in #1591 (comment)

[paul-buerkner]

Thanks for reporting this issue! Anybody else seeing this? I cannot reproduce it unfortunately. Which version of rstan do you have installed?

[lunafazio]

Yes, managed to reproduce. I think it might actually be related to the version of R itself.

On 4.3.1:

diagnostics <- as.data.frame(matrix(nrow = 1, ncol = 0))
diagnostics[1:3]
# 1 NULL NULL NULL

On 4.4.0:

diagnostics <- as.data.frame(matrix(nrow = 1, ncol = 0))
diagnostics[1:3]
# Error in `[.data.frame`(diagnostics, 1:3) : undefined columns selected

The problem object is created here and the error is triggered when attempting to subset by rstan_diagn_order, here.

I traced the difference in behavior to this line of [.data.frame, which seems to produce a different output in R 4.4.0, which triggers the error two lines later. Couldn't figure out what it is that NextMethod calls, but I suppose this is enough information to figure out a workaround.

[paul-buerkner]

Thank you! This is very helpful!

[paul-buerkner]

I think I fixed it but I currently don't have an R 4.4.0 install to test it myself. Can one of you quickly check it out?

[lunafazio]

It works now!

[paul-buerkner]

Great! Thank you!