Interpreting residual plot for small sample size

[Hkhutch27]

Good morning,

I am relatively new to using DHARMa for model diagnostics, so I apologize if my question is simple! I am modeling bird counts across 40 sites, with an offset for effort, and I would appreciate any guidance you can provide. Here is my model formula:

model1 <- glmmTMB(counts ~ offset(log(effort)) + predictor1 + predictor2,
data = bird,
zi = ~0,
family = nbinom2)

For model1, the AICc is 342.74. Below is the DHARMa residual output for this model. Overall the model fit seems okay, despite the significant deviations from residual quantiles. However, some significant patterns show up when I plot the residuals for each predictor.

I also fit a simpler model, model2:

model2 <- glmmTMB(counts ~ offset(log(effort)) + predictor2,
data = bird,
zi = ~0,
family = nbinom2)

The AICc for model2 is 343.4, and I have included the DHARMa output below as well.

I have a few specific questions:

1. Model Usability: Based on the patterns in the DHARMa residuals (particularly the significant deviations), how can I determine if the model is still considered "usable"? I understand that significant results in residual diagnostics don’t always render a model invalid, but I’m unsure where to draw the line between acceptable and problematic deviations.

2. AICc Comparison: Given that model1 has a slightly lower AICc (342.7) compared to model2 (343.4), but both show significant residual patterns, would either of these models be considered superior? Should I rely solely on AICc here, or would the residual patterns take precedence?

3. Small Sample Size: Could the relatively small sample size (40 sites) contribute to the issues with residuals, and if so, how might that affect model interpretation?

Thank you in advance for your help and for the very informative vignette—it has already been extremely helpful!

[melina-leite]

Hi @Hkhutch27,
my 50 cents concerning your questions:

1. I suspect that the relationship of your response variable with predictor2 is not as linear as you think. Have you tried another relationship, such as a quadratic or even smooth function (gam model)?

2. Regarding the AIC, I would interpret both models as equally plausible and both with similar issues about residual diagnostics, especially regarding predictor2.

3. I don't believe the small sample size has anything to do with the patterns of the residuals.

Maybe @florianhartig has more insight to add here?

Best, Melina

[florianhartig]

Hi @Hkhutch27,

first of all, what I always say: residual checks are not a model selection criteria, as they do not account for complexity. Thus, if you want to know if you should add complexity to a model, use a model selection criteria. As Melina said, probably both models are fine. As you have tried them anyway, I would stay with the predictor1 + predictor2 model.

Residual checks are meant to inquire if your model shows some significant misfit. In your case, it looks indeed as if there is some nonlinearity and maybe even some dispersion changes with pred2. It's true that you have very few data points, so could be a fluke, but I think more likely than not there is something.

The question is now if this is a bit problem, and give your small sample size, if you should play around with this model to fix the problem. It will likely be complicated to create a fix because the pattern doesn't seem to a simple quadratic effect etc.

I would probably try some simple fixes, e.g. transformation of predictors, and maybe play around with the zi and dispersion term. Also, note that in practice, I have often found that even in cases where you would expect that counts are proportional to effort, the relationship is not 1-1. In such a case, changing from offset(log(effort)) to log(effort) can help. These are just ideas.

I wouldn't do too much though - for me, residual deviations, although significant, aren't so large that I would expect your conclusions to be wrong.