Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Functionality: Have the functional claims of the software been confirmed? #3563

Closed
Tracked by #3444
MikeKaller opened this issue Aug 4, 2021 · 5 comments
Closed
Tracked by #3444

Comments

@MikeKaller
Copy link

MikeKaller commented Aug 4, 2021

I had good success testing several functions (h_vb_stan, h_gompertz_stan, and catch_curve) with my data and with public data sets (BlackDrum2001 and Brook Trout). The Gompertz model was spot on to my previous estimates.

When testing h_vb_stan, the estimate for L infinity was 5000 mm more than it should have been, when I requested t0 = TRUE. The estimates were fine for t0 = FALSE. This occurred with one data set (mine), but was fine with BlackDrum2001 (from Ogle). My data are biased toward very young fish (age 1-2), with few fish over age 4, which might be the problem. However, I have been able to achieve reasonable estimates in R and SAS. These data are in press, but I attached for the authors to check. Please consider them not open data at the moment. It might have been me or it might have been the data, but then again, it might have been the package. I attempted to attach, hope it worked.
MG.xlsx

#good estimate with t0=FALSE

rstan::summary(vb_out$model_par,

  •            prob = 0.5, pars = "growth_coef")$summary
                      mean     se_mean          sd         50%    n_eff     Rhat
    

growth_coef[1,1] 412.9389142 3.015959709 12.27809496 412.5017118 16.57337 1.119560
growth_coef[1,2] 416.1777699 3.479534558 12.83390134 418.3515820 13.60427 1.149403
growth_coef[2,1] 0.5350705 0.004839283 0.02816545 0.5380143 33.87438 1.082948
growth_coef[2,2] 0.5284937 0.011779580 0.04672065 0.5215366 15.73107 1.127339

#poor estimate with t-=TRUE

iter <- 100
vb_out <-

  • h_vb_stan(
    
  •     fish_length = fish_data %>% pull(fish_length),
    
  •     fish_age = fish_data %>% pull(fish_age),
    
  •     jj = fish_data %>% pull(jj),
    
  •     t0 = TRUE,
    
  •     n_proj = 6,
    
  •     iter = iter,
    
  •     refresh = 0
    
  • )
    

Warning messages:
1: There were 90 divergent transitions after warmup. See
http://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
to find out why this is a problem and how to eliminate them.
2: There were 87 transitions after warmup that exceeded the maximum treedepth. Increase max_treedepth above 10. See
http://mc-stan.org/misc/warnings.html#maximum-treedepth-exceeded
3: There were 1 chains where the estimated Bayesian Fraction of Missing Information was low. See
http://mc-stan.org/misc/warnings.html#bfmi-low
4: Examine the pairs() plot to diagnose sampling problems

5: The largest R-hat is NA, indicating chains have not mixed.
Running the chains for more iterations may help. See
http://mc-stan.org/misc/warnings.html#r-hat
6: Bulk Effective Samples Size (ESS) is too low, indicating posterior means and medians may be unreliable.
Running the chains for more iterations may help. See
http://mc-stan.org/misc/warnings.html#bulk-ess
7: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
Running the chains for more iterations may help. See
http://mc-stan.org/misc/warnings.html#tail-ess

rstan::summary(vb_out$model_par,

  •            prob = 0.5, pars = "growth_coef")$summary
                      mean      se_mean          sd        50%    n_eff      Rhat
    

growth_coef[1,1] 5401.598901 6281.2936447 8943.229314 364.954084 2.027172 25.783682
growth_coef[1,2] 314.949252 144.7459496 205.940111 344.254750 2.024273 23.937948
growth_coef[2,1] 4.233159 3.6421330 5.183172 1.697671 2.025254 30.495047
growth_coef[2,2] 1.512192 0.9655945 1.397608 1.304163 2.094987 12.136972
growth_coef[3,1] -5.815315 2.7314191 3.883504 -6.567190 2.021486 43.233233
growth_coef[3,2] -6.505797 2.6907258 3.847480 -8.504928 2.044625 9.551402

@whedon
Copy link

whedon commented Aug 4, 2021

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

@whedon
Copy link

whedon commented Aug 4, 2021

Downloading of the repository (to check the bibtex) failed for issue #3563 failed with the following error:

/app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/lib/whedon/processor.rb:28:in `initialize': undefined method `[]' for nil:NilClass (NoMethodError)
	from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/lib/whedon.rb:379:in `new'
	from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/lib/whedon.rb:379:in `download'
	from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/bin/whedon:38:in `download'
	from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
	from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
	from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
	from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
	from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/bin/whedon:131:in `<top (required)>'
	from /app/vendor/bundle/ruby/2.6.0/bin/whedon:23:in `load'
	from /app/vendor/bundle/ruby/2.6.0/bin/whedon:23:in `<main>'

@whedon
Copy link

whedon commented Aug 4, 2021

PDF failed to compile for issue #3563 with the following error:

 /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/lib/whedon/processor.rb:28:in `initialize': undefined method `[]' for nil:NilClass (NoMethodError)
	from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/lib/whedon.rb:379:in `new'
	from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/lib/whedon.rb:379:in `download'
	from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/bin/whedon:38:in `download'
	from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
	from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
	from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
	from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
	from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/bin/whedon:131:in `<top (required)>'
	from /app/vendor/bundle/ruby/2.6.0/bin/whedon:23:in `load'
	from /app/vendor/bundle/ruby/2.6.0/bin/whedon:23:in `<main>'

@whedon
Copy link

whedon commented Aug 4, 2021

Downloading of the repository (to analyze the language) for issue #3563 failed with the following error:

 /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/lib/whedon/processor.rb:28:in `initialize': undefined method `[]' for nil:NilClass (NoMethodError)
	from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/lib/whedon.rb:379:in `new'
	from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/lib/whedon.rb:379:in `download'
	from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/bin/whedon:38:in `download'
	from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
	from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
	from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
	from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
	from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-b63fc70cc085/bin/whedon:131:in `<top (required)>'
	from /app/vendor/bundle/ruby/2.6.0/bin/whedon:23:in `load'
	from /app/vendor/bundle/ruby/2.6.0/bin/whedon:23:in `<main>'

@danielskatz
Copy link

Please manually open an issue in the software source code's repository - the github "feauture" you used doesn't work correctly - it opens the issue in the wrong repo, and doesn't tell the submitter what the exact problem and potential solution are.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants