Update documentation / default arg metrics in score()

[nikosbosse]

Suggestion would be to have metrics be an argument of score(), i.e. score <- function(data, metrics, ...) which then has a default for all the methods.

For the documentation, could write something like

#' @param metrics An optional named list of scoring functions. Names will be used as
#' column names in the output. See [metrics_point()], [metrics_binary()],
#' `metrics_quantile()`, and [metrics_sample()] for more information on the
#' default metrics used.

[nikosbosse]

I tested this and it didn't really work. The problem is in score.default():

Either metrics doesn't get passed down to the actual method, as in this case:

score.default <- function(data, metrics, ...) {
  data <- validate(data)
  score(data, ...)
}

or metrics can't be empty and it will error if it's missing, as in this case:

score.default <- function(data, metrics, ...) {
  data <- validate(data)
  score(data, metrics, ...)
}

We can still have the description be "an optional list", but I'm not sure adding metrics as an argument to score() works very well / we'd have to make it work in a way that I'm not sure adds that much

[nikosbosse]

@seabbs Do you care strongly about this or do you think we can close this for? I don't really have a strong preference, but if you care I'm happy to keep it open

[nikosbosse]

Closing this for now, but please feel free to re-open

[seabbs]

So this surfaces something we could think about which would be forcing the user to explicitly call as_forecast themselves vs doing it for them in score.default this would get us away from having to call score inside score which is a little abusive.

[nikosbosse]

I opened up a new issue, #507 which I think mostly includes this issue as well:

• if we decide to force users to call as_forecast(), then metrics should be an argument of score()
• if we don't then we leave it out as an argument