Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CI/SE, fix quantile for StratifiedSample #87

Closed
iuliadmtru opened this issue Nov 14, 2022 · 7 comments
Closed

Add CI/SE, fix quantile for StratifiedSample #87

iuliadmtru opened this issue Nov 14, 2022 · 7 comments
Assignees
Labels
bug Something isn't working enhancement New feature or request

Comments

@iuliadmtru
Copy link
Contributor

We made a lot of changes to the design lately but svyquantile was not tested. @sayantikaSSG can you write some tests for svyquantile? And maybe @smishr can take a look at the source code and update it.

@smishr
Copy link
Contributor

smishr commented Nov 29, 2022

Will look into this in next few days

@smishr
Copy link
Contributor

smishr commented Dec 6, 2022

The current quantile function doesnt quite work for both SimpleRandomSample or StratifiedSample.

In R:

`library(survey)
data(api)
srs = svydesign(id=~1, fpc = ~fpc, data=apisrs)
svyquantile(~enroll,srs,c(0.1,0.2,0.5,0.75,0.95))

$enroll
quantile ci.2.5 ci.97.5 se
0.1 247 213 276 15.97398
0.2 318 290 344 13.69199
0.5 453 413 531 29.91952
0.75 682 621 734 28.65175
0.95 1589 1293 1888 150.86539
attr(,"hasci")
[1] TRUE
attr(,"class")
[1] "newsvyquantile"

dstrat = svydesign(id=~1, strata=~stype,fpc = ~fpc, data=apisrs)
svyquantile(~enroll,dstrat,0.5)

$enroll
quantile ci.2.5 ci.97.5 se
0.1 267 196 328 33.46722
0.2 354 318 419 25.60749
0.5 639 570 734 41.58048
0.75 1135 1016 1410 99.89458
0.95 1888 1678 NaN NaN
attr(,"hasci")
[1] TRUE
attr(,"class")
[1] "newsvyquantile"`

But in Julia (after modifying the printing layout to be like R):
`
julia> apisrs = load_data("apisrs");

julia> srs = SimpleRandomSample(apisrs;popsize=:fpc);

julia> quantile(:enroll, srs, [0.1,0.2,0.5,0.75,0.95])
5×2 DataFrame
Row │ qth_quantile quantile
│ Float64 Float64
─────┼────────────────────────
1 │ 0.1 245.5
2 │ 0.2 317.6
3 │ 0.5 453.0
4 │ 0.75 668.5
5 │ 0.95 1473.1

julia> strat = load_data("apistrat");

julia> dstrat = StratifiedSample(strat, :stype; popsize=:fpc);

julia> quantile(:enroll, dstrat, [0.1,0.2,0.5,0.75,0.95])
5×2 DataFrame
Row │ qth_quantile quantile
│ Float64 Float32
─────┼────────────────────────
1 │ 0.1 307.882
2 │ 0.2 385.01
3 │ 0.5 712.02
4 │ 0.75 1305.98
5 │ 0.95 2130.95
`

Which means currently, only the median (0.5th qauntile) is coming correctly for SimpleRandomSample, no other quantiles, and nothing is correct for StratifiedSample. I checked that both R quantile and Julia Statistics.quantile use the same Type=7 default. Am investigating. I think there is some conflict in the method quantile that is being used, as Statistics and StatsBase have different definitions from what i could see.

https://docs.julialang.org/en/v1/stdlib/Statistics/#Statistics.quantile
https://juliastats.org/StatsBase.jl/stable/scalarstats/#Statistics.quantile

@smishr
Copy link
Contributor

smishr commented Dec 6, 2022

Further, R returns the CIs (default alpha = 0.05) and SE by default. This option should be investigated.

@smishr smishr changed the title svyquantile hasn't been updated in a while Fix quantile for new survey designs Dec 6, 2022
@smishr smishr pinned this issue Dec 6, 2022
@smishr smishr mentioned this issue Dec 7, 2022
@smishr smishr unpinned this issue Dec 8, 2022
@smishr
Copy link
Contributor

smishr commented Dec 12, 2022

For now, quantile with StratifiedSample returns some form of quantiles, which I couldn't identify from the Hyndman and Fan defintions and their weighted versions applied in R.

@smishr
Copy link
Contributor

smishr commented Dec 12, 2022

Also, CI/SE havent been implemented yet

@smishr smishr added bug Something isn't working enhancement New feature or request labels Dec 12, 2022
@smishr smishr modified the milestones: 0.2.0 release, 0.3.0 release Dec 12, 2022
@smishr smishr changed the title Fix quantile for new survey designs Add CI/SE, fix quantile for StratifiedSample Dec 12, 2022
@smishr smishr pinned this issue Dec 12, 2022
@smishr
Copy link
Contributor

smishr commented Dec 12, 2022

See #130 as well

@smishr smishr unpinned this issue Jan 3, 2023
@smishr
Copy link
Contributor

smishr commented Jan 4, 2023

We have stopped creating domain-specific functions like mean, total, or quantile for now, using arbitrary user defined functions now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants