Low number suppression within measures

[LFISHER7]

I have been using this function that @alexwalkercebm wrote to suppress low numbers in measures outputs. It seems to me that if you are using measures, you would always want to do this and it could be implemented by default.

I would suggest a snippet similar to above but would mask the numerator and denominator only (before calculating 'value' col). This way it's obvious which groups have been suppressed at what time.

[alexwalkerepi]

Yes I think that would be valuable, perhaps as an option within the generate_measures command. I say as an option because there are plausible use cases where you generate a measure with one level of aggregation, then subsequently aggregate at a higher level. If you've already redacted at the lower level then you'll have some missing numbers where it might not have been necessary to react.

[LFISHER7]

That makes sense, I was wondering if there was a use case. By having it as an option, at least it forces people to recognise that they should be thinking about it.

[sebbacon]

Sounds great - please add an issue in the cohortextractor repo describing the desired behaviour

[wjchulme]

This function redacts values that are 5 or less. If the marginal totals are known (eg in a table elsewhere, or because you know the size of the population) then these could be recovered.

My approach has been to ensure all values less than 5 are redacted and the sum of all redacted values exceeds 5. So for example:

• for [3,10], both values have to be redacted
• for [1,2,10], all three values have to be redacted.
• for [1,2,8,10], 1, 2, and 8 all have to be redacted, but as 1+2+8>5 then 10 can stay unredacted.

eg here: https://github.com/opensafely/covid-vaccine-effectiveness-research/blob/5c2aedebe1fe4b238765d1e12d9086cb34f8924c/lib/redaction_functions.R#L8 -- this function produces the mask, which is then used elsewhere for actual redaction.

Is this worth implemented here too?

[wjchulme]

Thanks, @evansd that's much clearer. In practice, what it means is that we redact all values <= 5, and if the sum of those redacted values is also <= 5, then we must additionally redact the smallest remaining unredacted number.

For a count table of the form [A=1, B=2, C=10] the key point is that if we were to combine categories A and B into a new category with count 1+2=3, then that is still a redactable count and we can recover the 3 if we know the total is 13. So we need to redact the 10 too. As an example, consider for [short-term immunosuppressed = 1, long-term immunosuppressed = 2, not immunosuppressed = 10] and the combined [any immunosuppressed = 3, not immunosuppressed = 10]. In both cases we should redact all numbers if we know that total =13.

For [A=1, B=2, C=8, D=10], combining A and B gives a redactable count 3, but combining A B and C gives 11, which is fine, so D can stay unredacted.

[wjchulme]

@LFISHER7 I think [4, 4, 10] is fine. The first redaction step (redact all values <= 5) would lead to [*, *, 10]. The second redaction step (redact smallest unredacted number if sum(*) <=5) doesn't apply, as by combining redacted categories, we're left with [8, 10], which is fine.

[LFISHER7]

Ah I see, slightly less recoverable than I thought. Apologies if I still haven't got it, but you would still know that the 8 here is formed from either [5, 3], [3, 5] or [4, 4]?

[wjchulme]

It could be any of [1,7], [2, 6], ..., [7, 1], as for all these combinations it would be necessary to redact both numbers. Or have I missed something?

[LFISHER7]

My bad, I misunderstood the redact next smallest unredacted value aspect. This looks like a good solution then