NewSeeded: rename new → try_new, add new_with_fallback #21
Conversation
Rename OsRng::new → try_new too for consitency Add dependency on time crate (Note: better to use chrono, but has no real equivalent to precise_time_ns().)
|
Yes, thank you for asking! As a test I printed 100 numbers generated with A very small PRNG seeded with these 32 bits should give more 'random' results than taking any more from the system clock. So while the system clock should be good enough for one When using the system clock ass a fallback, what do you think of the following: Initialize a 32-bit 'PCG RXS M XS (LCG)' with it, and use that to initialize the target RNG? PCG is not as difficult as it seems. With the constants written out a step takes just 4 lines of code. I have the start of something here (different version though). |
|
Can't we use the standard library instead of the time crate? It seems to provide everything we need with |
|
@pitdicker That sounds much smarter than the code I wrote. Assembling roughly a The lower precision part of the time (up to a few days, e.g. 10^6 sec) still provides some varying numbers, but is much more predictable. Whether or not it counts as "entropy" depends on the application (i.e. if you know this was calculated in response to a web query in the last second there is significantly less entropy than if it happened when the server was last rebooted — not that we're going to recommend anyone use system time as entropy for cryptography of course). But using a high-precision If we wanted to make the fallback as secure as possible, we could require the use of some RNG wrapper, like |
|
I agree that making more calls to Even with very basic statistics the numbers don't look very random. If I sort by the In rust-lang/rfcs#722 there has been some discussion about reseeding. The linked article https://eprint.iacr.org/2013/338.pdf seems to be the state of the art of analysis around reseeding. It is to way difficult for me, at least without spending days to read it. The takeaway seems to be that extracting entropy and integrating it into an RNGs state is hard. I don't trust myself (or maybe us) to come up with a good way to really extract more than 32 bits of entropy from the system time. With repeated runs I suspect there are a lot of details that start to matter: actual precision of the hardware timer (in the cpu?), rounding in the kernel, variance of the time between runs... For a fall-back I would go with something simple. |
If I understand you right, this means we could construct an u64? If we only needed one random number, I agree. But if someone were to initialize multiple RNGs, they would not be as different as when each of them was based on a u32 extracted from the system time. |
Constructing a
Given enough time, and mixed scheduling (not making all the "time" readings in a tight loop), it should be possible to extract an unlimited amount of entropy. But not doing stuff in a tight loop is really going to help, so I think making only one or two readings each time Perhaps best would be to combine the two approaches: construct a simple PRNG, and each time I still wouldn't trust this approach to withstand a serious attack, but for anything else (including low-resource attacks) it should be fine. Makes me wonder, if we can extract a little entropy from the system timer at a very low cost, maybe we should mix that into the output of |
|
I did a little more work to try to estimate available entropy. The good news is that output of the high-resolution counter is well distributed, so there isn't too much rounding going on (on my test system at least). The bad news is that diffs between samples are not at all well distributed, but there may still be 2-4 bits of entropy per call. That example is simply calling the timer in a tight loop and doing very little work per cycle; if the timer is called less frequently or with allocs or yields between calls there could be far more entropy available. |
|
Wow, you are serious about estimating the entropy of the system clock!
I also saw that the difference is much less predictable for the first two rounds of the loop. What would you suggest after testing? Constructing a single |
|
That's true, the difference between the first two samples may be much greater. I initially had differences greater than 1ms, but realised it was probably caused by a I'm thinking a hybrid approach may be best. |
Remove dependency on external crate Use variation on PCG to mix internal state and generate output Use multiple init cycles
|
@pitdicker maybe you'd like to review my latest version? I'm not sure how to estimate entropy, but I think it should be reasonably good (well, still not close to the 64-bits of state I suppose). It's a tad slower than PCG however... |
|
FYI |
|
See last commit for another approach which is simpler but should be stronger (though slower). |
|
To test the statistical quality I have run a few variants trough PractRand. d6f8ebf fails after 256 MiB, with only 3 tests not failing. If I take a306a69 and apply the MurmurHash finalizer on it, it fails after 256 MiB, with 65 test not failing. fn next_u32(&mut self) -> u32 {
let mut state = get_time() as u32;
state = (state ^ (state >> 16)).wrapping_mul(0x85ebca6b);
state = (state ^ (state >> 13)).wrapping_mul(0xc2b2ae35);
state ^ (state >> 16);
state as u32
}So the experiments to extract more randomness from the system clock work against us :-(. But a hybrid approach works great! Combining a hash of the clock as source of some extra entropy with PCG (messy implementation): #[derive(Debug)]
pub struct ClockRng {
state: u32,
}
impl ClockRng {
/// Create a `ClockRng` (very low cost)
pub fn new() -> ClockRng {
let mut state = precise_time_ns() as u32;
state = (state ^ (state >> 16)).wrapping_mul(0x85ebca6b);
state = (state ^ (state >> 13)).wrapping_mul(0xc2b2ae35);
state ^ (state >> 16);
ClockRng { state: state }
}
}
impl Rng for ClockRng {
fn next_u32(&mut self) -> u32 {
// Input 1: high 32 bits of the current time in nanoseconds, hashed with
// the MurmurHash finalizer.
let mut time = precise_time_ns() as u32;
time = (time ^ (time >> 16)).wrapping_mul(0x85ebca6b);
time = (time ^ (time >> 13)).wrapping_mul(0xc2b2ae35);
time ^ (time >> 16);
// Input 2: PCG 32 RXS M XS
let mut state = self.state;
self.state = state * 747796405 + 2891336453;
state = ((state >> ((state >> 28) + 4)) ^ state) * 277803737;
state = (state >> 22) ^ state;
time ^ state
}
/* other methods */
}A PractRand run is now at 32 GiB without anything to report. |
|
@pitdicker PractRand is designed to measure bias, not entropy. While bias in our output isn't a good thing, it's much less important than entropy. Remember, a fully deterministic PRNG can pass PractRand (not forever maybe, but to more data than you could possibly run). PractRand may still be useful for trying to remove bias I guess. I am surprised that 64f1f63 does worse than the previous commit. a306a69 does no mixing at all yet gets one of the best results. This surprises me. What happens if you run the same test a couple more times? The other funny thing is |
|
Directly measuring entropy via the Shannon method is possible, so I did that (roughly 4-5.5 bits per call, but only last 2 bits have high entropy). So if we mix the result well we should get a good quality generator, right? Lets try that... |
Also appears to double performance; this might be because the >sec parts of the time are now ignored and/or algorithm being more parallelizable.
|
@pitdicker if you care to, please test the new I'm considering removing |
Yes it surprises me to. But I only had time to run the test, not to figure out the cause... PractRand has very reliable tests. I have run them several times but the results stay the same.
No problem. It is better, but not much, sorry: A few small experiments from my side fared much worse though. |
Good work! About 4-5 bits matches my feeling. This paper (which I have not yet read fully) explains on page 6 and 7 that Shannon entropy is the best case, and we can realistically expect to be able to extract a little less.
Yes, this is better than my proposal to seed a small PRNG once with the system clock. When speaking in terms of entropy, every round then adds a little, and every little bit helps.
I am really learning as I go. You are right about PractRand. But I expect a good method to extract entropy and mix it to produce practically unbiased results. So it is useful as an indication. I do believe being unbiased is a very important property, especially when seeding other RNGs.
What did you think about #21 (comment)? It only adds about 4 bits of entropy each round as you calculated. It is statistically good, I canceled PractRand after 1 TiB of no anomalies. I think we either have to go with a simple solution, or do good research. Wikipedia has among others the suggestion to use a Cryptographic hash function to extract entropy. |
| @@ -294,14 +299,37 @@ mod thread_local; | |||
| #[cfg(feature="std")] | |||
| pub trait NewSeeded: Sized { | |||
| /// Creates a new instance, automatically seeded via `OsRng`. | |||
| fn new() -> Result<Self, Error>; | |||
| fn try_new() -> Result<Self, Error>; | |||
There was a problem hiding this comment.
Has this been discussed somewhere? What are we trying to be consistent with?
When I started learning about rand, I was impressed by how easy it was to do the right thing. new just seeds an RNG from the OsRng. So I am a bit of a fan of the simple new().
The API guidelines seem to suggest using new is fine.
Would you like to open an issue to discuss the rename?
There was a problem hiding this comment.
new() here would be fine I guess; it's just that the last proposal had try_new. The bit about being consistent is renaming in OsRng to match. Maybe that was a bad idea.
There was a problem hiding this comment.
I'd personally prefer just new as well, should be clear from the type what's going on.
| @@ -122,7 +122,8 @@ pub struct ReseedWithNew; | |||
| #[cfg(feature="std")] | |||
| impl<R: Rng + NewSeeded> Reseeder<R> for ReseedWithNew { | |||
| fn reseed(&mut self, rng: &mut R) { | |||
| match R::new() { | |||
| // TODO: should we use new_with_fallback instead? | |||
There was a problem hiding this comment.
Reseeding a strong RNG like ISAAC from something much weaker like the system timer does not sound like a good idea.
ReseedingRng wraps an already seeded RNG, so we only have to care about reseeding, not initializing.
In some situations it makes sense to ignore an error, like with TreadRng. Instead of taking down a process, it seems preferable to continue using the existing RNG. But someone relying on reseeding to keep the wrapped RNG secure would probably like to be informed of an error. It should be passed on through try_fill, and otherwise panic.
So a best default is not clear to me. Maybe we should just do the simplification of TreadRng to not rely on ReseedingRng, and handle errors different for them.
In both cases I don't think new_with_fallback is a good idea.
There was a problem hiding this comment.
If a ReseedingRng was initially seeded from the OS then this failed when reseeding, then yes, it would be better to keep using the old state or mix in new entropy than replace. But the only case I have heard of OsRng failing is that "broken DLL" problem on Windows (i.e. never works) or (possibly) slow initialisation (doesn't work at first, does later).
We could in any case always mix in fresh entropy instead of replace. I don't know.
As for "not seeding a strong RNG like ISAAC from a weak source", why not? It's still better than seeding a weak RNG from a weak source. As that bug report about Firefox shows, sometimes a fallback option is needed, but there's no reason they need to use a weak RNG too.
| @@ -51,7 +51,8 @@ impl Rng for ThreadRng { | |||
|
|
|||
| thread_local!( | |||
| static THREAD_RNG_KEY: Rc<RefCell<ReseedingStdRng>> = { | |||
| let r = match StdRng::new() { | |||
| // TODO: consider using new_with_fallback instead of try_new | |||
| let r = match StdRng::try_new() { | |||
There was a problem hiding this comment.
Seems exactly what Firefox would need rust-lang/rust#44911
Uh, sorry but there are a few issues with that code.
You have two lines of code which don't do anything.
Time never affects the state, it only affects the output with a simple XOR. So basically you have a simple LCG like PCG with some extra tweaking of the output; you're never accumulating entropy. Further, you never try to assemble much entropy. I still think @burdges I wonder if you are able to help us out here? |
| let xorshifted = ((state >> 18) ^ state) >> 27; | ||
| let rot = state >> 59; | ||
| let rot2 = (-rot) & w(31); | ||
| ((xorshifted >> rot.0 as usize) | (xorshifted << rot2.0 as usize)).0 as u32 |
There was a problem hiding this comment.
It is strange that this version fails so quickly on PractRand. What is happening on these lines? Are the two rotations partly cancelling out each other? In would like to test it with the normal PCG version.
|
Does this sound right to you?
My previous thought was to set up a small RNG with the result from the first round. You want to go with something stronger, and that makes sense. That would mean mixing in the new values from In principle I like the idea behind Hopefully it can give both the good entropy we want, and be unbiased. |
|
@pitdicker you can try that variant on Overall though I still don't think bias in the crypto keys should be a problem aside from the reduction in entropy, and I'm not sure how to measure that (I would expect it to be small, but don't know). I still think something like |
That part was a good idea, and it works well. I only did a quick test during breakfast, where I replaced the output function. I think there is the problem with the output function, that is different from normal PCG. |
|
I still feel uneasy about coming up with something ourselves. What do you think about creating an implementation based on this Jitter RNG? It also uses the the system clock as source, and has excellent documentation and analysis. It would be quite some work though, there are about 1000 lines of C code. The license is BSD or alternatively GPL. |
|
Apparently that is also the method Linux uses to extract entropy from the timer. |
|
Nice find; that does sound like what we want. At this level of complexity it might also make sense to use an external crate, or to start with an external implementation and consider importing into But this is getting a bit beyond the scope of the |
Currently I have an implementation of Xorshift* and PCG mostly finished, and polishing it a bit and waiting an the discussion around initialization. Also How to proceed with this PR? Remove |
|
How to proceed for now — not sure; see #23. |
|
On closer look it seems quite doable to implement jitterentropy-library. I will give it a try. |
|
|
Rename OsRng::new → try_new too for consitency
Add dependency on time crate
(Note: better to use chrono, but has no real equivalent to precise_time_ns().)
@pitdicker, can you review?