Figure out if we're using the correct clocks everywhere

[njsmith]

Trio's standard clock is intended to be a monotonic clock that ticks only while the system is running, not while it's suspended. (Rationale: this is the kind of clock that blocking primitives like epoll/GetQueuedCompletionEvents/kqueue use for their timeout parameter, and we want times on the Trio clock to be convertable into timeouts for these calls.)

Is this true? Currently we use time.perf_counter() for our clock.

Linux

We're ok: time.perf_counter() eventually calls clock_gettime(CLOCK_MONOTONIC, ...), and on Linux, CLOCK_MONOTONIC has the properties mentioned above.

macOS

We're ok: time.perf_counter() eventually calls mach_absolute_time(), whose docs say:

Returns current value of a clock that increments monotonically in tick units (starting at an arbitrary point), this clock does not increment while the system is asleep.

(Note: on macOS, clock_gettime(CLOCK_MONOTONIC, ...) does not work like this – it keeps ticking while the system is suspended. So it's good that Python's time module doesn't use this!)

FreeBSD

I'm pretty sure we do the wrong thing: time.perf_counter() eventually ends up on pymonotonic, which ends up using CLOCK_MONOTONIC, and on FreeBSD this does keep ticking during suspend: https://www.freebsd.org/cgi/man.cgi?query=clock_gettime

Instead, we should use clock_gettime(CLOCK_UPTIME, ...).

(Also, perhaps, file a bug on Python? Not sure what the contract for time.monotonic()/time.perf_counter() is supposed to be.)

Windows

I'm pretty sure we do the wrong thing: time.perf_counter() calls QueryPerformanceCounter, and the docs say:

QueryPerformanceCounter reads the performance counter and returns the total number of ticks that have occurred since the Windows operating system was started, including the time when the machine was in a sleep state such as standby, hibernate, or connected standby.

...It is super unclear what you are supposed to do instead.

[pause to scrounge around weird corners of the internet]

OK so it sounds like the official options are:

• QueryUnbiasedInterruptTime – but this is relatively low resolution, usually the same 15 ms resolution that we originally switched to QueryPerformanceCounter to avoid, see Which clock should we use on Windows? #33

• QueryUnbiasedInterruptTimePrecise – exactly what we want, but the docs imply that it might be kinda slow. We should measure to find out if that's actually true.

• Use QueryPerformanceCounter (which includes time spend suspended) plus some eldritch magic to figure out how long the system has spent suspended, so we can subtract it off again to "correct" the QueryPerformanceCounter output. Specifically, user-space processes can apparently find the number of 100 ns ticks that the system has spent suspended stored as an unsigned 64-bit value at memory address 0x7ffe03b0.

  I swear I'm not making this up.

  • https://stackoverflow.com/questions/24330496/how-do-i-create-monotonic-clock-on-windows-which-doesnt-tick-during-suspend
  • https://www.geoffchappell.com/studies/windows/km/ntoskrnl/structs/kuser_shared_data/index.htm

[njsmith]

Now that my world has been shaken a bit, I feel like we should also double-check that I'm right that the GetQueuedCompletionEventEx function uses a monotonic-without-suspend clock for its timeout argument.

[njsmith]

@vstinner BTW, if you check out this issue it documents some quirks in how time.monotonic() and time.perf_counter() have inconsistent semantics across platforms. I have no idea if these are considered bugs or not :-). Do you want me to file any BPO issues?

[vstinner]

I have no idea if these are considered bugs or not :-)

You should dig into https://www.python.org/dev/peps/pep-0418/ and search for "suspend". Handling time is hard :-)

[njsmith]

@vstinner Ha, you don't have to tell me :-)

It does look like this statement from PEP 418 may not be true anymore:

Some recent operating systems provide two clocks, one including time elapsed during system suspend, one not including this time. Most operating systems only provide one kind of clock.

In particular, all supported Windows versions now have both clocks available, but Python only wraps the keeps-ticking-during-suspend version.

(Hmm, I guess this is actually a bug in asyncio as well, since they also want a clock that stops ticking during suspend...)

[ghost]

GetTickCount64 is the right one for Windows.

[njsmith]

@FastInvite2k2 we switched away from using that a few years ago – see #33 for discussion

[njsmith]

Also, aside from the resolution issues we ran into in #33, from the docs it looks like GetTickCount64 keeps ticking while suspended, which is not what we want.