precise time.time() under Windows 8: use GetSystemTimePreciseAsFileTime

[pitrou]

BPO	19007
Nosy	@malemburg, @tim-one, @abalkin, @pitrou, @vstinner, @tjguk, @4kir4, @zware, @eryksun, @zooba
Files	time_precise.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2013-09-12.19:44:34.318>
labels = ['type-feature', 'library', 'OS-windows', '3.11']
title = 'precise time.time() under Windows 8: use GetSystemTimePreciseAsFileTime'
updated_at = <Date 2021-06-13.22:48:09.141>
user = 'https://github.com/pitrou'

bugs.python.org fields:

activity = <Date 2021-06-13.22:48:09.141>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'Windows']
creation = <Date 2013-09-12.19:44:34.318>
creator = 'pitrou'
dependencies = []
files = ['40088']
hgrepos = []
issue_num = 19007
keywords = ['patch']
message_count = 9.0
messages = ['197538', '197539', '197541', '224358', '247024', '247778', '247779', '247808', '395767']
nosy_count = 10.0
nosy_names = ['lemburg', 'tim.peters', 'belopolsky', 'pitrou', 'vstinner', 'tim.golden', 'akira', 'zach.ware', 'eryksun', 'steve.dower']
pr_nums = []
priority = 'low'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue19007'
versions = ['Python 3.11']

Linked PRs

• gh-63207: Use GetSystemTimePreciseAsFileTime() in time.time() #116822

[pitrou]

For whoever is interested: Windows 8 apparently has a new API named GetSystemTimePreciseAsFileTime which returns the system API with a much better resolution than GetSystemTimeAsFileTime does ("The GetSystemTimePreciseAsFileTime function retrieves the current system date and time with the highest possible level of precision (<1us)").
http://msdn.microsoft.com/en-us/library/windows/desktop/hh706895%28v=vs.85%29.aspx

(spawned from this topic on the tulip mailing-list: https://groups.google.com/forum/#!topic/python-tulip/vX9vOZB1FOI )

[vstinner]

time.time() is sometimes used in performance critical code. Is GetSystemTimePreciseAsFileTime() as fast as GetSystemTimeAsFileTime()?

Linux has the opposite: CLOCK_REALTIME_COARSE. This clock is less accurate but may be faster.
http://lwn.net/Articles/342018/

[pitrou]

"GetSystemTimePreciseAsFileTime() has very little overhead; the new function is even a little faster than GetSystemTimeAsFileTime(), a call takes a few ten nanoseconds."

from http://www.windowstimestamp.com/description#C_2

[vstinner]

My patch for the issue bpo-22043 replaces the _PyTime_timeval structure (10^-6 second resolution, 1 us) with a _PyTime_timespec structure (10^-9 second resolution, 1 ns). It may help for this issue.

[vstinner]

Python 3.5 has a new C function _PyTime_GetSystemClock() which uses the new _PyTime_t type. It now has a resolution of 1 nanosecond.

Sorry, I don't have Windows 8 at home, so I cannot work on this issue.

I guess that we should check at runtime if GetSystemTimePreciseAsFileTime() is available, as we did for GetTickCount64() in the past:

hKernel32 = GetModuleHandleW(L"KERNEL32");
*(FARPROC*)&Py_GetTickCount64 = GetProcAddress(hKernel32,
                                               "GetTickCount64");

Is there any Windows developer interested to write a short patch to implement it?

[vstinner]

Good news! I got a new fresh Windows 8.1 VM with Visual Studio 2015. I'm now able to work on this issue.

I wrote a patch: time_precise.patch.

Resolution computed in Python by https://hg.python.org/peps/file/tip/pep-0418/clock_resolution.py:

GetSystemTimePreciseAsFileTime(): 715 ns
GetSystemTimeAsFileTime(): 14 ms

Obviously, the resolution is better...

GetSystemTimePreciseAsFileTime() uses internally the QueryPerformanceCounter() so I chose to use QueryPerformanceFrequency() to fill time.get_clock_info('time').resolution, same code than time.get_clock_info('perf_counter').resolution

[vstinner]

Note: GetSystemTimePreciseAsFileTime() is restricted to desktop applications.

The windowstimestamp.com has a warning on this function:

http://www.windowstimestamp.com/description

"""
2.1.4.2. Desktop Applications: GetSystemTimePreciseAsFileTime()

(...)
The function shall also be used with care when a system time adjustment is active. Current Windows versions treat the performance counter frequency as a constant. The high resolution of GetSystemTimePreciseAsFileTime() is derived from the performance counter value at the time of the call and the performance counter frequency. However, the performance counter frequency should be corrected during system time adjustments to adapt to the modified progress in time. Current Windows versions don't do this. The obtained microsecond part may be severely affected when system time adjustments are active. Seconds may consist of more or less than 1.000.000 microseconds. Microsoft may or not fix this in one of the next updates/versions.
(...)
As of May, 2015 the inaccuracy of GetSystemTimePreciseAsFileTime() during system time adjustments persists for the preview versions of Windows 10.
"""

Is it ok to switch to GetSystemTimePreciseAsFileTime() for Python time.time()?

[zooba]

What are the expected benefits from changing? Just a higher resolution? I'm not sure that's worth anything if it's inaccurate.

[eryksun]

What are the expected benefits from changing? Just a higher
resolution? I'm not sure that's worth anything if it's inaccurate.

GetSystemTimePreciseAsFileTime() returns an accurate timestamp, which is the current system time plus the difference between the current value of the performance counter and its value when the system time was last updated. This can't be replicated simply with separate calls to GetSystemTimeAsFileTime() and QueryPerformanceCounter(). It relies on the following undocumented fields in the KUSER_SHARED_DATA record: BaselineSystemTimeQpc, QpcSystemTimeIncrement, and QpcSystemTimeIncrementShift.

The system uses a similar procedure in QueryInterruptTimePrecise() to extend the precision of QueryInterruptTime(). Ditto for QueryUnbiasedInterruptTime() vs QueryUnbiasedInterruptTimePrecise().

"GetSystemTimePreciseAsFileTime() has very little overhead; the new
function is even a little faster than GetSystemTimeAsFileTime(), a
call takes a few ten nanoseconds."

The above claim was corrected back in 2014. GetSystemTimePreciseAsFileTime() is considerably more expensive than GetSystemTimeAsFileTime(). It costs about 10 times more on a system that supports an invariant TSC. It would be more expensive on an older system that use the chipset's HPET or ACPI PM timer. That said, I doubt this matters very much when calling time.time() in interpreted code. It's a difference measured in tens of nanoseconds.

[vstinner]

I wrote PR gh-116822 to use GetSystemTimePreciseAsFileTime() in time.time().

By the way, I also modified modified time.monotonic() to use QueryPerformanceCounter(): commit 846ad5a of issue gh-88494.

[vstinner]

Feature implemented in 1d95451. For the delay @pitrou, it took 11 years to implement the function :-) I suppose that the delay was until all Windows versions supported by Python provide the function.