Direct Time Zone Access: Elm's use case

[rtfeldman]

@littledan suggested I post about this here!

Elm's core time package works with time zones, as does Moment Timezone.

In Elm, it's important for us to be able to take a time zone string like "America/New_York"—such as one we'd get from Intl—and turn it into a Time.Zone value which holds information about that time zone.

Currently the way we do this is the same way Moment Timezone does: there's an Elm package which ships IANA timezone data and provides an API for turning the time zone strings into Elm's Time.Zone values.

This has a few problems, which this writeup explains in greater detail. (Moment Timezone faces the same issues we do.)

The key for us is being able to expose an Elm function which takes a String and returns a Time.Zone value which contains up-to-date information about that time zone. It would be great if someday we could get that information directly from the browser!

[ljqx]

Wouldn't this time by time introduce

1. breaking change
2. inconsistency of result for same code in different (version) browsers

?

As time zone data changes frequently. As an example for IANA tz data: 2018e => 2018f

[rtfeldman]

@ljqx That concern applies to dealing with browser-supplied time zones in general, and is much broader than this issue. (For example, the concern applies equally to ZonedInstant in the current proposal.)

I would suggest opening a separate issue to discuss it!

[littledan]

@rtfeldman Thanks for the pointer; I didn't know about the TimeZone proposal.

To be clear, you're interested in Time.Zone and not Time.TimeZone, right?

This proposal (and ECMA-402) use the IANA tz strings directly, rather than a wrapper class. I'm wondering, what sort of data do you need in Elm that you can't get by passing the timezone to the right temporal API?

[rtfeldman]

To be clear, you're interested in Time.Zone and not Time.TimeZone, right?

Correct!

[rtfeldman]

This proposal (and ECMA-402) use the IANA tz strings directly, rather than a wrapper class. I'm wondering, what sort of data do you need in Elm that you can't get by passing the timezone to the right temporal API?

This is a great question! Let me take a moment to outline why the current Temporal proposal doesn't address Elm's use case.

First, the only API in the current Temporal proposal that accesses up-to-date TZDB information is ZonedInstant.

In contrast, Elm's elm/time API is designed to decouple Zone from Instant. At first glance, it might appear that elm/time's API could be implemented using ZonedInstant behind the scenes, by doing something like this:

1. Have Zone internally store only the time zone string (as opposed to actual offset information, like it currently does)
2. When an elm/time user calls a function involving both a Zone and an Instant, behind the scenes construct a Temporal ZonedInstant, use that to perform the calculations using the up-to-date time zone data, and then immediately discard it.

There are two major problems with this approach. One is an API limitation that does not currently exist, and the other is a show-stopper that prevents elm/time from using ZonedInstant at all.

First, the API limitation.

Problem 1: Listing Time Zones

The hypothetical approach I mentioned earlier rules out being able to display for the user a dropdown of available time zones to choose from.

This use case is currently supported through a third-party Elm package as well as in Moment TZ.

Users who need this functionality would have to download a separate bundle which would naturally get out of sync with the time zones known to Temporal.

Problem 2: Purity Requirements

This is the show-stopper that prevents elm/time from using ZonedInstant at all.

All Elm functions are pure. If I write an Elm program and I pass it the same inputs, it must do the same thing every time I run it. This is a foundational guarantee that the language is built around, and all sorts of things would break if it were violated.

By design, the ZonedInstant constructor is not pure; it can return different values when given the same arguments—because the most up-to-date information for that time zone may have changed.

This means if we want to instantiate a ZonedInstant from Elm, that operation must be treated as an asynchronous Task. (This is why elm/time's function for getting the current UTC Offset returns a Task. It doesn't have a choice!)

However, this means that the "create a ZonedInstant behind the scenes just long enough to do the calculation" approach will not work. For example, take the toHour function in elm/time:

toHour : Time.Zone -> Time.Posix -> Int

This would not be implementable using the current Temporal proposal. It would have to change to something like:

toHour : Time.Zone -> Time.Posix -> Task x Int

Since Tasks are asynchronous, this would mean it would no longer be possible to synchronously determine the hour of a timestamp in a given time zone using elm/time.

This would be such an ergonomics catastrophe that I doubt anyone would use it over the status quo (where they have to download time zone bundles separately, even if the browser has all that information already), so it's more likely that what would happen is that elm/time would not use Temporal. 😄

Hopefully that clarifies, but happy to elaborate further!

[littledan]

This helps a lot, thanks!

1. I can see how this enumeration function would be useful. It reminds me of the demand for enumeration of languages, scripts, regions, currencies, etc which Intl faces. As these need to be localized for display, I wonder if they would make sense as part of the Intl.DisplayNames proposal. What do you think?
2. I am not sure the tz database will be updated so promptly in practice. I thought it would usually take a browser update and restart to get the new one. (Is that right?) If we wrote in the spec that the database will stay constant over the lifetime of the page, would that address the purity issue?

[ljharb]

@rtfeldman when you say pure, do you mean pure within the life of the program? Or that forever, the same inputs would have to produce the same outputs, meaning that you need to be able to permanently snapshot timezone data? (In other words, I’d be surprised if time zone data would change during the lifetime of a realm - I’d expect the data to be constant for the life of the realm, in other words.

[rtfeldman]

I haven't looked into Intl.DisplayNames, but I will read up on it - thanks for pointing it out!

when you say pure, do you mean pure within the life of the program?

It's important that if I compile an Elm Zone -> Int function, it always returns the same Int when given the same Zone - regardless of what else is going on in the browser. The only way it should return a different String is if I recompile and change my implementation.

Having said that, it's totally fine if an Elm program gets back a different Zone value from the same time zone String (like "America/New_York"), as long as it's through a Task - that won't cause any problems!

If we wrote in the spec that the database will stay constant over the lifetime of the page, would that address the purity issue?

That would be better than not specifying when the updates happen (and I think it's worth including that in the Temporal specification regardless!) but it wouldn't be enough for us to safely use ZonedInstant to implement elm/time.

If purity gets broken, even if between runs of the same program, the symptoms are really difficult to track down to the root cause—because it's such a foundational assumption Elm programmers depend on.

We made the mistake early on of thinking it would be okay to use JavaScript's built-in Date under the hood because it would be "pure in practice," but ended up having to stop using Date altogether, because of edge case purity problems.

Making impure calls to ZonedInstant and guaranteeing they'd be "pure in practice" would likely get us back in the same place we were with Date!

[littledan]

I haven't looked into Intl.DisplayNames, but I will read up on it - thanks for pointing it out!

Intl.DisplayNames is extremely early. I hope we can make progress on it soon, but there isn't much to read yet. Use cases like this will be very useful in its design!

It's important that if I compile an Elm Zone -> Int function, it always returns the same Int when given the same Zone - regardless of what else is going on in the browser.

I still don't quite understand. What do you mean by, "what else is going on in the browser"? Does it need to be stable across a browser version upgrade and restart?

If so, it's not clear to me how the browser could even provide tz data without messing with Task, as the browser will sometimes need to update the tz database. Would evancz's time zone proposal meet your requirements of being usable without returning a Task?

Date has far more impurities than ZonedInstant, e.g. it updates the local timezone based on where the UA is within the lifetime of a page.

[rtfeldman]

I still don't quite understand. What do you mean by, "what else is going on in the browser"? Does it need to be stable across a browser version upgrade and restart?

Sorry for my lack of clarity!

I'm trying to choose my words carefully because it's not that the time zone itself needs to be stable across a browser version upgrade and restart (which would defeat the whole point), but rather that compiled Elm functions which don't return Task need to be stable across browser version upgrades.

It might be helpful to give some context on how Elm tends to interact with JavaScript APIs. In general:

• If the JS function is pure, like String.startsWith, Elm can expose a synchronous function which uses that JS function under the hood.
• If the JS function is not pure, like Math.random(), Elm will expose it as an asynchronous Task (or as an asynchronous Cmd - the distinction between Task and Cmd is whether the effects are chainable, which isn't relevant here, so I'll stick to Task for simplicity's sake).

So the fundamental question behind "Can elm/time's toHour : Time.Zone -> Time.Posix -> Int function use a particular JS API under the hood?" is whether that JS API is all of these things:

• Pure
• Takes a materialized time zone value and a number (ms since epoch)
• Returns the hour as a number

ZonedInstant works with a time zone String and a number, and provides ways to get the hour from those.

However, because it takes a String instead of a materialized time zone value (with the actual offsets that will be used in the calculation, so that the calculation consistently gives the same answer every time), ZonedInstant can't be pure, which means Elm has to use Task to represent that API.

If so, it's not clear to me how the browser could even provide tz data without messing with Task, as the browser will sometimes need to update the tz database. Would evancz's time zone proposal meet your requirements of being usable without returning a Task?

Evan's proposal would do the trick, yeah.

The key is that if the browser exposes a function like getTimeZone("America/New_York") which returns a materialized time zone value (with offsets and all that, for consistently reproducible calculations) then we can implement that in Elm as getTimeZone : String -> Task x Zone (which is what Elm programmers already do when starting up their applications, except that today they only get UTC offset) and then toHour : Time.Zone -> Time.Posix -> Int can be implemented on top of that with no problems.

However, it's not possible to use ZonedInstant to implement that getTimeZone : String -> Task x Zone function in Elm, because ZonedInstant has the additional requirement of providing the instant up front.

Date has far more impurities than ZonedInstant, e.g. it updates the local timezone based on where the UA is within the lifetime of a page.

That's totally fair! The problem is that for us, it's all or nothing; more impurities are worse, but any at all means we can't use the API to address our use case. 😄

Thanks for your patience on this, by the way! I really appreciate your taking time to understand the situation thoroughly. ❤️

[ljharb]

Given that time zone data is never constant over time (it changes based on legal decisions, for example), it kind of seems like inherently you’d need it to always be asynchronous in any paradigm/language, with the constraints you’ve outlined.

[rtfeldman]

I'd say it's inherently effectful for that reason.

In Elm, all effects are necessarily handled asynchronously due to purity requirements (so effectful implies asynchronous), but in JS it's possible to run effects synchronously - e.g. localStorage.setItem().

[ljharb]

@rtfeldman ok - so by those definitions and under those constraints, it would seem to me that the only way the language could provide timezone data that would meet your requirements for synchronous usage is if the engine retained all historical and future snapshots of timezone data, and if you were able to determine a fixed point in time with which to apply time zone logic. Am I understanding the situation correctly?

[gibson042]

@ljharb I don't think that is necessary, because it seems like the necessary escape hatch already exists. @rtfeldman is asking for a way to get the data for a time zone (e.g., a characterization of every UTC offset shift) so that the only asynchronous aspect of temporal arithmetic in Elm will be retrieval via Task (a one-time cost), followed by synchronous pure use (e.g., "according to given tzdata [such data being static but not necessarily identical to the next result for its time zone string], what is the local time at instant t?").

[ljharb]

Ahhh, thank you for clarifying. So the ask is to be able to somehow provide an asynchronously retrieved snapshot of timezone data, rather than a string (that could produce different data over time), to specific date/time operations?

[rtfeldman]

So the ask is to be able to somehow provide an asynchronously retrieved snapshot of timezone data, rather than a string (that could produce different data over time), to specific date/time operations?

Precisely! 😃

[ljharb]

I've already communicated to the champions that I want to be able to pass around a TimeZone object rather than a string (ie, the two would be interchangeable), although I was imagining no such guarantee as above - but I'm not sure it would harm any of my use cases to have it (ie, that timezone data was frozen at construction time of a TimeZone object).

[rtfeldman]

Nice! Were you thinking that the TimeZone object would hold enough info necessary to do the calculations?

[ljharb]

Not in a user-visible way, of course, but yes - my thinking was that just like the internals see a string, and know how to look up the data - they'd know how to get the data based off of a specific TimeZone object.

[rtfeldman]

Gotcha!

🤔 Is there a reason the calculation info couldn't be exposed in a user-visible way?

[ljharb]

No idea, but I'd assume that exposing that data directly would be a much larger API for browsers to ship and adhere to, which might preclude some optimizations and make it difficult to get consensus.

[littledan]

@rtfeldman Thanks for bearing with me; your explanation in #103 (comment) makes sense about the act of getting the timezone returning a Task.

I am wondering, could the core of Elm "cheat" and provide this same getTimeZone interface returning a Task x Time.Zone, but actually have Time.Zone be wrapping a String? This string would then logically own the tz info for that time zone. In the implementation, the data wouldn't be read from the TZ database until constructing a ZonedInstant later, but it's hard for me to see how the result would differ in terms of observability/purity.

[StoneCypher]

Speaking as someone who had to do timezones in Java in the years that Oracle withdrew from the standard timezone database for hardware security, I can say that the ambiguities present in Javascript's TZ implementation are easily repaired and fundamentally currently insurmountable

I'll repeat what I said else-ticket. My opinion is that all we actually need is:

1. What is the current local timezone
   1. With daylight savings
   2. Without
2. The ability to create new Date objects while specifying a timezone
3. The ability to ask a Date object whether it is in DST

Those five things would allow a user-land implementation to fill any needed gaps at any level of complexity I'm aware of (short of things like (leap seconds and support for historic Russian calendars)[https://www.youtube.com/watch?v=-5wpm-gesOY])

I am enthusiastically in support of adding a handful of methods to repair this

[littledan]

The W3C TAG expressed strong support for including a low-level API for access to the timezone database in w3ctag/design-reviews#311 . Could we do do this in the TC39 Temporal proposal, alongside the high-level API?

[StoneCypher]

you're the guy who closed my proposal years ago

but now that the right people support it, you want one

[littledan]

I see multiple people asking for the same thing as a useful signal and data point. We can push back on feature requests to a point, but eventually it's time to reconsider. Thanks for your help in getting to this point.

[jswalden]

tzdb data exposure seems like a fine thing to have, for sure. Adding it to this proposal, feels maybe scope-creepy?

I had thought -- I could be wrong, edumacate me if necessary -- the aim of the proposal was to rectify the inadequacies of Date in terms of representing stuff that an unadorned milliseconds-from-epoch could not. Exposing intricate time zone information feels afield from that, or perhaps orthogonal. And my suspicion is the majority of value to improving all this broad area of functionality, lies in the APIs already roughly envisioned.

Now, maybe that's just solvable by doing everything in this proposal, then -- if the existing ideas are firmed up sooner, uplifting just them and letting the tzdb lower-level bits follow when they're ready. I don't know if our processes allow that. I'm not much of a rigid process-adherence fanatic, myself, for any of this -- I just care that stuff gets done. But if we could fork off CivilDate and other things if they firm up sooner, that would fully address my worry about tzdb stuff delaying everything else too much.

[littledan]

Well, maybe I'm being naive, but I think we could provide the timezone data in a way that fits in well with a high-level API. The current proposal accepts strings as timezone parameters. We can keep doing this, while also making a Timezone class, which takes a string as a parameter, and can be used as the timezone argument in various places. This Timezone class can have a method for getting at the underlying timezone data.

If we decide we want a Duration type, we could do something similar, of continuing to allow the convenient object literals, while also supporting this more strongly typed concept.

One thing that I really don't want to happen is for Temporal to be a JavaScript standard, and underlying timezone database access to be theoretically a web-only feature. That layering feels wrong.

[kaizhu256]

i also feel tzdb is a hard-to-generalize, client-specific/presentation-logic problem that's perhaps out-of-scope.

most business-logic need only utc-value and timezoneOffset to do their calculations/aggregations:

{
    "utc": "2019-02-08T00:00:00.000Z",
    "timezoneOffset": -360 
}

complicating it with tzdb-lookups at that stage, rather than at the [implementation-specific] input-validation/presentation stage is usually an anti-pattern.

[rtfeldman]

most business-logic need only utc-value and timezoneOffset to do their calculations/aggregations

This is not enough information to account for daylight savings time changes, among other edge cases.

There's a reason tzdb stores more information than these two data points!

[littledan]

Right, I definitely wouldn't want to expose something incorrect like @kaizhu256 is describing. This proposal's schema looks reasonable to me, though we might leave describing the timezones to Intl.DisplayNames rather than this API (since both the abbreviation and long name are locale-specific (yes, there are multiple locales within a timezone)).

[pipobscure]

I do not understand problem 2. Your claim is that the ZonedDateTime constructor (formerly ZonedInstant) returns a different result given the same inputs.

That is true if and only iff the underlying timezone database changes (i.e. a seamless browser/os update while elm is executing). In which case exposing the nderlying timezone database directly would trigger the same problem.

If the underlying database changes, then you would have the exact same problem in pure elm.

In short,

let instant = new Instant(1553262381001000000n);
let zoned = new ZonedInstant(instant, 'America/New_York');

will return the same result every time. Now considering that the value itself is an object, the only way to solve that is to use WeakMap to dedupe objects.

let map = new WeakMap();
function makeZoned(instant, zone) {
  let submap = map.get(instant);
  if (!submap) {
    submap = new Map();
    map.set(instant, submap);
  }
  let zoned = submap.get(zone);
  if (!zoned) {
    zoned = new ZonedDateTime(instant, zone);
    submap.set(zone, zoned);
  }
  return zoned;
}

Now for any given input you have === guarantees for the result. So problem 2 (the show stopper) really is a non-issue if I understood it correctly.

As for problem 1, a listing of all timezones may well be worth while. I think it's just out of scope for this proposal and easily doable individually now.

I am thinking of proposing a direct tzdata API as well. But I'd like to get temporal done first.

[jorroll]

@pipobscure perhaps you missed this comment clarifying the issue? #103 (comment).

The way I understand it, the problem (simplified) is:

Bad (impure) function:

1. A function takes a timezone string and a local time as arguments. The function uses the timezone string to ask the timezone database to convert the local time to a UTC time. If the function is run again, and the timezone DB has changed, then, for the same inputs, the output has changed.

Good (pure) function:

2. A function takes a timezone object, which represents a snapshot of all the timezone information at that instant, as well as a local time as arguments. The function uses the timezone information contained within the timezone object to convert the local time to the UTC time. If the timezone DB changes then a different timezone object is produced, which means that the function's result only changes when the inputs change.

The problem is that (apparently) there is no current way to get the necessary timezone object, so you are forced to go with scenerio #1.

[maggiepint]

Given that a TimeZone object has been added, is this issue possible to close?

[ryzokuken]

@maggiepint thanks! Closing this.