API proposal/RFC - System.Time namespace

[Clockwork-Muse]

(aka "programmers for good dates")

Related:
#14089 - Adding Date and Time types.
#14052 - Deprecating DateTime.Now.

See also:
What's wrong with DateTime anyway?

Date and Time are foundational concepts in our daily lives, and in programming; it's why we have DateTime.
Unfortunately, DateTime isn't sufficient. Date and time are part of a larger ecosystem of concepts, ranging from an absolute point in time something occurred (an Instant), to the value reported on a wristwatch (LocalTime). They're related, and you can (usually) transition or convert from one to the other, but they are very definitely not the same thing.

But wait! We have NodaTime! Well, the biggest problem with NodaTime is... it's a separate, third-party library. It means we can't take dependencies on it. This results in things like OData having a separate Date type for XML compliance (and there are other examples in different namespaces). There's a lot of stuff that NodaTime does, and does well, and most of it is stuff that the default libraries should be handling (like, say, time zone ids from non-windows sources).

What about patching some common holes with new, standard types (ie, Matt's Date and Time types)? This alleviates some problems for us, but leaves the problem when interacting with NodaTime. That is, "Why do I have to use the deficient framework types/something else to do XML serialization!?". (Note NodaTime has been looking to get the library installable on SQLCLR; I don't think it's made it into a release yet). Oh, and it means we have to keep three time zone databases in sync, instead of two (at least on windows).

---

So: I'm proposing a new namespace as a modern, good implementation of a date and time library, our own version of JSR310/java.time . We need a good set of narrow value types, and a range of support classes to handle conversions and many other use cases.

Unfortunately... I... don't have an API on hand: I'm creating this issue to get things started. Coming from the Java world, I really like the new java.time library, but don't know if we can (or should, really) do a straight port. It's possible we might just use NodaTime; however that library was deliberately implemented with limited extensibility, something we would need to consider (ie, no custom calendars). It would be good to ask Jon Skeet what version 8.0 would look like, if we went that route.

---

Known pain points:

• VB aliases DateTime to Date. This limits type names somewhat.

---

End-of-life considerations:

First, even if we had an implementation now, corefx isn't releaseable yet. Then, there's whatever delay is involved in possibly getting such a library into the desktop release, and into wider distribution. So nothing is going anywhere any time soon.

• NodaTime: The aim (regardless of whether we include it or write a new one) is to supersede the standalone library. We should be covering most, if not all, of its current use-cases.
• DateTime (and other existing types): eventually, mark obsolete/deprecate (and remove?). Timeline for this is probably multiple versions after release, although documentation should be changed to point to the new types as part of the initial one.
• VB aliasing: Tricky. The current usage implies the wrong type (that is, it's not just a calendar date). It's possible the aliasing should be removed altogether, if the old types are removed. Leaving it behind would be a major confusion point.

---

Interested parties (?):
@jasonwilliams200OK , @mj1856 , @gafter , @paulirwin , @tarekgh, @eatdrinksleepcode , @michaelstaib , @HaloFour , @ellismg

@jskeet (Jon Skeet - NodaTime lead)
@jodastephen (Stephen Colebourne - JodaTime and java.time implementer - informational/possible wisdom only)

[mattjohnsonpint]

WRT VB's aliasing of Date, See mattjohnsonpint/corefx-dateandtime#29. One can use brackets to prevent the aliasing, as in [Date], but it only works under the new compiler. The previous VS2013 compiler refuses to accept another type named [Date] living in the System namespace other than the one aliased to System.DateTime. (Not sure if that matters or not if the new API is for .NET Core only.)

If this new API lives in a System.Time namespace, that would resolve the compiler issue, but VB devs would still have to remember to use brackets to prevent the aliasing.

[HaloFour]

I think that instead of Date and Time that new types be called LocalDate and LocalTime. That eliminates the problems with VB's alias. This is also the naming convention used by JSR310, presumably because Date was also already taken.

I think that there are a lot of good ideas between Joda/Noda and JSR310. However, before we can really move forward fixing the time issue I think that .NET needs a better answer to the time zone issue. The current TimeZoneInfo class is better but it's still not nearly as flexible as tzdb. I think that this is fundamental to designing a proper date/time library.

[Clockwork-Muse]

@HaloFour - That's one of the reasons I posted the issue; to get extra stuff to look into.
In truth, I was already thinking about something like that anyways, if for no other reason than to make it part of a unified namespace. Things like System.Globalization.Calendar would be merged (or otherwise ignored, and eventually removed).

Borrowing and tweaking the table you posted in an earlier issue, we need (at minimum) narrow types for the following areas (ignore names for now):

BCL Type	Purpose
Date	Represents only a date, ignorant of time or time zone.
Time	Represents only a time, ignorant of date or time zone
(Local) DateTime	Represents the combination of Date and Time, ignorant of time zone (essentially DateTimeKind.Local similar to DateTimeKind.Unspecified)
TimeZone	Represents a named time zone complete with historical and transition rules ZonedDateTime
TimeOffset	Represent an offset from UTC (ie, + 2:00)
OffsetDateTime	Represents the combination of a Date and Time with an offset from UTC

... a big question at that point is, do we go the JodaTime/NodaTime route and have Date maintain references to a calendar, or go the Java 8 route and have distinct date types for each calendar (and they and Date implement some common interface).

Then there's the common utility types, which are slightly less important but still nice to have .

BCL Type	Purpose
Instant	An absolute point in time, and knows nothing of dates or calendars (and maybe time?). Usually historical (should log with this)
Period	Difference between two Dates
Duration	Difference between two Times
Clock	Allow repeatable test results for calls to Date.Today, and similar.

[jskeet]

DateTimeKind.Local isn't equivalent to "ignorant of time zone" - it means "in the system default time zone". DateTimeKind.Unspecified is closer to "ignorant of time zone". This may be a good reason to avoid using the term "local" in any new API; I somewhat regret using it in Noda Time, although I don't currently have a better suggestion.

I would also suggest TimeOfDay instead of Time - partly as that enables System.Time as a namespace.

[Clockwork-Muse]

@jskeet - Thanks. Guess I'm too used to the Java 8/JodaTime convention. Haven't had the opportunity to do much in C# yet.
Names being taken sucks. System.Time would be a nice namespace, but there's also the System.Timers namespace, which might cause some confusion (I just picked something easy to understand).

[jodastephen]

For a namespace, might be worth considering System.temporal.

For classes, might be worth considering JSR-310 names as is, simply to aid developers moving between languages. (The "Local" part of LocalDate comes from ISO-8601's terminology).

Step 1 in a process like this is to understand the mistakes of the past design, examine the bugs that get raised. Step 2 is to decide on the value types. Step 3 is to figure out the level of extensibility you want and find a way to make that work. (JSR-310 is extensible in calendar system, fields, units and value types for example)

[Clockwork-Muse]

@jodastephen - re Step 1: any insight WRT java.time?

[jodastephen]

See https://github.com/ThreeTen/threeten/wiki for some captured thoughts.

[paulirwin]

Thanks for tagging me due to my suggestion #14089. I'm happy to see this conversation kicked up again.

My only input in looking at the Java world is to make sure that our equivalent of getMonth() isn't zero-based :-) (I kid, I kid... I know that java.time.LocalDate finally righted this wrong.)

I did not know about the LocalDate name coming from ISO-8601, thanks @jodastephen for that info. That changes my mind and makes me okay with using that name for the .NET version.

I'm sorry I don't have much more input here at the moment, I exhausted my input on #14089.

[ellismg]

Thanks for tagging me. I'll admit that I have not gone deep on a bunch of the improvements NodaTime brings to .NET, but I'll try to help out however I can.

One question I have to start with (which will probably lead to a bunch of follow up questions) is about this statement:

Well, the biggest problem with NodaTime is... it's a separate, third-party library. It means we can't take dependencies on it.

It's unclear to me what "we" means here. Does this mean that types in CoreFX can't take a dependency on it? Is it that other libraries outside CoreFX don't want to take a dependency on NodaTime? Something else entirely? I'm interested in understanding the differences between some new time APIs in a future version of .NET Core vs a future version of Noda Time that runs on top of the .NET Core surface area and hence can be used by any library targeting .NET Core. What does the former provide that the latter does not?

When all of the libraries in .NET Core ship on NuGet and can version independently from one another and have no ties to the runtime itself, I struggle to see the difference between something in CoreFX vs a popular "third-party" library like NodaTime, JSON.net, etc.

[Clockwork-Muse]

@ellismg - the 'we' here was intended to be corefx. Some in-house devs may also not be allowed to take 3rd party library dependencies, but this is rare and not an interesting reason. The former (doing a new library) would allow us to pull in tertiary types (like OData's Date type), and to allow wider use (OData's Date doesn't work directly with the globalization calendars, you have to take a trip through DateTime). It's enabling corefx to accept or return a better, more specific type than DateTime.

If the future deployment model would allow us to use NodaTime types in corefx, I could probably work/live with that.

[mattjohnsonpint]

Looking at the roadmap, I see System.Data.SqlClient is pending. This is a good example of an assembly that would rely on the changes we're discussing here.

Would it be reasonable for System.Data.SqlClient to depend on a .NET Core version of Noda Time? Does having a Microsoft made System.Time assembly change anything? Does it make a difference?

Or perhaps the time-portions would be broken out into System.Data.SqlClient.Time which then took the dependency on our time library while the main assembly didn't. But that could lead to a lot more splintering... How much fragmentation is too much?

What about backwards compatibility? If we are suggesting that these new types be created independently, are we also suggesting dropping the current date and time types? What about all of the existing code that folks will want to port to .NET Core? It would be quite frustrating to find that you had to make tons of changes because DateTime wasn't there any more.

Besides, there's really very little wrong with DateTimeOffset and TimeSpan. They are perfectly usable and functional. And DateTime works just fine for UTC and Unspecified kinds. It's really only the Local kind that gets folks into trouble (and just not being aware of Kind in the first place). Heck, even TimeZoneInfo isn't all that bad from an API perspective. It's really only the underlying data that is the issue there.

I'm not trying to be a naysayer, but IMHO, thrusting a whole new and different time API on folks is bound to be problematic. Likewise, doing nothing when we know there are issues also has it's consequences. I think the key to success here is balance. Add the missing pieces, while keeping the good parts intact.

Here's a detailed proposal that I think achieves everyone's goals (please correct if I'm mistaken):

• Create a System.Time nuget package, and move the following to it:
  • System.DateTimeOffset, System.DateTime, System.DateTimeKind, System.DayOfWeek (and any other supporting types I may have forgotten).
  • System.TimeZoneInfo and subclasses (but see below for changes)
  • System.Globalization.DateTimeFormatInfo, System.Globalization.DateTimeStyles
  • System.Globalization.Calendar, System.Globalization.GregorianCalendar
• Create a System.Time.Calendars nuget package for all of the other calendars, (or possibly create a separate package per calendar)
• Leave System.TimeSpan where it currently resides, in mscorlib. There are too many things that need a measure of elapsed time that shouldn't need a dependency on a date/time library.
• Include the code from mj1856/corefx-dateandtime, which includes:
  • new System.Date and System.TimeOfDay types, which plug two large holes in the current API.
  • Some extension methods that can be upgraded to instance or static methods during the migration.
• Keep all of the existing namespaces, for full compatibility.

Other important changes that would be introduced:

• Upgrade the System.TimeZoneInfo class to make it use a provider model.
  • Create a public System.TimeZoneInfoProvider abstract base class.
  • Create a System.SystemTimeZoneInfoProvider class to be the default provider. It would provide a pass-through to whatever was on the host OS. In other words, the Windows implementation would read from the registry as it does today and return Microsoft time zones. Other OS's implementation would read from their system's zoneinfo, returning IANA data. This would sit alongside the base class in the System.Time nuget package.
  • Create a System.MicrosoftTimeZoneInfoProvider class and put it in its own nuget package (eg, System.TimeZoneInfo.Microsoft) for Microsoft time zones that embeds the time zone data, rather than taking it from the operating system. (Useful for cloud projects, since one can't necessarily apply time zone hotfixes to a paas environment)
  • Create a System.IanaTimeZoneInfoProvider class and put it in its own nuget package (eg, System.TimeZoneInfo.IANA) which embeds IANA tz data.
  • Third parties would be free to develop other time zone providers if desired. Examples of other providers that might be written:
    • An IATA provider for the airline industry.
    • If the tzdist project takes off, one might want to write a provider that pulls IANA data directly from tzdist servers.

There are really only a few other things that Noda Time provides that are absent from the above. We can debate/discuss these:

• Period - a type to represent any combination of of distinct calendrical units or time units (ex. "2 years", "4 months", or "7 years, 3 months, 2 days and 5 seconds"). Aligns with ISO8601 "durations".
• ZonedDateTime - a type to combine a DateTimeOffset with a TimeZoneInfo, making certain operations "time zone aware", such as adding time across a DST boundary.
• Instant - a type that represents a distinct moment in time, which can also be thought of as "always being in UTC". Equivalent to DateTimeOffset with zero offset, or to DateTime with UTC kind.

There are a few trivial bits also, such as Offset and Interval, but I don't see a whole lot of value in those coming over.

After all is said and done, there would be a few new .NET Core nuget packages, which I would expect assemblies like the aforementioned System.Data.SqlClient to take a dependency on.

@Clockwork-Muse - Do you think this goes far enough? Or are you really set on the death of DateTime? 😉

@ellismg @tarekgh - does this sound reasonable? Or is it overreaching?

[Clockwork-Muse]

@mj1856 - I think I'd prefer the death of DateTime through neglect (nobody uses it) in favor of something better. I don't want people yelling at us for it being gone - at least at first I'd prefer gentle encouragement to use something better (say, a StyleCop rule or something). Make the new API available but not even deprecate the old one (at least for a while). People could use the existing libraries with minimal/no changes, and update to the new API as time allowed. (All of a sudden I'm wondering if we could distribute a Roslyn syntax rewiter to do gross conversions, based on types and method calls... Tricky though).
I mostly proposed a new, separate API for a cleaner break from the old, and to give any new design more potential freedom in implementation.

One of the biggest problems is dealing with DateTimeKind, and the issues it causes. I have little issue with TimeSpan or DateTimeOffset as they're (probably) implemented (although DateTimeOffset.Date returns a DateTime, which isn't really correct). (The purist in me also wants TimeSpan in here, too, but I mostly get what you're saying about the dependency.)

ZonedDateTime doesn't feel to me like it has DateTimeOffset in it. Yeah, you can get it without much trouble, but it's really Date and Time with 'location' attached. Ehhh, semantics really.

Dealing with time zone info providers does sound like an interesting idea. I'm not as up on some of the deployment/management side of things, though.

I personally (just for right now) dislike how you have to use the current calendar system, but don't have problems with it otherwise. A good Period class would help in that regard, anyways.

[ashmind]

Just a few points based on my .NET experience.
(I'm on mobile so sorry for any formatting issues)

1. Whatever the final API is, I suggest it should be harder to use non-timezoned date by default. I've seen so many people not being properly aware of DateTimeOffset, and it is a pain and leads to many hacks when people wake up to the reality of timezones. It would be cool if YourNewNamespace.DateTime was timezoned, and some explicit name like InvariantDateTime was used for a date that should be the same in all timezones.
2. One thing missing in current API is timezone abbreviations (PDT, GST, etc) and corresponding parsing/formatting. Those are somewhat ambiguous, which complicates parsing/formatting API, but at least having them listed on timezone would be very useful for human-friendly date representation.

[tugberkugurlu]

@ashmind

Whatever the final API is, I suggest it should be harder to use non-timezoned date by default

I believe UTC is not a time zone and its use should be completely encouraged. As far as I can understand, you want the DateTimeKind.Local usage to be discouraged which is a very valid point.

[tugberkugurlu]

In NodaTime, I really like the way how the time zone info (e.g. DST, etc.) gets updated. AFAIK, this information changes based on the government rules. So, I should be able to update this by myself or the official update channel should be very responsive to changes and releases should be frequent.

[HaloFour]

IIRC TimeZoneInfo would require various other internal enhancements as well. I think it only supports the concept of a single "base offset" which doesn't reflect how potentially complex time zone rules have changed over time apart from just DST changes. Then there is TimeZone, would be nice if that class can either get folded into the mix or just go away.

I'm on the fence about System.DateTime. I'd love to find a way to include it within the rest of this family simply because it's so prevalent and because it'll likely remain the first date-related type that any new developer will find. It's not that far off from what a LocalDateTime type would be, although Kind does throw a little bit of a monkey wrench into it.

@ashmind The Microsoft time zone information doesn't contain those abbreviations at all. The IANA tzdb does for supported time zones, but they are mostly useless in parsing scenarios. You'd need a known time zone as a starting point, at which point the abbreviation can aid in resolving ambiguous times at DST boundaries.

[tarekgh]

I believe we need to take some time to think more about the whole space. exposing a lot more date/time classes will create more complexity. I understand the need of such types but we need to think how all of this will fit with what we have today. this is why we suggested to start with what @mj1856 creating and have the experiment to know what most needed types and how simple/complex it is. also we'll see what is wrong in such design. #14089 has the detailed discussion about this direction.

I want to second @ellismg point. we need to encourage people to use and consume the third parity packages. Json.Net is very good example of that. not every type should be part of the framework core. we need to understand why people can't take dependency on such Third parity libraries. deploying such third parity library shouldn't be different than deploying the corefx itself.

@mj1856 as we have talked offline a couple of months ago regarding different issues. and I have shared my thoughts with you. mostly what you have mentioned align with what we thought but we need to spend some more time exploring different options and design to make sure everything will fit nicely especially with types like TimeZoneInfo which is shared with the desktop framework.

[mattjohnsonpint]

@HaloFour - The single base-offset issue was described in KB3012229, and has already been fixed in .NET Core and .NET 4.6RC. However, there's still a related API issue open in #14457.

WRT the TimeZone class, it's already been removed.

WRT abbreviations, only English abbreviations are in TZDB, and the TZDB list has discussed several times that many were just made up by the group to fill in the gaps. Some non-English ones are also in CLDR, but in general, not every zone has an abbreviation. I've got the CLDR abbreviations and names wrapped nicely in TimeZoneNames, which I plan to do a .NET Core release in a future update. I'm not sure if it would make sense to pull this in as a core dependency - though it would solve the globalization issues with TimeZoneInfo's properties (DisplayName, StandardName, DaylightName) which are affected by machine locale rather than the .NET CurrentCulture.

[paulirwin]

Third party libraries are just fine for non-primitive types or primitive types of niche usage. A calendar date is a primitive type that is universally used. JsonConvert is not a primitive type and not much framework code would call into it. However, quite a bit of framework code could make use of a primitive Date type, in particular SqlClient as @mj1856 mentioned. SqlClient not being able to have a 1:1 mapping to SQL Server's date type is the primary reason why I created #14089, but also things like HTML5's input type="date" mapping to an MVC parameter. My team has spent time - this week already, in fact - resolving issues arising from DateTime including time (usually midnight) when the domain doesn't need it.

[mattjohnsonpint]

@paulirwin - From your perspective, can you clarify the meaning of "primitive"? There is some debate on what this term actually means.

The C# Language Specification uses this word only twice, and in an indirect way.

The VB.Net Language Specification defines it more definitively as:

7.3 Primitive Types
The primitive types are identified through keywords, which are aliases for predefined types in the System namespace. A primitive type is completely indistinguishable from the type it aliases: writing the reserved word Byte is exactly the same as writing System.Byte. Primitive types are also known as intrinsic types.

Furthermore, VB lists Date as a primitive type, while C# doesn't mention DateTime in the language spec at all. Nor does it list it on the "Built-In Types Table" on MSDN.

If we apply C#'s definition to VB, then DateTime is not a primitive, and DateTimeOffset, TimeSpan, etc. are not primitives in either language.

So when we say "primitive", are we talking about a framework concept of primitive in some way? Or is there some other distinguishing factor in your mind that makes date/time related classes and value types a lower-level feature?

I can understand this lower-levelness at least with TimeSpan, given that it's used along with timers, tasks, threads, etc. But the rest of them seem like they aren't really at the lowest level to be called "primitive".

[Clockwork-Muse]

@tarekgh - I agree we should in general be encouraging people to use 3rd party libraries, but I'm pretty sure JSON.Net is a terrible example to use here:

1. As near as I can tell, it's packaged with the .Net library (they took a dependency on it).
2. There were already extension points for doing your own serialization/parsing.

In the case of date and time, there's a lot of stuff that is "stuck" returning DateTime, and can't have a more appropriate type (or custom type) returned instead.

I didn't really have any problem with @mj1856 's experiments. The problem was that the direction at the end of #14089 seemed more like, "We'll just do these few types, and tell people to use NodaTime for the rest". I firmly believe that's not a good solution long-term:

1. It still leaves most of the pain points currently present here, for coders to stumble into (DateTime.Kind Local?)
2. It forces anybody using NodaTime to need to switch to a completely equivalent (hopefully!) but different type for some common actions.

[paulirwin]

@mj1856 Sorry, I wasn't intending to use the word primitive in any kind of way as to imply something specification-related but I appreciate the digging into the meaning. What I meant by primitive was a value type that is a fundamental building block of framework code (IDataReader, System.Convert, etc), such as Guid, DateTime, TimeSpan, etc., even though these types aren't primitive in the sense that they can be literals (although perhaps they should).

[LordZoltan]

@mj1856 - I'm personally all for sourcing time zone info from the CLDR, it's an excellent resource and, from what I can tell, very accurate; plus it's updated regularly. Its format lends itself to generating the code via a tool.

There's also the possibility (getting ahead of us all, here, but just throwing it out there) that this could pave the way to start driving some of the culture and region information from it, too - the data is much more comprehensive than Windows, certainly.

That said, it would make the most sense to have a provider model (as mentioned earlier) for timezones et al, of which a CLDR-driven source would be just one implementation.

As wonderful as NodaTime is, I'm not entirely comfortable with the framework taking a dependency on it for something as fundamental as date and time. Its reach is pervasive, and I'd wager would very quickly make NodaTime seem to be a core 'part' of the wider framework, when in fact it isn't.

I'm very happy for a new, more flexible, Date/Time API to be introduced into the framework that supersedes the old one, but feel it should fall under the general .Net umbrella, not a third party umbrella.

I realise that that, along with much else besides, is a debatable issue, however, and that is also true for whether anything I've just said here has really added anything to the discussion!

[rebizu]

Hi there i am new here,
As far as i can see, nobody mentioned leep second, there is one at 2015-06-30 23:59:60
Have anybody thought about how a leep second can be handled in a new implementation of datetime?

For reference here is some light readings about leep seconds
https://en.wikipedia.org/wiki/Leap_second
http://www.timeanddate.com/time/leapseconds.html

[jskeet]

For Noda Time, I made a very conscious decision not to include leap seconds. Originally I believe JSR-310 was going to support multiple timelines, but ended up deciding that clock-smearing (with a simple "60 seconds per minute, always" model) was a better fit. My contention is that few applications require understanding of leap seconds, and it's reasonable for those applications to require other libraries or custom handling rather than imposing the complexity of leap seconds on all other applications.

More interesting to me is whether there's a clean way of representing a time-of-day value of 24:00, which has far more practical applications - but still ends up causing pain in various ways.

[Clockwork-Muse]

@rebizu -
I'm with @jskeet - we're unlikely to include leap seconds for pretty much exactly that reason. There's leap-second aware stuff for java.time in the 310-extras package (among other things), and I would recommend creating something similar for not-rare-but-not-common-enough use cases.

@jskeet - WRT to 24:00, one of the big problems being "does adding it to a date give the next day?" (when combining a date and time). It certainly would be nice for exclusive upper-bounds checking.
On the one hand I want to say that doing such a combination would be an error...("You should be using tomorrow!"), and maybe just have a special check or item. It would depend on how any new types combine date and time. Something else to consider, though, yeah.

[tarekgh]

I agree with @jskeet regarding the leap seconds too.

@Clockwork-Muse

but I'm pretty sure JSON.Net is a terrible example to use here
I just mentioned JSON.Net as example of the third parity library that is used by many application out there and people take dependency on it. I didn't mean we should not follow the packaging mechanism of this component.

I didn't really have any problem with @mj1856 's experiments. The problem was that the direction at the end of #14089 seemed more like, "We'll just do these few types, and tell people to use NodaTime for the rest". I firmly believe that's not a good solution long-term

as discussed in #14089 Matt's library is experimental components and any type proven in the field we'll consider adding it to the core. it is matter of time. the long term direction is we'll include all needed types in the core as long as it make sense these types to be there and in same time most of apps will need such types. as I mentioned before I am really worried exposing many date/time types which will add more complexity to what we are currently have and in end up not many people will use such new types. we still can add more types to Matt's library as needed.
recommending NodaTime also can help telling what apps really need from such library and the volume of the demand too. but in general we want to understand why people cannot take dependency on NodaTime? @jskeet may have more info here to understand the pain points with NodaTime

[eatdrinksleepcode]

At this point I feel the need to remind us all:

I am honestly very confused by the resistance to using NodaTime itself. @jskeet and @mj1856 (and others) have put a lot of effort into understanding both the (extremely complex) domain and the myriad of ways that it trips up developers every day, and designing a library to help expose these critical concepts as a "pit of success". IMHO they have been quite successful in that effort. I think we would be crazy to throw all of that away just to recreate it again. If there are problems that NodaTime doesn't solve, let's solve them. If there are design issues that make NodaTime less than ideal, let's fix them. But I have yet to see any justification for starting from scratch.

As near as I can tell, [JSON.net is] packaged with the .Net library (they took a dependency on it).

That's the whole idea: when we identify a gap in Core, and find that there is an existing library that solves the problem well, Core should take a dependency on that library rather than building it over again.

In the case of date and time, there's a lot of stuff that is "stuck" returning DateTime, and can't have a more appropriate type (or custom type) returned instead.

This is an unfortunate problem, but re-building a new date/time library in Core does not solve it. Other than taking massive breaking changes in the framework - which I am pretty sure is off the table - the best thing I know to do is ensure that our new date/time library has good interop with the existing types. But we're in luck! NodaTime already has such interop, allowing developers to convert back and forth in a reasonable way.

As a disclaimer, I should acknowledge that I have a couple of small commits in NodaTime from the relatively early days. But I don't consider myself particularly invested in NodaTime, nor is my concern whether my (minor) work in particular survives. My concern is this...

Well, the biggest problem with NodaTime is... it's a separate, third-party library. It means we can't take dependencies on it.

...the apparently still-prevalent philosophy that because a library is not published by Microsoft and doesn't have the "System.*" namespace that it can't be considered "part of the framework". This is the philosophy that has been holding back .NET open source for over a decade, but it has no place in the world of .NET Core. Needlessly re-inventing the wheel - as has already been done so often in the history of .NET - only reinforces the stereotype that .NET open source projects can't have a real impact because only "the framework" matters. We need to kill that stereotype if .NET is going to compete with all of the other popular development ecosystems whose beating heart is community-driven open source.

</soapbox>

[HaloFour]

@eatdrinksleepcode

While I agree with a lot of what you say I do think that if a part of the .NET framework takes a dependency on something that is not a part of the .NET framework then that something must become a part of the framework. It would be impossible for System.Data.dll to have a dependency on NodaTime without NodaTime then being distributed and maintained as a part of the whole because neither would work without the other. CoreFX being more modular might alleviate some of that issue.

I think NodaTime is good but it's also very complicated, moreso than the typical scenario requires. Chronologies throw a lot of common assumptions out the window in order to support a very small minority of use cases. Separation of chronologies is one reason that Java 8 did not just use JodaTime. Mutability is another. I don't doubt that ownership and maintainability was also a part of that conversation.

[whoisj]

Maybe what we need is a struct UniversalDateTime which is agnostic of any calandaring, and a abstract class LocalDateTime which requires specialized calendaring to implemented against a well crafted API. What that API is, I'd leave to the date/time experts who actually care.

I've pushed up an old type I created for myself (and updated it to use interpolated strings - yum), just for reference here: https://github.com/whoisj/xtypes/blob/master/Src/Timestamp.cs

[HaloFour]

@mj1856 I agree. It's amazing how much software really gets it wrong but companies either hack around it or deal with the pain through manual workarounds. I think the vast majority of that software is born out of simple needs running out of a single branch office location where the default local behavior just happens to be correct. Then they open a new office, or sign a new client in another country (or Arizona/Hawaii).

@whoisj That's pretty much how these libraries work. Instant being a value representing an offset in some unit from a fixed point in UTC time. Other types exist to interpret that value in the context of a given calendar, time zone, offset, etc. An additional data type to represent a "date" with no further granularity is also critical, something immune to reinterpretation due to time or time zone, etc.

[whoisj]

@whoisj That's pretty much how these libraries work. Instant being a value representing an offset in some unit from a fixed point in UTC time. Other types exist to interpret that value in the context of a given calendar, time zone, offset, etc. An additional data type to represent a "date" with no further granularity is also critical, something immune to reinterpretation due to time or time zone, etc.

Sounds perfect. My strongest push here is that there be a "common" type by which all others can be represented, such that there's a way to translate between the various time types both current and future. By agreeing to the statement any point in time can represented as an amount of elapsed time from a given agreed upon common point in time, then we can create such a "common" type.

I'm recommending we use milliseconds (because Ticks are far too granular for this king of thing), the date and time of 1-1-0000 00:00:00.0000, and allow both positive and negative values (both preceding and following the common-point-in-time.

From there, various calendaring schemes are moot. People can do as they please, 'cause I really, really do not care if somebody claims to live in a world where the number of hours in a day shifts around constantly or not, and years have 360 days in them. Have at it.

[HaloFour]

@whoisj The exact unit isn't really that relevant because you wouldn't be exposed to the value as a raw number. Given that .NET and NodaTime already both use 100-nanosecond increment "ticks" I think it makes sense to at least keep that level of precision. Java 8 Instant goes down to individual nanoseconds but also has a range of 2 trillion years.

[whoisj]

@whoisj The exact unit isn't really that relevant because you wouldn't be exposed to the value as a raw number. Given that .NET and NodaTime already both use 100-nanosecond increment "ticks" I think it makes sense to at least keep that level of precision. Java 8 Instant goes down to individual nanoseconds but also has a range of 2 trillion years.

Limiting ourselves to plus or minus 10,000 should be fine -- but can we agree to support minus 10,000 years? Some of us do care about dates in classical history and most of that is before the common era.

[ljw1004]

@whoisj - do you care about instants-in-time in classical history? or about dates in classical history? I imagine normally the latter, and that a simple DateTime would serve you fine. I can't imagine what kind of UTC/DST issues you'd even face with this.

[whoisj]

@whoisj - do you care about instants-in-time in classical history? or about dates in classical history? I imagine normally the latter, and that a simple DateTime would serve you fine. I can't imagine what kind of UTC/DST issues you'd even face with this.

Given that detailed record were kept as far back as 5,000 BCE - having the hour of the day would be useful. But no, I do not think anyone really cares about the seconds value.

[HaloFour]

@whoisj Looks like NodaTime already supports a range between the years -9998 and 9999 (actually it seems that Instant has probably a larger range but the API doesn't support making use of it in any practical way). Probably the only argument against either making the type more precise or expanding the range is exceeding the range of a 64-bit value. The further away from contemporary time we get the less relevant the precision matters, anyway. Classical dates aren't going to bother with milliseconds. Geologic dates aren't going to bother with days or months.

[whoisj]

@HaloFour agreed, but it would still be nice to be able to say "given this arbitrary date/time construct you gave me, how many years is it after Julius Caesar was born?" simply because it is useful to do so.

Additionally, dates stored in a database regarding simple date+time (down to minutes maybe) should be represent-able in the structure without the need to implement a custom structure. This allows utilities like SqlDataReader to remain useful without having to do funky conversions to Int64 and stuff which then invalidates the logic in the database itself.

I can see a need to do something like

using (var reader = command.ExecuteReader())
{
    while (reader.Read())
    {
        m_value.DateTime = (ClassicalDateTime)reader.ReadTimestamp();
    }
}

By ensuring the base type is common, this is trivial to implement. All of the "I don't care about minutes, seconds, etc can be handled by the ClassicalDateTime implementation. Can even enumerate the months using the old classical callendar complete with Latin names (if you so like).

[HaloFour]

@whoisj

My opinion is that very specialized use cases like that are probably the domain of a separate library. Trying to resolve the hour of a day 6,000 years ago is far from a common scenario.

[Clockwork-Muse]

@whoisj - the types you want are already being considered. I'm actually surprised that DateTime.MinValue is 0001-01-01 - I personally would want to make most types able to handle the BC range. That said, accuracy and translation of ancient dates are often inexact, so you may need a different library. Then there's the problem that some definitions have changed over the years; like the fact that March (?) used to be the first month.

@tarekgh - you fail to understand just how quickly things become advanced. Trying to get the duration between two dates, and you only care about the local time zone? This returns the wrong value if the range crosses only one DST boundary (or possibly more, if the offsets change):

public static TimeSpan TimeSince(DateTime earlierLocalDate) 
{
    return DateTime.Now - earlierLocalDate;
}

... because the subtraction doesn't actually care about DST. Even if it did, adding time doesn't either:

// Assume US tz that experiences DST
new DateTime(2015, 3, 8, 1, 30, 0, DateTimeKind.Local).AddHours(1);

... returns a value happily printing :2:30, even though this time does not exist. Oh you plan on doing all math in UTC? Unfortunately, that doesn't work too well either:

// Remember, some (non-US) time zones jump at midnight, not 2 AM, 
// so DateTime.Now.Date occasionally returns invalid values.
incomingUtcDateTime = new DateTime(2015, 3, 8, 0, 0, 0, DateTimeKind.Utc);
nextUtcDay = incomingUtcDateTime.AddDays(1);
// US-Pacific is at +8 here...
mappedLocalStart = incomingUtcDatetime.AddHours(8).ToLocalTime();
// ...but is at +7 here
mappedLocalEnd = nextUtcDay.AddHours(7).ToLocalTime();

... plan on keeping track of and applying the offsets yourself?

[whoisj]

My opinion is that very specialized use cases like that are probably the domain of a separate library. Trying to resolve the hour of a day 6,000 years ago is far from a common scenario.

@HaloFour I agree, but why actively design a system which cannot support it?

[HaloFour]

@whoisj

but why actively design a system which cannot support it?

I wouldn't, but I also wouldn't actively design a system that supported it
if that meant compromising the simplicity of efficiency of the common use
cases.

All that said, Noda demonstrates that a 64-bit instance can cover those
ranges so I don't see what the complaints are. However, we're stuck with
the reality that DateTime can't support those values.

On Wed, Jul 1, 2015, 5:13 PM J Wyman [email protected] wrote:

My opinion is that very specialized use cases like that are probably the
domain of a separate library. Trying to resolve the hour of a day 6,000
years ago is far from a common scenario.

@HaloFour https://github.com/HaloFour I agree, but why actively design
a system which cannot support it?

—
Reply to this email directly or view it on GitHub
https://github.com/dotnet/corefx/issues/2116#issuecomment-117825946.

[whoisj]

All that said, Noda demonstrates that a 64-bit instance can cover those
ranges so I don't see what the complaints are. However, we're stuck with
the reality that DateTime can't support those values.

Agreed (and to be fair I've not actually looked at Noda yet).

[mattjohnsonpint]

FYI - I created an issue for the DateTimeZoned vs method overloads in mattjohnsonpint/corefx-dateandtime#34. Let's move that part of this conversation there please. Thanks.

[Code-Grump]

I feel like I've arrived late to this party. Date and time is something I care a lot about and have felt pain on, especially around accurate scheduling. I've used NodaTime a lot recently, so the idea of the BCL moving past the mistakes of DateTime is very appealing.

One of the most valuable things the core libraries provide is a central "standard" that other, more specific libraries can work around. DateTime is supported almost universally by serialization and data-access libraries, despite the flaws, because it is the ubiquitous type for date/time representation. The issue with working with third-party libraries like NodaTime is that you face barriers trying to interoperate with other APIs expecting different representations.

Is there value in defining a few primitive types or interfaces, like Instant, which can become standards for representing specific values, and let libraries like NodaTime target these types to provide useful behaviours? Converting an Instant into your flavour of time representation gives us a way to choose the date-time library that's useful for our needs, and I think the vast majority of date-time related code is generally most interested with capturing, storing and comparing instants, converting to local time representations just for presentation.

A subject I haven't seen addressed yet is mechanisms of obtaining the current instant. One of the key things NodaTime did for my projects was to include an IClock interface. It's proven valuable in many unit testing scenarios and I would hope there are others out there that agree it to be a meaningful addition.

[whoisj]

@Tragedian 👍

[hmansell]

I just want to chime in from the perspective of the apps we have at BlueMountain Capital, since some folks (e.g. @tarekgh) are arguing that the need for something better than DateTime is niche.

We have prevalent problems with DateTime, mostly due to the offset being ambiguous and the fact that we run things in multiple timezones. DateTimeOffset is better, but the fact that framework and 3rd party libraries don't really use it means that we end up converting in lots of places, and that encourages developers who don't know the error of their ways to allow DateTime to infect our own codebase.

The implicit coercions between DateTimeOffset and DateTime are really unhelpful here, because people don't even know they are doing it wrong and end up with incorrect DateTimeOffset values because they are converted from DateTimes.

We also felt the need to define our own Date type (one of many) since it makes an API much clearer to know if it works in terms of dates or date/times. It also allows the use of adding/subtracting days, whereas adding days is essentially undefined for date/times unless they are zone-away (which neither DateTime not DateTimeOffset is.

This area is badly in need of improvement. And PLEASE don't do implicit conversions between the bad old DateTime and the new clean types.

My 2c.

[tarekgh]

@hmansell

since some folks (e.g. @tarekgh) are arguing that the need for something better than DateTime is niche.

this is not my position. it looks my points was not clear. here is some points I hope it clarify and summarize what I was saying:

• some people suggested deprecating DateTime and I was trying to tell this is not feasible. if we need to do some improvements we may need to have a new type which avoid all DateTime problems.
• I was not saying DateTime don't have problems. I am aware of its problems and this is why DateTimeOffset was originally introduced in the framework to address some of these problems. I agree we still have problems.
• I was trying to tell we also need to support the third party libraries (like NodaTime) and figure out what are the pain point of using such libraries to try improve the story there.
• I clearly mentioned before any type that make sense to be in the core should be in the core. the issue here is we need to make sure whatever type we introduce in the core should be really what majority of users want and also have the right design to address all problems and issues. this is why we have introduced Matt's library as one step validating the requirements and design before integrating it to the core. you see in this discussion there are a lot of opinions which need to be filtered and come up with right decisions. we already have the plan to introduce the needed type in Matt's library so we can start incorporate all needed types there and can move forward.

I hope this is clear enough to everyone.

[mattjohnsonpint]

Thanks @Clockwork-Muse for starting this this discussion, and to everyone who participated.

Moving forward:

• We're going to close this thread. No disrespect is intended - it's just too long to be productive in its current state.
• We're going to migrate the existing prototype over to the dotnet/corefxlab repo. This will happen sometime this weekend, or early next week.
• Once migrated, please provide feedback in the form of individual issues and/or pull requests in the dotnet/corefxlab repo. Please keep them focused on specific details, so we can move forward in a productive manner.

Thanks!

[Clockwork-Muse]

Closed for length issues.

[Neme12]

@tarekgh

I want to second @ellismg point. we need to encourage people to use and consume the third parity packages. Json.Net is very good example of that. not every type should be part of the framework core. we need to understand why people can't take dependency on such Third parity libraries. deploying such third parity library shouldn't be different than deploying the corefx itself.

It seems that times have changed (looking at System.Text.Json 😄)

[Clockwork-Muse]

With Windows now natively supporting leap seconds, and PTP becoming more of a thing, I've been contemplating a design that would support leap seconds from the beginning.

Not only do the current types not fully support leap seconds (because of legitimate back-compat concerns), NodaTime doesn't either, that I'm aware of. For that matter java.time also doesn't (and by default slews the clock for +/- some hours around it), although at least the extra package has TAI instants

[vladd]

I'm a little confused: is there a plan to include the new date/time API in the .NET 5 BCL? [Maybe there is a summary page for the planned/actively developed upcoming APIs, but I failed to find it.]

[stephentoub]

is there a plan to include the new date/time API in the .NET 5 BCL?

No, there is no plan to do so.