Support datetime data types as input #242
Comments
|
We should also support Python's datetime as input for datetime plots. |
leouieda
changed the title
|
Agreed. Pandas can already handle this type of data. I confess that I've never had this use case so I'd love to hear what people need from this. |
|
I would love to see this feature as well in pygmt. GMT has the best support for easy-to-read and beautiful time series plots. I use that feature even more than plotting maps. I especially like the interval annotation (where, for example, the name of the month is shown in between two tick marks, instead of underneath the first day of the month). Would love to be able to do all of this from pygmt. The basemap features with the nice annotation already work, but plot (psxy) does not yet accept datetime data. There are many date/time representations in Python. Plain-old command-line GMT uses ISO strings "2019-03-18T17:48:00.000", which would be a good place to start, but native datetime module objects and numpy.datetime64 would be nice as well. Otherwise pandas can easily help to do any conversions. |
|
@PaulWessel What's the correct way to pass string vectors to GMT? Like this one: The |
|
Did not anticipate this. One way would be to convert those to UNIX seconds since 1970 and pass that as a double. However, if you want to pass datetime strings then we would need to pass type = GMT_DATETIME and that would need to trigger a conversion from those strings to internal time. Would you also need to reverse, i.e., calling GMT_Get_Vector and if that column is abs time then return string array? Or perhaps we could make a GMT_DateTime () function that takes your string and returns a double for internal floating point time and then you pass that? What would be best from your perspective? |
|
Since telling GMT that you have abstime etc is done via -J or -f, that is separate from loading up an array in a column. So maybe for me to extend Get/Put to do the datestring to time and back is the simplest? |
|
Before we do, what is the equivalent issue in Julia and MATLAB, @joa-quim ? Regarding time representations. |
|
It's not only about datetimes. See this command-line example: It seems there are no API functions to pass the third columns (varying fonts) to GMT. |
|
Sure? All that stuff is part of trailing text, i.e., 5 1 is the two numerical columns and "12p,0,red 0 TL GMT TEXT1" is the trailing text. That is what GMT_Put_Strings is meant to do. Trailing text is its own special "column" |
|
IN contrast, your datetime string is meant to be a numerical column but it is given as a string that needs conversion. |
|
I like the GMT_Put/Get_Vector extension to GMT_DATETIME. With no interruptions (...) I would make a branch for that today. |
I thought the 4th column (the text angles) should be passed as a numerical column. |
Yes, I think that's a good and useful extension. |
Don't know. Never tried to plot with time |
|
pstext goes way back. The order of things were fixed and it mixed up text and numbers. Then we added the -F+j+a+f modifiers to specify the order, and even pull some of them out (+a45+jCB). IT is one of those things were backwards compatibility is a problem. Since angle is a number then it should always be part of the numerics (if it is given). I think from the externals you need to place Put_Strings with whatever order you have and let -F tell us. |
seisman
changed the title
seisman
changed the title
|
In PR GenericMappingTools/gmt#3396, the GMT API function With that PR merged, now it's possible for us to pass datetime vectors to GMT. See #464 for the possible implementation. Comments and suggestions are welcomed. |
|
Just had a quick look at #464 and it looks really promising! Been working with some time series data recently so I'd be keen to test it out once GMT 6.1.0 is out (looking at the calendar team!). One quick question though, how will Not-a-Time (i.e. |
|
Any datetime types (raw strings, datetime, np.datetim64 and pandas.DatetimeIndex) are converted to strings ( |
|
What is the definition of a NaT? |
|
NaT is also new to me. There are some documentation from Matlab and Python numpy |
|
NaT is basically like NaN, but for time. To be honest, I'm not sure why they don't just use NaN, but yeah, it exists...
Yeah, a warning might be better here. Not sure if there is an ISO datetime for NaT. |
|
Revisiting this thread a little as I'm trying to implement support for Apache Arrow's date32 and date64 dtypes in PyGMT at #2845, and would like some advice on the implementation. Currently at 5d16103, I've converted data stored in PyArrow's Also, what's the temporal resolution of |
|
Internally in GMT, absolute (and relative) time is stored in doubles. From wikipedia it say the smallest increment between two doubles is 2.22e-16 and since the default unit is seconds since some epoch (e.g. UNIX time 1970) it would seem that is pretty small. We never really used high-precision time when we wrote GMT so I would think 2.22e-16 seconds is pretty small. But not sure if that is always the smallest increment. Should cover nano seconds. |
|
It's a little more elaborated. The but since we are already 53 years away from 1970, the eps now is much larger If the time reference is the year 0 (not uncommon) with doubles one cannot have better resolution then ~10 micro sec And definitely doubles must be used. The panorama with singles (float32) is dramatic Ideally time should be stored in 64 bits ints. |
|
Ok, so if I understand correctly, the default To be safe and on the conservative side, it sounds like 1 microsecond is ok (and I just remembered @seisman mentioning this at #464 (comment))... |
|
Yes, but when reference is year zero |
Just read the |
Well, who would use year zero and look for high precision in recent years. You can decide you internal epoch if you really need nanoseconds around some epoch. Presumably one does not need nanoseconds from 0-2023? |
Description of the problem
I want to plot some datetime data on a map. However, gmt-python doesn't accept string as input.
Full code that generated the error
Full error message
System information
conda listbelow:output of conda list
The text was updated successfully, but these errors were encountered: