Skip to content
This repository has been archived by the owner on Jun 28, 2022. It is now read-only.

Invalid DateTime under Android #2

Closed
daimpi opened this issue Oct 4, 2020 · 4 comments
Closed

Invalid DateTime under Android #2

daimpi opened this issue Oct 4, 2020 · 4 comments

Comments

@daimpi
Copy link

daimpi commented Oct 4, 2020

Hi,

I just noticed two issues under Android when it comes to the format timestamp which seem to throw ENA off:

  1. ENA seems to expect English names for the months. e.g. 3. October 2020, 02:54 will work but not 3. Oktober 2020, 02:54 but the latter is what my EN log contains: me-03.txt. Therefore I will see "Invalid DateTime" for the matches in October.

  2. I recently came across someone who's timestamp had a totally different format: 2020.10.3 09:40 (ArgentinaRDX-03.txt). I'm not entirely sure what's going on there, but I'll try to investigate and will let you know as soon as I know more.

@felixlen
Copy link
Owner

felixlen commented Oct 4, 2020

Hi,

I am using the luxon library for date parsing and locale conversion. Android writes formatted timestamp in dependence of the Android locale language into the log, so the parsing needs to be adapted for every locale setting (it would be much better and less ambiguous if they would be using ISO timestamp strings as iOS does). I tried to cover both english and german, but unfortunately don't have the capacity to test and implement every possible locale.

I now have implemented showing the original log timestamp in case that date parsing leads to an invalid date. I hope that this yields an acceptable workaround, as supporting all possible ways the timestamp could be possibly formatted is beyond my capacity (if someone has a pointer for a library capable of automatic this, I am happy to get a pointer).

@daimpi
Copy link
Author

daimpi commented Oct 4, 2020

Hi,
as always thanks for the quick reply ❤️.

Android writes formatted timestamp in dependence of the Android locale language into the log, so the parsing needs to be adapted for every locale setting (it would be much better and less ambiguous if they would be using ISO timestamp strings as iOS does).

This is exactly the core of the problem and I fully agree that the blame here lies squarely with Google.

Regarding the format I mentioned in

2. I recently came across someone who's timestamp had a totally different format: 2020.10.3 09:40 (ArgentinaRDX-03.txt). […]

Turns out this was actually caused by a language setting (Chinese) + manual editing of the EN log before sharing.

I now have implemented showing the original log timestamp in case that date parsing leads to an invalid date.

I think that's a great idea 👍.
One thing I've noticed when I tried using ENA today with the logs provided in OP: There might currently be an implementation problem because when I uploaded those files, ENA was continuously showing the spinning loading wheel without ever finishing.

(if someone has a pointer for a library capable of automatic this, I am happy to get a pointer)

Unfortunately I can't provide any real pointer only an observation: I've noticed is that Microsoft Excel seems to have some way of parsing timestamps. When I select "Datatype -> Date/Time" for the timestamp column when importing EN logs, it's actually properly parsing them even with the OP logs. But as Excel is closed source, it's very unlikely that their parser would be openly available unfortunately .

@felixlen
Copy link
Owner

felixlen commented Oct 4, 2020

Hi,

I think that's a great idea 👍.
One thing I've noticed when I tried using ENA today with the logs provided in OP: There might currently be an implementation problem because when I uploaded those files, ENA was continuously showing the spinning loading wheel without ever finishing.

Thanks for the feedback. I haven't tested the workaround as I didn't have a proper example. Thanks for the pointer to the example. I was a bit too quick in implementing and checked the validity of the wrong object --- it is fixed by now and is working for me. Can you confirm this?

(if someone has a pointer for a library capable of automatic this, I am happy to get a pointer)

Unfortunately I can't provide any real pointer only an observation: I've noticed is that Microsoft Excel seems to have some way of parsing timestamps. When I select "Datatype -> Date/Time" for the timestamp column when importing EN logs, it's actually properly parsing them even with the OP logs. But as Excel is closed source, it's very unlikely that their parser would be openly available unfortunately .

Yes, I am sure that commercial products can handle this and probably there is also an open version (pandas in python can do it as well). It is no rocket science to implement, but a tedious task: One needs to loop over several possible ways a date could have been formatted and check if it matches to the kind of timestamp at hand. The tedious task is collecting all the different format descriptions, e.g. for german I have to use "d. LLLL yyyy, HH:mm and for english d LLLL yyyy, HH:mm.

@daimpi
Copy link
Author

daimpi commented Oct 4, 2020

it is fixed by now and is working for me. Can you confirm this?

Yes! works perfectly now, thanks 😊.

Yes, I am sure that commercial products can handle this and probably there is also an open version (pandas in python can do it as well). It is no rocket science to implement, but a tedious task […]

I see, makes sense. I think the approach you've chosen is a really good idea in this context 👍.

@daimpi daimpi closed this as completed Oct 4, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants