Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replace regex with indexOf #8

Merged
merged 5 commits into from
May 22, 2023
Merged

Conversation

karussell
Copy link
Contributor

@karussell karussell commented May 22, 2023

For #7. Not sure how to do performance tests in kotlin. Is there a native alternative for JMH?

Update: although this improves the import for my smaller test PBF it does not seem to improve the memory problems for planet.
Update: was a build problem

Copy link
Owner

@westnordost westnordost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obligatory question: Did you test it? I.e. run the unit tests?


As mentioned earlier, the library does a lot of copying of lists, maps and sets where oftentimes it doesn't really have to, it was just simpler to write the code that way. E.g. keys.filter { it.startsWith("$key:") } creates a new set.

Now, I have recently read that copying data is one of those things modern CPUs are really good at, so oftentimes it doesn't really matter. However, maybe Java is different (because all collections are on the heap, not on the stack).
So, to traverse all elements on a collection lazily, you can use Sequences. Sequences are pretty much the same thing as Java Streams.

If using these at some places actually has an impact, I'd be curious to know. Might as well have a negative impact due to the added indirection, don't know 🤷 .

@karussell
Copy link
Contributor Author

karussell commented May 22, 2023

Obligatory question: Did you test it? I.e. run the unit tests?

Yes :)

Now, I have recently read that copying data is one of those things modern CPUs are really good at

Yeah, normally it is no problem and often the JVM can handle smaller frequently created garbage. But the GraphHopper import is delicate and it might not be the same (as a method is called millions of times). Will profile it again after this change. But the build makes me headaches as sometimes it worked to push it to the local maven repo and sometimes not.

@westnordost
Copy link
Owner

westnordost commented May 22, 2023

But the build makes me headaches as sometimes it worked to push it to the local maven repo and sometimes not.

Hm. Maybe the multiplatform setup is delicate... Is there an update for the multiplatform maven plugin?

@westnordost
Copy link
Owner

Also, I am not sure if publishing to local maven works if a local version with the same version number already exists. Maybe you need to delete the one you already published just then.

karussell and others added 3 commits May 22, 2023 22:50
…_speeds/LegalDefaultSpeeds.kt

Co-authored-by: Tobias Zwick <[email protected]>
…_speeds/LegalDefaultSpeeds.kt

Co-authored-by: Tobias Zwick <[email protected]>
…_speeds/LegalDefaultSpeeds.kt

Co-authored-by: Tobias Zwick <[email protected]>
@westnordost
Copy link
Owner

So from my point of view, this could be merged now. Or do you want to also try out asSequence to not create unnecessary new lists/sets/maps in this same PR first?

@karussell karussell mentioned this pull request May 22, 2023
@karussell
Copy link
Contributor Author

Sure, feel free to merge it :) (I can't :) )

If you like or want it might be beneficial to setup the benchmark mentioned in #7 before merging, but this is up to you.

Or do you want to also try out asSequence to not create unnecessary new lists/sets/maps in this same PR first?

No, I think it is fine for me to approach this in smaller steps.

Obligatory question: Did you test it? I.e. run the unit tests?

Maybe you activate github actions to run the tests automatically?

@westnordost
Copy link
Owner

Maybe you activate github actions to run the tests automatically?

It would be prudent to do so... but I am too lazy to do this now. I'd have to read up a bit about github actions first. A PR that enables that however would be welcome :-)

@westnordost westnordost merged commit 7329b62 into westnordost:master May 22, 2023
@karussell
Copy link
Contributor Author

btw: feel free to copy stuff from us https://github.com/graphhopper/graphhopper/blob/master/.github/workflows/build.yml (the node cache is probably not necessary)

@karussell karussell deleted the no_regex branch September 2, 2024 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants