Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid storing timestamp and absolute paths in cache #87

Closed
wants to merge 1 commit into from

Conversation

gsmet
Copy link

@gsmet gsmet commented Jan 25, 2024

Hi,

This is using an approach similar to what was done for the formatter plugin (reusing the exact same code for writing the properties to the cache).

To give a bit a bit more context: we are using this plugin to manage imports in the Quarkus project and we are leveraging the Develocity build cache and I would like to store the cache file in the Develocity build cache which is shared among hosts.
We are also using the formatter plugin and in the formatter plugin, things were already handled this way so I mimicked exactly the behavior by adjusting the cache key to remove the base dir and removing the timestamp from the properties file.

Formatter code is here if you want to check: https://github.com/revelc/formatter-maven-plugin/blob/main/src/main/java/net/revelc/code/formatter/FormatterMojo.java#L606

Let me know if you see anything incorrect.

/cc @jprinet

Thanks!

This is using an approach similar to what was done for the formatter
plugin (reusing the exact same code for writing the properties to the
cache).
Copy link
Member

@ctubbsii ctubbsii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change really seems like a one-time upgrade. It doesn't seem worth it to read-sort-strip-write the file every time. Could probably add a version number to the file, either in the filename or as the first line, to determine if it needs to be rewritten with a new format.

@ctubbsii ctubbsii added this to the 1.10.0 milestone Jan 25, 2024
@gsmet
Copy link
Author

gsmet commented Jan 26, 2024

@ctubbsii not really. The timestamp is added every time by Properties#store().

Now, we could write the file ourselves instead of relying on Properties#store(). That would be more optimized.
I decided against it because I wanted both plugins to be consistent and preferred reusing the code in the formatter plugin.

@ctubbsii
Copy link
Member

ctubbsii commented Feb 8, 2024

@gsmet I'm not too familiar with the caching implementation. It was added by others to my plugin, and I've allowed that, even though I don't use it. I'm okay with these changes, but I think if they were modeled after formatter-maven-plugin's implementation, there may be a problem. @hazendaz had identified some problem with the caching behavior, that I'm not too familiar with. It may be necessary to fix that behavior in the formatter-maven-plugin before the behavior is mimicked here.

@hazendaz
Copy link
Member

@ctubbsii The issue on formatter plugin you mention has to do with configuration that was added to it in recent versions which broke its usage causing the cache to be rewritten constantly as the key changes every time now (revelc/formatter-maven-plugin@d1d5852). So we use an older version still until that isn't broken that way.

The point here is to make sure the cache is not being written constantly as well as fixing paths. I have the same notes on this plugin needing that fixed so we never used the cache here either as its useless if it keeps getting updated (as it pertains to storage). However this plugin was always pretty fast so it was not a huge concern. There are other plugins that are substantially worse with no cache (ie license-maven-plugin for example).

The broader point here is storing the cache to ensure builds don't do wasteful activities. If you recall, originally I had set the default .cache to the root of the project on formatter-maven-plugin. I'll call that the poor mans solution as we didn't have develocity (gradle enterprise) at that time and maven didn't yet have a cache (still seemingly work in progress).

This improvement would allow us to do the same here. But again this plugin is super fast in general.

@gsmet
Copy link
Author

gsmet commented Feb 10, 2024

But again this plugin is super fast in general.

The in general is important :). For a project the size of Quarkus, when building without tests, it starts to show :).

@gsmet
Copy link
Author

gsmet commented Feb 10, 2024

I might be biased but really this PR is about making the cache file only containing information that is portable and reproducible. It only makes it better IMO and I don't think we are introducing any new issues.

Copy link
Member

@ctubbsii ctubbsii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are problems with this implementation.
First, it writes a file using Properties.store, which appends to any existing file. That could be a problem. I think that problem exists in the current implementation as well.

It then reads the entire contents of the file into memory, which it already had in memory before. I don't think it needs to do that.

It then sorts before it removes the timestamp. But, when it removes the timestamp, it makes a big assumption that there's only one comment line and it's at the top after sorting. It should have removed the comment line before sorting.

I have proposed changes that are much smaller, more efficient, and I think probably better, in #90. However, I would like others to review it.

@gsmet
Copy link
Author

gsmet commented Mar 5, 2024

Couldn’t agree more that the impl is not ideal.

As I said, I copied the code of the other plugin in the hopes it would avoid too many discussions but we can close this if you prepared something better.

@gsmet gsmet closed this Mar 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants