Sort the cache and remove timestamp
Closed this issue · 7 comments
There are two considerations here.
First, cache uses properties file from java to build. This in turn uses hashtable. As a result the output is not deterministic. A new property is needed to sort the resulting file. This can be default off.
Situation where needed: I'm turning on check-in of the cache to an automated pipeline that auto formats and commits. With non-deterministic output, its clear that this doesn't always sort in same order resulting in unnecessary rewriting of the file order. For this type of automated situation, it just fights itself with changes. A quick solution seems to be this https://stackoverflow.com/questions/10275862/how-to-sort-properties-in-java. Most often this doesn't seem to be a problem and ironically seems more of a problem on windows. It likely changes depending on larger file sets.
Second, again because of using properties file, java writes a timestamp in there which we don't need. We cannot stop it from doing that but its super easy to write after and remove it. This is in line with same situation above. It decides to write the file automatically and we end up with potentially nothing but a timestamp (well we would if we sort).
Now this may not actually end up with doing all of above but rather looking further at this rather quickly. It should not be writing the file to start with if it had nothing to do but I'm seeing it switch positions so it feels more like we have a bug and maybe order doesn't matter. Regardless of the situation here need to look tonight and wanted to document what I'm seeing thus self assigned as I'll take care of this.
I found the issue, its a bit complicated. The first I was thinking of doing, not one above, much shorter I have applied to fix the issue. I haven't sent yet but will do so tomorrow. Its isolated around maven plugins. When the help mojo is created, that happens early on. Then that is formatted which flags the cache to be written but later it moves the file elsewhere. That seems to trigger us to write the cache when not expected to do so. If we run just the formatter, it doesn't write cache and every time its says its one file causing this. There were also protections to make sure this did not happen but this is such an odd case. The MojoHelp gets moved into the project space later on with maven plugin plugin. I'm not quite sure what to think about that really. But having this protect on the write to make sure it does not contain the timestamp making it reproducable is a good thing and further have it sorted helps if we detect other issues.
This could be a maven bug with maven plugin plugin 3.6.2+ but getting some other odditing after my fix where I have a half hash not with a file. Everything else looks fine so I'll have to look at that more tomorrow to see what caused that. I should have a fix sometime tomorrow night then I'll try getting with jsoup again to get that piece ready. If we hang too much more than few weeks we will be looking at back to back releases to cover 2021-12 and 2022-03.
This is definitely a maven situation that affects us. It creates HelpMojo then moves it with the product group id. The first creation causes it to get formatted, then formatting occurrs, then it is different which causes the protections in place to not work and it writes the file. Based on the timings I've done thus far with quick read file back in, sort it, delete the timestamp, and rewrite it back, I don't feel at the moment it needs further flags to do this. Basically if caching is enabled this will be by default unless we get some metrics to show any delay in processing that is too great.
Confirmed completed and verified in 2.18.0
@ctubbsii This was part of 2.18.0, not sure how to deal with milestone here since the prior is closed now. Would it make sense to just remove it? Or do we reopen and apply it back to 2.18.0?
@ctubbsii figured it out, have opened 2.18.0 back up and going through what was addressed. Will close when done.
FWIW, the milestone doesn't need to be open to assign an issue to it. There is a "Closed" tab with the closed milestones. Just click the gear next to milestones, then click the Closed tab, and select the appropriate milestone.