Improve performance when loading load order

[Ortham]

Plugin validity checks are pretty much the only expensive operation in libloadorder, as they involve reading the whole plugin file, which can be quite large and so reading can be slow. The performance of actual plugin reading is up to libespm, but there are a couple of things libloadorder can do to improve its performance:

• At the moment, validity is checked then the plugin is loaded, which involves two file reads (though the second only reads the file header). Given that libloadorder only deals with file headers, this could be combined into a single step, where a Plugin object is constructed, and if an exception is thrown then it is handled as an invalid plugin, and if not that constructed object is stored. Profiling shows this doesn't bring any performance benefit, and it has less clear semantics, so this idea is dropped.
• When the load order is reloaded because filesystem changes have been detected, all data is discarded and re-loaded. Depending on the changes though, it could be that most or all of the existing plugin objects can be reused. Plugin modification times would need to be compared with their cached times to see if the objects need re-loading or not.

[Utumno]

where a Plugin object is constructed, and if an exception is thrown then it is handled as an invalid plugin, and if not that constructed object is stored

Of course that constructed object should be a few KBs big if not less

Go for a static cache in the plugin class. I have commented on this somewhere - key should be modtime, size, name - value a slim Plugin instance - IIRC

[Ortham]

This issue has been greatly mitigated by 4cc5a5f, which loads only plugin headers when validating. As such, there's no pressing need to make the changes suggested here, but the suggestions remain valid.

[Ortham]

OK, actually got around to doing some profiling, and the results show that loading data from scratch typically takes 50 ms with 268 plugins installed and 255 of them active. With 13 plugins and none active, it takes < 1 ms, so there's a clear correlation to number of plugins.

This is on a VirtualBox virtual machine running on top of a Core i5-4300U with an SSD, so it's not a particularly fast environment anyway.

An initial implementation of re-using data where possible according to point 2 in the OP (not including inside the loading from file and loading active plugins functions) speeds up subsequent loads so that 1000 loads with no changes between them take only 4 times as long as a single load. As such, I think the changes are worth making.

[Ortham]

More profiling, and combining the isValid() checks with the Plugin constructors doesn't lead to any performance improvement, so I'm striking that from the OP.