[FEATURE] Option to pickle the whole strategy

[alexander-soare]

Right now strategy.save produces a JSON with the log. Any reason you didn't opt for (or have an option of) pickling the whole strategy? Two motivations for this:

1. Not having to re-init the strategy with all the args/kwargs
2. Not having to loop through tell! SMBO can take quite some time to do this.

[RobertTLange]

Hi Alex, I chose the .json storage for easier readability, being more lightweight and convenience. But you are right .pkl storage would be a nice option. What do you think about adding a as_pkl boolean option to the save method? The load method would then have to be adapted to replace self in the case of the log file being a pickle.

[alexander-soare]

@RobertTLange I think your proposal sounds good. I guess you could also consider whether people would want both? So there's an as_json as well which defaults to True?

Regarding your second question I was curious and had a look around (for example: https://stackoverflow.com/questions/21998007/python-avoid-defining-both-a-classmethod-and-an-instancemethod) and it seems the "right" way to do that would simply be to have a module level function called load_strategy. Like you, I was hoping there was an accepted way to overload the method.

If you go with that, perhaps the load method would need a more descriptive name? Like load_history or load_log? Dunno... don't want to start going down a rabbit hole here... :P

[alexander-soare]

Another thing to consider is that maybe people can just pickle it themselves... lol. Unless you're doing other checks (on top of simple pickling unpickling the standard way), maybe it's too trivial to implement.

[RobertTLange]

Sorry for the long wait. I just addressed your request in PR #5. You can now save the state of your strategy as a pickle file as follows:

strategy = RandomSearch(...)
# Explicitly call `save` method
strategy.save("<pkl_filename>.pkl")
# ... or save the strategy with a `tell` call
strategy.tell(configs, scores, save=True, save_fname="<pkl_filename>.pkl")

You can then reload the log using strategy.load("<pkl_filename>.pkl") or providing the path at instantiation. You can now either store the log as easily readable .json/.yaml or pickle depending on the file extension provided by the desired path! When loading the pickle file, we replace the __dict__ which stores all writable attributes:

mle-hyperopt/mle_hyperopt/strategy.py

Line 266 in d72a400

self.__dict__ = load_strategy(reload_path).__dict__

It will be part of the next release (hopefully coming soon -- I am pretty excited about the new HalvingSearch, HyperbandSearch and PBTSearch implementations). I hope that this addresses your request. Let me know if you have any other recommendations/issues. I will close this for now. Feel free to reopen the issues if something comes up.