pm.sample cannot pickle LimbDark object

[lgrcia]

Error
Getting TypeError: cannot pickle 'exoplanet.theano_ops.driver.LimbDark' object when sampling a model containing LimbDarkLightCurve

Reproduction
Fresh virtualenv with exoplanet, pymc3, tqdm installed

python3 -m venv xotest
source ./xotest/bin/activate.fish
pip install tqdm exoplanet pymc3

and running docs/tutorials/transit.py

cd xotest
curl -O https://raw.githubusercontent.com/exoplanet-dev/exoplanet/main/docs/tutorials/transit.py
python ./transit.py

Setup:

• exoplanet==0.4.0
• macOS 10.14.6
• Python 3.8.5
• pip 20.1.1

[dfm]

You won't be able to run that as a script on macOS with Python 3.8 because of issues with how multiprocessing now works. You'll need to wrap your script in a if __name__ == "__main__" and then make sure that multiprocessing is using fork instead of spawn or forkserver.

Unfortunately there's not much to do from within exoplanet except add a page to the docs so I'm going to relabel this as a documentation issue rather than a bug.

Related issues:

• Parallel sampling hangs on macOS with Python 3.8 with pickling error pymc-devs/pymc#3844
• TypeError: cannot pickle 'starry._c_ops.Ops' object rodluger/starry#254
• Parallel sampling with Theano Op fails on MacOS 10.15 pymc-devs/pymc#4085
• Simple GP example causes RecursionError #111

[lgrcia]

I get this error as well on Linux when simply trying to pickle the model in docs/tutorials/transit.py. Like:

with open("test.pickle", "wb") as f:
    pickle.dump(model, f)

TypeError: cannot pickle 'exoplanet.theano_ops.driver.LimbDark' object

It works with exoplanet 0.3.2 but not 0.4.0 (At the end I was able to sample on macOS by downgrading the package)

[dfm]

OK - I'll take a look into this because it would be good to be able to pickle these models, but I don't understand why I haven't run into a sampling issue before if you're still getting it. I use parallel sampling on macos and ubuntu every day with v0.4 and the tutorial is automatically run on CI for every push.

[lgrcia]

This is puzzling indeed. I will try to investigate as well what could be wrong in my process.

For reference: Before running into the last point I tried using if __name__ == "__main__" with fork (trying both mp_ctx and multiprocessing.set_start_method as mentioned in pymc-devs/pymc#3844 (comment)) with Python 3.8.5 and Python 3.6.5 on macOS, leading to the same error in v0.4.0.

[dfm]

This will all have something to do with running in a Jupyter environment vs a script. These tutorials are designed to be run using jupytext, not directly as a script. #120 fixes the issue with pickling and #121 will track documenting best practices for running parallel sampling as a script (this is where the __name__ part comes in!).