[BUG] pickleable base C++ classes can incur object slicing if derived typeid is not registered with pybind11

[EricCousineau-TRI]

Issue description

Context: #2972 (comment)

If you have C++ base class that is exposed to pybind11, but then you have a derived class that is not exposed to pybind11, then you will encounter object slicing (wikipedia) via a pickle -> unpickle round trip.
Slicing may mean that the unpickled object has the wrong vtable, data, etc.

Reproducible example code

master...EricCousineau-TRI:issue-3062 (derived from fixes in #2972)
commit: 7ce1ad0

Upon running the test:

$ make pytest
...
    def test_roundtrip_simple_cpp_derived():
        p = m.make_SimpleCppDerivedAsBase()
        p.num = 404
        assert m.get_value_via_vtable_cpp(p) == 10
...
        data = pickle.dumps(p, pickle.HIGHEST_PROTOCOL)
        p2 = pickle.loads(data)
        assert isinstance(p2, m.SimpleBase)
        assert p2.num == 404
>       assert m.get_value_via_vtable_cpp(p2) == 10
E       assert 1 == 10  # Egad! Error!

\cc @rwgk @YannickJadoul @elkhrt

[EricCousineau-TRI]

Suggested (but invasive) solution:

Fail fast if we attempt to pickle an object whose typeid is not registered.
EDIT: Ah, but ensure that we add an option for users to allow it if they have some virtual methods of their own to add in pickle-based serialization+deserialization.

Less invasive solution:

Document this sharp edge, and tell people to be careful / provide an example of how to do this properly.

[rwgk]

Oh, I saw your get_value_via_vtable_cpp addition here only after I added the dynamic_cast test under #2972, but it looks like the same basic idea.

In a way the __dict__ issue @elkhrt ran into is a warning flag that we ignored. Edward, what's your take on the "implicit upcasting" going through a pickling/unpickling cycle?

How could we detect the situation when pickling? (I'm currently drawing a blank.)

[elkhrt]

Agree this is an issue. It doesn't affect the open spiel use case because we effectively have our own type registration - the unpickling code will call a factory method, which might be native C++, or a Python callable, which ensures we get the right derived type. (This wasn't deliberate, but just a consequence of the general design).