[Core] update protobuf to 3.19.4

[scv119]

Why are these changes needed?

The error message in #25638 indicates we should use protobuf>3.19.0 to generated code so that we can work with python protobuf >= 4.21.1. Try generating wheels to see if this works.

Related issue number

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • Manually test
  • This PR is not tested :(

[scv119]

Manually tested with this we can work with both protobuf == 4.21.1 and protobuf == 3.20.1

[matthewdeng]

To simplify this even more, this should fail with just import ray right?

[pcmoritz]

Thanks a ton for fixing this, this is great! Do we need the upper bound on 4.21.1 in the requirements.txt? It would be great to not constrain it so the CI will detect if there is other versions in the future that cause problems :)

[rkooo567]

LGMT if tests pass!

[fishbone]

test_protobuf_compatibility failed

[scv119]

well, we are hitting a windows bug: protocolbuffers/protobuf#10075

[scv119]

@pcmoritz @rkooo567 so basically this works for linux and mac... except for Windows.
Turns out there is an open issue protocolbuffers/protobuf#10075 that python protobuf-4.21.1 package doesn't work with C++ protobuf-3.19.4 generated python file, on Windows.

What's our option here?

1. we upgrade C++ protobuf to 3.19.4, change the pip requirement to protobuf >= 3.12.0, < 4.0.0. In this way:

• by default, all user should install a protobuf version that's compatible (between 3.12.0 to 4).
• linux/mac user with protobuf between 3.12.0 to 4.21.1 (latest) will work.
• windows user, if their installation didn't respect our requirement, with protobuf 4.21.1 doesn't work.

2. we upgrade C++ protobuf to 3.19.4, change the pip requirement to **protobuf >= 3.12.0 **. In this way:

• linux/mac user with protobuf between 3.12.0 to 4.21.1 (latest) will work.
• windows user with latest protobuf 4.21.1, ray doesn't work.

3. do nothing?

[pcmoritz]

There is also 4., which is upgrade C++ protobuf to 3.19.4 only on non-Windows (and also unpin), and keep the old one on Windows until protocolbuffers/protobuf#10075 is fixed :)

That one actually seems optimal given the situation, what do you think? We could even keep pinning on Windows so stuff works there too unless the pin is ignored.