Make C++ implementation C++11 only. #2780
Comments
|
Hello, Thank you for collecting feedback about this. My project consists of shared libraries on linux. Some libraries are built with c++11 and others must be built with c++98 (for compatibility with some other libraries). We link protobuf dynamically so this may not an issue for us, unless any c++11 language features will be used in header files that our c++98 libraries use. |
|
This would be an issue for our project. We support back to centos5 (gcc 4.1) and a number of platforms in between for which we are not yet able to turn on the c++11 flag. One of the attractive things about protobuf was its impressive backwards support, and would be sad to have older platforms be stuck with less. |
|
(PS. I feel bad saying this since I would love to move to c++11 only for our project too, haha! It is a sad world that we can't use "new" features from 6 years go yet. I'm sure libraries feel this pain more than applications do even. All of which to say, I feel your pain! 😭 ) |
|
Hi there, we made a related decision in gRPC - version 1.0 supports older compilers with only limited C++11 support (not as few as protobuf, though), but version 1.1 and beyond only support true C++11 compilers (as of grpc/grpc#8602 ). I understand, though, that protobuf's predicament is greater since it is a package that has been public for much longer, has more users, and has been supporting C++98. |
|
I strongly favor moving to a 4.x C++11 only branch. Users tight to pre-c++11 can continue using the 3.x branch which we support in maintenance mode. Note that linking pre-c++11 and post-c++11 libraries is already very tricky due to the standard libraries being slightly incompatible (see https://gcc.gnu.org/wiki/Cxx11AbiCompatibility). Hence I think the impact of not being able to link libs depending on pre-c++11 and post-c++11 proto runtime is minor, given you already have dangerous issues lurking around. Furthermore, for c++ protos, we already have the rule that all proto files in a binary have to be compiled with the exact same version of the compiler and runtime. |
|
We have C++14 enabled at work so this change would be welcome. Protobuf has great compatibility w.r.t. the serialized format, so people who are stuck with old environments can continue using the old libs. |
|
Hello all, Now I'm using gcc-5.4.0 to compile protobuf. However it couldn't be built because of C++11 new features. Solution: I used command "configure CXXFLAGS=-std=c++11" to config protobuf. |
|
+1 , backporting gcc5.4.x /7.x on old unmaintained distros is straightforward also with proper stripping unneeded language target at configure build flag. |
|
Requiring C++11 will make it basically impossible to support protobuf on some Python versions on Windows. In particular, Python 2.7 support will not be possible and Python 3.5 will need to be a minimum requirement. This is assuming that protobuf continues to need to have compiled portions for these Python versions. The reason being that CPython ties each major minor version of the interpreter to a particular Visual Studio version on Windows. As a result, any extension one builds needs to be built with the same runtime as the interpreter or risk segfaults. A full listing can be found in this reference. Note that Python 2.6 and 2.7 requires Visual Studio 2008 (VC 9), which does not have C++11 support. Also Python 3.4 requires Visual Studio 2010 (VC 10), which also does not have C++11 support. Only Python 3.5 and 3.6, which requires Visual Studio 2015 (VC 14), have C++11 support. cc @xfxyjwf |
|
With respect to @jakirkham's note, Python does not let people build their recent recent releases with older Visual Studio versions. IMO Protobuf should follow suit. There will be no way to move forward otherwise. Also note that Visual Studio 2010 has basic C++11 support, see the official reference. |
|
The googlei18n/libphonenumber project which uses protobuf currently doesn't support C++11. See google/libphonenumber#1594 for the discussion. |
|
Make it a |
2017-04-05 version 3.3.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Planned Future Changes
* There are some changes that are not included in this release but are
planned for the near future:
- Preserve unknown fields in proto3: please read this doc:
https://docs.google.com/document/d/1KMRX-G91Aa-Y2FkEaHeeviLRRNblgIahbsk4wA14gRk/view
for the timeline and follow up this github issue:
protocolbuffers/protobuf#272
for discussion.
- Make C++ implementation C++11 only: we plan to require C++11 to build
protobuf code starting from 3.4.0 or 3.5.0 release. Please join this
github issue:
protocolbuffers/protobuf#2780
to provide your feedback.
C++
* Fixed map fields serialization of DynamicMessage to correctly serialize
both key and value regardless of their presence.
* Parser now rejects field number 0 correctly.
* New API Message::SpaceUsedLong() that’s equivalent to
Message::SpaceUsed() but returns the value in size_t.
* JSON support
- New flag always_print_enums_as_ints in JsonPrintOptions.
- New flag preserve_proto_field_names in JsonPrintOptions. It will instruct
the JSON printer to use the original field name declared in the .proto
file instead of converting them to lowerCamelCase when printing JSON.
- JsonPrintOptions.always_print_primtive_fields now works for oneof message
fields.
- Fixed a bug that doesn’t allow different fields to set the same json_name
value.
- Fixed a performance bug that causes excessive memory copy when printing
large messages.
* Various performance optimizations.
Java
* Map field setters eagerly validate inputs and throw NullPointerExceptions
as appropriate.
* Added ByteBuffer overloads to the generated parsing methods and the Parser
interface.
* proto3 enum's getNumber() method now throws on UNRECOGNIZED values.
* Output of JsonFormat is now locale independent.
Python
* Added FindServiceByName() in the pure-Python DescriptorPool. This works only
for descriptors added with DescriptorPool.Add(). Generated descriptor_pool
does not support this yet.
* Added a descriptor_pool parameter for parsing Any in text_format.Parse().
* descriptor_pool.FindFileContainingSymbol() now is able to find nested
extensions.
* Extending empty [] to repeated field now sets parent message presence.
PHP
* Added file option php_class_prefix. The prefix will be prepended to all
generated classes defined in the file.
* When encoding, negative int32 values are sign-extended to int64.
* Repeated/Map field setter accepts a regular PHP array. Type checking is
done on the array elements.
* encode/decode are renamed to serializeToString/mergeFromString.
* Added mergeFrom, clear method on Message.
* Fixed a bug that oneof accessor didn’t return the field name that is
actually set.
* C extension now works with php7.
* This is the first GA release of PHP. We guarantee that old generated code
can always work with new runtime and new generated code.
Objective-C
* Fixed help for GPBTimestamp for dates before the epoch that contain
fractional seconds.
* Added GPBMessageDropUnknownFieldsRecursively() to remove unknowns from a
message and any sub messages.
* Addressed a threading race in extension registration/lookup.
* Increased the max message parsing depth to 100 to match the other languages.
* Removed some use of dispatch_once in favor of atomic compare/set since it
needs to be heap based.
* Fixes for new Xcode 8.3 warnings.
C#
* Fixed MapField.Values.CopyTo, which would throw an exception unnecessarily
if provided exactly the right size of array to copy to.
* Fixed enum JSON formatting when multiple names mapped to the same numeric
value.
* Added JSON formatting option to format enums as integers.
* Modified RepeatedField<T> to implement IReadOnlyList<T>.
* Introduced the start of custom option handling; it's not as pleasant as it
might be, but the information is at least present. We expect to extend code
generation to improve this in the future.
* Introduced ByteString.FromStream and ByteString.FromStreamAsync to
efficiently create a ByteString from a stream.
* Added whole-message deprecation, which decorates the class with [Obsolete].
Ruby
* Fixed Message#to_h for messages with map fields.
* Fixed memcpy() in binary gems to work for old glibc, without breaking the
build for non-glibc libc’s like musl.
Javascript
* Added compatibility tests for version 3.0.0.
* Added conformance tests.
* Fixed serialization of extensions: we need to emit a value even if it is
falsy (like the number 0).
* Use closurebuilder.py in favor of calcdeps.py for compiling JavaScript.
|
support,use c++ 11 later |
|
Hi, we use WindRiver compiler, and there is no support for C++11 there. Unfortunately it is not possible to switch to different compiler for us (company defined compiler...). If you switch, do you plan to support older releases? |
|
If we go with C++11 only, we will probably maintain a branch and only back port severe bug fixes. |
|
Why don't you just fast forward to C++14? |
|
There are a lot of projects that aren't C++11 compatible. |
|
I notice that move constructors for generated messages has landed in master. Fantastic! But there still aren't move constructors/assignments for the containers (I hope it's OK that I cross posted this from #2791; this seems to be the main report.) |
|
According to the plan, 3.5 will start to reserve unknown fields. I hope 3.5 will be C++98 compatible, so old compilers will get this important feature in their final release. |
|
@zzm3145 the next 3.5.0 release will not require C++11. We will likely start to require C++11 when we migrate to use Abseil: Google's C++ common libraries: |
|
I do not know how efficient are the calls guarded by As implemented, unless doing link-time code generation, it seems to me that |
|
@kkm000 With C++11 support removing GoogleOnceInit is indeed something we want to do. However GoogleOnceInit is implemented smart enough that after inlining the function it's a atomic read on a global variable and only a call in the initialization case. However using c++11 support will remove this dependency and allow the compiler to do whatever is best for the platform and is more readable. |
|
@gerben-s: Yup, my bad, I misread the inline piece. It only calls Yes, c++11 and its library removes a lot of low-level stuff, like atomics, mutexes, threads, futures, you name it. Great stuff! |
|
An update from protobuf team: it's decided starting from version 3.6.0 protobuf will require C++11 to compile. A 3.5.x branch will be maintained for pre-C++11 compilers but only bug fixes will be accepted in the branch. We are already accepting C++11 features into our internal code base and will accept pull requests using C++11 features on the github repo as well. |
|
FWIW had seen some C++11 features in 3.5.0 as well. So it would be good to tidy that up if it is intended to work on pre-C++11 compilers. See these comments for details. Haven't checked recently to see if they have been fixed. |
|
@jakirkham I checked your build log but it's using v3.4.1. Have you tried 3.5.0? The issue you pointed to doesn't exist in 3.5.0 as far as I can tell. |
|
Had backported some patches from 3.5.0 to 3.4.1, but I agree that is not the same as building 3.5.0. Have retried fresh with 3.5.0 and ran into issue ( #4064 ). |
|
Ran into an issue with 3.5.1 that we missed. Raised as issue ( #4094 ). |
2017-12-20 version 3.5.1 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Planned Future Changes
* Make C++ implementation C++11 only: we plan to require C++11 to build
protobuf code starting from 3.6.0 release. Please join this github issue:
protocolbuffers/protobuf#2780 to provide your feedback.
protoc
* Fixed a bug introduced in 3.5.0 and protoc in Windows now accepts non-ascii
characters in paths again.
C++
* Removed several usages of C++11 features in the code base.
* Fixed some compiler warnings.
PHP
* Fixed memory leak in C-extension implementation.
* Added discardUnknokwnFields API.
* Removed duplicatd typedef in C-extension headers.
* Avoided calling private php methods (timelib_update_ts).
* Fixed Any.php to use fully-qualified name for DescriptorPool.
Ruby
* Added Google_Protobuf_discard_unknown for discarding unknown fields in
messages.
C#
* Unknown fields are now preserved by default.
* Floating point values are now bitwise compared, affecting message equality
check and Contains() API in map and repeated fields.
|
@jayantkolhe @pherl Is it possible to build portable C++11 Linux release binaries on any system that isn't RHEL6? Here's some knowledge from my notes file:
I'd like to know if it's possible to use something like musl-cross-make to statically link musl-libc (MIT) and LLVM libc++ (MIT). In that case, we'd be able to write C++14 code, compile it with GCC6+ (which can do things like |
|
At conda-forge, we are migrating to some new compilers produced by Anaconda (they already use them) that are based on GCC 7.2 that @mingwandroid worked on. We would still use them to build on CentOS 6, but as glibc is generally backwards compatible, the binaries built should still work on newer Linux systems. Maybe this is interesting to you? |
|
Are you able to compile a gcc7 toolchain that runs on rhel6 and produces
dynamically linked binaries that are ABI compatible with libstdc++4.4?
…On Fri, Apr 27, 2018, 10:57 AM jakirkham ***@***.***> wrote:
At conda-forge, we are migrating to some new compilers produced by
Anaconda (they already use them) that are based on GCC 7.2 that
@mingwandroid <https://github.com/mingwandroid> worked on. We would still
use them to build on CentOS 6, but as glibc is generally backwards
compatible, the binaries built should still work on newer Linux systems.
Maybe this is interesting to you?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#2780 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AADAbhojNdzz0izQtbXb5Ab3b8diFOYkks5ts1wFgaJpZM4MQFgz>
.
|
|
No, we provide our own libstdc++. You could link to it statically I guess (we don't though). |
|
The toolchain is a CentOS6 glibc 2.12 targeting (and hosted) pseudo-cross compiler. It is used to build all of the |
|
That's one solution but protobuf can't assume an Anaconda subsystem is available when distributing binaries on PyPi. PEP 513 says Take for example gcc-7.2.0-i486-linux-musl.tar.xz which is 31mB; runs on RHEL4+ i486+; can turn helloworld.c into a 5kb static binary that runs on any Linux and can be optimized for any microarchitecture, e.g. Skylake. It also supports C++17. The catch is GPLv3 if you want portable binaries. |
|
Yup, and thanks for paying attention to the I like that musl toolchain; very cool. |
|
I heard from @gunan that |
|
Any more details you can provide about this |
|
@yifeif to comment on manylinux issues
…On Sat, Apr 28, 2018 at 1:22 AM Ray Donnelly ***@***.***> wrote:
Any more details you can provide about this manylinux2010 issue would be
welcome.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2780 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AHlCORxBMY7qEdzhj9BrR-JteWgiHmudks5ttCa8gaJpZM4MQFgz>
.
|
|
I think manylinux2010 should be okay? But it might not be fully rolled out yet pypa/manylinux#179 |
|
I'm going to close this issue because this is now pretty much done and we're using C++11 now. Feel free to comment if you have any C++11-related issues, though. |
and others
C++11 has been out for several years and we are thinking of moving to allow C++11 features from being used in the code base. That means the implementation will not be able to compile if you don't have a C++11 compiler.
We may create a branch that works for C++98. The branch will only accept bug fixes, but not new features, optimization, etc
Please reply in this thread if you think this would be an issue for your project.
The text was updated successfully, but these errors were encountered: