-
Notifications
You must be signed in to change notification settings - Fork 15.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
python: SIGSEGV when use PyImport_Import import symbol_database #5979
Comments
Would you please provide your code for importing python protobuf? |
Sorry for the delay... After minimize the crash scenario,I find after I link the protobuf library with a binary whose logic is importing a Python layer's generated code, causes the binary crash when execute. (For some reason, the full binary use protobuf library to decode buffer in C++ layer, so I link with the protobuf library). Here is a simple POC code: #include <Python.h>
#include <stdio.h>
static PyObject* py_module = NULL;
int main(int argc, char *argv[]){
Py_Initialize();
char* module_name = getenv("MUTATOR_MODULE");
printf("load python module %s\n",module_name);
if (module_name) {
PyObject* py_name = PyUnicode_FromString(module_name);
py_module = PyImport_Import(py_name);
Py_DECREF(py_name);
}
} compile command: clang++ -std=c++11 -g -O2 -fno-omit-frame-pointer test_crash.cpp `python3-config --cflags --ldflags` `pkg-config --cflags --libs protobuf` - o test_crash trigger command:
And I get some symbolized information from the backtrace: #0 0x00007ffff6a605e6 in google::protobuf::Arena::OnArenaAllocation (this=<optimized out>,
allocated_type=0x7ffff6e13c58 <typeinfo for google::protobuf::FileDescriptorProto>, n=0xd8) at google/protobuf/arena.cc:409
#1 0x00007ffff6ad3119 in google::protobuf::Arena::AllocHook (n=<optimized out>,
allocated_type=0x7ffff6e13c58 <typeinfo for google::protobuf::FileDescriptorProto>, this=0x7ffff047e131) at ./google/protobuf/arena.h:515
#2 google::protobuf::Arena::AllocateInternal<google::protobuf::FileDescriptorProto> (skip_explicit_ownership=0x1, this=0x7ffff047e131)
at ./google/protobuf/arena.h:525
#3 google::protobuf::Arena::DoCreateMessage<google::protobuf::FileDescriptorProto> (this=0x7ffff047e131) at ./google/protobuf/arena.h:600
#4 google::protobuf::Arena::CreateMessageInternal<google::protobuf::FileDescriptorProto> (arena=0x7ffff047e131) at ./google/protobuf/arena.h:496
#5 google::protobuf::Arena::CreateMaybeMessage<google::protobuf::FileDescriptorProto> (arena=0x7ffff047e131)
at google/protobuf/descriptor.pb.cc:16712
#6 0x00007ffff03f1038 in google::protobuf::MessageLite::ParseFromArray(void const*, int) () from /usr/lib/x86_64-linux-gnu/libprotobuf.so.9
#7 0x00007ffff043a1b6 in google::protobuf::EncodedDescriptorDatabase::Add(void const*, int) () from /usr/lib/x86_64-linux-gnu/libprotobuf.so.9
#8 0x00007ffff03fb9b8 in google::protobuf::DescriptorPool::InternalAddGeneratedFile(void const*, int) ()
from /usr/lib/x86_64-linux-gnu/libprotobuf.so.9
#9 0x00007ffff042a48c in google::protobuf::protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto() ()
from /usr/lib/x86_64-linux-gnu/libprotobuf.so.9
#10 0x00007ffff7de76ba in call_init (l=<optimized out>, argc=argc@entry=0x1, argv=argv@entry=0x7fffffffe068, env=env@entry=0x7fffffffe078)
at dl-init.c:72
#11 0x00007ffff7de77cb in call_init (env=0x7fffffffe078, argv=0x7fffffffe068, argc=0x1, l=<optimized out>) at dl-init.c:30
#12 _dl_init (main_map=main_map@entry=0x61a000062480, argc=0x1, argv=0x7fffffffe068, env=0x7fffffffe078) at dl-init.c:120
#13 0x00007ffff7dec8e2 in dl_open_worker (a=a@entry=0x7fffffff9360) at dl-open.c:575
#14 0x00007ffff7de7564 in _dl_catch_error (objname=objname@entry=0x7fffffff9350, errstring=errstring@entry=0x7fffffff9358,
mallocedp=mallocedp@entry=0x7fffffff934f, operate=operate@entry=0x7ffff7dec4d0 <dl_open_worker>, args=args@entry=0x7fffffff9360) at dl-error.c:187
#15 0x00007ffff7debda9 in _dl_open (
file=0x7ffff0bc5ec0 "/usr/local/lib/python3.5/dist-packages/protobuf-3.7.1-py3.5-linux-x86_64.egg/google/protobuf/pyext/_message.cpython-35m-x86_64-linux-gnu.so", mode=0x80000002, caller_dlopen=0x4a3770 <__interceptor_dlopen(char const*, int)+112>, nsid=0xfffffffffffffffe, argc=<optimized out>,
argv=<optimized out>, env=0x7fffffffe078) at dl-open.c:660
#16 0x00007ffff732ff09 in dlopen_doit (a=a@entry=0x7fffffff9590) at dlopen.c:66
#17 0x00007ffff7de7564 in _dl_catch_error (objname=0x781430 <alloc_memory_for_dlsym+16>, errstring=0x781438 <alloc_memory_for_dlsym+24>,
mallocedp=0x781428 <alloc_memory_for_dlsym+8>, operate=0x7ffff732feb0 <dlopen_doit>, args=0x7fffffff9590) at dl-error.c:187
#18 0x00007ffff7330571 in _dlerror_run (operate=operate@entry=0x7ffff732feb0 <dlopen_doit>, args=args@entry=0x7fffffff9590) at dlerror.c:163
#19 0x00007ffff732ffa1 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#20 0x00000000004a3770 in __interceptor_dlopen (
filename=0x7ffff0bc5ec0 "/usr/local/lib/python3.5/dist-packages/protobuf-3.7.1-py3.5-linux-x86_64.egg/google/protobuf/pyext/_message.cpython-35m-x86_64-linux-gnu.so", flag=0x2)
at /media/arnow117/Data/build-clang/llvm/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:5949 |
@anandolee Any udpate on this ticket? I also did similar as @arnow117 using protobuf 3.6.1 : write a c++ program which calls PyImport_Import() to import a generated protobuf python code. I also got a SIGSEGV when I call PyImport_Import (backtrace also leads me to _message.cpython-35m-x86_64-linux-gnu.so). But when I import the generated protobuf python code from python interpreter, it doesn't crash. |
I also get this problem when loading tensorflow via pybind if I have the same version of protobuf also in my c++ code |
I also have this exact problem with python 3.6 as well. Upgrading to 3.8 seems to fix it. As is the case with cjolivier01 it only seems to happen if I have the same proto native in the C++ program I'm trying to load into. |
We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment. This issue is labeled |
We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please reopen it. This issue was closed and archived because there has been no new activity in the 14 days since the |
What version of protobuf and what language are you using?
Version: 3.7.0(native library & python module)
Language: Python
What operating system (Linux, Windows, ...) and version?
ubuntu
What runtime / compiler are you using (e.g., python version or gcc version)
python 3.5.2
What did you do?
I write a c program which call PyImport_Import() to import a generated protobuf python language code. However I got a SIGSEGV when I call PyImport_Import and it will import symbol_database. While after simple debug I think it maybe a bug in _message.cpython-35m-x86_64-linux-gnu.so.
context in gdb-peda below:
[----------------------------------registers-----------------------------------]
RAX: 0x6160000bf480 --> 0x0
RBX: 0x3
RCX: 0x0
RDX: 0x0
RSI: 0x0
RDI: 0x6160000bf4c8 --> 0x0
RBP: 0x60400002fc10 --> 0x1
RSP: 0x7fffffffb900 --> 0x647fffffff --> 0x0
RIP: 0x7ffff05ae6b3 (<_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE5clearEv+51>: mov rdx,QWORD PTR [rbx+0x8])
R8 : 0x60400002fc10 --> 0x1
R9 : 0x0
R10: 0x0
R11: 0x6030000c2d10 --> 0x6030000c2d20 ("optional_string")
R12: 0x7ffff6972140 --> 0x0
R13: 0x6160000bf4c8 --> 0x0
R14: 0x7fffffffb90f --> 0x608000039a2000
R15: 0x7ffff0485690 --> 0x4
EFLAGS: 0x10206 (carry PARITY adjust zero sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
0x7ffff05ae6a4 <_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE5clearEv+36>:
jmp 0x7ffff05ae6b3 <_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE5clearEv+51>
0x7ffff05ae6a6 <_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE5clearEv+38>: nop WORD PTR cs:[rax+rax*1+0x0]
0x7ffff05ae6b0 <_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE5clearEv+48>: mov rbx,rbp
=> 0x7ffff05ae6b3 <_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE5clearEv+51>: mov rdx,QWORD PTR [rbx+0x8]
0x7ffff05ae6b7 <_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE5clearEv+55>: mov rbp,QWORD PTR [rbx]
0x7ffff05ae6ba <_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE5clearEv+58>: lea rdi,[rdx-0x18]
0x7ffff05ae6be <_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE5clearEv+62>: cmp rdi,r12
0x7ffff05ae6c1 <_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE5clearEv+65>:
jne 0x7ffff05ae704 <_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE5clearEv+132>
[------------------------------------stack-------------------------------------]
0000| 0x7fffffffb900 --> 0x647fffffff --> 0x0
0008| 0x7fffffffb908 --> 0x7ffff7a7c8b1 (<google::protobuf::DescriptorProto::IsInitialized() const+33>: test al,al)
0016| 0x7fffffffb910 --> 0x608000039a20 --> 0x60400002fc10 --> 0x1
0024| 0x7fffffffb918 --> 0x60400002fc10 --> 0x1
0032| 0x7fffffffb920 --> 0x6030000c2c20 --> 0x6030000c2c30 ("afs.proto")
0040| 0x7fffffffb928 --> 0x7ffff04735e8 --> 0x1
0048| 0x7fffffffb930 --> 0x7ffff0485690 --> 0x4
0056| 0x7fffffffb938 --> 0x7ffff059dd48 (<_ZNK6google8protobuf14DescriptorPool14FindFileByNameERKSs+72>: mov rax,QWORD PTR [rbx+0x20])
[------------------------------------------------------------------------------]
Legend: code, data, rodata, value
Stopped reason: SIGSEGV
0x00007ffff05ae6b3 in std::_Hashtable<std::string, std::string, std::allocatorstd::string, std::__detail::_Identity, std::equal_tostd::string, std::hashstd::string, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, true, true> >::clear()
() from /home/arnow117/.local/lib/python3.5/site-packages/google/protobuf/pyext/_message.cpython-35m-x86_64-linux-gnu.so
gdb-peda$ bt
#0 0x00007ffff05ae6b3 in std::_Hashtable<std::string, std::string, std::allocatorstd::string, std::__detail::_Identity, std::equal_tostd::string, std::hashstd::string, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, true, true> >::clear() () from /home/arnow117/.local/lib/python3.5/site-packages/google/protobuf/pyext/_message.cpython-35m-x86_64-linux-gnu.so
#1 0x00007ffff059dd48 in google::protobuf::DescriptorPool::FindFileByName(std::string const&) const ()
from /home/arnow117/.local/lib/python3.5/site-packages/google/protobuf/pyext/_message.cpython-35m-x86_64-linux-gnu.so
#2 0x00007ffff05660e8 in ?? () from /home/arnow117/.local/lib/python3.5/site-packages/google/protobuf/pyext/_message.cpython-35m-x86_64-linux-gnu.so
#3 0x00007ffff745a165 in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#4 0x00007ffff74eacbc in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#5 0x00007ffff74ead93 in PyEval_EvalCodeEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#6 0x00007ffff7372bc5 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#7 0x00007ffff742755e in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#8 0x00007ffff731e085 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#9 0x00007ffff7330f93 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#10 0x00007ffff742755e in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#11 0x00007ffff745493f in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#12 0x00007ffff74eacbc in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#13 0x00007ffff74ead93 in PyEval_EvalCodeEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#14 0x00007ffff74529db in PyEval_EvalCode () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#15 0x00007ffff745f68d in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#16 0x00007ffff734e029 in PyCFunction_Call () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#17 0x00007ffff745b87e in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#18 0x00007ffff74eacbc in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#19 0x00007ffff7458f49 in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#20 0x00007ffff745a649 in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#21 0x00007ffff745a649 in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#22 0x00007ffff745a649 in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#23 0x00007ffff74eacbc in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#24 0x00007ffff74ead93 in PyEval_EvalCodeEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#25 0x00007ffff7372ac8 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#26 0x00007ffff742755e in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#27 0x00007ffff7414fef in _PyObject_CallMethodIdObjArgs () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#28 0x00007ffff7491964 in PyImport_ImportModuleLevelObject () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#29 0x00007ffff745e6c8 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#30 0x00007ffff734e049 in PyCFunction_Call () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#31 0x00007ffff742755e in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#32 0x00007ffff74e9947 in PyEval_CallObjectWithKeywords () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#33 0x00007ffff745404d in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#34 0x00007ffff74eacbc in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#35 0x00007ffff74ead93 in PyEval_EvalCodeEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#36 0x00007ffff74529db in PyEval_EvalCode () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#37 0x00007ffff745f68d in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#38 0x00007ffff734e029 in PyCFunction_Call () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#39 0x00007ffff745b87e in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#40 0x00007ffff74eacbc in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#41 0x00007ffff7458f49 in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#42 0x00007ffff745a649 in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#43 0x00007ffff745a649 in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#44 0x00007ffff745a649 in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#45 0x00007ffff74eacbc in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#46 0x00007ffff74ead93 in PyEval_EvalCodeEx () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#47 0x00007ffff7372ac8 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#48 0x00007ffff742755e in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#49 0x00007ffff7414fef in _PyObject_CallMethodIdObjArgs () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#50 0x00007ffff7491964 in PyImport_ImportModuleLevelObject () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#51 0x00007ffff745e6c8 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#52 0x00007ffff734e049 in PyCFunction_Call () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#53 0x00007ffff742755e in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#54 0x00007ffff74276bf in PyObject_CallFunction () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
#55 0x00007ffff7491df6 in PyImport_Import () from /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
The text was updated successfully, but these errors were encountered: