failed to compile from source code #4

zhaiwx1987 · 2014-04-02T07:49:33Z

Hi,
I got an error while trying to compile from source code. And after commenting out the conflict codes, everything goes well ...

bellow is the error message:
./libmysqld.a(zutil.c.o):(.data.rel.ro.local+0x0): multiple definition of z_errmsg' ../libmysqld.a(zutil.c.o):(.data.rel.ro.local+0x0): first defined here ../libmysqld.a(adler32.c.o): In functionadler32_combine_':
/u01/project/offical/mysql-5.6-webscalesql-5.6.16-47/storage/innobase/zlib_embedded/adler32.c:141: multiple definition of adler32_combine' ../libmysqld.a(adler32.c.o):/u01/project/offical/mysql-5.6-webscalesql-5.6.16-47/zlib/adler32.c:138: first defined here ../libmysqld.a(crc32.c.o): In functioncrc32_combine':
/u01/project/offical/mysql-5.6-webscalesql-5.6.16-47/storage/innobase/zlib_embedded/crc32.c:432: multiple definition of `crc32_combine'
../libmysqld.a(crc32.c.o):/u01/project/offical/mysql-5.6-webscalesql-5.6.16-47/zlib/crc32.c:381: first defined here
collect2: error: ld returned 1 exit status
make[2]: *** [libmysqld/examples/mysql_embedded] Error 1

The text was updated successfully, but these errors were encountered:

steaphan-fb-com · 2014-04-06T16:02:24Z

I'd need a lot more detail to be able to help you.

Which branch are you building, and which revision (git hash)?

What versions of tools (cmake, gcc, etc...) are you building with? On which platform?

Can you include the full output of the build?

zhaiwx1987 · 2014-04-08T08:03:49Z

I downloaded the source code from
https://github.com/facebook/mysql-5.6/archive/webscalesql-5.6.16-47.zip

$gcc --version
gcc (GCC) 4.7.2

$cmake --version
cmake version 2.8.8

$uname -a
Linux db031134.sqa.cm4 2.6.32-220.17.1.tb619.el6.x86_64 #1 SMP Fri Jun 8 13:48:13 CST 2012 x86_64 x86_64 x86_64 GNU/Linux

full error message..

../../../zlib/libzlib.a(zutil.c.o):(.data.rel.ro.local+0x0): multiple definition of z_errmsg' ../../innobase/libinnobase.a(zutil.c.o):(.rodata+0x0): first defined here ../../../zlib/libzlib.a(adler32.c.o): In functionadler32_combine':
/u01/project/offical/mysql-5.6-webscalesql-5.6.16-47/zlib/adler32.c:138: multiple definition of adler32_combine' ../../innobase/libinnobase.a(adler32.c.o):/u01/project/offical/mysql-5.6-webscalesql-5.6.16-47/storage/innobase/zlib_embedded/adler32.c:141: first defined here ../../../zlib/libzlib.a(crc32.c.o): In functioncrc32_combine':
/u01/project/offical/mysql-5.6-webscalesql-5.6.16-47/zlib/crc32.c:381: multiple definition of crc32_combine' ../../innobase/libinnobase.a(crc32.c.o):/u01/project/offical/mysql-5.6-webscalesql-5.6.16-47/storage/innobase/zlib_embedded/crc32.c:432: first defined here collect2: error: ld returned 1 exit status make[2]: *** [storage/perfschema/unittest/pfs_connect_attr-t] Error 1 make[1]: *** [storage/perfschema/unittest/CMakeFiles/pfs_connect_attr-t.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... ../zlib/libzlib.a(crc32.c.o): In functioncrc32_combine':
/u01/project/offical/mysql-5.6-webscalesql-5.6.16-47/zlib/crc32.c:381: multiple definition of crc32_combine' ../storage/innobase/libinnobase.a(crc32.c.o):/u01/project/offical/mysql-5.6-webscalesql-5.6.16-47/storage/innobase/zlib_embedded/crc32.c:432: first defined here ../zlib/libzlib.a(zutil.c.o):(.data.rel.ro.local+0x0): multiple definition ofz_errmsg'
../storage/innobase/libinnobase.a(zutil.c.o):(.rodata+0x0): first defined here
../zlib/libzlib.a(adler32.c.o): In function adler32_combine': /u01/project/offical/mysql-5.6-webscalesql-5.6.16-47/zlib/adler32.c:138: multiple definition ofadler32_combine'
../storage/innobase/libinnobase.a(adler32.c.o):/u01/project/offical/mysql-5.6-webscalesql-5.6.16-47/storage/innobase/zlib_embedded/adler32.c:141: first defined here
collect2: error: ld returned 1 exit status
make[2]: *** [sql/mysqld] Error 1
make[1]: *** [sql/CMakeFiles/mysqld.dir/all] Error 2
make: *** [all] Error 2

steaphan-fb-com · 2014-04-08T15:20:01Z

I just tested the current version here myself, and it builds fine.

What were your full "cmake" and "make" commands? The intent is that neither should need any options. I built with just "cmake . && make".

eonghk · 2014-04-12T16:13:19Z

I have the same problem if I just type cmake ./ && make in the source directory.
Anyway, I'm using gentoo and if I use the system zlib, it compiles fine.
There are three zlib in the source, any reason?

mictadlo · 2014-08-15T05:24:09Z

I failed to compile it on Manjaro linux:

$ git clone https://github.com/facebook/mysql-5.6.git
$ cmake -DCMAKE_INSTALL_PREFIX:PATH=/home/mictadlo/apps/ && make all install
 ...
85%] Building CXX object sql/CMakeFiles/mysqld.dir/main.cc.o
Linking CXX executable mysqld
../zlib/libzlib.a(crc32.c.o): In function `crc32_combine':
/home/mictadlo/Downloads/mysql-5.6/zlib/crc32.c:381: multiple definition of `crc32_combine'
../storage/innobase/libinnobase.a(crc32.c.o):/home/mictadlo/Downloads/mysql-5.6/storage/innobase/zlib_embedded/crc32.c:432: first defined here
../zlib/libzlib.a(zutil.c.o):(.data.rel.ro.local+0x0): multiple definition of `z_errmsg'
../storage/innobase/libinnobase.a(zutil.c.o):(.rodata+0x0): first defined here
../zlib/libzlib.a(adler32.c.o): In function `adler32_combine':
/home/mictadlo/Downloads/mysql-5.6/zlib/adler32.c:138: multiple definition of `adler32_combine'
../storage/innobase/libinnobase.a(adler32.c.o):/home/mictadlo/Downloads/mysql-5.6/storage/innobase/zlib_embedded/adler32.c:141: first defined here
collect2: error: ld returned 1 exit status
sql/CMakeFiles/mysqld.dir/build.make:114: recipe for target 'sql/mysqld' failed
make[2]: *** [sql/mysqld] Error 1
CMakeFiles/Makefile2:4521: recipe for target 'sql/CMakeFiles/mysqld.dir/all' failed
make[1]: *** [sql/CMakeFiles/mysqld.dir/all] Error 2
Makefile:147: recipe for target 'all' failed
make: *** [all] Error 2

steaphan-fb-com · 2014-10-14T19:12:05Z

If anyone is still having problems with the latest branch, please open a new issue with details.

…that unique keys do not have index extensions defined. Also add suite/rocksdb/type_int_indexes.test

- Implemented efficient storage for VARCHARs in Keys. - Also, NULL values are stored as one NULL-byte (without "default value" as in KeyTupleFormat).

…that unique keys do not have index extensions defined. Also add suite/rocksdb/type_int_indexes.test

- Implemented efficient storage for VARCHARs in Keys. - Also, NULL values are stored as one NULL-byte (without "default value" as in KeyTupleFormat).

Summary: I saw this intermittent ASAN failure ==662590== ERROR: AddressSanitizer: SEGV on unknown address 0x000000000098 (pc 0x00000112fc64 sp 0x7f0324c71710 bp 0x7f0324c71760 T21) AddressSanitizer can not provide additional info. #0 0x112fc63 in my_pthread_fastmutex_lock /data/users/jenkins/workspace/myrocks-prod/BUILD_TYPE/ASan/CLIENT_MODE/Sync/PAGE_SIZE/32/TEST_SET/MixedOther/label/mysql/mysql/mysys/thr_mutex.c:479 #1 0x18fb4b0 in _ZN23Dropped_indices_manager14remove_indicesERKSt13unordered_setIjSt4hashIjESt8equal_toIjESaIjEEP12Dict_manager /data/users/jenkins/workspace/myrocks-prod/BUILD_TYPE/ASan/CLIENT_MODE/Sync/PAGE_SIZE/32/TEST_SET/MixedOther/label/mysql/mysql/include/mysql/psi/mysql_thread.h:688 #2 0x18b031a in _Z17drop_index_threadPv /data/users/jenkins/workspace/myrocks-prod/BUILD_TYPE/ASan/CLIENT_MODE/Sync/PAGE_SIZE/32/TEST_SET/MixedOther/label/mysql/mysql/storage/rocksdb/ha_rocksdb.cc:4927 #3 0x7f0337d611e8 in _ZN6__asan10AsanThread11ThreadStartEv ??:0 #4 0x7f03376dbfa7 in start_thread ??:0 #5 0x7f0335a5f5bc in __clone ??:0 Thread T21 created by T0 here: #0 0x7f0337d5121b in pthread_create ??:0 #1 0x18aec6a in _ZL17rocksdb_init_funcPv /data/users/jenkins/workspace/myrocks-prod/BUILD_TYPE/ASan/CLIENT_MODE/Sync/PAGE_SIZE/32/TEST_SET/MixedOther/label/mysql/mysql/storage/rocksdb/ha_rocksdb.cc:1765 #2 0x60a039 in _Z24ha_initialize_handlertonP13st_plugin_int /data/users/jenkins/workspace/myrocks-prod/BUILD_TYPE/ASan/CLIENT_MODE/Sync/PAGE_SIZE/32/TEST_SET/MixedOther/label/mysql/mysql/sql/handler.cc:673 #3 0xab04e0 in _ZL17plugin_initializeP13st_plugin_int /data/users/jenkins/workspace/myrocks-prod/BUILD_TYPE/ASan/CLIENT_MODE/Sync/PAGE_SIZE/32/TEST_SET/MixedOther/label/mysql/mysql/sql/sql_plugin.cc:1137 #4 0xac8fe3 in _Z11plugin_initPiPPci /data/users/jenkins/workspace/myrocks-prod/BUILD_TYPE/ASan/CLIENT_MODE/Sync/PAGE_SIZE/32/TEST_SET/MixedOther/label/mysql/mysql/sql/sql_plugin.cc:1431 #5 0x5ef74c in _ZL22init_server_componentsv .cc:5482 #6 0x5f0cce in _Z11mysqld_mainiPPc .cc:6254 #7 0x7f033597defe in __libc_start_main ??:0 #8 0x5d86a8 in _start /home/engshare/third-party2/glibc/2.17/src/glibc-2.17/csu/../sysdeps/x86_64/start.S:123 ==662590== ABORTING It looks like the Dropped_indices_manager mutex is accessed by the thread after the mutex was cleaned up. I'm moving the cleanup of the manager to the thread in hopes that this will fix the failure. Test Plan: ran rocksdb suite, nothing failed Reviewers: hermanlee4 Reviewed By: hermanlee4 Differential Revision: https://reviews.facebook.net/D42909

Summary: A single argument constructor could accidentely be used as a type conversion constructor. Add 'explicit' keyword to avoid this. Squash with Issue #4: Efficient VARCHAR storage for keys Test Plan: MTR Reviewers: jtolmer, hermanlee4 Reviewed By: hermanlee4 Differential Revision: https://reviews.facebook.net/D48069

…that unique keys do not have index extensions defined. Also add suite/rocksdb/type_int_indexes.test

- Implemented efficient storage for VARCHARs in Keys. - Also, NULL values are stored as one NULL-byte (without "default value" as in KeyTupleFormat). Test Plan: MTR Reviewers: jtolmer, hermanlee4 Reviewed By: hermanlee4 Differential Revision: https://reviews.facebook.net/D48069

…that unique keys do not have index extensions defined. Also add suite/rocksdb/type_int_indexes.test

Summary: [Porting Notes] We want to dump raft logs to vanilla async replicas regardless of whether it's the relay log or binlog. Effectively after this change we'll dump relay logs on the followers and binlogs on the leader. When the raft role changes, the logs to the dumped are also changed. Dump_log class is introduced as a thin wrapper/continer around mysql_bin_log or rli->relay_log and is inited with mysql_bin_log to emulate vanilla mysql behavior. Dump threads use the global dump_log object instead of mysql_bin_log directly. We switch the log in dump log only when raft role changes (in binlog_change_to_binlog() and binlog_change_to_apply_log()). During raft role change we take all log releated locks (LOCK_log, LOCK_index, LOCK_binlog_end_pos, and dump log lock) to serialize it with other log operations like dumping logs. Related doc - https://fb.quip.com/oTVAAdgEi4zY This diff contains below 7 patches: D23013977 D24766787 D24716539 D24900223 D24955284 D25174166 D25775525 Reviewed By: luqun Differential Revision: D26141496 ------------------------------------------------------------------------------- Passing raw_log pointer to wait_with_heartbeat() and wait_without_heartbeat() Summary: When enable_raft_plugin is OFF Dump_log::lock() is a no-op. Which means that when enable_raft_plugin is OFF there can be a race between log switching and dump threads. This could lead to a scenario where the raw_log that wait_next_event() is working on might be different than what wait_with_heartbeat()/wait_without_heartbeat() is working on. This can cause deadlocks because wait_with_heartbeat()/wait_without_heartbeat()'s mysql_cond_wait would unlock and then lock a different log's LOCK_binlog_end_pos mutex which would then never be unlocked by wait_next_event(). Reviewed By: anirbanr-fb Differential Revision: D32152658 ----------------------------------------------------------------------------------------- Fix rpl_raft_dump_raft_logs Summary: This tests completes but fails because the following warning exists: ``` 2022-08-30T16:28:00.159525Z 11 [ERROR] [MY-013114] [Repl] Slave I/O for channel '': Got fatal error 1236 from master when reading data from binary log: 'Slave has more GTIDs than the master has, using the master's SERVER_UUID. This may indicate that the end of the binary log was truncated or that the last binary log file was lost, e.g., after a power or disk failure when sync_binlog != 1. The master may or may not have rolled back transactions that were already replicated to the slave. Suggest to replicate any transactions that master has rolled back from slave to master, and/or commit empty transactions on master to account for transactions that have been', Error_code: MY-013114 ``` Since the MTR result file is valid, we can suppress this error. Reviewed By: yichenshen Differential Revision: D39141846 ------------------------------------------------------------------------------- Fix heap overflow in group_relay_log_name handling Summary: We were accessing group_relay_log_name in Query_log_event::do_apply_event_worker() but it's assigned only after the coordinator thread encounters an end event (i.e. xid event or a query event with "COMMIT" or "ROLLBACK" query). This was causing a race between accessing group_relay_log_name in the worker thread and writing it on the coordinator thread. We don't need to set transaction position in events other than end event, so now we set transaction position in query event only if it's an end event. The race is eliminated because group_relay_log_name is set before enqueuing the event to the worker thread (in both dep repl and vanilla mts). Reviewed By: lth Differential Revision: D28767430 ------------------------------------------------------------------------------- fix memory during MYSQL_BIN_LOG::open_existing_binlog Summary: asandebug complain there are memory leaks during MYSQL_BIN_LOG open Direct leak of 50 byte(s) in 1 object(s) allocated from: #0 0x67460ef in malloc #1 0x93f0777 in my_raw_malloc(unsigned long, int) #2 0x93f064a in my_malloc(unsigned int, unsigned long, int) facebook#3 0x93f0eb0 in my_strdup(unsigned int, char const*, int) facebook#4 0x8af01a6 in MYSQL_BIN_LOG::open(unsigned int, char const*, char const*, unsigned int) facebook#5 0x8af8064 in MYSQL_BIN_LOG::open_binlog(char const*, char const*, unsigned long, bool, bool, bool, Format_description_log_event*, unsigned int, RaftRotateInfo*, bool) facebook#6 0x8b00c00 in MYSQL_BIN_LOG::new_file_impl(bool, Format_description_log_event*, RaftRotateInfo*) facebook#7 0x8d65e47 in rotate_relay_log(Master_info*, bool, bool, bool, RaftRotateInfo*) facebook#8 0x8d661c0 in rotate_relay_log_for_raft(RaftRotateInfo*) facebook#9 0x8c7696a in process_raft_queue facebook#10 0xa0fa1fd in pfs_spawn_thread(void*) facebook#11 0x7f8c9a12b20b in start_thread release these memory before assign them Reviewed By: Pushapgl Differential Revision: D28819752

Under load, if START SLAVE IO_THREAD and STOP SLAVE execute concurrently, the following deadlock is possible: - Thread 44 is executing STOP SLAVE, has read-locked the channel map, and is waiting for the slave I/O thread to quit. - Thread 55 is executing START SLAVE, and is blocked waiting for the channel map write lock. - Thread 54 is the SLAVE I/O thread, and is trying to read-lock the channel map lock. The request of T54 is compatible with the current lock state, but according to POSIX, once a write request is pending, it is up to the implementation whether to satisfy them or block. For the fix, observe that the starting replica I/O thread only tries to signal the stats thread to start, thus move this code to the START REPLICA command-executing thread instead, which already happens to hold the channel map lock. This also forces to move the stopping of the stats thread from the replica I/O thread to the STOP REPLICA command-executing thread. This fixes intermittent but often-seen failures on rpl.rpl_multi_source_channel_map_stress under macOS. Squash with b015dd3 Stacktraces: Thread 44: frame facebook#5: 0x0000000106483108 mysqld`inline_mysql_cond_timedwait(that=0x000000014585d438, mutex=0x000000014585d2f0, abstime=0x000000016db61588, src_file="/Users/laurynas/vilniusdb/fb-mysql/sql/rpl_replica.cc", src_line=2367) at mysql_cond.h:224:16 frame facebook#6: 0x0000000106460818 mysqld`terminate_slave_thread(thd=0x00000001458fb800, term_lock=0x000000014585d2f0, term_cond=0x000000014585d438, slave_running=0x000000014585d4f4, stop_wait_timeout=0x000000016db61700, need_lock_term=false, force=false) at rpl_replica.cc:2367:9 frame facebook#7: 0x0000000106460188 mysqld`terminate_slave_threads(mi=0x000000014585ce00, thread_mask=1, stop_wait_timeout=31536000, need_lock_term=false) at rpl_replica.cc:2204:18 frame facebook#8: 0x000000010645873c mysqld`stop_slave(thd=0x0000000145952800, mi=0x000000014585ce00, net_report=true, for_one_channel=true, push_temp_tables_warning=0x000000016db61a37, invoked_by_raft=false) at rpl_replica.cc:10032:9 frame facebook#9: 0x00000001064593c4 mysqld`stop_slave_cmd(thd=0x0000000145952800) at rpl_replica.cc:971:13 Thread 55: frame facebook#2: 0x000000018e5bf73c libsystem_pthread.dylib`_pthread_rwlock_lock_slow + 720 frame facebook#3: 0x0000000104c3d2ac mysqld`native_rw_wrlock(rwp=0x00006000039dc0e8) at thr_rwlock.h:101:10 frame facebook#4: 0x0000000104c3d21c mysqld`inline_mysql_rwlock_wrlock(that=0x00006000039dc0e8, src_file="/Users/laurynas/vilniusdb/fb-mysql/sql/rpl_gtid.h", src_line=473) at mysql_rwlock.h:410:12 frame facebook#5: 0x0000000104c3c6f0 mysqld`Checkable_rwlock::wrlock(this=0x00006000039dc0e0) at rpl_gtid.h:473:5 frame facebook#6: 0x00000001054dc6e0 mysqld`Multisource_info::wrlock(this=0x000000010a182980) at rpl_msr.h:441:25 frame facebook#7: 0x0000000106458bb0 mysqld`start_slave_cmd(thd=0x0000000130008a00) at rpl_replica.cc:832:15 Thread 54: frame facebook#2: 0x000000018e5bf73c libsystem_pthread.dylib`_pthread_rwlock_lock_slow + 720 frame facebook#3: 0x0000000104b62d90 mysqld`native_rw_rdlock(rwp=0x00006000039dc0e8) at thr_rwlock.h:82:10 frame facebook#4: 0x0000000104b62d00 mysqld`inline_mysql_rwlock_rdlock(that=0x00006000039dc0e8, src_file="/Users/laurynas/vilniusdb/fb-mysql/sql/rpl_gtid.h", src_line=464) at mysql_rwlock.h:340:12 frame facebook#5: 0x0000000104b62bcc mysqld`Checkable_rwlock::rdlock(this=0x00006000039dc0e0) at rpl_gtid.h:464:5 frame facebook#6: 0x0000000104b61cec mysqld`Multisource_info::rdlock(this=0x000000010a182980) at rpl_msr.h:415:25 frame facebook#7: 0x0000000104b618f0 mysqld`start_handle_slave_stats_daemon() at slave_stats_daemon.cc:233:15 frame facebook#8: 0x00000001064627dc mysqld`handle_slave_io(arg=0x000000014585ce00) at rpl_replica.cc:6050:36

Summary: some MTR failed in ubsan due to num of rows/records calculated in records_in_range() is a negative values which is caused by m_actual_disk_size is a negative value ``` storage/rocksdb/ha_rocksdb.cc:: runtime error: -56.8272 is outside the range of representable values of type 'unsigned long long' #0 myrocks::ha_rocksdb::records_in_range_internal(unsigned int, key_range*, key_range*, long, long, unsigned long long*, unsigned long long*) /data/sandcastle/boxes/trunk-git-mysql/storage/rocksdb/ha_rocksdb.cc:14855:16 #1 myrocks::ha_rocksdb::records_in_range(unsigned int, key_range*, key_range*) /data/sandcastle/boxes/trunk-git-mysql/storage/rocksdb/ha_rocksdb.cc:14760:3 #2 handler::multi_range_read_info_const(unsigned int, RANGE_SEQ_IF*, void*, unsigned int, unsigned int*, unsigned int*, Cost_estimate*) /data/sandcastle/boxes/trunk-git-mysql/sql/handler.cc:6608:26 #3 myrocks::ha_rocksdb::multi_range_read_info_const(unsigned int, RANGE_SEQ_IF*, void*, unsigned int, unsigned int*, unsigned int*, Cost_estimate*) /data/sandcastle/boxes/trunk-git-mysql/storage/rocksdb/ha_rocksdb.cc:19002:18 #4 check_quick_select(THD*, RANGE_OPT_P ``` Due to m_actual_disk_size is an estimated value, always reset to 0 if it i becomes negative. Differential Revision: D50531919 fbshipit-source-id: e1fc02e03040c9ee05bf8015b59c87ada0a16469

Summary: After bumping the thread stack size to 10MB for ASAN, this test started failing because the Bison parser ran out of stack before the thread did. This test deliberately tries to cause a thread stack overflow by generating an arbitrarily deeply nested SQL query to exercise the `check_stack_overrun` logic. Because our thread stack is so large in ASAN, we hit the Bison "stack" instead first. From the error log: ``` CURRENT_TEST: main.execution_constants mysqltest: At line 42: Query '$query_head 0 $query_tail' failed with wrong error 3950: 'Out of memory', should have failed with any of '0,1436' errors. ``` Parser OOM Stack: ``` (lldb) bt * thread #42, name = 'connection', stop reason = breakpoint 1.1 * frame #0: 0x000000000a33ea53 mysqld`my_error(nr=<unavailable>, MyFlags=<unavailable>) at my_error.cc:217:3 frame #1: 0x0000000008065a21 mysqld`MYSQLerror(location=<unavailable>, thd=<unavailable>, (null)=<unavailable>, s=<unavailable>) (.__uniq.242601085889238811981650692569837157750) at sql_yacc.yy:302:5 [artificial] frame #2: 0x0000000008034304 mysqld`MYSQLparse(YYTHD=<unavailable>, parse_tree=<unavailable>) at sql_yacc.cc:25323:9 frame #3: 0x0000000008009085 mysqld`THD::sql_parser(this=<unavailable>) at sql_class.cc:3811:7 frame #4: 0x00000000082b1f4c mysqld`parse_sql(thd=0x0000628000300100, parser_state=<unavailable>, creation_ctx=<unavailable>) at sql_parse.cc:7963:34 frame #5: 0x00000000082a060d mysqld`dispatch_sql_command(thd=<unavailable>, parser_state=<unavailable>, last_timer=<unavailable>) at sql_parse.cc:5993:11 frame #6: 0x000000000829bd7b mysqld`dispatch_command(thd=<unavailable>, com_data=<unavailable>, command=<unavailable>) at sql_parse.cc:2445:7 frame #7: 0x000000000829e5e9 mysqld`do_command(thd=<unavailable>) at sql_parse.cc:1636:18 frame #8: 0x00000000086c5e34 mysqld`handle_connection(arg=0x00006030001e0940) (.__uniq.188644437719200549207133117087266310382) at connection_handler_per_thread.cc:307:13 frame #9: 0x000000000b3e123b mysqld`pfs_spawn_thread(arg=0x000061600044f4a0) (.__uniq.322994040891040637281203315118403052672) at pfs.cc:2983:3 frame #10: 0x00007ffff7c9abaf libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:434:8 frame #11: 0x00007ffff7d2d17c libc.so.6`__clone3 at clone3.S:81 ``` sql/sql_yacc.yy: ``` static void MYSQLerror(YYLTYPE *location, THD *thd, Parse_tree_root **, const char *s) { if (strcmp(s, "syntax error") == 0) { thd->syntax_error_at(*location); } else if (strcmp(s, "memory exhausted") == 0) { my_error(ER_DA_OOM, MYF(0)); ... ``` mysql-fork/_build-8.0-ASanDebug/sql/sql_yacc.cc: ``` yyoverflow (YY_("memory exhausted"), &yyss1, yysize * YYSIZEOF (*yyssp), &yyvs1, yysize * YYSIZEOF (*yyvsp), &yyls1, yysize * YYSIZEOF (*yylsp), &yystacksize); ``` Differential Revision: D51233935 fbshipit-source-id: 04391603e7baef0c6e6fa4f308de0aad545474c2

Under load, if START SLAVE IO_THREAD and STOP SLAVE execute concurrently, the following deadlock is possible: - Thread 44 is executing STOP SLAVE, has read-locked the channel map, and is waiting for the slave I/O thread to quit. - Thread 55 is executing START SLAVE, and is blocked waiting for the channel map write lock. - Thread 54 is the SLAVE I/O thread, and is trying to read-lock the channel map lock. The request of T54 is compatible with the current lock state, but according to POSIX, once a write request is pending, it is up to the implementation whether to satisfy them or block. For the fix, observe that the starting replica I/O thread only tries to signal the stats thread to start, thus move this code to the START REPLICA command-executing thread instead, which already happens to hold the channel map lock. This also forces to move the stopping of the stats thread from the replica I/O thread to the STOP REPLICA command-executing thread. This fixes intermittent but often-seen failures on rpl.rpl_multi_source_channel_map_stress under macOS. Squash with b015dd3 Stacktraces: Thread 44: frame facebook#5: 0x0000000106483108 mysqld`inline_mysql_cond_timedwait(that=0x000000014585d438, mutex=0x000000014585d2f0, abstime=0x000000016db61588, src_file="/Users/laurynas/vilniusdb/fb-mysql/sql/rpl_replica.cc", src_line=2367) at mysql_cond.h:224:16 frame facebook#6: 0x0000000106460818 mysqld`terminate_slave_thread(thd=0x00000001458fb800, term_lock=0x000000014585d2f0, term_cond=0x000000014585d438, slave_running=0x000000014585d4f4, stop_wait_timeout=0x000000016db61700, need_lock_term=false, force=false) at rpl_replica.cc:2367:9 frame facebook#7: 0x0000000106460188 mysqld`terminate_slave_threads(mi=0x000000014585ce00, thread_mask=1, stop_wait_timeout=31536000, need_lock_term=false) at rpl_replica.cc:2204:18 frame facebook#8: 0x000000010645873c mysqld`stop_slave(thd=0x0000000145952800, mi=0x000000014585ce00, net_report=true, for_one_channel=true, push_temp_tables_warning=0x000000016db61a37, invoked_by_raft=false) at rpl_replica.cc:10032:9 frame facebook#9: 0x00000001064593c4 mysqld`stop_slave_cmd(thd=0x0000000145952800) at rpl_replica.cc:971:13 Thread 55: frame facebook#2: 0x000000018e5bf73c libsystem_pthread.dylib`_pthread_rwlock_lock_slow + 720 frame facebook#3: 0x0000000104c3d2ac mysqld`native_rw_wrlock(rwp=0x00006000039dc0e8) at thr_rwlock.h:101:10 frame facebook#4: 0x0000000104c3d21c mysqld`inline_mysql_rwlock_wrlock(that=0x00006000039dc0e8, src_file="/Users/laurynas/vilniusdb/fb-mysql/sql/rpl_gtid.h", src_line=473) at mysql_rwlock.h:410:12 frame facebook#5: 0x0000000104c3c6f0 mysqld`Checkable_rwlock::wrlock(this=0x00006000039dc0e0) at rpl_gtid.h:473:5 frame facebook#6: 0x00000001054dc6e0 mysqld`Multisource_info::wrlock(this=0x000000010a182980) at rpl_msr.h:441:25 frame facebook#7: 0x0000000106458bb0 mysqld`start_slave_cmd(thd=0x0000000130008a00) at rpl_replica.cc:832:15 Thread 54: frame facebook#2: 0x000000018e5bf73c libsystem_pthread.dylib`_pthread_rwlock_lock_slow + 720 frame facebook#3: 0x0000000104b62d90 mysqld`native_rw_rdlock(rwp=0x00006000039dc0e8) at thr_rwlock.h:82:10 frame facebook#4: 0x0000000104b62d00 mysqld`inline_mysql_rwlock_rdlock(that=0x00006000039dc0e8, src_file="/Users/laurynas/vilniusdb/fb-mysql/sql/rpl_gtid.h", src_line=464) at mysql_rwlock.h:340:12 frame facebook#5: 0x0000000104b62bcc mysqld`Checkable_rwlock::rdlock(this=0x00006000039dc0e0) at rpl_gtid.h:464:5 frame facebook#6: 0x0000000104b61cec mysqld`Multisource_info::rdlock(this=0x000000010a182980) at rpl_msr.h:415:25 frame facebook#7: 0x0000000104b618f0 mysqld`start_handle_slave_stats_daemon() at slave_stats_daemon.cc:233:15 frame facebook#8: 0x00000001064627dc mysqld`handle_slave_io(arg=0x000000014585ce00) at rpl_replica.cc:6050:36

Summary: add histogram for rpl_semi_sync_master_trx_wait. 8.0 porting notes: Keeps the same histogram status variables as before since these are already being read by various applications. We should eventually remove this. Reference Patch: facebook@d1a1394 Reference Patch: facebook@15333b2e6f9 Differential Revision: D21832889 ---------------------------------------------------------------------- Fix semi_sync histogram reporting Summary: Fix a porting bug with semi_sync histograms. Reviewed By: george-reynya Differential Revision: D40964563 ---------------------------------------------------------------------- Semisync histogram double free (facebook#1290) Summary: Avoid double free on latency histogram data Before this fix, rpl.rpl_semi_sync_alias test under ASan with ``` ================================================================= ==65389==ERROR: AddressSanitizer: heap-use-after-free on address 0x0001742e17d4 at pc 0x000107febaf0 bp 0x00016ea8f710 sp 0x00016ea8f708 READ of size 4 at 0x0001742e17d4 thread T80 #0 0x107febaec in my_free(void*) my_malloc.cc:135 facebook#1 0x103cb9828 in free_latency_histogram_sysvars(SHOW_VAR*) mysqld.cc:4668 facebook#2 0x103cb99bc in prepare_latency_histogram_vars(latency_histogram*, SHOW_VAR*, unsigned long long*) mysqld.cc:4692 facebook#3 0x17c65826c in rpl_semi_sync_master_trx_wait_histogram(THD*, SHOW_VAR*, char*) semisync_source_plugin.cc:581 facebook#4 0x10be1b4cc in PFS_status_variable_cache::manifest(THD*, SHOW_VAR const*, System_status_var*, char const*, bool, bool) pfs_variable.cc:1366 facebook#5 0x10be1ba90 in PFS_status_variable_cache::do_materialize_all(THD*) pfs_variable.cc:1172 facebook#6 0x10c0ab33c in PFS_variable_cache<Status_variable>::materialize_all(THD*) pfs_variable.h:536 facebook#7 0x10c0ab294 in table_session_status::rnd_init(bool) table_session_status.cc:111 facebook#8 0x10bceb790 in ha_perfschema::rnd_init(bool) ha_perfschema.cc:1686 facebook#9 0x1033c7cec in handler::ha_rnd_init(bool) handler.cc:3157 facebook#10 0x103975380 in TableScanIterator::Init() basic_row_iterators.cc:230 facebook#11 0x103a33a18 in FilterIterator::Init() composite_iterators.h:82 facebook#12 0x103982ec0 in MaterializeIterator::MaterializeQueryBlock(MaterializeIterator::QueryBlock const&, unsigned long long*) composite_iterators.cc:845 facebook#13 0x103981410 in MaterializeIterator::Init() composite_iterators.cc:660 facebook#14 0x1049fc518 in Query_expression::ExecuteIteratorQuery(THD*) sql_union.cc:1293 facebook#15 0x1049fd358 in Query_expression::execute(THD*) sql_union.cc:1355 facebook#16 0x1047ae7ac in Sql_cmd_dml::execute_inner(THD*) sql_select.cc:870 facebook#17 0x1047ac344 in Sql_cmd_dml::execute(THD*) sql_select.cc:618 facebook#18 0x1047ffcc8 in Sql_cmd_show::execute(THD*) sql_show.cc:232 facebook#19 0x10480ab58 in Sql_cmd_show_status::execute(THD*) sql_show.cc:894 facebook#20 0x1045cea6c in mysql_execute_command(THD*, bool, unsigned long long*) sql_parse.cc:5323 facebook#21 0x1045c5dcc in dispatch_sql_command(THD*, Parser_state*, unsigned long long*) sql_parse.cc:6093 facebook#22 0x1045bb92c in dispatch_command(THD*, COM_DATA const*, enum_server_command) sql_parse.cc:2444 facebook#23 0x1045c06f8 in do_command(THD*) sql_parse.cc:1636 facebook#24 0x104cc4cc4 in handle_connection(void*) connection_handler_per_thread.cc:307 facebook#25 0x10bd130d4 in pfs_spawn_thread(void*) pfs.cc:2983 facebook#26 0x18ad47fa4 in _pthread_start+0x90 (libsystem_pthread.dylib:arm64e+0x6fa4) facebook#27 0x18ad42d9c in thread_start+0x4 (libsystem_pthread.dylib:arm64e+0x1d9c) 0x0001742e17d4 is located 4 bytes inside of 40-byte region [0x0001742e17d0,0x0001742e17f8) freed by thread T80 here: #0 0x139ff6de4 in wrap_free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x3ede4) facebook#1 0x107febcfc in my_raw_free(void*) my_malloc.cc:269 facebook#2 0x107feba48 in my_free(void*) my_malloc.cc:141 facebook#3 0x103cb9828 in free_latency_histogram_sysvars(SHOW_VAR*) mysqld.cc:4668 facebook#4 0x17c6231e8 in ReplSemiSyncMaster::~ReplSemiSyncMaster() semisync_source.cc:517 facebook#5 0x17c623488 in ReplSemiSyncMaster::~ReplSemiSyncMaster() semisync_source.cc:516 facebook#6 0x17c651484 in semi_sync_master_plugin_deinit(void*) semisync_source_plugin.cc:833 facebook#7 0x10467aa90 in plugin_deinitialize(st_plugin_int*, bool) sql_plugin.cc:1123 facebook#8 0x1046730b0 in reap_plugins() sql_plugin.cc:1192 facebook#9 0x1046863b4 in mysql_uninstall_plugin(THD*, MYSQL_LEX_CSTRING) sql_plugin.cc:2602 facebook#10 0x104685374 in Sql_cmd_uninstall_plugin::execute(THD*) sql_plugin.cc:3731 facebook#11 0x1045cea6c in mysql_execute_command(THD*, bool, unsigned long long*) sql_parse.cc:5323 facebook#12 0x1045c5dcc in dispatch_sql_command(THD*, Parser_state*, unsigned long long*) sql_parse.cc:6093 facebook#13 0x1045bb92c in dispatch_command(THD*, COM_DATA const*, enum_server_command) sql_parse.cc:2444 facebook#14 0x1045c06f8 in do_command(THD*) sql_parse.cc:1636 facebook#15 0x104cc4cc4 in handle_connection(void*) connection_handler_per_thread.cc:307 facebook#16 0x10bd130d4 in pfs_spawn_thread(void*) pfs.cc:2983 facebook#17 0x18ad47fa4 in _pthread_start+0x90 (libsystem_pthread.dylib:arm64e+0x6fa4) facebook#18 0x18ad42d9c in thread_start+0x4 (libsystem_pthread.dylib:arm64e+0x1d9c) ``` It seems that the double invocation of `free_latency_histogram_sysvars` is correct in this case, thus protect against the double free with resetting the pointers to nullptr. Pull Request resolved: facebook#1290 Reviewed By: sunshine-Chun Differential Revision: D45277600 Pulled By: hermanlee

steaphan-fb-com closed this as completed Oct 14, 2014

jonahcohen referenced this issue in MySQLOnRocksDB/mysql-5.6 Dec 19, 2014

Fix Issue #4: Crash when using pseudo-unique keys

c066766

jonahcohen referenced this issue in MySQLOnRocksDB/mysql-5.6 Dec 19, 2014

Fix Issue #4, part#2: make other members of class RDBSE_KEYDEF aware …

d1d7de0

…that unique keys do not have index extensions defined. Also add suite/rocksdb/type_int_indexes.test

jonahcohen referenced this issue in MySQLOnRocksDB/mysql-5.6 Dec 19, 2014

Issue #4: Efficient VARCHAR storage for keys

74c4946

- Implemented efficient storage for VARCHARs in Keys. - Also, NULL values are stored as one NULL-byte (without "default value" as in KeyTupleFormat).

hermanlee pushed a commit that referenced this issue Sep 14, 2015

Fix Issue #4: Crash when using pseudo-unique keys

2cb9e48

hermanlee pushed a commit that referenced this issue Sep 14, 2015

Fix Issue #4, part#2: make other members of class RDBSE_KEYDEF aware …

c1b0d6f

…that unique keys do not have index extensions defined. Also add suite/rocksdb/type_int_indexes.test

hermanlee pushed a commit that referenced this issue Sep 14, 2015

Issue #4: Efficient VARCHAR storage for keys

a61d690

- Implemented efficient storage for VARCHARs in Keys. - Also, NULL values are stored as one NULL-byte (without "default value" as in KeyTupleFormat).

mariadb-DanielLee mentioned this issue Nov 27, 2015

mysqld some times crashes at the end of Random Query Generator test #111

Closed

This was referenced Dec 8, 2015

MyRocks slave crashed because of the wrong errkey. #120

Closed

mysqld crashed when running a RQG transactions test, not reproducible at will #123

Closed

This was referenced Dec 28, 2015

handle_fatal_signal (sig=11) in thd_ha_data #129

Closed

Assertion `v->cfd_->internal_comparator().Compare(start, end) <= 0' failed. #131

Closed

spetrunia added a commit that referenced this issue Jan 5, 2016

Fix Issue #4: Crash when using pseudo-unique keys

4bff942

spetrunia added a commit that referenced this issue Jan 5, 2016

Fix Issue #4, part#2: make other members of class RDBSE_KEYDEF aware …

db852b7

…that unique keys do not have index extensions defined. Also add suite/rocksdb/type_int_indexes.test

rameshvs02 mentioned this issue Feb 18, 2016

InnoDB: Failing assertion: no_trx || parent_trx->is_primary in file ha_innodb.cc line 11527 #170

Closed

mariadb-DanielLee mentioned this issue Apr 28, 2016

RQG test failed with a core dump #237

Closed

RickPizzi mentioned this issue Oct 21, 2016

MyRocks: 1 hour stall in DelayWrite() during bulk load #348

Closed

spetrunia mentioned this issue Nov 16, 2016

rocksdb.bulk_load test times out #428

Closed

alxyang mentioned this issue Jan 23, 2017

Flush memtable and compact L0 files for read only workloads #427

Open

hermanlee pushed a commit that referenced this issue Jan 31, 2017

Fix Issue #4: Crash when using pseudo-unique keys

a082f45

hermanlee pushed a commit that referenced this issue Jan 31, 2017

Fix Issue #4, part#2: make other members of class RDBSE_KEYDEF aware …

edfec5f

…that unique keys do not have index extensions defined. Also add suite/rocksdb/type_int_indexes.test

hermanlee mentioned this issue Aug 23, 2024

Implement LAST_INSERT_IDS function #1469

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

failed to compile from source code #4

failed to compile from source code #4

zhaiwx1987 commented Apr 2, 2014

steaphan-fb-com commented Apr 6, 2014

zhaiwx1987 commented Apr 8, 2014

steaphan-fb-com commented Apr 8, 2014

eonghk commented Apr 12, 2014

mictadlo commented Aug 15, 2014

steaphan-fb-com commented Oct 14, 2014

failed to compile from source code #4

failed to compile from source code #4

Comments

zhaiwx1987 commented Apr 2, 2014

steaphan-fb-com commented Apr 6, 2014

zhaiwx1987 commented Apr 8, 2014

steaphan-fb-com commented Apr 8, 2014

eonghk commented Apr 12, 2014

mictadlo commented Aug 15, 2014

steaphan-fb-com commented Oct 14, 2014