Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

link error: duplicate symbol: memcpy #3315

Closed
derekbruening opened this issue Dec 15, 2018 · 5 comments
Closed

link error: duplicate symbol: memcpy #3315

derekbruening opened this issue Dec 15, 2018 · 5 comments
Assignees
@derekbruening
Labels
Component-Build OpSys-Linux

Comments

@derekbruening
Copy link
Contributor

The private version of memcpy and other routines in x86_shared.asm are linked into drfrontendlib which is statically linked into various "regular" applications. Some users have reported duplicate symbol errors with some toolchains: details will be added here. We can likely solve by adding WEAK(memcpy), etc. into x86_shared.asm.
@derekbruening derekbruening self-assigned this Dec 15, 2018
@derekbruening
Copy link
Contributor Author

@hgreving2304 -- I remember you hit this somewhere? Could you supply details on what platform/toolchain it was?

@hgreving2304
Copy link

@derekbruening , I think that was in Google3 when the -nostdlib compiler (driver, not linker) switch was missing.

@derekbruening derekbruening mentioned this issue Feb 5, 2019
Open
@derekbruening
Copy link
Contributor Author

This is the plan:

Core DR's copies of memset and memcpy were placed into drhelper for
#1504, to avoid glibc-versioned symbols in drrun and other shipped
binaries. They are linked into the static drfrontendlib and
drinjectlib libraries. They are also in core DR when built as a
static library. In these static library cases they can conflict with
symbols of the same name elsewhere in the linking application.

To solve this, we first separate memset and memcpy from drhelper into
their own library, drmemfuncs. We simply avoid linking it into static
core DR, and into drdecode. For static core DR we assume that
whatever versions the application or glibc provides will be
re-entrant.

We can't avoid linking it into drfrontendlib and drinjectlib (we hit
the glibc versioning problem), but we reduce the chance of conflicts
with these libraries by marking memset and memcpy as weak.

derekbruening added a commit that referenced this issue Feb 12, 2019
@derekbruening
i#3315: avoid conflicts with memset and memcpy
5bbb313 
Core DR's copies of memset and memcpy were placed into drhelper to
avoid glibc-versioned symbols in our shipped binaries (#1504).
They are linked into the static drfrontendlib and drinjectlib
libraries.  They are also in core DR when built as a static library.
In these static library cases they can conflict with symbols of the
same name elsewhere in the linking application.

To solve this, we first separate memset and memcpy from drhelper into
their own library, drmemfuncs.  We simply avoid linking it into static
core DR, and into drdecode.  For static core DR we assume that
whatever versions the application or glibc provides will be
re-entrant.

We can't avoid linking it into drfrontendlib and drinjectlib (we hit
the glibc versioning problem), but we reduce the chance of conflicts
with these libraries by marking memset and memcpy as weak.

Fixes #3315
@derekbruening derekbruening mentioned this issue Feb 12, 2019
Merged
derekbruening added a commit that referenced this issue Feb 12, 2019
@derekbruening
i#3315: avoid conflicts with memset and memcpy (#3374)
da98a60 
Core DR's copies of memset and memcpy were placed into drhelper to
avoid glibc-versioned symbols in our shipped binaries (#1504).
They are linked into the static drfrontendlib and drinjectlib
libraries.  They are also in core DR when built as a static library.
In these static library cases they can conflict with symbols of the
same name elsewhere in the linking application.

To solve this, we first separate memset and memcpy from drhelper into
their own library, drmemfuncs.  We simply avoid linking it into static
core DR, and into drdecode.  For static core DR we assume that
whatever versions the application or glibc provides will be
re-entrant.

We can't avoid linking it into drfrontendlib and drinjectlib (we hit
the glibc versioning problem), but we reduce the chance of conflicts
with these libraries by marking memset and memcpy as weak.

Fixes #3315
derekbruening added a commit that referenced this issue Feb 12, 2019
@derekbruening
i#3315 memcpy conflict: fix Mac build
cc23f64 
Turns WEAK into a nop in asm files as the nasm assembler has no
support for weak symbols, so we live with potential conflicts on Mac.

Temporarily runs Mac on PR builds to ensure this fixes Mac on Travis
(tested locally and it does fix it there).

Issue: #3315
@derekbruening derekbruening mentioned this issue Feb 12, 2019
Merged
@derekbruening
Copy link
Contributor Author

There is a downside to having static DR use glibc memcpy. It lets sanitizers or other interceptors override
DR's memcpy. However, we already have interoperability concerns with sanitizers, so I was assuming we could live with that.

derekbruening added a commit that referenced this issue Feb 12, 2019
@derekbruening
i#3315 memcpy conflict: fix Mac build (#3375)
a808417 
Turns WEAK into a nop in asm files as the nasm assembler has no
support for weak symbols, so we live with potential conflicts on Mac.

Tested by temporarily runing Mac on PR builds to ensure this fixes Mac on Travis,
after first testing locally on Mac.

Issue: #3315
derekbruening added a commit that referenced this issue Feb 12, 2019
@derekbruening
i#3315 memcpy conflict: expand comments on interception
b5afac9 
Expands comments on interception issues when static DR uses libc's
memcpy and memset.

Issue: #3315
@derekbruening derekbruening mentioned this issue Feb 12, 2019
Merged
derekbruening added a commit that referenced this issue Feb 12, 2019
@derekbruening
i#3315 memcpy conflict: expand comments on interception (#3376)
a46b557 
Expands comments on interception issues when static DR uses libc's
memcpy and memset.

Issue: #3315
derekbruening added a commit that referenced this issue Feb 14, 2019
@derekbruening
i#3315 memcpy conflict: mark _check aliases as weak
5c8473e 
Marks the _-memcpy_check and __memset_chk aliases as weak to further
help avoid conflicts with static linking.

Issue: #3315
@derekbruening derekbruening mentioned this issue Feb 14, 2019
Merged
derekbruening added a commit that referenced this issue Feb 14, 2019
@derekbruening
i#3315 memcpy conflict: mark _chk aliases as weak (#3378)
fecfad4 
Marks the _-memcpy_chk and __memset_chk aliases as weak to further
help avoid conflicts with static linking.

Issue: #3315
derekbruening added a commit that referenced this issue Mar 7, 2019
@derekbruening
i#3315 memfuncs: Add memmove to drmemfuncs
c78acf8 
Adds a memmove implementation to drmemfuncs to avoid an import when
the compiler generates a call to memmove in the rare cases it cannot
prove that memcpy is safe.

Issue: #3315
@derekbruening derekbruening mentioned this issue Mar 7, 2019
Merged
derekbruening added a commit that referenced this issue Mar 8, 2019
@derekbruening
i#3315 memfuncs: Add memmove to drmemfuncs (#3428)
a2050d9 
Adds a memmove implementation to drmemfuncs to avoid an import when
the compiler generates a call to memmove in the rare cases it cannot
prove that memcpy is safe.

Issue: #3315
@derekbruening derekbruening mentioned this issue Mar 13, 2019
Closed
derekbruening added a commit that referenced this issue Mar 13, 2019
@derekbruening
i#3437: Add generated header dependence to drmemfuncs
4e57b15 
Adds a dependence on the generated event headers to drmemfuncs to fix
a build race.

Issue: #3315
Fixes #3437
@derekbruening derekbruening mentioned this issue Mar 13, 2019
Merged
derekbruening added a commit that referenced this issue Mar 13, 2019
@derekbruening
i#3437: Add generated header dependence to drmemfuncs (#3438)
7e97817 
Adds a dependence on the generated event headers to drmemfuncs to fix
a build race.

Issue: #3315
Fixes #3437
derekbruening added a commit that referenced this issue Jun 2, 2020
@derekbruening
i#3315,i#3348: Move mem{cpy,set,move} into memfuncs on AArchXX
fca7519 
Provides isolation of mem{cpy,set,move} in the DR static library on
AArchXX.  Moves the assembly mem{cpy,set} into a separate memfuncs.asm
for arm and aarch64, and links in memmove.c.  Marks the
__mem{cpy,set}_chk aliases as weak.  This brings AArchXX into parity
with x86 on library symbol isolation.

Removes the exceptions for AArchXX in the CMake_symbol_check test.
Confirms the build-time test fails without the code changes here:

  CMake Error at xxx/core/CMake_symbol_check.cmake:98 (message):
    *** Error:
    xxx/build_a64_dbg_tests/lib64/debug/libdynamorio_static_nohide.a
    contains a likely-to-conflict symbol: 4279: 00000000002cea3c 0 FUNC GLOBAL
    HIDDEN 248 memcpy

Issue: #3315, #3348
@derekbruening derekbruening mentioned this issue Jun 2, 2020
Merged
derekbruening added a commit that referenced this issue Jun 3, 2020
@derekbruening
i#3315,i#3348: Move mem{cpy,set,move} into memfuncs on AArchXX (#4309)
ef2c973 
Provides isolation of mem{cpy,set,move} in the DR static library on
AArchXX.  Moves the assembly mem{cpy,set} into a separate memfuncs.asm
for arm and aarch64, and links in memmove.c.  Marks the
__mem{cpy,set}_chk aliases as weak.  This brings AArchXX into parity
with x86 on library symbol isolation.

Removes the exceptions for AArchXX in the CMake_symbol_check test.
Confirms the build-time test fails without the code changes here:

  CMake Error at xxx/core/CMake_symbol_check.cmake:98 (message):
    *** Error:
    xxx/build_a64_dbg_tests/lib64/debug/libdynamorio_static_nohide.a
    contains a likely-to-conflict symbol: 4279: 00000000002cea3c 0 FUNC GLOBAL
    HIDDEN 248 memcpy

Issue: #3315, #3348
@TACIXAT
Copy link

TACIXAT commented Nov 10, 2023 •
edited
Loading

Running into this issue on Windows.

libvcruntime.lib(memcpy.obj) : error LNK2005: memcpy already defined in dynamorio.lib(dynamorio.dll)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@derekbruening derekbruening

Labels
Component-Build OpSys-Linux
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@TACIXAT @derekbruening @hgreving2304