Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Undefined behavior in uloc.cpp in icu4c #6560

Open
mbautin opened this issue Dec 5, 2020 · 3 comments
Open

Undefined behavior in uloc.cpp in icu4c #6560

mbautin opened this issue Dec 5, 2020 · 3 comments
Assignees
@mbautin

Comments

@mbautin
Copy link
Contributor

mbautin commented Dec 5, 2020 •
edited by frozenspider
Loading

Since icu4c started being instrumented with ASAN/UBSAN, we've been getting htis:

ybd asan --cxx-test bfql_bfql-test --gtest_filter BfqlTest.TestBuiltinToJson

uloc.cpp:1206:5: runtime error: null pointer passed as argument 1, which is declared to never be null
--
31 | /usr/include/string.h:43:28: note: nonnull attribute specified here
32 | #0 0x7fb24552bcfa in ulocimp_getLanguage_67 (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x794cfa)
33 | #1 0x7fb24552fb16 in uloc_getScript_67 (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x798b16)
34 | #2 0x7fb2455776a6 in createTagStringWithAlternates(char const*, int, char const*, int, char const*, int, char const*, int, char const*, icu_67::ByteSink&, UErrorCode*) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x7e06a6)
35 | #3 0x7fb245575f1f in createLikelySubtagsString(char const*, int, char const*, int, char const*, int, char const*, int, icu_67::ByteSink&, UErrorCode*) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x7def1f)
36 | #4 0x7fb24557124c in _uloc_addLikelySubtags(char const*, icu_67::ByteSink&, UErrorCode*) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x7da24c)
37 | #5 0x7fb245572696 in _uloc_minimizeSubtags(char const*, icu_67::ByteSink&, UErrorCode*) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x7db696)
38 | #6 0x7fb245571bc8 in ulocimp_minimizeSubtags_67 (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x7dabc8)
39 | #7 0x7fb245545ccc in icu_67::Locale::minimizeSubtags(UErrorCode&) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x7aeccc)
40 | #8 0x7fb246ba19c9 in icu_67::Calendar::setWeekData(icu_67::Locale const&, char const*, UErrorCode&) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicui18n.so.67+0xba09c9)
41 | #9 0x7fb246ba46ad in icu_67::Calendar::Calendar(icu_67::TimeZone const&, icu_67::Locale const&, UErrorCode&) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicui18n.so.67+0xba36ad)
42 | #10 0x7fb246bd0426 in icu_67::GregorianCalendar::GregorianCalendar(icu_67::TimeZone const&, UErrorCode&) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicui18n.so.67+0xbcf426)
43 | #11 0x7fb248cd1a62 in yb::(anonymous namespace)::CreateCalendar() /nfusr/centos-gcp-cloud/jenkins-worker-335/jenkins/jenkins-github-yugabyte-db-phabricator-64408/build/asan-clang-dynamic-ninja/../../src/yb/util/date_time.cc:82:21
44 | #12 0x7fb248cd1034 in yb::DateTime::DateFromString(string const&) /nfusr/centos-gcp-cloud/jenkins-worker-335/jenkins/jenkins-github-yugabyte-db-phabricator-64408/build/asan-clang-dynamic-ninja/../../src/yb/util/date_time.cc:233:27
45 | #13 0x55d335 in yb::bfql::BfqlTest_TestBuiltinToJson_Test::TestBody() /nfusr/centos-gcp-cloud/jenkins-worker-335/jenkins/jenkins-github-yugabyte-db-phabricator-64408/build/asan-clang-dynamic-ninja/../../src/yb/util/bfql/bfql-test.cc:584:27
46 | #14 0x7fb24b99b421 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2402:10
47 | #15 0x7fb24b99b421 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2438
48 | #16 0x7fb24b9143db in testing::Test::Run() /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2474:5
49 | #17 0x7fb24b917189 in testing::TestInfo::Run() /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2656:11
50 | #18 0x7fb24b919c07 in testing::TestCase::Run() /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2774:28
51 | #19 0x7fb24b93f817 in testing::internal::UnitTestImpl::RunAllTests() /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:4649:43
52 | #20 0x7fb24b9a40fc in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2402:10
53 | #21 0x7fb24b9a40fc in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2438
54 | #22 0x7fb24b93e48b in testing::UnitTest::Run() /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:4257:10
55 | #23 0x7fb24ccf292b in main /nfusr/centos-gcp-cloud/jenkins-worker-335/jenkins/jenkins-github-yugabyte-db-phabricator-64408/build/asan-clang-dynamic-ninja/../../src/yb/util/test_main.cc:104:13
56 | #24 0x7fb23ed38824 in __libc_start_main /tmp/glibc-20181130-26094-cs1x60/glibc-2.23/csu/../csu/libc-start.c:289
57 | #25 0x4388b8 in _start /tmp/glibc-20181130-26094-cs1x60/glibc-2.23/csu/../sysdeps/x86_64/start.S:118
58 |  
59 | SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior uloc.cpp:1206:5 in
60 | uloc.cpp:1254:5: runtime error: null pointer passed as argument 1, which is declared to never be null
61 | /usr/include/string.h:43:28: note: nonnull attribute specified here
62 | #0 0x7fb24552ccca in ulocimp_getScript_67 (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x795cca)
63 | #1 0x7fb24552ff7f in uloc_getCountry_67 (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x798f7f)
64 | #2 0x7fb245577751 in createTagStringWithAlternates(char const*, int, char const*, int, char const*, int, char const*, int, char const*, icu_67::ByteSink&, UErrorCode*) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x7e0751)
65 | #3 0x7fb245575f1f in createLikelySubtagsString(char const*, int, char const*, int, char const*, int, char const*, int, icu_67::ByteSink&, UErrorCode*) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x7def1f)
66 | #4 0x7fb245572850 in _uloc_minimizeSubtags(char const*, icu_67::ByteSink&, UErrorCode*) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x7db850)
67 | #5 0x7fb245571bc8 in ulocimp_minimizeSubtags_67 (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x7dabc8)
68 | #6 0x7fb245545ccc in icu_67::Locale::minimizeSubtags(UErrorCode&) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicuuc.so.67+0x7aeccc)
69 | #7 0x7fb246ba19c9 in icu_67::Calendar::setWeekData(icu_67::Locale const&, char const*, UErrorCode&) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicui18n.so.67+0xba09c9)
70 | #8 0x7fb246ba46ad in icu_67::Calendar::Calendar(icu_67::TimeZone const&, icu_67::Locale const&, UErrorCode&) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicui18n.so.67+0xba36ad)
71 | #9 0x7fb246bd0426 in icu_67::GregorianCalendar::GregorianCalendar(icu_67::TimeZone const&, UErrorCode&) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/installed/asan/lib/libicui18n.so.67+0xbcf426)
72 | #10 0x7fb248cd1a62 in yb::(anonymous namespace)::CreateCalendar() /nfusr/centos-gcp-cloud/jenkins-worker-335/jenkins/jenkins-github-yugabyte-db-phabricator-64408/build/asan-clang-dynamic-ninja/../../src/yb/util/date_time.cc:82:21
73 | #11 0x7fb248cd1034 in yb::DateTime::DateFromString(string const&) /nfusr/centos-gcp-cloud/jenkins-worker-335/jenkins/jenkins-github-yugabyte-db-phabricator-64408/build/asan-clang-dynamic-ninja/../../src/yb/util/date_time.cc:233:27
74 | #12 0x55d335 in yb::bfql::BfqlTest_TestBuiltinToJson_Test::TestBody() /nfusr/centos-gcp-cloud/jenkins-worker-335/jenkins/jenkins-github-yugabyte-db-phabricator-64408/build/asan-clang-dynamic-ninja/../../src/yb/util/bfql/bfql-test.cc:584:27
75 | #13 0x7fb24b99b421 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2402:10
76 | #14 0x7fb24b99b421 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2438
77 | #15 0x7fb24b9143db in testing::Test::Run() /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2474:5
78 | #16 0x7fb24b917189 in testing::TestInfo::Run() /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2656:11
79 | #17 0x7fb24b919c07 in testing::TestCase::Run() /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2774:28
80 | #18 0x7fb24b93f817 in testing::internal::UnitTestImpl::RunAllTests() /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:4649:43
81 | #19 0x7fb24b9a40fc in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2402:10
82 | #20 0x7fb24b9a40fc in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:2438
83 | #21 0x7fb24b93e48b in testing::UnitTest::Run() /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20201111062958-7047513803-centos7-linuxbrew/src/gmock-1.8.0/googletest/src/gtest.cc:4257:10
84 | #22 0x7fb24ccf292b in main /nfusr/centos-gcp-cloud/jenkins-worker-335/jenkins/jenkins-github-yugabyte-db-phabricator-64408/build/asan-clang-dynamic-ninja/../../src/yb/util/test_main.cc:104:13
85 | #23 0x7fb23ed38824 in __libc_start_main /tmp/glibc-20181130-26094-cs1x60/glibc-2.23/csu/../csu/libc-start.c:289
86 | #24 0x4388b8 in _start /tmp/glibc-20181130-26094-cs1x60/glibc-2.23/csu/../sysdeps/x86_64/start.S:118
87 |  
88 | SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior uloc.cpp:1254:5 in
@mbautin mbautin self-assigned this Dec 5, 2020
@tedyu
Copy link
Contributor

tedyu commented Jan 16, 2021

Patch is pending : https://unicode-org.atlassian.net/browse/ICU-21118

Hopefully there would be a new release soon.

@mbautin
Copy link
Contributor Author

mbautin commented Apr 1, 2021

Filed #7900 to upgrade ICU

@bmatican
Copy link
Contributor

bmatican commented Mar 3, 2022

@tedyu think you can followup and also upgrade ICU to 69.1, then close out both this and #7900?

nathanhjli added a commit that referenced this issue Apr 14, 2022
@nathanhjli
[#7854] YSQL: Updating thirdparty_archives version to include krb5
686837e 
Summary:
To support GSSAPI/Kerberos authentication, ran build_thirdparty tool to update the thirdparty_archives.yml file.

The new releases referenced in `thirdparty_archives.yml` contain two new `yugabyte-db-thirdparty` changes:
- Adding krb5 (yugabyte/yugabyte-db-thirdparty@e488f7f)
- Upgrade Icu4c to 70.1 (yugabyte/yugabyte-db-thirdparty@3c175c3)

Since the Icu4c version upgrade is packed into this new release too, I will resolve the version change issues in this diff. Icu4c was updated due to undefined behaviour in previous icu4c version (#6560). These include:
- Changing TRUE/FALSE for UBool to using standard true/false
- Adding a new migration to fix collations, since there are new collations added and some old collations removed with the version upgrade.

Test Plan: Jenkins

Reviewers: pjain, mbautin, zyu, myang, sergei, alex

Reviewed By: mbautin, myang, sergei, alex

Subscribers: amartsinchyk, sergei, yql

Differential Revision: https://phabricator.dev.yugabyte.com/D16239
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mbautin mbautin

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tedyu @bmatican @mbautin