Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Try newer version of odbc driver #9484

Merged
merged 2 commits into from
Mar 10, 2020
Merged

Try newer version of odbc driver #9484

merged 2 commits into from
Mar 10, 2020

Conversation

filimonov
Copy link
Contributor

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • Non-significant (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Detailed description / Documentation draft:
Continuation of #9348 - let's try new driver build.

@filimonov
Copy link
Contributor Author 

docker run -it --rm yandex/clickhouse-stateless-test ldd /usr/local/lib/libclickhouseodbc.so
	linux-vdso.so.1 (0x00007fff35f8e000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1583a1a000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1583816000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f158360e000)
	libodbcinst.so.2 => /usr/lib/x86_64-linux-gnu/libodbcinst.so.2 (0x00007f15833f9000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f158305b000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1582c6a000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f15843ca000)
	libltdl.so.7 => /usr/lib/x86_64-linux-gnu/libltdl.so.7 (0x00007f1582a60000)


docker run -it --rm yandex/clickhouse-stateless-test nm -D /usr/local/lib/libclickhouseodbc.so
0000000000122670 T SQLAllocHandle
000000000011d700 T SQLBindCol
000000000011a020 T SQLBindParameter
000000000011c030 T SQLBrowseConnect
0000000000119ed0 T SQLBulkOperations
000000000011bf70 T SQLCancel
0000000000119d80 T SQLCancelHandle
000000000011c180 T SQLCloseCursor
000000000011dbf0 T SQLColAttribute
000000000011aeb0 T SQLColumnPrivileges
000000000011ca10 T SQLColumns
0000000000119c30 T SQLCompleteAsync
0000000000121710 T SQLConnect
0000000000118c80 T SQLCopyDesc
000000000011dac0 T SQLDescribeCol
000000000011ac30 T SQLDescribeParam
000000000011d0f0 T SQLDisconnect
0000000000121600 T SQLDriverConnect
0000000000118c70 T SQLDummyOrdinal
0000000000119930 T SQLEndTran
0000000000120ee0 T SQLExecDirect
0000000000121170 T SQLExecute
000000000011aae0 T SQLExtendedFetch
000000000011da00 T SQLFetch
000000000011d920 T SQLFetchScroll
000000000011a990 T SQLForeignKeys
0000000000122280 T SQLFreeHandle
0000000000122040 T SQLFreeStmt
0000000000121930 T SQLGetConnectAttr
000000000011be20 T SQLGetCursorName
000000000011d810 T SQLGetData
00000000001196c0 T SQLGetDescField
00000000001193d0 T SQLGetDescRec
000000000011cd00 T SQLGetDiagField
000000000011cef0 T SQLGetDiagRec
0000000000121a30 T SQLGetEnvAttr
000000000011b690 T SQLGetFunctions
0000000000121de0 T SQLGetInfo
                 U SQLGetPrivateProfileString
0000000000121830 T SQLGetStmtAttr
000000000011c7d0 T SQLGetTypeInfo
000000000011d2f0 T SQLMoreResults
000000000011c380 T SQLNativeSql
000000000011c5c0 T SQLNumParams
0000000000120e10 T SQLNumResultCols
000000000011b540 T SQLParamData
0000000000121370 T SQLPrepare
000000000011a840 T SQLPrimaryKeys
000000000011a6f0 T SQLProcedureColumns
000000000011a5a0 T SQLProcedures
000000000011b3f0 T SQLPutData
000000000011d4f0 T SQLRowCount
0000000000121c10 T SQLSetConnectAttr
000000000011b2a0 T SQLSetCursorName
0000000000119170 T SQLSetDescField
0000000000118ea0 T SQLSetDescRec
0000000000121d00 T SQLSetEnvAttr
000000000011a450 T SQLSetPos
0000000000121b20 T SQLSetStmtAttr
000000000011b150 T SQLSpecialColumns
000000000011b000 T SQLStatistics
000000000011a300 T SQLTablePrivileges
000000000011cbb0 T SQLTables
                 w _ITM_RU1
                 w _ITM_RU8
                 w _ITM_addUserCommitAction
                 w _ITM_deregisterTMCloneTable
                 w _ITM_memcpyRnWt
                 w _ITM_memcpyRtWn
                 w _ITM_registerTMCloneTable
                 w _ZGTtdlPv
                 w _ZGTtnam
                 U __ctype_b_loc
                 U __ctype_get_mb_cur_max
                 U __ctype_tolower_loc
                 U __cxa_atexit
                 w __cxa_finalize
                 U __duplocale
                 U __errno_location
                 U __freelocale
                 U __fxstat
                 U __fxstat64
                 w __gmon_start__
                 U __h_errno_location
                 U __iswctype_l
                 U __lxstat64
                 U __newlocale
                 U __nl_langinfo_l
                 w __pthread_key_create
                 U __register_atfork
                 U __stack_chk_fail
                 U __strcoll_l
                 U __strdup
                 U __strftime_l
                 U __strndup
                 U __strtod_l
                 U __strtof_l
                 U __strxfrm_l
                 U __timezone
                 U __tls_get_addr
                 U __towlower_l
                 U __towupper_l
                 U __tzname
                 U __uselocale
                 U __wcscoll_l
                 U __wcsftime_l
                 U __wcsxfrm_l
                 U __wctype_l
                 U __xpg_strerror_r
                 U __xstat
                 U __xstat64
                 U _exit
                 U abort
                 U accept
                 U asprintf
                 U bind
                 U bind_textdomain_codeset
                 U bindtextdomain
                 U btowc
                 U calloc
                 U chdir
                 U chmod
                 U clock_gettime
                 U close
                 U closedir
                 U closelog
                 U connect
                 U dgettext
                 U dl_iterate_phdr
                 U dup2
                 U epoll_create
                 U epoll_ctl
                 U epoll_wait
                 U execvp
                 U exit
                 U fclose
                 U fcntl
                 U fdopen
                 U feof
                 U ferror
                 U fflush
                 U fgetc
                 U fgets
                 U fileno
                 U fopen
                 U fopen64
                 U fork
                 U fprintf
                 U fputc
                 U fputs
                 U fread
                 U free
                 U freeaddrinfo
                 U fseek
                 U fseeko64
                 U fstatfs
                 U fstatvfs
                 U fsync
                 U ftell
                 U ftello64
                 U fwrite
                 U getaddrinfo
                 U getc
                 U getcwd
                 U getegid
                 U getenv
                 U geteuid
                 U gethostname
                 U getnameinfo
                 U getpagesize
                 U getpeername
                 U getpgid
                 U getpid
                 U getppid
                 U getpriority
                 U getpwuid
                 U getrusage
                 U getservbyname
                 U getsid
                 U getsockname
                 U getsockopt
                 U gettext
                 U gettimeofday
                 U getuid
                 U getwc
                 U gmtime_r
                 U if_indextoname
                 U if_nametoindex
                 U inet_aton
                 U inet_pton
                 U ioctl
                 U isalpha
                 U isspace
                 U kill
                 U link
                 U listen
                 U localtime
                 U localtime_r
                 U lseek
                 U lseek64
                 U malloc
                 U mbrtowc
                 U mbsnrtowcs
                 U mbsrtowcs
                 U memchr
                 U memcmp
                 U memcpy
                 U memmove
                 U memset
                 U mkdir
                 U mktime
                 U mmap
                 U munmap
                 U nanosleep
                 U open
                 U open64
                 U opendir
                 U openlog
                 U pipe
                 U poll
                 U printf
                 U pthread_attr_destroy
                 U pthread_attr_init
                 U pthread_attr_setstacksize
                 U pthread_cond_broadcast
                 U pthread_cond_destroy
                 U pthread_cond_init
                 U pthread_cond_timedwait
                 U pthread_cond_wait
                 U pthread_condattr_destroy
                 U pthread_condattr_init
                 U pthread_condattr_setclock
                 U pthread_create
                 U pthread_detach
                 U pthread_getspecific
                 U pthread_join
                 U pthread_key_create
                 U pthread_key_delete
                 U pthread_mutex_destroy
                 U pthread_mutex_init
                 U pthread_mutex_lock
                 U pthread_mutex_timedlock
                 U pthread_mutex_unlock
                 U pthread_mutexattr_destroy
                 U pthread_mutexattr_init
                 U pthread_mutexattr_settype
                 w pthread_once
                 U pthread_rwlock_destroy
                 U pthread_rwlock_init
                 U pthread_rwlock_rdlock
                 U pthread_rwlock_unlock
                 U pthread_rwlock_wrlock
                 U pthread_self
                 U pthread_setschedparam
                 U pthread_setspecific
                 U pthread_sigmask
                 U putc
                 U putenv
                 U putwc
                 U qsort
                 U raise
                 U read
                 U readdir
                 U readdir64
                 U realloc
                 U recv
                 U recvfrom
                 U rename
                 U rmdir
                 U sched_get_priority_max
                 U sched_get_priority_min
                 U sched_yield
                 U send
                 U sendto
                 U setlocale
                 U setsockopt
                 U setvbuf
                 U shutdown
                 U sigaction
                 U sigaddset
                 U sigemptyset
                 U siglongjmp
                 U signal
                 U sigpending
                 U sigprocmask
                 U snprintf
                 U socket
                 U sprintf
                 U sscanf
                 U statfs
                 U statfs64
                 U statvfs
                 U stderr
                 U stdin
                 U stdout
                 U strcasecmp
                 U strchr
                 U strcmp
                 U strcpy
                 U strdup
                 U strerror
                 U strerror_r
                 U strftime
                 U strlen
                 U strncasecmp
                 U strncmp
                 U strncpy
                 U strrchr
                 U strtod
                 U strtof
                 U strtol
                 U strtold_l
                 U strtoll
                 U strtoul
                 U strtoull
                 U symlink
                 U syscall
                 U sysconf
                 U syslog
                 U tcgetattr
                 U tcsetattr
                 U time
                 U timegm
                 U timezone
                 U toupper
                 U truncate64
                 U tzname
                 U tzset
                 U uname
                 U ungetc
                 U ungetwc
                 U unlink
                 U utime
                 U vasprintf
                 U vfprintf
                 U vsnprintf
                 U waitpid
                 U wcrtomb
                 U wcscmp
                 U wcslen
                 U wcsnrtombs
                 U wctob
                 U wmemchr
                 U wmemcmp
                 U wmemcpy
                 U wmemmove
                 U wmemset
                 U write
                 U writev

@qoega
Copy link
Member

qoega commented Mar 3, 2020

Actually it can't be tested in CI.

@filimonov
Copy link
Contributor Author

Actually it can't be tested in CI.

Docker image will be rebuild only after merge in master. Did you mean that?

@alexey-milovidov
Copy link
Member

unbundled - 501 Not Implemented.

It is expected:

#if USE_POCO_SQLODBC || USE_POCO_DATAODBC

I will add an exception.

@alexey-milovidov
Copy link
Member

Actually it can't be tested in CI.

What do you mean?

@alexey-milovidov
Copy link
Member

Address Sanitizer: No message received

It's also expected, because we dlopen third-party library that is not instrumented. But it allocates memory. This memory range is not accounted by ASan and ODBC Bridge fails.

I will add another exception.

@alexey-milovidov
Copy link
Member

Memory Sanitizer (stateless)

2020.03.03 19:43:29.982882 [ 201 ] {} <Error> Application: DB::Exception: Listen [::]:8443 failed: Code: 344, e.displayText() = DB::Exception: HTTPS protocol is disabled because Poco library was built without NetSSL support. (version 20.3.1.2552)

It is completely broken in another PR, the server doesn't start.

@alexey-milovidov
Copy link
Member

Debug, stateless

2020-03-03 19:22:05 [35ab94dc7135] 2020.03.03 19:22:05.111320 [ 215 ] {3c7ac337-8893-4279-9e81-1fdc3fe89ca5} <Error> executeQuery: Code: 86, e.displayText() = DB::Exception: Received error from remote server /columns_info?connection_string=DSN%3D%7BClickHouse%20DSN%20%28ANSI%29%7D&schema=system&table=tables&external_table_functions_use_nulls=true. HTTP status code: 500 Internal Server Error, body: Error getting columns from ODBC 'Poco::Exception. Code: 1000, e.code() = 0, e.displayText() = Connection attempt failed: Connection:Not applicable
2020-03-03 19:22:05 Server:Not applicable
2020-03-03 19:22:05 ===========================
2020-03-03 19:22:05 ODBC Diagnostic record #1:
2020-03-03 19:22:05 ===========================
2020-03-03 19:22:05 SQLSTATE = IM002
2020-03-03 19:22:05 Native Error Code = 0
2020-03-03 19:22:05 [unixODBC][Driver Manager]Data source name not found, and no default driver specified

Do not understand the reason. @filimonov Please try to reproduce.

@filimonov
Copy link
Contributor Author

filimonov commented Mar 4, 2020 •
edited
Loading

Do not understand the reason. @filimonov Please try to reproduce.

It's because "Debug, stateless" uses another docker image yandex/clickhouse-stateless-with-coverage-test which is not inherited for some reason from yandex/clickhouse-stateless-test.

We can eather copy&paste the same change there (sounds quite stupid), or make yandex/clickhouse-stateless-with-coverage-test use yandex/clickhouse-stateless-test as parent, can do that change if is't ok for you. (It will give some more advantages, like less images to download on CI).

UPD: the difference is llvm-9 vs llvm-8. @alesapin can you comment? Still it seems better to inherint from yandex/clickhouse-stateless-test uninstall unneeded llvm-8 and install needed llvm-9.

@alexey-milovidov
Copy link
Member

@filimonov We don't use system LLVM even for "unbundled" build.
https://clickhouse-builds.s3.yandex.net/9484/add201401da7ee7f90929ba383971da7bd93b0f6/build_log_622335845_1583258392.txt

Installation of LLVM in Docker images is pointless.

@alexey-milovidov
Copy link
Member

Actually it is used for symbolization of messages from sanitizers:

ln -s /usr/lib/llvm-8/bin/llvm-symbolizer /usr/bin/llvm-symbolizer

@alexey-milovidov
Copy link
Member

But we don't need it for "stateless with coverage".

@alexey-milovidov
Copy link
Member

We can eather copy&paste the same change there (sounds quite stupid), or make yandex/clickhouse-stateless-with-coverage-test use yandex/clickhouse-stateless-test as parent, can do that change if is't ok for you. (It will give some more advantages, like less images to download on CI).

Both options are Ok, let's try whatever you prefer more.

@alexey-milovidov
Copy link
Member

I have added suppression for ASan and Unbundled builds.

@alexey-milovidov alexey-milovidov added the st-waiting-for-fix We are waiting for fixes in this issue or pull request label Mar 5, 2020
@alexey-milovidov
Copy link
Member

@filimonov Copy paste is Ok, let's finish this PR.

@filimonov
Copy link
Contributor Author

Check now (not sure if the docker image will be rebuilt automatically).

@filimonov filimonov removed the st-waiting-for-fix We are waiting for fixes in this issue or pull request label Mar 9, 2020
@filimonov
Copy link
Contributor Author

Yep. It look like image yandex/clickhouse-stateless-test-with-coverage was not rebuild

@alesapin alesapin merged commit 7520d4f into ClickHouse:master Mar 10, 2020
@robot-clickhouse robot-clickhouse added the pr-not-for-changelog This PR should not be mentioned in the changelog label Mar 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
no-docs-needed pr-not-for-changelog This PR should not be mentioned in the changelog
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@filimonov @qoega @alexey-milovidov @alesapin @robot-clickhouse