SNOW-644849: Add telemetry about imported pacakages at runtime

[sfc-gh-jdu]

Please answer these questions before submitting your pull requests. Thanks!

1. What GitHub issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

   SNOW-644849

2. Fill out the following pre-review checklist:

   • I am adding a new automated test(s) to verify correctness of my new code
   • I am adding new logging messages
   • I am adding a new telemetry message
   • I am modifying authorization mechanisms
   • I am adding new credentials
   • I am modifying OCSP code
   • I am adding a new dependency

3. Please describe how your code solves the related issue.

   We would like to know what packages are imported at Python runtime along with connector, which can help us understand the user scenarios of python connector. The message will looks like this:

{
  "source": "PythonSnowpark",
  "type": "client_imported_packages",
  "value": "{'pdb', 'bz2', 'abc', 'ssl', 'faulthandler', 'fnmatch', 'pytest', 'shlex', 'fractions', 'enum', 'queue', 'certifi', 'traceback', 'pprint', 'pwd', 'pickle', 'ast', 'uu', 'pkg_resources', 'codecs', 'gc', 'hmac', 'copy', 'errno', 'copyreg', 'subprocess', 'snowflake', 'weakref', 'decimal', 'logging', 'dataclasses', 'typing', 'numpy', 'pytest_forked', 'ipaddress', 'operator', 'signal', 'cmd', 'email', 'gettext', 'datetime', 'importlib', 'functools', 'base64', 'calendar', 'configparser', 'codeop', 'tempfile', 'cryptography', 'distutils', 'dis', 'contextvars', 'linecache', 'uuid', 'contextlib', 'posixpath', 'unittest', 'Cryptodome', 'selectors', 'iniconfig', 'binascii', 'locale', 'grp', 'select', 'encodings', 'posix', 'plistlib', 'sre_compile', 'platform', 'pycparser', 'ntpath', 'time', 'itertools', 'textwrap', 'csv', 'zlib', 'unicodedata', 'token', 'sys', 'struct', 'reprlib', 'keyword', 'charset_normalizer', 'sysconfig', 'cffi', 'attr', 'socket', 'pyexpat', 'numbers', 'jwt', 'teamcity', 'pathlib', 'random', 'opcode', 'py', 'json', 'threading', 'zipfile', 'ctypes', 'math', 'code', 'gzip', 'marshal', 'collections', 'genericpath', 'os', 'mimetypes', 'pytz', 'xml', 'concurrent', 'bdb', 'string', 'difflib', 'argparse', 'quopri', 'http', 'inspect', 'warnings', 'pluggy', 'hashlib', 'urllib', 'zipimport', 'OpenSSL', 'bisect', 'site', 'glob', 'stat', 'html', 'multiprocessing', 'atexit', 'io', 'secrets', 'sre_constants', 'lzma', 'heapq', 'tokenize', 'pkgutil', 'sre_parse', 'shutil', 'tomli', 'webbrowser', 'asyncio', 'asn1crypto', 'stringprep', 'test', 're', 'builtins', 'cython_runtime', 'types', 'array'}"
}

[codecov-commenter]

Codecov Report

Merging #1236 (b7719d2) into main (2c3f6d5) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #1236      +/-   ##
==========================================
+ Coverage   81.63%   81.65%   +0.01%     
==========================================
  Files          59       59              
  Lines        8522     8531       +9     
  Branches     1379     1382       +3     
==========================================
+ Hits         6957     6966       +9     
- Misses       1258     1259       +1     
+ Partials      307      306       -1     

Impacted Files	Coverage Δ
src/snowflake/connector/connection.py	90.78% <100.00%> (+0.27%)	⬆️
src/snowflake/connector/telemetry.py	92.75% <100.00%> (+0.05%)	⬆️
src/snowflake/connector/errors.py	89.78% <0.00%> (-0.54%)	⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[sfc-gh-sfan]

I kinda agree this is a little bit too hacky...(scratched my comment about sp as I thought this is snowpark by mistake).

[sfc-gh-jdu]

I kinda agree this is a little bit too hacky...(scratched my comment about sp as I thought this is snowpark by mistake).

It will not work in SP (similar to snowflakedb/snowpark-python#274), and I don't expect it works in SP because per our discussion, we can just collect pakcages information of a SP from snowhouse. Though it's pretty hacky, it still works (you can find in the test) and I'm just looking for a better way to do that...

[sfc-gh-jdu]

I changed to use sys.modules that we can capture the usages of packages other than pandas. Note that as long as pandas exists in the environment, it will be pulled into sys.modules. But our goal is actually to get some information about what packages are used along with connector (or snowpark) and pandas. We are not able to get pandas (also pyarrow) usage in this telemetry records.

[sfc-gh-kwagner]

LGTM

[sfc-gh-mkeller]

🚢 🚢