Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ingesting data for crypto assets yields UserWarning: Ignoring values because they are out of bounds for uint32 #216

Open
phelps-sg opened this issue Sep 16, 2023 · 1 comment
Open

Ingesting data for crypto assets yields UserWarning: Ignoring values because they are out of bounds for uint32 #216

phelps-sg opened this issue Sep 16, 2023 · 1 comment

Comments

@phelps-sg
Copy link

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

  • Operating System: (Windows Version or $ uname --all)
    Linux pop-os 6.4.6-76060406-generic #202307241739169271764522.04~5597803 SMP PREEMPT_DYNAMIC Tue A x86_64 x86_64 x86_64 GNU/Linux
  • Python Version: $ python --version
    Python 3.10.12
  • Python Bitness: $ python -c 'import math, sys;print(int(math.log(sys.maxsize + 1, 2) + 1))'
    64
  • How did you install Zipline: (pip, conda, or other (please explain))
    mamba
  • Python packages: $ pip freeze or $ conda list
    _libgcc_mutex 0.1 conda_forge conda-forge
    _openmp_mutex 4.5 2_gnu conda-forge
    aiofiles 0.8.0 pyhd8ed1ab_0 conda-forge
    aiohttp 3.8.1 py310h5764c6d_1 conda-forge
    aiohttp-cors 0.7.0 py_0 conda-forge
    aiosignal 1.3.1 pyhd8ed1ab_0 conda-forge
    alembic 1.12.0 pyhd8ed1ab_0 conda-forge
    alsa-lib 1.2.10 hd590300_0 conda-forge
    anaconda-client 1.12.0 pyhd8ed1ab_1 conda-forge
    anaconda-project 0.11.1 pyhd8ed1ab_0 conda-forge
    aniso8601 9.0.1 pyhd8ed1ab_0 conda-forge
    anyio 4.0.0 pyhd8ed1ab_0 conda-forge
    appdirs 1.4.4 pyh9f0ad1d_0 conda-forge
    argon2-cffi 23.1.0 pyhd8ed1ab_0 conda-forge
    argon2-cffi-bindings 21.2.0 py310h5764c6d_3 conda-forge
    arrow 1.2.3 pyhd8ed1ab_0 conda-forge
    astroid 2.15.6 py310hff52083_0 conda-forge
    asttokens 2.4.0 pyhd8ed1ab_0 conda-forge
    async-lru 2.0.4 pyhd8ed1ab_0 conda-forge
    async-timeout 4.0.3 pyhd8ed1ab_0 conda-forge
    attr 2.5.1 h166bdaf_1 conda-forge
    attrs 23.1.0 pyh71513ae_1 conda-forge
    babel 2.12.1 pyhd8ed1ab_1 conda-forge
    backcall 0.2.0 pyh9f0ad1d_0 conda-forge
    backports 1.0 pyhd8ed1ab_3 conda-forge
    backports.functools_lru_cache 1.6.5 pyhd8ed1ab_0 conda-forge
    bcolz-zipline 1.2.6 py310h0a54255_0 conda-forge
    beautifulsoup4 4.12.2 pyha770c72_0 conda-forge
    black 23.9.1 py310hff52083_0 conda-forge
    blackd 23.9.1 hd8ed1ab_0 conda-forge
    bleach 6.0.0 pyhd8ed1ab_0 conda-forge
    blessed 1.19.1 pyhe4f9e05_2 conda-forge
    blinker 1.6.2 pyhd8ed1ab_0 conda-forge
    blosc 1.21.5 h0f2a231_0 conda-forge
    bottleneck 1.3.7 py310h0a54255_0 conda-forge
    brotli 1.1.0 hd590300_0 conda-forge
    brotli-bin 1.1.0 hd590300_0 conda-forge
    brotlipy 0.7.0 py310h5764c6d_1005 conda-forge
    bzip2 1.0.8 h7f98852_4 conda-forge
    c-ares 1.19.1 hd590300_0 conda-forge
    c-blosc2 2.10.2 hb4ffafa_0 conda-forge
    ca-certificates 2023.7.22 hbcca054_0 conda-forge
    cached-property 1.5.2 hd8ed1ab_1 conda-forge
    cached_property 1.5.2 pyha770c72_1 conda-forge
    cachelib 0.9.0 pyhd8ed1ab_0 conda-forge
    cachetools 5.3.1 pyhd8ed1ab_0 conda-forge
    cairo 1.16.0 h0c91306_1017 conda-forge
    certifi 2023.7.22 pyhd8ed1ab_0 conda-forge
    cffi 1.15.1 py310h255011f_3 conda-forge
    cfgv 3.3.1 pyhd8ed1ab_0 conda-forge
    charset-normalizer 2.1.1 pyhd8ed1ab_0 conda-forge
    click 8.1.7 unix_pyh707e725_0 conda-forge
    clyent 1.2.2 py_1 conda-forge
    colorama 0.4.6 pyhd8ed1ab_0 conda-forge
    colorful 0.5.4 pyhd8ed1ab_0 conda-forge
    comm 0.1.4 pyhd8ed1ab_0 conda-forge
    conda-pack 0.7.1 pyhd8ed1ab_0 conda-forge
    conda-package-handling 2.2.0 pyh38be061_0 conda-forge
    conda-package-streaming 0.9.0 pyhd8ed1ab_0 conda-forge
    contourpy 1.1.0 py310hd41b1e2_0 conda-forge
    cryptography 41.0.3 py310h75e40e8_0 conda-forge
    cycler 0.11.0 pyhd8ed1ab_0 conda-forge
    dbus 1.13.6 h5008d03_3 conda-forge
    debugpy 1.8.0 py310hc6cd4ac_0 conda-forge
    decorator 5.1.1 pyhd8ed1ab_0 conda-forge
    defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
    dill 0.3.7 pyhd8ed1ab_0 conda-forge
    distlib 0.3.7 pyhd8ed1ab_0 conda-forge
    dnspython 2.4.2 pyhd8ed1ab_0 conda-forge
    docker-pycreds 0.4.0 py_0 conda-forge
    dominate 2.8.0 pyhd8ed1ab_0 conda-forge
    email-validator 2.0.0.post2 pyhd8ed1ab_0 conda-forge
    email_validator 2.0.0.post2 hd8ed1ab_0 conda-forge
    empyrical-reloaded 0.5.9 pyhd8ed1ab_0 conda-forge
    entrypoints 0.4 pyhd8ed1ab_0 conda-forge
    et_xmlfile 1.1.0 pyhd8ed1ab_0 conda-forge
    exceptiongroup 1.1.3 pyhd8ed1ab_0 conda-forge
    exchange-calendars 4.2.8 pyhd8ed1ab_1 conda-forge
    executing 1.2.0 pyhd8ed1ab_0 conda-forge
    expat 2.5.0 hcb278e6_1 conda-forge
    filelock 3.12.4 pyhd8ed1ab_0 conda-forge
    flask 2.3.3 pyhd8ed1ab_0 conda-forge
    flask-bootstrap 3.3.7.1 py_0 conda-forge
    flask-caching 2.0.2 pyhd8ed1ab_0 conda-forge
    flask-jwt-extended 4.5.2 pyhd8ed1ab_0 conda-forge
    flask-migrate 4.0.5 pyhd8ed1ab_0 conda-forge
    flask-restful 0.3.10 pyhd8ed1ab_0 conda-forge
    flask-sqlalchemy 3.0.3 pyhd8ed1ab_0 conda-forge
    flask-wtf 1.1.1 pyhd8ed1ab_0 conda-forge
    fn 0.4.3 pyhd8ed1ab_0 conda-forge
    font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
    font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
    font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
    font-ttf-ubuntu 0.83 hab24e00_0 conda-forge
    fontconfig 2.14.2 h14ed4e7_0 conda-forge
    fonts-conda-ecosystem 1 0 conda-forge
    fonts-conda-forge 1 0 conda-forge
    fonttools 4.42.1 py310h2372a71_0 conda-forge
    fqdn 1.5.1 pyhd8ed1ab_0 conda-forge
    freetype 2.12.1 h267a509_2 conda-forge
    frozendict 2.3.8 py310h2372a71_0 conda-forge
    frozenlist 1.4.0 py310h2372a71_0 conda-forge
    gettext 0.21.1 h27087fc_0 conda-forge
    gitdb 4.0.10 pyhd8ed1ab_0 conda-forge
    gitpython 3.1.36 pyhd8ed1ab_0 conda-forge
    glib 2.78.0 hfc55251_0 conda-forge
    glib-tools 2.78.0 hfc55251_0 conda-forge
    gmp 6.2.1 h58526e2_0 conda-forge
    google-api-core 2.11.1 pyhd8ed1ab_0 conda-forge
    google-auth 2.23.0 pyh1a96a4e_0 conda-forge
    googleapis-common-protos 1.60.0 pyhd8ed1ab_0 conda-forge
    gpustat 1.1.1 pyhd8ed1ab_0 conda-forge
    graphite2 1.3.13 h58526e2_1001 conda-forge
    greenlet 2.0.2 py310hc6cd4ac_1 conda-forge
    grpcio 1.43.0 py310ha6b4e5e_0 conda-forge
    gst-plugins-base 1.22.5 h8e1006c_1 conda-forge
    gstreamer 1.22.5 h98fc4e7_1 conda-forge
    gunicorn 21.2.0 py310hff52083_0 conda-forge
    h11 0.14.0 pyhd8ed1ab_0 conda-forge
    h2 4.1.0 pyhd8ed1ab_0 conda-forge
    h5py 3.9.0 nompi_py310ha2ad45a_102 conda-forge
    harfbuzz 8.2.0 h3d44ed6_0 conda-forge
    hdf5 1.14.2 nompi_h4f84152_100 conda-forge
    hpack 4.0.0 pyh9f0ad1d_0 conda-forge
    html5lib 1.1 pyh9f0ad1d_0 conda-forge
    httpcore 0.18.0 pyhd8ed1ab_0 conda-forge
    hyperframe 6.0.1 pyhd8ed1ab_0 conda-forge
    icu 73.2 h59595ed_0 conda-forge
    identify 2.5.29 pyhd8ed1ab_0 conda-forge
    idna 3.4 pyhd8ed1ab_0 conda-forge
    importlib-metadata 6.8.0 pyha770c72_0 conda-forge
    importlib_metadata 6.8.0 hd8ed1ab_0 conda-forge
    importlib_resources 6.0.1 pyhd8ed1ab_0 conda-forge
    iniconfig 2.0.0 pyhd8ed1ab_0 conda-forge
    intervaltree 3.1.0 pyhd8ed1ab_1 conda-forge
    ipykernel 6.25.2 pyh2140261_0 conda-forge
    ipython 8.15.0 pyh0d859eb_0 conda-forge
    ipython_genutils 0.2.0 py_1 conda-forge
    ipywidgets 8.1.1 pyhd8ed1ab_0 conda-forge
    iso3166 2.1.1 pyhd8ed1ab_0 conda-forge
    iso4217 1.9.20220401 pyhd8ed1ab_0 conda-forge
    isoduration 20.11.0 pyhd8ed1ab_0 conda-forge
    isort 5.12.0 pyhd8ed1ab_1 conda-forge
    itsdangerous 2.1.2 pyhd8ed1ab_0 conda-forge
    jedi 0.19.0 pyhd8ed1ab_0 conda-forge
    jinja2 3.1.2 pyhd8ed1ab_1 conda-forge
    joblib 1.3.2 pyhd8ed1ab_0 conda-forge
    json5 0.9.14 pyhd8ed1ab_0 conda-forge
    jsonpointer 2.4 py310hff52083_2 conda-forge
    jsonschema 4.19.0 pyhd8ed1ab_1 conda-forge
    jsonschema-specifications 2023.7.1 pyhd8ed1ab_0 conda-forge
    jsonschema-with-format-nongpl 4.19.0 pyhd8ed1ab_1 conda-forge
    jupyter 1.0.0 py310hff52083_8 conda-forge
    jupyter-lsp 2.2.0 pyhd8ed1ab_0 conda-forge
    jupyter_client 8.3.1 pyhd8ed1ab_0 conda-forge
    jupyter_console 6.6.3 pyhd8ed1ab_0 conda-forge
    jupyter_core 5.3.1 py310hff52083_0 conda-forge
    jupyter_events 0.7.0 pyhd8ed1ab_2 conda-forge
    jupyter_server 2.7.3 pyhd8ed1ab_0 conda-forge
    jupyter_server_terminals 0.4.4 pyhd8ed1ab_1 conda-forge
    jupyterlab 4.0.6 pyhd8ed1ab_0 conda-forge
    jupyterlab_pygments 0.2.2 pyhd8ed1ab_0 conda-forge
    jupyterlab_server 2.25.0 pyhd8ed1ab_0 conda-forge
    jupyterlab_widgets 3.0.9 pyhd8ed1ab_0 conda-forge
    keyutils 1.6.1 h166bdaf_0 conda-forge
    kiwisolver 1.4.5 py310hd41b1e2_0 conda-forge
    korean_lunar_calendar 0.3.1 pyhd8ed1ab_0 conda-forge
    krb5 1.21.2 h659d440_0 conda-forge
    lame 3.100 h166bdaf_1003 conda-forge
    lazy-object-proxy 1.9.0 py310h1fa729e_0 conda-forge
    lcms2 2.15 h7f713cb_2 conda-forge
    ld_impl_linux-64 2.40 h41732ed_0 conda-forge
    lerc 4.0.0 h27087fc_0 conda-forge
    libabseil 20230802.0 cxx17_h59595ed_3 conda-forge
    libaec 1.0.6 hcb278e6_1 conda-forge
    libblas 3.9.0 18_linux64_openblas conda-forge
    libbrotlicommon 1.1.0 hd590300_0 conda-forge
    libbrotlidec 1.1.0 hd590300_0 conda-forge
    libbrotlienc 1.1.0 hd590300_0 conda-forge
    libcap 2.69 h0f662aa_0 conda-forge
    libcblas 3.9.0 18_linux64_openblas conda-forge
    libclang 15.0.7 default_h7634d5b_3 conda-forge
    libclang13 15.0.7 default_h9986a30_3 conda-forge
    libcups 2.3.3 h4637d8d_4 conda-forge
    libcurl 8.3.0 hca28451_0 conda-forge
    libdeflate 1.18 h0b41bf4_0 conda-forge
    libedit 3.1.20191231 he28a2e2_2 conda-forge
    libev 4.33 h516909a_1 conda-forge
    libevent 2.1.12 hf998b51_1 conda-forge
    libexpat 2.5.0 hcb278e6_1 conda-forge
    libffi 3.4.2 h7f98852_5 conda-forge
    libflac 1.4.3 h59595ed_0 conda-forge
    libgcc-ng 13.2.0 h807b86a_1 conda-forge
    libgcrypt 1.10.1 h166bdaf_0 conda-forge
    libgfortran-ng 13.2.0 h69a702a_1 conda-forge
    libgfortran5 13.2.0 ha4646dd_1 conda-forge
    libglib 2.78.0 hebfc3b9_0 conda-forge
    libgomp 13.2.0 h807b86a_1 conda-forge
    libgpg-error 1.47 h71f35ed_0 conda-forge
    libiconv 1.17 h166bdaf_0 conda-forge
    libjpeg-turbo 2.1.5.1 hd590300_1 conda-forge
    liblapack 3.9.0 18_linux64_openblas conda-forge
    libllvm15 15.0.7 h5cf9203_3 conda-forge
    libnghttp2 1.52.0 h61bc06f_0 conda-forge
    libnsl 2.0.0 h7f98852_0 conda-forge
    libogg 1.3.4 h7f98852_1 conda-forge
    libopenblas 0.3.24 pthreads_h413a1c8_0 conda-forge
    libopus 1.3.1 h7f98852_1 conda-forge
    libpng 1.6.39 h753d276_0 conda-forge
    libpq 15.4 hfc447b1_0 conda-forge
    libprotobuf 4.23.4 hf27288f_6 conda-forge
    libsndfile 1.2.2 hbc2eb40_0 conda-forge
    libsodium 1.0.18 h36c2ea0_1 conda-forge
    libsqlite 3.43.0 h2797004_0 conda-forge
    libssh2 1.11.0 h0841786_0 conda-forge
    libstdcxx-ng 13.2.0 h7e041cc_1 conda-forge
    libsystemd0 254 h3516f8a_0 conda-forge
    libta-lib 0.4.0 h166bdaf_1 conda-forge
    libtiff 4.6.0 h8b53f26_0 conda-forge
    libunwind 1.6.2 h9c3ff4c_0 conda-forge
    libuuid 2.38.1 h0b41bf4_0 conda-forge
    libuv 1.46.0 hd590300_0 conda-forge
    libvorbis 1.3.7 h9c3ff4c_0 conda-forge
    libwebp-base 1.3.2 hd590300_0 conda-forge
    libxcb 1.15 h0b41bf4_0 conda-forge
    libxkbcommon 1.5.0 h5d7e998_3 conda-forge
    libxml2 2.11.5 h232c23b_1 conda-forge
    libxslt 1.1.37 h0054252_1 conda-forge
    libzlib 1.2.13 hd590300_5 conda-forge
    lru-dict 1.2.0 py310h2372a71_0 conda-forge
    lxml 4.9.3 py310h9b7343a_0 conda-forge
    lz4-c 1.9.4 hcb278e6_0 conda-forge
    lzo 2.10 h516909a_1000 conda-forge
    mako 1.2.4 pyhd8ed1ab_0 conda-forge
    markupsafe 2.1.3 py310h2372a71_0 conda-forge
    matplotlib-base 3.7.2 py310hf38f957_0 conda-forge
    matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
    mccabe 0.7.0 pyhd8ed1ab_0 conda-forge
    mistune 3.0.1 pyhd8ed1ab_0 conda-forge
    mpg123 1.31.3 hcb278e6_0 conda-forge
    msgpack-python 1.0.5 py310hdf3cbec_0 conda-forge
    multidict 6.0.4 py310h1fa729e_0 conda-forge
    multipledispatch 0.6.0 py_0 conda-forge
    multitasking 0.0.9 pyhd8ed1ab_0 conda-forge
    munkres 1.1.4 pyh9f0ad1d_0 conda-forge
    mypy 1.5.1 py310h2372a71_0 conda-forge
    mypy_extensions 1.0.0 pyha770c72_0 conda-forge
    mysql-common 8.0.33 hf1915f5_4 conda-forge
    mysql-libs 8.0.33 hca2cd23_4 conda-forge
    nbclient 0.8.0 pyhd8ed1ab_0 conda-forge
    nbconvert 7.8.0 pyhd8ed1ab_0 conda-forge
    nbconvert-core 7.8.0 pyhd8ed1ab_0 conda-forge
    nbconvert-pandoc 7.8.0 pyhd8ed1ab_0 conda-forge
    nbformat 5.9.2 pyhd8ed1ab_0 conda-forge
    ncurses 6.4 hcb278e6_0 conda-forge
    nest-asyncio 1.5.6 pyhd8ed1ab_0 conda-forge
    networkx 3.1 pyhd8ed1ab_0 conda-forge
    nodeenv 1.8.0 pyhd8ed1ab_0 conda-forge
    nodejs 20.6.1 hb753e55_0 conda-forge
    nomkl 1.0 h5ca1d4c_0 conda-forge
    notebook 7.0.3 pyhd8ed1ab_0 conda-forge
    notebook-shim 0.2.3 pyhd8ed1ab_0 conda-forge
    nspr 4.35 h27087fc_0 conda-forge
    nss 3.92 h1d7d5a4_0 conda-forge
    numexpr 2.8.4 py310hd91493a_101 conda-forge
    numpy 1.25.2 py310ha4c1d20_0 conda-forge
    nvidia-ml-py 12.535.108 pyhd8ed1ab_0 conda-forge
    openai 0.28.0 pyhd8ed1ab_0 conda-forge
    opencensus 0.11.2 pyhd8ed1ab_0 conda-forge
    opencensus-context 0.1.3 py310hff52083_1 conda-forge
    openjpeg 2.5.0 h488ebb8_3 conda-forge
    openpyxl 3.1.2 py310h2372a71_0 conda-forge
    openssl 3.1.2 hd590300_0 conda-forge
    overrides 7.4.0 pyhd8ed1ab_0 conda-forge
    packaging 23.1 pyhd8ed1ab_0 conda-forge
    pandas 2.1.0 py310hcc13569_0 conda-forge
    pandas-datareader 0.10.0 pyh6c4a22f_0 conda-forge
    pandas-stubs 2.0.3.230814 pyhd8ed1ab_0 conda-forge
    pandoc 3.1.3 h32600fe_0 conda-forge
    pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge
    parso 0.8.3 pyhd8ed1ab_0 conda-forge
    pathspec 0.11.2 pyhd8ed1ab_0 conda-forge
    pathtools 0.1.2 py_1 conda-forge
    patsy 0.5.3 pyhd8ed1ab_0 conda-forge
    pcre2 10.40 hc3806b6_0 conda-forge
    pexpect 4.8.0 pyh1a96a4e_2 conda-forge
    pickleshare 0.7.5 py_1003 conda-forge
    pillow 10.0.1 py310h29da1c1_0 conda-forge
    pip 23.2.1 pyhd8ed1ab_0 conda-forge
    pixman 0.40.0 h36c2ea0_0 conda-forge
    pkgutil-resolve-name 1.3.10 pyhd8ed1ab_1 conda-forge
    platformdirs 3.10.0 pyhd8ed1ab_0 conda-forge
    plotly 5.17.0 pyhd8ed1ab_0 conda-forge
    pluggy 1.3.0 pyhd8ed1ab_0 conda-forge
    ply 3.11 py_1 conda-forge
    pre-commit 3.4.0 pyha770c72_1 conda-forge
    prometheus_client 0.17.1 pyhd8ed1ab_0 conda-forge
    prompt-toolkit 3.0.39 pyha770c72_0 conda-forge
    prompt_toolkit 3.0.39 hd8ed1ab_0 conda-forge
    protobuf 4.23.4 py310h620c231_2 conda-forge
    psutil 5.9.5 py310h1fa729e_0 conda-forge
    pthread-stubs 0.4 h36c2ea0_1001 conda-forge
    ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
    pulseaudio-client 16.1 hb77b528_5 conda-forge
    pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
    py-cpuinfo 9.0.0 pyhd8ed1ab_0 conda-forge
    py-spy 0.3.14 h87a5ac0_0 conda-forge
    pyasn1 0.4.8 py_0 conda-forge
    pyasn1-modules 0.2.7 py_0 conda-forge
    pycparser 2.21 pyhd8ed1ab_0 conda-forge
    pydantic 1.10.12 py310h2372a71_1 conda-forge
    pygments 2.16.1 pyhd8ed1ab_0 conda-forge
    pyjwt 2.8.0 pyhd8ed1ab_0 conda-forge
    pylint 2.17.5 pyhd8ed1ab_0 conda-forge
    pyluach 2.2.0 pyhd8ed1ab_0 conda-forge
    pyopenssl 23.2.0 pyhd8ed1ab_1 conda-forge
    pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge
    pyqt 5.15.9 py310h04931ad_4 conda-forge
    pyqt5-sip 12.12.2 py310hc6cd4ac_4 conda-forge
    pysocks 1.7.1 pyha2e5f31_6 conda-forge
    pytables 3.8.0 py310h374b01c_3 conda-forge
    pytest 7.4.2 pyhd8ed1ab_0 conda-forge
    pytest-mock 3.11.1 pyhd8ed1ab_0 conda-forge
    python 3.10.12 hd12c33a_0_cpython conda-forge
    python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
    python-fastjsonschema 2.18.0 pyhd8ed1ab_0 conda-forge
    python-interface 1.6.0 py_0 conda-forge
    python-json-logger 2.0.7 pyhd8ed1ab_0 conda-forge
    python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge
    python_abi 3.10 3_cp310 conda-forge
    pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge
    pyu2f 0.1.5 pyhd8ed1ab_0 conda-forge
    pyyaml 6.0.1 py310h2372a71_0 conda-forge
    pyzmq 25.1.1 py310h5bbb5d0_0 conda-forge
    qt-main 5.15.8 hc47bfe8_16 conda-forge
    qtconsole 5.4.4 pyhd8ed1ab_0 conda-forge
    qtconsole-base 5.4.4 pyha770c72_0 conda-forge
    qtpy 2.4.0 pyhd8ed1ab_0 conda-forge
    ray-core 2.6.3 py310h2ca9b2b_1 conda-forge
    ray-dashboard 2.6.3 py310ha1e06c9_1 conda-forge
    ray-default 2.6.3 py310hff52083_1 conda-forge
    readline 8.2 h8228510_1 conda-forge
    referencing 0.30.2 pyhd8ed1ab_0 conda-forge
    requests 2.31.0 pyhd8ed1ab_0 conda-forge
    requests-toolbelt 1.0.0 pyhd8ed1ab_0 conda-forge
    rfc3339-validator 0.1.4 pyhd8ed1ab_0 conda-forge
    rfc3986-validator 0.1.1 pyh9f0ad1d_0 conda-forge
    rpds-py 0.10.3 py310hcb5633a_0 conda-forge
    rsa 4.9 pyhd8ed1ab_0 conda-forge
    ruamel_yaml 0.15.80 py310h5764c6d_1008 conda-forge
    scikit-learn 1.3.0 py310hf7d194e_0 conda-forge
    scipy 1.11.2 py310hb13e2d6_1 conda-forge
    send2trash 1.8.2 pyh41d4057_0 conda-forge
    sentry-sdk 1.31.0 pyhd8ed1ab_0 conda-forge
    setproctitle 1.2.2 py310h5764c6d_2 conda-forge
    setuptools 68.2.2 pyhd8ed1ab_0 conda-forge
    sip 6.7.11 py310hc6cd4ac_0 conda-forge
    six 1.16.0 pyh6c4a22f_0 conda-forge
    smart_open 6.4.0 pyhd8ed1ab_0 conda-forge
    smmap 3.0.5 pyh44b312d_0 conda-forge
    snappy 1.1.10 h9fff704_0 conda-forge
    sniffio 1.3.0 pyhd8ed1ab_0 conda-forge
    sortedcontainers 2.1.0 py_0 conda-forge
    soupsieve 2.5 pyhd8ed1ab_1 conda-forge
    sqlalchemy 2.0.20 py310h2372a71_0 conda-forge
    stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
    statsmodels 0.14.0 py310h278f3c1_1 conda-forge
    ta-lib 0.4.19 py310h278f3c1_4 conda-forge
    tardis-client 1.3.4 py310_0 mesonomics
    tardis-dev 2.0.0a13 py310_0 mesonomics
    tenacity 8.2.3 pyhd8ed1ab_0 conda-forge
    terminado 0.17.1 pyh41d4057_0 conda-forge
    threadpoolctl 3.2.0 pyha21a80b_0 conda-forge
    tinycss2 1.2.1 pyhd8ed1ab_0 conda-forge
    tk 8.6.12 h27826a3_0 conda-forge
    toml 0.10.2 pyhd8ed1ab_0 conda-forge
    tomli 2.0.1 pyhd8ed1ab_0 conda-forge
    tomlkit 0.12.1 pyha770c72_0 conda-forge
    toolz 0.12.0 pyhd8ed1ab_0 conda-forge
    tornado 6.3.3 py310h2372a71_0 conda-forge
    tqdm 4.66.1 pyhd8ed1ab_0 conda-forge
    traitlets 5.10.0 pyhd8ed1ab_0 conda-forge
    types-click 7.1.8 pyhd8ed1ab_0 conda-forge
    types-flask 1.1.6 pyhd8ed1ab_0 conda-forge
    types-jinja2 2.11.9 pyhd8ed1ab_0 conda-forge
    types-markupsafe 1.1.10 pyhd8ed1ab_0 conda-forge
    types-pytz 2023.3.0.1 pyhd8ed1ab_0 conda-forge
    types-werkzeug 1.0.9 pyhd8ed1ab_0 conda-forge
    typing-extensions 4.7.1 hd8ed1ab_0 conda-forge
    typing_extensions 4.7.1 pyha770c72_0 conda-forge
    typing_utils 0.1.0 pyhd8ed1ab_0 conda-forge
    tzdata 2023c h71feb2d_0 conda-forge
    ukkonen 1.0.1 py310hbf28c38_3 conda-forge
    unicodedata2 15.0.0 py310h5764c6d_0 conda-forge
    uri-template 1.3.0 pyhd8ed1ab_0 conda-forge
    urllib3 1.26.15 pyhd8ed1ab_0 conda-forge
    virtualenv 20.21.0 pyhd8ed1ab_0 conda-forge
    visitor 0.1.3 pyh9f0ad1d_1 conda-forge
    wandb 0.15.10 pyhd8ed1ab_0 conda-forge
    wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge
    webcolors 1.13 pyhd8ed1ab_0 conda-forge
    webencodings 0.5.1 pyhd8ed1ab_2 conda-forge
    websocket-client 1.6.3 pyhd8ed1ab_0 conda-forge
    werkzeug 2.3.7 pyhd8ed1ab_0 conda-forge
    wheel 0.41.2 pyhd8ed1ab_0 conda-forge
    widgetsnbextension 4.0.9 pyhd8ed1ab_0 conda-forge
    wrapt 1.15.0 py310h1fa729e_0 conda-forge
    wtforms 3.0.1 pyhd8ed1ab_0 conda-forge
    xcb-util 0.4.0 hd590300_1 conda-forge
    xcb-util-image 0.4.0 h8ee46fc_1 conda-forge
    xcb-util-keysyms 0.4.0 h8ee46fc_1 conda-forge
    xcb-util-renderutil 0.3.9 hd590300_1 conda-forge
    xcb-util-wm 0.4.1 h8ee46fc_1 conda-forge
    xkeyboard-config 2.39 hd590300_0 conda-forge
    xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
    xorg-libice 1.1.1 hd590300_0 conda-forge
    xorg-libsm 1.2.4 h7391055_0 conda-forge
    xorg-libx11 1.8.6 h8ee46fc_0 conda-forge
    xorg-libxau 1.0.11 hd590300_0 conda-forge
    xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
    xorg-libxext 1.3.4 h0b41bf4_2 conda-forge
    xorg-libxrender 0.9.11 hd590300_0 conda-forge
    xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
    xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge
    xorg-xf86vidmodeproto 2.3.1 h7f98852_1002 conda-forge
    xorg-xproto 7.0.31 h7f98852_1007 conda-forge
    xz 5.2.6 h166bdaf_0 conda-forge
    yaml 0.2.5 h7f98852_2 conda-forge
    yarl 1.9.2 py310h2372a71_0 conda-forge
    yfinance 0.2.28 pyhd8ed1ab_0 conda-forge
    zeromq 4.3.4 h9c3ff4c_1 conda-forge
    zipline-reloaded 3.0.3 py310h278f3c1_0 conda-forge
    zipline-tardis-bundle 0.3.1 py310he53d0f1_0 mesonomics
    zipp 3.16.2 pyhd8ed1ab_0 conda-forge
    zlib 1.2.13 hd590300_5 conda-forge
    zlib-ng 2.0.7 h0b41bf4_0 conda-forge
    zstandard 0.19.0 py310h5764c6d_0 conda-forge
    zstd 1.5.5 hfc55251_0 conda-forge

Now that you know a little about me, let me tell you about the issue I am
having:

Description of Issue

When ingesting data for crypto pairs we often encounter large volumes which cannot be represented in uint32.

/home/sphelps/mambaforge/envs/zipline-tardis-bundle/lib/python3.10/site-packages/zipline/data/bcolz_daily_bars.py:341: UserWarning: Ignoring 422 values because they are out of bounds for uint32:                                                                                                                                                                                                                        open      high       low     close       volume
timestamp                                                                     
2022-03-30 00:00:00+00:00  0.087520  0.097370  0.084210  0.092865   9209813515
2022-03-31 00:00:00+00:00  0.092865  0.098105  0.088565  0.095280   7058370768
2022-04-01 00:00:00+00:00  0.095280  0.096950  0.088730  0.095310   7066482428
2022-04-02 00:00:00+00:00  0.095310  0.102800  0.094340  0.096920   9800596381
2022-04-03 00:00:00+00:00  0.096920  0.100835  0.094075  0.097590   6914043464
...                             ...       ...       ...       ...          ...
2023-05-27 00:00:00+00:00  0.025160  0.025505  0.025025  0.025425   5587944209
2023-05-28 00:00:00+00:00  0.025425  0.026690  0.025300  0.026265   6736129138                
2023-05-29 00:00:00+00:00  0.026265  0.027575  0.025835  0.026465  14296257656
2023-05-30 00:00:00+00:00  0.026465  0.026715  0.026115  0.026485  13632077291
2023-05-31 00:00:00+00:00  0.026485  0.026715  0.025165  0.025490  14446139237

Here is how you can reproduce this issue on your machine:

Reproduction Steps

  1. Install zipline-tardis-bundle
  2. Ingest coins with large volumes
  3. Above warning is displayed for coins with large volumes
    ...

What steps have you taken to resolve this already?

...

Anything else?

Can we use 64-bit int for representing volumes to support crypto?
...

Sincerely,
Steve Phelps
@MBounouar
Copy link

Hi,
By design zipline has put constraints on how the data is stored. Unfortunately, one would have to do the changes in the code. I am working on switching to parquet and relax some constraints, but I haven't had much time to finalize this.

@phelps-sg phelps-sg mentioned this issue Sep 16, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@phelps-sg @MBounouar