MAINT: Move to exchange-calendars 4.x and more (#158)

* replace deprecated abstractproperty * removed deprecated __unicode__ * bump up pytest to >= 7.2.0 * remove unecessary object class declaration * cleanup metaclass declaration * replace logbook with logging (#16) * fix B904 flak8-bugbear warnings * replaced logbook with logging * replace logbook with logging (#17) * fix B904 flak8-bugbear warnings * change import style for logbook * replaced with logging * replaced logbook with logging * changes to utils path * wip sqlalchemy2.0 * exchange-calendar 4 * mock is part of unittest * fix cmdline start end * separate lint from test workflow * enable black formatting workflow check * separate black and lint from workflow * drop python 3.7 support
stefan-jansen · Jan 7, 2023 · 2642caa · 2642caa
1 parent 0eb9c56
commit 2642caa
Show file tree

Hide file tree

Showing 190 changed files with 4,610 additions and 5,277 deletions.
diff --git a/.devcontainer/requirements.txt b/.devcontainer/requirements.txt
@@ -28,14 +28,12 @@ iso3166==2.1.1
 iso4217==1.11.20220401
 kiwisolver==1.4.4
 korean-lunar-calendar==0.3.1
-Logbook==1.5.3
 lru-dict==1.1.8
 lxml==4.9.1
 Mako==1.2.4
 MarkupSafe==2.1.1
 matplotlib==3.6.2
 mccabe==0.7.0
-mock==4.0.3
 multipledispatch==0.6.0
 multitasking==0.0.11
 mypy-extensions==0.4.3

diff --git a/.github/workflows/ci_tests_full.yml b/.github/workflows/ci_tests_full.yml
@@ -6,13 +6,35 @@ on:
     - cron: "0 9 * * 6"
 
 jobs:
+  black-format:
+    name: Formatting Check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: psf/black@stable
+        with:
+          options: "--check --diff"
+          src: "./src ./tests"
+
+  flake8-lint:
+    name: Lint Check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/[email protected]
+        with:
+          python-version: "3.10"
+
+      - name: flake8 Lint
+        uses: py-actions/flake8@v2
+
   tests:
     runs-on: ${{ matrix.os }}
     strategy:
       fail-fast: false
       matrix:
-        os: [ ubuntu-latest, windows-latest, macos-latest ]
-        python-version: [ '3.7', '3.8', '3.9', '3.10' ]
+        os: [ubuntu-latest, windows-latest, macos-latest]
+        python-version: ["3.8", "3.9", "3.10"]
         exclude:
           - os: windows-latest
             python-version: 3.9
@@ -51,10 +73,6 @@ jobs:
           python -m pip install tox tox-gh-actions
           python -m pip install .[test]
 
-      - name: Lint with flake8
-        run: |
-          flake8
-
       - name: Unittests with tox & pytest
         uses: nick-fields/retry@v2
         with:

diff --git a/.github/workflows/ci_tests_quick.yml b/.github/workflows/ci_tests_quick.yml
@@ -8,13 +8,35 @@ on:
       - main
 
 jobs:
+  black-format:
+    name: Formatting Check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: psf/black@stable
+        with:
+          options: "--check --diff"
+          src: "./src ./tests"
+
+  flake8-lint:
+    name: Lint Check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/[email protected]
+        with:
+          python-version: "3.10"
+
+      - name: flake8 Lint
+        uses: py-actions/flake8@v2
+
   tests:
     runs-on: ${{ matrix.os }}
     strategy:
       fail-fast: false
       matrix:
         os: [ubuntu-latest, windows-latest, macos-latest]
-        python-version: ['3.10']
+        python-version: ["3.10"]
 
     steps:
       - name: Checkout Zipline
@@ -50,10 +72,6 @@ jobs:
           python -m pip install tox tox-gh-actions
           python -m pip install .[test]
 
-      - name: Lint with flake8
-        run: |
-          flake8
-
       - name: Unittests with tox & pytest
         uses: nick-fields/retry@v2
         with:

diff --git a/README.md b/README.md
@@ -6,10 +6,10 @@
 
 # Backtest your Trading Strategies
 
-|Version Info| [![Python](https://img.shields.io/pypi/pyversions/zipline-reloaded.svg?cacheSeconds=2592000")](https://pypi.python.org/pypi/zipline-reloaded) [![Anaconda-Server Badge](https://anaconda.org/ml4t/zipline-reloaded/badges/platforms.svg)](https://anaconda.org/ml4t/zipline-reloaded) [![Release](https://img.shields.io/pypi/v/zipline-reloaded.svg?cacheSeconds=2592000)](https://pypi.org/project/zipline-reloaded/)  [![Anaconda-Server Badge](https://anaconda.org/ml4t/zipline-reloaded/badges/version.svg)](https://anaconda.org/ml4t/zipline-reloaded)|
-|----|----|
-|**Test** **Status** | [![CI Tests](https://github.com/stefan-jansen/zipline-reloaded/actions/workflows/ci_tests_full.yml/badge.svg)](https://github.com/stefan-jansen/zipline-reloaded/actions/workflows/unit_tests.yml) [![PyPI](https://github.com/stefan-jansen/zipline-reloaded/actions/workflows/build_wheels.yml/badge.svg)](https://github.com/stefan-jansen/zipline-reloaded/actions/workflows/build_wheels.yml) [![Anaconda](https://github.com/stefan-jansen/zipline-reloaded/actions/workflows/conda_package.yml/badge.svg)](https://github.com/stefan-jansen/zipline-reloaded/actions/workflows/conda_package.yml) [![codecov](https://codecov.io/gh/stefan-jansen/zipline-reloaded/branch/main/graph/badge.svg)](https://codecov.io/gh/stefan-jansen/zipline-reloaded) |
-|**Community**|[![Discourse](https://img.shields.io/discourse/topics?server=https%3A%2F%2Fexchange.ml4trading.io%2F)](https://exchange.ml4trading.io) [![ML4T](https://img.shields.io/badge/Powered%20by-ML4Trading-blue)](https://ml4trading.io) [![Twitter](https://img.shields.io/twitter/follow/ml4trading.svg?style=social)](https://twitter.com/ml4trading)|
+| Version Info        | [![Python](https://img.shields.io/pypi/pyversions/zipline-reloaded.svg?cacheSeconds=2592000")](https://pypi.python.org/pypi/zipline-reloaded) [![Anaconda-Server Badge](https://anaconda.org/ml4t/zipline-reloaded/badges/platforms.svg)](https://anaconda.org/ml4t/zipline-reloaded) [![Release](https://img.shields.io/pypi/v/zipline-reloaded.svg?cacheSeconds=2592000)](https://pypi.org/project/zipline-reloaded/) [![Anaconda-Server Badge](https://anaconda.org/ml4t/zipline-reloaded/badges/version.svg)](https://anaconda.org/ml4t/zipline-reloaded)                                                                                                                                                                                                 |
+| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Test** **Status** | [![CI Tests](https://github.com/stefan-jansen/zipline-reloaded/actions/workflows/ci_tests_full.yml/badge.svg)](https://github.com/stefan-jansen/zipline-reloaded/actions/workflows/unit_tests.yml) [![PyPI](https://github.com/stefan-jansen/zipline-reloaded/actions/workflows/build_wheels.yml/badge.svg)](https://github.com/stefan-jansen/zipline-reloaded/actions/workflows/build_wheels.yml) [![Anaconda](https://github.com/stefan-jansen/zipline-reloaded/actions/workflows/conda_package.yml/badge.svg)](https://github.com/stefan-jansen/zipline-reloaded/actions/workflows/conda_package.yml) [![codecov](https://codecov.io/gh/stefan-jansen/zipline-reloaded/branch/main/graph/badge.svg)](https://codecov.io/gh/stefan-jansen/zipline-reloaded) |
+| **Community**       | [![Discourse](https://img.shields.io/discourse/topics?server=https%3A%2F%2Fexchange.ml4trading.io%2F)](https://exchange.ml4trading.io) [![ML4T](https://img.shields.io/badge/Powered%20by-ML4Trading-blue)](https://ml4trading.io) [![Twitter](https://img.shields.io/twitter/follow/ml4trading.svg?style=social)](https://twitter.com/ml4trading)                                                                                                                                                                                                                                                                                                                                                                                                            |
 
 Zipline is a Pythonic event-driven system for backtesting, developed and used as the backtesting and live-trading engine by [crowd-sourced investment fund Quantopian](https://www.bizjournals.com/boston/news/2020/11/10/quantopian-shuts-down-cofounders-head-elsewhere.html). Since it closed late 2020, the domain that had hosted these docs expired. The library is used extensively in the book [Machine Larning for Algorithmic Trading](https://ml4trading.io)
 by [Stefan Jansen](https://www.linkedin.com/in/applied-ai/) who is trying to keep the library up to date and available to his readers and the wider Python algotrading community.
@@ -26,7 +26,7 @@ by [Stefan Jansen](https://www.linkedin.com/in/applied-ai/) who is trying to kee
 
 ## Installation
 
-Zipline supports Python >= 3.7 and is compatible with current versions of the relevant [NumFOCUS](https://numfocus.org/sponsored-projects?_sft_project_category=python-interface) libraries, including [pandas](https://pandas.pydata.org/) and [scikit-learn](https://scikit-learn.org/stable/index.html).
+Zipline supports Python >= 3.8 and is compatible with current versions of the relevant [NumFOCUS](https://numfocus.org/sponsored-projects?_sft_project_category=python-interface) libraries, including [pandas](https://pandas.pydata.org/) and [scikit-learn](https://scikit-learn.org/stable/index.html).
 
 If your system meets the pre-requisites described in the [installation instructions](https://zipline.ml4trading.io/install.html), you can install Zipline using pip by running:
 

diff --git a/docs/source/install.rst b/docs/source/install.rst
@@ -9,7 +9,7 @@ that runs on Windows, macOS, and Linux. In case you are installing `zipline-relo
 encounter [conflict errors](https://github.com/conda/conda/issues/9707), consider using
 [mamba](https://github.com/mamba-org/mamba) instead.
 
-Zipline runs on Python 3.7, 3.8 and 3.9. To install and use different Python versions in parallel as well as create
+Zipline runs on Python 3.8, 3.9 and 3.10. To install and use different Python versions in parallel as well as create
 a virtual environment, you may want to use `pyenv <https://github.com/pyenv/pyenv>`_.
 
 Installing with ``pip``

diff --git a/pyproject.toml b/pyproject.toml
@@ -18,7 +18,6 @@ classifiers = [
     'License :: OSI Approved :: Apache Software License',
     'Natural Language :: English',
     'Programming Language :: Python',
-    'Programming Language :: Python :: 3.7',
     'Programming Language :: Python :: 3.8',
     'Programming Language :: Python :: 3.9',
     'Programming Language :: Python :: 3.10',
@@ -32,7 +31,7 @@ classifiers = [
 
 license = { file = "LICENSE" }
 
-requires-python = '>=3.7'
+requires-python = '>=3.8'
 dependencies = [
     'alembic >=0.7.7',
     'bcolz-zipline >=1.2.6',
@@ -43,13 +42,12 @@ dependencies = [
     'intervaltree >=2.1.0',
     'iso3166 >=2.1.1',
     'iso4217 >=1.6.20180829',
-    'logbook >=1.0',
     'lru-dict >=1.1.4',
     'multipledispatch >=0.6.0',
     'networkx >=2.0',
     'numexpr >=2.6.1',
     'numpy >=1.14.5, <1.24',
-    'pandas >=1.1.0, <1.6',
+    'pandas >=1.3, <1.6',
     'patsy >=0.4.0',
     'python-dateutil >=2.4.2',
     'python-interface >=1.5.3',
@@ -62,8 +60,7 @@ dependencies = [
     'ta-lib >=0.4.09',
     'tables >=3.4.3',
     'toolz >=0.8.2',
-    'trading-calendars >=1.6.1',
-    'exchange-calendars <=3.3'
+    'exchange-calendars >=4.2.4'
 ]
 
 [project.urls]
@@ -77,32 +74,34 @@ requires = [
     "setuptools_scm[toml]>=6.2",
     'wheel>=0.36.0',
     'Cython>=0.29.21,<3',
-    'oldest-supported-numpy; python_version>="3.7"',
+    'oldest-supported-numpy; python_version>="3.8"',
 ]
 build-backend = 'setuptools.build_meta'
 
 [project.optional-dependencies]
 test = [
     'tox',
-    'pytest==6.2.5',
+    'pytest>=7.2.0',
     'pytest-cov >=3.0.0',
     'pytest-xdist >=2.5.0',
     'pytest-timeout >=1.4.2',
     'parameterized >=0.6.1',
-    'mock >=2.0.0p',
     'testfixtures >=4.1.2',
     'flake8 >=3.9.1',
     'matplotlib >=1.5.3',
     'responses >=0.9.0',
     'pandas-datareader >=0.2.1',
     'click <8.1.0',
     'coverage',
-    'pytest-rerunfailures'
+    'pytest-rerunfailures',
+    'psycopg2 ==2.9.4',
+    'pytest-postgresql ==3.1.3'
 ]
 dev = [
     'flake8 >=3.9.1',
     'black',
-    'pre-commit >=2.12.1'
+    'pre-commit >=2.12.1',
+    'Cython>=0.29.21,<3',
 
 ]
 docs = [
@@ -179,7 +178,6 @@ minversion = 3.23.0
 
 [gh-actions]
 python =
-    3.7: py37
     3.8: py38
     3.9: py39
     3.10: py310
@@ -192,8 +190,6 @@ setenv =
 changedir = tmp
 extras = test
 deps =
-    pandas11: pandas>=1.1.0,<1.2
-    pandas12: pandas>=1.2.0,<1.3
     pandas13: pandas>=1.3.0,<1.4
     pandas14: pandas>=1.4.0,<1.5
     pandas15: pandas>=1.5.0,<1.6

diff --git a/src/zipline/__main__.py b/src/zipline/__main__.py
@@ -2,7 +2,7 @@
 import os
 
 import click
-import logbook
+import logging
 import pandas as pd
 
 import zipline
@@ -48,8 +48,14 @@
 @click.pass_context
 def main(ctx, extension, strict_extensions, default_extension, x):
     """Top level zipline entry point."""
-    # install a logbook handler before performing any other operations
-    logbook.StderrHandler().push_application()
+    # install a logging handler before performing any other operations
+
+    logging.basicConfig(
+        format="[%(asctime)s-%(levelname)s][%(name)s]\n %(message)s",
+        level=logging.INFO,
+        datefmt="%Y-%m-%dT%H:%M:%S%z",
+    )
+
     create_args(x, zipline.extension_args)
     load_extensions(
         default_extension,
@@ -194,13 +200,13 @@ def _(*args, **kwargs):
 @click.option(
     "-s",
     "--start",
-    type=Date(tz="utc", as_timestamp=True),
+    type=Date(as_timestamp=True),
     help="The start date of the simulation.",
 )
 @click.option(
     "-e",
     "--end",
-    type=Date(tz="utc", as_timestamp=True),
+    type=Date(as_timestamp=True),
     help="The end date of the simulation.",
 )
 @click.option(
@@ -357,11 +363,13 @@ def zipline_magic(line, cell=None):
             # don't use system exit and propogate errors to the caller
             standalone_mode=False,
         )
-    except SystemExit as e:
+    except SystemExit as exc:
         # https://github.com/mitsuhiko/click/pull/533
         # even in standalone_mode=False `--help` really wants to kill us ;_;
-        if e.code:
-            raise ValueError("main returned non-zero status code: %d" % e.code)
+        if exc.code:
+            raise ValueError(
+                "main returned non-zero status code: %d" % exc.code
+            ) from exc
 
 
 @main.command()