Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update cme futures holidays and early closes #7524

Conversation

Marinovsky
Copy link
Collaborator

@Marinovsky Marinovsky commented Oct 20, 2023

Description

As pointed out here (#7501 (comment)) the general CME Future entry (Future-cme-[*]) contained dates that were not holidays and therefore needed to be removed, at least from the general entry. Some of those dates were actually early closes for futures of class Equity, FX and Interest Rate, others were in fact holidays, as defined in the officials CME Holiday Calendars (https://www.cmegroup.com/tools-information/holiday-calendar.html). For these reason, most of the CME group assets needed an update in its early closes and holidays entries in MHDB.

In order to update the MHDB, we used a Python notebook since there was a library called pandas_market_calendars (https://pypi.org/project/pandas-market-calendars/) that claimed to contain holiday and early closes dates for most of the markets. After, manually checking these dates were correct, we removed the ones that were not holidays from the general CME Future entry in MHDB and searched for which futures they were indeed holidays. We found that some of them were holidays for CME asset classes such as Energy, Metals, Grains, Livestock and Dairy, so we add those dates in their respective entries in MHDB.

On the other hand, another bug it was found that was for some CME Future assets, though the timezone was America/New York their early closes were in America/Chicago timezone (the CME Futures class of all of those found assets was Equity). Therefore, the early-closes entry of them were also updated.

In the beginning it was wanted a script that given one CME Group asset( or more), could return a new MHDB with its holidays and its early closes updated. However, due to uncertainty of the library veracity, the purpose of the script was changed so that it could be useful verifying the veracity of the holidays and early-closes dates. Still, that script realized to fix the underlying GH issue can be modified to fulfill its initial purpose (see https://colab.research.google.com/drive/1YhzVAeJnhRCujruuhu9jlOaoWldrS3tt?usp=sharing).

To sum up:

  • Wrong holiday dates in Futures-cme-[*] in MHDB were removed. The dates that were holidays for other CME Group Futures classes were added in the entries of the products of that class.
  • CME Group Futures entries in MHDB were updated in their early closes and late opens
  • CME Group Futures early closes, late opens and halts were extracted manually using the CME Group Market Calendars available in their webpage (since 2009).
  • LEAN was modified to handle HALTS, this is, to resume the market after an early close
  • Unit tests were added to assert the behavior of the HALTS
  • Regression test stats were updated

This changes were made using the following collaborative notebook: https://colab.research.google.com/drive/1YhzVAeJnhRCujruuhu9jlOaoWldrS3tt?usp=sharing

Related Issue

Closes #7501

Motivation and Context

With this change users will have more accurate data about the early closes and holidays of CME Group Futures

Requires Documentation Change

N/A

How Has This Been Tested?

The holidays and early-closes dates given by pandas_market_calendars library were manually asserted to be holidays or early closes using the files from CME Group Holiday Calendars (https://www.cmegroup.com/tools-information/holiday-calendar.html)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Refactor (non-breaking change which improves implementation)
  • Performance (non-breaking change which improves performance. Please add associated performance test and results)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Non-functional change (xml comments/documentation/etc)

Checklist:

  • My code follows the code style of this project.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>

Marinovsky and others added 14 commits November 20, 2023 11:05
Acoording to `pandas_market_calendars` there were some dates in
Future-cme-[*] who wasn't early_closes, so they needed to be removed
from there. On the other hand, the early closes list of Future-cme-ES were
shifted by 1 hour according to CME webpage. Besides, there were some
missing dates.
After debugging the tests it was found they were failing due to the last
change on SecurityExchangeHours.IsOpen(). That method wasn't taking into
account that even if there is a late open after an early close if the
timespan is after the early close but before the late open, the market
is still close.
@Marinovsky Marinovsky force-pushed the bug-7501-UpdateCMEFuturesHolidaysAndEarlyCloses branch from 7e94f68 to 1693b4f Compare November 20, 2023 16:12
Copy link
Member

@Martin-Molinero Martin-Molinero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you 🙌

@Marinovsky Marinovsky mentioned this pull request Nov 29, 2023
11 tasks
@Martin-Molinero
Copy link
Member

Included by #7605

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Futures Early Close Removing The Entire Day
2 participants