Merge branch 'release/0.8.0'

.github/workflows/draft-release.yml

@@ -17,7 +17,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       # Drafts your next Release notes as Pull Requests are merged into "master"
-      - uses: release-drafter/release-drafter@v5
+      - uses: release-drafter/release-drafter@v6
         with:
           config-name: release-drafter-config.yml
         env:

.pre-commit-config.yaml

@@ -1,6 +1,14 @@
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
     # Ruff version.
-    rev: v0.0.275
+    rev: v0.3.2
     hooks:
+      # Run the linter.
       - id: ruff
+      # Run the formatter.
+      - id: ruff-format
+  - repo: https://github.com/asottile/pyupgrade
+    rev: v3.15.2
+    hooks:
+      - id: pyupgrade
+        args: ["--py39-plus"]

docs/calendar_modifications.md

@@ -12,6 +12,28 @@ menu:
 
 Extended exchange calendars provide an API to support modifications at runtime. 
 
+## Dates, Times, and Day Types
+Calendar modifications are represented using common data types for dates, wall-clock times, and types of special 
+days. Thanks to Pydantic and custom annotated types, however, the API allows to pass in values in different formats that
+will safely be converted into the correct used internally.
+
+Wherever the API expects a `pandas.Timestamp`, represented by the type `TimestampLike`, it is possible to an actual 
+`pandas.Timestamp`, a `datetime.date` object, a string in ISO format `YYYY-MM-DD`, or any other valid value that can be 
+used to initialize a timestamp. Pydantic will validate such calls and enforce the correct data type.
+
+There is also the special type `DateLike` which is used to represent date-like Timestamps. Such timestamps are 
+normalized to midnight and are timezone-naive. They represent full days starting at midnight (inclusive) and ending at 
+midnight (exclusive) of the following day *in the context of the exchange and the corresponding timezone they are used 
+in*. A `DateLike` timestamp is typically used to specify a date for a specific exchange calendar that has a timezone 
+attached.
+
+Similar to timestamps, wall clock times in the form of `datetime.time` are represented by 
+`TimeLike` to allow passing an actual `datetime.time` or strings in the format 
+`HH:MM:SS` or `HH:MM`.
+
+The enumeration type `DayType` represents types of special days, API calls accept either enumeration members or 
+their string value. For example, `DayType.HOLIDAY` and `'holiday'` can be used equivalently.
+
 ## Adding Special Days
 
 The `exchange_calendars_extensions` module provides the following methods for adding special days:
@@ -24,7 +46,7 @@ The `exchange_calendars_extensions` module provides the following methods for ad
 
 For example,
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 import exchange_calendars as ec
 
@@ -40,7 +62,7 @@ always added as regular holidays, not as ad-hoc holidays, to allow for an indivi
 
 Adding special open or close days works similarly, but needs the respective special open or close time:
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 
 ecx.apply_extensions()
 import exchange_calendars as ec
@@ -55,7 +77,7 @@ assert '2022-12-28' in calendar.special_opens_all.holidays()
 A more generic way to add a special day is via `add_day(...)` which takes either a `DaySpec` (holidays, 
 monthly/quarterly expiries) or `DaySpecWithTime` (special open/close days) Pydantic model:
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 import exchange_calendars as ec
 
@@ -73,7 +95,7 @@ The `DayType` enum enumerates all supported special day types.
 
 Thanks to Pydantic, an even easier way just uses suitable dictionaries: 
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 import exchange_calendars as ec
 
@@ -88,11 +110,11 @@ assert '2022-12-28' in calendar.special_opens_all.holidays()
 ```
 The dictionary format makes it particularly easy to read in changes from an external source like a file.
 
-## Removing Special Sessions
+## Removing Special Days
 
 To remove a day as a special day (of any type) from a calendar, use `remove_day`. For example,
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 import exchange_calendars as ec
 
@@ -108,22 +130,11 @@ will remove the holiday on 27 December 2022 from the calendar, thus turning this
 Removing a day via `remove_day(...)` that is not actually a special day, results in no change and does not throw an 
 exception.
 
-## Dates, Times, and Day Types
-Thanks to Pydantic, dates, times, and the type of a special day can typically be specified in different formats and will 
-safely be parsed into the correct data type that is used internally.
-
-For example, wherever the API expects a date, you may pass in a `pandas.Timestamp`, a `datetime.date` object, or simply
-a string in ISO format `YYYY-MM-DD`. Similarly, wall clock times can be passed as `datetime.time` objects or as strings 
-in the format `HH:MM:SS` or `HH:MM`.
-
-The enumeration type `ecx.DayType` represents types of special days, API calls accept either enumeration members or 
-their string value. For example, `ecx.DayType.HOLIDAY` and `'holiday'` can be used equivalently.
-
 ## Visibility of Changes
 Whenever a calendar has been modified programmatically, the changes are only reflected after obtaining a new exchange 
 calendar instance.
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 import exchange_calendars as ec
 
@@ -165,7 +176,7 @@ exchange. When a new calendar instance is created, the changes are applied to th
 
 It is also possible to create a changeset separately and then associate it with a particular exchange:
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 import exchange_calendars as ec
 
@@ -183,7 +194,7 @@ assert '2022-12-28' in calendar.holidays_all.holidays()
 Again, an entire changeset can also be created from a suitably formatted dictionary, making it particularly easy to read
 in and apply changes from an external source like a file.
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 import exchange_calendars as ec
 
@@ -204,7 +215,7 @@ assert '2022-12-28' in calendar.holidays_all.holidays()
 The API permits to add and remove the same day as a special day. For example, the following code will add a holiday on
 28 December 2022 to the calendar, and then remove the same day as well.
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 import exchange_calendars as ec
 
@@ -220,7 +231,7 @@ more sense in a case where a day is added to change its type of special day. Con
 holiday for the calendar `XLON` in the original version of the calendar. The following code will change the type of 
 special day to a special open by first removing the day (as a holiday), and then adding it back as a special open day:
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 import exchange_calendars as ec
 
@@ -239,7 +250,7 @@ allows to change the type of special day in an existing calendar from one to ano
 In fact, internally, each added days is always implicitly also removed from the calendar first, so that it strictly is 
 not necessary (but allowed) to explicitly remove a day, and then adding it back as a different type of special day:
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 import exchange_calendars as ec
 
@@ -259,7 +270,7 @@ As seen above, changesets may contain the same day both in the list of days to a
 However, changesets enforce consistency and will raise an exception if the same day is added more than once.
 For example, the following code will raise an exception:
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 
 ecx.add_holiday('XLON', date='2022-12-28', name='Holiday')
@@ -268,7 +279,7 @@ ecx.add_special_open('XLON', date='2022-12-28', name='Special Open', time='11:00
 In contrast, removing a day is an idempotent operation, i.e. doing it twice will not raise an exception and keep the 
 corresponding changeset the same as after the first removal. 
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 
 ecx.remove_day('XLON', date='2022-12-27')
@@ -281,7 +292,7 @@ It is sometimes necessary to revert individual changes made to a calendar. To th
 `reset_day`:
 
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 import exchange_calendars as ec
 
@@ -306,7 +317,7 @@ assert '2022-12-28' not in calendar.holidays_all.holidays()
 To reset an entire calendar to its original state, use the method `reset_calendar` or update the calendar with an 
 empty ChangeSet:
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 import exchange_calendars as ec
 
@@ -331,7 +342,7 @@ assert '2022-12-28' not in calendar.holidays_all.holidays()
 ## Retrieving Changes
 For any calendar, it is possible to retrieve a copy of the associated changeset:
 ```python
-import exchange_calendars_extensions as ecx
+import exchange_calendars_extensions.core as ecx
 ecx.apply_extensions()
 
 ecx.add_holiday('XLON', date='2022-12-28', name='Holiday')

docs/features.md

@@ -63,14 +63,14 @@ Calendars for expiry day sessions are currently only available for the following
 {{% /note %}}
 
 ## Calendar Modifications
-This package also adds the ability to modify existing calendars at runtime. This can be used to add or remove
+It is also possible to modify existing calendars at runtime. This can be used to add or remove
 - holidays (regular and ad-hoc)
 - special open days (regular and ad-hoc)
 - special close days (regular and ad-hoc)
 - quarterly expiry days
 - monthly expiry days
 
-This is useful to fix incorrect information from `exchange-calendars`. This regularly happens, e.g., when an 
+This may be useful to fix incorrect information from `exchange-calendars`. This regularly happens, e.g., when an 
 exchange announces a change to the regular trading schedule on short notice and an updated release of the upstream 
 package is not yet available. After some time, modifications can typically be removed when the upstream package has
 been updated.
@@ -81,3 +81,18 @@ in the [`exchange-calendars`](https://github.com/gerrymanoim/exchange_calendars)
 calendars should only be used as a last resort and to bridge the time until the information has been updated at the 
 root.
 {{% /warning %}}
+
+## Metadata
+In some situations, it may be useful to be able to associate arbitrary metadata with certain dates. Here, metadata can 
+be a set of string tags and/or a string comment.
+
+{{% note %}}
+For example, a tag could be used to mark days on which the exchange 
+deviated from the regular trading schedule in an unplanned way, e.g. a delayed open due to technical issues. That is, 
+tags or a comment could be useful to incorporate additional user-owned information that would normally be outside the 
+scope of the exchange calendars core functionality.
+{{% /note %}}
+
+This package provides functionality to add metadata in the form of tags and/or comments to any date in any calendar.
+It is then possible to filter dates by their metadata to retrieve only dates within a certain time period that e.g. 
+have a certain tag set.

docs/metadata.md

@@ -0,0 +1,135 @@
+---
+title: "Metadata"
+draft: false
+type: docs
+layout: "single"
+
+menu:
+  docs_extensions:
+      weight: 60
+---
+# Metadata
+
+Metadata in the form of tags and comments can be associated with specific dates. Metadata can be a combination of a 
+single string comment and/or a set of string tags. For example, 
+```python
+import pandas as pd
+
+import exchange_calendars_extensions.core as ecx
+ecx.apply_extensions()
+import exchange_calendars as ec
+
+# Add metadata.
+ecx.set_comment('XLON', '2022-01-01', "This is a comment.")
+ecx.set_tags('XLON', '2022-01-01', {'tag1', 'tag2'})
+
+calendar = ec.get_calendar('XLON')
+
+# Get metadata.
+meta_dict = calendar.meta()
+print(len(meta_dict))
+
+# The value for the first and only date.
+meta = meta_dict[pd.Timestamp('2022-01-01')]
+
+print(meta.comment)
+print(meta.tags)
+```
+will print
+```text
+1
+This is a comment.
+{'tag1', 'tag2'}
+```
+
+The `meta()` method returns an ordered dictionary of type `Dict[pd.Timestamp, DayMeta]` that contains all days that have 
+metadata associated with them, ordered by date. The keys are `DateLike` timezone-naive Pandas timestamps normalized to 
+midnight. Each timestamp represents a full day starting at midnight (inclusive) and ending at midnight (exclusive) of 
+the following day within the relevant timezone for the exchange. 
+
+
+{{% note %}}
+Currently, the dictionary returned by the `meta()` method does not support lookup from values other than 
+`pandas.Timestamp`. This means that it is not possible to look up metadata for a specific date using a string.
+{{% /note %}}
+
+Dates can be filtered by a start and an end timestamp. For example, 
+```python
+import pandas as pd
+
+import exchange_calendars_extensions.core as ecx
+ecx.apply_extensions()
+import exchange_calendars as ec
+
+# Add metadata for two dates.
+ecx.set_comment('XLON', '2022-01-01', "This is a comment.")
+ecx.set_tags('XLON', '2022-01-01', {'tag1', 'tag2'})
+ecx.set_comment('XLON', '2022-01-02', "This is another comment.")
+ecx.set_tags('XLON', '2022-01-02', {'tag3', 'tag4'})
+
+calendar = ec.get_calendar('XLON')
+
+# Get metadata only for 2022-01-01.
+meta_dict = calendar.meta(start='2022-01-01', end='2022-01-01')
+print(len(meta_dict))
+
+# The value for the first and only date.
+meta = meta_dict[pd.Timestamp('2022-01-01')]
+
+print(meta.comment)
+print(meta.tags)
+```
+will print
+```text
+1
+This is a comment.
+{'tag1', 'tag2'}
+```
+
+The `meta()` method supports `TimestampLike` `start` and `end` arguments which must be either both timezone-naive or 
+timezone-aware. Otherwise, a `ValueError` is raised.
+
+The returned dictionary includes all days with metadata that have a non-empty intersection with the period between 
+the `start` and `end`. This result is probably what one would usually expect, even in situations where `start` and/or 
+`end` are not aligned to midnight. In the above example, if `start` were `2022-01-01 06:00:00` and `end` were 
+`2022-01-01 18:00:00`, the result would be the same since the intersection with the full day `2022-01-01` is non-empty.
+
+When `start` and `end` are timezone-naive, as in the examples above, the timezone of the exchange does not matter. Like
+`start` and `end`, the timestamps that mark the beginning and end of a day are used timezone-naive. Effectively, any 
+comparison uses timestamps with a wall-clock time component.
+
+In contrast, when `start` and `end` timestamps are timezone-aware, all other timestamps also used timezone-aware and 
+with the exchange's native timezone. Comparisons are then done between instants, i.e. actual points on the timeline.
+
+The difference between the two cases is illustrated in the following example which considers the date 2024-03-31. In 
+timezones that are based on Central European Time (CET), a transition to Central European Summer Time (CEST) occurs on
+this date. The transition happens at 02:00:00 CET, which is 03:00:00 CEST, i.e. clocks advance by one hour and the day 
+is 23 hours long.
+```python
+import pandas as pd
+
+import exchange_calendars_extensions.core as ecx
+from collections import OrderedDict
+from exchange_calendars_extensions.api.changes import DayMeta
+ecx.apply_extensions()
+import exchange_calendars as ec
+
+# Add metadata.
+day = pd.Timestamp("2024-03-31")
+meta = DayMeta(tags=[], comment="This is a comment")
+ecx.set_meta('XETR', day, meta)
+
+calendar = ec.get_calendar('XETR')
+
+# Get metadata for 2024-03-31, timezone-naive.
+assert calendar.meta(start='2024-03-31 00:00:00') == OrderedDict([(day, meta)])
+assert calendar.meta(start='2024-03-31 23:59:59') == OrderedDict([(day, meta)])
+
+# Get metadata for 2024-03-31, timezone-aware.
+# 2024-03-30 23:00:00 UTC is 2024-03-31 00:00:00 CET.
+assert calendar.meta(start=pd.Timestamp('2024-03-30 23:00:00').tz_localize("UTC")) == OrderedDict([(day, meta)])
+# 2024-03-31 21:59:59 UTC is 2024-03-31 23:59:59 CEST.
+assert calendar.meta(start=pd.Timestamp('2024-03-31 21:59:59').tz_localize("UTC")) == OrderedDict([(day, meta)])
+# 2024-03-31 22:00:00 UTC is 2024-03-31 00:00:00 CEST.
+assert calendar.meta(start=pd.Timestamp('2024-03-31 22:00:00').tz_localize("UTC")) == OrderedDict([])
+```