Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Regression in v2.2.3 for annual events #151

Closed
sanieldalib opened this issue Jul 30, 2024 · 16 comments · Fixed by #158
Closed

bug: Regression in v2.2.3 for annual events #151

sanieldalib opened this issue Jul 30, 2024 · 16 comments · Fixed by #158
Labels
bug Something isn't working

Comments

@sanieldalib
Copy link

sanieldalib commented Jul 30, 2024

I am running into an issue where the same code:

calendar = Calendar.from_ical(ical_str.read())
parsed_cal = recurring_ical_events.of(calendar)
parsed_cal.between(start_time, end_time):

returns different events on macOS compared to a linux docker container (python:3.9.11). Specifically, it is an annually recurring event missing when running within the linux container (python:3.9.11). I have confirmed that the difference is between OSes, as I get the difference in behavior on the same machine, python version, and package version - the only difference is macOS vs the docker image.

I'll try to create a minimum reproduction of this and include it.

Is there anything you might be aware of that would be causing this issue?


We're using Polar.sh so you can upvote and help fund this issue. We receive the funding once the issue is completed & confirmed by you. Thank you in advance for helping prioritize & fund our work. Fund with Polar
@sanieldalib sanieldalib added the bug Something isn't working label Jul 30, 2024
@fabien-michel
Copy link
Collaborator

Can you provide the ics of the event?
Is there a difference in the system / container locale?

@sanieldalib
Copy link
Author

sanieldalib commented Jul 30, 2024

Thank you @fabien-michel!

Here is the ics: https://calendar.google.com/calendar/ical/themarinas%40gmail.com/public/basic.ics

With the following code

from urllib.request import urlopen
from datetime import datetime, timezone

from icalendar import Calendar
import recurring_ical_events

with urlopen(
    "https://calendar.google.com/calendar/ical/themarinas%40gmail.com/public/basic.ics"
) as ical_str:

    calendar = Calendar.from_ical(ical_str.read())
    parsed_cal = recurring_ical_events.of(calendar)

    start_time = datetime.fromtimestamp(1722564000, timezone.utc)
    end_time = datetime.fromtimestamp(1722567600, timezone.utc)
    events = parsed_cal.between(start_time, end_time)

    print(f"Found {len(events)} events")
    print(events)

I get the following output on macOS:

Found 3 events
[VEVENT({'DTSTART': vDDDTypes(2024-08-01 19:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTEND': vDDDTypes(2024-08-01 20:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTSTAMP': vDDDTypes(2024-07-30 18:35:35+00:00, Parameters({})), 'UID': vText('b'[email protected]''), 'CREATED': vDDDTypes(2014-03-31 19:26:50+00:00, Parameters({})), 'LAST-MODIFIED': vDDDTypes(2015-07-31 02:23:26+00:00, Parameters({})), 'SEQUENCE': 1, 'STATUS': vText('b'CONFIRMED''), 'SUMMARY': vText('b'Vespers for the Feast of St. Joseph''), 'TRANSP': vText('b'OPAQUE'')}, VALARM({'ACTION': vText('b'NONE''), 'TRIGGER': vDDDTypes(1976-04-01 00:55:45+00:00, Parameters({'VALUE': 'DATE-TIME'})), 'X-WR-ALARMUID': vText('b'0D3A9816-AC61-499A-A594-930AA281666B''), 'UID': vText('b'0D3A9816-AC61-499A-A594-930AA281666B'')})), VEVENT({'DTSTART': vDDDTypes(2024-08-02, Parameters({'VALUE': 'DATE'})), 'DTEND': vDDDTypes(2024-08-03, Parameters({'VALUE': 'DATE'})), 'DTSTAMP': vDDDTypes(2024-07-30 18:35:35+00:00, Parameters({})), 'UID': vText('b'040000008200E00074C5B7101A82E0080000000080791746E0B7C801000000000000000010000000296A1C3ED541F040AE0829C0C67BD7E0''), 'CREATED': vDDDTypes(2008-05-17 12:41:40+00:00, Parameters({})), 'LAST-MODIFIED': vDDDTypes(2023-02-21 06:29:03+00:00, Parameters({})), 'SEQUENCE': 0, 'STATUS': vText('b'CONFIRMED''), 'SUMMARY': vText('b'Feast of St Joseph the Carpenter''), 'TRANSP': vText('b'OPAQUE''), 'X-APPLE-TRAVEL-ADVISORY-BEHAVIOR': vText('b'AUTOMATIC'')}, VALARM({'ACTION': vText('b'AUDIO''), 'TRIGGER': vDDDTypes(-1 day, 9:00:00, Parameters({})), 'X-WR-ALARMUID': vText('b'F02F9E2A-F0FB-4E4F-A9AB-9405F33BBA19''), 'UID': vText('b'F02F9E2A-F0FB-4E4F-A9AB-9405F33BBA19''), 'ATTACH': 'Chord', 'X-APPLE-DEFAULT-ALARM': vText('b'TRUE''), 'ACKNOWLEDGED': vText('b'20200802T001626Z'')})), VEVENT({'DTSTART': vDDDTypes(2024-08-01 19:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTEND': vDDDTypes(2024-08-01 20:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTSTAMP': vDDDTypes(2024-07-30 18:35:35+00:00, Parameters({})), 'UID': vText('b'3DF21382-D125-4051-98BA-AE1412967940''), 'CREATED': vDDDTypes(2024-07-30 02:36:49+00:00, Parameters({})), 'LAST-MODIFIED': vDDDTypes(2024-07-30 02:36:49+00:00, Parameters({})), 'SEQUENCE': 0, 'STATUS': vText('b'CONFIRMED''), 'SUMMARY': vText('b'Vespers: Feast of St. Joseph the Carpenter''), 'TRANSP': vText('b'OPAQUE''), 'X-APPLE-TRAVEL-ADVISORY-BEHAVIOR': vText('b'AUTOMATIC'')})]

However, running within the python:3.9.11 container yields the following:

Found 2 events
[VEVENT({'DTSTART': vDDDTypes(2024-08-02, Parameters({'VALUE': 'DATE'})), 'DTEND': vDDDTypes(2024-08-03, Parameters({'VALUE': 'DATE'})), 'DTSTAMP': vDDDTypes(2024-07-30 18:47:01+00:00, Parameters({})), 'UID': vText('b'040000008200E00074C5B7101A82E0080000000080791746E0B7C801000000000000000010000000296A1C3ED541F040AE0829C0C67BD7E0''), 'CREATED': vDDDTypes(2008-05-17 12:41:40+00:00, Parameters({})), 'LAST-MODIFIED': vDDDTypes(2023-02-21 06:29:03+00:00, Parameters({})), 'SEQUENCE': 0, 'STATUS': vText('b'CONFIRMED''), 'SUMMARY': vText('b'Feast of St Joseph the Carpenter''), 'TRANSP': vText('b'OPAQUE''), 'X-APPLE-TRAVEL-ADVISORY-BEHAVIOR': vText('b'AUTOMATIC'')}, VALARM({'ACTION': vText('b'AUDIO''), 'TRIGGER': vDDDTypes(-1 day, 9:00:00, Parameters({})), 'X-WR-ALARMUID': vText('b'F02F9E2A-F0FB-4E4F-A9AB-9405F33BBA19''), 'UID': vText('b'F02F9E2A-F0FB-4E4F-A9AB-9405F33BBA19''), 'ATTACH': 'Chord', 'X-APPLE-DEFAULT-ALARM': vText('b'TRUE''), 'ACKNOWLEDGED': vText('b'20200802T001626Z'')})), VEVENT({'DTSTART': vDDDTypes(2024-08-01 19:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTEND': vDDDTypes(2024-08-01 20:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTSTAMP': vDDDTypes(2024-07-30 18:47:01+00:00, Parameters({})), 'UID': vText('b'3DF21382-D125-4051-98BA-AE1412967940''), 'CREATED': vDDDTypes(2024-07-30 02:36:49+00:00, Parameters({})), 'LAST-MODIFIED': vDDDTypes(2024-07-30 02:36:49+00:00, Parameters({})), 'SEQUENCE': 0, 'STATUS': vText('b'CONFIRMED''), 'SUMMARY': vText('b'Vespers: Feast of St. Joseph the Carpenter''), 'TRANSP': vText('b'OPAQUE''), 'X-APPLE-TRAVEL-ADVISORY-BEHAVIOR': vText('b'AUTOMATIC'')})]

Here is the Dockerfile I used:

FROM python:3.9.11
WORKDIR /code
COPY test.py /code/
RUN pip install recurring-ical-events~=2.2.3
CMD python3 test.py

@sanieldalib
Copy link
Author

Hi @fabien-michel thank you for your response. Just wanted to check if this is enough information for you to potentially reproduce?

@niccokunzmann
Copy link
Owner

niccokunzmann commented Aug 6, 2024 via email

@sanieldalib
Copy link
Author

@niccokunzmann @fabien-michel Here is the ICS file:
basic.ics.txt

Github does not access .ics so I added the txt extension.

@niccokunzmann
Copy link
Owner

niccokunzmann commented Aug 7, 2024

@sanieldalib Could you check this? I copied your code and now, I get this output:

~/recurring-ical-events$ .tox/py39/bin/python test.py 
Found 0 events
[]

Code of test.py

from urllib.request import urlopen
from datetime import datetime, timezone

from icalendar import Calendar
import recurring_ical_events

with urlopen(
    "https://github.com/user-attachments/files/16516741/basic.ics.txt"
) as ical_str:

    calendar = Calendar.from_ical(ical_str.read())
    parsed_cal = recurring_ical_events.of(calendar)

    start_time = datetime.fromtimestamp(1722564000, timezone.utc)
    end_time = datetime.fromtimestamp(1722567600, timezone.utc)
    events = parsed_cal.between(start_time, end_time)

    print(f"Found {len(events)} events")
    print(events)

@niccokunzmann
Copy link
Owner

niccokunzmann commented Aug 7, 2024

@sanieldalib, I opened #153 for this but I think the calendar might have changed as the output is different ...

@sanieldalib
Copy link
Author

@niccokunzmann Sorry, the ics was updated from my first posting to only include the troublesome event within the specified timestamps.

Here is the output of your test on my macOS machine:

✗ python3 test.py 
Found 1 events
[VEVENT({'DTSTART': vDDDTypes(2024-08-01 19:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTEND': vDDDTypes(2024-08-01 20:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTSTAMP': vDDDTypes(2024-08-07 00:03:26+00:00, Parameters({})), 'UID': vText('b'[email protected]''), 'CREATED': vDDDTypes(2014-03-31 19:26:50+00:00, Parameters({})), 'LAST-MODIFIED': vDDDTypes(2015-07-31 02:23:26+00:00, Parameters({})), 'SEQUENCE': 1, 'STATUS': vText('b'CONFIRMED''), 'SUMMARY': vText('b'Vespers for the Feast of St. Joseph''), 'TRANSP': vText('b'OPAQUE'')}, VALARM({'ACTION': vText('b'NONE''), 'TRIGGER': vDDDTypes(1976-04-01 00:55:45+00:00, Parameters({'VALUE': 'DATE-TIME'})), 'X-WR-ALARMUID': vText('b'0D3A9816-AC61-499A-A594-930AA281666B''), 'UID': vText('b'0D3A9816-AC61-499A-A594-930AA281666B'')}))]

vs in docker (python:3.9.11) :

Found 0 events                                                              
[]      

@niccokunzmann
Copy link
Owner

Hm. I am under MacOS and I cannot reproduce this.

niccokunzmann@Niccos-MacBook-Pro recurring-ical-events % tox -re py38
py38: remove tox env folder /Users/niccokunzmann/recurring-ical-events/.tox/py38
py38: install_deps> python -I -m pip install -r /Users/niccokunzmann/recurring-ical-events/requirements.txt -r /Users/niccokunzmann/recurring-ical-events/test-requirements.txt --pre
py38: commands[0]> pytest --basetemp=/Users/niccokunzmann/recurring-ical-events/.tox/py38/tmp
======================================================== test session starts ========================================================
platform darwin -- Python 3.8.2, pytest-8.3.2, pluggy-1.5.0
cachedir: .tox/py38/.pytest_cache
rootdir: /Users/niccokunzmann/recurring-ical-events
configfile: pyproject.toml
plugins: cov-5.0.0
collected 1272 items                                                                                                                

test/test_after.py ....................................................................ss                                     [  5%]
test/test_at_function.py ............................................                                                         [  8%]
test/test_bad_rrule_format.py ....                                                                                            [  9%]
test/test_convert_inputs.py ........................                                                                          [ 11%]
test/test_daylight_saving_time.py ........                                                                                    [ 11%]
test/test_deleted_entries.py ........                                                                                         [ 12%]
test/test_duration.py ................................................................                                        [ 17%]
test/test_end_before_start_event.py ....                                                                                      [ 17%]
test/test_event_values_and_edits.py .............................................                                             [ 21%]
test/test_examples.py ....................................................................................................... [ 29%]
.............                                                                                                                 [ 30%]
test/test_issue_101_select_components.py ....................................................................                 [ 35%]
test/test_issue_107_omitting_last_event.py ....                                                                               [ 36%]
test/test_issue_113_period_in_rdate.py ........                                                                               [ 36%]
test/test_issue_117_until_before_dtstart.py ....                                                                              [ 37%]
test/test_issue_128_only_first_event.py ..........                                                                            [ 37%]
test/test_issue_148_ignored_exdate_in_higher_sequence.py ................................................................     [ 42%]
test/test_issue_15.py ....                                                                                                    [ 43%]
test/test_issue_151_macos_linux_difference.py FFFF                                                                            [ 43%]
test/test_issue_18_cancel_status.py ........................                                                                  [ 45%]
test/test_issue_20_exdate_ignored.py ........................................................................                 [ 51%]
test/test_issue_27.py ........                                                                                                [ 51%]
test/test_issue_28_timezone_with_z.py ....                                                                                    [ 51%]
test/test_issue_36_recurrence_id_format.py ............                                                                       [ 52%]
test/test_issue_4.py ....................                                                                                     [ 54%]
test/test_issue_44_day_event_reported_twice.py ............................                                                   [ 56%]
test/test_issue_48_daylight.py ........................                                                                       [ 58%]
test/test_issue_48_dst.py ............                                                                                        [ 59%]
test/test_issue_61.py ....                                                                                                    [ 59%]
test/test_issue_62_moved_event.py ................................................                                            [ 63%]
test/test_issue_6_copy_subcomponents.py ........                                                                              [ 64%]
test/test_issue_7_datetime_and_date_start_stop.py ....................                                                        [ 65%]
test/test_issue_86_x_wr_timezone_but_no_tzid_in_dt.py ....                                                                    [ 66%]
test/test_issue_97_simple_recurrent_todos_and_journals.py ....................................                                [ 68%]
test/test_keep_recurrence_attributes.py ............                                                                          [ 69%]
test/test_multiple_rrule.py ....                                                                                              [ 70%]
test/test_properties.py ........................                                                                              [ 72%]
test/test_rdate.py ................................................................                                           [ 77%]
test/test_readme.py .                                                                                                         [ 77%]
test/test_recurrence_sequence_number.py ....                                                                                  [ 77%]
test/test_repeated_properties.py ....                                                                                         [ 77%]
test/test_repetitions_do_not_change.py ....................                                                                   [ 79%]
test/test_simple_recurrent_events.py ....................................                                                     [ 82%]
test/test_single_events.py ....................                                                                               [ 83%]
test/test_skip_bad_events.py ........                                                                                         [ 84%]
test/test_time_arguments.py ........                                                                                          [ 85%]
test/test_time_span_contains_event.py .............................................                                           [ 88%]
test/test_time_zones_differ.py ....................................                                                           [ 91%]
test/test_x_wr_timezone.py .........................................................                                          [ 95%]
test/test_zero_size_events.py ....................................                                                            [ 98%]
test/test_zoneinfo_issue_57.py ................                                                                               [100%]

============================================================= FAILURES ==============================================================
_________________________________________ test_count_events_from_issue[Calendars-use_pytz] __________________________________________

calendars = Calendars(use_pytz)

    def test_count_events_from_issue(calendars):
        """The number of events seems to differ from OS to OS."""
    
        start_time = datetime.fromtimestamp(1722564000, timezone.utc)
        end_time = datetime.fromtimestamp(1722567600, timezone.utc)
        events = calendars.issue_151_macos_linux_difference.between(start_time, end_time)
        for event in events:
            print(event["UID"], event["DTSTART"], event["SUMMARY"])
>       assert len(events) == 4
E       assert 0 == 4
E        +  where 0 = len([])

test/test_issue_151_macos_linux_difference.py:17: AssertionError
_______________________________________ test_count_events_from_issue[Calendars-use_zoneinfo] ________________________________________

calendars = Calendars(use_zoneinfo)

    def test_count_events_from_issue(calendars):
        """The number of events seems to differ from OS to OS."""
    
        start_time = datetime.fromtimestamp(1722564000, timezone.utc)
        end_time = datetime.fromtimestamp(1722567600, timezone.utc)
        events = calendars.issue_151_macos_linux_difference.between(start_time, end_time)
        for event in events:
            print(event["UID"], event["DTSTART"], event["SUMMARY"])
>       assert len(events) == 4
E       assert 0 == 4
E        +  where 0 = len([])

test/test_issue_151_macos_linux_difference.py:17: AssertionError
___________________________________ test_count_events_from_issue[ReversedCalendars-use_zoneinfo] ____________________________________

calendars = ReversedCalendars(use_zoneinfo)

    def test_count_events_from_issue(calendars):
        """The number of events seems to differ from OS to OS."""
    
        start_time = datetime.fromtimestamp(1722564000, timezone.utc)
        end_time = datetime.fromtimestamp(1722567600, timezone.utc)
        events = calendars.issue_151_macos_linux_difference.between(start_time, end_time)
        for event in events:
            print(event["UID"], event["DTSTART"], event["SUMMARY"])
>       assert len(events) == 4
E       assert 0 == 4
E        +  where 0 = len([])

test/test_issue_151_macos_linux_difference.py:17: AssertionError
_____________________________________ test_count_events_from_issue[ReversedCalendars-use_pytz] ______________________________________

calendars = ReversedCalendars(use_pytz)

    def test_count_events_from_issue(calendars):
        """The number of events seems to differ from OS to OS."""
    
        start_time = datetime.fromtimestamp(1722564000, timezone.utc)
        end_time = datetime.fromtimestamp(1722567600, timezone.utc)
        events = calendars.issue_151_macos_linux_difference.between(start_time, end_time)
        for event in events:
            print(event["UID"], event["DTSTART"], event["SUMMARY"])
>       assert len(events) == 4
E       assert 0 == 4
E        +  where 0 = len([])

test/test_issue_151_macos_linux_difference.py:17: AssertionError
====================================================== short test summary info ======================================================
FAILED test/test_issue_151_macos_linux_difference.py::test_count_events_from_issue[Calendars-use_pytz] - assert 0 == 4
FAILED test/test_issue_151_macos_linux_difference.py::test_count_events_from_issue[Calendars-use_zoneinfo] - assert 0 == 4
FAILED test/test_issue_151_macos_linux_difference.py::test_count_events_from_issue[ReversedCalendars-use_zoneinfo] - assert 0 == 4
FAILED test/test_issue_151_macos_linux_difference.py::test_count_events_from_issue[ReversedCalendars-use_pytz] - assert 0 == 4
============================================ 4 failed, 1266 passed, 2 skipped in 20.76s =============================================
py38: exit 1 (25.50 seconds) /Users/niccokunzmann/recurring-ical-events> pytest --basetemp=/Users/niccokunzmann/recurring-ical-events/.tox/py38/tmp pid=780
  py38: FAIL code 1 (40.53=setup[15.03]+cmd[25.50] seconds)
  evaluation failed :( (40.65 seconds)
niccokunzmann@Niccos-MacBook-Pro recurring-ical-events %     

This is in #153.

@niccokunzmann
Copy link
Owner

Could you run this command in the environment? pip list

@niccokunzmann
Copy link
Owner

Also, could you follow the setup documentation and run the tests on your machine? I am wondering where and how this problem occurs. I cannot reproduce it at the moment.

@sanieldalib
Copy link
Author

sanieldalib commented Aug 8, 2024

@niccokunzmann I dug into this some more. It appears that the issue is a regression between versions 2.2.2 and 2.2.3. The event appears on any OS in version 2.2.2, but once I upgrade to 2.2.3, the event no longer appears.

I initially thought this was an issue with the OS as my Docker image, was pulling the latest patch version, which my venv on macOS already had v2.2.2 installed. I apologize for the red herring here.

@niccokunzmann
Copy link
Owner

No problem, this can happen. Would you say that the issue is still open and needs work on it? What as required to close it?

1 similar comment
@niccokunzmann
Copy link
Owner

No problem, this can happen. Would you say that the issue is still open and needs work on it? What as required to close it?

@sanieldalib
Copy link
Author

Yes. There is still a regression from 2.2.2 -> 2.2.3. The test you wrote should return 1 event, not 0.

@sanieldalib sanieldalib changed the title bug: Difference in behavior between macOS and Linux bug: Regression in v2.2.3 for annual events Aug 14, 2024
@niccokunzmann
Copy link
Owner

niccokunzmann commented Aug 20, 2024

This is fixed in v3.0.0.

This took me 12 hours - 720$ - as an invitation if you or someone else sees the importance to contribute to my FOSS work :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants