Skip to content

Visual regression

Visual regression #24

GitHub Actions / Test Report failed Jun 29, 2024 in 1s

2 passed, 8 failed and 0 skipped

Tests failed

❌ e2e/report.xml

10 tests were completed in 106s with 2 passed, 8 failed and 0 skipped.

Test suite Passed Failed Skipped Time
TestCafe Tests: Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04 2✅ 8❌ 106s

❌ TestCafe Tests: Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04

Filters
  ❌ Name (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-1)
	1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
General
  ❌ Hotkey help (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-2)
	1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
Tasks
  ❌ Adding a task (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-3)
	1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
  ❌ Starting and stopping the task (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-4)
	1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
  ❌ Changing task status (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-5)
	1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
  ❌ Renaming the task (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-6)
	1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
  ✅ Deleting the task
  ❌ Export and import (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-8)
	1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
  ❌ Editing a session (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-9)
	1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
  ✅ Moving a session

Annotations

Check failure on line 0 in e2e/report.xml

See this annotation in the file changed.

@github-actions github-actions / Test Report

TestCafe Tests: Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04 ► Filters ► Name (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-1)

Failed test found in:
  e2e/report.xml
Error:
        1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
Raw output
    
      1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }

         + expected - actual

          {
         -  diff: '/home/runner/work/timer-5/timer-5/e2e/visual-regression-screenshots/[Filters] Name - empty filter.diff.png'
         -  match: false
         +  match: true
          }


         Browser: Chrome 126.0.0.0 / Ubuntu 22.04
         Screenshot: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-1/run-3/Chrome_126.0.0.0_Ubuntu_22.04/errors/1.png

            11 |  await screenTasks.addTask('FOO');
            12 |  await screenTasks.addTask('BAR');
            13 |  await screenTasks.addTask('BAZ');
            14 |  // Send "ctrl+f" keys
            15 |  await t.pressKey('ctrl+f');
          > 16 |  await t.expect(await comparePageScreenshot('empty filter')).eql(VISUAL_REGRESSION_OK);
            17 |  // Fill in the "Name filter", in lowercase
            18 |  await t.typeText(screenTasks.filter.name.input, 'game');
            19 |  // Assert current task is still opened
            20 |  await t.expect(screenTask.name.textContent).eql('BAZ');
            21 |  // Assert the displayed tasks match the filter

            at <anonymous> (/home/runner/work/timer-5/timer-5/e2e/fixtures/filters.ts:16:63)
            at fulfilled (/home/runner/work/timer-5/timer-5/e2e/fixtures/filters.ts:5:58)
    
    

Check failure on line 0 in e2e/report.xml

See this annotation in the file changed.

@github-actions github-actions / Test Report

TestCafe Tests: Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04 ► General ► Hotkey help (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-...

Failed test found in:
  e2e/report.xml
Error:
        1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
Raw output
    
      1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }

         + expected - actual

          {
         -  diff: '/home/runner/work/timer-5/timer-5/e2e/visual-regression-screenshots/[General] Hotkey help - only tasks.diff.png'
         -  match: false
         +  match: true
          }


         Browser: Chrome 126.0.0.0 / Ubuntu 22.04
         Screenshot: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-2/run-3/Chrome_126.0.0.0_Ubuntu_22.04/errors/1.png

             7 |test('Hotkey help', async (t) => {
             8 |  // Send "shift+?", assert the hotkey cheatsheet is opened
             9 |  await t.pressKey('shift+?');
            10 |  // Assert it displays all hotkeys, in english only
            11 |  await t.expect(dialogHotkeyCheatsheet.descriptions.count).eql(8);
          > 12 |  await t.expect(await comparePageScreenshot('only tasks')).eql(VISUAL_REGRESSION_OK);
            13 |  // Assert "Close" closes the dialog
            14 |  await t.click(dialogHotkeyCheatsheet.buttonDismiss);
            15 |  await t.expect(dialogHotkeyCheatsheet.dialog.exists).notOk();
            16 |  // Open a session, assert session hotkeys are displayed
            17 |  await screenTasks.addTask('Keys');

            at <anonymous> (/home/runner/work/timer-5/timer-5/e2e/fixtures/general.ts:12:61)
            at fulfilled (/home/runner/work/timer-5/timer-5/e2e/fixtures/general.ts:5:58)
    
    

Check failure on line 0 in e2e/report.xml

See this annotation in the file changed.

@github-actions github-actions / Test Report

TestCafe Tests: Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04 ► Tasks ► Adding a task (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-...

Failed test found in:
  e2e/report.xml
Error:
        1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
Raw output
    
      1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }

         + expected - actual

          {
         -  diff: '/home/runner/work/timer-5/timer-5/e2e/visual-regression-screenshots/[Tasks] Adding a task - just opened.diff.png'
         -  match: false
         +  match: true
          }


         Browser: Chrome 126.0.0.0 / Ubuntu 22.04
         Screenshot: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-3/run-3/Chrome_126.0.0.0_Ubuntu_22.04/errors/1.png

            16 |
            17 |test('Adding a task', async (t) => {
            18 |  // Click the empty state "Add task" button
            19 |  await t.click(screenTasks.emptyStateAddTaskButton);
            20 |  // Unfocus the input, asset validation is show
          > 21 |  await t.expect(await comparePageScreenshot('just opened')).eql(VISUAL_REGRESSION_OK);
            22 |  await t.pressKey('tab').expect(dialogPrompt.validationError.textContent).contains('Value is required');
            23 |  await t.expect(await comparePageScreenshot('invalid')).eql(VISUAL_REGRESSION_OK);
            24 |  // Submit the dialog with "Enter", assert form is not submitted
            25 |  await t.click(dialogPrompt.input).pressKey('enter');
            26 |  await t.expect(dialogPrompt.title.textContent).contains('Create task');

            at <anonymous> (/home/runner/work/timer-5/timer-5/e2e/fixtures/tasks.ts:21:62)
            at fulfilled (/home/runner/work/timer-5/timer-5/e2e/fixtures/tasks.ts:5:58)
    
    

Check failure on line 0 in e2e/report.xml

See this annotation in the file changed.

@github-actions github-actions / Test Report

TestCafe Tests: Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04 ► Tasks ► Starting and stopping the task (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/...

Failed test found in:
  e2e/report.xml
Error:
        1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
Raw output
    
      1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }

         + expected - actual

          {
         -  diff: '/home/runner/work/timer-5/timer-5/e2e/visual-regression-screenshots/[Tasks] Starting and stopping the task - active menu.diff.png'
         -  match: false
         +  match: true
          }


         Browser: Chrome 126.0.0.0 / Ubuntu 22.04
         Screenshot: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-4/run-3/Chrome_126.0.0.0_Ubuntu_22.04/errors/1.png

            62 |  await t.expect(app.favicon.getAttribute('href')).contains('/favicon.svg');
            63 |  // Assert the task is marked as active/not-running in the task list item, task list item context action, task view title and task view context action
            64 |  await t.expect(screenTasks.taskStateIcon.getAttribute('data-mat-icon-name')).contains('play_circle');
            65 |  await t.click(screenTasks.buttonTaskAction.nth(1));
            66 |  await t.expect(menuTaskActions.selectorState.textContent).contains('State: Active');
          > 67 |  await t.expect(await comparePageScreenshot('active menu')).eql(VISUAL_REGRESSION_OK);
            68 |  await t.pressKey('esc');
            69 |  await t.expect(screenTask.stateIcon.getAttribute('data-mat-icon-name')).contains('play_circle');
            70 |  await t.click(screenTask.buttonTaskAction);
            71 |  await t.expect(menuTaskActions.selectorState.textContent).contains('Active');
            72 |  await t.pressKey('esc');

            at <anonymous> (/home/runner/work/timer-5/timer-5/e2e/fixtures/tasks.ts:67:62)
            at fulfilled (/home/runner/work/timer-5/timer-5/e2e/fixtures/tasks.ts:5:58)
    
    

Check failure on line 0 in e2e/report.xml

See this annotation in the file changed.

@github-actions github-actions / Test Report

TestCafe Tests: Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04 ► Tasks ► Changing task status (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29...

Failed test found in:
  e2e/report.xml
Error:
        1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
Raw output
    
      1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }

         + expected - actual

          {
         -  diff: '/home/runner/work/timer-5/timer-5/e2e/visual-regression-screenshots/[Tasks] Changing task status - active.diff.png'
         -  match: false
         +  match: true
          }


         Browser: Chrome 126.0.0.0 / Ubuntu 22.04
         Screenshot: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-5/run-3/Chrome_126.0.0.0_Ubuntu_22.04/errors/1.png

            121 |    .click(screenTasks.emptyStateAddTaskButton)
            122 |    .typeText(dialogPrompt.input, 'Task')
            123 |    .click(dialogPrompt.buttonSubmit);
            124 |  // Cycle task statuses Active->Finished->Dropped->Active using task list context action, assert the status changes
            125 |  await t.click(screenTasks.buttonTaskAction).click(menuTaskActions.selectorState);
          > 126 |  await t.expect(await comparePageScreenshot('active')).eql(VISUAL_REGRESSION_OK);
            127 |  await t.click(menuTaskActions.optionFinished).pressKey('esc');
            128 |  await t.expect(screenTask.stateIcon.getAttribute('data-mat-icon-name')).eql('check_circle');
            129 |  await t.expect(await comparePageScreenshot('is finished')).eql(VISUAL_REGRESSION_OK);
            130 |  await t.click(app.buttonFinishedTasks);
            131 |  await t.expect(screenTasks.taskStateIcon.getAttribute('data-mat-icon-name')).eql('check_circle');

            at <anonymous> (/home/runner/work/timer-5/timer-5/e2e/fixtures/tasks.ts:126:57)
            at fulfilled (/home/runner/work/timer-5/timer-5/e2e/fixtures/tasks.ts:5:58)
    
    

Check failure on line 0 in e2e/report.xml

See this annotation in the file changed.

@github-actions github-actions / Test Report

TestCafe Tests: Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04 ► Tasks ► Renaming the task (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07...

Failed test found in:
  e2e/report.xml
Error:
        1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
Raw output
    
      1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }

         + expected - actual

          {
         -  diff: '/home/runner/work/timer-5/timer-5/e2e/visual-regression-screenshots/[Tasks] Renaming the task - context menu.diff.png'
         -  match: false
         +  match: true
          }


         Browser: Chrome 126.0.0.0 / Ubuntu 22.04
         Screenshot: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-6/run-3/Chrome_126.0.0.0_Ubuntu_22.04/errors/1.png

            206 |    .click(screenTasks.emptyStateAddTaskButton)
            207 |    .typeText(dialogPrompt.input, 'Task')
            208 |    .click(dialogPrompt.buttonSubmit)
            209 |    .click(screenTasks.buttonTaskAction);
            210 |  await t.expect(screenTask.screen.exists).ok();
          > 211 |  await t.expect(await comparePageScreenshot('context menu')).eql(VISUAL_REGRESSION_OK);
            212 |  await t.click(menuTaskActions.buttonRename);
            213 |  // Erase the value, assert a validation message is shown
            214 |  await t.expect(await comparePageScreenshot('just opened')).eql(VISUAL_REGRESSION_OK);
            215 |  await t.click(dialogPrompt.input).pressKey('ctrl+a').pressKey('delete').pressKey('tab');
            216 |  await t.expect(dialogPrompt.validationError.textContent).eql('Value is required');

            at <anonymous> (/home/runner/work/timer-5/timer-5/e2e/fixtures/tasks.ts:211:63)
            at fulfilled (/home/runner/work/timer-5/timer-5/e2e/fixtures/tasks.ts:5:58)
    
    

Check failure on line 0 in e2e/report.xml

See this annotation in the file changed.

@github-actions github-actions / Test Report

TestCafe Tests: Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04 ► Tasks ► Export and import (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07...

Failed test found in:
  e2e/report.xml
Error:
        1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
Raw output
    
      1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }

         + expected - actual

          {
         -  diff: '/home/runner/work/timer-5/timer-5/e2e/visual-regression-screenshots/[Tasks] Export and import - menu.diff.png'
         -  match: false
         +  match: true
          }


         Browser: Chrome 126.0.0.0 / Ubuntu 22.04
         Screenshot: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-8/run-3/Chrome_126.0.0.0_Ubuntu_22.04/errors/1.png

            279 |    ],
            280 |  };
            281 |  await writeFile(`e2e/downloads/${testId}/data.json`, JSON.stringify(referenceData, null, '  '));
            282 |  // Import the data
            283 |  await t.click(app.buttonImportExport);
          > 284 |  await t.expect(await comparePageScreenshot('menu')).eql(VISUAL_REGRESSION_OK);
            285 |  await t.setFilesToUpload(app.inputImport, [`../downloads/${testId}/data.json`]);
            286 |  // Assert the tasks are added
            287 |  await t.expect(screenTasks.taskItem.count).eql(3);
            288 |  await t.expect(screenTasks.taskName.nth(0).textContent).eql('Task 1');
            289 |  await t.expect(screenTasks.taskName.nth(1).textContent).eql('Task 2');

            at <anonymous> (/home/runner/work/timer-5/timer-5/e2e/fixtures/tasks.ts:284:55)
            at fulfilled (/home/runner/work/timer-5/timer-5/e2e/fixtures/tasks.ts:5:58)
    
    

Check failure on line 0 in e2e/report.xml

See this annotation in the file changed.

@github-actions github-actions / Test Report

TestCafe Tests: Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04, Chrome 126.0.0.0 / Ubuntu 22.04 ► Tasks ► Editing a session (screenshots: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07...

Failed test found in:
  e2e/report.xml
Error:
        1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }
Raw output
    
      1) AssertionError: expected { Object (match, diff) } to deeply equal { match: true }

         + expected - actual

          {
         -  diff: '/home/runner/work/timer-5/timer-5/e2e/visual-regression-screenshots/[Tasks] Editing a session - running.diff.png'
         -  match: false
         +  match: true
          }


         Browser: Chrome 126.0.0.0 / Ubuntu 22.04
         Screenshot: /home/runner/work/timer-5/timer-5/e2e/screenshots/2024-06-29_07-47-36/test-9/run-3/Chrome_126.0.0.0_Ubuntu_22.04/errors/1.png

            311 |  // Change a running task session start time, assert the total changes
            312 |  await t.expect(screenTask.sessionDuration.nth(0).textContent).eql(' 4 s', { timeout: 5_000 });
            313 |  await t.expect(screenTask.taskDuration.textContent).eql(' 7 s');
            314 |  await t
            315 |    .expect(await comparePageScreenshot('running', { ignore: [duration, screenTask.sessionStart] }))
          > 316 |    .eql(VISUAL_REGRESSION_OK);
            317 |  await t.click(screenTask.buttonSessionAction).click(screenTask.menuSession.buttonEdit);
            318 |  const now = utcDateToLocalDate(new Date());
            319 |  now.setHours(now.getHours() - 3);
            320 |  await dialogEditSession.setStart(now);
            321 |  await t

            at <anonymous> (/home/runner/work/timer-5/timer-5/e2e/fixtures/tasks.ts:316:6)
            at fulfilled (/home/runner/work/timer-5/timer-5/e2e/fixtures/tasks.ts:5:58)