Unit Test coverage for @carbon/react components ☂️

[guidari]

The goal of this workstream is to increase confidence in each component, not just to increase coverage. It’s better to have 80% coverage with high-quality tests than 100% coverage with low-quality tests.

If you want to tackle any research, just create an issue to keep your notes from the research. Would be nice to look for repos that are using that on their PRs, to take as an example.

Important

Read the kickoff notes below for implementation instructions and tips

Add code coverage CI check

Preview Give feedback

1. Research and decide code coverage tooling #17123
   type: infrastructure 🤖 +1
   

   
2. Implement code coverage tooling #17124
   type: infrastructure 🤖 +1
   

   
3. Test Coverage: Enforce the % threshold, make the status check required #17985
   +0
   

High priority components under 80% coverage

Preview Give feedback

1. Test coverage: Organize and define steps to take for each component #17414
   afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
   
2. Add test coverage for DefinitionTooltip #17415
   afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
   

   
3. Increase ContainedList unit test coverage #17470
   afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
   
4. Increase ListBox/next/ListBoxSelection unit test coverage #17471
   afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
   
5. Increase ProgressIndicator.Skeleton unit test coverage #17472
   afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
   
6. Increase Layout unit test coverage #17473
   afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
   
7. Add test coverage for Toggle.Skeleton #17429
   +0
   
8. Increase RadioTile unit test coverage #17474
   afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
   
9. Increase TextInput.Skeleton unit test coverage #17475
   afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
   
10. Increase ControlledPasswordInput unit test coverage #17476
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
11. Increase Dropdown unit test coverage #17477
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
12. Increase FluidTextArea.Skeleton unit test coverage #17478
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
13. Increase FilterableMultiSelect unit test coverage #17479
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
14. Increase MultiSelect unit test coverage #17480
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
15. Increase Tile unit test coverage #17481
    package: @carbon/react role: dev 🤖 type: infrastructure 🤖 version: 11 +4
    
16. Increase Theme unit test coverage #17482
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
17. Increase FileUploader unit test coverage #17483
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
18. Increase FileUploaderDropContainer unit test coverage #17484
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
19. Increase ModalWrapper unit test coverage #17485
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
20. Increase FluidComboBox.Skeleton unit test coverage #17486
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
21. Increase FluidSelect.Skeleton unit test coverage #17487
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
22. Increase HeaderContainer unit test coverage #17488
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
23. Increase HeaderMenu unit test coverage #17489
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
24. Increase HeadePanel unit test coverage #17490
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
25. Increase SideNav unit test coverage #17491
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
26. Increase SideNavMenu unit test coverage #17492
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
27. Increase Switcher unit test coverage #17493
    package: @carbon/react role: dev 🤖 type: infrastructure 🤖 version: 11 +4
    
28. Increase SwitcherItem unit test coverage #17494
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
29. Increase Pagination.Skeleton unit test coverage #17495
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
30. Increase Pagination unit test coverage #17496
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
31. Increase TimePicker unit test coverage #17497
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
32. Increase Toggletip unit test coverage #17498
    package: @carbon/react role: dev 🤖 type: infrastructure 🤖 version: 11 +4
    
33. Increase OverflowMenu unit test coverage #17499
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
34. Add test coverage for Tabs #17441
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
35. Add test coverage for Tabs.Skeleton #17442
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
36. Increase Slug unit test coverage #17500
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
37. Increase TreeNode unit test coverage #17501
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
38. Increase TreeView unit test coverage #17502
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
39. Increase Tag unit test coverage #17503
    package: @carbon/react role: dev 🤖 type: infrastructure 🤖 version: 11 +4
    
40. Increase DismissibleTag unit test coverage #17504
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
41. Increase OperationalTag unit test coverage #17505
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
42. Add test coverage for SelectableTag #17437
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
43. Add test coverage for Tag.Skeleton #17438
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
44. Add test coverage for ChatButton.Skeleton #17432
    +0
    
45. Add test coverage for ChatButton #17431
    +0
    
46. Increase ToggleSmall.Skeleton unit test coverage #17506
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
47. Increase OverflowMenuV2 unit test coverage #17507
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
48. Increase StructuredList unit test coverage #17508
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
49. Increase StructuredList.Skeleton unit test coverage #17509
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
50. Increase FluidDatePicker.Skeleton unit test coverage #17510
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
51. Increase FluidDropdown.Skeleton unit test coverage #17511
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
52. Increase FluidNumberInput.Skeleton unit test coverage #17512
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
53. Increase FluidSearch.Skeleton unit test coverage #17513
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
54. Increase FluidPasswordInput unit test coverage #17514
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
55. Increase FluidTextInput.Skeleton unit test coverage #17515
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
56. Increase FluidTimePicker.Skeleton unit test coverage #17516
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
57. Increase FluidTimePicker unit test coverage #17517
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
58. Increase FluidMultiSelect.Skeleton unit test coverage #17518
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    
59. Increase FluidMultiSelect unit test coverage #17519
    afrohacks good first issue 👋 hacktoberfest package: @carbon/react role: dev 🤖 status: help wanted 👐 type: infrastructure 🤖 version: 11 +8
    

Flaky tests to fix

Preview Give feedback

1. Menu-test.avt.e2e.js:25:3 › @avt Menu › @avt-keyboard-nav Menu
2. Slider › behaves as expected - Two Handle Slider Component API › Error handling, expected behavior from event handlers › gracefully tolerates empty event passed to _onDrag

[tay1orjones]

I'd love to see this include adding something like a codecov ci check on our PRs.

• https://about.codecov.io/product/features/
• https://docs.codecov.com/docs/flags
• https://www.curtiscode.dev/post/tools/codecov-monorepo/

[guidari]

Related issue: Research and implement a testing strategy for types provided through @carbon/react
#16361

[tay1orjones]

... increase confidence in each component, not just to increase coverage

One example of this that I just ran into that we should consider:

Tests covering callback props should always assert both how many times it's called and what it's called with

expect(onChange).toHaveBeenCalledTimes(1);
expect(onChange).toHaveBeenCalledWith(arg1, arg2, ...);

Numerous times we've fixed bugs relating to callbacks being called multiple times or not at all. We've also had plenty of times where the contents of the callback args change unintentionally. Let's make sure we always test both.

There's probably not a way to automatically enforce this, but it's a style/practice we should adopt everywhere imo

[tay1orjones]

Additionally we should look at any playwright test that is

• evaluating as "slow" locally
• using .locator() directly, as this can produce flakiness

We recently found a case were the tile avt tests were actually failing locally, but passing in CI. It seems the reason was that the tests were timing out and not producing an error in ci. Locally they would fail. We should evaluate to ensure all the tests we have pass individually locally.

[elycheea]

@tay1orjones The timing of your comment was impeccable. @ariellalgilmore and I were investigating some flakiness on our end as you wrote this. 😹

[tay1orjones]

Kickoff notes

Collecting coverage

• Entire repo - yarn test --collectCoverage from the root
• Collect coverage for a specific component (will update the html report)
  • yarn test packages/react/src/components/Tooltip/__tests__/DefinitionTooltip-test.js --collectCoverage
• Get coverage for a specific component in your terminal
  • yarn test packages/react/src/components/Tooltip/__tests__/DefinitionTooltip-test.js --coverage --collectCoverageFrom=packages/react/src/components/Tooltip/DefinitionTooltip.tsx

Reading the coverage report

• The report is placed in /coverage - open in finder and navigate to the .html file and open it
• Red = lines not covered
• Yellow - conditional branch not covered
• Green left margin e.g. 85x is how many times the line was hit across all the tests. This number can largely be ignored

Other notes

• Default of 3 for effort on these issues

Tips for test writing

• 100% coverage is the goal, but despite the number the intention is for every prop in the component api to be covered by a logical test that ensures the prop functionality behaves as expected.
• ...

[ariellalgilmore]

adding https://codewithhugo.com/jest-exclude-coverage/ as a resource. In Jest 100% coverage might not be possible because it's not a "real" browser setting. For example, when testing for overflow the tests will never pick it up. The linked guide explains how to ignore or exclude files when necessary