Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check for duplicate crate dependencies in CI #2986

Merged
merged 7 commits into from
Nov 1, 2021

Conversation

teor2345
Copy link
Contributor

@teor2345 teor2345 commented Nov 1, 2021

Motivation

When we add duplicate dependencies, Zebra compiles slower, and its binaries will be larger.
This slows down CI, and can fill up CI disks.

In some cases, duplicate dependencies can result in hard-to-diagnose bugs in Zebra.
This happens because code expects a single instance of some structs, but there are multiple instances running.

Duplicate dependencies and unexpected crate sources can also be a security risk.

Scheduling

This is unexpected work in Sprint 21, to make sure that tokio is upgraded correctly.

It should be merged before the tokio roll-up PR.

Solution

This PR detects new duplicate dependencies and new unexpected crate sources.
Current duplicates and sources are allowed. I opened separate tickets to remove them.

CI:

  • Add a cargo deny check bans step to CI
  • Add a cargo deny check sources step to CI
  • Make sure Cargo.lock is up to date in CI, because it is used by these checks

Configuration:

  • Allow the current set of duplicates and sources in the deny.toml config
    (so this PR doesn't actually need to change any dependencies)

Review

@jvff should review this PR.

It blocks the tokio roll-up PR.

Reviewer Checklist

  • CI passes
  • deny.toml makes sense
  • new GitHub actions make sense

Follow Up Work

Tickets:

@teor2345 teor2345 added A-dependencies Area: Dependency file updates A-devops Area: Pipelines, CI/CD and Dockerfiles C-enhancement Category: This is an improvement P-Medium labels Nov 1, 2021
@teor2345 teor2345 added this to the 2021 Sprint 21 milestone Nov 1, 2021
@teor2345 teor2345 requested a review from jvff November 1, 2021 00:04
@teor2345 teor2345 self-assigned this Nov 1, 2021
@teor2345 teor2345 changed the title Check for new duplicate crate dependencies in CI Check for duplicate crate dependencies in CI Nov 1, 2021
Also cleanup trailing whitespace.
Allow the current set of duplicates and sources,
with references to the tickets that will fix them.
Also check for:
- unexpected crate sources
- outdated Cargo.lock
  (required for accurate duplicate and source checks)
@teor2345 teor2345 force-pushed the duplicate-dependencies-ci branch from 5f88b00 to 3539dc5 Compare November 1, 2021 01:56
@teor2345 teor2345 mentioned this pull request Nov 1, 2021
3 tasks
@teor2345 teor2345 enabled auto-merge (squash) November 1, 2021 20:32
@teor2345 teor2345 merged commit 9963471 into main Nov 1, 2021
@teor2345 teor2345 deleted the duplicate-dependencies-ci branch November 1, 2021 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-dependencies Area: Dependency file updates A-devops Area: Pipelines, CI/CD and Dockerfiles C-enhancement Category: This is an improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants