Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: a linter to deprecate imported modules #20987

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

adomani
Copy link
Collaborator

@adomani adomani commented Jan 23, 2025

This is still a prototype, but writing

import Bar1
import Bar2

deprecate_module since yyyy-mm-dd

in file A.lean means that any file that imports A will have import A flagged with a suggestion to import Bar1 and Bar2 instead.

Zulip discussion


Open in Gitpod

@github-actions github-actions bot added the t-linter Linter label Jan 23, 2025
Copy link

github-actions bot commented Jan 23, 2025

PR summary 5191403211

Import changes exceeding 2%

% File
+3.57% Mathlib.Tactic.Linter

Import changes for modified files

No significant changes to the import graph

Import changes for all files
Files Import difference
There are 4850 files with changed transitive imports taking up over 207482 characters: this is too many to display!
You can run scripts/import_trans_difference.sh all locally to see the whole output.

Declarations diff

+ addModuleDeprecation
+ deprecateModuleLinter

You can run this locally as follows
## summary with just the declaration names:
./scripts/declarations_diff.sh <optional_commit>

## more verbose report:
./scripts/declarations_diff.sh long <optional_commit>

The doc-module for script/declarations_diff.sh contains some details about this script.


No changes to technical debt.

You can run this locally as

./scripts/technical-debt-metrics.sh pr_summary
  • The relative value is the weighted sum of the differences with weight given by the inverse of the current value of the statistic.
  • The absolute value is the relative value divided by the total sum of the inverses of the current values (i.e. the weighted average of the differences).

@eric-wieser
Copy link
Member

Maybe deprecate_module would be a better name?

@github-actions github-actions bot added the large-import Automatically added label for PRs with a significant increase in transitive imports label Jan 23, 2025
@adomani
Copy link
Collaborator Author

adomani commented Jan 23, 2025

!bench

1 similar comment
@adomani
Copy link
Collaborator Author

adomani commented Jan 23, 2025

!bench

Copy link
Collaborator

@grunweg grunweg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have strong opinions on the environment extension set-up (for lack for time spent with them); the remaining infrastructure looks very good. Find below my usual collection of small polish comments.

`addModuleDeprecation` adds to the `deprecateModuleExt` extension the pair consisting of the
current module name and the array of its direct imports.
It ignores `Init`, since this is a special module that is expected to be imported by all files.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps: "It ignores direct imports in Init"? (I was thinking whether you meant "the current module name in Init"... but that doesn't make much sense.)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote "It ignores the Init import", since usually Init is not explicitly written and it is implicitly imported anyway.

Mathlib/Tactic/Linter/DeprecateModule.lean Outdated Show resolved Hide resolved
Mathlib/Tactic/Linter/DeprecateModule.lean Outdated Show resolved Hide resolved
Mathlib/Tactic/Linter/DeprecateModule.lean Outdated Show resolved Hide resolved
Mathlib/Tactic/Linter/DeprecateModule.lean Outdated Show resolved Hide resolved
Mathlib/Tactic/Linter/DeprecateModule.lean Outdated Show resolved Hide resolved
elab "show_deprecated_modules" : command => do
let directImports := deprecateModuleExt.getState (← getEnv)
logInfo <| "\n".intercalate <|
directImports.fold (init := ["Deprecated modules\n"]) fun nms (i, deps) =>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure there's room for bikeshedding this formatting. I think we can land something and iterate.

Comment on lines +106 to +122
let fm ← getFileMap
let md := (getMainModuleDoc (← getEnv)).toArray
-- The end of the first module doc-string, or the end of the file if there is none.
let firstDocModPos := match md[0]? with
| none => fm.positions.back!
| some doc => fm.ofPosition doc.declarationRange.endPos
unless stx.getTailPos?.getD default ≤ firstDocModPos do
return
-- We try to parse the file up to `firstDocModPos`.
let upToStx ← parseUpToHere firstDocModPos <|> (do
-- If parsing failed, there is some command which is not a module docstring.
-- In that case, we parse until the end of the modules and add an extra `section` afterwards,
-- so we trigger a "no module doc-string" warning.
let fil ← getFileName
let (stx, _) ← Parser.parseHeader { input := fm.source, fileName := fil, fileMap := fm }
parseUpToHere (stx.getTailPos?.getD default) "\nsection")
let importIds := getImportIds upToStx
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These lines are also present in the headerLinter. Can you factor them into a common function?
(I think it's fine if this linter imports the Header linter: didn't we soft-decide that anyway at some point?)

Mathlib/Tactic/Linter/DeprecateModule.lean Outdated Show resolved Hide resolved
Mathlib/Tactic/Linter/DeprecateModule.lean Outdated Show resolved Hide resolved
adomani and others added 3 commits January 23, 2025 17:55
@leanprover-bot
Copy link
Collaborator

Here are the benchmark results for commit c2b50b2.
There were no significant changes against commit 8fac96b.

Copy link

File Instructions %
build +327.990⬝10⁹ (+0.21%)
Mathlib.RingTheory.Kaehler.JacobiZariski +1.42⬝10⁹ (+0.12%)
CI run

Copy link
Collaborator Author

@adomani adomani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Michael!

I acted on all comments, except extracting the code from the header linter, since that would take a little more thought.


/--
The `deprecateModule` linter emits a warning when a file that has been renamed or split
is imported.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a sentence!

`addModuleDeprecation` adds to the `deprecateModuleExt` extension the pair consisting of the
current module name and the array of its direct imports.
It ignores `Init`, since this is a special module that is expected to be imported by all files.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote "It ignores the Init import", since usually Init is not explicitly written and it is implicitly imported anyway.

Mathlib/Tactic/Linter/DeprecateModule.lean Outdated Show resolved Hide resolved
Mathlib/Tactic/Linter/DeprecateModule.lean Outdated Show resolved Hide resolved
Mathlib/Tactic/Linter/DeprecateModule.lean Outdated Show resolved Hide resolved
Mathlib/Tactic/Linter/DeprecateModule.lean Outdated Show resolved Hide resolved
Mathlib/Tactic/Linter/DeprecateModule.lean Outdated Show resolved Hide resolved
Mathlib/Tactic/Linter/DeprecateModule.lean Outdated Show resolved Hide resolved
@adomani
Copy link
Collaborator Author

adomani commented Jan 23, 2025

!bench

@leanprover-bot
Copy link
Collaborator

Here are the benchmark results for commit 5191403.
There were no significant changes against commit 8fac96b.

Copy link

File Instructions %
build +208.707⬝10⁹ (+0.13%)
Mathlib.Data.Array.Lemmas +1.495⬝10⁹ (+50.67%)
CI run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
large-import Automatically added label for PRs with a significant increase in transitive imports t-linter Linter
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants