Check types for privacy #42125

petrochenkov · 2017-05-20T17:03:02Z

This PR implements late post factum checking of type privacy, as opposed to early preventive "private-in-public" checking.
This will allow to turn private-in-public checks into a lint and make them more heuristic-based, and more aligned with what people may expect (e.g. reachability-based behavior).

Types are privacy-checked if they are written explicitly, and also if they are inferred as expression or pattern types.
This PR checks "semantic" types and does it unhygienically, this significantly restricts what macros 2.0 (as implemented in #40847) can do (sorry @jseyfried) - they still can use private names, but can't use private types.
This is the most conservative solution, but hopefully it's temporary and can be relaxed in the future, probably using macro contexts of expression/pattern spans.

Traits are also checked in preparation for trait aliases, which will be able to leak private traits, and macros 2.0 which will be able to leak pretty much anything.

This is a [breaking-change], but the code that is not contrived and can be broken by this patch should be guarded by private_in_public lint. Previous crater run discovered a few abandoned crates that weren't updated since private_in_public has been introduced in 2015.

cc #34537 https://internals.rust-lang.org/t/lang-team-minutes-private-in-public-rules/4504
Fixes #30476
Fixes #33479

cc @nikomatsakis
r? @eddyb

petrochenkov · 2017-05-20T17:07:24Z

src/librustc_privacy/lib.rs

+                None
+            }
+            ty::TyAnon(..) => {
+                // FIXME: How to visit `impl Trait` type?


This needs to be fixed.
IIRC, my attempts to visit it with TypeVisitor either infinitely recursed or ICEd.

Shouldn't impl Trait be able to "export" private types? cc @rust-lang/lang

Yes, in anonymized form, but I'm talking about preventing use of

impl PrivateTrait ^^^^^^^^^^^^

and

impl Trait<PrivateType> ^^^^^^^^^^^

here, not use of private underlying types.

Oh. So using API that involves private types? The private types would show up elsewhere if they were involved. Like, using any method on Trait would end up with that part in its Substs.

petrochenkov · 2017-05-20T17:09:03Z

src/librustc_privacy/lib.rs

+                return true;
+            } else if let ty::TyFnDef(..) = ty.sty {
+                // Inherent static methods don't have self type in substs,
+                // we have to check it additionally.


For some reason Self for static inherent methods is not listed in substs.
IIRC, it's not the first time I see it requiring special treatment for this reason.

It's not because Self is a mere alias in them. The substitutions for impls (inherent or trait impls, but you never get the latter from type-checking, only e.g. trans or MIR inlining can see them) are literally for the <...> in impl<...>.

However, it's (AFAIK) impossible to access an inherent method without one of:

method call, which means there is an expression containing the Self type

T::method which is sugar for <T>::method, and the semantic version of T can be read (see rustdoc and rustc_save_analysis)

petrochenkov · 2017-05-20T17:10:23Z

src/test/compile-fail/private-inferred-type.rs

+    // ext::Z; // ERROR type `ext::Z` is private
+            // ERROR type `fn(u8) -> ext::Z {ext::Z::{{constructor}}}` is private
+    // ext::Z1; // ERROR type `fn(u8) -> ext::Z1 {ext::Z1::{{constructor}}}` is private
+    // ext::Z1::k; // ERROR type `fn() {ext::Z1::k}` is private


Some of these cases cannot be tested until macros 2.0 are merged.

nikomatsakis · 2017-05-24T14:21:42Z

@petrochenkov I'm very excited to see this PR! I have a few questions. I think there are some key questions that we were wrestling with:

Is it legal to define an associated type in a "public impl" that references a private type?
- Update: In this PR, no. Maybe later.
Is it legal to make a pub type Foo = PrivateType? (Presumably it would be illegal to reference it.)
- Update: In this PR, no. Maybe later.
Is it legal to make a pub use self::foo::PrivateThing? (Presumably it would be illegal to reference it.)
- Update: No. No changes planned.
Are these the right questions? Other corner cases or interesting cases I've forgotten about?
- Update: None suggested.

I'm sure I'll know the answers once I skim the code, but I thought it'd be helpful to try and summarize these key points so that people don't have to do so.

nikomatsakis · 2017-05-24T14:22:21Z

Also, I think we should FCP this before we land it. I don't anticipate a lot of back-and-forth but this is basically an insta-stable language change, so I think it makes sense.

petrochenkov · 2017-05-24T15:06:10Z

@nikomatsakis

Is it legal to define an associated type in a "public impl" that references a private type?
Is it legal to make a pub type Foo = PrivateType? (Presumably it would be illegal to reference it.)

I didn't relax the private-in-public checker yet, so these two are still errors. I'm going to send a separate PR with private-in-public changes later. That PR will certainly require an FCP, this one - I'm not so sure.

My preliminary plan is that pub type Foo = PrivateType will be legal, prohibiting it achieves nothing in presence of macros 2.0 (pub macro Foo { PrivateType } does the same thing).
pub associated_type Foo = PrivateType is a thorny case, discussed at length in the internals thread. I think I'll follow the decision from that discussion and keep it an error. It's viable because this check is based on semantic types and traits and not affected by macros. But this is a restriction that can certainly be relaxed in the future if all types are normalized before checking for privacy.

Is it legal to make a pub use self::foo::PrivateThing? (Presumably it would be illegal to reference it.)

This is a separate case unrelated to this PR and its successors, and it will continue being an error.
With current resolution model this is basically an "unused import"/"use importing nothing" error in disguise.
pub([visibility]) [pattern] filters names matching [pattern] using [visibility] and imports what remains. This works for everything - globs, names matching in several namespaces and single names as well. pub use self::foo::PrivateThing just filters away all matching names and imports nothing, then it reports an error instead of being useless.

brson · 2017-05-24T18:32:56Z

Started builds for crater

crater Check types for privacy #42125
- http://github.com/petrochenkov/rust
- before 0bd9e1f
- https://tools.taskcluster.net/task-inspector/#ucQGJfMgSsav2NSo8LUmXA
- after 18d9739c5b897cd6a111402ed585d2bfa30601e0
- https://tools.taskcluster.net/task-inspector/#jfw59nukSfmVTPIIG6oTSw

brson · 2017-05-25T02:49:18Z

I've started the crate builds for crater.

brson · 2017-05-25T17:41:31Z

https://gist.github.com/anonymous/557f11cdeae48a28a39574a729c74be1

nikomatsakis · 2017-05-25T18:10:38Z

Inline (not yet analyzed)

Coverage

9476 crates tested: 7024 working / 2169 broken / 61 regressed / 27 fixed / 195 unknown.

Regressions

There are 37 root regressions
There are 61 regressions

Root regressions, sorted by rank:

rmp-serde-0.13.1 (before) (after)
mysql-11.1.0 (before) (after)
slog-async-2.0.1 (before) (after)
mach-0.1.0 (before) (after)
locale-0.2.1 (before) (after)
measurements-0.4.0 (before) (after)
microbench-0.3.2 (before) (after)
mush-0.1.2 (before) (after)
netmap_sys-0.1.3 (before) (after)
google-deploymentmanager2-1.0.4+20161209 (before) (after)
secret-service-0.3.1 (before) (after)
staticdir-0.3.1 (before) (after)
vk_api-1.0.36 (before) (after)
hcap-0.0.2 (before) (after)
couchdb-0.5.1 (before) (after)
coyoneda-0.5.2 (before) (after)
mango_smoothie-0.2.0 (before) (after)
math-text-transform-0.1.1 (before) (after)
matrixstack-0.1.4 (before) (after)
maybe-owned-0.2.0 (before) (after)
metagener-0.1.0 (before) (after)
mpd-0.0.11 (before) (after)
ndarray-parallel-0.3.0 (before) (after)
nickel_session-0.1.0 (before) (after)
noise-rust-crypto-0.0.3 (before) (after)
nom-operator-0.0.1 (before) (after)
pid_control-0.7.2 (before) (after)
reql-0.0.8 (before) (after)
rgmk-0.2.0 (before) (after)
rules-0.0.2 (before) (after)
rustpather-0.1.0 (before) (after)
slog-perf-0.1.0 (before) (after)
stacktrace-0.2.1 (before) (after)
swindon-0.4.0 (before) (after)
transcriptome_translation-0.1.0 (before) (after)
unbound-0.4.0 (before) (after)
carboxyl_time-0.0.2 (before) (after)

Mark-Simulacrum · 2017-05-25T21:04:16Z

Actual root regressions:

rmp-serde-0.13.1 (before) (after)
mysql-11.1.0 (before) (after)
slog-async-2.0.1 (before) (after)
locale-0.2.1 (before) (after)
secret-service-0.3.1 (before) (after)
couchdb-0.5.1 (before) (after)
coyoneda-0.5.2 (before) (after)
mpd-0.0.11 (before) (after)
reql-0.0.8 (before) (after)
rgmk-0.2.0 (before) (after)
rules-0.0.2 (before) (after)
rustpather-0.1.0 (before) (after)
slog-perf-0.1.0 (before) (after)
swindon-0.4.0 (before) (after)
carboxyl_time-0.0.2 (before) (after) -- actual origin in carboxyl

petrochenkov · 2017-05-26T14:44:39Z

Analysis:
🌳 rmp-serde-0.13.1 - type inference is too smart (#33479)
🍂 coyoneda-0.5.2 - actually morphism-0.4.0, type inference is too smart (#33479)
🌳 mysql-11.1.0 - type inference is too smart (similar to #33479, but with a trait impl)
🍂 locale-0.2.1 - unfixed private_in_public in the same crate, fixed on master but unpublished
🍂 couchdb-0.5.1 - unfixed private_in_public in the same crate, fixed on master but unpublished
🍂 mpd-0.0.11 - unfixed private_in_public in the same crate, fixed on master but unpublished
🍂 rgmk-0.2.0 - unfixed private_in_public in the same crate, fixed on master but unpublished
🍂 rules-0.0.2 - unfixed private_in_public in the same crate, not fixed on master
🍂 rustpather-0.1.0 - unfixed private_in_public in the same crate, the repository doesn't exist anymore
🍂 swindon-0.4.0 - allow(private_in_public) in the same crate, not fixed on master
🌳 slog-async-2.0.1 - allow(private_in_public) in the same project (but other crate, slog)
🍂 slog-perf-0.1.0 - allow(private_in_public) in the same project (but other crate, slog)
🍂 reql-0.0.8 - allow(private_in_public) in another project (slog)
🌳 secret-service-0.3.1 - actually rust-gmp-0.2.10 (old version), unfixed private_in_public in the same crate
🍂 carboxyl_time-0.0.2 - actually carboxyl-0.1.2 (old version), unfixed private_in_public in the same crate

Crates that have other crates dependent on them are marked with 🌳.

I think it'll be okay to send PRs with fixes and/or version bumps, and then land this without additional warnings.

nikomatsakis · 2017-05-26T15:45:44Z

unfixed private_in_public in the same crate, fixed on master but unpublished

what does "unfixed" mean? oh, unfixed in that crate, you mean? (i.e., not on rustc master)

petrochenkov · 2017-05-26T15:46:21Z

@nikomatsakis

unfixed in that crate, you mean?

Yes

petrochenkov · 2017-05-27T10:53:55Z

PRs/issues are submitted to all affected crates.

arielb1 · 2017-05-27T20:47:25Z

this significantly restricts what macros 2.0 (as implemented in #40847) can do (sorry @jseyfried) - they still can use private names, but can't use private types.

It's hard to offer a strong boundary using private types in macros anyway, because of inference leakages. Unreachable-but-public types can be used if you want to hide a type name.

Cleanup checking of inherent static methods and fix checking of inherent associated constants Add more tests

Fix regressions after rebase

petrochenkov · 2017-07-06T22:28:29Z

Rebased.
@bors r=nikomatsakis

bors · 2017-07-06T22:28:30Z

📌 Commit a27f8cf has been approved by nikomatsakis

bors · 2017-07-07T01:32:20Z

⌛ Testing commit a27f8cf with merge b80b659...

@jseyfried

Check types for privacy This PR implements late post factum checking of type privacy, as opposed to early preventive "private-in-public" checking. This will allow to turn private-in-public checks into a lint and make them more heuristic-based, and more aligned with what people may expect (e.g. reachability-based behavior). Types are privacy-checked if they are written explicitly, and also if they are inferred as expression or pattern types. This PR checks "semantic" types and does it unhygienically, this significantly restricts what macros 2.0 (as implemented in #40847) can do (sorry @jseyfried) - they still can use private *names*, but can't use private *types*. This is the most conservative solution, but hopefully it's temporary and can be relaxed in the future, probably using macro contexts of expression/pattern spans. Traits are also checked in preparation for [trait aliases](#41517), which will be able to leak private traits, and macros 2.0 which will be able to leak pretty much anything. This is a [breaking-change], but the code that is not contrived and can be broken by this patch should be guarded by `private_in_public` lint. [Previous crater run](#34537 (comment)) discovered a few abandoned crates that weren't updated since `private_in_public` has been introduced in 2015. cc #34537 https://internals.rust-lang.org/t/lang-team-minutes-private-in-public-rules/4504 Fixes #30476 Fixes #33479 cc @nikomatsakis r? @eddyb

bors · 2017-07-07T03:43:05Z

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing b80b659 to master...

alexcrichton · 2017-08-12T22:36:57Z

@rust-lang/compiler just FYI but there's 5 open regressions against beta for this PR, all linked above this comment.

@eddyb

Record semantic types for all syntactic types in bodies ... and use recorded types in type privacy checking (types are recorded after inference, so there are no `_`s left). Also use `hir_ty_to_ty` for types in signatures in type privacy checking. This could also be potentially useful for save-analysis and diagnostics. Fixes #42125 (comment) r? @eddyb

@eddyb

Type privacy polishing Various preparations before implementing rust-lang/rfcs#2145 containing final minor breaking changes (mostly for unstable code or code using `allow(private_in_public)`). (Continuation of #42125, #44633 and #41332.) It would be good to run crater on this. r? @eddyb

rust-highfive assigned eddyb May 20, 2017

petrochenkov commented May 20, 2017

View reviewed changes

eddyb added the S-waiting-on-crater Status: Waiting on a crater run to be completed. label May 20, 2017

Mark-Simulacrum added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 20, 2017

Mark-Simulacrum added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label May 28, 2017

petrochenkov added 2 commits July 6, 2017 23:50

Check adjustments and node substs

08a1d45

Cleanup checking of inherent static methods and fix checking of inherent associated constants Add more tests

Address review comments

a27f8cf

Fix regressions after rebase

petrochenkov force-pushed the privty branch from 4fb032e to a27f8cf Compare July 6, 2017 22:27

bors merged commit a27f8cf into rust-lang:master Jul 7, 2017

tbg mentioned this pull request Jul 8, 2017

Downgrade ProjectionTy's TraitRef to its substs #42388

Merged

This was referenced Aug 5, 2017

carboxyl_time-0.0.2 beta regression #43658

Closed

coyoneda-0.5.2 beta regression #43659

Closed

mysql beta regression #43662

Closed

keyring-0.1.1 beta regression #43671

Closed

This was referenced Aug 5, 2017

rulinalg-0.3.7 beta regression #43678

Closed

rust-gmp-0.2.10 beta regression #43679

Closed

nightly-rust beta regression (via slog-async-2.0.1) #43686

Closed

This was referenced Aug 22, 2017

rustpather-0.1.0 beta regression #43953

Closed

phage beta regression #43954

Closed

twists beta regression #43955

Closed

petrochenkov mentioned this pull request Aug 24, 2017

Modules rust-lang/rfcs#2121

Closed

petrochenkov deleted the privty branch August 26, 2017 00:16

This was referenced Sep 15, 2017

RFC: Type privacy and private-in-public lints rust-lang/rfcs#2145

Merged

Record semantic types for all syntactic types in bodies #44633

Merged

petrochenkov mentioned this pull request Nov 18, 2017

Type privacy polishing #46083

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check types for privacy #42125

Check types for privacy #42125

petrochenkov commented May 20, 2017 •

edited

Loading

petrochenkov May 20, 2017

eddyb May 20, 2017

petrochenkov May 20, 2017

eddyb May 24, 2017

petrochenkov May 20, 2017

eddyb May 20, 2017 •

edited

Loading

petrochenkov May 20, 2017

nikomatsakis commented May 24, 2017 •

edited

Loading

nikomatsakis commented May 24, 2017

petrochenkov commented May 24, 2017

brson commented May 24, 2017

brson commented May 25, 2017

brson commented May 25, 2017

nikomatsakis commented May 25, 2017

Mark-Simulacrum commented May 25, 2017

petrochenkov commented May 26, 2017 •

edited

Loading

nikomatsakis commented May 26, 2017

petrochenkov commented May 26, 2017

petrochenkov commented May 27, 2017

arielb1 commented May 27, 2017

petrochenkov commented Jul 6, 2017

bors commented Jul 6, 2017

bors commented Jul 7, 2017

bors commented Jul 7, 2017

alexcrichton commented Aug 12, 2017

Check types for privacy #42125

Check types for privacy #42125

Conversation

petrochenkov commented May 20, 2017 • edited Loading

petrochenkov May 20, 2017

Choose a reason for hiding this comment

eddyb May 20, 2017

Choose a reason for hiding this comment

petrochenkov May 20, 2017

Choose a reason for hiding this comment

eddyb May 24, 2017

Choose a reason for hiding this comment

petrochenkov May 20, 2017

Choose a reason for hiding this comment

eddyb May 20, 2017 • edited Loading

Choose a reason for hiding this comment

petrochenkov May 20, 2017

Choose a reason for hiding this comment

nikomatsakis commented May 24, 2017 • edited Loading

nikomatsakis commented May 24, 2017

petrochenkov commented May 24, 2017

brson commented May 24, 2017

brson commented May 25, 2017

brson commented May 25, 2017

nikomatsakis commented May 25, 2017

Coverage

Regressions

Root regressions, sorted by rank:

Mark-Simulacrum commented May 25, 2017

petrochenkov commented May 26, 2017 • edited Loading

nikomatsakis commented May 26, 2017

petrochenkov commented May 26, 2017

petrochenkov commented May 27, 2017

arielb1 commented May 27, 2017

petrochenkov commented Jul 6, 2017

bors commented Jul 6, 2017

bors commented Jul 7, 2017

bors commented Jul 7, 2017

alexcrichton commented Aug 12, 2017

petrochenkov commented May 20, 2017 •

edited

Loading

eddyb May 20, 2017 •

edited

Loading

nikomatsakis commented May 24, 2017 •

edited

Loading

petrochenkov commented May 26, 2017 •

edited

Loading