From 8955d6a4d0db430a28114b32ac35ee0c406a5319 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Mon, 25 Jun 2018 17:59:54 -0700 Subject: [PATCH 01/46] WIP --- RFCs/FS-1060-nullable-reference-types.md | 208 +++++++++++++++++++++++ 1 file changed, 208 insertions(+) create mode 100644 RFCs/FS-1060-nullable-reference-types.md diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md new file mode 100644 index 00000000..161fad59 --- /dev/null +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -0,0 +1,208 @@ +# F# RFC FS-1060 - Nullable Reference Types + +The design suggestion [Add non-nullable instantiations of nullable types, and interop with proposed C# 8.0 non-nullable reference types](https://github.com/fsharp/fslang-suggestions/issues/577) has been marked "approved in principle". This RFC covers the detailed proposal for this suggestion. + +* [x] Approved in principle +* [ ] [Suggestion](https://github.com/fsharp/fslang-suggestions/issues/577) +* [ ] Details: [TODO](https://github.com/fsharp/fslang-design/issues/FILL-ME-IN) +* [ ] Implementation: [TODO](https://github.com/Microsoft/visualfsharp/pull/FILL-ME-IN) + +# Summary +[summary]: #summary + +The main goal of this feature is to address the pains of dealing with implicitly nullable reference types by providing syntax and behavior that distinguishes between nullable and non-nullable reference types. This will be done in lockstop with C# 8.0, which will also have this as a feature, and F# will interoperate with C# by using and emitting the same metadata adornment that C# will emit. + +Conceptually, reference types can be thought of as having two forms: + +* Normal reference types +* Nullable reference types + +These will be distinguished via syntax, type signatures, and tools. Additionally, F# will offer flow analysis to allow you to determine when a nullable reference type is non-null and can be "viewed" as a reference type. + +Warnings are emitted when reference and nullable reference types are not used according with their intent, and these warnings are tunable as errors. + +# Motivation +[motivation]: #motivation + +Today, any reference type (e.g., `string`) could be `null`, forcing developers to account for this with special checks at the top-level of their functions or methods. Especially in F#, where types are almost always assumed to be non-null, this is a pain that can often be forgotten, resulting in a `NullReferenceException`, which has traditionally been difficult to diagnose. Moreover, one of the primary benefits in working with F# is with immutable and **non-null** data. This is in line with the general goals of F#. + +The `[]` attribute allows you to declare F# reference types as nullable when you need it, but this "infects" any place where the type might be used. In practice, F# types are not normally checked for `null`, which could mean that there is a lot of code out there that does not account for `null` when using a type that is decorated with `[]`. + +Additionally, nullable reference types is a headlining feature of C# 8.0 that will have a dramatic impact on .NET and its ecosystem. It is vital that we interoperate with this change smoothly and can emit the appropriate information so that C# consumers of F# components can consume things as if they were also C# 8.0 components. + +The desired outcome over time is that significantly less `NullReferenceException`s are produced by code at runtime. This feature should allow F# programmers to more safely eject `null` from their concerns when interoperating with other .NET components, and make it more explicit when `null` could be a problem. + +# Principles + +The following principles guide all further considerations and the design of this feature. + +1. On the surface, F# is "almost" as simple to use as it is today. In practice, it must feel simpler due to nullability of types like `string` being explicit rather than implicit. +2. The value for F# is primarily in flowing non-nullable reference types into F# code from .NET Libraries and from F# code into .NET libraries. +3. Adding nullable annotations should not be part of routine F# programming. +4. Nullability annotations/information is carefully suppressed and simplified in tooling (tooltips, FSI) to avoid extra information overload. +5. F# users are typically concerned with this feature at the boundaries of their system so that they can flow non-null data into the inner parts of their system. +6. The feature produces warnings by default, with different classes of warnings (nullable, non-nullable) offering opt-in/opt-out mechanisms. +7. All existing F# projects compile with warnings turned off by default. Only new projects have warnings on by default. Compiling older F# code with newer F# code may opt-in the older F# code. +8. F# non-nullness is reasonably "strong and trustworthy" today for F#-defined types and routine F# coding. This includes the explicit use of `option` types to represent the absence of information. No compile-time guarantees today will be "lowered" to account for this feature. +9. The F# compiler will strive to provide flow analysis such that common null-check scenarios guarantee null-safety when working with nullable reference types. +10. This feature is useful for F# in other contexts, such as interop with `string` in JavaScript via [Fable](http://fable.io/). +11. F# programmers can start to phase out their use of `[]` and the `null` type constraint when they need ad-hoc nullability for reference types declared in F#. +12. Syntax for the feature feels "baked in" to the type system, and should not be horribly unpleasant to use. + +# Detailed design +[design]: #detailed-design + +This is the bulk of the RFC. Explain the design in enough detail for somebody familiar +with the language to understand, and for somebody familiar with the compiler to implement. +This should get into specifics and corner-cases, and include examples of how the feature is used. + +## Overview and syntax + +Nullable reference types can be thought of as a special case of [Erased type-tagged anonymous union types](https://github.com/fsharp/fslang-suggestions/issues/538). This is because, conceptually, reference types are already a union of the type and `null`. That is, `string` is implicitly either a `string` or `null`. This concept is lifted into syntax: + +``` +reference-type | null +``` + +Examples in F# code: + +```fsharp +// Parameter to a function +let len (str: string | null) = + match str with + | null -> -1 + | s -> s.Length + +// Return type +let findOrNull (index: int) (list: 'T list) : 'T | null = + match List.tryItem index list with + | Some item -> item + | None -> null + +// Declared type at let-binding +let maybeAValue : int | null = hopefullyGetAnInteger() + +// Generic type parameter +type IMyInterface<'T | null, > = + abstract DoStuff<'T | null> : unit -> 'T | null + +// Array type signature +let f (arr: (float | null) []) = () + +// Nullable types with nullable generic types +type C<'T | null>() = + member __.M(param1: List | null> | null) = () +``` + +The syntax accomplishes two goals: + +1. Lifts the concept that a reference type is either that type or null. +2. Becomes a bit unweildly when nullability is used a lot. + +Because it is a design goal to flow **non-null** reference types, and **avoid** nullable reference types unless absolutely necessary. + +Note that there is no way to declare an F# type to be unioned with `null`, e.g., + +```fsharp +type (NotSupportedAtAll() | null)() = class end +``` + +This is because `[]` already exists, as we do not offer two syntaxes to accomplish the same thing. + +## Checking of nullable references + +TODO + +## Checking of non-nullable references + +TODO + +## Flow analysis + +TODO + +## Emission of metadata + +TODO + +## Type inference + +TODO + +## Asserting non-nullability + +TODO + +## Tooling considerations + +TODO + +# Breaking changes + +There are two forms of breaking changes with this feature: + +* Non-null warnings (i.e., not checking for `null`) on a `string | null` type +* Nullable warnings + +TODO + +# Compatibility with existing F# nullability features + +Today, classes declared in F# are non-null by default. To that end, F# has a way to express opt-in nullability with `[]`. Attempting to assign `null` to a class without this attribute is a compile error: + +```fsharp +type CNonNull() = class end + +let mutable c = CNonNull() +c <- null // ERROR: 'CNonNull' does not have 'null' as a proper value. + +[] +type CNullable() = class end + +let mutable c2 = CNullable() +c2 <- null // OK +``` + +Additionally, F# can constrain generic types to require types which support `null` as a proper value. Parameterizing a generic type with a type that does not support `null` as a proper value is a compile error: + +```fsharp +type CNonNull() = class end + +[] +type CNullable() = class end + +type C< ^T when ^T: null>() = class end + +let c1 = C() // ERROR: 'CNonNull' does not have 'null' as a proper value +let c2 = C() // OK +``` + +This behavior will remain unmodified. Any existing code that uses `[]` and the `null` constraint will be source-compatible with this feature and exhibit the same compile-time behavior as before. Neither feature is up for deprecation. + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +# Alternatives +[alternatives]: #alternatives + +What other designs have been considered? What is the impact of not doing this? + +## Syntax + +TODO `foo?` + +## Warnings + +TODO + +## Assering non-nullability + +TODO + +# Unresolved questions +[unresolved]: #unresolved-questions + +What parts of the design are still TBD? + From 9666ae3c5b3c0d91a50f7d1ff18f21724af31ef2 Mon Sep 17 00:00:00 2001 From: cartermp Date: Tue, 26 Jun 2018 10:10:15 -0700 Subject: [PATCH 02/46] Updates --- RFCs/FS-1060-nullable-reference-types.md | 328 ++++++++++++++++++----- 1 file changed, 267 insertions(+), 61 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 161fad59..3e117494 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -7,32 +7,32 @@ The design suggestion [Add non-nullable instantiations of nullable types, and in * [ ] Details: [TODO](https://github.com/fsharp/fslang-design/issues/FILL-ME-IN) * [ ] Implementation: [TODO](https://github.com/Microsoft/visualfsharp/pull/FILL-ME-IN) -# Summary +## Summary [summary]: #summary -The main goal of this feature is to address the pains of dealing with implicitly nullable reference types by providing syntax and behavior that distinguishes between nullable and non-nullable reference types. This will be done in lockstop with C# 8.0, which will also have this as a feature, and F# will interoperate with C# by using and emitting the same metadata adornment that C# will emit. +The main goal of this feature is to address the pains of dealing with implicitly nullable reference types by providing syntax and behavior that distinguishes between nullable and non-nullable reference types. This will be done in lockstep with C# 8.0, which will also have this as a feature, and F# will interoperate with C# by using and emitting the same metadata adornment that C# emits emit. Conceptually, reference types can be thought of as having two forms: * Normal reference types * Nullable reference types -These will be distinguished via syntax, type signatures, and tools. Additionally, F# will offer flow analysis to allow you to determine when a nullable reference type is non-null and can be "viewed" as a reference type. +These will be distinguished via syntax, type signatures, and tools. Additionally, there will be flow analysis to allow you to determine when a nullable reference type is non-null and can be "viewed" as a reference type. Warnings are emitted when reference and nullable reference types are not used according with their intent, and these warnings are tunable as errors. -# Motivation +## Motivation [motivation]: #motivation -Today, any reference type (e.g., `string`) could be `null`, forcing developers to account for this with special checks at the top-level of their functions or methods. Especially in F#, where types are almost always assumed to be non-null, this is a pain that can often be forgotten, resulting in a `NullReferenceException`, which has traditionally been difficult to diagnose. Moreover, one of the primary benefits in working with F# is with immutable and **non-null** data. This is in line with the general goals of F#. +Today, any reference type (e.g., `string`) could be `null`, forcing developers to account for this with special checks at the top-level of their functions or methods. Especially in F#, where types are almost always assumed to be non-null, this is a pain that can often be forgotten. Moreover, one of the primary benefits in working with F# is working with immutable and **non-null** data. By this statement alone, nullable reference types are in line with the general goals of F#. -The `[]` attribute allows you to declare F# reference types as nullable when you need it, but this "infects" any place where the type might be used. In practice, F# types are not normally checked for `null`, which could mean that there is a lot of code out there that does not account for `null` when using a type that is decorated with `[]`. +The `[]` attribute allows you to declare F# reference types as nullable, but this "infects" any place where the type might be used. In practice, F# types are not normally checked for `null`, which could mean that there is a lot of code out there that does not account for `null` when using a type that is decorated with `[]`. Additionally, nullable reference types is a headlining feature of C# 8.0 that will have a dramatic impact on .NET and its ecosystem. It is vital that we interoperate with this change smoothly and can emit the appropriate information so that C# consumers of F# components can consume things as if they were also C# 8.0 components. The desired outcome over time is that significantly less `NullReferenceException`s are produced by code at runtime. This feature should allow F# programmers to more safely eject `null` from their concerns when interoperating with other .NET components, and make it more explicit when `null` could be a problem. -# Principles +## Principles The following principles guide all further considerations and the design of this feature. @@ -45,20 +45,51 @@ The following principles guide all further considerations and the design of this 7. All existing F# projects compile with warnings turned off by default. Only new projects have warnings on by default. Compiling older F# code with newer F# code may opt-in the older F# code. 8. F# non-nullness is reasonably "strong and trustworthy" today for F#-defined types and routine F# coding. This includes the explicit use of `option` types to represent the absence of information. No compile-time guarantees today will be "lowered" to account for this feature. 9. The F# compiler will strive to provide flow analysis such that common null-check scenarios guarantee null-safety when working with nullable reference types. -10. This feature is useful for F# in other contexts, such as interop with `string` in JavaScript via [Fable](http://fable.io/). +10. This feature is useful for F# in other contexts, such as interoperation with `string` in JavaScript via [Fable](http://fable.io/). 11. F# programmers can start to phase out their use of `[]` and the `null` type constraint when they need ad-hoc nullability for reference types declared in F#. 12. Syntax for the feature feels "baked in" to the type system, and should not be horribly unpleasant to use. -# Detailed design +## Compatibility with existing F# nullability features + +Today, classes declared in F# are non-null by default. To that end, F# has a way to express opt-in nullability with `[]`. Attempting to assign `null` to a class without this attribute is a compile error: + +```fsharp +type CNonNull() = class end + +let mutable c = CNonNull() +c <- null // ERROR: 'CNonNull' does not have 'null' as a proper value. + +[] +type CNullable() = class end + +let mutable c2 = CNullable() +c2 <- null // OK +``` + +Additionally, F# can constrain generic types to require types which support `null` as a proper value. Parameterizing a generic type with a type that does not support `null` as a proper value is a compile error: + +```fsharp +type CNonNull() = class end + +[] +type CNullable() = class end + +type C< ^T when ^T: null>() = class end + +let c1 = C() // ERROR: 'CNonNull' does not have 'null' as a proper value +let c2 = C() // OK +``` + +This behavior will remain unmodified. Any existing code that uses `[]` and the `null` constraint will be source-compatible with this feature and exhibit the same compile-time behavior as before. Neither feature is up for deprecation. + +## Detailed design [design]: #detailed-design -This is the bulk of the RFC. Explain the design in enough detail for somebody familiar -with the language to understand, and for somebody familiar with the compiler to implement. -This should get into specifics and corner-cases, and include examples of how the feature is used. +## Core concept and syntax -## Overview and syntax +Nullable reference types can conceptually be thought of a union between a reference type and `null`. For example, `string` in F# 4.5 or C# 7.3 is implicitly either a `string` or `null`. More concretely, they are a special case of [Erased type-tagged anonymous union types](https://github.com/fsharp/fslang-suggestions/issues/538). -Nullable reference types can be thought of as a special case of [Erased type-tagged anonymous union types](https://github.com/fsharp/fslang-suggestions/issues/538). This is because, conceptually, reference types are already a union of the type and `null`. That is, `string` is implicitly either a `string` or `null`. This concept is lifted into syntax: +This concept is lifted into syntax: ``` reference-type | null @@ -96,40 +127,215 @@ type C<'T | null>() = The syntax accomplishes two goals: -1. Lifts the concept that a reference type is either that type or null. -2. Becomes a bit unweildly when nullability is used a lot. +1. Makes the distinction that a nullable reference type can be `null` clear in syntax. +2. Becomes a bit unweildy when nullability is used a lot, particularly with nested types. + +Because it is a design goal to flow **non-null** reference types, and **avoid** flowing nullable reference types unless absolutely necessary, it is considered a feature that the syntax can get a bit unweildy. + +### Nullable reference type declarations in F\# + +There will be no way to declare an F# type as follows: + +```fsharp +type (NotSupportedAtAll | null)() = class end +``` + +This is because `[]` already exists, and we do not wish offer two syntaxes to accomplish the same thing. Thus, to declare a nullable reference type in F#, you must use `[]` as before: + +```fsharp +[] +type C() = class end +``` + +The same rules and behavior that exists today still applies. That is, if you attempt to assign `null` to a type declared in F# that is not decorated with `[]`, it will remain a compile error. + +### Null type constraint and nullable generic parameters + +The existing `null` type constraint still exists: + +```fsharp +type C< ^T when ^T: null>() = class end +``` -Because it is a design goal to flow **non-null** reference types, and **avoid** nullable reference types unless absolutely necessary. +When compiled, this actually just produces a class that requires a reference type as the `'T`: + +```csharp +// Compiled form +[Serializable] +[CompilationMapping(SourceConstructFlags.ObjectType)] +public class C where T : class +{ + public C() + : this() + { + } +} +``` + +However, the only time you can parameterize `C` with an F# class is if that class has been decorated with `[]`. Attempting to parameterize with an F# class without this attribute will still be a compile error as it is today. -Note that there is no way to declare an F# type to be unioned with `null`, e.g., +However, this is **not** equivalent to declaring a class where the parameterized type is a nullable reference type: ```fsharp -type (NotSupportedAtAll() | null)() = class end +type C1< ^T when ^T: null>() = class end + +// NOT THE SAME THING +type C2<'T | null>() = class end ``` -This is because `[]` already exists, as we do not offer two syntaxes to accomplish the same thing. + + +Additionally, the existing `null` type constraint is **not** equivalent to `C<'T | null>`. The `null` constraint expressed via Statically Resolved Type Parameters ## Checking of nullable references -TODO +There are a few common ways that F# programmers check for null today. Unless explicitly mentioned as unsupported, support for flow analysis is under consideration for certain null-checking patterns. -## Checking of non-nullable references +### Pattern matching with alias -TODO +```fsharp +let len (str: string | null) = + match str with + | null -> -1 + | s -> s.Length // OK - we know 's' is string +``` -## Flow analysis +This is by far the most common pattern in F# programming. In the previous code sample, `str` is of type `string | null`, but `s` is now of type `string`. This is because we know that based on the `null` has been accounted for by the `null` pattern. -TODO +### Pattern matching with wildcard -## Emission of metadata +```fsharp +let len (str: string | null) = + match str with + | null -> -1 + | _ -> str.Length // OK - 'str' is string +``` -TODO +A slight riff that is less common is treating `str` differently when in a wildcard pattern's scope. To reach that code path, `str` must indeed by non-null, so this could potentially be supported. -## Type inference +### If check + +```fsharp +let len (str: string | null) = + if str = null then + -1 + else + str.Length // OK - 'str' is a string +``` + +Although less common, this is a valid way to check for `null` and could be enabled by the compiler. We know for certain that `str` is a `string` in the `else` branch. The inverse check is also true. + +### Unsupported: isNull + +```fsharp +let len (str: string | null) = + if isNull str then + -1 + else + str.Length // WARNING: could be null +``` + +The `isNull` function is considered to be a good way to check for `null`. Unfortunately, because it is a function that returns `bool`, the logic needed to support this is deeply foreign to F# and troublesome to implement. For example, consider the following: + +```fsharp +let len (str: string | null) = + let someCondition = getCondition() + if isNull str && someCondition then + -1 + else + str.Length // WARNING: could be null +``` + +There is no telling what `someCondition` would evaluate to, so we cannot guarantee that `str` will be `string` in the `else` clause. + +### Unsupported: boolean-based checks for null + +Generally speaking, beyond highly-specialized cases, we cannot guarantee non-nullability that is checked via a boolean. Consider the following: + +```fsharp +let myIsNull item = isNull item + +let len (str: string | null) = + if myIsNull str then + -1 + else + str.Length // WARNING: could be null +``` + +Although this simple example could potentially work, it could quickly get out of hand and complicate the compiler. Also, other languages that support nullable reference types (C# 8.0, Scala, Kotlin, Swift, TypeScript) do not do this. Instead, non-nullability is asserted when the programmer knows something will not be `null`. + +### Asserting non-nullability + +To get around scenarios where flow analysis cannot establish a non-null situation, a programmer can use `!` to assert non-nullability: + +```fsharp +let len (str: string | null) = + if isNull str then + -1 + else + str!.Length // OK: 'str' is asserted to be 'null' +``` + +This will not generate a warning, but it is unsafe and can be the cause of a `NullReferenceException` if the reference was indeed `null` under some condition. + +### Warnings + +In all other situations not covered by flow analysis, a warning is emitted by the compiler if a nullable reference is dereferenced or casted to a non-null type: + +```fsharp +let f (ns: string | null) = + printfn "%d" ns.Length // WARNING: 'ns' may be null + + let mutable ns2 = ns + printfn "%d" ns2.Length // WARNING: 'ns2' may be null + + let s = ns :> string // WARNING: 'ns' may be null +``` + +A warning is given when casting from `'S[]` to `('T | null)[]` and from `('S | null)[]` to `'T[]`. + +A warning is given when casting from `C<'S>` to `C<'T | null>` and from `C<'S | null>` to `C<'T>`. + +### Errors + +As previously mentioned, the following two cases are always errors: + +1. Attempting to assign `null` to an F#-declared reference type that is not decorated with `[]`. +2. Attempting to parameterize a type with the `null` constraint with a reference type that is non-nullable. + +Additionally, all warnings associated with nullability can also be tuned as errors separately from the `--warnaserror` switch. This is because some F# programmers may want this feature to be strict separately from how they deal with other warnings. + +## Checking of non-null references + +A warning is given if `null` is assigned to a non-null reference type or passed as a parameter where a non-null reference type is expected. If the type is declared in F# and is not annotated with `[]`, an error is given as it is today: + +```fsharp +let mutable s: string = "hello" +s <- null // WARNING + +let f (s: string) = () +f null // WARNING + +type C() = class end +let g (c: C) = () +g null // ERROR +``` + +A warning is given if a nullable reference type `P` is used to parameterize another type `C` that accepts non-nullable reference types. If `C` is declared using Statically Resolved Type Parameters, an error is given as it is today. + +```fsharp +type CNullParam<'T>() = class end + +let c1 = C // WARNING +``` + +## Metadata TODO -## Asserting non-nullability +`System.Runtime.CompilerServices.NullableAttribute` + +## Type inference TODO @@ -137,7 +343,7 @@ TODO TODO -# Breaking changes +## Breaking changes There are two forms of breaking changes with this feature: @@ -146,62 +352,62 @@ There are two forms of breaking changes with this feature: TODO -# Compatibility with existing F# nullability features - -Today, classes declared in F# are non-null by default. To that end, F# has a way to express opt-in nullability with `[]`. Attempting to assign `null` to a class without this attribute is a compile error: +## Drawbacks +[drawbacks]: #drawbacks -```fsharp -type CNonNull() = class end +Why should we *not* do this? -let mutable c = CNonNull() -c <- null // ERROR: 'CNonNull' does not have 'null' as a proper value. +## Alternatives +[alternatives]: #alternatives -[] -type CNullable() = class end +What other designs have been considered? What is the impact of not doing this? -let mutable c2 = CNullable() -c2 <- null // OK -``` +### Syntax -Additionally, F# can constrain generic types to require types which support `null` as a proper value. Parameterizing a generic type with a type that does not support `null` as a proper value is a compile error: +#### Question mark ```fsharp -type CNonNull() = class end +let ns: string? = "hello" -[] -type CNullable() = class end +type C<'T?>() = class end +``` -type C< ^T when ^T: null>() = class end +Issue: looks horrible with F# optional parameters -let c1 = C() // ERROR: 'CNonNull' does not have 'null' as a proper value -let c2 = C() // OK +```fsharp +type C() = + memeber __.M(?x: string?) = "BLEGH" ``` -This behavior will remain unmodified. Any existing code that uses `[]` and the `null` constraint will be source-compatible with this feature and exhibit the same compile-time behavior as before. Neither feature is up for deprecation. +#### Nullable -# Drawbacks -[drawbacks]: #drawbacks +Find a way to make semantics with the existing `System.Nullable<'T>` type. -Why should we *not* do this? +Issues: -# Alternatives -[alternatives]: #alternatives +* Back-compat issue? +* Really ugly nesting +* No syntax for this makes F# be "behind" C# for such a cornerstone thing -What other designs have been considered? What is the impact of not doing this? +#### Option + +Find a way to make this work with the existing Option type. -## Syntax +Issues: -TODO `foo?` +* Back compat issue? +* `None` already emits `null`, but it's a distinct case and not erased +* References and Nullable references are fundamentally the same thing, just one has an assembly-level attribute -## Warnings +### Warnings TODO -## Assering non-nullability +### Assering non-nullability TODO -# Unresolved questions +## Unresolved questions [unresolved]: #unresolved-questions What parts of the design are still TBD? From 847c1959457f5ce06073964cce43476069c3629d Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Tue, 26 Jun 2018 18:50:47 -0700 Subject: [PATCH 03/46] hoopty --- RFCs/FS-1060-nullable-reference-types.md | 103 ++++++++++++++--------- 1 file changed, 65 insertions(+), 38 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 3e117494..88e73707 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -49,39 +49,6 @@ The following principles guide all further considerations and the design of this 11. F# programmers can start to phase out their use of `[]` and the `null` type constraint when they need ad-hoc nullability for reference types declared in F#. 12. Syntax for the feature feels "baked in" to the type system, and should not be horribly unpleasant to use. -## Compatibility with existing F# nullability features - -Today, classes declared in F# are non-null by default. To that end, F# has a way to express opt-in nullability with `[]`. Attempting to assign `null` to a class without this attribute is a compile error: - -```fsharp -type CNonNull() = class end - -let mutable c = CNonNull() -c <- null // ERROR: 'CNonNull' does not have 'null' as a proper value. - -[] -type CNullable() = class end - -let mutable c2 = CNullable() -c2 <- null // OK -``` - -Additionally, F# can constrain generic types to require types which support `null` as a proper value. Parameterizing a generic type with a type that does not support `null` as a proper value is a compile error: - -```fsharp -type CNonNull() = class end - -[] -type CNullable() = class end - -type C< ^T when ^T: null>() = class end - -let c1 = C() // ERROR: 'CNonNull' does not have 'null' as a proper value -let c2 = C() // OK -``` - -This behavior will remain unmodified. Any existing code that uses `[]` and the `null` constraint will be source-compatible with this feature and exhibit the same compile-time behavior as before. Neither feature is up for deprecation. - ## Detailed design [design]: #detailed-design @@ -307,7 +274,7 @@ Additionally, all warnings associated with nullability can also be tuned as erro ## Checking of non-null references -A warning is given if `null` is assigned to a non-null reference type or passed as a parameter where a non-null reference type is expected. If the type is declared in F# and is not annotated with `[]`, an error is given as it is today: +A warning is given if `null` is assigned to a non-null reference type or passed as a parameter where a non-null reference type is expected. ```fsharp let mutable s: string = "hello" @@ -318,10 +285,10 @@ f null // WARNING type C() = class end let g (c: C) = () -g null // ERROR +g null // WARNING ``` -A warning is given if a nullable reference type `P` is used to parameterize another type `C` that accepts non-nullable reference types. If `C` is declared using Statically Resolved Type Parameters, an error is given as it is today. +A warning is given if a nullable reference type `P` is used to parameterize another type `C` that accepts non-nullable reference types. ```fsharp type CNullParam<'T>() = class end @@ -407,8 +374,68 @@ TODO TODO -## Unresolved questions +### Unresolved questions [unresolved]: #unresolved-questions -What parts of the design are still TBD? +## Compatibility with existing F# nullability features + +Today, classes declared in F# are non-null by default. To that end, F# has a way to express opt-in nullability with `[]`. Attempting to assign `null` to a class without this attribute is a compile error: + +```fsharp +type CNonNull() = class end + +let mutable c = CNonNull() +c <- null // ERROR: 'CNonNull' does not have 'null' as a proper value. + +[] +type CNullable() = class end + +let mutable c2 = CNullable() +c2 <- null // OK +``` + +Additionally, F# can constrain generic types to require types which support `null` as a proper value. Parameterizing a generic type with a type that does not support `null` as a proper value is a compile error: + +```fsharp +type CNonNull() = class end + +[] +type CNullable() = class end + +type C< ^T when ^T: null>() = class end + +let c1 = C() // ERROR: 'CNonNull' does not have 'null' as a proper value +let c2 = C() // OK +``` + +This behavior will remain unmodified. Any existing code that uses `[]` and the `null` constraint will be source-compatible with this feature and exhibit the same compile-time behavior as before. Neither feature is up for deprecation. + +### Compatibility with existing null constraint + +The existing `null` constraint for generic types prevents programmers from parameterizing a type with an F# reference type that does not have `null` as a proper value. It is expressed as follows: + +```fsharp +[] +type C() = class end +type D< ^T when ^T: null>() = class end +``` + +`D` will compile into this: + +```csharp +[Serializable] +[CompilationMapping(SourceConstructFlags.ObjectType)] +public class D where T : class +{ + public D() + : this() + { + } +} +``` + +However, this will be highly confusing as per the [C# 8.0 proposal](https://github.com/dotnet/csharplang/blob/master/proposals/nullable-reference-types.md#generics): + +> The `class` constraint is **non-null**. We can consider whether `class?` should be a valid nullable constraint denoting "nullable reference type". +What this means is that F# syntax that declares a type as accepting only "nullable" types (note: not the same thing, but can and will be interpreted that way by programmers) as type parameters actually accepts _only_ non-nullable types! This behavior is the exact opposite of what programmers likely believe what this syntax does. \ No newline at end of file From 4c559b65bb8acc6249d8ba6af35e7eca21ded11b Mon Sep 17 00:00:00 2001 From: cartermp Date: Tue, 26 Jun 2018 23:39:04 -0700 Subject: [PATCH 04/46] goodnight --- RFCs/FS-1060-nullable-reference-types.md | 227 +++++++++++++---------- 1 file changed, 133 insertions(+), 94 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 88e73707..81f7cb52 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -52,7 +52,34 @@ The following principles guide all further considerations and the design of this ## Detailed design [design]: #detailed-design -## Core concept and syntax +## Metadata + +Nullable reference types are emitted as a reference type with an assembly-level attribute in C# 8.0. That is, the following code: + +```csharp +public class D { + public void M(string? s) {} +} +``` + +Is the same as the following: + +```csharp +public class D +{ + public void M([System.Runtime.CompilerServices.Nullable] string s) + { + } +} +``` + +Languages that are unaware of this attribute will see `M` as a method that takes in a `string`. That is, to be unaware of and failing to respect this attribute means that you will view incoming reference types exactly as before. + +F# will be aware of and respect this attribute, treating reference types as nullable reference types if they have this attribute. If they do not, then those reference types will be non-nullable. + +F# will also emit this attribute when it produces a nullable reference type for other languages to consume. + +## F# concept and syntax Nullable reference types can conceptually be thought of a union between a reference type and `null`. For example, `string` in F# 4.5 or C# 7.3 is implicitly either a `string` or `null`. More concretely, they are a special case of [Erased type-tagged anonymous union types](https://github.com/fsharp/fslang-suggestions/issues/538). @@ -107,58 +134,11 @@ There will be no way to declare an F# type as follows: type (NotSupportedAtAll | null)() = class end ``` -This is because `[]` already exists, and we do not wish offer two syntaxes to accomplish the same thing. Thus, to declare a nullable reference type in F#, you must use `[]` as before: - -```fsharp -[] -type C() = class end -``` - -The same rules and behavior that exists today still applies. That is, if you attempt to assign `null` to a type declared in F# that is not decorated with `[]`, it will remain a compile error. - -### Null type constraint and nullable generic parameters - -The existing `null` type constraint still exists: - -```fsharp -type C< ^T when ^T: null>() = class end -``` - -When compiled, this actually just produces a class that requires a reference type as the `'T`: - -```csharp -// Compiled form -[Serializable] -[CompilationMapping(SourceConstructFlags.ObjectType)] -public class C where T : class -{ - public C() - : this() - { - } -} -``` - -However, the only time you can parameterize `C` with an F# class is if that class has been decorated with `[]`. Attempting to parameterize with an F# class without this attribute will still be a compile error as it is today. - -However, this is **not** equivalent to declaring a class where the parameterized type is a nullable reference type: - -```fsharp -type C1< ^T when ^T: null>() = class end - -// NOT THE SAME THING -type C2<'T | null>() = class end -``` - - - -Additionally, the existing `null` type constraint is **not** equivalent to `C<'T | null>`. The `null` constraint expressed via Statically Resolved Type Parameters - ## Checking of nullable references There are a few common ways that F# programmers check for null today. Unless explicitly mentioned as unsupported, support for flow analysis is under consideration for certain null-checking patterns. -### Pattern matching with alias +### Likely supported: Pattern matching with alias ```fsharp let len (str: string | null) = @@ -169,7 +149,7 @@ let len (str: string | null) = This is by far the most common pattern in F# programming. In the previous code sample, `str` is of type `string | null`, but `s` is now of type `string`. This is because we know that based on the `null` has been accounted for by the `null` pattern. -### Pattern matching with wildcard +### Possibly supported: Pattern matching with wildcard ```fsharp let len (str: string | null) = @@ -180,7 +160,7 @@ let len (str: string | null) = A slight riff that is less common is treating `str` differently when in a wildcard pattern's scope. To reach that code path, `str` must indeed by non-null, so this could potentially be supported. -### If check +### Possibly supported: If check ```fsharp let len (str: string | null) = @@ -192,7 +172,19 @@ let len (str: string | null) = Although less common, this is a valid way to check for `null` and could be enabled by the compiler. We know for certain that `str` is a `string` in the `else` branch. The inverse check is also true. -### Unsupported: isNull +### Possibly supported: After null check within boolean expression + +```fsharp +let len (str: string | null) = + if (str <> null && str.Length > 0) then // OK - `str` must be null + str.Length // OK here too + else + -1 +``` + +Note that the reverse (`str.Length > 0 && str <> null`) would give a warning, because we attempt to dereference before the `null` check. This would only hold with AND checks. More generally, if the boolean expression to the left of the dereference involves a `x <> null` check, then the dereference is safe. + +### Likely unsupported: isNull ```fsharp let len (str: string | null) = @@ -215,7 +207,7 @@ let len (str: string | null) = There is no telling what `someCondition` would evaluate to, so we cannot guarantee that `str` will be `string` in the `else` clause. -### Unsupported: boolean-based checks for null +### Likely unsupported: boolean-based checks for null Generally speaking, beyond highly-specialized cases, we cannot guarantee non-nullability that is checked via a boolean. Consider the following: @@ -231,6 +223,22 @@ let len (str: string | null) = Although this simple example could potentially work, it could quickly get out of hand and complicate the compiler. Also, other languages that support nullable reference types (C# 8.0, Scala, Kotlin, Swift, TypeScript) do not do this. Instead, non-nullability is asserted when the programmer knows something will not be `null`. +### Likely unsupported: nested functions accessing outer scopes + +```fsharp +let len (str: string | null) = + let doWork() = + str.Length // WARNING: maybe too complicated? + + match str with + | null -> -1 + | _ -> doWork() +``` + +Although `doWork()` is called in a place where `str` would be non-null, this may too complex to implement properly. + +**Note:** C# supports the equivalent of this with local functions. + ### Asserting non-nullability To get around scenarios where flow analysis cannot establish a non-null situation, a programmer can use `!` to assert non-nullability: @@ -263,32 +271,25 @@ A warning is given when casting from `'S[]` to `('T | null)[]` and from `('S | n A warning is given when casting from `C<'S>` to `C<'T | null>` and from `C<'S | null>` to `C<'T>`. -### Errors - -As previously mentioned, the following two cases are always errors: - -1. Attempting to assign `null` to an F#-declared reference type that is not decorated with `[]`. -2. Attempting to parameterize a type with the `null` constraint with a reference type that is non-nullable. - -Additionally, all warnings associated with nullability can also be tuned as errors separately from the `--warnaserror` switch. This is because some F# programmers may want this feature to be strict separately from how they deal with other warnings. - ## Checking of non-null references -A warning is given if `null` is assigned to a non-null reference type or passed as a parameter where a non-null reference type is expected. +### Null assignment and passing + +A warning is given if `null` is assigned to a non-null value or passed as a parameter where a non-null reference type is expected: ```fsharp let mutable s: string = "hello" s <- null // WARNING +let s2: string = null // WARNING + let f (s: string) = () f null // WARNING - -type C() = class end -let g (c: C) = () -g null // WARNING ``` -A warning is given if a nullable reference type `P` is used to parameterize another type `C` that accepts non-nullable reference types. +### Nullable parameterization + +A warning is given if a nullable reference type `P` is used to parameterize another type `C` that does not accept nullable reference types: ```fsharp type CNullParam<'T>() = class end @@ -296,18 +297,50 @@ type CNullParam<'T>() = class end let c1 = C // WARNING ``` -## Metadata +Today, it is already a compile error if all fields in an F# class are not initialized by constructor calls. This behavior is unchanged. See [Compatibility with Microsoft.FSharp.Core.DefaultValueAttribute](FS-1060-nullable-reference-types.md#compatibility-with-microsoft.fsharp.core.defaultValueAttribute) for considerations about code that uses `[]` with reference types today, which produces `null`. -TODO +### F# Collection initialization + +A warning is given if `null` is used in an F# array/list/seq expression, where the type is already annotated to be non-nullable: + +```fsharp +let xs: string[] = [| ""; ""; null |] // WARNING +let ys: string list = [ ""; ""; null ] // WARNING +let zs: seq = seq { yield ""; yield ""; yield null } // WARNING +``` + +### Unchecked.defaultOf<'T> + +Today, `Unchecked.defaultof<'T>` will generate `null` when `'T` is a reference type. Given this, it is reasonable to do either of the following: + +* The return type of `Unchecked.defaultof<'T>` is now `'T | null`. Calling `Unchecked.defaultof<'T | null>` produces the same output. +* Give a warning when `Unchecked.defaultof<'T>` is called when `'T` is a non-nullable reference type, indicating that you must call `Unchecked.defaultof<'T | null>`. -`System.Runtime.CompilerServices.NullableAttribute` +### Array.zeroCreate<'T> + +Today, `Array.zeroCreate<'T>` will generate `null`s when `'T` is a reference type. Given this, it is reasonable to do either of the following: + +* The return type of `Array.zeroCreate<'T>` is now `('T | null) []`. Calling `Array.zeroCreate<'T | null>` produces the same output. +* Give a warning when `Array.zeroCreate<'T>` is called where `'T` is a non-nullable reference type, indicating that you must call `Array.zeroCreate<'T | null>`. ## Type inference -TODO +### F# Collection initialization + +Using `null` will "infect" ## Tooling considerations +### F# tooltips + +TODO + +### Signatures in FSI + +TODO + +### F# scripting + TODO ## Breaking changes @@ -346,6 +379,8 @@ type C() = memeber __.M(?x: string?) = "BLEGH" ``` +But it is a lot easier to implement since it doesn't require ad-hoc, structurally-typed unions. + #### Nullable Find a way to make semantics with the existing `System.Nullable<'T>` type. @@ -394,33 +429,19 @@ let mutable c2 = CNullable() c2 <- null // OK ``` -Additionally, F# can constrain generic types to require types which support `null` as a proper value. Parameterizing a generic type with a type that does not support `null` as a proper value is a compile error: - -```fsharp -type CNonNull() = class end - -[] -type CNullable() = class end - -type C< ^T when ^T: null>() = class end - -let c1 = C() // ERROR: 'CNonNull' does not have 'null' as a proper value -let c2 = C() // OK -``` - -This behavior will remain unmodified. Any existing code that uses `[]` and the `null` constraint will be source-compatible with this feature and exhibit the same compile-time behavior as before. Neither feature is up for deprecation. +This behavior is quite similar to nullable reference types, but is unfortunately a bit of a cleaver when all you need is a paring knife. It's arguable that the value of nullability for F# programmers is _ad-hoc_ in nature, which is something that is squarely accomplished with nullable reference types. Instead, `[]` sort of "pollutes" things by making any instantiation a nullable instantiation. ### Compatibility with existing null constraint -The existing `null` constraint for generic types prevents programmers from parameterizing a type with an F# reference type that does not have `null` as a proper value. It is expressed as follows: +The existing `null` constraint for generic types prevents programmers from parameterizing a type with an F# reference type that does not have `null` as a proper value (i.e., decorated with `[]`). + +The following class `D`: ```fsharp -[] -type C() = class end -type D< ^T when ^T: null>() = class end +type D<'T when 'T: null>() = class end ``` -`D` will compile into this: +will compile into this: ```csharp [Serializable] @@ -434,8 +455,26 @@ public class D where T : class } ``` -However, this will be highly confusing as per the [C# 8.0 proposal](https://github.com/dotnet/csharplang/blob/master/proposals/nullable-reference-types.md#generics): +Which is fine in today's world where nullable reference types are not explicit. After all `class` requires `T` to be a reference type, which can implicitly be `null`. + +However, this will be highly confusing in C# 8.0 as per the [C# 8.0 proposal](https://github.com/dotnet/csharplang/blob/master/proposals/nullable-reference-types.md#generics): > The `class` constraint is **non-null**. We can consider whether `class?` should be a valid nullable constraint denoting "nullable reference type". -What this means is that F# syntax that declares a type as accepting only "nullable" types (note: not the same thing, but can and will be interpreted that way by programmers) as type parameters actually accepts _only_ non-nullable types! This behavior is the exact opposite of what programmers likely believe what this syntax does. \ No newline at end of file +What this means is that F# syntax that declares a type as accepting only types that have `null` as a proper value as type parameters actually accepts _only_ non-nullable types! This behavior is the exact opposite of what programmers likely believe what this syntax does, and is not compatible with a world where nullable reference types exist. + +## Compatibility with Microsoft.FSharp.Core.DefaultValueAttribute + +Today, the `[]` attribute is required on explicit fields in classes that have a primary constructor. These fields must support zero-initialization, which in the case of reference types, is `null`. + +Consider the following code: + +```fsharp +type C() = + [] + val mutable Whoops : string + +printfn "%d" (C().Whoops.Length) +``` + +Today, this produces a `NullReferenceException` at runtime. This means that `Whoops` is actually a nullable reference type, but it will be annotated as if it were a non-nullable reference type. It is the presence of the attribute that changes this, which we'll have to rationalize somehow. \ No newline at end of file From 6e8735bc299e7190b58fe924cb94a97068e5090a Mon Sep 17 00:00:00 2001 From: cartermp Date: Wed, 27 Jun 2018 15:13:37 -0700 Subject: [PATCH 05/46] Mostly done --- RFCs/FS-1060-nullable-reference-types.md | 272 ++++++++++++++++++----- 1 file changed, 221 insertions(+), 51 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 81f7cb52..b2a5e9bc 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -52,7 +52,7 @@ The following principles guide all further considerations and the design of this ## Detailed design [design]: #detailed-design -## Metadata +#### Metadata Nullable reference types are emitted as a reference type with an assembly-level attribute in C# 8.0. That is, the following code: @@ -79,7 +79,7 @@ F# will be aware of and respect this attribute, treating reference types as null F# will also emit this attribute when it produces a nullable reference type for other languages to consume. -## F# concept and syntax +### F# concept and syntax Nullable reference types can conceptually be thought of a union between a reference type and `null`. For example, `string` in F# 4.5 or C# 7.3 is implicitly either a `string` or `null`. More concretely, they are a special case of [Erased type-tagged anonymous union types](https://github.com/fsharp/fslang-suggestions/issues/538). @@ -126,7 +126,7 @@ The syntax accomplishes two goals: Because it is a design goal to flow **non-null** reference types, and **avoid** flowing nullable reference types unless absolutely necessary, it is considered a feature that the syntax can get a bit unweildy. -### Nullable reference type declarations in F\# +#### Nullable reference type declarations in F\# There will be no way to declare an F# type as follows: @@ -134,11 +134,11 @@ There will be no way to declare an F# type as follows: type (NotSupportedAtAll | null)() = class end ``` -## Checking of nullable references +### Checking of nullable references There are a few common ways that F# programmers check for null today. Unless explicitly mentioned as unsupported, support for flow analysis is under consideration for certain null-checking patterns. -### Likely supported: Pattern matching with alias +#### Likely supported: Pattern matching with alias ```fsharp let len (str: string | null) = @@ -149,7 +149,7 @@ let len (str: string | null) = This is by far the most common pattern in F# programming. In the previous code sample, `str` is of type `string | null`, but `s` is now of type `string`. This is because we know that based on the `null` has been accounted for by the `null` pattern. -### Possibly supported: Pattern matching with wildcard +##### Possibly supported: Pattern matching with wildcard ```fsharp let len (str: string | null) = @@ -160,7 +160,7 @@ let len (str: string | null) = A slight riff that is less common is treating `str` differently when in a wildcard pattern's scope. To reach that code path, `str` must indeed by non-null, so this could potentially be supported. -### Possibly supported: If check +#### Possibly supported: If check ```fsharp let len (str: string | null) = @@ -172,7 +172,7 @@ let len (str: string | null) = Although less common, this is a valid way to check for `null` and could be enabled by the compiler. We know for certain that `str` is a `string` in the `else` branch. The inverse check is also true. -### Possibly supported: After null check within boolean expression +#### Possibly supported: After null check within boolean expression ```fsharp let len (str: string | null) = @@ -184,7 +184,7 @@ let len (str: string | null) = Note that the reverse (`str.Length > 0 && str <> null`) would give a warning, because we attempt to dereference before the `null` check. This would only hold with AND checks. More generally, if the boolean expression to the left of the dereference involves a `x <> null` check, then the dereference is safe. -### Likely unsupported: isNull +#### Likely unsupported: isNull ```fsharp let len (str: string | null) = @@ -207,7 +207,7 @@ let len (str: string | null) = There is no telling what `someCondition` would evaluate to, so we cannot guarantee that `str` will be `string` in the `else` clause. -### Likely unsupported: boolean-based checks for null +#### Likely unsupported: boolean-based checks for null Generally speaking, beyond highly-specialized cases, we cannot guarantee non-nullability that is checked via a boolean. Consider the following: @@ -223,7 +223,7 @@ let len (str: string | null) = Although this simple example could potentially work, it could quickly get out of hand and complicate the compiler. Also, other languages that support nullable reference types (C# 8.0, Scala, Kotlin, Swift, TypeScript) do not do this. Instead, non-nullability is asserted when the programmer knows something will not be `null`. -### Likely unsupported: nested functions accessing outer scopes +#### Likely unsupported: nested functions accessing outer scopes ```fsharp let len (str: string | null) = @@ -239,7 +239,7 @@ Although `doWork()` is called in a place where `str` would be non-null, this may **Note:** C# supports the equivalent of this with local functions. -### Asserting non-nullability +#### Asserting non-nullability To get around scenarios where flow analysis cannot establish a non-null situation, a programmer can use `!` to assert non-nullability: @@ -253,7 +253,7 @@ let len (str: string | null) = This will not generate a warning, but it is unsafe and can be the cause of a `NullReferenceException` if the reference was indeed `null` under some condition. -### Warnings +#### Warnings In all other situations not covered by flow analysis, a warning is emitted by the compiler if a nullable reference is dereferenced or casted to a non-null type: @@ -271,9 +271,9 @@ A warning is given when casting from `'S[]` to `('T | null)[]` and from `('S | n A warning is given when casting from `C<'S>` to `C<'T | null>` and from `C<'S | null>` to `C<'T>`. -## Checking of non-null references +### Checking of non-null references -### Null assignment and passing +#### Null assignment and passing A warning is given if `null` is assigned to a non-null value or passed as a parameter where a non-null reference type is expected: @@ -287,19 +287,7 @@ let f (s: string) = () f null // WARNING ``` -### Nullable parameterization - -A warning is given if a nullable reference type `P` is used to parameterize another type `C` that does not accept nullable reference types: - -```fsharp -type CNullParam<'T>() = class end - -let c1 = C // WARNING -``` - -Today, it is already a compile error if all fields in an F# class are not initialized by constructor calls. This behavior is unchanged. See [Compatibility with Microsoft.FSharp.Core.DefaultValueAttribute](FS-1060-nullable-reference-types.md#compatibility-with-microsoft.fsharp.core.defaultValueAttribute) for considerations about code that uses `[]` with reference types today, which produces `null`. - -### F# Collection initialization +#### F# Collection initialization A warning is given if `null` is used in an F# array/list/seq expression, where the type is already annotated to be non-nullable: @@ -309,59 +297,207 @@ let ys: string list = [ ""; ""; null ] // WARNING let zs: seq = seq { yield ""; yield ""; yield null } // WARNING ``` -### Unchecked.defaultOf<'T> +#### Unchecked.defaultOf<'T> Today, `Unchecked.defaultof<'T>` will generate `null` when `'T` is a reference type. Given this, it is reasonable to do either of the following: * The return type of `Unchecked.defaultof<'T>` is now `'T | null`. Calling `Unchecked.defaultof<'T | null>` produces the same output. * Give a warning when `Unchecked.defaultof<'T>` is called when `'T` is a non-nullable reference type, indicating that you must call `Unchecked.defaultof<'T | null>`. -### Array.zeroCreate<'T> +#### Array.zeroCreate<'T> Today, `Array.zeroCreate<'T>` will generate `null`s when `'T` is a reference type. Given this, it is reasonable to do either of the following: * The return type of `Array.zeroCreate<'T>` is now `('T | null) []`. Calling `Array.zeroCreate<'T | null>` produces the same output. * Give a warning when `Array.zeroCreate<'T>` is called where `'T` is a non-nullable reference type, indicating that you must call `Array.zeroCreate<'T | null>`. -## Type inference +#### Constructor initialization -### F# Collection initialization +Today, it is already a compile error if all fields in an F# class are not initialized by constructor calls. This behavior is unchanged. See [Compatibility with Microsoft.FSharp.Core.DefaultValueAttribute](FS-1060-nullable-reference-types.md#compatibility-with-microsoft.fsharp.core.defaultValueAttribute) for considerations about code that uses `[]` with reference types today, which produces `null`. -Using `null` will "infect" +### Generics -## Tooling considerations +A type parameter is assumed to have non-nullable constraints when declared "normally": -### F# tooltips +```fsharp +type C<'T>() = // 'T is non-nullable +class end +``` -TODO +A warning is given if a nullable reference type is used to parameterize another type `C` that does not accept nullable reference types: + +```fsharp +type C<'T>() = class end -### Signatures in FSI +let c = C // WARNING +``` -TODO +#### Constraints -### F# scripting +Today, there are two relevant constraints in F# - `null` and `not struct`: -TODO +```fsharp +type C1<'T when 'T: null>() = class end +type C2<'T when 'T: not struct>() = class end +``` + +Both are actually the same thing for interoperation purposes, as they emit `class` as a constraint. See [Unresolved questions](nullable-reference-types.md#unresolved-questions) for more. -## Breaking changes +If an F# class `B` has nullability as a constraint, then an inheriting class `D` also inherits this constraint, as per existing inheritance rules: -There are two forms of breaking changes with this feature: +```fsharp +type B<'T | null>() = class end +type D<'T | null>() + inherit B<'T> +``` + +That is, nullabulity constraints propagate via inheritance. + +### Type inference + +Nullability is propagated through type inference: + +```fsharp +let makeTupleWithNull (a) +// Inferred signature: +// +// val makeNullIfEmpty : str:string -> string | null +let makeNullIfEmpty (str: string) = + match str with + | "" -> null + | _ -> str + + // val xs : (string | null) list +let xs = [ ""; ""; null ] + +// val ys : (string | null) [] +let ys = [| ""; null; "" |] + +// val zs : seq +let zs = seq { yield ""; yield null } + +// val x : 'a | null +let x = null + +// val s : string +let mutable s = "hello" + +// s is now (s | null), despite the warning +s <- null +``` + +This includes function composition: + +```fsharp +// val f1 : 'a -> 'a +let f1 s = s + +// val f2 : string -> (string | null) +let f2 s = if s <> "" then "hello" else null + +// val f3 : string -> string +let f3 (s: string) = + match s with + | null -> "" + | _ -> s + +// val pipeline : string -> (string | null) +let pipeline = f1 >> f2 >> f3 // WARNING - flowing possible null into non-null parameter +``` + +Existing nullability rules for F# types hold (i.e., you cannot suddenly assign `null` to an F# record). + +### Tooling considerations + +To remain in line our first principle: + +> On the surface, F# is "almost" as simple to use as it is today. In practice, it must feel simpler due to nullability of types like `string` being explicit rather than implicit. + +We'll consider careful suppression of nullability in F# tooling so that the extra information doesn't appear overwhelming. For example, imagine the following signature in tooltips: + +```fsharp +member Join : source2: (Expression | null> | null) * key1:(Expression | null> | null)) * key2:(Expression | null> | null)) +``` + +AGH! This signature is challenging enough already, but now nullability has made it significantly worse. + +#### F# tooltips + +In QuickInfo tooltips today, we reduce the length of generic signatures with a section below the signature. We can do a similar thing for saying if something is nullable: + +``` +member Foo : source: (seq<'T>) * predicate: ('T -> 'U- > bool) * item: 'U + + where 'T, 'U are nullable + +Generic parameters: +'T is string +'U is int +``` + +Although this uses more vertical space in the tooltip, it reduces the length of the signature, which can already difficult to parse for many members in .NET. + +In Signature Help tooltips today, we embed the concrete type for a generic type parameter in the tooltip. We may consider doing a similar thing as with QuickInfo, where nullability is expressed separately from the signature. However, this would also likely require a change where we factor out concrete types from the type signature at the parameter site. For example: + +``` +List(System.Collections.Generics.IEnumerable<'T>) + + where 'T is (string | null) + +Initializes an instance of a [...] +``` + +#### Signatures in F# Interactive + +To have parity with F# signatures (note: not tooltips), signatures printed in F# interactive will have nullability expression as if it were in code: + +``` +> let s: string | null = "hello";; +> +> val s : string | null = "hello" +``` + +#### F# scripting + +F# scripts will now also give warnings when using the compiler that implements this feature. This also necessitates a new feature for F# scripting that allows you to specify a language version so that you can opt-out of any nullability warnings. Example: + +```fsharp +#langlevel "4.5" + +let s: string = null // OK, since 4.5 doesn't implement nullable references +``` + +By default, F# scripts that use a newer compiler via F# interactive will understand nullability. + +### Breaking changes and tunability + +Non-null warnings are an obvious breaking change with existing code, and thus will be opt-in for existing projects. New projects will have non-null warnings be on by default. + +Type inference will infer a reference type to be nullable in cases where it can be the result of an F# expression. This will generate warnings for existing code utilizing type inference. It will obviously also do so when dereferencing a reference type inferred to be nullable. Thus, warnings for nullability must be off by default for existing projects and on by default for new ones. + +Additionally, adding nullable annotations to an API will be a breaking change for users who have opted into warnings when they upgrade a library to use the new API. This also warrants the ability to turn off warnings. + +Finally, F# code in particular is quite aggressive with respect to non-null today. It is expected that a lot of F# users would like these warnings to actually be errors, independently of turning all warnings into errors. This should also be possible. + +In summary: + +| Warning kind | Existing projects | New projects | Tun into an error? | +|--------------|-------------------|--------------|--------------------| +| Nullable warnings | Off by default | On by default | Yes | +| Non-nullable warnings | Off by default | On by default | Yes | +| Warnings from other files | Off by default | On by default | Yes | +| Warnings from other assemblies | Off by default | On by default | Yes | -* Non-null warnings (i.e., not checking for `null`) on a `string | null` type -* Nullable warnings -TODO ## Drawbacks [drawbacks]: #drawbacks -Why should we *not* do this? +Although it could be argued that this simplifies reference types by making the fact that they could be `null` explicit ## Alternatives [alternatives]: #alternatives -What other designs have been considered? What is the impact of not doing this? - ### Syntax #### Question mark @@ -441,7 +577,7 @@ The following class `D`: type D<'T when 'T: null>() = class end ``` -will compile into this: +Is equivalent to this: ```csharp [Serializable] @@ -463,7 +599,41 @@ However, this will be highly confusing in C# 8.0 as per the [C# 8.0 proposal](ht What this means is that F# syntax that declares a type as accepting only types that have `null` as a proper value as type parameters actually accepts _only_ non-nullable types! This behavior is the exact opposite of what programmers likely believe what this syntax does, and is not compatible with a world where nullable reference types exist. -## Compatibility with Microsoft.FSharp.Core.DefaultValueAttribute +**Recommendation:** Change to emit as a nullable reference type constraint. + +### Compatibility with existing not struct constraint + +The existing `not struct` constraint for generic types requires the parameterizing type to be a .NET referennce type. + +The following class `C`: + +```fsharp +type C<'T when 'T : not struct>() = class end +``` + +Is equivalent to this: + +```csharp +[Serializable] +[CompilationMapping(SourceConstructFlags.ObjectType)] +public class C2 where T : class +{ + public C2() + : this() + { + } +} +``` + +Which is fine in today's world. However, the `class` constraint will now mean **non-null** reference type in C# 8.0: + +> The `class` constraint is **non-null**. We can consider whether `class?` should be a valid nullable constraint denoting "nullable reference type". + +Although it's true that "not struct" still maps to **non-null** reference type, this might be unexpected. + +**Recommendation:** Change to emit to a nullable reference type constraint. + +### Compatibility with Microsoft.FSharp.Core.DefaultValueAttribute Today, the `[]` attribute is required on explicit fields in classes that have a primary constructor. These fields must support zero-initialization, which in the case of reference types, is `null`. @@ -473,7 +643,7 @@ Consider the following code: type C() = [] val mutable Whoops : string - + printfn "%d" (C().Whoops.Length) ``` From 3dab702f551870d3ca9b57d802e3a0d65b12ac83 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Wed, 27 Jun 2018 15:39:37 -0700 Subject: [PATCH 06/46] Completed draft --- RFCs/FS-1060-nullable-reference-types.md | 88 +++++++++++++++++------- 1 file changed, 63 insertions(+), 25 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index b2a5e9bc..3b06f032 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -353,6 +353,18 @@ type D<'T | null>() That is, nullabulity constraints propagate via inheritance. +If an unconstrained type parameter comes from an older F# or C# assembly, it is assumed to be nullable, and warnings are given when non-nullability is assumed. + +### Interaction between older and newer projects + +#### New consuming old + +All reference types from older projects are assumed to be nullable until they are compiled with a newer F# compiler. That is, including a newer project into an older system should not "infect" those older projects. + +### Old consuming new + +All reference types consumped from a newer project are "normal" reference types as far as the older project is considered. Because the older version of the F# compiler does not respect the `[Nullable]` attribute, there is no difference from the view of that project. + ### Type inference Nullability is propagated through type inference: @@ -402,7 +414,7 @@ let f3 (s: string) = | _ -> s // val pipeline : string -> (string | null) -let pipeline = f1 >> f2 >> f3 // WARNING - flowing possible null into non-null parameter +let pipeline = f1 >> f2 >> f3 // WARNING - flowing possible null as input to function that assumes a non-null input ``` Existing nullability rules for F# types hold (i.e., you cannot suddenly assign `null` to an F# record). @@ -453,8 +465,8 @@ To have parity with F# signatures (note: not tooltips), signatures printed in F# ``` > let s: string | null = "hello";; -> -> val s : string | null = "hello" + + val s : string | null = "hello" ``` #### F# scripting @@ -475,7 +487,7 @@ Non-null warnings are an obvious breaking change with existing code, and thus wi Type inference will infer a reference type to be nullable in cases where it can be the result of an F# expression. This will generate warnings for existing code utilizing type inference. It will obviously also do so when dereferencing a reference type inferred to be nullable. Thus, warnings for nullability must be off by default for existing projects and on by default for new ones. -Additionally, adding nullable annotations to an API will be a breaking change for users who have opted into warnings when they upgrade a library to use the new API. This also warrants the ability to turn off warnings. +Additionally, adding nullable annotations to an API will be a breaking change for users who have opted into warnings when they upgrade a library to use the new API. This also warrants the ability to turn off warnings for specific assemblies. Finally, F# code in particular is quite aggressive with respect to non-null today. It is expected that a lot of F# users would like these warnings to actually be errors, independently of turning all warnings into errors. This should also be possible. @@ -485,22 +497,39 @@ In summary: |--------------|-------------------|--------------|--------------------| | Nullable warnings | Off by default | On by default | Yes | | Non-nullable warnings | Off by default | On by default | Yes | -| Warnings from other files | Off by default | On by default | Yes | | Warnings from other assemblies | Off by default | On by default | Yes | - +These kinds of warnings should be individually tunable on a per-project basis. ## Drawbacks [drawbacks]: #drawbacks -Although it could be argued that this simplifies reference types by making the fact that they could be `null` explicit +Although it could be argued that this simplifies reference types by making the fact that they could be `null` explicit, it does give programmers another thing that must explicitly account for. Although it fits within the "spirit" of F# to make nullability of reference types explicit, it's arguable that F# programmers need to account for enough things already. + +This is also a very complicated feature, with lots of edge cases, that has significantly reduces utilitity if any of the following are true: + +* There is insufficient flow analysis such that F# programmers are asserting non-nullability all over the place just to avoid unnecessary warnings +* Existing, working code is somehow infected without the programmer explicitly opting into this new behavior (indeed, this would be horrible) +* The rest of the .NET ecosystem simply does not adopt non-nullability, thus making adornments the new normal + +Although we cannot control the third point, ensuring that the implement is complete and up to spec with respect to what C# also does (while accounting for F#-isms) is the only way to do our part in helping the ecosystem produce less nulls. + +Additionally, we now have the following possible things to account for with respect to accessing underlying data: + +* Nullable reference types +* Option +* ValueOption +* Choice +* Result + +These are a lot of ways to send data to various parts of the system, and there is a danger that people could use nullable reference types for more than just the boundary levels of their system. ## Alternatives [alternatives]: #alternatives ### Syntax -#### Question mark +#### Question mark nullability annotation ```fsharp let ns: string? = "hello" @@ -517,38 +546,47 @@ type C() = But it is a lot easier to implement since it doesn't require ad-hoc, structurally-typed unions. -#### Nullable +#### Nulref<'T> -Find a way to make semantics with the existing `System.Nullable<'T>` type. +Have a wrapper type for any reference type `'T` that could be null. Issues: -* Back-compat issue? -* Really ugly nesting -* No syntax for this makes F# be "behind" C# for such a cornerstone thing +* Really ugly nesting is going to happen +* No syntax for this makes F# seen as "behind" C# for such a cornerstone feature of the .NET ecosystem -#### Option +#### Option or ValueOption Find a way to make this work with the existing Option type. Issues: -* Back compat issue? -* `None` already emits `null`, but it's a distinct case and not erased -* References and Nullable references are fundamentally the same thing, just one has an assembly-level attribute +* Could this even work in the first place? +* `None` already emits `null`, but it's a distinct case and not erased at compile time +* References and Nullable references are fundamentally the same thing, just one has an assembly-level attribute. This is simply not the same as an optional type. -### Warnings +### Assering non-nullability -TODO +**`!!` postfix operator?** -### Assering non-nullability +Why two `!` when one could suffice? -TODO +**`.get()` member?** + +No idea if this could even work. + +**Explicit cast required?** + +Not possible with current plan of emitting warnings with casting. ### Unresolved questions [unresolved]: #unresolved-questions -## Compatibility with existing F# nullability features +#### Handling warnings + +What about the ability to turn off nullability warnings when annotations are coming from a particular file? + +#### Compatibility with existing F# nullability features Today, classes declared in F# are non-null by default. To that end, F# has a way to express opt-in nullability with `[]`. Attempting to assign `null` to a class without this attribute is a compile error: @@ -567,7 +605,7 @@ c2 <- null // OK This behavior is quite similar to nullable reference types, but is unfortunately a bit of a cleaver when all you need is a paring knife. It's arguable that the value of nullability for F# programmers is _ad-hoc_ in nature, which is something that is squarely accomplished with nullable reference types. Instead, `[]` sort of "pollutes" things by making any instantiation a nullable instantiation. -### Compatibility with existing null constraint +#### Compatibility with existing null constraint The existing `null` constraint for generic types prevents programmers from parameterizing a type with an F# reference type that does not have `null` as a proper value (i.e., decorated with `[]`). @@ -601,7 +639,7 @@ What this means is that F# syntax that declares a type as accepting only types t **Recommendation:** Change to emit as a nullable reference type constraint. -### Compatibility with existing not struct constraint +#### Compatibility with existing not struct constraint The existing `not struct` constraint for generic types requires the parameterizing type to be a .NET referennce type. @@ -633,7 +671,7 @@ Although it's true that "not struct" still maps to **non-null** reference type, **Recommendation:** Change to emit to a nullable reference type constraint. -### Compatibility with Microsoft.FSharp.Core.DefaultValueAttribute +#### Compatibility with Microsoft.FSharp.Core.DefaultValueAttribute Today, the `[]` attribute is required on explicit fields in classes that have a primary constructor. These fields must support zero-initialization, which in the case of reference types, is `null`. From d5c31dc57f925247bb4a6a0defc122aafb496ea4 Mon Sep 17 00:00:00 2001 From: cartermp Date: Thu, 5 Jul 2018 10:44:01 -0700 Subject: [PATCH 07/46] Clarify constraints --- RFCs/FS-1060-nullable-reference-types.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 3b06f032..c17493ff 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -353,7 +353,7 @@ type D<'T | null>() That is, nullabulity constraints propagate via inheritance. -If an unconstrained type parameter comes from an older F# or C# assembly, it is assumed to be nullable, and warnings are given when non-nullability is assumed. +If an unconstrained type parameter comes from an older C# assembly, it is assumed to be nullable, and warnings are given when non-nullability is assumed. If it comes from F# and it does not have the `null` constraint, then it is assumed to be non-nullable. If it does, then it is assumed to be nullable. ### Interaction between older and newer projects @@ -506,13 +506,13 @@ These kinds of warnings should be individually tunable on a per-project basis. Although it could be argued that this simplifies reference types by making the fact that they could be `null` explicit, it does give programmers another thing that must explicitly account for. Although it fits within the "spirit" of F# to make nullability of reference types explicit, it's arguable that F# programmers need to account for enough things already. -This is also a very complicated feature, with lots of edge cases, that has significantly reduces utilitity if any of the following are true: +This is also a very complicated feature, with lots of edge cases, that has significantly reduced utility if any of the following are true: * There is insufficient flow analysis such that F# programmers are asserting non-nullability all over the place just to avoid unnecessary warnings -* Existing, working code is somehow infected without the programmer explicitly opting into this new behavior (indeed, this would be horrible) +* Existing, working code is infected without the programmer explicitly opting into this new behavior * The rest of the .NET ecosystem simply does not adopt non-nullability, thus making adornments the new normal -Although we cannot control the third point, ensuring that the implement is complete and up to spec with respect to what C# also does (while accounting for F#-isms) is the only way to do our part in helping the ecosystem produce less nulls. +Although we cannot control the third point, ensuring that the implementation is complete and up to spec with respect to what C# also does (while accounting for F#-isms) is the only way to do our part in helping the ecosystem produce less nulls. Additionally, we now have the following possible things to account for with respect to accessing underlying data: @@ -541,12 +541,12 @@ Issue: looks horrible with F# optional parameters ```fsharp type C() = - memeber __.M(?x: string?) = "BLEGH" + member __.M(?x: string?) = "BLEGH" ``` But it is a lot easier to implement since it doesn't require ad-hoc, structurally-typed unions. -#### Nulref<'T> +#### Nullref<'T> Have a wrapper type for any reference type `'T` that could be null. From 5ea04a2507b48d1ef6e6bbcf439603b2b0e085d3 Mon Sep 17 00:00:00 2001 From: cartermp Date: Thu, 5 Jul 2018 11:24:23 -0700 Subject: [PATCH 08/46] More on constraints and move the defaultof/zerocreate down to unresolved --- RFCs/FS-1060-nullable-reference-types.md | 67 +++++++++++++++--------- 1 file changed, 42 insertions(+), 25 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index c17493ff..21de3572 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -52,7 +52,7 @@ The following principles guide all further considerations and the design of this ## Detailed design [design]: #detailed-design -#### Metadata +### Metadata Nullable reference types are emitted as a reference type with an assembly-level attribute in C# 8.0. That is, the following code: @@ -79,11 +79,21 @@ F# will be aware of and respect this attribute, treating reference types as null F# will also emit this attribute when it produces a nullable reference type for other languages to consume. -### F# concept and syntax +#### Nullability obliviousness + +It is desireable for some assemblies to declare themselves as nullability-oblivious, even if they were compiled with a compiler that has this as a concept. This is a top-level attribute that will be respected by the C# compiler. F# should also respect this as an attribute. + +#### Handling nullability + +C# 8.0 will also respect a new attribute that can mark a method as "handling null". This is to get around the non-determinism in checking if any arbitrary method call (and its calls) check for `null` and do not re-assign it somehow. -Nullable reference types can conceptually be thought of a union between a reference type and `null`. For example, `string` in F# 4.5 or C# 7.3 is implicitly either a `string` or `null`. More concretely, they are a special case of [Erased type-tagged anonymous union types](https://github.com/fsharp/fslang-suggestions/issues/538). +This attribute (we'll call it `[]`) will also be respected by F#, so that obvious calls such as `String.IsNullorEmpty` that wrap a dereference don't trigger a warning. + +This attribute can be abused, since you can attach it to something that absolutely does not check for `null`. + +### F# concept and syntax -This concept is lifted into syntax: +Nullable reference types can conceptually be thought of a union between a reference type and `null`. For example, `string` in F# 4.5 or C# 7.3 is implicitly either a `string` or `null`. This concept is lifted into syntax: ``` reference-type | null @@ -124,7 +134,7 @@ The syntax accomplishes two goals: 1. Makes the distinction that a nullable reference type can be `null` clear in syntax. 2. Becomes a bit unweildy when nullability is used a lot, particularly with nested types. -Because it is a design goal to flow **non-null** reference types, and **avoid** flowing nullable reference types unless absolutely necessary, it is considered a feature that the syntax can get a bit unweildy. +Because it is a design goal to flow **non-null** reference types, and **avoid** flowing nullable reference types unless absolutely necessary, it is considered a feature that the syntax can get a bit unweildy if used everywhere. That is, we want to discourage the use of `null` unless it is absolutely necessary. #### Nullable reference type declarations in F\# @@ -232,7 +242,7 @@ let len (str: string | null) = match str with | null -> -1 - | _ -> doWork() + | _ -> doWork() // But after this line is written it's fine? ``` Although `doWork()` is called in a place where `str` would be non-null, this may too complex to implement properly. @@ -297,20 +307,6 @@ let ys: string list = [ ""; ""; null ] // WARNING let zs: seq = seq { yield ""; yield ""; yield null } // WARNING ``` -#### Unchecked.defaultOf<'T> - -Today, `Unchecked.defaultof<'T>` will generate `null` when `'T` is a reference type. Given this, it is reasonable to do either of the following: - -* The return type of `Unchecked.defaultof<'T>` is now `'T | null`. Calling `Unchecked.defaultof<'T | null>` produces the same output. -* Give a warning when `Unchecked.defaultof<'T>` is called when `'T` is a non-nullable reference type, indicating that you must call `Unchecked.defaultof<'T | null>`. - -#### Array.zeroCreate<'T> - -Today, `Array.zeroCreate<'T>` will generate `null`s when `'T` is a reference type. Given this, it is reasonable to do either of the following: - -* The return type of `Array.zeroCreate<'T>` is now `('T | null) []`. Calling `Array.zeroCreate<'T | null>` produces the same output. -* Give a warning when `Array.zeroCreate<'T>` is called where `'T` is a non-nullable reference type, indicating that you must call `Array.zeroCreate<'T | null>`. - #### Constructor initialization Today, it is already a compile error if all fields in an F# class are not initialized by constructor calls. This behavior is unchanged. See [Compatibility with Microsoft.FSharp.Core.DefaultValueAttribute](FS-1060-nullable-reference-types.md#compatibility-with-microsoft.fsharp.core.defaultValueAttribute) for considerations about code that uses `[]` with reference types today, which produces `null`. @@ -355,6 +351,8 @@ That is, nullabulity constraints propagate via inheritance. If an unconstrained type parameter comes from an older C# assembly, it is assumed to be nullable, and warnings are given when non-nullability is assumed. If it comes from F# and it does not have the `null` constraint, then it is assumed to be non-nullable. If it does, then it is assumed to be nullable. +Additionally, C# will introduce the `class?` constraint. This syntax sugar will compile into the `class` constraint, but the class itself will be decorated with an attribute that tells consumers that the `'T` is a nullable reference type. This attribute will need to be respected by the F# compiler so that C# + ### Interaction between older and newer projects #### New consuming old @@ -637,11 +635,13 @@ However, this will be highly confusing in C# 8.0 as per the [C# 8.0 proposal](ht What this means is that F# syntax that declares a type as accepting only types that have `null` as a proper value as type parameters actually accepts _only_ non-nullable types! This behavior is the exact opposite of what programmers likely believe what this syntax does, and is not compatible with a world where nullable reference types exist. -**Recommendation:** Change to emit as a nullable reference type constraint. +Additionally, C# 8.0 will introduce the `class?` constraint. This is syntax sugar for an attribute that will decorate the class, informing consumers that `T` is a nullable reference type. So `class` and `class?` are distinctly different things in C# now. + +**Recommendation:** Change to emit as a `class` constraint but with the same attribute decorating the class that C# uses. #### Compatibility with existing not struct constraint -The existing `not struct` constraint for generic types requires the parameterizing type to be a .NET referennce type. +The existing `not struct` constraint for generic types requires the parameterizing type to be a .NET reference type. The following class `C`: @@ -667,9 +667,9 @@ Which is fine in today's world. However, the `class` constraint will now mean ** > The `class` constraint is **non-null**. We can consider whether `class?` should be a valid nullable constraint denoting "nullable reference type". -Although it's true that "not struct" still maps to **non-null** reference type, this might be unexpected. +Today in F# you cannot assign `null` as a proper value to `C` when it is constrained with `not struct` within F#. So it is already semantically similar to how things will work in C# 8.0. What this does mean is that C# 8.0 code that consumes this class will see `T` as a non-nullable reference type. This is consistent with the changes being made to constraints in C# 8.0. -**Recommendation:** Change to emit to a nullable reference type constraint. +**Recommendation:** Make no change. #### Compatibility with Microsoft.FSharp.Core.DefaultValueAttribute @@ -685,4 +685,21 @@ type C() = printfn "%d" (C().Whoops.Length) ``` -Today, this produces a `NullReferenceException` at runtime. This means that `Whoops` is actually a nullable reference type, but it will be annotated as if it were a non-nullable reference type. It is the presence of the attribute that changes this, which we'll have to rationalize somehow. \ No newline at end of file +Today, this produces a `NullReferenceException` at runtime. + +`Whoops` is annotated as a reference type, but it is actuall a nullable reference type due to being decorated with the attribute. This means that we'll either have to: + +(a) Do nothing and leave this as a confusing thing that emits a warning at the call site, but not at the declaration site +(b) Emit a warning saying that we're calling a nullable reference type a non-nullable reference type + +Either way, we'll need to respect the `[]` attribute. + +#### Unchecked.defaultOf<'T> + +Today, `Unchecked.defaultof<'T>` will generate `null` when `'T` is a reference type. However, `'T` is non-null, and we'll want to support a `'T | null` being passed in as well. Given this, the signature could be changed to `'T | null`. This could compile down to be an assembly-level attribute, so older F# compilers consuming a newer FSharp.Core wouldn't see the constraint. + +However, would this still work for value types being passed in? It has to. + +#### Array.zeroCreate<'T> + +The same question for `Unchecked.defaultof<'T>` exists for `Array.zeroCreate<'T>`. We'll want to make it clear that `'T` is nullable, but this has to be fully backwards-compatible. And older compilers should still be able to use this function from a newer FSharp.Core. \ No newline at end of file From 0c76201f6c4cc8800d4d843de10411ceea8484ff Mon Sep 17 00:00:00 2001 From: cartermp Date: Thu, 5 Jul 2018 11:32:00 -0700 Subject: [PATCH 09/46] Update flow analysis to account for handlesnull annotation --- RFCs/FS-1060-nullable-reference-types.md | 43 ++++++++++-------------- 1 file changed, 17 insertions(+), 26 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 21de3572..cf21faa2 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -159,6 +159,20 @@ let len (str: string | null) = This is by far the most common pattern in F# programming. In the previous code sample, `str` is of type `string | null`, but `s` is now of type `string`. This is because we know that based on the `null` has been accounted for by the `null` pattern. +#### Likely supported: isNull and others using null-handling annotation + +```fsharp +let len (str: string | null) = + if isNull str then + -1 + else + str.Length // OK +``` + +The `isNull` function is considered to be a good way to check for `null`. We can annotate it in FSharp.Core with the attribute used for functions handling null (we'll call it `[]`). + +Similarly, CoreFX will annotate methods like `String.IsNullOrEmpty` as handling `null`. This means that the most common of null-checking methods can be respected by flow analysis and reduce the amount of nullability annotations. + ##### Possibly supported: Pattern matching with wildcard ```fsharp @@ -194,30 +208,7 @@ let len (str: string | null) = Note that the reverse (`str.Length > 0 && str <> null`) would give a warning, because we attempt to dereference before the `null` check. This would only hold with AND checks. More generally, if the boolean expression to the left of the dereference involves a `x <> null` check, then the dereference is safe. -#### Likely unsupported: isNull - -```fsharp -let len (str: string | null) = - if isNull str then - -1 - else - str.Length // WARNING: could be null -``` - -The `isNull` function is considered to be a good way to check for `null`. Unfortunately, because it is a function that returns `bool`, the logic needed to support this is deeply foreign to F# and troublesome to implement. For example, consider the following: - -```fsharp -let len (str: string | null) = - let someCondition = getCondition() - if isNull str && someCondition then - -1 - else - str.Length // WARNING: could be null -``` - -There is no telling what `someCondition` would evaluate to, so we cannot guarantee that `str` will be `string` in the `else` clause. - -#### Likely unsupported: boolean-based checks for null +#### Likely unsupported: boolean-based checks for null that lack an annotation Generally speaking, beyond highly-specialized cases, we cannot guarantee non-nullability that is checked via a boolean. Consider the following: @@ -231,7 +222,7 @@ let len (str: string | null) = str.Length // WARNING: could be null ``` -Although this simple example could potentially work, it could quickly get out of hand and complicate the compiler. Also, other languages that support nullable reference types (C# 8.0, Scala, Kotlin, Swift, TypeScript) do not do this. Instead, non-nullability is asserted when the programmer knows something will not be `null`. +Although this simple example could potentially work, it could quickly get out of hand. Also, other languages that support nullable reference types (C# 8.0, Scala, Kotlin, Swift, TypeScript) do not do this. Instead, non-nullability is asserted when the programmer knows something will not be `null`. #### Likely unsupported: nested functions accessing outer scopes @@ -245,7 +236,7 @@ let len (str: string | null) = | _ -> doWork() // But after this line is written it's fine? ``` -Although `doWork()` is called in a place where `str` would be non-null, this may too complex to implement properly. +Although `doWork()` is called only in a scope where `str` would be non-null, this may too complex to implement properly. **Note:** C# supports the equivalent of this with local functions. From 89a470217dc85a8168530b592598d5ca714bd3db Mon Sep 17 00:00:00 2001 From: cartermp Date: Thu, 5 Jul 2018 11:32:53 -0700 Subject: [PATCH 10/46] Fix --- RFCs/FS-1060-nullable-reference-types.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index cf21faa2..b8d29e20 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -159,7 +159,7 @@ let len (str: string | null) = This is by far the most common pattern in F# programming. In the previous code sample, `str` is of type `string | null`, but `s` is now of type `string`. This is because we know that based on the `null` has been accounted for by the `null` pattern. -#### Likely supported: isNull and others using null-handling annotation +#### Likely supported: isNull and others using null-handling decoration ```fsharp let len (str: string | null) = @@ -171,7 +171,7 @@ let len (str: string | null) = The `isNull` function is considered to be a good way to check for `null`. We can annotate it in FSharp.Core with the attribute used for functions handling null (we'll call it `[]`). -Similarly, CoreFX will annotate methods like `String.IsNullOrEmpty` as handling `null`. This means that the most common of null-checking methods can be respected by flow analysis and reduce the amount of nullability annotations. +Similarly, CoreFX will annotate methods like `String.IsNullOrEmpty` as handling `null`. This means that the most common of null-checking methods can be respected by flow analysis and reduce the amount of nullability assertions. ##### Possibly supported: Pattern matching with wildcard @@ -208,7 +208,7 @@ let len (str: string | null) = Note that the reverse (`str.Length > 0 && str <> null`) would give a warning, because we attempt to dereference before the `null` check. This would only hold with AND checks. More generally, if the boolean expression to the left of the dereference involves a `x <> null` check, then the dereference is safe. -#### Likely unsupported: boolean-based checks for null that lack an annotation +#### Likely unsupported: boolean-based checks for null that lack handlesnull decoration Generally speaking, beyond highly-specialized cases, we cannot guarantee non-nullability that is checked via a boolean. Consider the following: From 33a12174a3ce8b4cbcbf51f6d8fe9af856711615 Mon Sep 17 00:00:00 2001 From: cartermp Date: Thu, 5 Jul 2018 11:45:45 -0700 Subject: [PATCH 11/46] remove falsehood --- RFCs/FS-1060-nullable-reference-types.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index b8d29e20..2c8108e0 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -533,8 +533,6 @@ type C() = member __.M(?x: string?) = "BLEGH" ``` -But it is a lot easier to implement since it doesn't require ad-hoc, structurally-typed unions. - #### Nullref<'T> Have a wrapper type for any reference type `'T` that could be null. From e4fb4ff152c6b1e444f7dadcde7be229e7af6b27 Mon Sep 17 00:00:00 2001 From: cartermp Date: Thu, 5 Jul 2018 11:46:30 -0700 Subject: [PATCH 12/46] spelling fix --- RFCs/FS-1060-nullable-reference-types.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 2c8108e0..a211d911 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -552,7 +552,7 @@ Issues: * `None` already emits `null`, but it's a distinct case and not erased at compile time * References and Nullable references are fundamentally the same thing, just one has an assembly-level attribute. This is simply not the same as an optional type. -### Assering non-nullability +### Asserting non-nullability syntax **`!!` postfix operator?** From 827b8fcfaae3c7eb9ef15ab68e6a325795b2b413 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Thu, 5 Jul 2018 17:03:17 -0700 Subject: [PATCH 13/46] Clarify nullable declarations --- RFCs/FS-1060-nullable-reference-types.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index a211d911..5096ea28 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -144,6 +144,8 @@ There will be no way to declare an F# type as follows: type (NotSupportedAtAll | null)() = class end ``` +And in existing F# today, the only way to declare an F# reference type that could be `null` in F# code is to use `[]`. This would not change with this proposal; that is, the only way to declare a reference type in F# that could have `null` as a value in F# code is `[]`. + ### Checking of nullable references There are a few common ways that F# programmers check for null today. Unless explicitly mentioned as unsupported, support for flow analysis is under consideration for certain null-checking patterns. From 618761e35d78808efd0771cc70328183c33bf3df Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Thu, 5 Jul 2018 17:20:56 -0700 Subject: [PATCH 14/46] Add recommendation for F#-declared reference types --- RFCs/FS-1060-nullable-reference-types.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 5096ea28..96f29e7b 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -462,12 +462,12 @@ To have parity with F# signatures (note: not tooltips), signatures printed in F# #### F# scripting -F# scripts will now also give warnings when using the compiler that implements this feature. This also necessitates a new feature for F# scripting that allows you to specify a language version so that you can opt-out of any nullability warnings. Example: +F# scripts will now also give warnings when using the compiler that implements this feature. This also necessitates a new feature for F# scripting that allows you to opt-out of any nullability warnings. Example: ```fsharp -#langlevel "4.5" +#nullability "false" -let s: string = null // OK, since 4.5 doesn't implement nullable references +let s: string = null // OK, since we don't track nullability warnings ``` By default, F# scripts that use a newer compiler via F# interactive will understand nullability. @@ -594,6 +594,8 @@ c2 <- null // OK This behavior is quite similar to nullable reference types, but is unfortunately a bit of a cleaver when all you need is a paring knife. It's arguable that the value of nullability for F# programmers is _ad-hoc_ in nature, which is something that is squarely accomplished with nullable reference types. Instead, `[]` sort of "pollutes" things by making any instantiation a nullable instantiation. +**Recommendation**: Relax this restriction from an error to a warning, and then treat any reference to types decorated with `[]` as if they were nullable reference types. This is not a breaking change, but it does mean that F#-declared reference types are now a bit different once this feature is in place. + #### Compatibility with existing null constraint The existing `null` constraint for generic types prevents programmers from parameterizing a type with an F# reference type that does not have `null` as a proper value (i.e., decorated with `[]`). @@ -678,12 +680,12 @@ printfn "%d" (C().Whoops.Length) Today, this produces a `NullReferenceException` at runtime. -`Whoops` is annotated as a reference type, but it is actuall a nullable reference type due to being decorated with the attribute. This means that we'll either have to: +`Whoops` is annotated as a reference type, but it is actually a nullable reference type due to being decorated with the attribute. This means that we'll either have to: (a) Do nothing and leave this as a confusing thing that emits a warning at the call site, but not at the declaration site -(b) Emit a warning saying that we're calling a nullable reference type a non-nullable reference type +(b) Emit a warning saying that decoration with `[]` means that the type annotation is incorrect -Either way, we'll need to respect the `[]` attribute. +Either way, we'll need to respect the `[]` attribute generating a `null` to remain compatible with existing code. #### Unchecked.defaultOf<'T> From 626ce3e4bb9757fde9716bb64118bd9899e35b44 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Thu, 5 Jul 2018 17:22:02 -0700 Subject: [PATCH 15/46] Better formatting --- RFCs/FS-1060-nullable-reference-types.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 96f29e7b..b459126d 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -682,8 +682,8 @@ Today, this produces a `NullReferenceException` at runtime. `Whoops` is annotated as a reference type, but it is actually a nullable reference type due to being decorated with the attribute. This means that we'll either have to: -(a) Do nothing and leave this as a confusing thing that emits a warning at the call site, but not at the declaration site -(b) Emit a warning saying that decoration with `[]` means that the type annotation is incorrect +* Do nothing and leave this as a confusing thing that emits a warning at the call site, but not at the declaration site +* Emit a warning saying that decoration with `[]` means that the type annotation is incorrect Either way, we'll need to respect the `[]` attribute generating a `null` to remain compatible with existing code. From 16a31ed5b259b47e96c41c219333b505504c41f9 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Thu, 5 Jul 2018 17:30:52 -0700 Subject: [PATCH 16/46] team redundancy team --- RFCs/FS-1060-nullable-reference-types.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index b459126d..26e4785d 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -10,7 +10,7 @@ The design suggestion [Add non-nullable instantiations of nullable types, and in ## Summary [summary]: #summary -The main goal of this feature is to address the pains of dealing with implicitly nullable reference types by providing syntax and behavior that distinguishes between nullable and non-nullable reference types. This will be done in lockstep with C# 8.0, which will also have this as a feature, and F# will interoperate with C# by using and emitting the same metadata adornment that C# emits emit. +The main goal of this feature is to address the pains of dealing with implicitly nullable reference types by providing syntax and behavior that distinguishes between nullable and non-nullable reference types. This will be done in lockstep with C# 8.0, which will also have this as a feature, and F# will interoperate with C# by respecting and emitting the same metadata that C# emits. Conceptually, reference types can be thought of as having two forms: From 551ccbb34716b7368e33f11664b9f997c0fc8cb9 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Thu, 5 Jul 2018 17:32:27 -0700 Subject: [PATCH 17/46] warning about my bad english --- RFCs/FS-1060-nullable-reference-types.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 26e4785d..5bf855e2 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -17,10 +17,12 @@ Conceptually, reference types can be thought of as having two forms: * Normal reference types * Nullable reference types -These will be distinguished via syntax, type signatures, and tools. Additionally, there will be flow analysis to allow you to determine when a nullable reference type is non-null and can be "viewed" as a reference type. +These will be distinguished via syntax, type signatures, and tools. Additionally, there will be flow analysis to allow you to determine when a nullable reference type is non-null and can be treated as a reference type. Warnings are emitted when reference and nullable reference types are not used according with their intent, and these warnings are tunable as errors. +For the purposes of this document, the terms "reference type" and "non-nullable reference type" may be used interchangeably. They refer to the same thing. + ## Motivation [motivation]: #motivation From d38f063bbb35255150a40fe666adc2bd65a0c8d1 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Thu, 5 Jul 2018 17:35:09 -0700 Subject: [PATCH 18/46] does not --- RFCs/FS-1060-nullable-reference-types.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 5bf855e2..6d9b1df3 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -44,7 +44,7 @@ The following principles guide all further considerations and the design of this 4. Nullability annotations/information is carefully suppressed and simplified in tooling (tooltips, FSI) to avoid extra information overload. 5. F# users are typically concerned with this feature at the boundaries of their system so that they can flow non-null data into the inner parts of their system. 6. The feature produces warnings by default, with different classes of warnings (nullable, non-nullable) offering opt-in/opt-out mechanisms. -7. All existing F# projects compile with warnings turned off by default. Only new projects have warnings on by default. Compiling older F# code with newer F# code may opt-in the older F# code. +7. All existing F# projects compile with warnings turned off by default. Only new projects have warnings on by default. Compiling older F# code with newer F# code does not opt-in the older F# code. 8. F# non-nullness is reasonably "strong and trustworthy" today for F#-defined types and routine F# coding. This includes the explicit use of `option` types to represent the absence of information. No compile-time guarantees today will be "lowered" to account for this feature. 9. The F# compiler will strive to provide flow analysis such that common null-check scenarios guarantee null-safety when working with nullable reference types. 10. This feature is useful for F# in other contexts, such as interoperation with `string` in JavaScript via [Fable](http://fable.io/). From 68ecc69420a6ad27f7f65cbf14d219cdff1d1681 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Thu, 5 Jul 2018 17:42:39 -0700 Subject: [PATCH 19/46] broaden example --- RFCs/FS-1060-nullable-reference-types.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 6d9b1df3..1e154585 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -242,7 +242,7 @@ let len (str: string | null) = Although `doWork()` is called only in a scope where `str` would be non-null, this may too complex to implement properly. -**Note:** C# supports the equivalent of this with local functions. +**Note:** C# supports the equivalent of this with local functions. TypeScript does not support this. #### Asserting non-nullability From d98746b685d9d611f1b1b0ec7a2b61e977e65c63 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Thu, 5 Jul 2018 17:43:45 -0700 Subject: [PATCH 20/46] myIsNull --- RFCs/FS-1060-nullable-reference-types.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 1e154585..a51bb2bb 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -249,8 +249,10 @@ Although `doWork()` is called only in a scope where `str` would be non-null, thi To get around scenarios where flow analysis cannot establish a non-null situation, a programmer can use `!` to assert non-nullability: ```fsharp +let myIsNull item = isNull item + let len (str: string | null) = - if isNull str then + if myIsNull str then -1 else str!.Length // OK: 'str' is asserted to be 'null' From a4a2a7e0ffaa75adf267a21c61af8dd012ca62c3 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Thu, 5 Jul 2018 17:46:33 -0700 Subject: [PATCH 21/46] finish sentence --- RFCs/FS-1060-nullable-reference-types.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index a51bb2bb..3f6ba711 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -348,7 +348,7 @@ That is, nullabulity constraints propagate via inheritance. If an unconstrained type parameter comes from an older C# assembly, it is assumed to be nullable, and warnings are given when non-nullability is assumed. If it comes from F# and it does not have the `null` constraint, then it is assumed to be non-nullable. If it does, then it is assumed to be nullable. -Additionally, C# will introduce the `class?` constraint. This syntax sugar will compile into the `class` constraint, but the class itself will be decorated with an attribute that tells consumers that the `'T` is a nullable reference type. This attribute will need to be respected by the F# compiler so that C# +Additionally, C# will introduce the `class?` constraint. This syntax sugar will compile into the `class` constraint, but the class itself will be decorated with an attribute that tells consumers that the `'T` is a nullable reference type. This attribute will need to be respected by the F# compiler. ### Interaction between older and newer projects From fe6a787f7c6b2ecab850aa8b3ed846a89cb54c5d Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Thu, 5 Jul 2018 17:47:35 -0700 Subject: [PATCH 22/46] get rid of lingering thingie --- RFCs/FS-1060-nullable-reference-types.md | 1 - 1 file changed, 1 deletion(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 3f6ba711..cac7338c 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -365,7 +365,6 @@ All reference types consumped from a newer project are "normal" reference types Nullability is propagated through type inference: ```fsharp -let makeTupleWithNull (a) // Inferred signature: // // val makeNullIfEmpty : str:string -> string | null From 77573f52776cbf0cce1aa77f29fa2eb08ee17948 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Mon, 9 Jul 2018 15:03:28 -0700 Subject: [PATCH 23/46] Format strings, option consideration, and option emission --- RFCs/FS-1060-nullable-reference-types.md | 56 ++++++++++++++++++++++-- 1 file changed, 52 insertions(+), 4 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index cac7338c..e9c12cbe 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -553,9 +553,8 @@ Find a way to make this work with the existing Option type. Issues: -* Could this even work in the first place? -* `None` already emits `null`, but it's a distinct case and not erased at compile time -* References and Nullable references are fundamentally the same thing, just one has an assembly-level attribute. This is simply not the same as an optional type. +* Massively breaking change for any F# code consuming existing reference types. In many cases, this break is so severe that entire routines would need to be rewritten. +* Massively breaking change for any C# code consuming F# option types. This simply breaks C# consumers no matter which way you swing it. ### Asserting non-nullability syntax @@ -698,4 +697,53 @@ However, would this still work for value types being passed in? It has to. #### Array.zeroCreate<'T> -The same question for `Unchecked.defaultof<'T>` exists for `Array.zeroCreate<'T>`. We'll want to make it clear that `'T` is nullable, but this has to be fully backwards-compatible. And older compilers should still be able to use this function from a newer FSharp.Core. \ No newline at end of file +The same question for `Unchecked.defaultof<'T>` exists for `Array.zeroCreate<'T>`. We'll want to make it clear that `'T` is nullable, but this has to be fully backwards-compatible. And older compilers should still be able to use this function from a newer FSharp.Core. + +#### Format specifiers + +Today, the following specifiers work with reference types: + +* `%s`, for strings +* `%O`, which calls `ToString()` +* `%A`/`%+A`, for anything, using reflection to find values to print +* `%a` and `%t`, which work with `System.IO.TextWriter` + +Making these work with non-nullable reference types would be a breaking change, so it's likely that the underlying type be the appropriate `T | null` for each specifier. For example, `%s` would work with `string | null`. + +**Recommendation:** These now operate assuming incoming types are nullable reference types. + +#### Emitting F# options types for C# consumption + +F# options types emit `None` as `null`. For example, considering the following public F# API: + +```fsharp +type C() = + member __.M(doot) = if doot then Some "doot" else None +``` + +This is equivalent to the following C# 7.x code: + +```csharp +[Serializable] +[CompilationMapping(SourceConstructFlags.ObjectType)] +public class C +{ + public C() + { + ((object)this)..ctor(); + } + + public FSharpOption M(bool doot) + { + if (doot) + { + return FSharpOption.Some("doot"); + } + return null; + } +} +``` + +This means that `FSharpOption` is a nullable reference type if it were to be consumed by C# 8.0. Indeed, there is definitely C# code out there that checks for `null` when consuming an F# option type to account for the `None` case. This is because there is no other way to safely extract the underlying value in C#! + +**Recommendation:** ROLL INTO A BALL AND CRY \ No newline at end of file From bea9ed60e497b4ed4b9090b6ca68a902ab9c5a8f Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Wed, 11 Jul 2018 15:02:06 -0700 Subject: [PATCH 24/46] More considerations for syntax, interaction model, and unresolved questions --- RFCs/FS-1060-nullable-reference-types.md | 88 +++++++++++++++++++----- 1 file changed, 69 insertions(+), 19 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index e9c12cbe..c313945d 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -3,7 +3,7 @@ The design suggestion [Add non-nullable instantiations of nullable types, and interop with proposed C# 8.0 non-nullable reference types](https://github.com/fsharp/fslang-suggestions/issues/577) has been marked "approved in principle". This RFC covers the detailed proposal for this suggestion. * [x] Approved in principle -* [ ] [Suggestion](https://github.com/fsharp/fslang-suggestions/issues/577) +* [x] [Suggestion](https://github.com/fsharp/fslang-suggestions/issues/577) * [ ] Details: [TODO](https://github.com/fsharp/fslang-design/issues/FILL-ME-IN) * [ ] Implementation: [TODO](https://github.com/Microsoft/visualfsharp/pull/FILL-ME-IN) @@ -350,16 +350,6 @@ If an unconstrained type parameter comes from an older C# assembly, it is assume Additionally, C# will introduce the `class?` constraint. This syntax sugar will compile into the `class` constraint, but the class itself will be decorated with an attribute that tells consumers that the `'T` is a nullable reference type. This attribute will need to be respected by the F# compiler. -### Interaction between older and newer projects - -#### New consuming old - -All reference types from older projects are assumed to be nullable until they are compiled with a newer F# compiler. That is, including a newer project into an older system should not "infect" those older projects. - -### Old consuming new - -All reference types consumped from a newer project are "normal" reference types as far as the older project is considered. Because the older version of the F# compiler does not respect the `[Nullable]` attribute, there is no difference from the view of that project. - ### Type inference Nullability is propagated through type inference: @@ -475,6 +465,40 @@ let s: string = null // OK, since we don't track nullability warnings By default, F# scripts that use a newer compiler via F# interactive will understand nullability. +### Interaction model + +The following details how an F# compiler with nullable reference types will behave when interacting with different kinds of components. + +#### F# 5.0 consuming non-F# assemblies that do not have nullability + +All non-F# assemblies built with a compiler that does not understand nullability are treated such that all of their reference types are nullable. + +Example: a `.dll` coming from a NuGet package built with C# 7.1 will be interpreted as if all reference types are nullable, including constraints, generics, etc. + +This means that warnings will be emitted when `null` is not properly accounted for. Additionally, when older non-F# components are eventually recompiled with C# 8.0, it will likely be an API-breaking change that will force new F# code to go back and do away with any redundant `null` checks. + +#### F# 5.0 consuming F# assemblies that do not have nullability + +All F# assemblies built with a previous F# compiler will be treated as such: + +* All reference types that are _not_ declared in F# are nullable. +* All F#-declared reference types that have `null` as a proper value are treated as if they are nullable. +* All F#-declared reference types that do not have `null` as a proepr value are treated as non-nullable reference types. + +#### Ignoring F# 5.0 assemblies that do have nullability + +Users may want to progressively work through `null` warnings by treating a given assembly as "nullability obliviousness". To respect this, F# 5.0 will have to ignore any potentially unsafe dereference of `null` until sucha time that "nullability obliviousness" is turned off for that assembly. + +**Potential issue:** How are these types annotated in code? `string` or `string | null`, with the latter simply not triggering any warnings? + +#### Older F# components consuming non-F# assemblies that do have nullability + +Because the nullability attribute is only understood by F# 5.0, their presence has no impact on existing F# codebases. Nothing is different. + +#### Older F# components consuming F# assemblies that do have nullability + +F# components are no different than non-F# components when it comes to being consumed by an older F# codebase. The nullability attribute is simply ignored. + ### Breaking changes and tunability Non-null warnings are an obvious breaking change with existing code, and thus will be opt-in for existing projects. New projects will have non-null warnings be on by default. @@ -556,7 +580,7 @@ Issues: * Massively breaking change for any F# code consuming existing reference types. In many cases, this break is so severe that entire routines would need to be rewritten. * Massively breaking change for any C# code consuming F# option types. This simply breaks C# consumers no matter which way you swing it. -### Asserting non-nullability syntax +### Asserting non-nullability **`!!` postfix operator?** @@ -570,12 +594,31 @@ No idea if this could even work. Not possible with current plan of emitting warnings with casting. +**New keyword with a scope?** + +Something like this may be considered: + +```console +notnull { expr } +``` + +Where `notnull` returns the type of `expr`. But this may be viewed as too complicated when compared with a postfix operator, especially since a postfix operator is already a standard established by C#, Kotlin, TypeScript, and Swift. + +**Not have it?** + +This is untenable, unless we also implement [the ability to disable a warning just in one place within a file](https://github.com/fsharp/fslang-suggestions/issues/278). Otherwise, users will either: + +* Have a nonzero number of warnings they cannot disable via a non-nullability assertion +* Ignore all warnings in a file + +Both options will likely be considered unacceptable by F# programmers. + ### Unresolved questions [unresolved]: #unresolved-questions #### Handling warnings -What about the ability to turn off nullability warnings when annotations are coming from a particular file? +What about the ability to turn off nullability warnings when annotations are coming from a particular file? Or only have nullability warnings on a per-file basis? Answer: that would be weird. #### Compatibility with existing F# nullability features @@ -596,7 +639,7 @@ c2 <- null // OK This behavior is quite similar to nullable reference types, but is unfortunately a bit of a cleaver when all you need is a paring knife. It's arguable that the value of nullability for F# programmers is _ad-hoc_ in nature, which is something that is squarely accomplished with nullable reference types. Instead, `[]` sort of "pollutes" things by making any instantiation a nullable instantiation. -**Recommendation**: Relax this restriction from an error to a warning, and then treat any reference to types decorated with `[]` as if they were nullable reference types. This is not a breaking change, but it does mean that F#-declared reference types are now a bit different once this feature is in place. +**Recommendation**: Relax this restriction from an error to a warning in F# 5.0 only, and then treat any reference to types decorated with `[]` as if they were nullable reference types. This is not a breaking change, but it does mean that F#-declared reference types are now a bit different once this feature is in place. #### Compatibility with existing null constraint @@ -632,7 +675,7 @@ What this means is that F# syntax that declares a type as accepting only types t Additionally, C# 8.0 will introduce the `class?` constraint. This is syntax sugar for an attribute that will decorate the class, informing consumers that `T` is a nullable reference type. So `class` and `class?` are distinctly different things in C# now. -**Recommendation:** Change to emit as a `class` constraint but with the same attribute decorating the class that C# uses. +**Recommendation:** Change this in F# 5.0 to emit as a `class` constraint but with the same attribute decorating the class that C# uses. #### Compatibility with existing not struct constraint @@ -664,7 +707,7 @@ Which is fine in today's world. However, the `class` constraint will now mean ** Today in F# you cannot assign `null` as a proper value to `C` when it is constrained with `not struct` within F#. So it is already semantically similar to how things will work in C# 8.0. What this does mean is that C# 8.0 code that consumes this class will see `T` as a non-nullable reference type. This is consistent with the changes being made to constraints in C# 8.0. -**Recommendation:** Make no change. +**Recommendation:** Make no change for F# 5.0 #### Compatibility with Microsoft.FSharp.Core.DefaultValueAttribute @@ -691,13 +734,20 @@ Either way, we'll need to respect the `[]` attribute generating a #### Unchecked.defaultOf<'T> -Today, `Unchecked.defaultof<'T>` will generate `null` when `'T` is a reference type. However, `'T` is non-null, and we'll want to support a `'T | null` being passed in as well. Given this, the signature could be changed to `'T | null`. This could compile down to be an assembly-level attribute, so older F# compilers consuming a newer FSharp.Core wouldn't see the constraint. +Today, `Unchecked.defaultof<'T>` will generate `null` when `'T` is a reference type. However, `'T` means non-null, and it's obvious that this will return `null` when parameterized with a .NET reference type such as `string`. We have some options: + +* Keep the existing signature, despite being a bit confusing, and have it act as an unconstrained generic function that will return `null` for reference types that have `null` as a proper value +* Create a variant of this function that works in terms of nullable reference types, and somehow convey that you need to call this for reference types, and the existing function for non-reference types -However, would this still work for value types being passed in? It has to. +The latter option is terrible, so the former is probably better. #### Array.zeroCreate<'T> -The same question for `Unchecked.defaultof<'T>` exists for `Array.zeroCreate<'T>`. We'll want to make it clear that `'T` is nullable, but this has to be fully backwards-compatible. And older compilers should still be able to use this function from a newer FSharp.Core. +The same issue for `Unchecked.defaultof<'T>` exists for `Array.zeroCreate<'T>`. + +#### typeof<'T> and typedefof<'T> + +We cannot actually guarantee that the underlying `System.Type` derived from `'T` is nullable or not. This is due to there being no mechanism in reflection today to understand the nullability attribute. This represents an inherent unsoundness of the nullability feature unless some approach for dealing with this is derived. #### Format specifiers From f1f7e857d68c789669c6e9cd1a6b16627d63b115 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Fri, 13 Jul 2018 10:30:31 -0700 Subject: [PATCH 25/46] unsoundness --- RFCs/FS-1060-nullable-reference-types.md | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index c313945d..c9163cbe 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -91,7 +91,7 @@ C# 8.0 will also respect a new attribute that can mark a method as "handling nul This attribute (we'll call it `[]`) will also be respected by F#, so that obvious calls such as `String.IsNullorEmpty` that wrap a dereference don't trigger a warning. -This attribute can be abused, since you can attach it to something that absolutely does not check for `null`. +It's worth noting that this attribute could be abused. ### F# concept and syntax @@ -143,7 +143,8 @@ Because it is a design goal to flow **non-null** reference types, and **avoid** There will be no way to declare an F# type as follows: ```fsharp -type (NotSupportedAtAll | null)() = class end +// This will not be possible! +type (SomeClass | null)() = class end ``` And in existing F# today, the only way to declare an F# reference type that could be `null` in F# code is to use `[]`. This would not change with this proposal; that is, the only way to declare a reference type in F# that could have `null` as a value in F# code is `[]`. @@ -522,6 +523,8 @@ These kinds of warnings should be individually tunable on a per-project basis. ## Drawbacks [drawbacks]: #drawbacks +### Complexity + Although it could be argued that this simplifies reference types by making the fact that they could be `null` explicit, it does give programmers another thing that must explicitly account for. Although it fits within the "spirit" of F# to make nullability of reference types explicit, it's arguable that F# programmers need to account for enough things already. This is also a very complicated feature, with lots of edge cases, that has significantly reduced utility if any of the following are true: @@ -542,9 +545,23 @@ Additionally, we now have the following possible things to account for with resp These are a lot of ways to send data to various parts of the system, and there is a danger that people could use nullable reference types for more than just the boundary levels of their system. +### Unsoundness + +This feature is inherently unsound. For starters, warnings don't prevent people from flowing `null` through their code like errors do. Compare this with the F# option type, which offers zero way to attempt to coerce `None` into a `Some x`. + +More subtly, there is no way to extract nullability information with reflection today without a change in the .NET runtime. This means that it can be impossible to know, at least by reflection, if the type of something is actually nullable or non-nullable. This means that reflection-based libraries (such as JSON.NET) may be able to define a contract that they cannot fulfill; i.e., make an API's return type a `'T` when it could still be `null`. + +Finally, the "handles null" attribute can be trivially abused, and it may be likely that the F# compiler could be "tricked" into thinking something will no longer be `null`. + +All of this means that this feature is entrusting acting in good faith on the greater .NET ecosystem, rather than preventing acting in bad faith with compile-time guarantees. + ## Alternatives [alternatives]: #alternatives +The primary alternative is to simply not do this. + +That said, it will become quite evident that this feature is necessary if F# is to continue advancing on the .NET platform. Despite originating as a C# feature, this is fundamentally a **platform-level shift** for .NET, and in a few years, will result in .NET looking very different than it is today. + ### Syntax #### Question mark nullability annotation From 94a1045b4beb667b73ad2c68f0cd98ee94716a28 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Mon, 16 Jul 2018 10:15:29 -0700 Subject: [PATCH 26/46] More reasons for no question mark --- RFCs/FS-1060-nullable-reference-types.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index c9163cbe..c2cab90e 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -579,6 +579,13 @@ type C() = member __.M(?x: string?) = "BLEGH" ``` +Issue: how to rationalize this with dynamic lookup + +```fsharp +// This will look confusing +let y: NullableDynamicMember? = Something?SomeDynamicMember +``` + #### Nullref<'T> Have a wrapper type for any reference type `'T` that could be null. From d33eb7ab4d9dccb64b8979e132ac682ebb125960 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Wed, 25 Jul 2018 17:03:35 -0700 Subject: [PATCH 27/46] Partial feedback --- RFCs/FS-1060-nullable-reference-types.md | 110 ++++++++++++++++++++--- 1 file changed, 98 insertions(+), 12 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index c2cab90e..9c76c67a 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -10,7 +10,7 @@ The design suggestion [Add non-nullable instantiations of nullable types, and in ## Summary [summary]: #summary -The main goal of this feature is to address the pains of dealing with implicitly nullable reference types by providing syntax and behavior that distinguishes between nullable and non-nullable reference types. This will be done in lockstep with C# 8.0, which will also have this as a feature, and F# will interoperate with C# by respecting and emitting the same metadata that C# emits. +The main goal of this feature is to address the pains of dealing with implicitly nullable reference types by providing syntax and behavior that distinguishes between nullable and non-nullable reference types. Conceptually, reference types can be thought of as having two forms: @@ -21,18 +21,25 @@ These will be distinguished via syntax, type signatures, and tools. Additionally Warnings are emitted when reference and nullable reference types are not used according with their intent, and these warnings are tunable as errors. +This will be done in lockstep with C# 8.0, which will also have this as a feature, and F# will interoperate with C# by respecting and emitting the same metadata that C# emits. + For the purposes of this document, the terms "reference type" and "non-nullable reference type" may be used interchangeably. They refer to the same thing. ## Motivation [motivation]: #motivation -Today, any reference type (e.g., `string`) could be `null`, forcing developers to account for this with special checks at the top-level of their functions or methods. Especially in F#, where types are almost always assumed to be non-null, this is a pain that can often be forgotten. Moreover, one of the primary benefits in working with F# is working with immutable and **non-null** data. By this statement alone, nullable reference types are in line with the general goals of F#. +A dramatic shift is about to occur in the .NET ecosystem. Starting with C# 8.0, C# will distinguish between explicitly nullable reference types and reference types that cannot be `null`. That is, `string` will not be implicitly `null` in C# 8.0 or higher, and attempting to make it `null` will be a warning. + +This problem with reference types in .NET - that they are _implicitly_ `null` - is a longstanding tension between the non-null nature of F# programming. Now that the most-used language in .NET will emit information indicating explicit nullability, this tension has the possibility of being eased considerably for F# programmers. Explicit representation of `null` is very much in line with general F# programming for two primary reasons: -The `[]` attribute allows you to declare F# reference types as nullable, but this "infects" any place where the type might be used. In practice, F# types are not normally checked for `null`, which could mean that there is a lot of code out there that does not account for `null` when using a type that is decorated with `[]`. +1. The explicit representation of critical information in types, with the compiler enforcing this representation, is very much the "F# way" of doing things. Although backwards-compatible explicit nullability is not sound in the same way that F# options are (read on to find out why), it is nonetheless in the spirit of how F# does things. +2. One of the goals of F# programmers is to evict `null` values as early as possible from the edges of their system. Although there are some scenarios where this cannot be done, and `null` values must flow through a program, these scenarios are usually few and far between, or avoided entirely. When nullability is explicit, it is harder to "forget" that an incoming type may carry a `null` value. This makes it easier to evict `null` values from the rest of an F# program. +3. Another goal of F# programmers is to ensure that they do not produce `null` values unless it is absolutely necessary in the context in which they are working. When nullability as a concept is explicit, an F# programmer must do more work to produce a `null` value. When it is an annoyance to produce `null` values (especially of the implicit variety), then programmers will try to avoid producing them. +4. A variant of (3) is that it is possible to accidentaly produce a `null` value in F# code. With explicit nullability as a concept, this accidental behavior can be recognized by a compiler and a warning can be given to let the F# programmer know that they may not have intended sending `null` somewhere. -Additionally, nullable reference types is a headlining feature of C# 8.0 that will have a dramatic impact on .NET and its ecosystem. It is vital that we interoperate with this change smoothly and can emit the appropriate information so that C# consumers of F# components can consume things as if they were also C# 8.0 components. +F# has some existing mitigations against `null`, including that F#-declared reference types cannot be `null` unless explicitly decorated with `[]`. This is useful for when an F# programmers absolutely must flow `null` into a component. However, this action "infects" any other place where the type might be used, forcing those places to also account for `null`. This is rarely what F# programmers desire. -The desired outcome over time is that significantly less `NullReferenceException`s are produced by code at runtime. This feature should allow F# programmers to more safely eject `null` from their concerns when interoperating with other .NET components, and make it more explicit when `null` could be a problem. +The desired outcome over time is that significantly less `NullReferenceException`s are produced by code at runtime when itneroperating with C# libraries and other F# components that declare reference types as capable of having `null` as a proper value. This feature should allow F# programmers to more safely eject `null` from their concerns when interoperating with other .NET components, and make it more explicit when `null` could be a problem. ## Principles @@ -48,23 +55,57 @@ The following principles guide all further considerations and the design of this 8. F# non-nullness is reasonably "strong and trustworthy" today for F#-defined types and routine F# coding. This includes the explicit use of `option` types to represent the absence of information. No compile-time guarantees today will be "lowered" to account for this feature. 9. The F# compiler will strive to provide flow analysis such that common null-check scenarios guarantee null-safety when working with nullable reference types. 10. This feature is useful for F# in other contexts, such as interoperation with `string` in JavaScript via [Fable](http://fable.io/). -11. F# programmers can start to phase out their use of `[]` and the `null` type constraint when they need ad-hoc nullability for reference types declared in F#. +11. F# programmers may be able to start to phase out their use of `[]`, `Unchecked.defaultof<_>`, and the `null` type constraint when they need ad-hoc nullability for reference types declared in F#. 12. Syntax for the feature feels "baked in" to the type system, and should not be horribly unpleasant to use. ## Detailed design [design]: #detailed-design -### Metadata +### .NET Metadata + +To remain backwards-compatible with existing codebases and interoperate with other .NET languages that support nullability as a concept, F# will emit and respect .NET metadata that concers to the following: + +* If a type is a nullable reference type +* If an assembly has nullability as a concept (regardless of what it was compiled with) +* If a generic type constraint is nullable +* If a given method/function "handles null", such as `String.IsNullOrWhiteSpace` + +C# 8.0 will emit and respect metadata for these scenarios, with a well-known set of names for each attribute. These attributes will also be what F# uses, though the behavior of F# in the face of these attributes is not necessarily identical to how C# 8.0 behaves in the face of these attributes. -Nullable reference types are emitted as a reference type with an assembly-level attribute in C# 8.0. That is, the following code: +#### Representing nullable types + +The following attribute will be used to represent a type that is marked as nullable: ```csharp -public class D { +namespace System.Runtime.CompilerServices +{ + [AttributeUsage( + AttributeTargets.Class | + AttributeTargets.GenericParameter | + AttributeTargets.Event | + AttributeTargets.Field | + AttributeTargets.Property | + AttributeTargets.Parameter | + AttributeTargets.ReturnValue, + AllowMultiple = false)] + public sealed class NullableAttribute : Attribute + { + public NullableAttribute() { } + public NullableAttribute(bool[] b) { } + } +} +``` + +For example, the following code: + +```csharp +public class D +{ public void M(string? s) {} } ``` -Is the same as the following: +Is the same as: ```csharp public class D @@ -77,13 +118,58 @@ public class D Languages that are unaware of this attribute will see `M` as a method that takes in a `string`. That is, to be unaware of and failing to respect this attribute means that you will view incoming reference types exactly as before. -F# will be aware of and respect this attribute, treating reference types as nullable reference types if they have this attribute. If they do not, then those reference types will be non-nullable. +F# will be aware of and respect this attribute, treating reference types as nullable reference types if they have this attribute. If they do not, then those reference types will be treated as non-nullable. F# will also emit this attribute when it produces a nullable reference type for other languages to consume. +### Representing nullability as a concept + +As descibed in the [Interaction Model](FS-1060-nullable-reference-types.md#interaction-model), there are certain behaviors involved in how to work with assemblies that may or may not express nullability. + +C# 8.0 code will emit the `[NonUllTypes(true|false)]` attribute: + +```csharp +namespace System.Runtime.CompilerServices +{ + [AttributeUsage(AttributeTargets.Class | + AttributeTargets.Constructor | + AttributeTargets.Delegate | + AttributeTargets.Enum | + AttributeTargets.Event | + AttributeTargets.Field | + AttributeTargets.Interface | + AttributeTargets.Method | + AttributeTargets.Module | + AttributeTargets.Property | + AttributeTargets.Struct, + AllowMultiple = false)] + public sealed class NonNullTypesAttribute : Attribute + { + public NonNullTypesAttribute(bool enabled = true) { } + } +} +``` +From the C# spec on this: + +> Unannotated reference types are non-nullable or [null-oblivious](FS-1060-nullable-reference-types.md#nullability-obliviousness)) depending on whether the containing scope includes `[NonNullTypes(true|false)]`. +> +> `[NonNullTypes(true|false)]` is not synthesized by the compiler. If the attribute is used explicitly in source, the type declaration must be provided explicitly to the compilation. + +In other words, this attribute specifies a way to mark a class, constructor, delegate, enum, event, field, interface, method, module, property, or struct as having nullability expressable or not in a containing scope. + +This attribute could potentially be used in F# to allow for opt-in/opt-out nullability at a fine-grained level. + +F# will respect this attribute, with the `true` caseindicating that the scope distinguishes between nullable and non-nullable reference types. + +F# will treat scopes with the `false` case as if its contained reference types are all nullable reference types. + +In other words, if we cannot be sure that a given reference type is non-nullable, we assume it is nullable. + #### Nullability obliviousness -It is desireable for some assemblies to declare themselves as nullability-oblivious, even if they were compiled with a compiler that has this as a concept. This is a top-level attribute that will be respected by the C# compiler. F# should also respect this as an attribute. +Nullability obliviousness is a concept that C# 8.0 has. A null-oblivious type is one that no assumptions can be made about. Once assigned to a nullable or non-nullable variable, it is treated as if it is nullable or non-nullable, respectively. + +**F# will not have this concept. Reference types we cannot determine to be non-nullable will be assumed to be nullable.** #### Handling nullability From 97601e2761fa3470d4fba2d460e7adbe4df2a731 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Fri, 27 Jul 2018 16:59:16 -0700 Subject: [PATCH 28/46] Update round 2 --- RFCs/FS-1060-nullable-reference-types.md | 51 +++++++++++++++--------- 1 file changed, 32 insertions(+), 19 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 9c76c67a..dd2c3a05 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -165,19 +165,38 @@ F# will treat scopes with the `false` case as if its contained reference types a In other words, if we cannot be sure that a given reference type is non-nullable, we assume it is nullable. -#### Nullability obliviousness +As a final note, this attribute _can_ be abused. For example, if a method is annotated with `[NonNullTypes(true)]` + +#### Null obliviousness Nullability obliviousness is a concept that C# 8.0 has. A null-oblivious type is one that no assumptions can be made about. Once assigned to a nullable or non-nullable variable, it is treated as if it is nullable or non-nullable, respectively. **F# will not have this concept. Reference types we cannot determine to be non-nullable will be assumed to be nullable.** -#### Handling nullability +This is slightly controversial, because we are making an assumption about code that we do not really know about: + +* The code could not be handling `null` at all, and thus could either produce a `NullReferenceException` or produce a value that is nullable +* The code could be handling `null` just fine, and could pass back a reference type that can never be `null` +* We have no way to tell if any of the previous two points are true + +To remain "true" to the original code, we could conceivably introduce null obliviousness and thus treat the value coming out of a component as either nullable or non-nullable, depending on how we assign it. + +However, this is not actually possible in F#, because the majority of F# code uses type inference to infer a type. Null obliviousness would require explicit type annotations to work as we intend it! So we must assume nullability to remain safe. + +Additionally, treating all components we cannot guarantee as non-nullable as nullable is, we feel, "in the spirit of F#", which mandates safety and non-nullness wherever possible. -C# 8.0 will also respect a new attribute that can mark a method as "handling null". This is to get around the non-determinism in checking if any arbitrary method call (and its calls) check for `null` and do not re-assign it somehow. +#### Marking methods as handling null -This attribute (we'll call it `[]`) will also be respected by F#, so that obvious calls such as `String.IsNullorEmpty` that wrap a dereference don't trigger a warning. +There will be several attributes C# 8.0 code can emit: -It's worth noting that this attribute could be abused. +* `[NotNullWhenTrue]` - Indicates a method handles `null` if the result is `true`, e.g., a `TryGetValue` call. +* `[NotNullWhenFalse]` - Indicates a method handles `null` fi the result is `false`, e.g., a `string.IsNullOrEmpty` call. +* `[EnsuresNotNull]` - Indicates that the program cannot continue if a value is `null`, for example, a `ThrowIfNull` call. +* `[AssertsTrue]` and `[AssertsFalse]` - Used in assertion cases where `null` is concerned. + +These attributes can be used to aid in checking if `null` is properly accounted for in the body of a function or method. Common .NET methods and F#-specific functions can be annotated with these attributes. Respecting these attributes will be necessary to accurately and efficiently determine if `null` is properly accounted for before someone attempts to dereference a reference type. + +It is worth noting that these attributes can be abused and accidentally misused by applying them to something that does not account for `null` somehow, which can result in a `NullReferenceException` despite the compiler not indicating that it could be possible. That makes them dangerous, and third-party authors will need to be careful in applying them to their code. However, they do enable a class of checking that would otherwise be too computationally complex to do. ### F# concept and syntax @@ -187,7 +206,9 @@ Nullable reference types can conceptually be thought of a union between a refere reference-type | null ``` -Examples in F# code: +What this syntax means is that `reference-type` is non-null. + +Some examples in F# code: ```fsharp // Parameter to a function @@ -203,28 +224,20 @@ let findOrNull (index: int) (list: 'T list) : 'T | null = | None -> null // Declared type at let-binding -let maybeAValue : int | null = hopefullyGetAnInteger() - -// Generic type parameter -type IMyInterface<'T | null, > = - abstract DoStuff<'T | null> : unit -> 'T | null +let maybeAValue : string | null = hopefullyGetAString() // Array type signature -let f (arr: (float | null) []) = () - -// Nullable types with nullable generic types -type C<'T | null>() = - member __.M(param1: List | null> | null) = () +let f (arr: (string | null) []) = () ``` The syntax accomplishes two goals: 1. Makes the distinction that a nullable reference type can be `null` clear in syntax. -2. Becomes a bit unweildy when nullability is used a lot, particularly with nested types. +2. Becomes a bit unwieldy when nullability is used a lot, particularly with nested types. -Because it is a design goal to flow **non-null** reference types, and **avoid** flowing nullable reference types unless absolutely necessary, it is considered a feature that the syntax can get a bit unweildy if used everywhere. That is, we want to discourage the use of `null` unless it is absolutely necessary. +Because it is a design goal to support the use of non-null reference types by default, we want to encourage the programmer to **avoid** using nullable reference types unless absolutely necessary. -#### Nullable reference type declarations in F\# +#### No Nullable reference type declarations in F\# There will be no way to declare an F# type as follows: From 82aafeecaacfa1ccd858fbcc271e2a1e8f25a7d3 Mon Sep 17 00:00:00 2001 From: cartermp Date: Sun, 29 Jul 2018 17:43:09 -0700 Subject: [PATCH 29/46] Feedback round 2 --- RFCs/FS-1060-nullable-reference-types.md | 70 +++++++++++++++--------- 1 file changed, 45 insertions(+), 25 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index dd2c3a05..26594bd2 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -273,20 +273,9 @@ let len (str: string | null) = str.Length // OK ``` -The `isNull` function is considered to be a good way to check for `null`. We can annotate it in FSharp.Core with the attribute used for functions handling null (we'll call it `[]`). +The `isNull` function is considered to be a good way to check for `null`. We can annotate it in FSharp.Core with `[]`. -Similarly, CoreFX will annotate methods like `String.IsNullOrEmpty` as handling `null`. This means that the most common of null-checking methods can be respected by flow analysis and reduce the amount of nullability assertions. - -##### Possibly supported: Pattern matching with wildcard - -```fsharp -let len (str: string | null) = - match str with - | null -> -1 - | _ -> str.Length // OK - 'str' is string -``` - -A slight riff that is less common is treating `str` differently when in a wildcard pattern's scope. To reach that code path, `str` must indeed by non-null, so this could potentially be supported. +Similarly, CoreFX will annotate methods like `String.IsNullOrEmpty` as handling `null`. This means that the most common of null-checking methods can be respected by the compiler. #### Possibly supported: If check @@ -298,7 +287,7 @@ let len (str: string | null) = str.Length // OK - 'str' is a string ``` -Although less common, this is a valid way to check for `null` and could be enabled by the compiler. We know for certain that `str` is a `string` in the `else` branch. The inverse check is also true. +More generally, `value = null` or `value <> null`, where `value` is an F# expression that is a nullable reference type, is a pattern that will likely be expected to be respected. Although less common, `=` and `<>` are valid way to check for `null`, after all. Logically, we know for certain that `value` is either null (`=`) or non-null (`<>`). #### Possibly supported: After null check within boolean expression @@ -310,9 +299,38 @@ let len (str: string | null) = -1 ``` -Note that the reverse (`str.Length > 0 && str <> null`) would give a warning, because we attempt to dereference before the `null` check. This would only hold with AND checks. More generally, if the boolean expression to the left of the dereference involves a `x <> null` check, then the dereference is safe. +Note that the reverse (`str.Length > 0 && str <> null`) would give a warning, because we attempt to dereference before the `null` check. This would only hold with AND checks and if the value is immutable (thus cannot be assigned to in an expression). More generally, if the boolean expression to the left of the dereference involves a `x <> null` check, then the dereference is safe. -#### Likely unsupported: boolean-based checks for null that lack handlesnull decoration +##### Likely unsupported: Pattern matching with wildcard + +```fsharp +let len (str: string | null) = + match str with + | null -> -1 + | _ -> str.Length // OK - 'str' is string +``` + +A slight riff that is less common is treating `str` differently when in a wildcard pattern's scope. To reach that code path, `str` must indeed by non-null, so this could potentially be supported. + +This would actually inconsistent with typing rules today, so this may not be supported. For example, the following is a compile error + +```fsharp +let f (x: obj) = + match x with + | :? string -> x.Length // Error, as 'length' is not defined on obj +``` + +You must give it a different name to get the code to compile: + +```fsharp +let f (x: obj) = + match x with + | :? string as s -> s.Length // OK +``` + +Breaking this rule for F# would be inconsistent with behavior that already exists. + +#### Likely unsupported: boolean-based checks for null that lack an appropriate well-known attribute Generally speaking, beyond highly-specialized cases, we cannot guarantee non-nullability that is checked via a boolean. Consider the following: @@ -326,27 +344,27 @@ let len (str: string | null) = str.Length // WARNING: could be null ``` -Although this simple example could potentially work, it could quickly get out of hand. Also, other languages that support nullable reference types (C# 8.0, Scala, Kotlin, Swift, TypeScript) do not do this. Instead, non-nullability is asserted when the programmer knows something will not be `null`. +Although this trivial example could certainly be done, it could quickly get out of hand. Also, other languages that support nullable reference types (C# 8.0, Scala, Kotlin, Swift, TypeScript) do not do this. An annotation for `myIsNull` is needed, otherwise a programmer will have to rewrite their code or assert non-nullability and risk a `NullReferenceException`. #### Likely unsupported: nested functions accessing outer scopes ```fsharp let len (str: string | null) = let doWork() = - str.Length // WARNING: maybe too complicated? + str.Length // WARNING: maybe too complicated? Doesn't feel natural to F#? match str with | null -> -1 - | _ -> doWork() // But after this line is written it's fine? + | _ -> doWork() // But after this line is written it's fine? Not natural to F# though. ``` Although `doWork()` is called only in a scope where `str` would be non-null, this may too complex to implement properly. -**Note:** C# supports the equivalent of this with local functions. TypeScript does not support this. +**Note:** C# supports the equivalent of this with local functions. TypeScript does not support this. There is no precedent for this kind of support to further motivate the work. #### Asserting non-nullability -To get around scenarios where flow analysis cannot establish a non-null situation, a programmer can use `!` to assert non-nullability: +To get around scenarios where compiler analysis cannot establish a non-null situation, a programmer can use `!` to assert non-nullability: ```fsharp let myIsNull item = isNull item @@ -360,9 +378,11 @@ let len (str: string | null) = This will not generate a warning, but it is unsafe and can be the cause of a `NullReferenceException` if the reference was indeed `null` under some condition. +This sort of feature is inherently unsafe, and is by definition not something being enthusiastically considered. But it may be a necessity if the compiler cannot analyze code well enough. + #### Warnings -In all other situations not covered by flow analysis, a warning is emitted by the compiler if a nullable reference is dereferenced or casted to a non-null type: +In all other situations not covered by compiler analysis, a warning is emitted by the compiler if a nullable reference is dereferenced or cast to a non-null type: ```fsharp let f (ns: string | null) = @@ -558,7 +578,7 @@ To have parity with F# signatures (note: not tooltips), signatures printed in F# F# scripts will now also give warnings when using the compiler that implements this feature. This also necessitates a new feature for F# scripting that allows you to opt-out of any nullability warnings. Example: ```fsharp -#nullability "false" +#nowarn "nullness" let s: string = null // OK, since we don't track nullability warnings ``` @@ -674,7 +694,7 @@ type C<'T?>() = class end Issue: looks horrible with F# optional parameters ```fsharp -type C() = +type C() = member __.M(?x: string?) = "BLEGH" ``` @@ -703,7 +723,7 @@ Issues: * Massively breaking change for any F# code consuming existing reference types. In many cases, this break is so severe that entire routines would need to be rewritten. * Massively breaking change for any C# code consuming F# option types. This simply breaks C# consumers no matter which way you swing it. -### Asserting non-nullability +### Non-nullability Assertions **`!!` postfix operator?** From 9e1e3919708186b85c28a45d3bb75bc2911e5487 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Mon, 30 Jul 2018 17:02:47 -0700 Subject: [PATCH 30/46] Feedback round 3 (4?) --- RFCs/FS-1060-nullable-reference-types.md | 68 +++++++++++++++++++----- 1 file changed, 56 insertions(+), 12 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 26594bd2..4df16bc6 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -118,6 +118,8 @@ public class D Languages that are unaware of this attribute will see `M` as a method that takes in a `string`. That is, to be unaware of and failing to respect this attribute means that you will view incoming reference types exactly as before. +C# 8.0 will also [distinguish between nullable and non-nullable constraints](https://github.com/dotnet/roslyn/blob/features/NullableReferenceTypes/docs/features/nullable-reference-types.md#type-parameters). This is accomplished with the same attribute. + F# will be aware of and respect this attribute, treating reference types as nullable reference types if they have this attribute. If they do not, then those reference types will be treated as non-nullable. F# will also emit this attribute when it produces a nullable reference type for other languages to consume. @@ -330,6 +332,8 @@ let f (x: obj) = Breaking this rule for F# would be inconsistent with behavior that already exists. +Instead, we could offer a warning that suggests giving the type a new name. This would improve the existing error message today, so it's probably worth doing with this feature. + #### Likely unsupported: boolean-based checks for null that lack an appropriate well-known attribute Generally speaking, beyond highly-specialized cases, we cannot guarantee non-nullability that is checked via a boolean. Consider the following: @@ -424,27 +428,35 @@ let ys: string list = [ ""; ""; null ] // WARNING let zs: seq = seq { yield ""; yield ""; yield null } // WARNING ``` +When types are not annotated, rules for [Type Inference](FS-1060-nullable-reference-types.md#type-inference) are applied. + #### Constructor initialization Today, it is already a compile error if all fields in an F# class are not initialized by constructor calls. This behavior is unchanged. See [Compatibility with Microsoft.FSharp.Core.DefaultValueAttribute](FS-1060-nullable-reference-types.md#compatibility-with-microsoft.fsharp.core.defaultValueAttribute) for considerations about code that uses `[]` with reference types today, which produces `null`. ### Generics -A type parameter is assumed to have non-nullable constraints when declared "normally": +Unconstrainted types follow existing nullability rules today. ```fsharp type C<'T>() = // 'T is non-nullable class end ``` -A warning is given if a nullable reference type is used to parameterize another type `C` that does not accept nullable reference types: +Giving a reference type that could be `null` is fine and in accordance with existing code: ```fsharp type C<'T>() = class end -let c = C // WARNING +let c1 = C() // OK + +type B() = class end + +let c2 = C() // OK ``` +The same holds for generic function types. + #### Constraints Today, there are two relevant constraints in F# - `null` and `not struct`: @@ -454,13 +466,13 @@ type C1<'T when 'T: null>() = class end type C2<'T when 'T: not struct>() = class end ``` -Both are actually the same thing for interoperation purposes, as they emit `class` as a constraint. See [Unresolved questions](nullable-reference-types.md#unresolved-questions) for more. +Both are actually the same thing for interoperation purposes today, as they emit `class` as a constraint. See [Unresolved questions](nullable-reference-types.md#unresolved-questions) for more. C# 8.0 will interpret the `class` constraint as a non-nullable reference type, so we will have to change that in F#. If an F# class `B` has nullability as a constraint, then an inheriting class `D` also inherits this constraint, as per existing inheritance rules: ```fsharp -type B<'T | null>() = class end -type D<'T | null>() +type B<'T when 'T: null>() = class end +type D<'T when 'T: null>() inherit B<'T> ``` @@ -468,11 +480,25 @@ That is, nullabulity constraints propagate via inheritance. If an unconstrained type parameter comes from an older C# assembly, it is assumed to be nullable, and warnings are given when non-nullability is assumed. If it comes from F# and it does not have the `null` constraint, then it is assumed to be non-nullable. If it does, then it is assumed to be nullable. -Additionally, C# will introduce the `class?` constraint. This syntax sugar will compile into the `class` constraint, but the class itself will be decorated with an attribute that tells consumers that the `'T` is a nullable reference type. This attribute will need to be respected by the F# compiler. +Additionally, C# will introduce the `class?` constraint, which is sugar for an application of the `[]` attribute (see [Metadata](FS-1060-nullable-reference-types.md#net-metadata) for more). + +### A note on nullability and fundamental F# type representations + +Although the concept of nullability will surface as syntax and generate warnings, these annotations are ignored for all other purposes. For example: + +* Nullable annotations are ignored when deciding type equivalents, though warnings are emitted for mismatches. +* Nullable annotations are ignored when deciding type subsumption, though warnings are emitted for mismatches. +* Nullable annotations are ignored when deciding method overload resolution, though warnings are emitted for mismatches in argument and return types once an overload is committed. +* Nullable annotations are ignored for abstract slot inference, though warnings are emitted for mismatches. +* Nullable annotations are ignored when checking for duplicate methods. + +To re-iterate: nullable reference types are about separating distinguishing the implicit `null` from a reference type, but they are not a new _kind_ of reference type. They are still the same reference type and, despite warnings, can still compile when a nullability rule is violated. ### Type inference -Nullability is propagated through type inference: +Nullability should be propagated through type inference. Today, it will currently not be, so we will need to create a "nullness type variable" to represent the uncertainty, then unify it away as more information becomes available. These would be ignored for most purposes (just like nullability annotations) and would not be generalizable (i.e., committed to non-null in the absence of other information. + +Some examples: ```fsharp // Inferred signature: @@ -505,23 +531,31 @@ s <- null This includes function composition: ```fsharp +// At the time of writing this function: +// // val f1 : 'a -> 'a let f1 s = s +// At the time of writing this function: +// // val f2 : string -> (string | null) let f2 s = if s <> "" then "hello" else null +// At the time of writing this function: +// // val f3 : string -> string let f3 (s: string) = match s with - | null -> "" + | "" -> "Empty" | _ -> s -// val pipeline : string -> (string | null) -let pipeline = f1 >> f2 >> f3 // WARNING - flowing possible null as input to function that assumes a non-null input +// val pipeline : string -> string +let pipeline = f1 >> f2 >> f3 // Warning: nullness mismatch. The type ... did not match the type ... ``` -Existing nullability rules for F# types hold (i.e., you cannot suddenly assign `null` to an F# record). +Nullness like this will likely be checked through standard type inference, so warnings will come out of that. + +Existing nullability rules for F# types hold, and ad-hoc assignment of `null` to a type that does not have `null` as a proper value is not suddenly possible. ### Tooling considerations @@ -683,6 +717,16 @@ That said, it will become quite evident that this feature is necessary if F# is ### Syntax +### Overall approach to nullability + +* Design a fully sound (apart from runtime casts/reflection, akin to F# units of measure) and fully checked non-nullness system for F# (which gives errors rather than warnings), then interoperate with C# (giving warnings around the edges for interop). + +---------> Possibly, though there may be compatibility concerns. + +* Allow code to be generic w.r.t nullness. For example, so you can express, "if you give me a `null` in, I might give you a `null` back; if you give me a non-`null` in, I will give you a non-`null` back". + +---------> Questions about usability arise, and if we do something similar to null obliviousness. + #### Question mark nullability annotation ```fsharp From 7f4ff74f8fafc821539300ef2449b2e1286a96df Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Tue, 31 Jul 2018 09:39:14 -0700 Subject: [PATCH 31/46] Final feedback --- RFCs/FS-1060-nullable-reference-types.md | 62 +++++++++++++++++------- 1 file changed, 45 insertions(+), 17 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 4df16bc6..46ba1bd0 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -368,21 +368,17 @@ Although `doWork()` is called only in a scope where `str` would be non-null, thi #### Asserting non-nullability -To get around scenarios where compiler analysis cannot establish a non-null situation, a programmer can use `!` to assert non-nullability: +To get around scenarios where compiler analysis cannot establish a non-null situation, a programmer can use the `notNull` function to convert a nullable reference type to a non-nullable reference type: ```fsharp -let myIsNull item = isNull item - -let len (str: string | null) = - if myIsNull str then - -1 - else - str!.Length // OK: 'str' is asserted to be 'null' +let len (ns: string | null) = + let s = notNull(ns) // unsafe, but lets you avoid a warning + s.Length ``` -This will not generate a warning, but it is unsafe and can be the cause of a `NullReferenceException` if the reference was indeed `null` under some condition. +This function has a signature of `('T | null -> 'T)`. It will throw a `NullReferenceException` at runtime if the input is `null`. -This sort of feature is inherently unsafe, and is by definition not something being enthusiastically considered. But it may be a necessity if the compiler cannot analyze code well enough. +This sort of feature is inherently unsafe, and is by definition not something being enthusiastically considered. But it will be a necessity for when the compiler cannot analyze code well enough. #### Warnings @@ -731,8 +727,6 @@ That said, it will become quite evident that this feature is necessary if F# is ```fsharp let ns: string? = "hello" - -type C<'T?>() = class end ``` Issue: looks horrible with F# optional parameters @@ -742,7 +736,7 @@ type C() = member __.M(?x: string?) = "BLEGH" ``` -Issue: how to rationalize this with dynamic lookup +Issue: dynamic member lookup ```fsharp // This will look confusing @@ -755,7 +749,7 @@ Have a wrapper type for any reference type `'T` that could be null. Issues: -* Really ugly nesting is going to happen +* Really, really ugly nesting is going to happen * No syntax for this makes F# seen as "behind" C# for such a cornerstone feature of the .NET ecosystem #### Option or ValueOption @@ -769,13 +763,27 @@ Issues: ### Non-nullability Assertions +**`!` operator?** + +```fsharp +let myIsNull item = isNull item + +let len (str: string | null) = + if myIsNull str then + -1 + else + str!.Length // OK: 'str' is asserted to be 'null' +``` + +Specialized syntax for this sort of thing is not really the F# way. + **`!!` postfix operator?** Why two `!` when one could suffice? **`.get()` member?** -No idea if this could even work. +No idea if this could even work. Probably not. **Explicit cast required?** @@ -789,7 +797,7 @@ Something like this may be considered: notnull { expr } ``` -Where `notnull` returns the type of `expr`. But this may be viewed as too complicated when compared with a postfix operator, especially since a postfix operator is already a standard established by C#, Kotlin, TypeScript, and Swift. +Where `notnull` returns the type of `expr`. But this may be viewed as too complicated when compared with a function postfix operator, especially since a postfix operator is already a standard established by C#, Kotlin, TypeScript, and Swift. **Not have it?** @@ -919,6 +927,26 @@ Today, this produces a `NullReferenceException` at runtime. Either way, we'll need to respect the `[]` attribute generating a `null` to remain compatible with existing code. +Additionally, we'll want to consider the issue: [DefaultValue attribute should require 'mutable' when used in classes and records](https://github.com/fsharp/fslang-suggestions/issues/484), as the existing `[]` can be used with immutable values, meaning that checking is imperfect and the attribute is a source of unchecked `null` values. This design bug could be rectified, and we can warn on using it with an immutable value. + +#### CLIMutable attribute + +Today, `CLIMutable` is typically used to decorate F# record types so that the labels can be "filled in" by .NET libraries that cannot normally work with immutable data. One such set of libraries are ORMs, which require the mutation to be able to do change tracking. + +However, the following code is perfectly valid today and can produce `null`: + +```fsharp +[] +type R = { StringVal: string } +``` + +It is very likely that `StringVal` can be given a `null` value. But with this feature, the type of `StringVal` is a non-nullable string! Similar to `[]`, this once valid code is now nonsense. This means we'll have to either: + +* Do nothing and leave this slightly confusing signature as-is, warning at the call site when you attempt to dereference `StringVal`. +* Warn at the declaration site, saying that type annotation for `StringVal` needs to be a nullable reference type. + +We will continue to respect the fact that `[]` can result in `null` values. + #### Unchecked.defaultOf<'T> Today, `Unchecked.defaultof<'T>` will generate `null` when `'T` is a reference type. However, `'T` means non-null, and it's obvious that this will return `null` when parameterized with a .NET reference type such as `string`. We have some options: @@ -983,4 +1011,4 @@ public class C This means that `FSharpOption` is a nullable reference type if it were to be consumed by C# 8.0. Indeed, there is definitely C# code out there that checks for `null` when consuming an F# option type to account for the `None` case. This is because there is no other way to safely extract the underlying value in C#! -**Recommendation:** ROLL INTO A BALL AND CRY \ No newline at end of file +**Recommendation:** ROLL INTO A BALL AND CRY - ...and discuss further with the C# team to see if something can be done about this. \ No newline at end of file From e32fcf07be0047c6905d10d7c787b6802e472ec7 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Tue, 31 Jul 2018 09:47:07 -0700 Subject: [PATCH 32/46] Add project configurability --- RFCs/FS-1060-nullable-reference-types.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 46ba1bd0..07e48ce0 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -615,6 +615,25 @@ let s: string = null // OK, since we don't track nullability warnings By default, F# scripts that use a newer compiler via F# interactive will understand nullability. +#### Project configurability + +There are a few concerns here described further in the [Interaction Model](FS-1060-nullable-reference-types.md#interaction-model). When working in an IDE such as visual studio, I may wish to adopt this feature with any of the following configuration toggles: + +1. Turn off nullability globally +2. Turn off nullability on a per-project basis +3. Turn off nullability for a given assembly (e.g., a package reference) +4. Turn off warnings for checking of nullable references +5. Turn off warnings for checking of non-nullable references +6. Make checking of nullable references an error, separately from `warnaserror` +7. Make checking of non-nullable references an error, separately from `warnaserror` +8. Make all nullability warnings an error, separately from `warnaserror` + +This implies that there are toggles in the tooling to support this. + +* The project properties page for F# projects will require toggles for 2, 4, 5, 6, 7, and 8. +* There will need to be a right-click gesture that you can use to turn off nullability for a referenced assembly. +* Turning it off globally is not something that we can control, so (1) is TBD until we can arrive at a tooling solution for all languages. + ### Interaction model The following details how an F# compiler with nullable reference types will behave when interacting with different kinds of components. From 7f2674d7759e048d9e42976fa8b87252b359436b Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Tue, 31 Jul 2018 14:54:45 -0700 Subject: [PATCH 33/46] Feedback from Julien --- RFCs/FS-1060-nullable-reference-types.md | 40 +++++++++++++++++++----- 1 file changed, 32 insertions(+), 8 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 07e48ce0..bf1e691c 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -63,9 +63,9 @@ The following principles guide all further considerations and the design of this ### .NET Metadata -To remain backwards-compatible with existing codebases and interoperate with other .NET languages that support nullability as a concept, F# will emit and respect .NET metadata that concers to the following: +To remain backwards-compatible with existing codebases and interoperate with other .NET languages that support nullability as a concept, F# will emit and respect .NET metadata that concerns to the following: -* If a type is a nullable reference type +* If a type is a nullable reference type (**NOTE:** C# has no notion of this, so an assembly-level marker would be pending discussion) * If an assembly has nullability as a concept (regardless of what it was compiled with) * If a generic type constraint is nullable * If a given method/function "handles null", such as `String.IsNullOrWhiteSpace` @@ -167,7 +167,7 @@ F# will treat scopes with the `false` case as if its contained reference types a In other words, if we cannot be sure that a given reference type is non-nullable, we assume it is nullable. -As a final note, this attribute _can_ be abused. For example, if a method is annotated with `[NonNullTypes(true)]` +As a final note, this attribute _can_ be abused. For example, if a method is annotated with `[NonNullTypes(true)]` and does not actually perform a `null` check, further code can still produce a `NullReferenceException` and not have a warning associated with it. #### Null obliviousness @@ -183,7 +183,7 @@ This is slightly controversial, because we are making an assumption about code t To remain "true" to the original code, we could conceivably introduce null obliviousness and thus treat the value coming out of a component as either nullable or non-nullable, depending on how we assign it. -However, this is not actually possible in F#, because the majority of F# code uses type inference to infer a type. Null obliviousness would require explicit type annotations to work as we intend it! So we must assume nullability to remain safe. +However, this is not actually possible in F#, because the majority of F# code uses type inference to infer a type. Null obliviousness would require explicit type annotations to work as we intend it! Failing the propagation of a null-oblivous type through type inference (see [Type Inferece](FS-1060-nullabl-reference-types.md#type-inference)), we must assume nullability to remain safe. (**NOTE:** this is also under discussion for C# with `var x = ...`). Additionally, treating all components we cannot guarantee as non-nullable as nullable is, we feel, "in the spirit of F#", which mandates safety and non-nullness wherever possible. @@ -368,11 +368,11 @@ Although `doWork()` is called only in a scope where `str` would be non-null, thi #### Asserting non-nullability -To get around scenarios where compiler analysis cannot establish a non-null situation, a programmer can use the `notNull` function to convert a nullable reference type to a non-nullable reference type: +To get around scenarios where compiler analysis cannot establish a non-null situation, a programmer can use the `Unchecked.notNull` function to convert a nullable reference type to a non-nullable reference type: ```fsharp let len (ns: string | null) = - let s = notNull(ns) // unsafe, but lets you avoid a warning + let s = Unchecked.notNull(ns) // unsafe, but lets you avoid a warning s.Length ``` @@ -412,6 +412,10 @@ let s2: string = null // WARNING let f (s: string) = () f null // WARNING + +let g (s: string) = if s.Length > 0 then s else null + +let x: string = g "Hello" // WARNING ``` #### F# Collection initialization @@ -553,6 +557,26 @@ Nullness like this will likely be checked through standard type inference, so wa Existing nullability rules for F# types hold, and ad-hoc assignment of `null` to a type that does not have `null` as a proper value is not suddenly possible. +Note that asserting non-nullability may be required in some situations where types are inferred to be nullable. For example: + +```fsharp +let neverNull (str: string) = + match str with + | "" -> "" + | _ -> makeNullIfEmpty str // val makeNullIfEmpty: 'T -> ('T | null) +``` + +We know that this will never be `null` because the empty string is accounted for, but the compiler may well infer `neverNull` to return a `(string | null)`. To get around this, asserting non-nullability may be required: + +```fsharp +let neverNull (str: string) = + match str with + | "" -> "" + | _ -> + makeNullIfEmpty str + |> Unchecked.notNull +``` + ### Tooling considerations To remain in line our first principle: @@ -644,7 +668,7 @@ All non-F# assemblies built with a compiler that does not understand nullability Example: a `.dll` coming from a NuGet package built with C# 7.1 will be interpreted as if all reference types are nullable, including constraints, generics, etc. -This means that warnings will be emitted when `null` is not properly accounted for. Additionally, when older non-F# components are eventually recompiled with C# 8.0, it will likely be an API-breaking change that will force new F# code to go back and do away with any redundant `null` checks. +This means that warnings will be emitted when `null` is not properly accounted for. Additionally, when older non-F# components are eventually recompiled with C# 8.0, `null`-safe code may be doing redundant `null` checks. It may be worth calling that out as a warning. #### F# 5.0 consuming F# assemblies that do not have nullability @@ -652,7 +676,7 @@ All F# assemblies built with a previous F# compiler will be treated as such: * All reference types that are _not_ declared in F# are nullable. * All F#-declared reference types that have `null` as a proper value are treated as if they are nullable. -* All F#-declared reference types that do not have `null` as a proepr value are treated as non-nullable reference types. +* All F#-declared reference types that do not have `null` as a proper value are treated as non-nullable reference types. #### Ignoring F# 5.0 assemblies that do have nullability From d57f9161704aaadb557845615c7ac4bbae33ae30 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Wed, 1 Aug 2018 11:07:51 -0700 Subject: [PATCH 34/46] Add FSharp.Core considerations --- RFCs/FS-1060-nullable-reference-types.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index bf1e691c..b057c09a 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -577,6 +577,25 @@ let neverNull (str: string) = |> Unchecked.notNull ``` +### FSharp.Core + +Although we do not officially support forwards-compatibility, we do strive to ensure that older compilers can reference newer versions and "use the new features". At the same time, if something like `String.replicate` were to return a nullable string, that would be bad and antithetical to the spirit of F#. + +So, FSharp.Core will also need to be selective annotated, applying nullability to things only when we actually intend `null` values to be accepted or come out of a function. Specifically, we can: + +* Apply `[]` at the module level for the assembly +* Apply `[]` on every input type we wish to accept `null` values for +* Apply `[]` on every output type (this implies we may need to explicitly annotate a function) +* Apply `[]` and/or `[]` on public functions that test for `null` +* Apply `[]` if applicable to anything that may throw on `null` (if that exists) +* Apply `[]` and/or `[]` if applicable to anything that may assert + +This also means that some internal helper functions could be done away with. For example, [`String.emptyIfNull`](https://github.com/Microsoft/visualfsharp/blob/master/src/fsharp/FSharp.Core/string.fs#L16) is not publically consumable, and is effectively a way to enforce that incoming `string` types aren't `null`. This would make all `String.` functions now only accept non-nullable strings. + +This will be a nontrivial effort, not unlike efforts to selectively annotate CoreFX libraries. + +We need to take care that doing this does not affect binary compatibility in any way. It shouldn't, since the attributes being applied will just be ignored by an earlier compiler, but this still needs to be verified. + ### Tooling considerations To remain in line our first principle: From dcde16ed319c1c5b8224ab167dee4beb71207406 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Wed, 1 Aug 2018 13:19:35 -0700 Subject: [PATCH 35/46] Clarification --- RFCs/FS-1060-nullable-reference-types.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index b057c09a..02dd07da 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -557,7 +557,7 @@ Nullness like this will likely be checked through standard type inference, so wa Existing nullability rules for F# types hold, and ad-hoc assignment of `null` to a type that does not have `null` as a proper value is not suddenly possible. -Note that asserting non-nullability may be required in some situations where types are inferred to be nullable. For example: +Note that asserting non-nullability may be required in some situations where types are inferred to be nullable. Consider the following (confusing) function that does not do what its name says it does: ```fsharp let neverNull (str: string) = @@ -590,7 +590,7 @@ So, FSharp.Core will also need to be selective annotated, applying nullability t * Apply `[]` if applicable to anything that may throw on `null` (if that exists) * Apply `[]` and/or `[]` if applicable to anything that may assert -This also means that some internal helper functions could be done away with. For example, [`String.emptyIfNull`](https://github.com/Microsoft/visualfsharp/blob/master/src/fsharp/FSharp.Core/string.fs#L16) is not publically consumable, and is effectively a way to enforce that incoming `string` types aren't `null`. This would make all `String.` functions now only accept non-nullable strings. +This also means that some internal helper functions could be done away with. For example, [`String.emptyIfNull`](https://github.com/Microsoft/visualfsharp/blob/master/src/fsharp/FSharp.Core/string.fs#L16) is not publically consumable, and is effectively a way to enforce that incoming `string` types aren't `null`. This would make all `String.` functions now only accept non-nullable strings (should such a decision be made). This will be a nontrivial effort, not unlike efforts to selectively annotate CoreFX libraries. From 1bfa407dfc38445e0fa5e2fc4cd9ade94c2b6c6d Mon Sep 17 00:00:00 2001 From: cartermp Date: Tue, 7 Aug 2018 23:39:45 -0700 Subject: [PATCH 36/46] Quick changes based on C# meeting --- RFCs/FS-1060-nullable-reference-types.md | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 02dd07da..8b8a32e5 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -17,9 +17,9 @@ Conceptually, reference types can be thought of as having two forms: * Normal reference types * Nullable reference types -These will be distinguished via syntax, type signatures, and tools. Additionally, there will be flow analysis to allow you to determine when a nullable reference type is non-null and can be treated as a reference type. +These will be distinguished via syntax, type signatures, and tools. Additionally, there will be analysis run by the compiler to determine the `null`-safety of your code when it works with reference types. -Warnings are emitted when reference and nullable reference types are not used according with their intent, and these warnings are tunable as errors. +Warnings are emitted when reference types and nullable reference types are not used according with their intent, and these warnings are tunable as errors. This will be done in lockstep with C# 8.0, which will also have this as a feature, and F# will interoperate with C# by respecting and emitting the same metadata that C# emits. @@ -30,7 +30,7 @@ For the purposes of this document, the terms "reference type" and "non-nullable A dramatic shift is about to occur in the .NET ecosystem. Starting with C# 8.0, C# will distinguish between explicitly nullable reference types and reference types that cannot be `null`. That is, `string` will not be implicitly `null` in C# 8.0 or higher, and attempting to make it `null` will be a warning. -This problem with reference types in .NET - that they are _implicitly_ `null` - is a longstanding tension between the non-null nature of F# programming. Now that the most-used language in .NET will emit information indicating explicit nullability, this tension has the possibility of being eased considerably for F# programmers. Explicit representation of `null` is very much in line with general F# programming for two primary reasons: +This problem with reference types in .NET - that they are _implicitly_ `null` - is a longstanding tension between the non-null nature of F# programming and the .NET ecosystem. Now that the most-used language in .NET will emit information indicating explicit nullability, this tension has the possibility of being eased considerably for F# programmers. Explicit representation of `null` is very much in line with general F# programming for two primary reasons: 1. The explicit representation of critical information in types, with the compiler enforcing this representation, is very much the "F# way" of doing things. Although backwards-compatible explicit nullability is not sound in the same way that F# options are (read on to find out why), it is nonetheless in the spirit of how F# does things. 2. One of the goals of F# programmers is to evict `null` values as early as possible from the edges of their system. Although there are some scenarios where this cannot be done, and `null` values must flow through a program, these scenarios are usually few and far between, or avoided entirely. When nullability is explicit, it is harder to "forget" that an incoming type may carry a `null` value. This makes it easier to evict `null` values from the rest of an F# program. @@ -596,6 +596,18 @@ This will be a nontrivial effort, not unlike efforts to selectively annotate Cor We need to take care that doing this does not affect binary compatibility in any way. It shouldn't, since the attributes being applied will just be ignored by an earlier compiler, but this still needs to be verified. +### Language oddities + +#### Some null + +Today, you can write this perfectly valid F# expression, which is sure to elicit a few laughs: + +```fsharp +let x = Some null +``` + +This will be a warning now. + ### Tooling considerations To remain in line our first principle: @@ -687,7 +699,7 @@ All non-F# assemblies built with a compiler that does not understand nullability Example: a `.dll` coming from a NuGet package built with C# 7.1 will be interpreted as if all reference types are nullable, including constraints, generics, etc. -This means that warnings will be emitted when `null` is not properly accounted for. Additionally, when older non-F# components are eventually recompiled with C# 8.0, `null`-safe code may be doing redundant `null` checks. It may be worth calling that out as a warning. +This means that warnings will be emitted when `null` is not properly accounted for. #### F# 5.0 consuming F# assemblies that do not have nullability @@ -699,7 +711,9 @@ All F# assemblies built with a previous F# compiler will be treated as such: #### Ignoring F# 5.0 assemblies that do have nullability -Users may want to progressively work through `null` warnings by treating a given assembly as "nullability obliviousness". To respect this, F# 5.0 will have to ignore any potentially unsafe dereference of `null` until sucha time that "nullability obliviousness" is turned off for that assembly. +Users may want to progressively work through `null` warnings by treating a given assembly as "nullability obliviousness". To respect this, F# 5.0 will have to ignore any potentially unsafe dereference of `null` until such a time that "nullability obliviousness" is turned off for that assembly. + +Note: redundant `null` checks should not produce a warning. **Potential issue:** How are these types annotated in code? `string` or `string | null`, with the latter simply not triggering any warnings? From 1bd5fa5877fb502edfe71a475a7bdb6f4a3e4853 Mon Sep 17 00:00:00 2001 From: Phillip Carter Date: Mon, 13 Aug 2018 08:42:24 -0700 Subject: [PATCH 37/46] Feedback and intent-based warnings --- RFCs/FS-1060-nullable-reference-types.md | 52 ++++++++++++++++++++---- 1 file changed, 44 insertions(+), 8 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 02dd07da..5231efd9 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -30,7 +30,7 @@ For the purposes of this document, the terms "reference type" and "non-nullable A dramatic shift is about to occur in the .NET ecosystem. Starting with C# 8.0, C# will distinguish between explicitly nullable reference types and reference types that cannot be `null`. That is, `string` will not be implicitly `null` in C# 8.0 or higher, and attempting to make it `null` will be a warning. -This problem with reference types in .NET - that they are _implicitly_ `null` - is a longstanding tension between the non-null nature of F# programming. Now that the most-used language in .NET will emit information indicating explicit nullability, this tension has the possibility of being eased considerably for F# programmers. Explicit representation of `null` is very much in line with general F# programming for two primary reasons: +This problem with reference types in .NET - that they are _implicitly_ `null` - is a longstanding tension between the non-null nature of F# programming. Now that the most-used language in .NET will emit information indicating explicit nullability, this tension has the possibility of being eased considerably for F# programmers. Explicit representation of `null` is very much in line with general F# programming for four primary reasons: 1. The explicit representation of critical information in types, with the compiler enforcing this representation, is very much the "F# way" of doing things. Although backwards-compatible explicit nullability is not sound in the same way that F# options are (read on to find out why), it is nonetheless in the spirit of how F# does things. 2. One of the goals of F# programmers is to evict `null` values as early as possible from the edges of their system. Although there are some scenarios where this cannot be done, and `null` values must flow through a program, these scenarios are usually few and far between, or avoided entirely. When nullability is explicit, it is harder to "forget" that an incoming type may carry a `null` value. This makes it easier to evict `null` values from the rest of an F# program. @@ -126,9 +126,9 @@ F# will also emit this attribute when it produces a nullable reference type for ### Representing nullability as a concept -As descibed in the [Interaction Model](FS-1060-nullable-reference-types.md#interaction-model), there are certain behaviors involved in how to work with assemblies that may or may not express nullability. +As descibed in the [Interaction Model](FS-1060-nullable-reference-types.md#interaction-model), there are certain behaviors involved in how to work with assemblies that may or may not express nullability. The most flexible system for this is one in which a given scope is defined to have nullability as a concept. This is quite granular, but in practice will be at the assembly module level, making it mostly as if it were a per-project configuration. -C# 8.0 code will emit the `[NonUllTypes(true|false)]` attribute: +C# 8.0 code will emit the `[NonNullTypes(true|false)]` attribute: ```csharp namespace System.Runtime.CompilerServices @@ -173,26 +173,28 @@ As a final note, this attribute _can_ be abused. For example, if a method is ann Nullability obliviousness is a concept that C# 8.0 has. A null-oblivious type is one that no assumptions can be made about. Once assigned to a nullable or non-nullable variable, it is treated as if it is nullable or non-nullable, respectively. -**F# will not have this concept. Reference types we cannot determine to be non-nullable will be assumed to be nullable.** +**F# may not have this concept. Reference types we cannot determine to be non-nullable will be assumed to be nullable.** -This is slightly controversial, because we are making an assumption about code that we do not really know about: +This is controversial, because we are making an assumption about code that we do not really know about: * The code could not be handling `null` at all, and thus could either produce a `NullReferenceException` or produce a value that is nullable * The code could be handling `null` just fine, and could pass back a reference type that can never be `null` * We have no way to tell if any of the previous two points are true +In practice, this could mean emitting thousands of warnings for a single project. Although emitting warnings for unsafe null usage is a feature, if the number of warnings gets too high, people will find it invasive. + To remain "true" to the original code, we could conceivably introduce null obliviousness and thus treat the value coming out of a component as either nullable or non-nullable, depending on how we assign it. However, this is not actually possible in F#, because the majority of F# code uses type inference to infer a type. Null obliviousness would require explicit type annotations to work as we intend it! Failing the propagation of a null-oblivous type through type inference (see [Type Inferece](FS-1060-nullabl-reference-types.md#type-inference)), we must assume nullability to remain safe. (**NOTE:** this is also under discussion for C# with `var x = ...`). -Additionally, treating all components we cannot guarantee as non-nullable as nullable is, we feel, "in the spirit of F#", which mandates safety and non-nullness wherever possible. +Additionally, treating all components we cannot guarantee as non-nullable as nullable is, we feel, "in the spirit of F#", which mandates safety and non-nullness wherever possible. This point may well be revisited if early usage indicates that lack of null obliviousness is too invasive. #### Marking methods as handling null There will be several attributes C# 8.0 code can emit: * `[NotNullWhenTrue]` - Indicates a method handles `null` if the result is `true`, e.g., a `TryGetValue` call. -* `[NotNullWhenFalse]` - Indicates a method handles `null` fi the result is `false`, e.g., a `string.IsNullOrEmpty` call. +* `[NotNullWhenFalse]` - Indicates a method handles `null` if the result is `false`, e.g., a `string.IsNullOrEmpty` call. * `[EnsuresNotNull]` - Indicates that the program cannot continue if a value is `null`, for example, a `ThrowIfNull` call. * `[AssertsTrue]` and `[AssertsFalse]` - Used in assertion cases where `null` is concerned. @@ -295,7 +297,7 @@ More generally, `value = null` or `value <> null`, where `value` is an F# expres ```fsharp let len (str: string | null) = - if (str <> null && str.Length > 0) then // OK - `str` must be null + if (str <> null && str.Length > 0) then // OK - `str` must be non-null str.Length // OK here too else -1 @@ -596,6 +598,40 @@ This will be a nontrivial effort, not unlike efforts to selectively annotate Cor We need to take care that doing this does not affect binary compatibility in any way. It shouldn't, since the attributes being applied will just be ignored by an earlier compiler, but this still needs to be verified. +### Intent-based warnings + +The `[]` and `[]` attributes can be used to great utility today, but can also result in declared code that is nonsensical with this feature: + +```fsharp +// Nonsense one: +type C() = + [] + val mutable Whoops : string + +printfn "%d" (C().Whoops.Length) + +// Nonsense two: +[] +type R = { Whoopsie: string } +``` + +Both of these declarations are saying conflicting things. The attribute implies nullability, because the default value for a reference type is `null` (at least prior to F# 5.0). However, they are declared with type `string`, which is non-null with `[]`! This is a contradiction. + +Because there is no way to interpret those declarations correctly, a warning is emitted. This is an **Intent-based warning**. Warning strings that follow are an example, not necessarily what will be emitted: + +```fsharp +type C() = + [] // WARNING - DefaultValue implies nullability, but 'Whoops' is declared as non-null. Consider a null annotation. + val mutable Whoops : string + +printfn "%d" (C().Whoops.Length) // WARNING - possible null dereference + +[] // WARNING - CLIMutable implies nullability, but 'Whoopsie' is declared as non-null. Consider a null annotation. +type R = { Whoopsie: string } +``` + +As a note, the previous example demonstrates that despite having contradicting declared types, usage of the declared types can still emit warnings. Intent-based warnings should not keep usage of these types from producing other nullability warnings. + ### Tooling considerations To remain in line our first principle: From aaad0debbc981d0c2a177c885b8fd1c0eec2c5b4 Mon Sep 17 00:00:00 2001 From: Don Syme Date: Fri, 24 Aug 2018 14:34:51 +0100 Subject: [PATCH 38/46] Update FS-1060-nullable-reference-types.md --- RFCs/FS-1060-nullable-reference-types.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 9b1834a9..45e04c41 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -30,11 +30,7 @@ For the purposes of this document, the terms "reference type" and "non-nullable A dramatic shift is about to occur in the .NET ecosystem. Starting with C# 8.0, C# will distinguish between explicitly nullable reference types and reference types that cannot be `null`. That is, `string` will not be implicitly `null` in C# 8.0 or higher, and attempting to make it `null` will be a warning. -<<<<<<< HEAD -This problem with reference types in .NET - that they are _implicitly_ `null` - is a longstanding tension between the non-null nature of F# programming. Now that the most-used language in .NET will emit information indicating explicit nullability, this tension has the possibility of being eased considerably for F# programmers. Explicit representation of `null` is very much in line with general F# programming for four primary reasons: -======= -This problem with reference types in .NET - that they are _implicitly_ `null` - is a longstanding tension between the non-null nature of F# programming and the .NET ecosystem. Now that the most-used language in .NET will emit information indicating explicit nullability, this tension has the possibility of being eased considerably for F# programmers. Explicit representation of `null` is very much in line with general F# programming for two primary reasons: ->>>>>>> 1bfa407dfc38445e0fa5e2fc4cd9ade94c2b6c6d +This problem with reference types in .NET - that they are _implicitly_ `null` - is a longstanding tension between the non-null nature of F# programming and the .NET ecosystem. Now that the most-used language in .NET will emit information indicating explicit nullability, this tension has the possibility of being eased considerably for F# programmers. Explicit representation of `null` is very much in line with general F# programming for four primary reasons: 1. The explicit representation of critical information in types, with the compiler enforcing this representation, is very much the "F# way" of doing things. Although backwards-compatible explicit nullability is not sound in the same way that F# options are (read on to find out why), it is nonetheless in the spirit of how F# does things. 2. One of the goals of F# programmers is to evict `null` values as early as possible from the edges of their system. Although there are some scenarios where this cannot be done, and `null` values must flow through a program, these scenarios are usually few and far between, or avoided entirely. When nullability is explicit, it is harder to "forget" that an incoming type may carry a `null` value. This makes it easier to evict `null` values from the rest of an F# program. @@ -1127,4 +1123,4 @@ public class C This means that `FSharpOption` is a nullable reference type if it were to be consumed by C# 8.0. Indeed, there is definitely C# code out there that checks for `null` when consuming an F# option type to account for the `None` case. This is because there is no other way to safely extract the underlying value in C#! -**Recommendation:** ROLL INTO A BALL AND CRY - ...and discuss further with the C# team to see if something can be done about this. \ No newline at end of file +**Recommendation:** ROLL INTO A BALL AND CRY - ...and discuss further with the C# team to see if something can be done about this. From 36b623459434f156d465a79ee0d6d647a8c46739 Mon Sep 17 00:00:00 2001 From: cartermp Date: Fri, 24 Aug 2018 08:44:26 -0700 Subject: [PATCH 39/46] Missed the fi --- RFCs/FS-1060-nullable-reference-types.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 8b8a32e5..ca97da17 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -192,7 +192,7 @@ Additionally, treating all components we cannot guarantee as non-nullable as nul There will be several attributes C# 8.0 code can emit: * `[NotNullWhenTrue]` - Indicates a method handles `null` if the result is `true`, e.g., a `TryGetValue` call. -* `[NotNullWhenFalse]` - Indicates a method handles `null` fi the result is `false`, e.g., a `string.IsNullOrEmpty` call. +* `[NotNullWhenFalse]` - Indicates a method handles `null` if the result is `false`, e.g., a `string.IsNullOrEmpty` call. * `[EnsuresNotNull]` - Indicates that the program cannot continue if a value is `null`, for example, a `ThrowIfNull` call. * `[AssertsTrue]` and `[AssertsFalse]` - Used in assertion cases where `null` is concerned. From fe767ce274984cba4ac5390012172d0fe4face8b Mon Sep 17 00:00:00 2001 From: cartermp Date: Fri, 24 Aug 2018 17:30:23 -0700 Subject: [PATCH 40/46] Clarify nullability obliviousness --- RFCs/FS-1060-nullable-reference-types.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 45e04c41..000ae45f 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -96,6 +96,8 @@ namespace System.Runtime.CompilerServices } ``` +Note: the `bool[]` is used to represent nested types. So `ResizeArray` would be represented with `[| false; true |]`, where `ResizeArray` is non-null, but `string | null` is `null`. + For example, the following code: ```csharp @@ -126,7 +128,7 @@ F# will also emit this attribute when it produces a nullable reference type for ### Representing nullability as a concept -As descibed in the [Interaction Model](FS-1060-nullable-reference-types.md#interaction-model), there are certain behaviors involved in how to work with assemblies that may or may not express nullability. The most flexible system for this is one in which a given scope is defined to have nullability as a concept. This is quite granular, but in practice will be at the assembly module level, making it mostly as if it were a per-project configuration. +As described in the [Interaction Model](FS-1060-nullable-reference-types.md#interaction-model), there are certain behaviors involved in how to work with assemblies that may or may not express nullability. The most flexible system for this is one in which a given scope is defined to have nullability as a concept. This is quite granular, but in practice will be at the assembly module level, making it mostly as if it were a per-project configuration. C# 8.0 code will emit the `[NonNullTypes(true|false)]` attribute: @@ -173,19 +175,15 @@ As a final note, this attribute _can_ be abused. For example, if a method is ann Nullability obliviousness is a concept that C# 8.0 has. A null-oblivious type is one that no assumptions can be made about. Once assigned to a nullable or non-nullable variable, it is treated as if it is nullable or non-nullable, respectively. -**F# may not have this concept. Reference types we cannot determine to be non-nullable will be assumed to be nullable.** - -This is controversial, because we are making an assumption about code that we do not really know about: +For example, imagine a C# 8.0 project consuming an older assembly that was compiled with an older C# compiler that has no notion of nullability, or it was compiled such that reference types are defined in a scope where `NonNullTypes(false)`. The C# behavior in this circumstance is, "we cannot say whether or not this is nullable or not, so we cannot assign nullability or non-nullability". Once assigned to a variable, the type it is assigned to dictates if it is nullable or not. -* The code could not be handling `null` at all, and thus could either produce a `NullReferenceException` or produce a value that is nullable -* The code could be handling `null` just fine, and could pass back a reference type that can never be `null` -* We have no way to tell if any of the previous two points are true +**F# may not have this concept due to type inference.** -In practice, this could mean emitting thousands of warnings for a single project. Although emitting warnings for unsafe null usage is a feature, if the number of warnings gets too high, people will find it invasive. +This is controversial, because we are making an assumption about code that we do not really know about because the code could be handling `null` just fine, and could pass back a reference type that can never be `null`, but we'd still force the programmer to account for `null`. -To remain "true" to the original code, we could conceivably introduce null obliviousness and thus treat the value coming out of a component as either nullable or non-nullable, depending on how we assign it. +This could mean emitting thousands of warnings for a single project. Although emitting warnings for unsafe null usage is a feature, if the number of warnings gets too high, people will find it invasive. -However, this is not actually possible in F#, because the majority of F# code uses type inference to infer a type. Null obliviousness would require explicit type annotations to work as we intend it! Failing the propagation of a null-oblivous type through type inference (see [Type Inferece](FS-1060-nullabl-reference-types.md#type-inference)), we must assume nullability to remain safe. (**NOTE:** this is also under discussion for C# with `var x = ...`). +To remain "true" to the original code, we could introduce null obliviousness as a "state" that a reference type could be in. However, this is not practically possible in F#, because the majority of F# code uses type inference to infer a type, and null obliviousness would require explicit type annotations to work properly! Additionally, treating all components we cannot guarantee as non-nullable as nullable is, we feel, "in the spirit of F#", which mandates safety and non-nullness wherever possible. This point may well be revisited if early usage indicates that lack of null obliviousness is too invasive. From 188fb092ca58d02583750a0ad71adc910839428e Mon Sep 17 00:00:00 2001 From: cartermp Date: Tue, 28 Aug 2018 12:19:37 -0700 Subject: [PATCH 41/46] Add synthesizing of nonnulltypes --- RFCs/FS-1060-nullable-reference-types.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 000ae45f..ac3afab5 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -159,15 +159,15 @@ From the C# spec on this: > > `[NonNullTypes(true|false)]` is not synthesized by the compiler. If the attribute is used explicitly in source, the type declaration must be provided explicitly to the compilation. -In other words, this attribute specifies a way to mark a class, constructor, delegate, enum, event, field, interface, method, module, property, or struct as having nullability expressable or not in a containing scope. +In other words, this attribute specifies a way to mark a class, constructor, delegate, enum, event, field, interface, method, module, property, or struct as having nullability expressible or not in a containing scope. This attribute could potentially be used in F# to allow for opt-in/opt-out nullability at a fine-grained level. -F# will respect this attribute, with the `true` caseindicating that the scope distinguishes between nullable and non-nullable reference types. +F# will respect this attribute, with the `true` case indicating that the scope distinguishes between nullable and non-nullable reference types. If `false` is used, then nullability is not a concept and F# behaves in accordance with [Null obliviousness](FS-1060-nullable-reference-types.md#null-obliviousness). -F# will treat scopes with the `false` case as if its contained reference types are all nullable reference types. +Additionally, the F# compiler will likely have to synthesize/embed this attribute upon usage. See a note from the C# team: -In other words, if we cannot be sure that a given reference type is non-nullable, we assume it is nullable. +> FYI, we’re currently working to have the C# compiler embed this attribute whenever it is used in source. As a result, we do not plan to include that attribute into frameworks. That means the F# compiler will likely have to synthesize/embed this attribute upon usage as well. As a final note, this attribute _can_ be abused. For example, if a method is annotated with `[NonNullTypes(true)]` and does not actually perform a `null` check, further code can still produce a `NullReferenceException` and not have a warning associated with it. From 2e2091b88bda4498cdb8dd6db8d8f8e59814d464 Mon Sep 17 00:00:00 2001 From: cartermp Date: Tue, 28 Aug 2018 16:40:20 -0700 Subject: [PATCH 42/46] Remove mistake on NonNullTypes(false) --- RFCs/FS-1060-nullable-reference-types.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index ac3afab5..72b967b1 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -163,7 +163,7 @@ In other words, this attribute specifies a way to mark a class, constructor, del This attribute could potentially be used in F# to allow for opt-in/opt-out nullability at a fine-grained level. -F# will respect this attribute, with the `true` case indicating that the scope distinguishes between nullable and non-nullable reference types. If `false` is used, then nullability is not a concept and F# behaves in accordance with [Null obliviousness](FS-1060-nullable-reference-types.md#null-obliviousness). +F# will respect this attribute, with the `true` case indicating that the scope distinguishes between nullable and non-nullable reference types. If `false` is used, then nullability is not a concept and F# treats the reference types exactly as previous versions of the language would (i.e., not complain on unsafe dereference). Additionally, the F# compiler will likely have to synthesize/embed this attribute upon usage. See a note from the C# team: From 216915a56af4104d57fbd8512acea15bccfd5db2 Mon Sep 17 00:00:00 2001 From: Don Syme Date: Tue, 16 Oct 2018 17:00:32 +0100 Subject: [PATCH 43/46] Update FS-1060-nullable-reference-types.md --- RFCs/FS-1060-nullable-reference-types.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 72b967b1..656bdf76 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -1122,3 +1122,23 @@ public class C This means that `FSharpOption` is a nullable reference type if it were to be consumed by C# 8.0. Indeed, there is definitely C# code out there that checks for `null` when consuming an F# option type to account for the `None` case. This is because there is no other way to safely extract the underlying value in C#! **Recommendation:** ROLL INTO A BALL AND CRY - ...and discuss further with the C# team to see if something can be done about this. + + + +### Meaning of `unbox` + +Here's something tricky. For an F# type C consider +1. `unbox(s)` +2. `unbox(s)` +3. `unbox(s)` +4. `unbox(s)` + +There are two unbox helpers - [`UnboxGeneric`](https://github.com/Microsoft/visualfsharp/blob/92247b886e4c3f8e637948de84b6d10f97b2b894/src/fsharp/FSharp.Core/prim-types.fs#L613) and [`UnboxFast`](https://github.com/Microsoft/visualfsharp/blob/92247b886e4c3f8e637948de84b6d10f97b2b894/src/fsharp/FSharp.Core/prim-types.fs#L620). Normally `unbox(s)` just gets inlined to become `UnboxGeneric`, e.g. this is what (1) becomes. However the F# optimizer converts calls to `UnboxFast` for (2), see [here](https://github.com/Microsoft/visualfsharp/blob/99c667b0ee24f18775d4250a909ee5fdb58e2fae/src/fsharp/Optimizer.fs#L2576) + +`UnboxGeneric` guards against the input being `null` and refuses to unbox a `null` value to `C` for example. It uses a runtime lookup on typeof to do this + +That is, the meaning of `unbox` in F# depends on whether the type carries a null value or not. If the target type *doesn't* carry null and is not a value type (i.e. is an F# reference type), the *slower* helper is used to do an early guard against a null value leaking in. If the target type *does* carry null, the *fast* helper is used (since null can leak through ok). For generic code the slow helper is used + +The problem of course is that once (3) is allowed we have an issue - using the slow helper is now no longer correct and will raise an exception, i.e. `unbox(null)` will raise an exception. + +I'll add this to the notes in the RFC From 54b6f90f0faf346cde0b983319213e5cec72f075 Mon Sep 17 00:00:00 2001 From: Don Syme Date: Tue, 16 Oct 2018 17:14:00 +0100 Subject: [PATCH 44/46] Update FS-1060-nullable-reference-types.md --- RFCs/FS-1060-nullable-reference-types.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 656bdf76..12192876 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -1141,4 +1141,16 @@ That is, the meaning of `unbox` in F# depends on whether the type carries a null The problem of course is that once (3) is allowed we have an issue - using the slow helper is now no longer correct and will raise an exception, i.e. `unbox(null)` will raise an exception. -I'll add this to the notes in the RFC +### Interaction with `UseNullAsTrueValue` + +F# option types use the obscure attribute `UseNullAsTrueValue` to ensure that `None` gets compiled as `null`. This was a design choice made early in F# and has been problematic for many reasons, e.g. `None.ToString()` raises an exception, and baroque special compilation rules are needed for `opt.HasValue`. However, this choice has been made and we must live with it. + +In theory the `UseNullAsTrueValue` attribute can be used with other F#' option types subject to limitations, e.g. this error message: +``` +1196,tcInvalidUseNullAsTrueValue,"The 'UseNullAsTrueValue' attribute flag may only be used with union types that have one nullary case and at least one non-nullary case" +``` + +IMPORTANT: Types that use `UseNullAsTrueValue` may *not* be made nullable, so `option | null` is **not** allowed. + +Additionally the semantics of type tests are adjusted slightly to account for the possibility that `null` is a legitimate value, e.g. so that `match None with :? int option -> true | _ -> false` returns `true`. Here `None` is represented as `null`. This is done through helpers `TypeTestGeneric` and `TypeTestFast`. We should consider whether this needs documenting or adjusting in the same way as `UnboxGeneric` and `UnboxFast`, though on first glance I don't believe it does. + From de0d0fe9099167e31b8eaded1bf37b334ec1df49 Mon Sep 17 00:00:00 2001 From: Don Syme Date: Tue, 16 Oct 2018 18:06:09 +0100 Subject: [PATCH 45/46] Update FS-1060-nullable-reference-types.md --- RFCs/FS-1060-nullable-reference-types.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 12192876..9674890a 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -1154,3 +1154,15 @@ IMPORTANT: Types that use `UseNullAsTrueValue` may *not* be made nullable, so `o Additionally the semantics of type tests are adjusted slightly to account for the possibility that `null` is a legitimate value, e.g. so that `match None with :? int option -> true | _ -> false` returns `true`. Here `None` is represented as `null`. This is done through helpers `TypeTestGeneric` and `TypeTestFast`. We should consider whether this needs documenting or adjusting in the same way as `UnboxGeneric` and `UnboxFast`, though on first glance I don't believe it does. + +### Interaction with default values + +F# does some analysis related to whether types have default values or not. This is used to +1. determine if a field can be left uninitialized after being annotated with `DefaultValue` +2. determine if a type containing a field of this type itself has a default value +3. determine if the default struct constructor can be called, e.g. `DateTime()`. For F#-defined types this can only be called if + the struct type has a default value +4. For struct types, to determine if the `'T when 'T : (new : unit -> 'T)` is satisfied. + +Historically .NET reference types have always been considered to have default values. We should consider whether `string` is now considered to have a default value, or more dpecifically whether we should report optional nullability warnings when there is a difference between the cases of old and new-nullability rules. + From f131963eee29e380b6eadddfaed67047c819c7d6 Mon Sep 17 00:00:00 2001 From: Don Syme Date: Fri, 19 Oct 2018 16:28:00 +0100 Subject: [PATCH 46/46] Update FS-1060-nullable-reference-types.md --- RFCs/FS-1060-nullable-reference-types.md | 185 +++++++++++++---------- 1 file changed, 106 insertions(+), 79 deletions(-) diff --git a/RFCs/FS-1060-nullable-reference-types.md b/RFCs/FS-1060-nullable-reference-types.md index 9674890a..f0227203 100644 --- a/RFCs/FS-1060-nullable-reference-types.md +++ b/RFCs/FS-1060-nullable-reference-types.md @@ -10,7 +10,9 @@ The design suggestion [Add non-nullable instantiations of nullable types, and in ## Summary [summary]: #summary -The main goal of this feature is to address the pains of dealing with implicitly nullable reference types by providing syntax and behavior that distinguishes between nullable and non-nullable reference types. +The main goal of this feature is to interoperate with the future .NET ecosystem where reference types will be non-nullable by default. + +This will be done by providing syntax and behavior that distinguishes between nullable and non-nullable reference types. Conceptually, reference types can be thought of as having two forms: @@ -23,39 +25,69 @@ Warnings are emitted when reference types and nullable reference types are not u This will be done in lockstep with C# 8.0, which will also have this as a feature, and F# will interoperate with C# by respecting and emitting the same metadata that C# emits. -For the purposes of this document, the terms "reference type" and "non-nullable reference type" may be used interchangeably. They refer to the same thing. +For the purposes of this document, the terms "reference type" and "non-nullable reference type" may be used interchangeably. They refer to the same thing. We also use the terminology "a type does not have null as a normal value", which is the terminology for non-nullability used in the existing F# language specification. ## Motivation [motivation]: #motivation -A dramatic shift is about to occur in the .NET ecosystem. Starting with C# 8.0, C# will distinguish between explicitly nullable reference types and reference types that cannot be `null`. That is, `string` will not be implicitly `null` in C# 8.0 or higher, and attempting to make it `null` will be a warning. +A shift is about to occur in the .NET ecosystem. Starting with C# 8.0, C# will distinguish between explicitly nullable reference types and reference types that do not have `null` as a normal value. That is, `string` will not have `null` as a normal value in C# 8.0 or higher, and attempting to make it `null` will be a warning. -This problem with reference types in .NET - that they are _implicitly_ `null` - is a longstanding tension between the non-null nature of F# programming and the .NET ecosystem. Now that the most-used language in .NET will emit information indicating explicit nullability, this tension has the possibility of being eased considerably for F# programmers. Explicit representation of `null` is very much in line with general F# programming for four primary reasons: +F# has always taken the approach that F#-defined types do not have null as a normal value. This problem with reference types in .NET - that they are _implicitly_ `null` - is a longstanding tension between the non-null nature of F# programming and the .NET ecosystem. Now that the most-used language in .NET will emit information indicating explicit nullability, this tension has the possibility of being eased considerably for F# programmers. Explicit representation of `null` is very much in line with general F# programming for four primary reasons: 1. The explicit representation of critical information in types, with the compiler enforcing this representation, is very much the "F# way" of doing things. Although backwards-compatible explicit nullability is not sound in the same way that F# options are (read on to find out why), it is nonetheless in the spirit of how F# does things. + 2. One of the goals of F# programmers is to evict `null` values as early as possible from the edges of their system. Although there are some scenarios where this cannot be done, and `null` values must flow through a program, these scenarios are usually few and far between, or avoided entirely. When nullability is explicit, it is harder to "forget" that an incoming type may carry a `null` value. This makes it easier to evict `null` values from the rest of an F# program. + 3. Another goal of F# programmers is to ensure that they do not produce `null` values unless it is absolutely necessary in the context in which they are working. When nullability as a concept is explicit, an F# programmer must do more work to produce a `null` value. When it is an annoyance to produce `null` values (especially of the implicit variety), then programmers will try to avoid producing them. + 4. A variant of (3) is that it is possible to accidentaly produce a `null` value in F# code. With explicit nullability as a concept, this accidental behavior can be recognized by a compiler and a warning can be given to let the F# programmer know that they may not have intended sending `null` somewhere. -F# has some existing mitigations against `null`, including that F#-declared reference types cannot be `null` unless explicitly decorated with `[]`. This is useful for when an F# programmers absolutely must flow `null` into a component. However, this action "infects" any other place where the type might be used, forcing those places to also account for `null`. This is rarely what F# programmers desire. +The desired outcome over time is that significantly less `NullReferenceException`s are produced by code at runtime when interoperating with C# libraries and other F# components that declare reference types as capable of having `null` as a proper value. This feature should allow F# programmers to more safely eject `null` from their concerns when interoperating with other .NET components, and make it more explicit when `null` could be a problem. + +### Existing Mitigations against `null` in F# code + +F# has many existing mitigations against `null`. Indeed, much of the design of F# is based on the principle that it should be unnecessary to explicitly use `null` values at all in normal F# code, and `null` values are absent from ML dialects such as OCaml. + +Some existing mitigations include: + +1. F#-declared reference types do not support `null` as a normal value + +2. `let` and `let rec` bindings are always initialized prior to use. This includes `let` bindings in classes, where checks are made to ensure that the uninitialized `this` is not leaked, e.g. via virtual dispatch to members in subclasses. + +3. F# does allow F#-declared reference types to be decorated with `[]`. (This attribute is useful for when an F# programmers frequently uses `null` for a particular type. However, this action "infects" any other place where the type might be used, forcing those places to also account for `null`.) + +4. In F# code unboxing protects against null values. That is, the meaning of `unbox` in F# depends on whether the type has null as a normal value or not. For example, `unbox(null)` raises an exception, where `unbox(null)` does not. The same applies for type casts. This prevents unboxing being a backdoor route for creating `null` -The desired outcome over time is that significantly less `NullReferenceException`s are produced by code at runtime when itneroperating with C# libraries and other F# components that declare reference types as capable of having `null` as a proper value. This feature should allow F# programmers to more safely eject `null` from their concerns when interoperating with other .NET components, and make it more explicit when `null` could be a problem. +5. F# does some analysis related to whether types have default values or not, covered below. + +6. The `null` literal may only be used with types that support it, ## Principles The following principles guide all further considerations and the design of this feature. 1. On the surface, F# is "almost" as simple to use as it is today. In practice, it must feel simpler due to nullability of types like `string` being explicit rather than implicit. + 2. The value for F# is primarily in flowing non-nullable reference types into F# code from .NET Libraries and from F# code into .NET libraries. + 3. Adding nullable annotations should not be part of routine F# programming. + 4. Nullability annotations/information is carefully suppressed and simplified in tooling (tooltips, FSI) to avoid extra information overload. + 5. F# users are typically concerned with this feature at the boundaries of their system so that they can flow non-null data into the inner parts of their system. + 6. The feature produces warnings by default, with different classes of warnings (nullable, non-nullable) offering opt-in/opt-out mechanisms. + 7. All existing F# projects compile with warnings turned off by default. Only new projects have warnings on by default. Compiling older F# code with newer F# code does not opt-in the older F# code. + 8. F# non-nullness is reasonably "strong and trustworthy" today for F#-defined types and routine F# coding. This includes the explicit use of `option` types to represent the absence of information. No compile-time guarantees today will be "lowered" to account for this feature. + 9. The F# compiler will strive to provide flow analysis such that common null-check scenarios guarantee null-safety when working with nullable reference types. + 10. This feature is useful for F# in other contexts, such as interoperation with `string` in JavaScript via [Fable](http://fable.io/). + 11. F# programmers may be able to start to phase out their use of `[]`, `Unchecked.defaultof<_>`, and the `null` type constraint when they need ad-hoc nullability for reference types declared in F#. + 12. Syntax for the feature feels "baked in" to the type system, and should not be horribly unpleasant to use. ## Detailed design @@ -66,8 +98,11 @@ The following principles guide all further considerations and the design of this To remain backwards-compatible with existing codebases and interoperate with other .NET languages that support nullability as a concept, F# will emit and respect .NET metadata that concerns to the following: * If a type is a nullable reference type (**NOTE:** C# has no notion of this, so an assembly-level marker would be pending discussion) + * If an assembly has nullability as a concept (regardless of what it was compiled with) + * If a generic type constraint is nullable + * If a given method/function "handles null", such as `String.IsNullOrWhiteSpace` C# 8.0 will emit and respect metadata for these scenarios, with a well-known set of names for each attribute. These attributes will also be what F# uses, though the behavior of F# in the face of these attributes is not necessarily identical to how C# 8.0 behaves in the face of these attributes. @@ -96,7 +131,7 @@ namespace System.Runtime.CompilerServices } ``` -Note: the `bool[]` is used to represent nested types. So `ResizeArray` would be represented with `[| false; true |]`, where `ResizeArray` is non-null, but `string | null` is `null`. +Note: the `bool[]` is used to represent nested types. So `ResizeArray` would be represented with `[| false; true |]`, where `ResizeArray` is non-null, but `string?` is `null`. For example, the following code: @@ -177,8 +212,6 @@ Nullability obliviousness is a concept that C# 8.0 has. A null-oblivious type is For example, imagine a C# 8.0 project consuming an older assembly that was compiled with an older C# compiler that has no notion of nullability, or it was compiled such that reference types are defined in a scope where `NonNullTypes(false)`. The C# behavior in this circumstance is, "we cannot say whether or not this is nullable or not, so we cannot assign nullability or non-nullability". Once assigned to a variable, the type it is assigned to dictates if it is nullable or not. -**F# may not have this concept due to type inference.** - This is controversial, because we are making an assumption about code that we do not really know about because the code could be handling `null` just fine, and could pass back a reference type that can never be `null`, but we'd still force the programmer to account for `null`. This could mean emitting thousands of warnings for a single project. Although emitting warnings for unsafe null usage is a feature, if the number of warnings gets too high, people will find it invasive. @@ -208,48 +241,44 @@ Nullable reference types can conceptually be thought of a union between a refere reference-type | null ``` +For technical reasons we may use this syntax instead: + +``` +reference-type? +``` + What this syntax means is that `reference-type` is non-null. Some examples in F# code: ```fsharp // Parameter to a function -let len (str: string | null) = +let len (str: string?) = match str with | null -> -1 | s -> s.Length -// Return type -let findOrNull (index: int) (list: 'T list) : 'T | null = - match List.tryItem index list with - | Some item -> item - | None -> null - // Declared type at let-binding -let maybeAValue : string | null = hopefullyGetAString() +let maybeAValue : string? = hopefullyGetAString() // Array type signature -let f (arr: (string | null) []) = () +let f (arr: string?[]) = () + +// Generic code, note 'T must be constrained to be a reference type +let findOrNull (index: int) (list: 'T list) : 'T? when 'T : not struct = + match List.tryItem index list with + | Some item -> item + | None -> null ``` The syntax accomplishes two goals: 1. Makes the distinction that a nullable reference type can be `null` clear in syntax. + 2. Becomes a bit unwieldy when nullability is used a lot, particularly with nested types. Because it is a design goal to support the use of non-null reference types by default, we want to encourage the programmer to **avoid** using nullable reference types unless absolutely necessary. -#### No Nullable reference type declarations in F\# - -There will be no way to declare an F# type as follows: - -```fsharp -// This will not be possible! -type (SomeClass | null)() = class end -``` - -And in existing F# today, the only way to declare an F# reference type that could be `null` in F# code is to use `[]`. This would not change with this proposal; that is, the only way to declare a reference type in F# that could have `null` as a value in F# code is `[]`. - ### Checking of nullable references There are a few common ways that F# programmers check for null today. Unless explicitly mentioned as unsupported, support for flow analysis is under consideration for certain null-checking patterns. @@ -257,18 +286,18 @@ There are a few common ways that F# programmers check for null today. Unless exp #### Likely supported: Pattern matching with alias ```fsharp -let len (str: string | null) = +let len (str: string?) = match str with | null -> -1 | s -> s.Length // OK - we know 's' is string ``` -This is by far the most common pattern in F# programming. In the previous code sample, `str` is of type `string | null`, but `s` is now of type `string`. This is because we know that based on the `null` has been accounted for by the `null` pattern. +This is by far the most common pattern in F# programming. In the previous code sample, `str` is of type `string?`, but `s` is now of type `string`. This is because we know that based on the `null` has been accounted for by the `null` pattern. -#### Likely supported: isNull and others using null-handling decoration +#### Possibly supported: isNull and others using null-handling decoration ```fsharp -let len (str: string | null) = +let len (str: string?) = if isNull str then -1 else @@ -282,7 +311,7 @@ Similarly, CoreFX will annotate methods like `String.IsNullOrEmpty` as handling #### Possibly supported: If check ```fsharp -let len (str: string | null) = +let len (str: string?) = if str = null then -1 else @@ -294,7 +323,7 @@ More generally, `value = null` or `value <> null`, where `value` is an F# expres #### Possibly supported: After null check within boolean expression ```fsharp -let len (str: string | null) = +let len (str: string?) = if (str <> null && str.Length > 0) then // OK - `str` must be non-null str.Length // OK here too else @@ -306,7 +335,7 @@ Note that the reverse (`str.Length > 0 && str <> null`) would give a warning, be ##### Likely unsupported: Pattern matching with wildcard ```fsharp -let len (str: string | null) = +let len (str: string?) = match str with | null -> -1 | _ -> str.Length // OK - 'str' is string @@ -341,7 +370,7 @@ Generally speaking, beyond highly-specialized cases, we cannot guarantee non-nul ```fsharp let myIsNull item = isNull item -let len (str: string | null) = +let len (str: string?) = if myIsNull str then -1 else @@ -353,7 +382,7 @@ Although this trivial example could certainly be done, it could quickly get out #### Likely unsupported: nested functions accessing outer scopes ```fsharp -let len (str: string | null) = +let len (str: string?) = let doWork() = str.Length // WARNING: maybe too complicated? Doesn't feel natural to F#? @@ -371,12 +400,12 @@ Although `doWork()` is called only in a scope where `str` would be non-null, thi To get around scenarios where compiler analysis cannot establish a non-null situation, a programmer can use the `Unchecked.notNull` function to convert a nullable reference type to a non-nullable reference type: ```fsharp -let len (ns: string | null) = - let s = Unchecked.notNull(ns) // unsafe, but lets you avoid a warning +let len (ns: string?) = + let s = notNull(ns) // unsafe, but lets you avoid a warning s.Length ``` -This function has a signature of `('T | null -> 'T)`. It will throw a `NullReferenceException` at runtime if the input is `null`. +This function has a signature of `('T? -> 'T)`. It will throw a `NullReferenceException` at runtime if the input is `null`. This sort of feature is inherently unsafe, and is by definition not something being enthusiastically considered. But it will be a necessity for when the compiler cannot analyze code well enough. @@ -385,7 +414,7 @@ This sort of feature is inherently unsafe, and is by definition not something be In all other situations not covered by compiler analysis, a warning is emitted by the compiler if a nullable reference is dereferenced or cast to a non-null type: ```fsharp -let f (ns: string | null) = +let f (ns: string?) = printfn "%d" ns.Length // WARNING: 'ns' may be null let mutable ns2 = ns @@ -394,9 +423,9 @@ let f (ns: string | null) = let s = ns :> string // WARNING: 'ns' may be null ``` -A warning is given when casting from `'S[]` to `('T | null)[]` and from `('S | null)[]` to `'T[]`. +A warning is given when casting from `'S[]` to `'T?[]` and from `'S?[]` to `'T[]`. -A warning is given when casting from `C<'S>` to `C<'T | null>` and from `C<'S | null>` to `C<'T>`. +A warning is given when casting from `C<'S>` to `C<'T?>` and from `C<'S?>` to `C<'T>`. ### Checking of non-null references @@ -448,7 +477,7 @@ Giving a reference type that could be `null` is fine and in accordance with exis ```fsharp type C<'T>() = class end -let c1 = C() // OK +let c1 = C() // OK type B() = class end @@ -503,28 +532,28 @@ Some examples: ```fsharp // Inferred signature: // -// val makeNullIfEmpty : str:string -> string | null +// val makeNullIfEmpty : str:string -> string? let makeNullIfEmpty (str: string) = match str with | "" -> null | _ -> str - // val xs : (string | null) list + // val xs : string? list let xs = [ ""; ""; null ] -// val ys : (string | null) [] +// val ys : string?[] let ys = [| ""; null; "" |] -// val zs : seq +// val zs : seq let zs = seq { yield ""; yield null } -// val x : 'a | null +// val x : 'a? let x = null // val s : string let mutable s = "hello" -// s is now (s | null), despite the warning +// s is now s?, despite the warning s <- null ``` @@ -538,7 +567,7 @@ let f1 s = s // At the time of writing this function: // -// val f2 : string -> (string | null) +// val f2 : string -> string? let f2 s = if s <> "" then "hello" else null // At the time of writing this function: @@ -563,10 +592,10 @@ Note that asserting non-nullability may be required in some situations where typ let neverNull (str: string) = match str with | "" -> "" - | _ -> makeNullIfEmpty str // val makeNullIfEmpty: 'T -> ('T | null) + | _ -> makeNullIfEmpty str // val makeNullIfEmpty: 'T -> 'T? ``` -We know that this will never be `null` because the empty string is accounted for, but the compiler may well infer `neverNull` to return a `(string | null)`. To get around this, asserting non-nullability may be required: +We know that this will never be `null` because the empty string is accounted for, but the compiler may well infer `neverNull` to return a `string?`. To get around this, asserting non-nullability may be required: ```fsharp let neverNull (str: string) = @@ -579,9 +608,9 @@ let neverNull (str: string) = ### FSharp.Core -Although we do not officially support forwards-compatibility, we do strive to ensure that older compilers can reference newer versions and "use the new features". At the same time, if something like `String.replicate` were to return a nullable string, that would be bad and antithetical to the spirit of F#. +Older compilers must be able to reference newer versions of FSharp.Core. -So, FSharp.Core will also need to be selective annotated, applying nullability to things only when we actually intend `null` values to be accepted or come out of a function. Specifically, we can: +FSharp.Core will be selective annotated, applying nullability to things only when we actually intend `null` values to be accepted or come out of a function. Specifically, we can: * Apply `[]` at the module level for the assembly * Apply `[]` on every input type we wish to accept `null` values for @@ -651,7 +680,7 @@ To remain in line our first principle: We'll consider careful suppression of nullability in F# tooling so that the extra information doesn't appear overwhelming. For example, imagine the following signature in tooltips: ```fsharp -member Join : source2: (Expression | null> | null) * key1:(Expression | null> | null)) * key2:(Expression | null> | null)) +member Join : source2: (Expression?>?) * key1:(Expression?>?)) * key2:(Expression?>?)) ``` AGH! This signature is challenging enough already, but now nullability has made it significantly worse. @@ -661,7 +690,7 @@ AGH! This signature is challenging enough already, but now nullability has made In QuickInfo tooltips today, we reduce the length of generic signatures with a section below the signature. We can do a similar thing for saying if something is nullable: ``` -member Foo : source: (seq<'T>) * predicate: ('T -> 'U- > bool) * item: 'U +member Foo : source: seq<'T> * predicate: ('T -> 'U -> bool) * item: 'U where 'T, 'U are nullable @@ -677,7 +706,7 @@ In Signature Help tooltips today, we embed the concrete type for a generic type ``` List(System.Collections.Generics.IEnumerable<'T>) - where 'T is (string | null) + where 'T is string? Initializes an instance of a [...] ``` @@ -686,10 +715,10 @@ Initializes an instance of a [...] To have parity with F# signatures (note: not tooltips), signatures printed in F# interactive will have nullability expression as if it were in code: -``` -> let s: string | null = "hello";; +```fsharp +> let s: string? = "hello";; - val s : string | null = "hello" +val s : string? = "hello" ``` #### F# scripting @@ -1085,11 +1114,22 @@ Today, the following specifiers work with reference types: Making these work with non-nullable reference types would be a breaking change, so it's likely that the underlying type be the appropriate `T | null` for each specifier. For example, `%s` would work with `string | null`. -**Recommendation:** These now operate assuming incoming types are nullable reference types. +**TBD:** These imply a nullability inference variable to allow to work with either kind + +### Interaction with `UseNullAsTrueValue` + +F# option types use the obscure attribute `UseNullAsTrueValue` to ensure that `None` gets compiled as `null`. This was a design choice made early in F# and has been problematic for many reasons, e.g. `None.ToString()` raises an exception, and baroque special compilation rules are needed for `opt.HasValue`. However, this choice has been made and we must live with it. + +In theory the `UseNullAsTrueValue` attribute can be used with other F#' option types subject to limitations, e.g. this error message: +``` +1196,tcInvalidUseNullAsTrueValue,"The 'UseNullAsTrueValue' attribute flag may only be used with union types that have one nullary case and at least one non-nullary case" +``` + +IMPORTANT: Types that use `UseNullAsTrueValue` may *not* be made nullable, so `option | null` is **not** allowed. -#### Emitting F# options types for C# consumption +Additionally the semantics of type tests are adjusted slightly to account for the possibility that `null` is a legitimate value, e.g. so that `match None with :? int option -> true | _ -> false` returns `true`. Here `None` is represented as `null`. This is done through helpers `TypeTestGeneric` and `TypeTestFast`. We should consider whether this needs documenting or adjusting in the same way as `UnboxGeneric` and `UnboxFast`, though on first glance I don't believe it does. -F# options types emit `None` as `null`. For example, considering the following public F# API: +In more detail, F# options types emit `None` as `null`. For example, considering the following public F# API: ```fsharp type C() = @@ -1121,7 +1161,7 @@ public class C This means that `FSharpOption` is a nullable reference type if it were to be consumed by C# 8.0. Indeed, there is definitely C# code out there that checks for `null` when consuming an F# option type to account for the `None` case. This is because there is no other way to safely extract the underlying value in C#! -**Recommendation:** ROLL INTO A BALL AND CRY - ...and discuss further with the C# team to see if something can be done about this. +**Recommendation:** discuss further with the C# team to see if something can be done about this. @@ -1141,19 +1181,6 @@ That is, the meaning of `unbox` in F# depends on whether the type carries a null The problem of course is that once (3) is allowed we have an issue - using the slow helper is now no longer correct and will raise an exception, i.e. `unbox(null)` will raise an exception. -### Interaction with `UseNullAsTrueValue` - -F# option types use the obscure attribute `UseNullAsTrueValue` to ensure that `None` gets compiled as `null`. This was a design choice made early in F# and has been problematic for many reasons, e.g. `None.ToString()` raises an exception, and baroque special compilation rules are needed for `opt.HasValue`. However, this choice has been made and we must live with it. - -In theory the `UseNullAsTrueValue` attribute can be used with other F#' option types subject to limitations, e.g. this error message: -``` -1196,tcInvalidUseNullAsTrueValue,"The 'UseNullAsTrueValue' attribute flag may only be used with union types that have one nullary case and at least one non-nullary case" -``` - -IMPORTANT: Types that use `UseNullAsTrueValue` may *not* be made nullable, so `option | null` is **not** allowed. - -Additionally the semantics of type tests are adjusted slightly to account for the possibility that `null` is a legitimate value, e.g. so that `match None with :? int option -> true | _ -> false` returns `true`. Here `None` is represented as `null`. This is done through helpers `TypeTestGeneric` and `TypeTestFast`. We should consider whether this needs documenting or adjusting in the same way as `UnboxGeneric` and `UnboxFast`, though on first glance I don't believe it does. - ### Interaction with default values