Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switch to storing enums as ints in JSON instead of strings #31100

Closed
roji opened this issue Jun 19, 2023 · 2 comments · Fixed by #31317
Closed

Switch to storing enums as ints in JSON instead of strings #31100

roji opened this issue Jun 19, 2023 · 2 comments · Fixed by #31317
Assignees
@maumar
Labels
area-json area-query breaking-change closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-enhancement
Milestone
8.0.0

Comments

@roji
Copy link
Member

roji commented Jun 19, 2023 •
edited
Loading

We currently serialize .NET enums into JSON as their string label; this is:

  1. Incompatible with the behavior of System.Text.Json and Newtonsoft.Json, which serialize to the underlying int value.
  2. Incompatible with our relational behavior, which also value-converts to the int value. This notably means that it's impossible to compare an enum value inside JSON with another enum value outside of JSON in a LINQ query.
@ajcvickers ajcvickers changed the title Switch to storing enums as ints in JSON instead of ints Switch to storing enums as ints in JSON instead of strings Jun 19, 2023
@petersladek
Copy link

We ran into this issue - we have json stored in db with enum values as ints. Is there any workaround right now? i.e. possibility of specifying value converter for json property?

@ajcvickers
Copy link
Contributor

@petersladek Use HasConversion<int> on the property configuration inside OnModelCreating.`

@ajcvickers ajcvickers added this to the 8.0.0 milestone Jul 7, 2023
maumar added a commit that referenced this issue Jul 20, 2023
@maumar
Fix to #31100 - Switch to storing enums as ints in JSON instead of st…
f2cda7f 
…rings

Problem was that in 7.0 we stored enums inside JSON as strings by default (by applying EnumToStringConverter by convention), but in 8.0 we are changing this to int.
This is a breaking change and it's extra problematic for databases that used EF JSON functionality in 7.0. This can easily create a scenario where there is a mix of string and int representation for an enum value within the same document.
(some data was added in 7.0, and then some in 8.0 before customer realized that the breaking change has been made). To mitigate this we are adding a fallback mechanism when reading enum data that is part of JSON entity. We try to read as int and if that fails we try to read again as string. This way should minimize the disruption, moreover any data saved back to the database will be saved in the new format, so over time everything should normalize.
We will still throw when projecting individual enum properties of a JSON entity (as opposed to the entire entity), because materialization for this goes through different path, indistinguishable from normal enum value read from column in relational table.

Fixes #31100
@maumar maumar mentioned this issue Jul 20, 2023
Merged
maumar added a commit that referenced this issue Jul 20, 2023
@maumar
Fix to #31100 - Switch to storing enums as ints in JSON instead of st…
cf27c6d 
…rings

Problem was that in 7.0 we stored enums inside JSON as strings by default (by applying EnumToStringConverter by convention), but in 8.0 we are changing this to int.
This is a breaking change and it's extra problematic for databases that used EF JSON functionality in 7.0. This can easily create a scenario where there is a mix of string and int representation for an enum value within the same document.
(some data was added in 7.0, and then some in 8.0 before customer realized that the breaking change has been made). To mitigate this we are adding a fallback mechanism when reading enum data that is part of JSON entity. We try to read as int and if that fails we try to read again as string. This way should minimize the disruption, moreover any data saved back to the database will be saved in the new format, so over time everything should normalize.
We will still throw when projecting individual enum properties of a JSON entity (as opposed to the entire entity), because materialization for this goes through different path, indistinguishable from normal enum value read from column in relational table.

Fixes #31100
maumar added a commit that referenced this issue Jul 20, 2023
@maumar
Fix to #31100 - Switch to storing enums as ints in JSON instead of st…
d374b81 
…rings

Problem was that in 7.0 we stored enums inside JSON as strings by default (by applying EnumToStringConverter by convention), but in 8.0 we are changing this to int.
This is a breaking change and it's extra problematic for databases that used EF JSON functionality in 7.0. This can easily create a scenario where there is a mix of string and int representation for an enum value within the same document.
(some data was added in 7.0, and then some in 8.0 before customer realized that the breaking change has been made). To mitigate this we are adding a fallback mechanism when reading enum data that is part of JSON entity. We try to read as int and if that fails we try to read again as string. This way should minimize the disruption, moreover any data saved back to the database will be saved in the new format, so over time everything should normalize.
We will still throw when projecting individual enum properties of a JSON entity (as opposed to the entire entity), because materialization for this goes through different path, indistinguishable from normal enum value read from column in relational table.

Fixes #31100
maumar added a commit that referenced this issue Jul 20, 2023
@maumar
Fix to #31100 - Switch to storing enums as ints in JSON instead of st…
48ae4b4 
…rings

Problem was that in 7.0 we stored enums inside JSON as strings by default (by applying EnumToStringConverter by convention), but in 8.0 we are changing this to int.
This is a breaking change and it's extra problematic for databases that used EF JSON functionality in 7.0. This can easily create a scenario where there is a mix of string and int representation for an enum value within the same document.
(some data was added in 7.0, and then some in 8.0 before customer realized that the breaking change has been made). To mitigate this we are adding a fallback mechanism when reading enum data that is part of JSON entity. We try to read as int and if that fails we try to read again as string. This way should minimize the disruption, moreover any data saved back to the database will be saved in the new format, so over time everything should normalize.
We will still throw when projecting individual enum properties of a JSON entity (as opposed to the entire entity), because materialization for this goes through different path, indistinguishable from normal enum value read from column in relational table.

Fixes #31100
ajcvickers pushed a commit that referenced this issue Aug 7, 2023
@maumar @ajcvickers
Fix to #31100 - Switch to storing enums as ints in JSON instead of st…
d3778f5 
…rings

Problem was that in 7.0 we stored enums inside JSON as strings by default (by applying EnumToStringConverter by convention), but in 8.0 we are changing this to int.
This is a breaking change and it's extra problematic for databases that used EF JSON functionality in 7.0. This can easily create a scenario where there is a mix of string and int representation for an enum value within the same document.
(some data was added in 7.0, and then some in 8.0 before customer realized that the breaking change has been made). To mitigate this we are adding a fallback mechanism when reading enum data that is part of JSON entity. We try to read as int and if that fails we try to read again as string. This way should minimize the disruption, moreover any data saved back to the database will be saved in the new format, so over time everything should normalize.
We will still throw when projecting individual enum properties of a JSON entity (as opposed to the entire entity), because materialization for this goes through different path, indistinguishable from normal enum value read from column in relational table.

Fixes #31100
ajcvickers pushed a commit that referenced this issue Aug 10, 2023
@maumar @ajcvickers
Fix to #31100 - Switch to storing enums as ints in JSON instead of st…
8a11953 
…rings

Problem was that in 7.0 we stored enums inside JSON as strings by default (by applying EnumToStringConverter by convention), but in 8.0 we are changing this to int.
This is a breaking change and it's extra problematic for databases that used EF JSON functionality in 7.0. This can easily create a scenario where there is a mix of string and int representation for an enum value within the same document.
(some data was added in 7.0, and then some in 8.0 before customer realized that the breaking change has been made). To mitigate this we are adding a fallback mechanism when reading enum data that is part of JSON entity. We try to read as int and if that fails we try to read again as string. This way should minimize the disruption, moreover any data saved back to the database will be saved in the new format, so over time everything should normalize.
We will still throw when projecting individual enum properties of a JSON entity (as opposed to the entire entity), because materialization for this goes through different path, indistinguishable from normal enum value read from column in relational table.

Fixes #31100
ajcvickers pushed a commit that referenced this issue Aug 10, 2023
@maumar @ajcvickers
Fix to #31100 - Switch to storing enums as ints in JSON instead of st…
f07d70e 
…rings

Problem was that in 7.0 we stored enums inside JSON as strings by default (by applying EnumToStringConverter by convention), but in 8.0 we are changing this to int.
This is a breaking change and it's extra problematic for databases that used EF JSON functionality in 7.0. This can easily create a scenario where there is a mix of string and int representation for an enum value within the same document.
(some data was added in 7.0, and then some in 8.0 before customer realized that the breaking change has been made). To mitigate this we are adding a fallback mechanism when reading enum data that is part of JSON entity. We try to read as int and if that fails we try to read again as string. This way should minimize the disruption, moreover any data saved back to the database will be saved in the new format, so over time everything should normalize.
We will still throw when projecting individual enum properties of a JSON entity (as opposed to the entire entity), because materialization for this goes through different path, indistinguishable from normal enum value read from column in relational table.

Fixes #31100

Taking over the PR

Tweaks from review
ajcvickers pushed a commit that referenced this issue Aug 10, 2023
@maumar
Fix to #31100 - Switch to storing enums as ints in JSON instead of st…
078c98f 
…rings (#31317)
@ajcvickers ajcvickers added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Aug 19, 2023
@ajcvickers ajcvickers modified the milestones: 8.0.0, 8.0.0-rc1 Aug 19, 2023
@ajcvickers ajcvickers mentioned this issue Oct 9, 2023
Closed
@ajcvickers ajcvickers modified the milestones: 8.0.0-rc1, 8.0.0 Nov 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@maumar maumar

Labels
area-json area-query breaking-change closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-enhancement
Projects
None yet
Milestone
8.0.0
Development

Successfully merging a pull request may close this issue.

Fix to #31100 - Switch to storing enums as ints in JSON instead of strings dotnet/efcore
4 participants
@ajcvickers @maumar @roji @petersladek