-
Notifications
You must be signed in to change notification settings - Fork 27.2k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Generic types over Vcs (vercel/turborepo#5738)
### Description This PR enables types generic over `Vc`s to work as value types. In other words, where we would previously need to declare all generic/container types over a given value type at the definition site, like the following: ```rust #[turbo_tasks::value] struct Thing; #[turbo_tasks::value(transparent)] struct Things(Vec<Vc<Thing>>); #[turbo_tasks::value(transparent)] struct ThingOption(Option<Vc<Thing>>); #[turbo_tasks::value(transparent)] struct ThingSet(IndexSet<Vc<Thing>>); #[turbo_tasks::value(transparent)] struct ThingMap(IndexMap<Vc<ThingKey>, Vc<Thing>>); let vec: Vc<Things> = Vc::cell(vec![Thing.cell()]); let option: Vc<ThingOption> = Vc::cell(Some(Thing.cell())); // etc. ``` We can now refer to `Vc<Vec<Vc<Thing>>>` directly, without needing to declare an intermediate transparent value type: ```rust #[turbo_tasks::value] struct Thing; let vec: Vc<Vec<_>> = Vc::cell(vec![Thing.cell()]); let option: Vc<Option<_>> = Vc::cell(Some(Thing.cell())); // etc. ``` The following generic types and methods are currently supported: * `Option<Vc<T>>` * `::is_some(self: Vc<Self>) -> Vc<bool>` * `::is_none(self: Vc<Self>) -> Vc<bool>` * `Vec<Vc<T>>` * `::is_empty(self: Vc<Self>) -> Vc<bool>` * `::len(self: Vc<Self>) -> Vc<usize>` * `IndexSet<Vc<T>>` * `::is_empty(self: Vc<Self>) -> Vc<bool>` * `::len(self: Vc<Self>) -> Vc<usize>` * `IndexMap<Vc<K>, Vc<V>>` * `::is_empty(self: Vc<Self>) -> Vc<bool>` * `::len(self: Vc<Self>) -> Vc<usize>` All these value types also implement `ValueDefault`, which means that you can instantiate them using `Default::default()` or `Vc::default()`: ```rust let vec: Vc<Vec<Vc<u32>>> = Default::default() ``` They also work recursively, but I would recommend creating intermediate types instead of this in most (all?) cases: ```rust let vec: Vc<Vec<Vc<Vec<Vc<u32>>>>> = Default::default() ``` ### Notes and implementation details 1. This only works when the generic type is wrapped in a `Vc`. In the example above, this would not cover `Vc<Vec<Thing>>` directly. The implementation relies on the fact that all `Type<Vc<T>>` are represented the exact same way in memory, and as such can all be cast to some common `Type<Vc<()>>` for storage. 2. Like primitives, these generic types can currently only be declared in `turbo_tasks` itself. Technically, if a crate owns a type `A`, it could also register `A` as a generic type, so we might want to allow this in the future (e.g. `AliasMap` in `turbopack`?) 3. There's some potential performance/correctness issues around `cell`ing, I'll defer to @sokra to figure out if this approach is worth it. ### Testing Instructions I added some tests to `all_in_one.rs`. --------- Co-authored-by: Justin Ridgewell <[email protected]> Co-authored-by: Tobias Koppers <[email protected]>
- Loading branch information
3 people
authored
Aug 24, 2023
1 parent
a636f30
commit e9b9610
Showing
24 changed files
with
743 additions
and
40 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
21 changes: 21 additions & 0 deletions
21
crates/turbo-tasks-macros-shared/src/generic_type_input.rs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| use syn::{ | ||
| parse::{Parse, ParseStream}, | ||
| Generics, Result, Token, Type, | ||
| }; | ||
|
|
||
| /// The input of the `generic_type` macro. | ||
| #[derive(Debug)] | ||
| pub struct GenericTypeInput { | ||
| pub generics: Generics, | ||
| pub ty: Type, | ||
| } | ||
|
|
||
| impl Parse for GenericTypeInput { | ||
| fn parse(input: ParseStream) -> Result<Self> { | ||
| let generics: Generics = input.parse()?; | ||
| let _comma: Token![,] = input.parse()?; | ||
| let ty: Type = input.parse()?; | ||
|
|
||
| Ok(GenericTypeInput { generics, ty }) | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,148 @@ | ||
| use proc_macro::TokenStream; | ||
| use quote::quote; | ||
| use syn::{parse_macro_input, spanned::Spanned, visit_mut::VisitMut, GenericParam, Lifetime, Type}; | ||
| use turbo_tasks_macros_shared::{get_type_ident, GenericTypeInput}; | ||
|
|
||
| use crate::value_macro::value_type_and_register; | ||
|
|
||
| pub fn generic_type(input: TokenStream) -> TokenStream { | ||
| let input = parse_macro_input!(input as GenericTypeInput); | ||
|
|
||
| for param in &input.generics.params { | ||
| match param { | ||
| syn::GenericParam::Type(ty) => { | ||
| if ty.ident == "Vc" { | ||
| ty.span() | ||
| .unwrap() | ||
| .error("Vc is a reserved name in generic_type") | ||
| .emit(); | ||
| } | ||
| } | ||
| syn::GenericParam::Lifetime(lt) => { | ||
| lt.span() | ||
| .unwrap() | ||
| .error("lifetime parameters are not supported in generic_type") | ||
| .emit(); | ||
| } | ||
| syn::GenericParam::Const(c) => { | ||
| c.span() | ||
| .unwrap() | ||
| .error("const parameters are not supported in generic_type") | ||
| .emit(); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| let (impl_generics, _, where_clause) = input.generics.split_for_impl(); | ||
|
|
||
| let repr = replace_generics_with_unit(input.generics.params.iter(), &input.ty); | ||
|
|
||
| let ty = input.ty; | ||
| let Some(ident) = get_type_ident(&ty) else { | ||
| return quote! { | ||
| // An error occurred while parsing the ident. | ||
| } | ||
| .into(); | ||
| }; | ||
|
|
||
| let mut generics_with_static = input.generics.clone(); | ||
| for param in &mut generics_with_static.params { | ||
| if let GenericParam::Type(param) = param { | ||
| param.bounds.push(syn::TypeParamBound::Lifetime(Lifetime { | ||
| ident: syn::Ident::new("static", param.ident.span()), | ||
| apostrophe: param.ident.span(), | ||
| })) | ||
| } | ||
| } | ||
|
|
||
| let value_type_and_register = value_type_and_register( | ||
| &ident, | ||
| quote! { #ty }, | ||
| Some(&generics_with_static), | ||
| quote! { | ||
| turbo_tasks::VcTransparentRead<#ty, #ty, #repr> | ||
| }, | ||
| quote! { | ||
| turbo_tasks::VcCellSharedMode<#ty> | ||
| }, | ||
| quote! { | ||
| turbo_tasks::ValueType::new_with_any_serialization::<#repr>() | ||
| }, | ||
| ); | ||
|
|
||
| quote! { | ||
| #value_type_and_register | ||
|
|
||
| impl #impl_generics Vc<#ty> #where_clause { | ||
| /// Converts this `Vc` to a generic representation. | ||
| fn to_repr(vc: Self) -> Vc<#repr> { | ||
| unsafe { | ||
| turbo_tasks::Vc::from_raw(Vc::into_raw(vc)) | ||
| } | ||
| } | ||
|
|
||
| /// Converts a generic representation of this `Vc` to the proper `Vc` type. | ||
| /// | ||
| /// # Safety | ||
| /// | ||
| /// The caller must ensure that the `repr` is a valid representation of this `Vc`. | ||
| unsafe fn from_repr(vc: Vc<#repr>) -> Self { | ||
| unsafe { | ||
| turbo_tasks::Vc::from_raw(Vc::into_raw(vc)) | ||
| } | ||
| } | ||
| } | ||
| } | ||
| .into() | ||
| } | ||
|
|
||
| struct ReplaceGenericsVisitor<'a> { | ||
| generics: &'a std::collections::HashSet<String>, | ||
| } | ||
|
|
||
| impl<'a> VisitMut for ReplaceGenericsVisitor<'a> { | ||
| fn visit_type_mut(&mut self, node: &mut Type) { | ||
| if let Type::Path(type_path) = node { | ||
| if type_path.qself.is_none() | ||
| && type_path.path.segments.len() == 1 | ||
| && type_path.path.segments[0].arguments.is_none() | ||
| && self | ||
| .generics | ||
| .contains(&type_path.path.segments[0].ident.to_string()) | ||
| { | ||
| // Replace the whole path with () | ||
| *node = syn::parse_quote! { () }; | ||
| return; | ||
| } | ||
| } | ||
|
|
||
| syn::visit_mut::visit_type_mut(self, node); | ||
| } | ||
| } | ||
|
|
||
| /// Replaces all instances of `params` generic types in `ty` with the unit type | ||
| /// `()`. | ||
| fn replace_generics_with_unit<'a, P>(params: P, ty: &Type) -> Type | ||
| where | ||
| P: IntoIterator<Item = &'a GenericParam>, | ||
| { | ||
| let generics_set: std::collections::HashSet<_> = params | ||
| .into_iter() | ||
| .filter_map(|param| { | ||
| if let GenericParam::Type(type_param) = param { | ||
| Some(type_param.ident.to_string()) | ||
| } else { | ||
| None | ||
| } | ||
| }) | ||
| .collect(); | ||
|
|
||
| let mut new_ty = ty.clone(); | ||
| let mut visitor = ReplaceGenericsVisitor { | ||
| generics: &generics_set, | ||
| }; | ||
|
|
||
| syn::visit_mut::visit_type_mut(&mut visitor, &mut new_ty); | ||
|
|
||
| new_ty | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.