Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental V2 renderer #479

Draft
wants to merge 16 commits into
base: release-2.0
Choose a base branch
from
Draft

Experimental V2 renderer #479

wants to merge 16 commits into from

Conversation

jhollinger
Copy link
Contributor

@jhollinger jhollinger commented Oct 23, 2024

An experimental serializer/renderer for the proposed V2 base. Rapidly evolving. I know it looks large, but most of that is tests. It's broken up into small commits so we can easily pull out pieces and PR them individually if we want. Notes:

  • Added a ton of extension hooks for V2 and a better hook runner.
  • A simple formatter DSL in Blueprints (plus an extension hook).
  • Extractors are still their own thing - extensions don't seem a great fit.
  • The serializer dogfoods the reflection API.
  • The serializer dogfoods our hooks: most logic and all options come from built-in extensions.
  • "Outer" and "inner" serializer calls to solve [Bug] Data is lost on nested associations when preload is enabled for new records blueprinter-activerecord#30.
  • Can be significantly faster than V1 - up to 65%! (details below).
  • Blueprints and extractors get instantiated:
    • During each render.
    • A single instance is shared throughout the render (e.g. a single WidgetBlueprint instance).
    • Field blocks, if blocks, formatter blocks, etc. are instance-evaled against the blueprint instance.
    • This allows for helper methods and state during the render.

Performance

The perf characteristics of V1 and V2 vary greatly based on:

  1. The total number of fields and associations in a Blueprint.
  2. The relative number of fields vs associations in a Blueprint.
  3. How many items are being rendered overall.
# 2 fields, 1 objects, 0 collections
10,000 widgets 10x: V2 39.33% faster (0.2439 sec)
1,000 widgets 100x: V2 33.85% faster (0.1908 sec)
500 widgets 100x: V2 35.08% faster (0.0995 sec)
250 widgets 100x: V2 35.07% faster (0.0494 sec)
100 widgets 250x: V2 33.96% faster (0.0481 sec)
25 widgets 500x: V2 28.38% faster (0.0198 sec)
5 widgets 1000x: V2 00.55% faster (0.0001 sec)
1 widgets 1000x: V2 49.95% slower (0.0061 sec)

# 10 fields, 5 objects, 2 collections
10,000 widgets 10x: V2 52.72% faster (3.5379 sec)
1,000 widgets 100x: V2 49.02% faster (3.0312 sec)
500 widgets 100x: V2 48.19% faster (1.4554 sec)
250 widgets 100x: V2 47.73% faster (0.7174 sec)
100 widgets 250x: V2 46.12% faster (0.6754 sec)
25 widgets 500x: V2 44.38% faster (0.3056 sec)
5 widgets 1000x: V2 36.58% faster (0.0934 sec)
1 widgets 1000x: V2 19.83% slower (0.0098 sec)

# 100 fields, 50 objects, 25 collections
10,000 widgets 10x: V2 57.36% faster (64.8405 sec)
1,000 widgets 100x: V2 65.60% faster (74.7372 sec)
500 widgets 100x: V2 65.64% faster (37.2329 sec)
250 widgets 100x: V2 64.54% faster (17.7782 sec)
100 widgets 250x: V2 53.74% faster (11.3225 sec)
25 widgets 500x: V2 57.40% faster (5.8170 sec)
5 widgets 1000x: V2 53.44% faster (2.1441 sec)
1 widgets 1000x: V2 37.89% faster (0.3217 sec)

Note that on the very small end of things (few items with few fields) V2 is actually slower. But we're talking fractions of a ms.

@jhollinger jhollinger force-pushed the jh/2.0-renderer branch 28 times, most recently from 0b5e538 to 9bcc80a Compare October 26, 2024 21:58
@jhollinger jhollinger force-pushed the jh/2.0-renderer branch 3 times, most recently from 76e930f to b7a8adc Compare November 12, 2024 18:58
@jhollinger
Copy link
Contributor Author

jhollinger commented Nov 12, 2024

@sandstrom

* Implement association as it's own thing, instead of having that be just a field. Fields are inherently different from associations, and it'll be easier to work with if that's reflected in the internal code as well. [related issue](https://github.com/procore-oss/blueprinter/issues/407)

I think this is accounted for. I've actually split "assocation" into "singular" and "multiple" variants (currently with the placeholder names object and collection). They're implemented as separate structs from field and can be fetched separately from the reflection API.

* Separate internal options from external ones, instead of having some 'blessed' or 'special' options among user-defined ones. [related issue](https://github.com/procore-oss/blueprinter/issues/405)

This is a little tricky given the hook-based design. To ensure that extensions can be just as powerful as most built-in functionality, all built-in options (if, unless, default, root, extractor, etc) are implemented with extension hooks. But I suppose we could still store those options separately if there's a consensus around it. I did move blueprint and view out of options, but for slightly different reasons.

@jhollinger jhollinger force-pushed the jh/2.0-renderer branch 5 times, most recently from 0ee79d7 to 630f6f1 Compare November 12, 2024 21:41
@sandstrom
Copy link

I think this is accounted for. I've actually split "assocation" into "singular" and "multiple" variants (currently with the placeholder names object and collection). They're implemented as separate structs from field and can be fetched separately from the reflection API.

Ah, I see now. I was looking at this PR diff, but the true diff seems to be this one:

main...jh/2.0-renderer

This is a little tricky given the hook-based design. To ensure that extensions can be just as powerful as most built-in functionality, all built-in options (if, unless, default, root, extractor, etc) are implemented with extension hooks. But I suppose we could still store those options separately if there's a consensus around it. I did move blueprint and view out of options, but for slightly different reasons.

I'm just providing feedback here, based on some thoughts I had when we implemented Blueprinter in our codebase.

I understand, though I guess I also slightly disagree with the idea to implement built-in functionality using extension hooks. I think I would have taken a different path, but it's also quite subjective so take this with a grain of salt 😄

Overall, great to see the new V2 taking shape! 🎉

@jhollinger jhollinger force-pushed the jh/2.0-renderer branch 3 times, most recently from 3c3af70 to 977e832 Compare November 18, 2024 20:18
Signed-off-by: Jordan Hollinger <[email protected]>
Signed-off-by: Jordan Hollinger <[email protected]>
Signed-off-by: Jordan Hollinger <[email protected]>
Signed-off-by: Jordan Hollinger <[email protected]>
Signed-off-by: Jordan Hollinger <[email protected]>
Signed-off-by: Jordan Hollinger <[email protected]>
Signed-off-by: Jordan Hollinger <[email protected]>
Signed-off-by: Jordan Hollinger <[email protected]>
…k hooks that aren't used

Signed-off-by: Jordan Hollinger <[email protected]>
Signed-off-by: Jordan Hollinger <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants