Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: GrpcClient.loadProtoJSON to load protobuf.js JSON proto #985

Merged
merged 4 commits into from
Apr 29, 2021

Conversation

alexander-fenster
Copy link
Contributor

This is the first part of the change to stop using fs during loading protos, replacing it with require(). This was inspired by the discussion in googleapis/nodejs-pubsub#1246.

Since @grpc/proto-loader v0.6+ supports loading protos from protobuf.js JSON representation (.fromJSON), we can now avoid using fs altogether and just require() the proto JSON file, and then pass it to the proto loader. This will make all the bundlers much happier.

The second part of this fix will be in the generator which would need to start calling the new .loadProtoJSON instead of the existing .loadProto.

Note: the change to isBrowser invocation in fallback.ts is unrelated, it just got caught by my linter and I fixed it. It's actually a bug!

@alexander-fenster alexander-fenster requested a review from a team as a code owner April 23, 2021 08:42
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Apr 23, 2021
@codecov
Copy link

codecov bot commented Apr 23, 2021

Codecov Report

Merging #985 (b736217) into master (f131ee0) will decrease coverage by 0.03%.
The diff coverage is 96.42%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #985      +/-   ##
==========================================
- Coverage   88.58%   88.55%   -0.04%     
==========================================
  Files          43       43              
  Lines        7098     7123      +25     
  Branches      612      617       +5     
==========================================
+ Hits         6288     6308      +20     
- Misses        800      805       +5     
  Partials       10       10              
Impacted Files Coverage Δ
src/fallback.ts 78.80% <91.30%> (-0.49%) ⬇️
src/grpc.ts 93.91% <100.00%> (+0.27%) ⬆️
src/iamService.ts 83.84% <100.00%> (-0.10%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f131ee0...b736217. Read the comment docs.

Copy link
Contributor

@feywind feywind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, this looks pretty good to me. Should be a fairly clean fix for pack utils :)

* In rare cases users might need to deallocate all memory consumed by loaded protos.
* This method will delete the proto cache content.
*/
static clearProtoCache() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually... I kind of want to expose this in Pub/Sub, but that's a separate discussion. It would probably fix an issue with proto memory leaks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it must be accessible from Pub/Sub, but let's talk about memory leaks separately: I believe that with caching there should be none :)

src/iamService.ts Outdated Show resolved Hide resolved
@alexander-fenster alexander-fenster added the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 29, 2021
@yoshi-kokoro yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 29, 2021
@alexander-fenster alexander-fenster merged commit 819b447 into master Apr 29, 2021
@alexander-fenster alexander-fenster deleted the load-proto-json branch April 29, 2021 04:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants