Skip to content

Latest commit

 

History

History
474 lines (377 loc) · 16 KB

MIGRATING.md

File metadata and controls

474 lines (377 loc) · 16 KB

Connect v1 to v2 migration guide

Connect v2 provides new features and simplifies some common APIs. In addition, it makes use of all the enhancements of Protobuf-ES v2. If you're currently using Connect v1, this document walks you through all you need to know to migrate and start using it right away.

Running the migration tool

To help with the process of migrating, we provide a tool called @connectrpc/connect-migrate which will take care of dependency and plugin updates as well as a few minor code changes. So, as a first step, you will need to run the migration tool. To do so, execute the following command:

npx @connectrpc/connect-migrate@latest

While the tool will do a lot of the dependency legwork for you, there are many use cases in code that it does not cover. Below, we list everything that has changed in v2. Changes that are handled by the migration tool are noted as such.

In summary, the suggested way to migrate to Connect v2 is to first run the migration tool and then go through the list below, fixing any other instances that weren't handled.

What's changed

Dependency updates

The next step in migrating is updating all dependencies. This includes the Protobuf-ES and Connect packages in package.json, plugin references in buf.gen.yaml, and any usage of generated SDKs in your package.json.

Protobuf-ES and Connect packages

One important dependency change to be aware of is that the plugin protoc-gen-connect-es has been removed in v2. Connect now relies on service descriptors generated by the Protobuf-ES v2 plugin protoc-gen-es and no longer generates code itself. Therefore, that dependency should be removed entirely from package.json. Your mileage may vary according to what @bufbuild and @connectrpc packages you depend on, but a list of relevant, compatible dependencies should look similar to the following:

  "dependencies": {
    "@bufbuild/protobuf": "^2.2.0",
    "@bufbuild/protoc-gen-es": "^2.2.0",
    "@connectrpc/connect": "^2.0.0",
-   "@connectrpc/protoc-gen-connect-es": "^1.0.0",
    "@connectrpc/connect-web": "^2.0.0",
    "@connectrpc/connect-node": "^2.0.0",
    "@connectrpc/connect-next": "^2.0.0",
    "@connectrpc/connect-fastify": "^2.0.0",
    "@connectrpc/connect-express": "^2.0.0",
    "@connectrpc/connect-query": "^2.0.0",
    "@connectrpc/connect-playwright": "^0.6.0"
  }

✅ The connect-migrate tool will handle this.

buf.gen.yaml

Next, any usage of the protoc-gen-connect-es plugin should be removed from buf.gen.yaml.

If you are using local plugins:

# buf.gen.yaml
version: v2
plugins:
  - local: protoc-gen-es
    out: src/gen
    opt: target=ts
- - local: protoc-gen-connect-es
-   out: src/gen
-   opt: target=ts

If you are using remote plugins:

# buf.gen.yaml
version: v2
plugins:
- - remote: buf.build/bufbuild/es:v1.10.0
+ - remote: buf.build/bufbuild/es:v2.2.0
    out: src/gen
    opt: target=ts
- - remote: buf.build/connectrpc/es
-   out: src/gen
-   opt: target=ts

✅ The connect-migrate tool will handle this.

Generated SDKs

The migration tool does not handle updating generated SDKs so these types of dependencies will need updated manually.

Since the Connect plugin no longer exists in v2, any generated SDK dependencies in your package.json that rely on this plugin (i.e. have connectrpc_es as part of their path) should be updated to use the Protobuf-ES v2 plugin instead.

The path for a generated SDK dependency is structured as follows:

@buf/{module_owner}_{module_name}.{plugin_owner}_{plugin_name}@{version}

We will walk through updating a generated SDK on the googleapis/googleapis module. To do this for other modules, simply replace googleapis/googleapis with your module owner and name (also note that your versions will differ per module).

A dependency on googleapis/googleapis that uses the Connect v1 plugin will look something like the following (note the connectrpc_es in the path). Remove this line from your package.json.

@buf/googleapis_googleapis.connectrpc_es@1.4.0-20241107203341-553fd4b4b3a6.3

To use the corresponding generated SDK with Protobuf-ES v2, use the command:

npm install @buf/googleapis_googleapis.bufbuild_es@latest

Your package.json should now resemble the following:

  "dependencies": {
    ...
-   "@buf/googleapis_googleapis.connectrpc_es": "^1.6.1-20241107203341-553fd4b4b3a6.1",
+   "@buf/googleapis_googleapis.bufbuild_es": "^2.2.2-20241107203341-553fd4b4b3a6.1",
    "@connectrpc/connect-web": "^2.0.0",
    "@bufbuild/protobuf": "^2.2.0",
    ...
  }

Generated code

Now that dependencies are updated, the next step is to re-generate code. The migration tool does not handle code generation, so be sure to do so in whatever way your project is configured. For example, npx buf generate, etc.

There are a few things to be aware with the switch to the Protobuf-ES v2 plugin as some of the options for that plugin have changed with its v2 release:

Editing buf.gen.yaml

If you are using Node16 module resolution and need the .js extension on all import paths, add the plugin option import_extension=js:

# buf.gen.yaml
version: v2
plugins:
 - local: protoc-gen-es
   out: src/gen
   opt:
     - target=ts
+    - import_extension=js

If you don't want the .js extension added to import paths, you can remove the plugin option import_extension=none - it's the default behavior now:

# buf.gen.yaml
version: v2
plugins:
  - local: protoc-gen-es
    out: src/gen
    opt:
      - target=ts
-     - import_extension=none

If you have been using the plugin option ts_nocheck=false, you can remove it as well - it's the default behavior now.

Removing unused files

When generating code, you should make sure to delete any now-unused *_connect.ts files. An easy way to do this is to use the clean option provided by the Buf CLI introduced in v1.36.0:

# buf.gen.yaml
version: v2
+ clean: true
plugins:
  - local: protoc-gen-es
    out: src/gen

With this option, buf generate will delete the contents of src/gen before generating code.

Import paths

Once your code is generated and the vestigial *_connect files are removed, import paths will need to be updated. This is usually an update from *_connect to *_pb:

- import { ElizaService } from "./gen/eliza_connect.js";
+ import { ElizaService } from "./gen/eliza_pb.js";

✅ The connect-migrate tool will handle this.

Working with generated code

As mentioned, Connect now solely uses the protoc-gen-es plugin provided by Protobuf-ES v2. As a result, working with generated code has changed in some areas.

Creating messages

To create a new instance, you now call the function create() from @bufbuild/protobuf and pass the generated schema:

- import { SayRequest } from "./gen/eliza_connect.js";
+ import { SayRequestSchema } from "./gen/eliza_pb.js";
+ import { create } from "@bufbuild/protobuf";

- const sayRequest = new SayRequest({
+ const sayRequest = create(SayRequestSchema, {
  sentence: "Hello",
});

Working with messages

When working with messages, there are a few noteworthy changes to be aware of:

  • The toPlainMessage function and the PlainMessage<T> type are no longer necessary. If you create a proto3 message with create(UserSchema), the returned object is already a plain object. You can replace the PlainMessage<User> types with User. The only difference is that User has a property $typeName, which is a simple string with the full name of the message like "example.User". This property makes sure you don't pass the wrong message to a function by accident.

  • The PartialMessage type has been mostly replaced by MessageInitShape, which extracts the init type from a message descriptor.

  • A message field using google.protobuf.Struct is now generated as a more-convenient JsonObject.

  • proto2 fields with default values are no longer generated as optional properties.

  • All well-known types and helpers have been moved to an export under @bufbuild/protobuf/wkt. For example, when converting a google.protobuf.Timestamp to an ECMAScript Date object, it is no longer possible via a method and must be done via helpers imported from this path.

  • JSON serialization options (passed to a transport or a server plugin) have been updated in Protobuf-ES v2. The option emitDefaultValues has been renamed:

    const transport = createConnectTransport({
      baseUrl: "https://demo.connectrpc.com",
      jsonOptions: {
    -   emitDefaultValues: true,
    +   alwaysEmitImplicit: true,
      },
    });
  • The JSON serialization option typeRegistry has also been renamed:

    import { createRegistry } from "@bufbuild/protobuf";
    import { createConnectTransport } from "@connectrpc/connect-web";
    - import { SayRequest } from "./gen/eliza_pb";
    + import { SayRequestSchema } from "./gen/eliza_pb";
    
    const transport = createConnectTransport({
      baseUrl: "https://demo.connectrpc.com",
      jsonOptions: {
    -   typeRegistry: createRegistry(SayRequest),
    -   registry: createRegistry(SayRequestSchema),
      },
    });

Note that registries have received a major update in Protobuf-ES v2 and are much more capable and flexible now. For more information, see the Protobuf registry documentation.

Service descriptors

Connect now relies on service descriptors generated by the Protobuf-ES v2 protoc-gen-es plugin. The v2 service descriptors aren't much different from the service descriptors previously generated by Connect v1. The same basic information (i.e. typed metadata) is still generated, but it's generated by protoc-gen-es now along with some additional information. For example, v2 now provides the full descriptor with features such as custom options, which can be very useful in interceptors.

One breaking change to be aware with service descriptors is that the access pattern of the MethodDescriptor has slightly changed.

Service descriptors in Connect v1 looked like this:

// protoc-gen-connect-es v1
import { ElizaService } from "./gen/eliza_connect";
ElizaService.typeName; // "connectrpc.eliza.v1.ElizaService"
ElizaService.methods.say;
ElizaService.methods.say.name; // "Say"
ElizaService.methods.say.kind; // MethodKind.Unary
ElizaService.methods.say.idempotency; // MethodIdempotency.NoSideEffects

Compare this to what is now generated by protoc-gen-es v2:

// protoc-gen-es v2
import { ElizaService } from "./gen/eliza_pb";
ElizaService.typeName; // "connectrpc.eliza.v1.ElizaService"
ElizaService.method.say;
ElizaService.method.say.name; // "Say"
ElizaService.method.say.methodKind; // "unary"
ElizaService.method.say.idempotency; //  MethodOptions_IdempotencyLevel.NoSideEffects

Note that:

  • instead of the plural form methods, it now uses the singular method.
  • the property kind for MethodKind has been renamed to methodKind and it now returns a string.
  • the idempotency property is now a string union instead of an enum.

For more information on all Protobuf-ES v2 changes, see the Protobuf-ES manual as well as the Protobuf-ES v2 migration guide.

Other breaking changes

  • Promise clients are now the default and the previously-deprecated createPromiseClient has been removed. Any callsites using createPromiseClient should be updated to use createClient.

    ✅ The connect-migrate tool will handle this.

  • The gRPC Transport now requires HTTP/2. If you are using createGrpcTransport and specifying an httpVersion, it will fail compilation. Remove the httpVersion property to use the default of HTTP/2.

    Note that if you were relying on HTTP/1.1 as part of your gRPC strategy, this may require bigger architectural changes, but the hope is that this is not a common problem.

  • Previously, Connect allowed request objects with matching shapes to be passed to API calls interchangeably as long as the passed object was a superset of the target type. For example, given the following proto definitions:

    syntax = "proto3";
    package example.v1;
    message MessageA {
      string field_a = 1;
    }
    message MessageB {
      string field_a = 1;
      int64 field_b = 2;
    }
    service ExampleService {
      rpc RequestA(MessageA) returns (Empty) {}
      rpc RequestB(MessageB) returns (Empty) {}
    }

    The following would have passed TypeScript compilation:

    client.requestA(new MessageA());
    client.requestA(new MessageB());

    This was an unintended bug and not a feature. In Connect v2, only the specified target type will pass compilation.

    If you intend to pass a message as a different message with the same fields, you can use object destructuring to drop the $typeName, and copy the rest of the properties:

    const messageA: MessageA;
    const { $typeName: _, ...properties } = messageA;
    const messageB = create(MessageBSchema, properties);
  • We have removed the credentials option from transports as well as the init option in interceptors. These two options were used to customize fetch routines. Users should now rely on the fetch option in transports as a way to perform these customizations. For example:

    createConnectTransport({
      baseUrl: "/",
      fetch: (input, init) => fetch(input, { ...init, credentials: "include" }),
    });

    In addition, as a replacement to determine whether an incoming request is a Connect GET request in server-side interceptors, the property requestMethod: string has been added to intercepted requests. This property is symmetrical to HandlerContext.requestMethod.

  • Errors details are now a pair of desc and init values. In Connect v1, error details were specified as message instances. In v2, error details are now an object that specifies both a schema and initialization object which are both passed to the create function of Protobuf-ES. For example:

    - import { LocalizedMessage } from "./gen/google/rpc/error_details_pb";
    - const details = [
    -   new LocalizedMessage({
    -       locale: "fr-CH",
    -       message: "Je n'ai plus de mots.",
    -   }),
    - ];
    + import { LocalizedMessageSchema } from "./gen/google/rpc/error_details_pb";
    + const details = [
    +   {
    +     desc: LocalizedMessageSchema,
    +     value: {
    +       locale: "fr-CH",
    +       message: "Je n'ai plus de mots.",
    +     }
    +   },
    + ];
    
    const metadata = new Headers({
      "words-left": "none"
    });
    throw new ConnectError(
      "I have no words anymore.",
      Code.ResourceExhausted,
      metadata,
      details
    );
  • MethodDescriptor is now self-sufficient. In v1, method descriptors always had to be used alongside the service descriptors. In v2, they can now be used standalone. This means that all callsites that previously passed a service and method descriptor should now just pass the method descriptor.

    A notable example is when using a Connect Router on the server-side:

    const routes = ({rpc}: Router) => {
    -   rpc(ElizaService, ElizaService.say, impl);
    +   rpc(ElizaService.say, impl);
    }
  • Interceptors for streaming RPCs now use appropriate stream types. In v1, the server used more exact types in interceptors, for example UnaryRequest for server-streaming rpcs while the client always used streaming variants. This was unintended behavior and has been fixed in v2. Now all streaming RPCs use the StreamRequest and StreamResponse types on the server as well.

  • Node 16 is no longer supported. Connect v2 now supports Node versions 18.14.1 and up.

  • Connect v2 now requires at least TypeScript v4.9.5.