Generated service not showing up as type `ServiceType`

Question

Generated service not showing up as type `ServiceType`

BrennerSpear opened this issue 6 months ago · 3 comments

Describe the bug

Hello all - I am folowing the Connect docs to implement a gRPC client to hit another service I have, and it looks like I am following it to the T, but getting type errors when i import my service.

BackfillOrchestrator, which is a service generated via buf generate is giving me a type error in the index.ts file

Any idea on what's going on?

# buf.gen.yaml

version: v1
plugins:
  - plugin: es
    out: ./packages/shadow-protos
    opt:
      - target=ts
  - plugin: connect-es
    out: ./packages/shadow-protos
    opt:
      - target=ts

// shadow-protos/services/backfill-orchestrator/v1/service_connect.ts

/**
 * @generated from service xyz.shadow.services.backfill_orchestrator.v1.BackfillOrchestrator
 */
export const BackfillOrchestrator = {
  typeName: "xyz.shadow.services.backfill_orchestrator.v1.BackfillOrchestrator",
  methods: {
    /**
     * @generated from rpc xyz.shadow.services.backfill_orchestrator.v1.BackfillOrchestrator.CreateBackfillJob
     */
    createBackfillJob: {
      name: "CreateBackfillJob",
      I: CreateBackfillJobRequest,
      O: CreateBackfillJobResponse,
      kind: MethodKind.Unary,
    },

    ...
}

// index.ts
import { TriggerAllHourlyExportWorkflowExecutionsResponse } from "@repo/shadow-protos/services/backfill-orchestrator/v1/service_pb";
import { BackfillOrchestrator } from "@repo/shadow-protos/services/backfill-orchestrator/v1/service_connect";
import { createConnectTransport } from "@connectrpc/connect-web";
import { createPromiseClient } from "@connectrpc/connect";

export async function triggerAllHourlyBackfills(): Promise<TriggerAllHourlyExportWorkflowExecutionsResponse> {
  const transport = createConnectTransport({
    baseUrl: "https://localhost:50052",
  });
  const client = createPromiseClient(BackfillOrchestrator, transport);

  //   let res = await client.triggerAllHourlyExportWorkflowExecutions({});
}

The error from BackfillOrchestrator is:

Argument of type '{ readonly typeName: "xyz.shadow.services.backfill_orchestrator.v1.BackfillOrchestrator"; readonly methods: { readonly createBackfillJob: { readonly name: "CreateBackfillJob"; readonly I: typeof CreateBackfillJobRequest; readonly O: typeof CreateBackfillJobResponse; readonly kind: any; }; ... 6 more ...; readonly tr...' is not assignable to parameter of type 'ServiceType'.
  Types of property 'methods' are incompatible.
    Type '{ readonly createBackfillJob: { readonly name: "CreateBackfillJob"; readonly I: typeof CreateBackfillJobRequest; readonly O: typeof CreateBackfillJobResponse; readonly kind: any; }; ... 6 more ...; readonly triggerAllHourlyExportWorkflowExecutions: { ...; }; }' is not assignable to type '{ [localName: string]: MethodInfo<AnyMessage, AnyMessage>; }'.
      Property 'createBackfillJob' is incompatible with index signature.
        Type '{ readonly name: "CreateBackfillJob"; readonly I: typeof CreateBackfillJobRequest; readonly O: typeof CreateBackfillJobResponse; readonly kind: any; }' is not assignable to type 'MethodInfo<AnyMessage, AnyMessage>'.
          Type '{ readonly name: "CreateBackfillJob"; readonly I: typeof CreateBackfillJobRequest; readonly O: typeof CreateBackfillJobResponse; readonly kind: any; }' is not assignable to type 'MethodInfoBiDiStreaming<AnyMessage, AnyMessage>'.
            The types returned by 'I.fromBinary(...)' are incompatible between these types.
              Type 'CreateBackfillJobRequest' is missing the following properties from type 'AnyMessage': equals, clone, fromBinary, fromJson, and 6 more.

generated CreateBackfillJobRequest via buf generate

/**
 * @generated from message xyz.shadow.services.backfill_orchestrator.v1.CreateBackfillJobRequest
 */
export class CreateBackfillJobRequest extends Message<CreateBackfillJobRequest> {
  /**
   * @generated from field: xyz.shadow.libs.common.v1.types.Chain chain = 1;
   */
  chain?: Chain;

  /**
   * @generated from field: xyz.shadow.libs.common.v1.types.Fork fork = 2;
   */
  fork?: Fork;

  /**
   * @generated from field: xyz.shadow.libs.common.v1.types.BlockRange block_range = 3;
   */
  blockRange?: BlockRange;

  constructor(data?: PartialMessage<CreateBackfillJobRequest>) {
    super();
    proto3.util.initPartial(data, this);
  }

  static readonly runtime: typeof proto3 = proto3;
  static readonly typeName = "xyz.shadow.services.backfill_orchestrator.v1.CreateBackfillJobRequest";
  static readonly fields: FieldList = proto3.util.newFieldList(() => [
    { no: 1, name: "chain", kind: "message", T: Chain },
    { no: 2, name: "fork", kind: "message", T: Fork },
    { no: 3, name: "block_range", kind: "message", T: BlockRange },
  ]);

  static fromBinary(bytes: Uint8Array, options?: Partial<BinaryReadOptions>): CreateBackfillJobRequest {
    return new CreateBackfillJobRequest().fromBinary(bytes, options);
  }

  static fromJson(jsonValue: JsonValue, options?: Partial<JsonReadOptions>): CreateBackfillJobRequest {
    return new CreateBackfillJobRequest().fromJson(jsonValue, options);
  }

  static fromJsonString(jsonString: string, options?: Partial<JsonReadOptions>): CreateBackfillJobRequest {
    return new CreateBackfillJobRequest().fromJsonString(jsonString, options);
  }

  static equals(a: CreateBackfillJobRequest | PlainMessage<CreateBackfillJobRequest> | undefined, b: CreateBackfillJobRequest | PlainMessage<CreateBackfillJobRequest> | undefined): boolean {
    return proto3.util.equals(CreateBackfillJobRequest, a, b);
  }
}

protofiles

syntax = "proto3";
package xyz.shadow.services.backfill_orchestrator.v1;

import "libs/common/v1/types.proto";

service BackfillOrchestrator {
  rpc CreateBackfillJob(CreateBackfillJobRequest)
      returns (CreateBackfillJobResponse);
}

message CreateBackfillJobRequest {
  .xyz.shadow.libs.common.v1.types.Chain chain = 1;
  .xyz.shadow.libs.common.v1.types.Fork fork = 2;
  .xyz.shadow.libs.common.v1.types.BlockRange block_range = 3;
}

message CreateBackfillJobResponse {
  string job_id = 1;
  .xyz.shadow.libs.common.v1.types.Fork fork = 2;
  .xyz.shadow.libs.common.v1.types.BlockRange block_range = 3;
  uint64 task_count = 4;
  uint64 blocks_per_task = 5;
  uint64 job_completion_estimate_in_minutes = 6;
}

syntax = "proto3";
package xyz.shadow.libs.common.v1.types;

message Chain {
  // Required.
  ChainId chain = 1;
  // Required.
  NetworkId network = 2;
}

enum ChainId {
  UnknownChain = 0;
  Ethereum = 1;
  Optimism = 2;
  Base = 3;
}

enum NetworkId {
  UnknownNetwork = 0;
  Mainnet = 1;
  Goerli = 2;
}

message Fork {
  // Required.
  string id = 1;
  // Optional. Default version is 0.
  uint64 version = 2;
}

// Right half-open interval denoting a list of blocks.
message BlockRange {
  // Start of the range (inclusive).
  uint64 start = 1;
  // End of the range (exclusive).
  uint64 end = 2;
}

Environment:

Node.js version: 18.19.0

{
  "dependencies": {
    "@bufbuild/protobuf": "^1.8.0",
    "@connectrpc/connect": "^1.4.0",
    "@connectrpc/connect-web": "^1.4.0",
    "@google-cloud/storage": "^7.9.0",
    "@google-cloud/workflows": "^3.2.0",
    "@repo/shadow-protos": "workspace:*"
  },
  "devDependencies": {
    "@repo/eslint-config": "workspace:*",
    "@repo/typescript-config": "workspace:*",
    "@types/eslint": "^8.56.5",
    "eslint": "^8.57.0",
    "typescript": "^5.3.3"
  }
}

// tsconfig
{
  "$schema": "https://json.schemastore.org/tsconfig",
  "display": "Default",
  "compilerOptions": {
    "declaration": true,
    "declarationMap": true,
    "esModuleInterop": true,
    "incremental": false,
    "isolatedModules": true,
    "lib": ["es2022", "DOM", "DOM.Iterable"],
    "module": "NodeNext",
    "moduleDetection": "force",
    "moduleResolution": "NodeNext",
    "noUncheckedIndexedAccess": true,
    "resolveJsonModule": true,
    "skipLibCheck": true,
    "strict": true,
    "target": "ES2022"
  }
}

Answer 1 · 2024-04-01T08:54:51.000Z

Hey Brenner, I copied your TypeScript config into our example and don't see an issue.

The cause of the error message you see is:

Type 'CreateBackfillJobRequest' is missing the following properties from type 'AnyMessage': equals, clone, fromBinary, fromJson, and 6 more.

This indicates that the base class Message from @bufbuild/protobuf could not be resolved when the compiler tries to compile the generated code.

You can remove the comment @ts-nocheck from the generated code to see an error message explaining why. (We generate the annotation for better BC, but it can be disable, see the documentation.)

I'm going to close this, but please feel free to re-open with a reproducible example.

Answer 2 · 2024-04-02T17:54:15.000Z

My fix was adding - import_extension=.ts

I'm using turbopack for my bundler

version: v1
plugins:
  # This will invoke protoc-gen-es
  - plugin: es
    out: ./packages/shadow-protos
    opt:
      - target=ts
      - import_extension=.ts
  # This will invoke protoc-gen-connect-es
  - plugin: connect-es
    out: ./packages/shadow-protos
    opt:
      - target=ts
      - import_extension=.ts

Answer 3 · 2024-04-03T10:27:41.000Z

Thanks for posting the resolution, Brenner!

If you are using next.js, this is a known issue, see vercel/next.js#59744. If you are not using next.js, the same issue likely applies to turbopack.

The ecosystem is in a bad state right now: You have to use the .js extension in imports for TypeScript's modern ESM resolution logic, but bundlers are slow to catch up.

The option import_extension=.ts or import_extension=none should both work in this case.