Using twirpscript with Buf.build is not working due to prettier
Closed this issue · 16 comments
Hi !
I am trying to build with the buf client but I am unable to achieve a generation :/
I created a example repository that you can play with : https://github.com/alexisvisco/protocol-example
Just clone it and run make run (it will take some time because I assume that you are not using buf so I created an image and make run build it)
Output:
/usr/lib/node_modules/twirpscript/node_modules/prettier/index.js:12990
throw error;
^
SyntaxError: Unexpected token (239:53)
237 | const put = json.put ?? json.put;if (put) {msg.put = put;}
238 | const post = json.post ?? json.post;if (post) {msg.post = post;}
> 239 | const delete = json.delete ?? json.delete;if (delete) {msg.delete = delete;}
| ^
240 | const patch = json.patch ?? json.patch;if (patch) {msg.patch = patch;}
241 | const custom = json.custom ?? json.custom;if (custom) {const m = CustomHttpPattern.initialize();CustomHttpPattern._readMessageJSON(m, custom);msg.custom = m;}
242 | const body = json.body ?? json.body;if (body) {msg.body = body;}
at ku (/usr/lib/node_modules/twirpscript/node_modules/prettier/parser-babel.js:1:54892)
at Ou (/usr/lib/node_modules/twirpscript/node_modules/prettier/parser-babel.js:1:55033)
at Object.parse (/usr/lib/node_modules/twirpscript/node_modules/prettier/parser-babel.js:1:381521)
at Object.parse$d [as parse] (/usr/lib/node_modules/twirpscript/node_modules/prettier/index.js:12975:19)
at coreFormat (/usr/lib/node_modules/twirpscript/node_modules/prettier/index.js:14525:16)
at formatWithCursor$1 (/usr/lib/node_modules/twirpscript/node_modules/prettier/index.js:14765:14)
at /usr/lib/node_modules/twirpscript/node_modules/prettier/index.js:60959:12
at format (/usr/lib/node_modules/twirpscript/node_modules/prettier/index.js:60979:12)
at writeFile (/usr/lib/node_modules/twirpscript/dist/compile.js:19:47)
at /usr/lib/node_modules/twirpscript/dist/compile.js:28:9 {
loc: { start: { line: 239, column: 53 } },
codeFrame: ' 237 | const put = json.put ?? json.put;if (put) {msg.put = put;}\n' +
' 238 | const post = json.post ?? json.post;if (post) {msg.post = post;}\n' +
'> 239 | const delete = json.delete ?? json.delete;if (delete) {msg.delete = delete;}\n' +
' | ^\n' +
' 240 | const patch = json.patch ?? json.patch;if (patch) {msg.patch = patch;}\n' +
' 241 | const custom = json.custom ?? json.custom;if (custom) {const m = CustomHttpPattern.initialize();CustomHttpPattern._readMessageJSON(m, custom);msg.custom = m;}\n' +
' 242 | const body = json.body ?? json.body;if (body) {msg.body = body;}'
}
Failure: failed to generate plugins concurrently: failed to execute plugins concurrently: failed to generate: exit status 1.
I create an example repo for you to test : https://github.com/alexisvisco/protocol-example
Patching the compile.js and removing the formatting by prettier fixed the issue (with lot of warning from buf), however it would be nice to keep the formatting because files look really bad haha
Hey @alexisvisco thanks for opening this and linking the reproduction. I haven't looked into running twirpscript from buf. Right now, TwirpScript assumes that it is compiling all of a project's proto files in a single pass, so there may be a few changes necessary to integrate well with buf.
Buf have different strategy to generate, the all strategy commented in the file does not solve the problem of prettier.
@tatethurston yep, using strategy: all resolve warnings, maybe I should add a tsconfig.json and a package.jsoin at the root of the generation 🤔
Cool, looks like the all strategy resolves the warnings (those are generated because duplicate files are being generated for google's well known types, which TwirpScript outputs in a separate file rather than inlining).
The next issue to tackle is figuring out why protoc-gen-openapiv2 is being generated -- looking at the go output that is not desirable.
Then we'll need to specify the language output (typescript auto-detection is in the twirpscript cli runner and not the compiler)
- name: protoc-gen-twirpscript
path: node_modules/twirpscript/dist/compiler.js
out: gen/ts
opt: language=typescript
strategy: all
The major issue for me is the problem with prettier that stop the generation haha :)
Adding a package.json (with twirpscript as a dependency) and tsconfig does not solve the prettier issue 😂
Maybe adding an option to disable prettier formatting, I can prettify myself those files with mhy own tooling, look like a quick win fix ?
I’ll look into this some more and figure out what is causing the prettier error. I suspect this is a bug in TwirpScript unrelated to buf and instead specific to an input proto file. I think the output of one or more of the files is likely malformed and not runnable. But do let me know if you’re able to move forward with prettier disabled.
I’ll look into this some more and figure out what is causing the prettier error. I suspect this is a bug in TwirpScript unrelated to buf and instead specific to an input proto file. I think the output of one or more of the files is likely malformed and not runnable. But do let me know if you’re able to move forward with prettier disabled.
I haven't tested it yet, I will do it tomorrow (00:01 France ) thank you man for your answer, I will give you feedback asap.
Hello @tatethurston I update the example repository with a server in go, however I am unable to call the service in nodejs (in browser it might works but I don't have time to test this morning) because of the fetch module.
I tried to import node-fetch but it does not works, which means that clients are unusable in node environment.
➜ protocol-example git:(main) ✗ ts-node test.ts
/Users/alexisviscogliosi/dev/protocol-example/node_modules/twirpscript/dist/runtime/client/index.js:9
return fetch(url, opts);
^
ReferenceError: fetch is not defined
at Object.fetchTransport [as rpcTransport] (/Users/alexisviscogliosi/dev/protocol-example/node_modules/twirpscript/dist/runtime/client/index.js:9:5)
at Array.<anonymous> (/Users/alexisviscogliosi/dev/protocol-example/node_modules/twirpscript/dist/runtime/client/index.js:71:42)
at runMiddleware (/Users/alexisviscogliosi/dev/protocol-example/node_modules/twirpscript/dist/runtime/client/index.js:38:35)
at makeRequest (/Users/alexisviscogliosi/dev/protocol-example/node_modules/twirpscript/dist/runtime/client/index.js:69:12)
at PBrequest (/Users/alexisviscogliosi/dev/protocol-example/node_modules/twirpscript/dist/runtime/client/index.js:104:12)
at Log (/Users/alexisviscogliosi/dev/protocol-example/gen/ts/protos/dummy/v1/service_dummy.pb.ts:38:35)
at test (/Users/alexisviscogliosi/dev/protocol-example/test.ts:4:34)
at Object.<anonymous> (/Users/alexisviscogliosi/dev/protocol-example/test.ts:11:1)
at Module._compile (node:internal/modules/cjs/loader:1101:14)
at Module.m._compile (/Users/alexisviscogliosi/dev/protocol-example/node_modules/ts-node/src/index.ts:1459:23)
Hey @alexisvisco, by default TwirpScript uses fetch to make requests. This works out of the box for browser clients, but Node.js doesn't yet have fetch builtin to the runtime. You can override the fetch default, the documentation for server side clients illustrates this:
https://github.com/tatethurston/TwirpScript#server-client
note this line:
client.rpcTransport = nodeHttpTransport;
alternatively, you can make fetch available as a global with a fetch npm package such as node-fetch:
import fetch from 'node-fetch';
globalThis.fetch = fetch;
https://www.npmjs.com/package/node-fetch#loading-and-configuring-the-module
@tatethurston Hey ! After fixing the little problem with fetch the code generated without prettier works well.
What is you insight
what did you find wrong about the proto generation?
The client is usable, I might test the server too to be sure the generation is okay.
Edit: nodejs server is working too.
Do you think you can find the root cause of the bad generation or I can create a pull request for the option I suggested above ?
I created a minimal working example using buf: https://github.com/tatethurston/TwirpScript#buf https://github.com/tatethurston/twirpscript/tree/main/examples/buf.
I'd like to avoid an option to disable prettier formatting. Once the bug is resolved, that step won't be necessary, and the bug should be fixed either way.
@alexisvisco I tracked down the generation issue and fixed it. I've published v0.0.48 with the fix. Try it out and LMK if you run into any issues
@tatethurston thanks you so much for your investigation ! It works well I am very happy.