Export OpenAPI as Typescript JSON files
Closed this issue · 11 comments
While working on Fastify + TypeScript project I've found the common practice of NOT providing JSON schemas as .json files but as TS files with as const assertion. Eg:
// mySchema.ts
export const mySchema = {
type: "object"
//...
} as const..this allows Fastify type providers to declare route schemas and infer relevant types in a single go.
I found myself writing a few utility line to:
- Generate referenced JSON schemas with
oas2json - Derefence them with
@apidevtools/json-schema-ref-parser - Convert JSON files to TS files with
as constassertion
Pseudo-code example:
import glob from 'tiny-glob'
import $RefParser from '@apidevtools/json-schema-ref-parser'
import fs from 'fs/promises'
import path from 'path'
import prettier from 'prettier'
// Generate schemas with openapi-transformer-toolkit oas2json
// Get all absolute paths just created JSON schemas
const schemas = await glob(`${dereferencedSchemasDirectory}/*.json`, {
absolute: true
})
for (const schemaPath of schemas) {
// Dereference schemas (at least local ones)
const schema = await $RefParser.dereference(schemaPath)
const modelName = path.parse(schemaPath).name
// Append necessary TS declarations
const tsSchema =
`export const ${modelName} = ` + JSON.stringify(schema) + 'as const'
// Might be necessary to do a format round since @apidevtools/json-schema-ref-parser seems to minify its output
const formattedSchema = prettier.format(tsSchema, {
parser: 'typescript'
})
await fs.writeFile(
path.resolve(schemasDirectory, `${modelName}.ts`),
formattedSchema
)
}...the generated JSON schema TS files, could then be used an input to generate the relevant TS files with json-schema-to-ts. Possibly equivalent to oas2ts output.
I understand this is far away from the original scope of this project, but I wanted to document this use case I case it could make it into this library.
Thanks for the awesome work done here!
My suggestion is to not load and pass json files in the fastify schema definition, but register each schema with ajv.
In the NF blog post related to this library there is an example about how this works
My suggestion is to not load and pass json files in the fastify schema definition, but register each schema with ajv.
Reading the article https://www.nearform.com/blog/simplify-your-api-development-with-our-openapi-code-generation-tool/ it seems that:
- the json is loaded by using
fastfiy.addSchema(json) - then reference those schemas with a json
{ $ref: ‘Customer.json#’ }
I think here the selling point could be adding a feature to improve the oas2ts utility
I was thinking about:
- an option to manipulate the ts export style
- an option to isolate every file (by resolving the
$refs as @toomuchdesign pointed out)
@toomuchdesign can you show an example of what the output of your approach looks like, compared to the ts output of this tool? I would like to understand how much effort it would be to tweak the tool to make it do that as well, in addition to what it already does. In other words, I'd like to understand what gap there is from the current ts output and the one you'd need to use the output as route schemas. My assumption is that the current TS output can't be used for that, right?
Hi all!
This is a quick repro to show current openapi-transformer-toolkit outputs vs. what Fastify needs to infer route requests object type from provided schemas.
Current state
Open API schema input
Note how Pet refers to another Owner component:
openapi: "3.0.0"
info:
version: 1.0.0
title: Swagger Petstore
paths:
/pets:
get:
responses:
"200":
description: A paged array of pets
content:
application/json:
schema:
type: array
items:
$ref: "#/components/schemas/Pet"
components:
schemas:
Pet:
type: "object"
required:
- "id"
- "name"
properties:
id:
type: "integer"
format: "int64"
name:
type: "string"
owner:
$ref: "#/components/schemas/Owner"
Owner:
type: "object"
required:
- "id"
- "name"
properties:
id:
type: "integer"
format: "int64"
name:
type: "string"
`oas2json` output
Pet.owner prop is a reference to Owner.json:
// Pet.json
{
"type": "object",
"required": [
"id",
"name"
],
"properties": {
"id": {
"type": "integer",
"format": "int64",
"minimum": -9223372036854776000,
"maximum": 9223372036854776000
},
"name": {
"type": "string"
},
"owner": {
"$ref": "Owner.json"
}
},
"title": "Pet",
"$id": "Pet.json"
}
// Owner.json
{
"type": "object",
"required": [
"id",
"name"
],
"properties": {
"id": {
"type": "integer",
"format": "int64",
"minimum": -9223372036854776000,
"maximum": 9223372036854776000
},
"name": {
"type": "string"
}
},
"title": "Owner",
"$id": "Owner.json"
}
`oas2ts` output
It brilliantly dereferences $ref definitions, but it's meant to export TS definitions of the schema, not the schema itself:
// Pet.d.ts
import { Owner } from './Owner'
export interface Pet {
id: number;
name: string;
owner?: Owner;
[k: string]: unknown;
}
// Owner.d.ts
export interface Owner {
id: number;
name: string;
[k: string]: unknown;
}Possible new feature
What I understand Fastify typed schemas need is:
- a JSON schema definition (basically the same that
oas2jsoncreated) - dereferenced (no foreign
$refreferences) - defined as a
.tsfile - provided with an
as constassertion
// Pet.ts
export const Pet = {
type: "object",
required: ["id", "name"],
properties: {
id: {
type: "integer",
format: "int64",
minimum: -9223372036854776000,
maximum: 9223372036854776000,
},
name: { type: "string" },
owner: {
type: "object",
required: ["id", "name"],
properties: {
id: {
type: "integer",
format: "int64",
minimum: -9223372036854776000,
maximum: 9223372036854776000,
},
name: { type: "string" },
},
title: "Owner",
$id: "Owner.json",
},
},
title: "Pet",
$id: "Pet.json",
} as const;
// Owner.ts
export const Owner = {
type: "object",
required: ["id", "name"],
properties: {
id: {
type: "integer",
format: "int64",
minimum: -9223372036854776000,
maximum: 9223372036854776000,
},
name: { type: "string" },
},
title: "Owner",
$id: "Owner.json",
} as const;Point 2 (deferencing) can be delegated to one of the few existing libraries written for this purpose. We might decide to restrict deferencing to only existing local definitions, or maybe not. It might also be re-implemented if the case.
It might interesting ensuring whether it could be possible avoid duplicated references in the dereferenced schemas by importing the relevant sub schemas (instead of inlining).
Point 4 (as const) is unfortunately needed because of this TS limitation.
Extra considerations
oas2jsondoes not dereference whileoas2tsdoes it by default. Shall we normalize dereferencing behaviour in all commands and possibly provide an option to enable/disable it?oas2tswrapsjson-schema-to-typescriptwhich uses a forked version of@apidevtools/json-schema-ref-parser. There seems to be something to be investigated :)oas2tscurrently generates TS types definitions, while the possible new feature would generate TS JSON schemas: this is something possibly worth considering to name things
Thanks for the detailed explanation @toomuchdesign, seems like a valid use case. We'll look into it
Can you point me to an example of how you use the tson output from openapi-transformer-toolkit with Fastify? I see a link to a repo in this comment, but it is 404 for me.
Imagine I have a blog API with posts, comments, and users. The Comment reply schema includes commenter, which references the User schema. The Post schema includes author, which references the User schema and an array comments with items of the Comment schema.
If I use fastify.addSchema() to add schemas using the tson imports and write a route schema using response: { 200: { $ref: 'Post.json#' } } , Fastify (Ajv) throws an error because the User schema is defined more than once (in Post and in the post's Comment array). Commenting the $id tags in the inner dereferenced schemas in the tson solves the problem.
Example repo: https://github.com/jmjf/oatk-tson-experiment
I'm probably doing it wrong and would appreciate an example of how to do it right. Thanks.
Hi @jmjf,
a typical use case for TS JSON schema consists of using the same schema to validate and infer types for validated data. Eg:
- https://github.com/fastify/fastify-type-provider-json-schema-to-ts
- https://ajv.js.org/guide/typescript.html
I believe the error you're running into is an issue coming from how openapi-transformer-toolkit generates the schemas: the $id prop should only be present in the root entities and not in nested/dereferenced ones.
@toomuchdesign Thanks for the feedback. I'll look into the type provider approach.
Just in case someone lands here looking for an example, here's the original version that was having issues. Notes below outline changes to use TSON.
https://github.com/jmjf/oatk-tson-experiment
Why did I do it this way? Write once, use many--avoid maintaining schemas in both OpenAPI and TypeScript code, which creates opportunities for the two to get out of sync. IMO, the key benefits of driving off of OpenAPI and using openapi-transformer-toolkit is ensuring the published interface and the interface the running code enforces are the same and providing TypeScript types to help developers (including me) ensure they write interface-compliant code. Real world, I consume APIs where the spec is out of sync with the parameters the API accepts and the data the API delivers and they are a pain. I don't want to inflict that pain on others.
Using the user path as an example
In the OpenAPI spec, I added UserIdParm and changed the user path as shown. There might be a way to do this with components/parameters, which I'll investigate later.
paths:
/user/{userId}:
get:
operationId: getUserById
security: []
summary: GET user endpoint for tson issue
parameters:
- in: path
$ref: '#/components/schemas/UserIdParam'
responses:
'200':
description: result
content:
application/json:
schema:
$ref: '#/components/schemas/User'
'4xx':
description: error
# ...
components:
schemas:
UserIdParam:
title: UserIdParam
type: object
properties:
userId:
type: number
required:
- userIdGenerate TSON from OpenAPI with openapi-transformer-toolkit oas2tson.
In the server:
import Fastify from 'fastify'
import { JsonSchemaToTsProvider } from '@fastify/type-provider-json-schema-to-ts'
import { Comment } from './tson/Comment'
import { Post } from './tson/Post'
import { User } from './tson/User'
import { UserIdParam } from './tson/UserIdParam'
const fastify = Fastify({
logger: true
}).withTypeProvider<JsonSchemaToTsProvider>()
fastify.get('/user/:userId', {
handler (req, reply) {
req.log.info({ params: req.params }, 'GET user/{userId}')
reply.send( { userId: req.params.userId.toString(), userNm: 'fred', random: 123} )
},
schema: {
params: UserIdParam,
response: {
200: User
}
}
})Build and run. Apply similar for other paths.
Example outputs:
# These examples exclude "random"
node@cbcc34699c96:/workspace$ curl localhost:3000/user/123
{"userId":"123","userNm":"fred"}
node@cbcc34699c96:/workspace$ curl localhost:3000/user/456
{"userId":"456","userNm":"fred"}
# userId's type is validated
node@cbcc34699c96:/workspace$ curl localhost:3000/user/123a
{"statusCode":400,"code":"FST_ERR_VALIDATION","error":"Bad Request","message":"params/userId must be number"}Matteo Collina did an 11 minute video demoing how to do this with TypeBox which is basically the same thing. https://www.youtube.com/watch?v=hx6jy3MzQzw (Left me thinking, "Is that all?" but Matteo was fun watch and I'm glad to see I'm not the only one who retypes stuff three times :) )