Middleware for nested `create`s
Opened this issue · 16 comments
Bug description
I am trying to write middleware e.g. to lowercase all my eMail and to hash password whereever a user gets created, but unfortunately it only works for direct creations and not for nested creations.
How to reproduce
Steps to reproduce the behavior:
- Go to any prisma example: e.g. https://github.com/prisma/prisma-examples/tree/latest/typescript/graphql-apollo-server
- Change the context.ts to something like this:
prisma.$use(async (params, next) => {
if (params.model == 'User' && params.action == 'create') {
params.args.data.email = params.args.data.email.toLowerCase()
}
return next(params)
})
create some nested create like this:
prisma.post.create({
data: {
author: {
create: {
email: "SoMe@EmaiL.com"
}
}
}
})- Run the instructions in the repo && prisma studio
- See error
--> SoMe@EmaiL.com in the user table
Expected behavior
Prisma information
Environment & setup
- OS: MacOS
- Database: PostgreSQL
- Node.js version: 14.1
- Prisma version:
@prisma/cli : 2.10.1
@prisma/client : 2.10.2
Current platform : darwin
Query Engine : query-engine 7d0087eadc7265e12d4b8d8c3516b02c4c965111 (at node_modules/@prisma/engines/query-engine-darwin)
Migration Engine : migration-engine-cli 7d0087eadc7265e12d4b8d8c3516b02c4c965111 (at node_modules/@prisma/engines/migration-engine-darwin)
Introspection Engine : introspection-core 7d0087eadc7265e12d4b8d8c3516b02c4c965111 (at node_modules/@prisma/engines/introspection-engine-darwin)
Format Binary : prisma-fmt 7d0087eadc7265e12d4b8d8c3516b02c4c965111 (at node_modules/@prisma/engines/prisma-fmt-darwin)
Studio : 0.304.0
Preview Features : connectOrCreate, transactionApi
Found a workaround for anyone who has the same issue. This recursively goes through any creation data and fixes the issues.
export function iterate(obj: any): any
{
Object.keys(obj).map((key: string) => {
switch (key) {
case 'eMail':
obj[key] = obj[key].toLowerCase()
break
case 'password':
obj[key] = hashSync(obj[key], 10)
break
}
if (typeof obj[key] === 'object') {
iterate(obj[key])
}
})
}
const prisma = new PrismaClient()
prisma.$use(async (params, next) => {
if (params.action == 'create') {
iterate(params.args.data)
}
return next(params)
})
Probably with a caviar though that it is fairly inefficient, but it works. :)
CASE
Hi prisma people, we are trying to use prisma middleware feature to implement subscriptions (publishing to the topic on the entity being changed). With some simplification, our code looks this way:
prisma.$use(async (params, next) => {
const result = await next(params)
// we only publish event when one of the tracked actions is called on a "subscribable" model
if (params.model === 'Address' && ['create', 'update', 'delete'].includes(params.action)) {
pubsub.publish('address_changed', {
operation: params.action,
data: result,
})
}
return result
})
This works perfectly for our case, but what we hit into are nested mutations, e.g.:
prisma.person.update({
data: {
addresses: {
create: {
....
}
}
}
...
})
In the case of nested mutations, middleware is not being called and we cannot react.
SOLUTION
That would be very helpful if we could specify an argument on the $use hook, to only track the operations on the root level, or on the nested entities as well.
If you could advise on if we can achieve this at the moment in a different way, or let us know if this is something we can expect from Prisma in the future, we would be grateful, thank you!
Refined method, based on @chrissisura. Quickly extended with async-handling und configurable via callbacks to be more flexible. May be of use for someone so I'll leave it here:
const applyMiddleware = async (data: any, keyToModify: string, cb: (data: any) => void) => {
await Promise.all(
Object.keys(data).map(async (key: string) => {
if (keyToModify === key) {
data[key] = await cb(data);
}
if (typeof data[key] === 'object') {
await applyMiddleware(data[key], keyToModify, cb);
}
})
);
};
prisma.$use(async (params, next) => {
// Slugify post-titles
switch (params.model) {
case 'Post': {
switch (params.action) {
case 'upsert':
case 'create':
case 'createMany':
case 'update':
case 'updateMany': {
await applyMiddleware(params.args.data, 'slug', (data) => slugify(data.title));
break;
}
}
}
// autohash user-passwords
case 'User': {
switch (params.action) {
case 'upsert':
case 'create':
case 'createMany':
case 'update':
case 'updateMany': {
await applyMiddleware(params.args.data, 'hashedPassword', (data) =>
hashPassword(data.hashedPassword)
);
break;
}
}
}
}
return next(params);
});Hi everyone,
I am also looking for a way to handle the execution of more complex logic before or after reads and writes. Due to the fact that there are no other ways of implementing some kind of life cycle events ("beforeUpdate", "afterDelete", ...) with Prisma, we have to rely on middlewares so far (see prisma/prisma-client-js#653). Even though they might be a first step in the right direction, they are quite limiting in comparison to actual life cycle events of conventional ORMs. Nested middlewares would give a lot more power to Prisma users eventually replacing the need of "real" life cycle events.
Use Cases for Nested Middlewares by Example
To give some examples, I will try to provide additional use cases for nested middlewares to show the importance of this feature in the long run.
Short Overview of a Client's Project
My fellow colleagues and I are building a GraphQL API with type-graphql (https://github.com/MichalLytek/type-graphql) and Prisma. Our database design consists out of two unique identifiers per model/table. I. e. a model has an auto-incrementing integer as primary key (generated by DB) and a so called "UID" which is - in this case - a nanoid (https://github.com/ai/nanoid), which can't be directly generated by the most DBMS. Users of the GraphQL API should not be able to guess the number of stored items in our database, so they can only query entities by their UID and will never be able to see the actual ID. Internally, we use the integer-based primary keys for better performance of database computations (like joins).
Use Cases
- We would love to have a middleware which automatically creates the UID for us on every model creation. With the current state of middlewares, this can only be archived on the root level of a model. See the following basic middleware as an example:
Example Middleware for the Creation of UIDs
import { Prisma } from '@prisma/client'
import { nanoid } from 'nanoid'
const uidGeneratorMiddleware: Prisma.Middleware = async (params, next) => {
if (params.action === 'create') {
if (params.args.data !== undefined && !params.args.data.uid) {
params.args.data['uid'] = nanoid()
} else if (params.args.data === undefined) {
params.args.data = { uid: nanoid() }
}
}
return next(params)
}
export default uidGeneratorMiddlewareOften, we create especially one-to-one (1-to-1) related models in a single Prisma query to profit from automatic transactions rather than calling prisma.$transaction(...).
Example: The model Car has two 1-to-1 relations to the models Engine and Transmission. They are always required for our exemplary data model to make sense (a car without an engine is probably not desirable...you get my point :D).
A GraphQL resolver for creating a Car would call:
Prisma Query
prisma.car.create({
data: {
uid: undefined, // unfortunately, this is needed because of https://github.com/prisma/prisma/issues/3696
// as soon as https://github.com/prisma/prisma/issues/3696 is fixed, there is no need of providing undefined here. Currently, as we pass in a value anyways, we could also call nanoid().
name: 'car name',
manufacturer: 'car manufacturer',
engine: {
create: {
uid: undefined, // even though it is annoying, we would pass undefined here too...
power: 150
}
},
transmission: {
create: {
uid: undefined, // ...and here
gears: 5,
}
}
}
})What happens with the query above is: Prisma calls the "UidMiddleware" only once for the root model Car. It would create the UID by calling nanoid() for it. But Engine and Transmission won't get a UID assigned. I could start to explicitly check for the existence of params.args.data['engine'].create and params.args.data['transmission'].create and extend my UidMiddleware as follows (simplified):
import { Prisma } from '@prisma/client'
import { nanoid } from 'nanoid'
const uidGeneratorMiddleware: Prisma.Middleware = async (params, next) => {
if (params.action === 'create') {
if (params.args.data !== undefined && !params.args.data.uid) {
params.args.data['uid'] = nanoid()
+ if (params.model === 'Car') {
+ if (params.args.data.engine?.create) {
+ params.args.data.engine.create['uid'] = nanoid()
+ }
+ }
} else if (params.args.data === undefined) {
params.args.data = { uid: nanoid() }
}
}
return next(params)
}
export default uidGeneratorMiddlewareIn this basic example that is already cumbersome but possible while in more complex environments with a lot of relations and models you simply cannot consider every possible combination in a time efficient manner.
- In addition to generating UIDs, we are encrypting/decrypting some values before writing them to database or reading them back. Especially user-given data like SMTP configurations or API keys to third parties. It would be great to do that in a simple middleware, which has to be defined only once. But have a look at the example above again. For some reason I would like to encrypt the number of gears of the transmission now; no matter how it was created (via
prisma.transmission.createor a relational query withcreatelike in my example) to ensuregearswill be definitely encrypted before it is written to database (or throw an error if the encryption fails to rollback the transaction). With middlewares, I have to remember to think of all possible "create combinations" and implement them by hand in every involved model's middleware. An alternative solution would be to just always useprisma.transmission.create()directly, but that defeats the whole point of Prisma IMHO. One could argue that you can archive a similar behavior by using custom models (https://www.prisma.io/docs/concepts/components/prisma-client/custom-models). Although this statement is true, you still have to manually think of possible combinations. Furthermore, you have to remember to actually use your custom models instead of directly accessing them withprisma.
I could list more use cases like 1) and 2) but I hope I was able to give you some insight on the possible added value which could come from nested middlewares which assure the execution for every nested model as well.
@chrissisura and @benbender have shown some ways of using middlewares for nested purposes (thank you for proposing these workarounds!). The problem of those solutions is, however, that they do not consider the type of the model once they start to recursively go through the data. Referring back to my 2nd use case, I cannot guarantee, that I want any field, which is called "username", to be encrypted. For instance, I certainly do not want to encrypt the username in the table "users" even though it has the same name as my SMTP configuration's username, which has to be encrypted. The following query would encrypt the username of User and SmtpConfiguration:
prisma.user.create({
data: {
username: 'anna',
smtpConfiguration: {
create: {
host: 'example.com',
username: 'anna@example.com',
// ...
}
}
})With improved nested Prisma middlewares, the above mentioned query would lead to two seperate middleware calls:
action: 'create'
model: 'User'
params.args.data = { username: 'anna' }
action: 'create'
model: 'SmtpConfiguration'
params.args.data = { host: 'example.com', username: 'anna@example.com' }
Thank you for reading through and, also, thanks a lot for the creation and maintenance of Prisma. It's a great tool and I am looking forward to see new features (for instance nested middlewares ;))!
Cheers.
Hi there, this was a problem for our team also as our existing code uses Loopback operation hooks (on save, on delete etc). In order to move over to Prisma we needed something that would make the transition easier.
In the end I implemented something in userland that is a version of @Charioteer's nested middleware suggestion.
The way it works is that you pass a middleware function to createNestedMiddleware which returns modified middleware that can be passed to client.$use:
client.$use(createNestedMiddleware((params, next) => {
// update params here
const result = await next(params)
// update result here
return result;
));This calls the middleware passed for the top-level case as well as for nested writes.
Edit: I've published the below code as a standalone npm package: prisma-nested-middleware.
Code
import { Prisma } from '@prisma/client';
import get from 'lodash/get';
import set from 'lodash/set';
const relationsByModel: Record<string, Prisma.DMMF.Model['fields']> = {};
Prisma.dmmf.datamodel.models.forEach((model) => {
relationsByModel[model.name] = model.fields.filter(
(field) => field.kind === 'object' && field.relationName
);
});
export type NestedAction = Prisma.PrismaAction | 'connectOrCreate';
export type NestedParams = Omit<Prisma.MiddlewareParams, 'action'> & {
action: NestedAction;
scope?: NestedParams;
};
export type NestedMiddleware<T = any> = (
params: NestedParams,
next: (modifiedParams: NestedParams) => Promise<T>
) => Promise<T>;
type WriteInfo = {
params: NestedParams;
argPath: string;
};
type PromiseCallbackRef = {
resolve: (result?: any) => void;
reject: (reason?: any) => void;
};
const writeOperationsSupportingNestedWrites: NestedAction[] = [
'create',
'update',
'upsert',
'connectOrCreate',
];
const writeOperations: NestedAction[] = [
...writeOperationsSupportingNestedWrites,
'createMany',
'updateMany',
'delete',
'deleteMany',
];
function isWriteOperation(key: any): key is NestedAction {
return writeOperations.includes(key);
}
function extractWriteInfo(
params: NestedParams,
model: Prisma.ModelName,
argPath: string
): WriteInfo[] {
const arg = get(params.args, argPath, {});
return Object.keys(arg)
.filter(isWriteOperation)
.map((operation) => ({
argPath,
params: {
...params,
model,
action: operation,
args: arg[operation],
scope: params,
},
}));
}
function extractNestedWriteInfo(
params: NestedParams,
relation: Prisma.DMMF.Field
): WriteInfo[] {
const model = relation.type as Prisma.ModelName;
switch (params.action) {
case 'upsert':
return [
...extractWriteInfo(params, model, `update.${relation.name}`),
...extractWriteInfo(params, model, `create.${relation.name}`),
];
case 'create':
// nested creates use args as data instead of including a data field.
if (params.scope) {
return extractWriteInfo(params, model, relation.name);
}
return extractWriteInfo(params, model, `data.${relation.name}`);
case 'update':
case 'updateMany':
case 'createMany':
return extractWriteInfo(params, model, `data.${relation.name}`);
case 'connectOrCreate':
return extractWriteInfo(params, model, `create.${relation.name}`);
default:
return [];
}
}
export function createNestedMiddleware<T>(
middleware: NestedMiddleware
): Prisma.Middleware<T> {
const nestedMiddleware: NestedMiddleware = async (params, next) => {
const relations = relationsByModel[params.model || ''] || [];
const finalParams = params;
const nestedWrites: {
relationName: string;
nextReached: Promise<unknown>;
resultCallbacks: PromiseCallbackRef;
result: Promise<any>;
}[] = [];
if (writeOperationsSupportingNestedWrites.includes(params.action)) {
relations.forEach((relation) =>
extractNestedWriteInfo(params, relation).forEach((nestedWriteInfo) => {
// store nextReached promise callbacks to set whether next has been
// called or if middleware has thrown beforehand
const nextReachedCallbacks: PromiseCallbackRef = {
resolve() {},
reject() {},
};
// store result promise callbacks so we can settle it once we know how
const resultCallbacks: PromiseCallbackRef = {
resolve() {},
reject() {},
};
const nextReached = new Promise<void>((resolve, reject) => {
nextReachedCallbacks.resolve = resolve;
nextReachedCallbacks.reject = reject;
});
const result = nestedMiddleware(
nestedWriteInfo.params,
(updatedParams) => {
// Update final params to include nested middleware changes.
// Scope updates to [argPath].[action] to avoid breaking params
set(
finalParams.args,
`${nestedWriteInfo.argPath}.${updatedParams.action}`,
updatedParams.args
);
// notify parent middleware that params have been updated
nextReachedCallbacks.resolve();
// only resolve nested next when resolveRef.resolve is called
return new Promise((resolve, reject) => {
resultCallbacks.resolve = resolve;
resultCallbacks.reject = reject;
});
}
).catch((e) => {
// reject nextReached promise so if it has not already resolved the
// parent will catch the error when awaiting it.
nextReachedCallbacks.reject(e);
// rethrow error so the parent catches it when awaiting `result`
throw e;
});
nestedWrites.push({
relationName: relation.name,
nextReached,
resultCallbacks,
result,
});
})
);
}
try {
// wait for all nested middleware to have reached next and updated params
await Promise.all(nestedWrites.map(({ nextReached }) => nextReached));
// evaluate result from parent middleware
const result = await middleware(finalParams, next);
// resolve nested middleware next functions with relevant slice of result
await Promise.all(
nestedWrites.map(async (nestedWrite) => {
// result cannot be null because only writes can have nested writes.
const nestedResult = get(result, nestedWrite.relationName);
// if relationship hasn't been included nestedResult is undefined.
nestedWrite.resultCallbacks.resolve(nestedResult);
// set final result relation to be result of nested middleware
set(result, nestedWrite.relationName, await nestedWrite.result);
})
);
return result;
} catch (e) {
// When parent rejects also reject the nested next functions promises
await Promise.all(
nestedWrites.map((nestedWrite) => {
nestedWrite.resultCallbacks.reject(e);
return nestedWrite.result;
})
);
throw e;
}
};
return (nestedMiddleware as unknown) as Prisma.Middleware;
}Usage / Explanation
What createNestedMiddleware does is call the middleware it's been passed for every nested relation. It does this by using the dmmf object which contains information about the relations defined in schema.prisma.
So for the following update:
client.country.update({
where: { id: 'imagination-land' },
data: {
nationalDish: {
update: {
where: { id: 'stardust-pie' },
data: {
keyIngredient: {
connectOrCreate: {
create: { name: 'Stardust' },
connect: { id: 'stardust' },
},
},
},
},
},
},
});It calls middleware function with params in the following order:
{ model: 'Recipe', action: 'update', args: { where: { id: 'stardust-pie' }, data: {...} } }{ model: 'Food', action: 'connectOrCreate', args: { create: {...}, connect: {...} } }{ model: 'Country', action: 'update', args: { where: { id: 'imagination-land', data: {...} } }
Then it waits for all the nested next functions to have been passed params, updates the top level params object with those objects and awaits the top level next function, in this case the next where model is 'Country'.
Once the top level next function resolves with a result the next functions of the nested middleware are resolved with the slice of the result relevent to them. So the middleware called for the 'Recipe' model receives the recipe object, the middleware for the 'Food' receives the food object.
Then the return values from the nested middleware are used to modify the top level result that is finally returned from the top level middleware.
There are a couple wrinkles:
- the list of actions that might be in params is expanded to include 'connectOrCreate'
- If a relation is not included using
includethen that middleware'snextfunction will resolve withundefined. - sometimes the parent params are relevent, for instance if being connected you need to know the parent you are being connected to. To resolve this I added a
scopeobject to params of nested middleware which is the parent params. - when handling nested
createactionsparams.argsdoes not include adatafield, that must be handled manually. You can use the existence ofparams.scopeto know when to handle a nestedcreate.
I haven't raised a PR since there is probably a better way to do this internally, however I haven't checked.... if this seems like a good solution to the Prisma team I'll happily do so 👍
Disclamer: we've written tests for our middleware but that doesn't mean this will work for you! Make sure you write your own tests before shipping anything that uses this.
Prisma client definitely needs nested middlewares, is this feature planned for the near future? Can we expect an API similar to the one proposed by @olivierwilkinson ?
In the end I implemented something in userland that is a version of @Charioteer's nested middleware suggestion.
Big thanks for sharing, @olivierwilkinson - this is fantastic.
Minor observation that it seems to work with Prisma v4 unchanged, without requiring the tweaks mentioned in the comments re: awaiting getDMMF(). (This seems in line with the v4 upgrade guide which states the required Prisma.dmmf.datamodel remains accessible in the Prisma Client.)
Big thanks for sharing, @olivierwilkinson - this is fantastic.
No problem, I'm glad you are finding it useful 😄
Minor observation that it seems to work with Prisma v4 unchanged, without requiring the tweaks mentioned in the comments re: awaiting
getDMMF().
Ah I must have misunderstood when I first read that, thank you for saying! 🙌 I'll update my code above (and elsewhere haha)
Could this be released as a standalone npm package while we wait for the core team to adopt something similar (or better) ?
Could this be released as a standalone npm package while we wait for the core team to adopt something similar (or better) ?
I can do that for the meantime for sure 😁. I should find some time this week to do it, I'll post on here once it's published 👍
@lewebsimple @andyjy I've published an npm package prisma-nested-middleware with the code I posted above. 👍
Is there any update on this topic? This should be relevant for all b2b saas cases IMO.
@JoshHupe check out the upcoming Prisma Client extensions.
(Note that extensions in their current state also do not support nested operations yet.)
Heya,
I've continued working on the prisma-nested-middleware package that I mentioned above. It now calls the middleware function for relations found in include, select and where objects, and can also change nested write actions to a different action type. Modifying results by model is also now supported through the select and include actions.
I wasn't sure if this warranted a new comment so apologies if I should have just edited one of my previous comments!
Just so everyone is aware, since middleware has been deprecated I've ported the work I did for the prisma-nested-middleware package to work for extensions, you can find it at prisma-extension-nested-operations. It is also published on npm under the same name 😁