Comparison with Tumau for potential collaboration / merge

Question

Comparison with Tumau for potential collaboration / merge

Opened this issue 5 years ago · 7 comments

Hi,

I'm writting a very similar software etienne-dldc/tumau so I think it would be interesting to compare the two to see how they differ and if we could merge them into a single project.

Note: I'm just writting the thing I can think of right now with my understanding of srv, this is probably incomplete and possibly wrong. If anyone want to add things / correct me feel free to do so.

Context

While srv does not have a concept of Context tumau has one. The reason behind this choice is that there are things that are not really related to request like authentication, external api... In fact in tumau the Request object is just there for typings really. And so if you want to parse JSON for example, you have to use a middleware that will parse the req as json and populate the Context so you can access it in other middlewares.
The drawback of this approach is that you have to type the context for your entire app (too complex to type for each middleware). So in the case of the JSON parser middleware you would type your context as { json?: any } and then you have to make sure the json key is populated before using it.
But in practice this is fine, in fact i'm quite satisfied with this.
Also note that the context is expected to be immutable. If a middleware want to change the context for the next one it need to pass it to next and/or to return a new context.

Middleware

The Middleware definition is a bit different, mainly because of the context.
The main differences are:

The middleware receive (ctx, next) as argument (the request is in the context)
The next function take a context as parameter
A middleware can return an object { ctx, response } to change the context on the way back
The next function return a Promise of an object { ctx, response }

Routing

In tumau routing is not part of the core bundle but in a separate one @tumau/router.
One of the main trouble I had while developping routing was to combine namespace (or prefix) while making the OPTION -> Allow header work...
This constraint explain most of the router API.

Debug

tumau does not have a debug system, but that's clearly missing !

Test / Benchmark / Repo

tumau is a monorepo (it was a bit hard to setup but I think it's OK now)
I've made a custom Jest matcher to snapshot HTTP Response Header (and a custom request runner)
I've tried to setup a benchmark but I'm not sure it's relaible...
I haven't setup a CI while srv has Travis

Answer 1 · 2019-10-11T03:16:58.000Z

Thanks for the overview, Etienne! I will try to provide a quick breakdown of the points you just mentioned.

Context

Not having a context is actually a very careful choice in the API design of SRV, since contexts are basically impossible to provide static typing for – much like global vars, actually.

It should very much be possible to define different sets of middlewares to be run for different routes, since some middlewares might be quite expensive to run and required functionalities might differ between routes quite a bit.

So the idea in SRV is (it's not well documented and still being tested, though) that a middleware is (request: SomeRequest, next: Handler) => Response, so the middleware calls next(downstreamRequest) where downstreamRequest can be just the request it was passed before or a new, extended request object based on it.

So a body parser middleware would take a random request instance (InputRequest extends Request), parse the request body and pass an extended request instance (OutputRequest extends InputRequest, JsonBodyRequest) down to the next middleware / route handler.

This way the request instance becomes the single source of truth and static type checking front to back is possible. Running the middlewares essentially becomes a simple compose(...middlewares)(request, r => r).

Routing

I tried to avoid having a separate package for the router. It might look beneficial to make the architecture super modular, but as a Koa poweruser I am always subtly annoyed by having to npm install and import two packages to just be able to write even the most basic services.

In the end almost everyone uses koa-router, anyway. I guess it would be the same situation here. And you could still opt-in to using an 3rd-party routing package.

Benchmark

I benchmarked it a while ago (gotta look it up again, but I also tweeted about it some time ago). The code has already been optimized quite a bit and reqs/sec are roughly similar to express and koa + koa-router.

Monorepo

Might make sense. Feel free to check out webpack-blocks, for instance, to see a battle-tested monorepo with a bunch of packages and Travis CI set up 🙂

Usability

I always try to optimize for the developer experience. Of course this project is in a quite early stage and a bunch of stuff is still missing, esp. documentation, for instance.

But I think it's important to have a short and pronouncable package name for this use case. So the @andywer/ scope would be dropped later on, but actually I have been thinking about a different package name already. We can talk about that a bit later.

Sorry for the wall of text 😅
Hope it all makes sense...

Answer 2 · 2019-10-12T13:37:09.000Z

I tried to avoid having a separate package for the router.

In the case of Tumau the point of splitting stuff into separate packages is to make it flexible.

I think it's important to have a short and pronouncable package name

I agree with that, I plan to release a tumau package that would just "glue" together the most common packages and expose an interface to them.
This mean that you can very quickly start using Tumau without wondering about sub packages but if later you want/need to replace something (like the router) you can.

Context

I just realized that in srv the request works really the way the context works in tumau (the main différence being the name and the fact that in tumau next let you access the context the next middleware returned...). So with that the main différence is typings.

srv: Try to type the Request / Context for each middleware
tumau: Globally type the Context / Request

I initially tried to type the Context for Each middleware but typings get really messy and DX is quite bad because you get cryptic TS errors when type mispatch.

Here is an example to illustrate how "messy" types gets:

In @tumau/router routes are matched in the order they are defined, in a route

you can return a response
you can return null in that case it will run the next route (this is for dynamic route matching)
call next will execute the middleware after the Router middleware

Server.create(
  Middleware.compose(
    Router(
      
    ),
    ctx => {
      // this middleware is executed if next is called inside a route middleware
    }
  )
)

Now typings this is quite hard because each route could have a different context type. You could give a CommonContext type but that mean you have to defined this type and manually add it.

Another problem is the compose function. Because TS does not support variadic you have to define override for each number of args:

compose<I1, I2, O2>(fn1: (ctx: I1) => I2, fn2: (ctx: I2) => O3): (ctx: I1) => O2;
compose<I1, I2, I3, O3>(fn1: (ctx: I1) => I2, fn2: (ctx: I2) => I3, fn3: (ctx: I3) => O3): (ctx: I1) => O3;
// ...

So my solution to this was to define a global Context type with optionnal properties. A little bit less safe but a better DX IMO because less TS madness overall...

Answer 3 · 2019-10-14T13:37:26.000Z

I initially tried to type the Context for Each middleware but typings get really messy and DX is quite bad because you get cryptic TS errors when type mispatch.

I think the main benefit of having first-class typescript support is to prevent errors like the route handler does not work, because I forgot a middleware. So I'd like to think that this is actually one of the most crucial spots to have working type checking.

Here is an example to illustrate how "messy" types gets:

To be fair, it seems like a lot of your past trouble stems from the fact that the API is not fully consistent. In SRV every route handler and every middleware takes a request and returns a response. Full stop, no exceptions. If you want to run the next route handler instead, you can return Response.Skip() which returns a valid Response instance that has a flag set indicating that the next route handler should be run instead.

I think it's way cleaner and allows type-safe code to pass things explicitly instead of following the old patterns that made middlewares in Express and Koa a mine field before (no offense). Consider a body parser middleware:

interface RequestWithBody<BodyType> extends Request {
  body?: BodyType
}

const BodyParserMiddleware = (
  request: Request,
  next: ((req: RequestWithBody<any>) => Response)
): Response => {
  const body = parseBody(request.rawRequest)
  const requestWithBody = request.derive({ body })
  return next(requestWithBody)
}

const BodyValidatorMiddleware = (rules: Rule[]) => (
  request: RequestWithBody<any>,
  next: ((req: RequestWithBody<any>) => Response)
) => {
  try {
    rules.every(rule => rule.validate(request.body))
  } catch (error) {
    return Response.Text(400, `Response body validation failed: ${error.message}`)
  }
  return next(request)
}

We can have perfect type-safety end-to-end regarding the data that the middlewares add. You can prevent adding the middlewares in the wrong order or forgetting one.

That's at least the idea behind it.

Answer 4 · 2019-10-15T20:12:24.000Z

We can have perfect type-safety end-to-end regarding the data that the middlewares add. You can prevent adding the middlewares in the wrong order or forgetting one.

In theory yes, but from what I have experienced in the past with functional APIs, this pattern tend to break down when you add generic middleware into the mix. Which brings me to the following question: have you tried to use srv in a real life project ? Cause if you have, I'm really curious to see how it looks 😃

Answer 5 · 2019-10-16T13:38:27.000Z

I am currently using it in some early-stage (closed-source) side projects of mine. Here is a snippet of the login routes file. I am calling the body parser explicitly here, not using it as a middleware yet. This will be the next step on the "testing srv in real life" journey.

Right now it's not yet proven that this approach will yield better code / good TypeScript errors. I do believe that the type checking should be a solvable problem, though, as support for compose()-like variadic functions has been constantly improved over the last couple of TS releases and it's a fairly common pattern. Hopefully I will know more in the next couple of days / weeks.

Here we go:

const routes = [
  Route.GET(paths.LOGIN, () => {
    return HTMLResponse(LoginPage(), "static")
  }),

  Route.POST(paths.LOGIN_ATTEMPT, async (request: Request) => {
    const { fields } = await parseBody(request)

    if (!fields.email || Array.isArray(fields.email)) {
      return Response.JSON(400, { details: `Parameter "email" is not set or not a single value.` })
    }

    const user = await queryUserByEmail(database, fields.email)

    if (!user) {
      return Response.JSON(404, { details: "User not found." })
    }
    if (!user.email_confirmed) {
      return Response.JSON(400, { details: "User did not confirm their email address yet." })
    }

    const jwt = createLoginToken(user)
    const loginConfirmationURL = new URL(routeTo.loginConfirmation(jwt), config.baseURL)

    await sendLoginConfirmation(user, String(loginConfirmationURL))

    return Response.JSON(200, {
      redirect: routeTo.loginAwaitConfirmation()
    })
  }),
  // ...
]

PS: In case you are wondering about the login logic and why you don't see anything about a password there – it's an email-based passwordless login 😉

Answer 6 · 2019-10-16T14:09:54.000Z

Hey, I have changed the way the context works in tumau (also there a single tumau package that exports everything 🎉 ).

While the previous context was an object that any middleware could mutate, it's now an abstraction that looks a bit like a Map where you have to pass a reference of a Context to get it's value.

Take a look at this file : etienne-dldc/tumau/examples/context

I know you will probably not like it 😆 because there are no garantee at compile time that middleware are where they are supposed to be but now you either have to explicitly tell it to throw if the context is missing (ctx.getOrThrow(NumContext)) or you have to deal with the case where the context is missing (ctx.get(NumContext) will return null if the context is missing and it does not have a default value, the type of the return is null | T).

I've updated the examples and also a project I'm working on where I'm using tumau and the DX is quite good because there is almost no explicit TS !

Answer 7 · 2019-10-16T15:26:51.000Z

PS: In case you are wondering about the login logic and why you don't see anything about a password there – it's an email-based passwordless login 😉

There is no session package for tumau for this exact same reason 😆