Before- and after-create hooks

[mohanklein]

Hey buddy!

I came across the Tus protocoll these days, as I need to upload huge files and then convert them to mp3. I use Companion, AWS S3 Multpart, Lambda & Elastic Transcoder for the moment but the costs seem to high. So I want to run my own Tus Server for the files. I checked your package first as I prefer Ruby over Node or Go, but I then started using the Go implementation "tusd" as it overs a hook system, which is very similar to my current AWS Lambda Trigger "S3 Object created" and then triggers conversion of audio files to mp3.
Any possibility of implementing something like before create hook for stuff like authentication and after create hook for stuff like "move this file somewhere and manipulate it"? Or did you not choose doing so on purpose?

Cheers from snowy Germany!

[janko]

Yeah, I've seen that tusd has hooks, but I thought it wasn't necessary to implement it directly, because it's already possible indirectly via Rack middleware. Tus::Server is a Roda app, so you can add middlewares with use.

For example, this would be an "after create" hook:

class TusHooksMiddleware
  def initialize(app)
    @app = app
  end

  def call(env)
    request = Rack::Request.new(env)

    result = @app.call(env)

    if request.post?
      # perform your "after create" hook
    end

    result
  end
end

Tus::Server.use TusHooksMiddleware

Likewise, authentication can be performed before the @app.call(env). If you're using an authentication framework like Rodauth, you don't even need any hooks, as Rodauth works on the Rack level and not Rails level.

Alternatively, you can use Roda's hooks plugin, if you want to use Roda when implementing your hooks:

Tus::Server.plugin :hooks
Tus::Server.before do
  # authentication
end
Tus::Server.after do
  if request.post?
    # after create hoook
  end
end

Let me know if that doesn't suit your needs.

[mohanklein]

@janko-m sorry, I meant "after finished", so I can move the file and/or process it ...

[janko]

You can check whether the upload has finished by checking whether the request was a PATCH request and whether Upload-Offset and Upload-Length headers are the same.

Tus::Server.plugin :hooks

Tus::Server.after do
  if request.patch? && response.headers["Upload-Offset"] == response.headers["Upload-Length"]
    # after create hook
  end
end

But I see it would be useful to have hooks so that you don't have to know this logic. I'll see which ones tusd has and probably add them to tus-ruby-server.

[mohanklein]

yeah man!

[mohanklein]

@janko-m I'm new to tus, maybe you are willing to answer me some questions: I upload large audio files and then need to process these (create an additional mp3 version). it seems to me, that the main purpose of all tus implementations is, to get files uploaded to the server and that's it, right? so anything like after upload moving and serving the files can be any creative implementation of myself. for the moment I use tusd for uploading and a shell script after finished upload to create a mp3 version and then move the file+mp3 version to another folder on e.g. /mnt/new-disk/.... the folder name is the tus id.
is there any best practice for serving the files as downloads and enable deleting via request by user? currently I'm feeling like I am creating a really glued architecture, with tusd for uploads, shell script for processing and pinging my backend after finish ... So if I now start writing an extra web app (rack or node) only to serve downloads and let users delete files this mess gets bigger and bigger :-)

sorry, maybe you can give me an advice on best practice for serving/deleting after upload, as I can't find much on this topic ...

cheers

[janko]

If you want some structure in dealing with the file after it has been uploaded, you can use Shrine with shrine-tus. Shrine is a gem that helps you attach uploaded files to database records, while shrine-tus integrates with tus-ruby-server. So, you can have Shrine move the file directly from the tus-ruby-server storage (by default the data/ directory) to your configured Shrine storage (/mnt/new-disk/...) when attaching to a record (and optionally process an mp3).

Then for downloading the file from the Shrine storage (/mnt/new-disk/...) you can use the download_endpoint Shrine plugin.

As for deleting, if the file is "attached" to a database record, you can either have files automatically deleted by deleting the database record, or you can use the remove_attachment Shrine plugin to "deassign" the attached file (which would also delete the files). If you're using the versions plugin, Shrine will automatically delete both the original file and your generated mp3.

I primarily wrote tus-ruby-server so that I can bring resumable uploads to Shrine 😃

[mohanklein]

hm yeah the thing is, I have a tus server, and a backend+frontend server ... so my database connection is the backend ...

[janko]

If your requirements don't include attach files to database records, then you should probably skip Shrine. Outside of that I don't have any guidelines to give you, as it sounds like a custom workflow.

[mohanklein]

yeah that seems right. but thank you very much! :-)

[janko]

Hooks have been released in version 2.2.0: https://github.com/janko-m/tus-ruby-server#hooks