This is a modified fork to demonstrate the following error as a reply to this question https://forums.swift.org/t/how-to-use-nonblockingfileio-for-repeated-writes/36206/18:
Precondition failed: blacklisted errno 9 Bad file descriptor in write(descriptor:pointer:size:)): file /Users/coder/prog/forks/VaporUploads/.build/checkouts/swift-nio/Sources/NIO/System.swift, line 112
To reproduce just clone this repository, start the Vapor server with swift Run
and issue a file upload similar to this twice:
curl --verbose --data-binary @/Users/coder/Desktop/SampleImage.jpg http://localhost:8080/stream
Files get sent over the internet in different ways. Vapor supports three that I’m aware of. Collecting data in the request, streaming data in a request, and streaming data in WebSockets.
Vapor provides a handy File type that can be used in your model to encode useful information like file name.
You can upload files on a POST
and collect all the incoming bytes with body: .collect(maxSize)
as seen in routes.swift:11.
This is straightforward. Increase the maxSize value and you’ll use that much more system RAM per active request. The body will collect common inbound encodings including JSON and form/multipart.
The obvious downside is that if you’re expecting many requests or large uploads your memory footprint will balloon. Imagine uploading a multigigabyte file into RAM!
You can also upload files on a POST
and stream the incoming bytes for handling as seen in routes.swift:18 and StreamController.swift:24
This is a bit trickier. You’ll need to understand:
- Promises
- Futures
- NIO’s file handling types:
NonBlockingFileIO
andNIOFileHandle
These aren’t extremely difficult concepts, but if its new domain expertise for you then you’ll have the benefit of learning something new.
The benefit is that the inbound bytes are handled and released from memory, keeping memory usage extremely low: KB instead of MB/GB. You can support many concurrent connections. You can stream very large files.
Not yet in this repository, but worth noting for its novelty and performance potential is that you can stream binary data over WebSockets. The strategy is fairly similar to HTTP Streaming because you’ll read inbound bytes (websocket.onBinary()
) and handle them with Promise
and Future
types, but the implementation relies on the WebSocket API.
So, you need to set up the websocket connection, handle inbound bytes, communicate outcomes to the client, and close the connection when appropriate. If you want, I can provide a proof of concept example. Let me know on twitter: @mike_critz