Any reasons not to use this for AWS SDK?
Closed this issue · 7 comments
I forked aws-sdk-lisp, updated the api definitions, and have started imparting all of my terrible opinions on it since it hasn't been updated in awhile.
One of my opinions is that it should be async by default, since most AWS api interactions are completely IO-bound. I will lose connection reuse and multi-part upload out of the gate; however, I can work on those later. Are there any reasons that you know of that I shouldn't use carrier? I noticed that you had s3 and dynamo drivers in your list of future work. This should theoretically take care of those I think?
Thanks in advance for any info you can provide!
I can't think of any reason it wouldn't work. Carrier is pretty light, so you're going to have to build all your logic on top of it (req signing and all that fun stuff) but in the end it's just verbs and headers, and carrier can do that stuff just fine.
Awesome. Thanks. I will let you know when I have it all wired up.
Just to let you know, I’ve already forked and fixed a bunch of issues with that library.
Personally, I’m a bit unwilling to use something like carrier because of the dependency on libuv: I’ve found that native dependencies overcomplicate the deployment story.
For most applications using a promise library and bordeaux-threads (maybe even with a thread pool) is probably preferable: like the JVM, we aren’t limited to a single thread of execution in Common Lisp.
Yeah, native libs can complicate things a little. Also, I'm the author of the entire cl-async stack and haven't touched lisp in like 4+ years now, so it's sort of in a "either it works or you need to fix it yourself" state. Luckily, people have been generous with pull requests and I'm fairly confident everything still works, and TBH trying to write it yourself is going to be a real pain in the ass.
In general, lisp/async don't mix super well, although libuv does make it a lot easier.
RE thread pools, I made a library a while back, pretend-event-loop, that mimics async without actually needing it (it just uses threadpools). Obviously, it's not going to be as performant as real async. If you're looking to run 10 simultaneous connections, do what @fiddlerwoaroof says and use a thread pool. If you're going to run 1000+ simultaneous connections, you might want real async (and the complexity that comes with it).
Just depends on what you're doing.
I saw that, @fiddlerwoaroof. I actually pulled a bunch of your fixes in when I started messing with the library originally. I decided to just run with it when I saw that you had PRs that had not been pulled in for years.
My thought was to not actually force any library in particular. I'd like to make it not close over the http client like it does today. Ideally people could use the right one for the job at hand, and if they don't care, it will just work how it does today. I'll leave the one it uses in there as the default, and maybe do another as an example. Every http client has a trade off, and I don't really think I want the SDK to make that decision for anyone.
Feel free to tell me I'm mistaken. I'm kinda just making it work how I wish it did.
My opinion was in my original comment on this thread has obviously changed in the last 24 hours. I thought I wanted async by default, but it started feeling off almost immediately. That's why I closed the ticket and started thinking about how I might do a cl equivalent of clojure's protocol for the http client.