mtwomey/http-client

HTTP client for Clojure and Babashka built on java.net.http

http-client

An HTTP client for Clojure and Babashka built on java.net.http.

API

See API.md.

Status

This library is in flux. Feedback is welcome. It can be used in production, but expect breaking changes. When this library is considered stable (API-wise) it will be built into babashka.

NOTE: The babashka.http-client library is built-in as of babashka version 1.1.171.

Installation

Use as a dependency in deps.edn or bb.edn:

org.babashka/http-client {:mvn/version "0.1.4"}

Rationale

Babashka has several built-in options for making HTTP requests, including:

• babashka.curl
• http-kit
• java.net.http

In addition, it allows to use several libraries to be used as a dependency:

• java-http-clj
• hato
• clj-http-lite

The built-in clients come with their own trade-offs. E.g. babashka.curl shells out to curl which on Windows requires your local curl to be updated. Http-kit buffers the entire response in memory. Using java.net.http directly can be a bit verbose.

Babashka's http-client aims to be a good default for most scripting use cases and is built on top of java.net.http and can be used as a dependency-free JVM library as well. The API is mostly compatible with babashka.curl so it can be used as a drop-in replacement. The other built-in solutions will not be removed any time soon.

Usage

The APIs in this library are mostly compatible with babashka.curl, which is in turn inspired by libraries like clj-http.

(require '[babashka.http-client :as http])
(require '[clojure.java.io :as io]) ;; optional
(require '[cheshire.core :as json]) ;; optional

GET

Simple GET request:

(http/get "https://httpstat.us/200")
;;=> {:status 200, :body "200 OK", :headers { ... }}

Headers

Passing headers:

(def resp (http/get "https://httpstat.us/200" {:headers {"Accept" "application/json"}}))
(json/parse-string (:body resp)) ;;=> {"code" 200, "description" "OK"}

Query parameters

Query parameters:

(->
  (http/get "https://postman-echo.com/get" {:query-params {"q" "clojure"}})
  :body
  (json/parse-string true)
  :args)
;;=> {:q "clojure"}

To send multiple params to the same key:

;; https://postman-echo.com/get?q=clojure&q=curl

(-> (http/get "https://postman-echo.com/get" {:query-params {:q ["clojure "curl"]}})
    :body (json/parse-string true) :args)
;;=> {:q ["clojure" "curl"]}

POST

A POST request with a :body:

(def resp (http/post "https://postman-echo.com/post" {:body "From Clojure"}))
(json/parse-string (:body resp)) ;;=> {"args" {}, "data" "From Clojure", ...}

Posting a file as a POST body:

(:status (http/post "https://postman-echo.com/post" {:body (io/file "README.md")}))
;; => 200

Posting a stream as a POST body:

(:status (http/post "https://postman-echo.com/post" {:body (io/input-stream "README.md")}))
;; => 200

Posting form params:

(:status (http/post "https://postman-echo.com/post" {:form-params {"name" "Michiel"}}))
;; => 200

Basic auth

Basic auth:

(:body (http/get "https://postman-echo.com/basic-auth" {:basic-auth ["postman" "password"]}))
;; => "{\"authenticated\":true}"

Streaming

With :as :stream:

(:body (http/get "https://github.com/babashka/babashka/raw/master/logo/icon.png"
    {:as :stream}))

will return the raw input stream.

Download binary

Download a binary file:

(io/copy
  (:body (http/get "https://github.com/babashka/babashka/raw/master/logo/icon.png"
    {:as :stream}))
  (io/file "icon.png"))
(.length (io/file "icon.png"))
;;=> 7748

To obtain an in-memory byte array you can use :as :bytes.

URI construction

Using the verbose :uri API for fine grained (and safer) URI construction:

(-> (http/request {:uri {:scheme "https"
                           :host   "httpbin.org"
                           :port   443
                           :path   "/get"
                           :query  "q=test"}})
    :body
    (json/parse-string true))
;;=>
{:args {:q "test"},
 :headers
 {:Accept "*/*",
  :Host "httpbin.org",
  :User-Agent "Java-http-client/11.0.17"
  :X-Amzn-Trace-Id
  "Root=1-5e63989e-7bd5b1dba75e951a84d61b6a"},
 :origin "46.114.35.45",
 :url "https://httpbin.org/get?q=test"}

Redirects

The default client is configured to always follow redirects. To opt out of this behaviour, construct a custom client:

(:status (http/get "https://httpstat.us/302" {:client (http/client {:follow-redirects :never})}))
;; => 302
(:status (http/get "https://httpstat.us/302" {:client (http/client {:follow-redirects :always})}))
;; => 200

Exceptions

An ExceptionInfo will be thrown for all HTTP response status codes other than #{200 201 202 203 204 205 206 207 300 301 302 303 304 307}.

user=> (http/get "https://httpstat.us/404")
Execution error (ExceptionInfo) at babashka.http-client.interceptors/fn (interceptors.clj:194).
Exceptional status code: 404

To opt out of an exception being thrown, set :throw to false.

(:status (http/get "https://httpstat.us/404" {:throw false}))
;;=> 404

Multipart

To perform a multipart request, supply :multipart with a sequence of maps with the following options:

• :name: The name of the param
• :part-name: Override for :name
• :content: The part's data. May be string or something that can be fed into clojure.java.io/input-stream
• :file-name: The part's file name. If the :content is a file, the name of the file will be used, unless :file-name is set.
• :content-type: The part's content type. By default, if :content is a string it will be text/plain; charset=UTF-8; if :content is a file it will attempt to guess the best content type or fallback to application/octet-stream.

An example request:

(http/post "https://postman-echo.com/post"
           {:multipart [{:name "title" :content "My Title"}
                        {:name "Content/type" :content "image/jpeg"}
                        {:name "file" :content (io/file "foo.jpg") :file-name "foobar.jpg"}]})

Compression

To accept gzipped or zipped responses, use:

(http/get "https://api.stackexchange.com/2.2/sites"
  {:headers {"Accept-Encoding" ["gzip" "deflate"]}})

The above server only serves compressed responses, so if you remove the header, the request will fail. Accepting compressed responses may become the default in a later version of this library.

Interceptors

Babashka http-client interceptors are similar to Pedestal interceptors. They are maps of :name (a string), :request (a function), :response (a function). An example is shown in this test:

(deftest interceptor-test
  (let [json-interceptor
        {:name ::json
         :description
         "A request with `:as :json` will automatically get the
         \"application/json\" accept header and the response is decoded as JSON."
         :request (fn [request]
                    (if (= :json (:as request))
                      (-> (assoc-in request [:headers :accept] "application/json")
                          ;; Read body as :string
                          ;; Mark request as amenable to json decoding
                          (assoc :as :string ::json true))
                      request))
         :response (fn [response]
                     (if (get-in response [:request ::json])
                       (update response :body #(json/parse-string % true))
                       response))}
        ;; Add json interceptor add beginning of chain
        ;; It will be the first to see the request and the last to see the response
        interceptors (cons json-interceptor interceptors/default-interceptors)
        ]
    (testing "interceptors on request"
      (let [resp (http/get "https://httpstat.us/200"
                             {:interceptors interceptors
                              :as :json})]
        (is (= 200 (-> resp :body
                       ;; response as JSON
                       :code)))))))

A :request function is executed when the request is built and the :response function is executed on the response. Default interceptors are in babashka.http-client.interceptors/default-interceptors. Interceptors can be configured on the level of requests by passing a modified :interceptors chain.

Async

To execute request asynchronously, use :async true. The response will be a CompletableFuture with the response map.

(-> (http/get "https://clojure.org" {:async true}) deref :status)
;;=> 200

Timeouts

Two different timeouts can be set:

• The connection timeout, :connect-timeout, in http/client
• The request :timeout in http/request

Alternatively you can use :async + deref with a timeout + default value:

(let [resp (http/get "https://httpstat.us/200?sleep=5000" {:async true})] (deref resp 1000 ::too-late))
;;=> :user/too-late

Test

$ bb test:clj
$ bb test:bb

Credits

This library has borrowed liberally from java-http-clj and hato, both available under the MIT license.

License

Copyright © 2022 - 2023 Michiel Borkent

Distributed under the MIT License. See LICENSE.