ExPool

A generic pooling library for Elixir.

Documentation for ExPool is available online.

Installation

Add ExPool to your list of dependencies in mix.exs:

def deps do
  [{:ex_pool, "~> 0.1.1"}]
end

Usage

ExPool uses a set of initialized processes kept ready to use rather than spawning and destroying them on demand.

When you run a function on the pool:

It requests a process from the pool.
Runs the function with the pid as only argument.
Returns the process to the pool.

If there are no processes available it blocks until a process is returned to the pool, and then runs the function.

The worker

The worker is a module that fits into a supervision tree (for example, a GenServer).

It is the process the pool will initialize and keep ready to use.

The following snippet shows an example of a worker that uses :timer.sleep\1 to simulate a long-lasting operation (like a CPU intensive task, an external http request or a database query).

defmodule HardWorker do
  use GenServer

  def start_link(_opts \\ []) do
    GenServer.start_link(__MODULE__, :ok, [])
  end

  def do_work(pid, milliseconds \\ 2000) do
    GenServer.call(pid, milliseconds)
  end

  def handle_call(milliseconds, _from, state) do
    :timer.sleep(milliseconds)
    IO.puts "Work done!"

    {:reply, :ok, state}
  end
end

Is recommended to always use blocking calls when using a worker (i.e. GenServer.call/2 instead of GenServer.cast/2).

If a non-blocking call is used ExPool will return the process to the pool and make it available for other requests even though it may be performing work.

The pool

You can start a pool with ExPool.start_link/1.

In the following example we create a pool and run a function on it.

{:ok, pool} = ExPool.start_link(worker_mod: HardWorker)

ExPool.run pool, fn (worker) ->
  HardWorker.do_work(worker, 1000)
end

# It will print:
#   Work done!

We have created a pool that spawns a set of workers (5 by default) and we have run a function on the pool.

If we run concurrently more functions on the pool than workers available the functions that overflow the number of workers will block until there is a worker available.

{:ok, pool} = ExPool.start_link(worker_mod: HardWorker, size: 2)

for _i <- 1..5 do
  spawn_link fn ->
    ExPool.run pool, &HardWorker.do_work(&1)
  end
end

# It will print:
#   Work done!
#   Work done!
#   Work done!
#   Work done!
#   Work done!

Using a pool on a supervision tree

To start a pool that will be supervised by your application add to your application supervisor (or any other supervisor of your choice) the following child.

defmodule MyApplication do
  use Application

  def start(_type, _args) do
    import Supervisor.Spec, warn: false

    children = [
      worker(ExPool, [[worker_mod: HardWorker, size: 10, name: :my_pool]])

      # ... more children
    ]

    opts = [strategy: :one_for_one, name: Transactions.Endpoint.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

The pool will be started with your application and can be used as follows.

ExPool.run :my_pool, fn (worker) ->
  HardWorker.do_work(worker)
end

Examples

A pool of redis connections

The following example will show how to use ExPool to keep a pool of redis connections on your application. We will use ExRedis to establish a redis connection and run commands.

First we add ExPool and ExRedis as dependencies of the application in our mix.exs.

  defp deps do
    [{:ex_pool, "~> 0.1.1"},
     {:exredis, ">= 0.2.2"}]
  end

Run mix deps.get to get the dependencies from Hex.

Add to our config/config.exs some configuration about the pool and the redis server.

config :redis_pool,
  worker_mod: ExRedis,
  size: 10,
  name: :redis

config :ex_redis,
  host: "127.0.0.1",
  port: 6379,
  password: "",
  db: 0

As ExRedis fits into a supervision tree there is no need to explicitly define a worker.

We have configured a pool with 10 redis connections named :redis. To start the pool when the application starts add it as a child of your application supervisor.

defmodule MyApplication do
  use Application

  @redis_pool_config Application.get_env(:redis_pool)

  def start(_type, _args) do
    import Supervisor.Spec, warn: false

    children = [
      worker(ExPool, [@redis_pool_config])
    ]

    opts = [strategy: :one_for_one, name: Transactions.Endpoint.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

Now you can run commands on redis from your application.

ExPool.run :redis, fn (client) ->
  client |> Exredis.query ["SET", "foo", "bar"]
end

ExPool.run :redis, fn (client) ->
  client |> Exredis.query ["GET", "foo"]
end
# => "bar"

License

ExPool source code is released under Apache 2 License. Check LICENSE file for more information.

jcabotc/ex_pool