A lightweight image processing module in Elixir using CImg, aimed at creating auxiliary routines for Deep Learning.
Note: Only a few image processing functions are available yet.
Image data handled by CImg is placed on the NIF side. On the Elixir side, image processing is performed through
RESOURCE
of images - generated by NIF - stored in the %CImg{} structure. You cannot directly read or process pixels
in an image. Instead, you must use the image processing functions provided by the CImg module.
Images are automatically subject to garbage collection when they are no longer used. The NIFs RESOURCE
management
mechanism is used to achieve this functionality.
Each function provided by CImg can be used alone or as a set of functions. In the latter case, a seed image is first
prepared using CImg.builder/1
or similar, and then a series of functions are put together using Elixir's pipe syntax.
I have confirmed it works in the following OS environment.
- Windows MSYS2/MinGW64
- WSL2/Ubuntu 20.04, 18.04
- Linux Mint 20 "Ulyana"
- Nerves rpi
python3 is required to build this module.
In addition, the following libraries are required to display images on a PC screen.
- GDI32 on Windows
- X11 on Linux
Add following dependency to your mix.exs
.
def deps do
[
{:cimg, "~> 0.1.19"}
]
end
and install dependencies:
$ mix deps.get
$ mix deps.compile
A sample program is in the demo directory. You can run it by following the steps below.
$ cd demo
$ mix deps.get
$ mix run -e "CImgDemo.demo1"
Closing the application window will stop the demo program.
Another demo:
$ mix run -e "CImgDemo.demo2"
$ mix run -e "CImgDemo.demo3"
The following code is an excerpt of the inference portion from the DeepLab app implemented using TflInterp. CImg preprocesses the input image and passes it through the DeepLab model. And then, the tensor output by DeepLab is transformed by CImg into a colormaedp image.
defmodule TflDemo.DeepLab3 do
alias TflDemo.DeepLab3.Prediction
def apply_deeplab3(img_file) do
img = CImg.load(img_file)
segments = Prediction.apply(img)
end
end
defmodule TflDemo.DeepLab3.Prediction do
use TflInterp, model: "priv/lite-model_deeplabv3_1_metadata_2.tflite"
@deeplab3_shape {257, 257}
def apply(img) do
# preprocess
bin = img
|> CImg.resize(@deeplab3_shape)
|> CImg.to_binary(range: {-1.0, 1.0})
# prediction
outputs =
__MODULE__
|> TflInterp.set_input_tensor(0, bin)
|> TflInterp.invoke()
|> TflInterp.get_output_tensor(0)
|> Nx.from_binary({:f, 32}) |> Nx.reshape({257, 257, :auto})
# postprocess
outputs
|> Nx.argmax(axis: 2)
|> Nx.as_type({:u, 8})
|> Nx.to_binary()
|> CImg.from_binary(257, 257, 1, 1, "<u1")
|> CImg.color_mapping(:lines) # convert into a colormaped image
end
end
cimg_ex is licensed under the Apache License Version 2.0.
- The "CImg" Library is licensed under the CeCILL-C/CeCILL.
- The "stb" - single-file public domain libraries for C/C++ - is public domain or MIT licensed.