/possumwood

Possumwood is a graph-based procedural authoring tool, in concept not dissimilar to popular CG packages like Houdini, Blender or Maya. It is intended to serve as a sandbox for computer graphics algorithms and libraries, providing a user-friendly and coding-free UI for libraries that would otherwise be inaccessible for an average user.

Primary LanguageC++MIT LicenseMIT

Synopsis

Possumwood UI

Possumwood (named after Sandbox Tree) is a graph-based procedural sandbox, implementing concepts of graph-based visual programming in a simple interface. It is intended to provide a user-friendly and accessible UI for experimenting with common computer graphics algorithms and libraries (e.g., OpenGL, CGAL, WildMagic, OpenCV).

Possumwood is built on top of a simple graph-evaluation engine and a Qt-based node graph editor, with an OpenGL viewport. Its main strength is its extensibility - it is trivial to implement new plugins supporting new libraries and data types, which allows free and user-friendly experimentation with parameters of each algorithm.

Possumwood is a sandbox, not a production tool. As a sandbox, it is quite open to radical changes and redesign, and for the time being it does not guarantee any form of backwards compatibility.

Build Status Snap Status Codacy Badge

Table of contents

Example setups

Possumwood comes with a number of example setups that can serve as a starting point for experimentation.

Opengl

             
             

A simple OpenGL demo

A simple demo showing how to load an .obj file (positions, normals and topology) and display it using OpenGL.

Named vertex attributes p and n are loaded from an .obj file, and passed through to an OpenGL vertex/fragment shader setup. In later demos, this setup is wrapped-up in a subnetwork.

             
             
             
             

Automatic normals

When an mesh file doesn't include explicit normals, it is relatively easy to use a fragment shader to "autogenerate" normals using screen-space derivatives dFdx() and dFdy(). This demo shows how to do that in Possumwood - just have a look at fragment_shared source.

             
             
             
             

Turntable demo

A simple shader setup passing a time value (i.e., the timeline value) as a uniform into the shader (together with viewport parameters). This value is then converted in the vertex_shader to a transformation matrix used to rotate a model.

             
             
             
             

Wireframe using a geometry shader

Wireframe mode is one of the rendering modes of OpenGL. A similar but more controllable effect can be achieved by processing a model using a geometry shader, which allows to convert primitives to a different number of primitives of the same or different type.

This setup shows how to emit a line for each polygon edge of the input geometry using a program in geometry_shader. It also shows how to adjust the line width to screen resolution, maintaining the width independent to view parameters.

             
             
             
             

Mesh subdivision in a geometry shader

While modern GPUs contain a bespoke functionality for subdivision, this simple demo shows how a similar (if limited) effect can be achieved using a simple geometry shader.

This demo implements a simple interpolative sudvision scheme with normal-based displacement. All subdivision computation is done in a geometry shader, with additional vertices and topology emitted by calling a recusive function. This approach is just a toy example, as it is strictly limited by the capabilities of individual GPUs - in practical applications, bespoke tesselation shaders perform much better.

             
             
             
             

PN Triangles in a geometry shader

Curved PN Triangles is a simple geometry subdivision scheme, replacing all triangles with a bezier patch directly on the GPU. The shape interpolation uses a cubic spline, and normals are interpolated using a quadratic spline to maintain continuity.

This implementation uses a geometry shader for emitting additional polygons.

Vlachos, Alex, et al. "Curved PN triangles." Proceedings of the 2001 symposium on Interactive 3D graphics. ACM, 2001.

             
             
             
             

Infinite ground plane

A simple way of generating procedural graphics using GLSL employs rendering a single polygon covering the whole frame, and a per-pixel fragment shader (with additional inputs) to compute the final colour. A number of examples of this approach can be found at ShaderToy.

This demo is a simple example of such approach - based on a ray generated from a near and far plane point corresponding to a screen pixel, it computes the position of ray's intersection with the ground plane (y==0), and uses this information to synthesize an "infinite" ground plane covered in checkerboard, with a "spotlight". The fragment shader does all the computation - from ray intersection, through colour and checkerboard generation, to depth-value computation to allow the ground plane to composit correctly with other objects in a scene.

             
             
             
             

Shadertoy

ShaderToy is a community of artists and programmers centered around generating interesting graphics content in GLSL using a single fragment shader.

One of the common techniques to generate procedural 3D content in a fragment shader is Raymarching - a simple method of tracing rays against shapes represented as Signed Distance Functions, and generating 3D content based on them.

This demo is a port of a ShaderToy demo by Indigo Quilezles, showing different SDFs and a few additional effects that can be achieved by raymarching. A comprehensive raymarching tutorial can be found at the RaymarchingWorkshop.

             
             
             
             

Skybox

A classical way of generating sky and background in games is to render a cube or spherical map on the far plane (behind all other objects). This method is called CubeMap, SkyBox or SkyDome.

This simple demo shows this technique on a spherical texture from HDRI Haven.

             
             
             
             

Reflection mapping

Reflection mapping is a simple image-based lighting technique for simulating purely reflective materials. It only behaves correcty for convex shapes, but even with this strict limitation, it has been long established as a simple method for reflective surfaces in computer graphics.

This demo combines a skybox with a "purely reflective" material.

             
             
             
             

Gourand shading

Gourand shading is one of the simplest method of polygonal mesh shading. It computes colour by linearly interpolating vertex colours in screen space, making it cheap but less physically accurate than Phong or physics-based shading methods.

This demo combines Gourand shading with Phong reflection model, reproducing an old "fixed-pipeline" shading model of OpenGL.

             
             
             
             

Phong shading

Phong shading computes lighting of a surface by interpolating per-pixel surface normal. Compared to Gourand shading, this method leads to smoother and more realistic surface looks better describing the shape of the object, and significantly more accurate high-frequency components computation (i.e., specular component in the Phong lighting model).

This demo implements Phong shading with a Phong reflection model and a single point light source. The setup is wrapped in a subnetwork - just double-click the blue node to "enter" it.

             
             
             
             

Material capture shader

MatCap or LitSphere is a simple technique for representing the properties of both the material and the environment light in a single texture. By "baking in" the two dimensions of BRDF representing incoming light, MatCap determines the final colour solely by using the difference vector between the view ray and the surface normal.

This simple approach can yield surprisingly realistic results, and has been used in games, scultping software and for capturing and representing artistic styles.

Sloan, Peter-Pike J., et al. "The lit sphere: A model for capturing NPR shading from art." Graphics interface. Vol. 2001. 2001.

             
             
             
             

Normal mapping

Normal mapping is a simple method for adding texture-based geometry details to a coarse model. It uses a texture to represent normal perturbation that would result from a significantly more detail in the surface mesh, without having to represent that detail explicitly in the underlying geometry. In games, this technique is often used to represent a complex model with all its detail, without having to explicitly store a heavy geometry.

This simple demo demonstrates a combination of normal mapping implemented in a fragment shader, together with displacement mapping implemented as a geometry shader. The lighting is implemented as a simple single light-source Phong reflection model.

Blinn, James F. "Simulation of wrinkled surfaces." ACM SIGGRAPH computer graphics. Vol. 12. No. 3. ACM, 1978. Mikkelsen, Morten. "Simulation of wrinkled surfaces revisited." (2008).

             
             
             
             

SDF-based text rendering

Rendering text in OpenGL is usually implemented using a simple texture. However, such text "pixelates" when viewed up-close. A simple method to improve on its appearance uses a Signed distance function to represent letter boundaries - by exploiting the interpolation abilities of GPU hardware, it is possible to represent a text boundary with significantly more precision by using the directional information represented in SDF texture differential.

This demo shows a simple implementation of this technique.

Green, Chris. "Improved alpha-tested magnification for vector textures and special effects." ACM SIGGRAPH 2007 courses. ACM, 2007.

             
             

Polymesh

             
             

OBJ file loading

This simple demo shows how to load an object from .obj file, and display it in the viewport.

The display code is contained in a subnetwork (double click the blue node to "enter" it), and is based on a trivial implementation of a vertex and fragment OpenGL shader.

             
             
             
             

CGAL normals generation

To display a polygonal mesh on a GPU, the model needs to contain surface-normal information for shading. While this information can be auto-generated, an explicitly represented normal can improve surface properties like smoothness or shape details.

This simple demo shows two methods of computing normals using the CGAL library.

             
             
             
             

CGAL mesh decimation

Mesh simplification (decimation) is a common operation in mesh processing - it aims to reduce the mesh complexity without removing its details, reducing the memory footprint without compromising the quality of the resulting model.

This demo shows how to use CGAL's implementation of mesh decimation in Possumwood, allowing to experiment with its various parameters.

             
             

Image

             
             

OpenCV image loading and display

This simple demo shows how to load an image using OpenCV's imread() method, and pass the resulting image data to OpenGL as a texture and a set of additional metadata represented by GLSL uniforms.

In following demos, this OpenGL setup is wrapped in a subnetwork.

             
             
             
             

Image loading

A simple demo showing how to load an image from a file, and display it using an OpenGL setup (wrapped in a subnetwork - enter it by double-clicking the blue node).

             
             
             
             

Fullscreen image display

While Possumwood's viewport is primarily intended for viewing 3D content, using a set of shaders it is possible to alter it to display 2D textures as well.

This demo uses shaders that ignore the perspective projection matrix and draw 2D image in a 2D front-facing view in the viewport.

             
             
             
             

Grayscale conversion

Color conversion in OpenCV is implemented in the cvtColor() function. This demo shows how to use the corresponding color node to first convert to grayscale, and then back to BGR to allow the image to be passed to OpenGL as RGB data.

             
             

Lua

             
             

Lua addition node

This simple demo shows how to create a subnetwork performing simple addition (and exposing input parameters) with a simple Lua script.

Double click the node too see its content.

             
             
             
             

Lua multiplication node

This simple demo shows how to create a subnetwork performing simple multiplication (and exposing input parameters) with a simple Lua script.

Double click the node too see its content.

             
             
             
             

Animated cube

A simple demo showing how to use a Lua script to create animated data, which are then fed as an input to a vector input of the cube node, animating its position.

             
             
             
             

Lua-based image synthesis

Possumwood contains a simple integration of the Lua scripting language, allowing to manipulate in-scene objects using code contained in nodes.

This demo shows how to synthesize an image programatically. It "injects" the images plugin into the input Lua context (i.e., the environment for running a script), and generates an image, which is then extracted from the output state using an image extract node. The result is then passed on an image-display subnetwork.

             
             
             
             

Lua expression-based image synthesis

This demo builds on the Lua Grid setup, extending it by additional parameters, and wrapping it in a subnetwork (double click any blue nodes to "enter" them).

             
             
             
             

Lua image tonemapping

Lua expressions in Possumwood can be also used to manipulate HDR images.

This demo shows how to implement simple Gamma compression tonemapping operator in Lua, and how to wrap it into a subnetwork with exposed parameters. Possumwood allows arbitrary nesting of nodes in this way, allowing to abstract-away any unnecessary complexity into simple nodes with clean interfaces.

             
             

Animation

             
             

Animation loading

This demo shows how to load animation data from a .bvh file (a common file used for this kind of data), and from a combination of .asf and amc files (another common file format, used for the CMU Mocap Library).

The loaded animation data are then processed (scaled, translated), and displayed in OpenGL using a simple skeleton render node.

             
             
             
             

GLSL skeleton

This demo uses GLSL (geometry shader) to generate a shaded polygonal stick figure from a set of line segments.

             
             
             
             

Periodic animation

A motion capture data source captures reality, and due to both capturing noise and imperfections, a locomotion or another periodic motion will never be perfectly periodic.

This demo uses a simple hierarchical linear blend to convert motion capture data to a periodic animation. To allow user to understand the input data well, the user interface uses a simple display of per-frame comparison error metric, similar to Kovar et al.'s Motion Graphs.

Kovar, Lucas, Michael Gleicher, and Frédéric Pighin. "Motion graphs." ACM SIGGRAPH 2008 classes. ACM, 2008.

             
             
             
             

Animation transition

A linear blend is the simplest way to transition between two animations. This demo shows how a hand-picked linear blend can be used in Possumwood to transition between two animations. To assist the user with picking the blending point, the editor window displays a visualisation of frame similarity metric, similar to Kovar et al.'s Motion Graphs.

Kovar, Lucas, Michael Gleicher, and Frédéric Pighin. "Motion graphs." ACM SIGGRAPH 2008 classes. ACM, 2008.

             
             
             
             

Motion graph random walk

A motion graph is a simple directional graph data structure describing possible transition points between a number of motion clips. Each transition is determined by comparing kinematic and dynamic properties of corresponding frames, which should lead to a transition that does not introduce any salient discontinuities or artifacts.

A random walk through such a graph would generate a variation of the original motion(s) by adding random transitions.

This demo shows a random walk through a motion graph. In its current state, it does not attempt to synthesize smooth continuous transitions - it just switches between different clips. (todo: introduce smooth transitions)

Kovar, Lucas, Michael Gleicher, and Frédéric Pighin. "Motion graphs." ACM Transactions on Graphics (TOG). Vol. 21. No. 3. ACM, 2002.

Kovar, Lucas, Michael Gleicher, and Frédéric Pighin. "Motion graphs." ACM SIGGRAPH 2008 classes. ACM, 2008.

             
             
             
             

Footstep detection

Animations represented as sequences of poses of hierarchical skeletons do not naturally represent information about character interaction with the environment. This information is necessary for motion editing to maintain desirable properties, such as realistic stationary footsteps. A number of methods has been used to perform this task, but a simple velocity thresholding has proven sufficient for most practical applications.

This demo shows how this simple technique can be used in Possumwood. The resulting constraint information can be further processed via median filtering, to remove any noise introduced in the detection process.

Kovar, Lucas, John Schreiner, and Michael Gleicher. "Footskate cleanup for motion capture editing." Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation. ACM, 2002.

Ikemoto, Leslie, Okan Arikan, and David Forsyth. "Knowing when to put your foot down." Proceedings of the 2006 symposium on Interactive 3D graphics and games. ACM, 2006.

             
             

Opencv

             
             

Face detection using Haar cascades

This demo follows the OpenCV's Haar Cascade tutorial - it allows to load a Haar Cascade .xml file in OpenCV's format, use it to detect faces in the input image, and then draw the result into the source image for display.

Lienhart, Rainer, and Jochen Maydt. "An extended set of haar-like features for rapid object detection." Proceedings. international conference on image processing. Vol. 1. IEEE, 2002.

             
             
             
             

Grayscale image denoising

This tutorial closely follows the OpenCV's denoising tutorial for grayscale images, removing the noise using the non-local means algorithm.

             
             
             
             

Color image denoising

This tutorial closely follows the OpenCV's denoising tutorial for color images, removing the noise using the non-local means algorithm.

             
             
             
             

Visualising luminance noise

This tutorial follows the OpenCV's denoising tutorial for grayscale images, but at the end of the processing it subtracts the denoised image from the original (leaving only extracted noise), and equalises the result using cv::equalizeHist() to show the noise structure.

             
             
             
             

Visualising color noise

This tutorial follows the OpenCV's denoising tutorial for color images, but at the end of the processing it subtracts the denoised image from the original (leaving only extracted noise), and equalises the result using cv::equalizeHist() to show the noise structure.

             
             
             
             

Discrete Fourier Transform and a simple low-pass filter

The Discrete Fourier Transform (implemented in OpenCV) converts a signal (image) into its complex frequency spectrum. By manipulating the spectrum, it is possible to implement simple frequency filters.

This demo shows how to perform a DFT on an image, and implements a simple frequency cutoff low-pass filter, showing both the image before and after applying this filter.

             
             
             
             

Video frames

OpenCV's VideoCapture object can be used to extract frames from a video file. This demo shows how to use a video_frame source together with a frame input to link the viewport timeline with a displayed video frame.

             
             
             
             

Dense optical flow

OpenCV provides a method to compute dense optical flow in the calcOpticalFlowFarneback() function, allowing to detect moving objects in frames without explicitly detecting feature points first.

This demo shows how to set up this type of optical flow in Possumwood, and how to display its results as colours on top of a grayscale animated image.

Farnebäck, Gunnar. "Two-frame motion estimation based on polynomial expansion." Scandinavian conference on Image analysis. Springer, Berlin, Heidelberg, 2003.

             
             
             
             

Calibration pattern detection (circles)

Pattern detection is the first step of camera calibration for many computer vision algorithms. This demo shows how a pattern build of circles can be detected using Hough transformation, and assembled into a grid for the use of camera intrinsic and extrinsic calibration.

             
             
             
             

Calibration pattern detection (chessboard)

Pattern detection is the first step of camera calibration for many computer vision algorithms. This demo shows how a chessboard pattern can be detected using corners feature detection, and assembled into a grid for the use of camera intrinsic and extrinsic calibration.

             
             
             
             

Camera calibration

The calibration of a camera's intrinsic parameters is an important step for most algorithms in machine vision.

This demo shows how to perform multi-image intrinsic camera calibration using a chessboard pattern, and how to use the solved parameters to undistort an image. This functionality is fully built using OpenCV's functions, wrapped in a set of Possumwood nodes.

The left side of the graph detects a pattern in a set of 4 images, accumulating the results in a points array; the calibrate_camera node computes the intrinsic calibration from a set of input sample points; the undistort node then uses the resulting calibration to correct the distortion of an input image.

             
             
             
             

Image inpainting using Fast Marching

Image inpainting is an image reconstruction technique, allowing to fill-in missing parts of an image based on existing pixels.

Telea's algorithm shown in this demo uses a Fast Marching method to inpaint pixels from a boundary based on surrounding pixels and boundary's normal direction.

Telea, Alexandru. "An image inpainting technique based on the fast marching method." Journal of graphics tools 9.1 (2004): 23-34.

             
             
             
             

Image inpainting using Fast Marching

Image inpainting is an image reconstruction technique, allowing to fill-in missing parts of an image based on existing pixels.

The Navier-Stokes-based algorithm shown in this demo uses a heuristic based on fluid dynamics to inpaint a large region of dark pixels in an image, obtained by image thresholding.

Bertalmio, Marcelo, Andrea L. Bertozzi, and Guillermo Sapiro. "Navier-stokes, fluid dynamics, and image and video inpainting." Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001. Vol. 1. IEEE, 2001.

             
             
             
             

Inpainting for image restoration

Image inpainting can be also used to attempt to restore an image where a large portion of the original data is missing.

This setup shows how an image can be partially restored using inpainting from a very sparse set of sample points.

             
             
             
             

Inpainting using the smoothness prior

A slightly different take on image inpainting can use the smoothness prior to try to reconstruct missing pixels in a single linear global solve step.

Cohen-Or, Daniel, et al. A sampler of useful computational tools for applied geometry, computer graphics, and image processing. AK Peters/CRC Press, 2015.

             
             
             
             

Image reconstruction using the smoothness prior

The smoothness prior can be also used to try to reconstruct missing pixels in a sparse set of samples using a single linear global solve step.

Cohen-Or, Daniel, et al. A sampler of useful computational tools for applied geometry, computer graphics, and image processing. AK Peters/CRC Press, 2015.

             
             

Hdr

             
             

HDR merge using Mertens algorithm

Multi-exposure merge is an procedure in high dynamic range imaging.

This setup merges 5 hand-held exposures of a scene to an HDR image using Mertens algorithm. As the inputs are not perfectly aligned, the output shows "ghosting" artifacts, as expected in this type of setup.

Mertens, Tom, Jan Kautz, and Frank Van Reeth. "Exposure fusion." 15th Pacific Conference on Computer Graphics and Applications (PG'07). IEEE, 2007.

             
             
             
             

Aligned HDR merge using Mertens algorithm

For a HDR merge algorithm to correctly process its inputs, the input images need to be precisely aligned. This can be either achived by careful capture, via a calibration and alignment step, or by a simpler image-alignment algorithm.

This demo shows how a simple alignment of hand-held multiple exposures can be achieved using the MTB algorithm, without the need of complex calibration. The resulting images are then merged to a HDR image using Mertens algorithm, completely eliminating the ghosting artifacts from the unaligned version of this demo.

Mertens, Tom, Jan Kautz, and Frank Van Reeth. "Exposure fusion." 15th Pacific Conference on Computer Graphics and Applications (PG'07). IEEE, 2007.

Ward, Greg. "Fast, robust image registration for compositing high dynamic range photographs from hand-held exposures." Journal of graphics tools 8.2 (2003): 17-30.

             
             
             
             

Aligned HDR merge using Debevec algorithm with response curve estimation

When merging multiple exposures for HDR imaging, it is often beneficial to use raw image data, avoiding any processing and gamma correction a camera might do to produce a perceptually good looking result. In some cases, when using cheaper cameras or a mobile phone, it might not be possible to use raw data, which means that an additional calibration step is required.

This setup shows how to use EXIF data from a set of JPG images to estimate the camera response curve using Debevec's algorithm, and then use the resulting curve for a more accurate HDR merge using Debevec's HDR merge algorithm. Compared to previous demos using Merten's algorithm, the dynamic range of the resulting image is much wider, with a more accurate representation of luminance information present in the scene.

Debevec, Paul E., and Jitendra Malik. "Recovering high dynamic range radiance maps from photographs." ACM SIGGRAPH 2008 classes. ACM, 2008.

             
             
             
             

Drago tonemapping

Tonemapping is the process of converting High Dynamic Range content to a standard display. The simplest operators are global (such as Gamma correction, while others act adaptively, mimicking the behaviour of human visual system.

This demo shows how to set up Drago's tonemapping algorithm in Possumwood.

Drago, Frédéric, et al. "Adaptive logarithmic mapping for displaying high contrast scenes." Computer graphics forum. Vol. 22. No. 3. Oxford, UK: Blackwell Publishing, Inc, 2003.

             
             
             
             

Reinhard's tonemapping

Tonemapping is the process of converting High Dynamic Range content to a standard display. The simplest operators are global (such as Gamma correction, while others act adaptively, mimicking the behaviour of human visual system.

This demo shows how to set up Reinhards's tonemapping algorithm in Possumwood.

Reinhard, Erik, and Kate Devlin. "Dynamic range reduction inspired by photoreceptor physiology." IEEE transactions on visualization and computer graphics 11.1 (2005): 13-24.

             
             
             
             

Mantiuk tonemapping

Tonemapping is the process of converting High Dynamic Range content to a standard display. The simplest operators are global (such as Gamma correction, while others act adaptively, mimicking the behaviour of human visual system.

This demo shows how to set up Mantiuk's tonemapping algorithm in Possumwood.

Mantiuk, Rafal, Karol Myszkowski, and Hans-Peter Seidel. "A perceptual framework for contrast processing of high dynamic range images." ACM Transactions on Applied Perception (TAP) 3.3 (2006): 286-308.

             
             

Lightfields

             
             

Lytro RAW lightfields reader

Lytro cameras output a proprietary raw image format, which has never been officially documented. However, it has been reverse-engineered, and the results were documented in detail by Jan KuÄŤera.

This demo implements a reader for Lytro RAW files, extracting both image data and metadata of each image.

KuÄŤera, Jan. Lytro Meltdown - the file format.

Kučera, Jan. "Výpočetní fotografie ve světelném poli a aplikace na panoramatické snímky." (2014).

Georgiev, Todor, et al. "Lytro camera technology: theory, algorithms, performance analysis." Multimedia Content and Mobile Devices. Vol. 8667. International Society for Optics and Photonics, 2013.

             
             
             
             

Lytro light field normalisation

Calibration of lightfield cameras is a complex process. One of the main issue facing microlens-based cameras is vignetting, caused both by the main lens, and by each of the microlenses.

A naive approach to address vignetting is by simply dividing the image pixels by the values of a white diffuse image captured with the same settings. While this approach addresses vignetting sufficiently, it doesn't take into account the camera response curve, leading to some artifacts still being present in the final image.

Georgiev, Todor, et al. "Lytro camera technology: theory, algorithms, performance analysis." Multimedia Content and Mobile Devices. Vol. 8667. International Society for Optics and Photonics, 2013.

             
             
             
             

Lytro lightfields debayern

The sensor of Lytro cameras are conventional Bayer-pattern color filters. In standard photography, one of the first steps of image processing involves the interpolation of values of neighbouring pixels to remove the Bayer pattern.

This demo shows how to obtain a debayered normalised lightfield in Possumwood.

Kucera, J. "Computational photography of light-field camera and application to panoramic photography." Institute of Information Theory and Automation (2014).

             
             
             
             

Lytro microlens pattern reconstruction

Reconstruction of the microlens pattern is essential for recovering the lightfield data from a Lytro image.

This demo shows how the information embedded in the metadata of a Lytro raw image can be used to reconstruct the microlens array, and visualise the result.

Georgiev, Todor, et al. "Lytro camera technology: theory, algorithms, performance analysis." Multimedia Content and Mobile Devices. Vol. 8667. International Society for Optics and Photonics, 2013.

             
             
             
             

Epipolar Images

Epipolar images show slices of the 4D lightfield along different axis.

The x-y image (center) shows a slice corresponding to a single dept-of-field image. By changing the UV offset or interval, it is possible to create images with different depth of field or focal plane.

The u-x and v-y images (left, right, top and bottom) show slices with "line direction" (based on incoming direction of rays) corresponding to individual depth planes.

Ng, Ren, et al. "Light field photography with a hand-held plenoptic camera." Computer Science Technical Report CSTR 2.11 (2005): 1-11.

             
             
             
             

UV EPIs

The epipolar images accumulated in UV space show images representing averages of lenslets in different part of the image. This provides a good measure for the quality of Lytro lenslet pattern detection and main lens calibration - if correct, all lenslets should look very similar.

Ng, Ren, et al. "Light field photography with a hand-held plenoptic camera." Computer Science Technical Report CSTR 2.11 (2005): 1-11.

             
             
             
             

Sub-aperture images from Lytro lightfields data

Lightfield data contain enough information to allow for reconstruction of multiple narrow-base sub-apreture images, each with slightly different "point of view".

This demo shows how to use demultiplexing to reconstruct multiple views from raw Lytro data.

Sabater, Neus, et al. "Accurate disparity estimation for plenoptic images." European Conference on Computer Vision. Springer, Cham, 2014.

             
             
             
             

Normalized sub-aperture images from Lytro lightfields data

Lightfield data contain enough information to allow for reconstruction of multiple narrow-base sub-apreture images, each with slightly different "point of view".

This demo shows how to use demultiplexing to reconstruct multiple views from normalized raw Lytro data. Normalization is performed by simple division of the image data by white diffuse image values.

Sabater, Neus, et al. "Accurate disparity estimation for plenoptic images." European Conference on Computer Vision. Springer, Cham, 2014.

             
             
             
             

Sub-apreture mosaic image sample counts

Demultiplexing a Bayern image leads to a significantly larger number of samples for green pixels, compared to red and blue values.

This demo addempts to visualise this difference explicitly, by showing both each channel of the demultiplexed image, and the number of samples used to reconstruct each of its pixels.

             
             
             
             

Sub-apreture image inpainting

To reconstruct missing data after obtaining sub-apreture images, this demo uses a simple inpainting technique from OpenCV to reconstruct missing data in each image separately.

Sabater, Neus, et al. "Accurate disparity estimation for plenoptic images." European Conference on Computer Vision. Springer, Cham, 2014.

Bertalmio, Marcelo, Andrea L. Bertozzi, and Guillermo Sapiro. "Navier-stokes, fluid dynamics, and image and video inpainting." Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001. Vol. 1. IEEE, 2001.

             
             
             
             

Sub-apreture image inpainting using smoothness prior

To reconstruct missing data after obtaining sub-apreture images, this demo uses a simple Laplacian-based inpainting technique to reconstruct missing data in each image separately.

Sabater, Neus, et al. "Accurate disparity estimation for plenoptic images." European Conference on Computer Vision. Springer, Cham, 2014.

Cohen-Or, Daniel, et al. A sampler of useful computational tools for applied geometry, computer graphics, and image processing. AK Peters/CRC Press, 2015.

             
             
             
             

Mosaic-based lightfields superresolution

Reconstructed mosaic data can be used to reconstruct a high-resolution image using a simple multiplexing algorithm.

This demo shows the result of this algorithm on an inpainted mosaic from previous demos.

             
             
             
             

Naive refocusing

The simplest method of lightfields digital refocusing is to directly use the lightfield's equation to determine the target pixel position for each sample, given its position in the lightfield image and its corresponding u and v values.

While very simple and fast, the number of samples for each pixel can vary depending on the bayern pattern and focal distance, often leading to noisy and/or low-resolution results with artifacts.

Ng, Ren, et al. "Light field photography with a hand-held plenoptic camera." Computer Science Technical Report CSTR 2.11 (2005): 1-11.

             
             
             
             

Naive refocusing detail

This demo shows the detail of the sampling and integration pattern of previous demo.

Ng, Ren, et al. "Light field photography with a hand-held plenoptic camera." Computer Science Technical Report CSTR 2.11 (2005): 1-11.

             
             
             
             

Naive refocusing with a de-bayern filter

Applying a de-Bayer filter to raw lightfield data reduces some artifacts seen in previous two demos.

Ng, Ren, et al. "Light field photography with a hand-held plenoptic camera." Computer Science Technical Report CSTR 2.11 (2005): 1-11.

             
             
             
             

Refocusing with radial-basis function regression

To reduce the gaps in refocused images caused by missing samples, and to exploit the information in samples that would otherwise be averaged-out into a single pixel, it is possible to use a radial-basis function regression superresolution framework.

This demo shows how a simple gaussian-based accumulation, similar to one used for mult-frame supersampling on modern phones, can be used to improve the resolution and quality of a super-sampled refocused image.

Ng, Ren, et al. "Light field photography with a hand-held plenoptic camera." Computer Science Technical Report CSTR 2.11 (2005): 1-11.

Bartlomiej Wronski, Ignacio Garcia-Dorado, Manfred Ernst, Damien Kelly, Michael Krainin, Chia-Kai Liang, Marc Levoy, and Peyman Milanfar. 2019. Handheld multi-frame super-resolution. ACM Trans. Graph. 38, 4, Article 28 (July 2019), 18 pages.

Fiss, Juliet, Brian Curless, and Richard Szeliski. "Refocusing plenoptic images using depth-adaptive splatting." 2014 IEEE international conference on computational photography (ICCP). IEEE, 2014.

             
             
             
             

Refocussing with radial-basis function regression - detail

This demo shows the detail of the sampling and integration pattern of previous demo.

             
             
             
             

Refocussing with radial-basis function regression and a de-Bayer filter

Applying a de-Bayer filter to raw lightfield data reduces some artifacts seen in previous two demos, while introducing a level of additional colour ambiguity (i.e., a less sharp result).

             
             
             
             

Refocussing with hierarchical B-spline regression

Briand, Thibaud, and Pascal Monasse. "Theory and Practice of Image B-Spline Interpolation." Image Processing On Line 8 (2018): 99-141.

             
             
             
             

Refocussing with hierarchical B-spline regression and de-bayern filter

Briand, Thibaud, and Pascal Monasse. "Theory and Practice of Image B-Spline Interpolation." Image Processing On Line 8 (2018): 99-141.

             
             
             
             

Comparison of details of different refocusing method

This demo shows the difference in integration patterns between different methods shown in previous demos.

             
             
             
             

Per-pixel correspondence metric (relative results)

Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013.

             
             
             
             

Per-pixel correspondence and defocus metrics (absolute values)

Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013.

             
             

Tutorials

Basics

Tutorials are intended to introduce the concepts in Possumwood, and help a user navigate its UIs. They are still work-in-progress, and more will be added over time.

This tutorial introduces the Lua integration of Possumwood, using a simple example of an addition node.

OpenGL / GLSL

Introduces basic concepts of GLSL shaders in Possumwood, via a simple setup displaying a rotating car. Its model is loaded from an .obj file, converted to an OpenGL Vertex Buffer Object and displayed using a combination of a vertex and a fragment shader.

A simple GLSL skybox setup, using fragment shader and the background node (based on gluUnProject) to render a background quad with spherically-mapped texture.

Environment map is a simple technique to approximate reflective surfaces in GLSL, without the need of actual raytracing. This tutorial builds on the previous Skybox tutorial, and extends it to make a teapot's material tool like polished metal.

Most implementation of OpenGL 4 remove the ability to influence the width of lines using the classical glLineWidth call. This tutorial describes in detail how to achieve a similar effect (and much more) using a geometry shader, by rendering each line as a camera-facing polygon strip.

Implementation of Curved PN Triangles tessellation technique, presented by Alex Vlachos on GDC 2011. This method tessellates input triangles using solely their vertex positions and normals (independently of the mesh topology) in a geometry shader.

Infinite ground plane, implemented using a single polygon and simple fragment shader "raytracing" against a plane with Y=0.

The Phong reflectance model, and Phong shading are two basic models of light behaviour in computer graphics. This tutorial explains their function, and provides a step-by-step implementation in GLSL, using a single orbiting light as the lightsource.

Image manipulation

This tutorial introduces basic concepts of image manipulation in Possumwood, and a simple shader-based OpenGL setup to display a texture in the scene.

Apart from rendering existing images loaded from a file, Possumwood can also generate and edit images using Lua scripting. In this tutorial, we explore a very simple setup which uses Lua and per-pixel expressions to generate an image.

Installation

Possumwood has been tested only on Linux (several distributions). While it should work on Windows, it has not been compiled or tested there. No support for MacOS is planned for the time being due to heavy dependency on OpenGL.

Launchpad PPA for Ubuntu 18.04+

On Ubuntu, the easiest way to install Possumwood is to use the snapshots PPA, which is built against daily builds of common CGI libraries in the cg-daily PPA:

sudo add-apt-repository ppa:martin-prazak/possumwood
sudo add-apt-repository ppa:cgi-daily-devs/cgi-daily
sudo apt-get update
sudo apt-get install possumwood

This will install Possumwood to your system, enabling to simply run possumwood command from any terminal.

Daily build of Possumwood is built against a number of packages that are newer than most distros provide out of the box. These are contained in the ppa:martin-prazak/cgi-daily PPA. Some packages might conflict with the system versions (OpenImageIO, OpenCV), requiring the original system packages to be uninstalled first.

Installation using Snap

Currently, Possumwood is released in Snap as a development/testing package only. The latest build and its status can be accessed here.

To install a testing version, please run:

sudo snap install --edge possumwood --devmode

This will download and install the latest successful build of Possumwood with its dependencies. To start the application, run possumwood from the command line. As a dev build, snap will not automatically update this installation. Moreover, snap skin support is currently rather rudimentary, making Possumwood not inherit the system look correctly.

Building from source

The project is structured as a standard CMake-built project. To build, just run these in the directory of the repository on any Linux distro:

mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=./install && make -j

This will build Possumwood and all its plugins, and install the result inside ./install directory. Installation is necessary to make sure Possumwood is able to find its plugins and its dependencies. After that, you can run the build via:

./install/bin/possumwood

Code Example

Possumwood is designed to be easily extensible. A simple addition node, using float attributes, can be implemented in a few lines of code:

#include <possumwood_sdk/node_implementation.h>

namespace {

// strongly-typed attributes
dependency_graph::InAttr<float> input1, input2;
dependency_graph::OutAttr<float> output;

// main compute function
dependency_graph::State compute(dependency_graph::Values& data) {
	// maintains attribute types
	const float a = data.get(input1);
	const float b = data.get(input2);

	data.set(output, a + b);

	// empty status = no error (both dependency_graph::State and exceptions are supported)
	return dependency_graph::State();
}

void init(possumwood::Metadata& meta) {
	// attribute registration
	meta.addAttribute(input1, "a");
	meta.addAttribute(input2, "b");
	meta.addAttribute(output, "out");

	// attribute dependencies
	meta.addInfluence(input1, output);
	meta.addInfluence(input2, output);

	// and setting up a compute method
	meta.setCompute(compute);
}

// static registration of a new node type
possumwood::NodeImplementation s_impl("maths/add", init);

}

API Reference

An auto-generated Doxygen reference can be found here

Contributors

Please feel free to contribute! I would welcome any form of feedback, opinions, bugs, code, or description of projects you are planning to use my code in!

At the moment, the project is in its prototype stage, and any feedback or help is welcome. In particular, it is in dire need of a MS Windows build, preferably using AppVeyor. Any help in that area would be greatly appreciated!

License

The code is released under the MIT license. Included example assets come with their own licenses, included in the directory of each asset.