Project Page | Paper | Video
This repository contains code for:
Field of Junctions: Extracting Boundary Structure at Low SNR
Dor Verbin and Todd Zickler
International Conference on Computer Vision (ICCV), 2021.
Please contact us by email for questions about our paper or code.
Our code is implemented in pytorch. It has been tested using pytorch 1.6 but it should work for other pytorch 1.x versions. The following packages are required:
- python 3.x
- pytorch 1.x
- numpy >= 1.14.0
To analyze an HxWxC
image into its field of junctions, you can simply run the following code snippet:
from field_of_junctions import FieldOfJunctions
foj = FieldOfJunctions(img, opts)
foj.optimize()
In addition to the input image, the FieldOfJunctions
class requires an object opts
with the following fields:
R Patch size
stride Stride of field of junctions (e.g. opts.stride == 1 is dense)
eta Width of Heaviside functions
delta Width of boundary maps
lr_angles Learning rate of angles
lr_x0y0 Learning rate of vertex positions
lambda_boundary_final Final value of spatial boundary consistency weight lambda_B
lambda_color_final Final value of spatial color consistency weight lambda_C
nvals Number of values to query in Algorithm 2 from the paper
num_initialization_iters Number of initialization iterations
num_refinement_iters Number of refinement iterations
greedy_step_every_iters Frequency of "greedy" iteration (applying Algorithm 2 with consistency)
parallel_mode Whether or not to run Algorithm 2 in parallel over all `nvals` values.
Note that setting parallel_mode
to True
typically results in faster optimization, but requires more memory during
initialization. For large images on a GPU with limited memory, you might need to set parallel_mode
to False
.
Instead of using foj.optimize()
which executes the entire optimization scheme, it is possible to access the field of junctions
during optimization by using the following equivalent code snippet:
foj = FieldOfJunctions(img, opts)
for i in range(foj.num_iters):
foj.step(i)
See Python notebook in the examples/
folder for a full usage example.
In order to compute the (global) boundary maps for a given field of junctions object foj
:
params = torch.cat([foj.angles, foj.x0y0], dim=1)
dists, _, patches = foj.get_dists_and_patches(params)
local_boundaries = foj.dists2boundaries(dists)
global_boundaries = foj.local2global(local_boundaries)[0, 0, :, :].detach().cpu().numpy()
In order to compute the boundary-aware smoothing of the input image given foj
, use:
params = torch.cat([foj.angles, foj.x0y0], dim=1)
dists, _, patches = foj.get_dists_and_patches(params)
smoothed_img = foj.local2global(patches)[0, :, :, :].permute(1, 2, 0).detach().cpu().numpy()
A zip file containing all of our synthetic data is available here. It contains the 300 images we used for quantitatively evaluating our algorithm, as well as ground truth locations of edges and corners/junctions.
For citing our paper, please use:
@InProceedings{verbin2021foj,
author = {Verbin, Dor and Zickler, Todd},
title = {Field of Junctions: Extracting Boundary Structure at Low {SNR}},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021},
pages = {6869-6878}
}