/imfuse

Wavelet-based and guided filtering image fusion algorithms

Primary LanguagePythonMIT LicenseMIT

Image fusion

Hans Brouwer & Aggariyo Wanagiri

Usage

Install dependencies

pip install -r requirements.txt

To fuse images using a CLI:

python guided_filter_fuse.py -h
python wavelet_fuse.py -h

The main results from the report can be recreated by running the evaluation code.

python evaluation.py

Image fusion with guided filtering

  • Guided filter
    • Filter output = linear transformation of guidance image in local window
      • O = a * I[w] + b for w in windows(I)
    • Linear coefficients estimated by minimizing squared difference b/w output and input image
      • Can solve directly with linear regression
      • a = sum(I[w] * P[w] - mean(I[w]) * mean(P[w])) / (len(w) * (var(I[w]) + eps))
      • b = mean(P[w]) - a * mean(I[w])
    • Naively, this causes value of output pixel values (O_i) to change in different windows
      • => Pre-calculate all a & b and then average for each window
      • a_bar = sum(a_k for k in w) / len(w), b_bar = sum(b_k for k in w) / len(w)
  • Image fusion
    1. Compute two-scale representations of images by average filtering
      • B_n = conv(I_n, Z) with Z = 31x31 average filter
      • D_n = I_n - B_n
    2. Construct weight map
      • H_n = conv(I_n, L) with L = 3x3 Laplacian filter
      • S_n = conv(abs(H_N), g) with g = (2r_g+1)x(2r_g+1) Gaussian low-pass with std. dev. = sig_g
      • P_n[k] = 1 if S_n[k] == max(S_i[k] for i in n) else 0
    3. Realign weight maps using spatial consistency
      • Guided filter on weight map with source image as guide
      • W_n = GuidedFilter(P_n, I_n) for B and D
      • Normalize weight maps to sum to 1
    4. Weighted average of base and detail images with respective weight maps, sum base and detail to final output

Schematic illustrating the image fusion approach

Multi-sensor image fusion using the wavelet transform

  • Image fusion
    1. Calculate wavelet transform
    2. At each resolution and point in the image take the max of coefficients of the two images
      • Windowed maximum over local area of coefficients
    3. Majority filter on binary weighting
      • Majority filter, negate, majority filter, negate
    4. Apply binary map to wavelets and transform back to image