singer-yang/DeepLens

DeepLens: a differentiable ray-tracer for computational lens design.

DeepLens is an open-source differentiable ray tracer for optical design, optics-network co-design, and optics-aware rendering.

Welcome to use DeepLens to (1) build your own pipeline or (2) compare as the baseline. We can provide code assistance if you plan to use DeepLens in your research, please contact Xinge Yang (xinge.yang@kaust.edu.sa) for more information. Manufacturing service for both refractive and diffractive elements is also avaliable via collaboration!

News

• DeepLens is finall published on Nature Communications, check it here!
• We now have a WeChat group. Please contact Xinge Yang (singeryang1999) to join the discussion!

What is DeepLens

Deep learning + lens design

1. More powerful lens design software with deep learning.
2. Next-generation computational cameras (optical encoding + deep learning decoding).

Key features

1. Open-source ray tracer with similar accuracy as Zemax.
2. Optimization power. Automated lens design from scratch!
3. Render camera captured image and start optics-network co-design with 5 lines of code!
4. Memory-efficient ray-tracing, millions of rays on a desk-machine.
5. Neural representation, represent a camera lens as a network.
6. More features can be obtained via request or collaboration:
   1. Hybrid ray-tracing-wave-propagation imaging model.
   2. Non-sequential ray-tracing.
   3. Polarization ray-tracing.
   4. Multi-GPU parallelization for billion-level ray-tracing.
   5. Illumination and display system optimization.

Applications

1. Automated lens design

Fully automated lens design from scratch. Try it at AutoLens!

2. End-to-End lens design

Lens-network co-design from scratch using final images (or classification/detection/segmentation) as objective.

3. Implicit Lens Representation

A surrogate network for fast (aberration + defocus) image simulation.

4. Hybrid Refractive-Difractive Lens Model

Design hybrid refractive-diffractive lenses with a new ray-wave model.

How to use

Here are two methods to use deeplens in your research:

Method 1

Clone this repo and write your code inside it.

git clone deeplens
cd deeplens
python 0_hello_deeplens.py
python your_code.py

Method 2

Clone the repo and install deeplens as a python package.

git clone deeplens
pip install -e ./deeplens

Then in your code:

import deeplens
lens = deeplens.GeoLens(filename='./lenses/cellphone80deg.json')

Directory

deeplens/
│
├── deeplens/
│   ├── optics/ (contain core functions for optical components)
|   ├── network/ (contain network architectures for image reconstruction and implicit representation)
|   ├── geolens (lensgroup using ray tracing)
│   └── diffraclens (lensgroup using wave propagation)
│
├── README.md
├── LICENSE
├── setup.py
├── requirements.txt
└── 0_hello_deeplens.py (main scripts)

Reference

Citations

DeepLens is first developed by Dr. Congli Wang (previously named dO), then developed and maintained by Xinge Yang.

If you use DeepLens in your research, please cite the corresponding papers:

• [TCI 2022] dO: A differentiable engine for deep lens design of computational imaging systems. Paper, BibTex
• [NatComm 2024] Curriculum Learning for ab initio Deep Learned Refractive Optics. Paper, BibTex
• [SiggraphAsia 2024] End-to-End Hybrid Refractive-Diffractive Lens Design with Differentiable Ray-Wave Model. Paper, BibTex

Projects built on top of DeepLens/dO

(If you donot want to list your paper here, we can remove it.)

• [TPAMI 2023] Aberration-Aware Depth-From-Focus. Paper, BibTex
• [Arxiv 2024] Centimeter-Scale Achromatic Hybrid Metalens Design: A New Paradigm Based on Differentiable Ray Tracing in the Visible Spectrum. Paper