tomhsiao1260/awesome-scroll-tools

tools to unroll and read the scrolls

📜 awesome scroll tools

In this repository you are going to find all the awesome awarded open source contributions from our community that will allow us to read the scrolls! 📚✨

They are divided in three categories: Segmentation, Ink Detection (3D, Scroll segment-based, Fragment-based), and Others.

Every category is subdivided in classes: 🌟 Highlighted (for popular contributions), ⚙️ Tools, 📦 Materials, 📝 Reports, and 📊 Visualization.

Some highlighted contributions are added to this repository as submodules.

We are trying to keep this repository updated as much as we can, but research moves fast! 🏃💨

For state-of-the-art contributions please join our Discord channel 💬⏰

Segmentation

🌟 Highlighted

🛠️ Semi-automatic tools

• Volume Cartographer is a tool to manually annotate segments of scrolls, supported by Optical Flow Segmentatin. It was first built by EduceLab, but the current fork used by the community is that by Philip Allgaier.

  • Optical Flow Segmentation by Julian Schilliger
    • Speed improvement by Julian Schilliger
  • Sharkbite fix by Chuck
  • Autorientation by Santiago Pelufo
  • Nextgen Volume Cartographer by Philip Allgaier
  • More Improvements by Philip Allgaier

• Khartes is a tool to manually create and visualize segment meshes, while also visualizing a preview of the rendered segment. It was first developed by Chuck

  • Multiple improvements by Chuck
  • Multiresolution chunked volumes by Chuck

⚙️ Automatic tools (work in progress)

• Thaumato Anakalyptor is an automatic tool that combines classical methods such as threshold gradient operator based edge detectors and Deep Learning based instance segmentation of point clouds to detect, merge and render segments. It was built by Julian Schilliger (part of Grand Prize winning submission)

🛠️ Tools

• Fast Segment Rendering by Julian Schilliger. Fast rendering of segments with GPU acceleration. Capable of saving the surface volume to multiple file formats.

  • CPU rendering by Julian Schilliger and Giorgio Angelotti

• Volumetric Vesuvius Labelling by James Darby. Provide custom tooling the Napari 3d viewer that will help manually annotate volumetric masks of the scrolls to train ML models for 3D segmentation.

• Autosegmentation preprocessing pipeline (work in progress) collection of scripts to preprocess volumes for autosegmentation. By Giorgio Angelotti

• Segment2Voxel by Giorgio Angelotti. Tool to create 1-voxel thick volumetric segment labels starting from mesh objs.

• Hraun is a collection of python tools for handling volumetric scroll data by Forrest McDonald

  • 3D segmentation prototype by Forrest McDonald

• Scroll compression and masking by Olivier Daubney. Script to compress and mask scroll data, greatly reducing storage requirements!

• Mesh merging by Julian Schilliger. Merges multiple overlapping meshes into one continuous mesh. Flattening not included.

  • Mesh merging prototype by Giorgio Angelotti. Different attempt to merge existing mesh of segments by projecting them in 2D and retriangulating in the plane.

• Meshing and chunking by Santiago Pelufo

• Volumetric segmentation model with labels, deep learning 3D model to separate papyrus from air, by Tim Skinner

• Superpixels and cells by Santiago Pelufo

• Segment Flattening by Julian Schilliger and Giorgio Angelotti. Improved flattening of scroll segments.

  • Slim-Flatboi previous implementation of the SLIM algorithm with minimization of isometric distortion to flatten scroll segments. Later included in ThaumatoAnakalyptor. By Giorgio Angelotti.

• Single Sheet Segmentation attempt by Brett Olsen

• vesuvius-blender by Santiago Pelufo. Explore the X-ray scans in Blender.

• vesuvius-build by Santiago Pelufo. Scripts to build files for progressive loading of the data. Convert the tif stack to grid cells or to h5 format that can be used by Ilastik.

• Volume Annotate A reimplementation of Volume Cartographer in Python by Moshe Levy. Missing all the recent improvements.

  • VA-Sheet Tracer by Trevor, Tom, Babak and Boaz

• vesuvius-image by Brett Olsen. Tool for storing and viewing data, including efficient Zarr loading of stack of tif images later included in Khartes.

• Quick Segment Created by EduceLab for annotating a large air gap in Scroll 1, and then projecting from that gap to either side to create two large segments, colloquially referred to as the “Monster Segment”. Hasn’t been used for more segmentation, since it was the only large air gap we could find.

📦 Materials

🌟 Highlighted

• Sheet instance annotation of cubes for Deep Learning models (work in progress)

  • More cubes to annotate, help us!

• Denoised and contrast enhanced volumes, download here, same path pattern for other scrolls.

📜 Segments

• Scroll 2 segments by Sean Johnson

• New segments by Sean Johnson

• Large segments by Sean Johnson

🏷️ Volumetric Labels

• Instance segmentation labels by James Darby

📝 Reports

• Technical report on ThaumatoAnakalyptor by Julian Schilliger

• Physical equalization of scrolls' brightness by Giorgio Angelotti

• Volumetric segmentation architecture investigation by James Darby

• Instance segmentation experiments by James Darby, Ryan Reszetnik, Liamo Pennimpede, Lucas Nelson

• Probabilistic view on the offset for surface volume creation by Giorgio Angelotti

📊 Visualization

• vesuvius-gui by Johannes Rudolph

• Browser-based scroll viewer by Yao Hsiao

• Crackle Viewer is a tool to browse and annotate surface volumes of rendered segments, by Julian Schilliger

• Point cloud extraction method comparer by Giorgio Angelotti. Tool to compare different point cloud extraction methods.

• Pipeline Visualize by Yao Hsiao. Tool to visualize the first steps of the Thaumato Anakalyptor pipeline.

• Cell viewer and segmentation comparison by Yao Hsiao

• Volume Viewer Used by the segmentation team primarily to see which segments they have worked on already. Hosted here. By Yao Hsiao

  • Vesuvius Challenge Whiteboard by Yao Hsiao and Dalufishe

• Scroll Viewer by Luke Farritor. A lightweight, extensible tool for viewing volumetric data, which runs in the browser, and is very fast.

Ink Detection

🏆 3D Ink Detection

🌟 Highlighted

• 3D (volumetric) Ink detection model by Ryan Chesler. Ink detection model that works on full scroll data in 3D, without segmentation nor flattening.

⚙️ Tools

• Large Scroll Model is a 3D Unet pretrained on scroll data, by Ryan Chesler

• UV predictions visualizer by Giorgio Angelotti. Script to quickly visualize the ink predictions output by Ryan Chesler's 3D model as a scatter pkot on segments. Needs the predictions Zarr for the full scroll.

• Volumetric ink detection attempt by Jorge Villaescusa

📦 Materials

• 3D Ink labels by Sean Johnsonn

• 3D Ink predictions by Ryan Chesler. Predictions of 3D Ink models on full scrolls in Zarr format.

🖋️ Scroll segments-based Ink Detection

🌟 Highlighted

• Grand Prize Winner Ink Detection model by Youssef Nader, Luke Farritor and Julian Schilliger

⚙️ Tools

• Unsupervised Ink Detection with DINO by Jorge García. Contains experiments related to detecting ink without labels, including a Colab notebook.

• Vesuvius GP+ by Jared Landau. Updated version of the Grand Prize Ink Detection script with extra features.

• Segment-to-segment label mapping by Oliver Daubney

• Runner Up Models, December, 2023

  • Ink detection model by SQ Mah

  • Ink detection model by Lou Schlessinger and Arefeh Sherafati

  • Ink detection model by lian Rafael Dal Prá, Sean Johnson, Leonardo Scabini, Raí Fernando Dal Prá, João Vitor Brentigani Torezan, Daniel Baldin Franceschini, Bruno Pereira Kellm, Marcelo Soccol Gris, Odemir Martinez Bruno

• Vesuvius Kintsugi is a tool to label floodfill surface volumes of rendered segments, by Giorgio Angelotti

• Omit is a pipeline that tries to detect ink with classical approaches (not deep learning) by Timo Meireman

• First Letters winning models, October 2023

  • Ink detection model by Luke Farritor
  • Ink detection model, 2nd place but more accurate by Youssef Nader

• Crackle Viewer is a tool to browse and annotate surface volumes of rendered segments, by Julian Schilliger

• Fourth placed Kaggle model finetuning on scroll data by Luke Farritor

• Scroll pretraining by Youssef Nader. Youssef’s original idea for pretraining on the scrolls and finetuning on the fragments, which led him to winning the First Letters Prize.

📦 Materials

• Ink Labels. Nicola Bodill produced more accurate labels for ink detection based on the prediction of the Grand Prize winner model

• Scroll 4 predictions. Youssef Nader produced some predictions on Scroll 4 from his Grand prize winner model. No sure trace of ink yet

• Ink detection masks. Anton Repushko shared some ink labels for Scroll 1, these labels were used by many participants for their final submission in December 2023.

• Crackle labels on Scroll 1 by Sean Johnson

📝 Reports

• Introduction to Ink Detection by Jared Landau

• Grand Prize Presentation by Youssef Nader and Julian Schilliger

• First Ink on scroll 1 by Casey Handmer

📊 Visualization

• Segment viewer. Used by the segmentation team primarily to see which segments they have worked on already. Hosted here By Yao Hsiao and Dalufishe

  • Improvements by Yao Hsiao

• Segment browser is a web-based tool to browse layers and open source ink detection results of all released segments. By Johannes Rudolph

📜 Fragment-based Ink Detection

🌟 Highlighted

• Kaggle competition on Fragments

  • 1st place by Ryan Chesler
  • 2nd place by RTX2309
  • 3rd place by wuyu
  • 4th place by POSCO DX -- Heeyoung Ahn
  • 5th place by Aksell
  • 6th place by chumajin
  • 7th place by OverthINKingSegmenter
  • 8th place by Luck is all you need
  • 9th place by still 1 fold, 2 net
  • 10th place by Feng Qilong

• 2.5D fragment segmentation (ink detection) baseline by Ryosuke Tanaka

• Ink ID by Stephen Parsons

📝 Reports

• Kaggle Challenge top ink detection model analysis by Ryan Chesler

• Ink detection model resolution analysis by Yannick Kirchhoff, Maximilian Rokuss and Benjamin Hamm

Others

⚙️ Tools

• Efficient Data Downloader Scripts to efficiently download data with rclone, by James Darby

📦 Materials

• CT scanning campfire scrolls Ahron Wayne replicated the carbonization process of a papyrus scroll and scanned it with his personal CT scanner

📝 Reports

• Hard-Hearted Scrolls, PhD Dissertation by Stephen Parsons

Contributions

If you want to contribute and add any resource please do a PR! 😊🚀