YuejiangLIU/causaltriplet-1

This repository is the official implementation of
Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning
In submission to CLeaR, 2023

Requirements

To install requirements:

pip install -r requirements.txt

Dataset

To generate paired images from ProcTHOR:

python procthor/generator.py --min_scene_idx=0 --max_scene_idx=9999

Examples of paired images, between which an 'open' action is performed:

           

Training

1. Single-object images under compositional distribution shifts

   To run the experiments encouraging independence between action class and object class:

   bash script/run_comp_critic.sh

   To run the experiments encouraging block-disentangled visual representations with the sparsity regularizer:

   bash script/run_comp_sparse.sh

2. Single-object images under systematic distribution shifts

   To run the experiments encouraging independence between action class and object class:

   bash script/run_syst_critic.sh

   To run the experiments encouraging block-disentangled visual representations with the sparsity regularizer:

   bash script/run_syst_sparse.sh

3. Simulated multi-object images under systematic distribution shifts

   To run the experiments approximating object-centric representations with instance masks:

   bash script/run_instance_mask.sh

   To run the experiments exploiting the latent structures in the Slot Attention with different matching modules:

   bash script/run_slot_thor.sh

4. Real-world multi-object images under compositional distribution shifts

   To run the experiments exploiting the latent structures in GroupVIT with different matching modules:

   bash script/run_group_epic.sh

Analysis

To analyze experiment results:

bash script/run_analysis.sh

Examples of experiment results from the saved logs:

• Effect of block-disentanglement on single-object images under compositional distribution shifts (left: ID, right: OOD)

• Effect of approximate object-centric representations on simulated multi-object images (left: ID, right: OOD)

• Effect of exploiting group structures on real-world multi-object images (left: ID, right: OOD)

• Effect of exploiting slot structures on simulated multi-object images (left: ID, right: OOD)

• Visualization of implicit Slot Attention on simulated multi-object images
  (left to right: input pair, reconstructed pair, segmentation masks)