/obj-pose-refiner

Primary LanguagePython

Object Pose Refiner

Refine coarse pose from previous output using RGB-D/RGB frames

Codes Structure

  • algs
    • folders for different methods implementation
    • avoid confilict with ffb6d
    • base.py
      • refiner base class definition
      • Some general use and post-processing/decode function implements
    • icp.py
      • ICP algorithms/related functions
      • Problems
        • Outlier Problem
        • If need sample from point set
        • Necessary to make it batchify
    • loss.py
      • different loss API
      • currently only implement L2-ADD
    • df_refiner.py
      • similar to densefusion but doing matches between transformed model and dmap
      • add one layer to try to help with matching
      • idea is matching -> featurea extraction -> calculate new (residual) pose from features
      • Problems
        • if change the first matching layer to fully connected
        • batch normalization
          • current layers all with bias
  • configs
    • linemod.yaml
      • configuration file for linemode data set
  • ref_utils
    • utils.py
      • helper functions
      • isrotMat(rot) -> bool
      • Euler2Mat(ai,aj,ak,axes='sxyz',if_dgree=True)->Matrix
      • CalRotMatDist(rotA,rotB,if_degree=True)->angle difference
        • with support from scipy of logm
          • can switch to cv2.Rodrigues()
        • no torch version currently
          • guess need to wait for logm-support for torch
          • or implement logm by myself :(
      • CalPoseMatDist(rotA,rotB,if_degree=True)->d_rot,d_trans
        • no torch version
      • Mat2Euler(rot,axes='sxyz',if_degree=True)-> ai,aj,ak
      • Has add torch-version to support batch size >1
        • with appendix _torch, same parameters
    • pose_utils.py
      • With Linemod default parameters
      • gen_coarse_pose_rand(src_pose,intrinsic=K_default,params=default_params,gen_num = gen_num_default) -> target pose, error
        • stimulate coarse pose using normal distribution
      • dmap2cloud(dmap,intrinsic=K_default)->point cloud
      • convert depthmap to point cloud
      • gen_coarse_pose_rand_batch(src_pose,parameters)-> poses
        • generate poses without test validity to support batch size > 1 without loop
      • project_pose_batch(rot,trans,points)-> projected target point
        • project model points with different rotation and translation
        • support batch size >1
        • batch size determined by bs of rot an trans
        • rot:[bs,3,3],trans:[bs,3],points:[N,3] or [1,N,3]
      • sample_points_rand_batch(points,num_points)->sampled point sets
      • points: [bs,M,3] or [M,3], num_poinst:N
      • sampled:[bs,min(M,N),3]
      • add padding to support situation M<N for later
  • containers
    • trainers and dataset from ffb6d
  • com_tuils
    • util functions from ffb6d
  • scripts
    • bash scripts to start experiments for all classes
  • train_ref.py
    • general training pipeline for algorithms
  • ref_config.py
    • hyperparmeters for experiments and models
    • modify self.model for different models
    • save by yaml.dump for every experiment

Dependencies

  • see in ope_ffb6d/requirement.txt
  • ope_ffb6d
    • many classes and functions are imported or inherited from ope_ffb6d
  • open3d 0.9.0

Running

  • Before running
    • modify configs(experiment name, wandb project name)
    • modify ref_config.py
    • set up experimetns parameters in main fuction of train_ref.py
  • Train
    • train all class of linemod at once 
      bash train_lm.sh
    • train one specific class
      • modify class name in configs
      python train_ref.py configs/linemod.yaml
  • Evaluate 
    python eval_refiner.py
    • no torch distributed for evaluation currently
    • add batch evaluation script later
    • using train script
    • set up model before running