SLOPER4D: A Scene-Aware Dataset for Global 4D Human Pose Estimation in Urban Environments (CVPR2023)

Yudi Dai · Yitai Lin · Xiping Lin · Chenglu Wen · Lan Xu · Hongwei Yi · Siqi Shen · Yuexin Ma · Cheng Wang

News and Updates

More info is coming soon…
03/18/2024: Blender Add-on for visualization
05/11/2023: Released a SAM-based tool for 2D mask generation and updated the data loader example.
04/2023: First part of the dataset V1.0 has released! (Dataset)
03/2023: Initial release of the visualization Tool (SMPL-Scene Viewer) (v1.0)

Dataset

15 sequences of 12 human subjects in
10 scenes in urban environments (1k – 30k $m^2$)
100k+ frames multi-source data (20 Hz)
including 2D / 3D annotations and 3D scenes; 7 km+ human motions.

Every human subject signed permission to release their motion data for research purposes.

Dataset breakdown

Num	Sequence	Traj. length ($m$)	Area size ($m^2$)	Frames	Motions
001	campus_001	908	13,400	16,202	Jogging downhill, tying shoelaces, jumping
002	football_002	221	200	4,665	Juggling, passing, and shooting a football
003	street_002	291	1,600	6,496	Taking photos, putting on/taking off a bag
004	library_001	440	2,300	9,949	Borrowing books, reading, descending stairs
005	library_002	474	2,300	8,901	Looking up, reading, returning a book
006	library_003	477	2,300	8,386	Admiring paintings, throwing rubbish, greeting
007	garden_001	217	3,000	5,994	Raising hand, sitting on bench, going upstairs
008	running_001	392	8,500	2,000	Running
009	running_002	985	30,000	8,113	Running
010	park_001	642	9,300	12,445	Visiting a park, walking up a small hill
011	park_002	1,025	11,000	1,000	Buying drinks, trotting, drinking
012	square_001	264	3,200	6,792	Making phone calls, waving, drinking
013	sunlightRock001	386	1,900	10,116	Climbing stairs, taking photos, walking
014	garden_002	209	4,200	5,604	Stooping, crossing a bridge, sitting cross-legged
015	plaza_001	365	2,700	7,989	Admiring sculptures, eating

Data processing

Please see processing pipeline.

Visualization

Please see visualization script.

More qualitative results

Comparison between IMU + ICP and our optimization results.

Borrowing and reading a book on a sofa.

Playing football.

Comparison between original extrinsic parameters and our optimization results.

Cross-Dataset Evaluation

LiDAR-based human pose estimation (HPE)

Camera-based HPE

Global Human Pose Estimation Comparison

License

The SLOPER4D codebase is published under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License. You must attribute the work in the manner specified by the authors, you may not use this work for commercial purposes and if you alter, transform, or build upon this work, you may distribute the resulting work only under the same license. Contact us if you are interested in commercial usage.

Citation

@InProceedings{Dai_2023_CVPR,
    author    = {Dai, Yudi and Lin, Yitai and Lin, Xiping and Wen, Chenglu and Xu, Lan and Yi, Hongwei and Shen, Siqi and Ma, Yuexin and Wang, Cheng},
    title     = {SLOPER4D: A Scene-Aware Dataset for Global 4D Human Pose Estimation in Urban Environments},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2023},
    pages     = {682-692}
}

climbingdaily/SLOPER4D