File "/home/robot/3D-Diffusion-Policy/third_party/pytorch3d_simplified/pytorch3d/ops/sample_farthest_points.py", line 139, in sample_farthest_points idx = _C.sample_farthest_points(points, lengths, K, start_idxs) RuntimeError: Not compiled with GPU support.
dbdxnuliba opened this issue ยท 2 comments
(dp3) robot@robot-LEGION-REN7000K-26IRB:~/3D-Diffusion-Policy$ bash scripts/train_policy.sh dp3 adroit_hammer 0112 0 0
gpu id (to use): 0
Train mode
/home/robot/anaconda3/envs/dp3/lib/python3.8/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'dp3.yaml': Defaults list is missing _self_
. See https://hydra.cc/docs/upgrades/1.0_to_1.1/default_composition_order for more information
warnings.warn(msg, UserWarning)
[DP3Encoder] point cloud shape: [512, 3]
[DP3Encoder] state shape: [24]
[DP3Encoder] imagination point shape: None
[PointNetEncoderXYZ] use_layernorm: True
[PointNetEncoderXYZ] use_final_norm: layernorm
[DP3Encoder] output dim: 128
[DiffusionUnetHybridPointcloudPolicy] use_pc_color: False
[DiffusionUnetHybridPointcloudPolicy] pointnet_type: pointnet
[2024-05-24 10:30:10,598][diffusion_policy_3d.model.diffusion.conditional_unet1d][INFO] - number of parameters: 2.551533e+08
Class name: DP3
Number of parameters: 255.2181M
_dummy_variable: 0.0000M (0.00%)
obs_encoder: 0.0648M (0.03%)
model: 255.1533M (99.97%)
mask_genrobottor: 0.0000M (0.00%)
Replay Buffer: state, shape (1000, 24), dtype float32, range -1.201.60
Replay Buffer: action, shape (1000, 26), dtype float32, range -1.001.00
Replay Buffer: point_cloud, shape (1000, 512, 6), dtype float32, range -0.25255.00
Replay Buffer: img, shape (1000, 84, 84, 3), dtype uint8, range 0.00255.00
Replay Buffer: action, shape (1000, 26), dtype float32, range -1.00
Replay Buffer: img, shape (1000, 84, 84, 3), dtype uint8, range 0.00
/home/robot/3D-Diffusion-Policy/third_party/gym-0.21.0/gym/logger.py:34: UserWarning: WARN: Box bound precision lowered by casting to float32
warnings.warn(colorize("%s: %s" % ("WARN", msg % args), "yellow"))
[MujocoPointcloudWrapper] use_point_crop: True
[MujocoPointcloudWrapper] sampling 512 points from point cloud using fps
[WandB] group: adroit_hammer-dp3-0112
[WandB] name: 0
wandb: Currently logged in as: robot_nuli. Use wandb login --relogin
to force relogin
wandb: Tracking run with wandb version 0.17.0
wandb: Run data is saved locally in /home/robot/3D-Diffusion-Policy/3D-Diffusion-Policy/data/outputs/adroit_hammer-dp3-0112_seed0/wandb/run-20240524_103013-zg67wyim
wandb: Run wandb offline
to turn off syncing.
wandb: Syncing run 0
wandb: โญ๏ธ View project at https://wandb.ai/robot_nuli/dp3
wandb: ๐ View run at https://wandb.ai/robot_nuli/dp3/runs/zg67wyim
Eval in Adroit hammer Pointcloud Env: 0%| | 0/20 [00:00<?, ?it/s]Found 1 GPUs for rendering. Using device 0.
test_mean_score: 0.0
test_mean_score: 1.0
test_mean_score: 1.0
test_mean_score: 1.0
Error executing job with overrides: ['task=adroit_hammer', 'training.debug=False', 'training.seed=0', 'training.device=cuda:0', 'exp_name=adroit_hammer-dp3-0112', 'logging.mode=online', 'checkpoint.save_ckpt=True']
Traceback (most recent call last):
File "train.py", line 506, in
main()
File "/home/robot/anaconda3/envs/dp3/lib/python3.8/site-packages/hydra/main.py", line 90, in decorated_main
_run_hydra(
File "/home/robot/anaconda3/envs/dp3/lib/python3.8/site-packages/hydra/_internal/utils.py", line 389, in _run_hydra
_run_app(
File "/home/robot/anaconda3/envs/dp3/lib/python3.8/site-packages/hydra/_internal/utils.py", line 452, in _run_app
run_and_report(
File "/home/robot/anaconda3/envs/dp3/lib/python3.8/site-packages/hydra/_internal/utils.py", line 216, in run_and_report
raise ex
File "/home/robot/anaconda3/envs/dp3/lib/python3.8/site-packages/hydra/_internal/utils.py", line 213, in run_and_report
return func()
File "/home/robot/anaconda3/envs/dp3/lib/python3.8/site-packages/hydra/_internal/utils.py", line 453, in
lambda: hydra.run(
File "/home/robot/anaconda3/envs/dp3/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 132, in run
_ = ret.return_value
File "/home/robot/anaconda3/envs/dp3/lib/python3.8/site-packages/hydra/core/utils.py", line 260, in return_value
raise self._return_value
File "/home/robot/anaconda3/envs/dp3/lib/python3.8/site-packages/hydra/core/utils.py", line 186, in run_job
ret.return_value = task_function(task_cfg)
File "train.py", line 503, in main
workspace.run()
File "train.py", line 257, in run
runner_log = env_runner.run(policy)
File "/home/robot/3D-Diffusion-Policy/3D-Diffusion-Policy/diffusion_policy_3d/env_runner/adroit_runner.py", line 102, in run
obs, reward, done, info = env.step(action)
File "/home/robot/3D-Diffusion-Policy/3D-Diffusion-Policy/diffusion_policy_3d/gym_util/multistep_wrapper.py", line 145, in step
observation, reward, done, info = super().step(act)
File "/home/robot/3D-Diffusion-Policy/third_party/gym-0.21.0/gym/core.py", line 289, in step
return self.env.step(action)
File "/home/robot/3D-Diffusion-Policy/3D-Diffusion-Policy/diffusion_policy_3d/gym_util/video_recording_wrapper.py", line 34, in step
result = super().step(action)
File "/home/robot/3D-Diffusion-Policy/third_party/gym-0.21.0/gym/core.py", line 289, in step
return self.env.step(action)
File "/home/robot/3D-Diffusion-Policy/3D-Diffusion-Policy/diffusion_policy_3d/gym_util/mjpc_diffusion_wrapper.py", line 177, in step
point_cloud, depth = self.get_point_cloud()
File "/home/robot/3D-Diffusion-Policy/3D-Diffusion-Policy/diffusion_policy_3d/gym_util/mjpc_diffusion_wrapper.py", line 166, in get_point_cloud
point_cloud = point_cloud_sampling(point_cloud=point_cloud,
File "/home/robot/3D-Diffusion-Policy/3D-Diffusion-Policy/diffusion_policy_3d/gym_util/mjpc_diffusion_wrapper.py", line 80, in point_cloud_sampling
_, sampled_indices = torch3d_ops.sample_farthest_points(points=point_cloud[...,:3], K=num_points)
File "/home/robot/3D-Diffusion-Policy/third_party/pytorch3d_simplified/pytorch3d/ops/sample_farthest_points.py", line 139, in sample_farthest_points
idx = _C.sample_farthest_points(points, lengths, K, start_idxs)
RuntimeError: Not compiled with GPU support.
wandb: \ 0.125 MB of 0.125 MB uploaded
wandb: Run history:
wandb: SR_test_L3 โโ
โโ
wandb: SR_test_L5 โโโโ
wandb: bc_loss โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
wandb: epoch โโโโโโโโโโโโโโโโโโโโโ
โ
โ
โ
โ
โ
โโโโโโโโโโโโโโ
wandb: global_step โโโโโโโโโโโโโโโโโโโโโ
โ
โ
โ
โ
โ
โโโโโโโโโโโโโโ
wandb: lr โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
wandb: mean_n_goal_achieved โโโโ
wandb: mean_success_rates โโโโ
wandb: test_mean_score โโโโ
wandb: train_action_mse_error โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
wandb: train_loss โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
wandb:
wandb: Run summary:
wandb: SR_test_L3 1.0
wandb: SR_test_L5 0.75
wandb: bc_loss 0.0002
wandb: epoch 800
wandb: global_step 6406
wandb: lr 9e-05
wandb: mean_n_goal_achieved 63.65
wandb: mean_success_rates 1.0
wandb: test_mean_score 1.0
wandb: train_action_mse_error 8e-05
wandb: train_loss 0.0002
wandb:
wandb: ๐ View run 0 at: https://wandb.ai/robot_nuli/dp3/runs/zg67wyim
wandb: โญ๏ธ View project at: https://wandb.ai/robot_nuli/dp3
wandb: Synced 6 W&B file(s), 4 media file(s), 2 artifact file(s) and 0 other file(s)
wandb: Find logs at: ./data/outputs/adroit_hammer-dp3-0112_seed0/wandb/run-20240524_103013-zg67wyim/logs
(dp3) robot@robot-LEGION-REN7000K-26IRB:~/3D-Diffusion-Policy$
Hi, does your computer have Nvidia GPU?
If you have, then maybe re-install pytorch3d under your current conda env.