Inference is not working giving error of Ninja

Question

Inference is not working giving error of Ninja

Opened this issue 5 months ago · 0 comments

python inference/real3d_infer.py --src_img data/raw/examples/Macron.png --drv_aud data/raw/examples/Obama_5s.wav --drv_pose data/raw/examples/May_5s.mp4 --bg_img data/raw/examples/bg.png --out_name output.mp4 --out_mode concat_debug
| WARN: checkpoints/240210_real3dportrait_orig/th1kh_512_audio2motion/base.yaml not exist.
| load 'model' from 'checkpoints/240210_real3dportrait_orig/audio2secc_vae/model_ckpt_steps_400000.ckpt', strict=True
| WARN: checkpoints/240210_real3dportrait_orig/secc2plane_torso_orig/secc_img2plane_orig.yaml not exist.
INFO:torch.distributed.nn.jit.instantiator:Created a temporary directory at /tmp/tmpq_0xt_9p
INFO:torch.distributed.nn.jit.instantiator:Writing /tmp/tmpq_0xt_9p/_remote_module_non_scriptable.py
| load 'model' from 'checkpoints/240210_real3dportrait_orig/secc2plane_torso_orig/model_ckpt_steps_100000.ckpt', strict=True
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1722340301.565697 9411 gl_context_egl.cc:85] Successfully initialized EGL. Major : 1 Minor: 5
I0000 00:00:1722340301.579896 9477 gl_context.cc:357] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 470.141.03), renderer: NVIDIA RTX A4000/PCIe/SSE2
W0000 00:00:1722340301.580273 9411 face_landmarker_graph.cc:174] Sets FaceBlendshapesGraph acceleration to xnnpack by default.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
W0000 00:00:1722340301.602778 9479 inference_feedback_manager.cc:114] Feedback manager requires a model with a single signature inference. Disabling support for feedback tensors.
W0000 00:00:1722340301.609719 9483 inference_feedback_manager.cc:114] Feedback manager requires a model with a single signature inference. Disabling support for feedback tensors.
face area percent 0.22489264198096937 larger than threshold 0.2, directly use the input image...
Extracted wav file (16khz) from data/raw/examples/Obama_5s.wav to data/raw/examples/Obama_5s_16k.wav.
Loading the HuBERT Model...
Loading the Wav2Vec2 Processor...
I0000 00:00:1722340306.606997 9411 gl_context_egl.cc:85] Successfully initialized EGL. Major : 1 Minor: 5
I0000 00:00:1722340306.620053 9552 gl_context.cc:357] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 470.141.03), renderer: NVIDIA RTX A4000/PCIe/SSE2
W0000 00:00:1722340306.628576 9553 inference_feedback_manager.cc:114] Feedback manager requires a model with a single signature inference. Disabling support for feedback tensors.
I0000 00:00:1722340306.926005 9411 gl_context_egl.cc:85] Successfully initialized EGL. Major : 1 Minor: 5
I0000 00:00:1722340306.943145 9568 gl_context.cc:357] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 470.141.03), renderer: NVIDIA RTX A4000/PCIe/SSE2
W0000 00:00:1722340306.943490 9411 face_landmarker_graph.cc:174] Sets FaceBlendshapesGraph acceleration to xnnpack by default.
W0000 00:00:1722340306.946731 9570 inference_feedback_manager.cc:114] Feedback manager requires a model with a single signature inference. Disabling support for feedback tensors.
W0000 00:00:1722340306.953479 9572 inference_feedback_manager.cc:114] Feedback manager requires a model with a single signature inference. Disabling support for feedback tensors.
batch 0 | loss_lan: 0.40, loss_reg_id: 0.27,loss_reg_exp: 0.97
| To extract pose from data/raw/examples/May_5s.mp4
lms_2d file not found, try to extract it from video... data/raw/examples/May_5s_lms.npy
I0000 00:00:1722340308.146324 9411 gl_context_egl.cc:85] Successfully initialized EGL. Major : 1 Minor: 5
I0000 00:00:1722340308.159511 9593 gl_context.cc:357] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 470.141.03), renderer: NVIDIA RTX A4000/PCIe/SSE2
W0000 00:00:1722340308.159799 9411 face_landmarker_graph.cc:174] Sets FaceBlendshapesGraph acceleration to xnnpack by default.
W0000 00:00:1722340308.162653 9594 inference_feedback_manager.cc:114] Feedback manager requires a model with a single signature inference. Disabling support for feedback tensors.
W0000 00:00:1722340308.169300 9598 inference_feedback_manager.cc:114] Feedback manager requires a model with a single signature inference. Disabling support for feedback tensors.
I0000 00:00:1722340308.171399 9411 gl_context_egl.cc:85] Successfully initialized EGL. Major : 1 Minor: 5
I0000 00:00:1722340308.188100 9602 gl_context.cc:357] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 470.141.03), renderer: NVIDIA RTX A4000/PCIe/SSE2
W0000 00:00:1722340308.188358 9411 face_landmarker_graph.cc:174] Sets FaceBlendshapesGraph acceleration to xnnpack by default.
W0000 00:00:1722340308.191934 9606 inference_feedback_manager.cc:114] Feedback manager requires a model with a single signature inference. Disabling support for feedback tensors.
W0000 00:00:1722340308.198802 9607 inference_feedback_manager.cc:114] Feedback manager requires a model with a single signature inference. Disabling support for feedback tensors.
| Extracted pose from data/raw/examples/May_5s.mp4
{'drv_audio_name': 'data/raw/examples/Obama_5s.wav', 'src_image_name': 'infer_out/tmp/cropped_src_img.png', 'a2m_ckpt': 'checkpoints/240210_real3dportrait_orig/audio2secc_vae', 'head_ckpt': '', 'torso_ckpt': 'checkpoints/240210_real3dportrait_orig/secc2plane_torso_orig', 'bg_image_name': 'data/raw/examples/bg.png', 'drv_pose_name': 'data/raw/examples/May_5s.mp4', 'blink_mode': 'period', 'temperature': 0.2, 'mouth_amp': 0.45, 'out_name': 'output.mp4', 'out_mode': 'concat_debug', 'map_to_init_pose': 'True', 'head_torso_threshold': None, 'seed': None, 'min_face_area_percent': 0.2, 'low_memory_usage': False}
| audio-to-motion finished
rendering drv secc: 0%| | 0/3 [00:00<?, ?it/s]create rasterizer on device cuda:0
rendering drv secc: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:02<00:00, 1.08it/s]
blinking secc: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 128/128 [00:00<00:00, 647.88it/s]
Real3D-Portrait is rendering frames: 0%| | 0/128 [00:00<?, ?it/s]Setting up PyTorch plugin "bias_act_plugin"... Failed!
Real3D-Portrait is rendering frames: 0%| | 0/128 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
subprocess.run(
File "/home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/inference/real3d_infer.py", line 625, in
GeneFace2Infer.example_run(inp)
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/inference/real3d_infer.py", line 561, in example_run
infer_instance.infer_once(inp)
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/inference/real3d_infer.py", line 187, in infer_once
out_name = self.forward_system(samples, inp)
File "/home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/inference/real3d_infer.py", line 547, in forward_system
out_fname = self.forward_secc2video(batch, inp)
File "/home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(args, **kwargs)
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/inference/real3d_infer.py", line 487, in forward_secc2video
gen_output = self.secc2video_model.forward(img=ref_img_head, camera=camera[i:i+1], cond=cond, ret={}, cache_backbone=True, use_cached_backbone=False)
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/./modules/real3d/img2plane_baseline.py", line 201, in forward
out = self.synthesis(img, camera, cond=cond, ret=ret, update_emas=update_emas, cache_backbone=cache_backbone, use_cached_backbone=use_cached_backbone, **synthesis_kwargs)
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/./modules/real3d/secc_img2plane.py", line 110, in synthesis
planes, cano_planes = self.cal_plane(img, cond, ret, **synthesis_kwargs)
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/./modules/real3d/secc_img2plane.py", line 85, in cal_plane
planes = self.cal_plane_given_cano(cano_planes, cond)
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/./modules/real3d/secc_img2plane.py", line 75, in cal_plane_given_cano
secc_planes = self.cal_secc_plane(cond) # [B, 3, CD, H, W]
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/./modules/real3d/secc_img2plane.py", line 70, in cal_secc_plane
secc_planes = self.secc_img2plane_backbone(inp_pncc)
File "/home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/./modules/real3d/segformer.py", line 713, in forward
x = self.prenet(x)
File "/home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in call_impl
return forward_call(*args, **kwargs)
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/./modules/eg3ds/models/networks_stylegan2.py", line 188, in forward
x = bias_act.bias_act(x, b, act=self.activation, gain=act_gain, clamp=act_clamp)
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/./modules/eg3ds/torch_utils/ops/bias_act.py", line 86, in bias_act
if impl == 'cuda' and x.device.type == 'cuda' and init():
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/./modules/eg3ds/torch_utils/ops/bias_act.py", line 43, in init
plugin = custom_ops.get_plugin(
File "/home/tignanshu/Real3DPortraitRoot/Real3DPortrait/./modules/eg3ds/torch_utils/custom_ops.py", line 138, in get_plugin
torch.utils.cpp_extension.load(name=module_name, build_directory=cached_build_dir,
File "/home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1284, in load
return jit_compile(
File "/home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1509, in jit_compile
write_ninja_file_and_build_library(
File "/home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1624, in write_ninja_file_and_build_library
run_ninja_build(
File "/home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1909, in run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error building extension 'bias_act_plugin': [1/3] /usr/bin/nvcc -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include/TH -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include/THC -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' --use_fast_math -std=c++17 -c /home/tignanshu/.cache/torch_extensions/py39_cu117/bias_act_plugin/b46266ff65f9fa53c32108953a1c6f16-nvidia-rtx-a4000/bias_act.cu -o bias_act.cuda.o
FAILED: bias_act.cuda.o
/usr/bin/nvcc -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include/TH -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include/THC -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' --use_fast_math -std=c++17 -c /home/tignanshu/.cache/torch_extensions/py39_cu117/bias_act_plugin/b46266ff65f9fa53c32108953a1c6f16-nvidia-rtx-a4000/bias_act.cu -o bias_act.cuda.o
nvcc fatal : Value 'c++17' is not defined for option 'std'
[2/3] c++ -MMD -MF bias_act.o.d -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include/TH -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include/THC -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -c /home/tignanshu/.cache/torch_extensions/py39_cu117/bias_act_plugin/b46266ff65f9fa53c32108953a1c6f16-nvidia-rtx-a4000/bias_act.cpp -o bias_act.o
FAILED: bias_act.o
c++ -MMD -MF bias_act.o.d -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include/TH -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include/THC -isystem /home/tignanshu/my_new_conda/envs/real3dportrait/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -c /home/tignanshu/.cache/torch_extensions/py39_cu117/bias_act_plugin/b46266ff65f9fa53c32108953a1c6f16-nvidia-rtx-a4000/bias_act.cpp -o bias_act.o
In file included from /home/tignanshu/.cache/torch_extensions/py39_cu117/bias_act_plugin/b46266ff65f9fa53c32108953a1c6f16-nvidia-rtx-a4000/bias_act.cpp:14:0:
/home/tignanshu/my_new_conda/envs/real3dportrait/lib/python3.9/site-packages/torch/include/ATen/cuda/CUDAContext.h:5:10: fatal error: cuda_runtime_api.h: No such file or directory
#include <cuda_runtime_api.h>
^~~~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.