vulkano-rs/vulkano

Disabling validation layers

Closed this issue · 14 comments

Version of vulkano: Latest
OS: Linux
GPU Driver: Nvidia 550

How can I disable validation layers as they seem to be tanking my command buffer creation cpu performance?

Vulkano doesn't enable validation layers itself. If you mean vulkano's homegrown validation, you can use the unchecked versions of the functions.

Can this homegrown validation cause command buffer (4k draw calls) build to take like 15ms?

I don't know. You will have to test that.

It's happening in my project, but I will check with the unchecked functions

(You should only need to use _unchecked draw commands. The rest should be fine.)

Thanks for the tip

It unfortunately changed nothing

That's quite strange. For me it usually cuts the recording time in half. I can't explain it doing quite literally nothing.

It needs to be noted however that vulkano's current command buffer implementation is very inefficient. It does a heap allocation per command + dynamic dispath when the command is actually recorded and when deallocating that heap allocation. I'm working on rewriting all of synchronization right now to fix this and a whole bunch of other issues.

You can use a profiler and get at the bottom of what the actual slowdown is due to. However I think it would be more optimal in any case if you recorded one draw command instead of 4k.

This is the command buffer recording code, is there something very wrong with it?

fn get_command_buffers(
    world: &World,
    assets: &AssetLibrary,
    state: &mut State,
    image_id: usize,
) -> Arc<PrimaryAutoCommandBuffer> {
    let now = Instant::now();

    let framebuffer = state
        .renderer
        .framebuffers
        .as_ref()
        .unwrap()
        .get(image_id)
        .unwrap();
    let mut builder = AutoCommandBufferBuilder::primary(
        state
            .renderer
            .command_buffer_allocator
            .as_ref()
            .unwrap()
            .as_ref(),
        state.renderer.queue.as_ref().unwrap().queue_family_index(),
        CommandBufferUsage::OneTimeSubmit,
    )
    .unwrap();

    builder
        .begin_render_pass(
            RenderPassBeginInfo {
                clear_values: vec![
                    Some([0.0, 0.0, 0.0, 1.0].into()),
                    Some([0.0, 0.0, 0.0, 1.0].into()),
                    Some(1f32.into()),
                ],
                ..RenderPassBeginInfo::framebuffer(framebuffer.clone())
            },
            SubpassBeginInfo {
                contents: SubpassContents::SecondaryCommandBuffers,
                ..Default::default()
            },
        )
        .unwrap();

    println!(" Start: {}ms", now.elapsed().as_millis());

    let now = Instant::now();
    let mut batches = world.entities.query::<(&DynamicMesh, &Transform)>();
    thread::scope(|scope| {
        let mut threads = Vec::new();
        for batch in batches.iter_batched(512) {
            let thread = scope.spawn(|| {
                let now = Instant::now();
                let mut sec_builder = AutoCommandBufferBuilder::secondary(
                    state
                        .renderer
                        .command_buffer_allocator
                        .as_ref()
                        .unwrap()
                        .as_ref(),
                    state.renderer.queue.as_ref().unwrap().queue_family_index(),
                    CommandBufferUsage::OneTimeSubmit,
                    CommandBufferInheritanceInfo {
                        render_pass: Some(CommandBufferInheritanceRenderPassType::BeginRenderPass(
                            CommandBufferInheritanceRenderPassInfo {
                                subpass: state
                                    .renderer
                                    .render_pass
                                    .as_ref()
                                    .unwrap()
                                    .clone()
                                    .first_subpass(),
                                framebuffer: Some(framebuffer.clone()),
                            },
                        )),
                        ..Default::default()
                    },
                )
                .unwrap();
                for (_, (dynamic_mesh, transform)) in batch {
                    if dynamic_mesh.vertex_buffer.is_none() || dynamic_mesh.index_buffer.is_none() {
                        continue;
                    }

                    let material = assets
                        .materials
                        .iter()
                        .find(|x| x.name == dynamic_mesh.material)
                        .unwrap();
                    let pipeline = state
                        .renderer
                        .pipelines
                        .get(&(
                            material.vertex_shader.clone(),
                            material.fragment_shader.clone(),
                        ))
                        .unwrap()
                        .clone();

                    sec_builder
                        .bind_pipeline_graphics(pipeline.clone())
                        .unwrap();

                    let vp_set = PersistentDescriptorSet::new(
                        state
                            .renderer
                            .descriptor_set_allocator
                            .as_ref()
                            .unwrap()
                            .as_ref(),
                        pipeline.layout().set_layouts().first().unwrap().clone(),
                        [WriteDescriptorSet::buffer(
                            0,
                            state
                                .renderer
                                .vp_buffers
                                .as_ref()
                                .unwrap()
                                .get(image_id)
                                .unwrap()
                                .clone(),
                        )],
                        [],
                    )
                    .unwrap();

                    let m_set = PersistentDescriptorSet::new(
                        state
                            .renderer
                            .descriptor_set_allocator
                            .as_ref()
                            .unwrap()
                            .as_ref(),
                        pipeline.layout().set_layouts().get(1).unwrap().clone(),
                        [WriteDescriptorSet::buffer(
                            0,
                            transform.buffer.as_ref().unwrap().buffer.clone(),
                        )],
                        [],
                    )
                    .unwrap();

                    if !material.attachments.is_empty() {
                        let att_set = PersistentDescriptorSet::new(
                            state
                                .renderer
                                .descriptor_set_allocator
                                .as_ref()
                                .unwrap()
                                .as_ref(),
                            pipeline.layout().set_layouts().get(2).unwrap().clone(),
                            material
                                .attachments
                                .iter()
                                .map(|attachement| {
                                    if let Attachment::Texture(tex) = attachement {
                                        let texture = assets
                                            .textures
                                            .iter()
                                            .find(|x| x.name == *tex)
                                            .unwrap();
                                        WriteDescriptorSet::image_view_sampler(
                                            0,
                                            texture.image_view.as_ref().unwrap().clone(),
                                            texture.sampler.as_ref().unwrap().clone(),
                                        )
                                    } else {
                                        panic!("not impl");
                                    }
                                })
                                .collect::<Vec<_>>(),
                            [],
                        )
                        .unwrap();

                        sec_builder
                            .bind_descriptor_sets(
                                PipelineBindPoint::Graphics,
                                pipeline.layout().clone(),
                                0,
                                (vp_set.clone(), m_set.clone(), att_set.clone()),
                            )
                            .unwrap();
                    } else {
                        sec_builder
                            .bind_descriptor_sets(
                                PipelineBindPoint::Graphics,
                                pipeline.layout().clone(),
                                0,
                                (vp_set.clone(), m_set.clone()),
                            )
                            .unwrap();
                    }

                    sec_builder
                        .bind_index_buffer(dynamic_mesh.index_buffer.as_ref().unwrap().clone())
                        .unwrap()
                        .bind_vertex_buffers(
                            0,
                            dynamic_mesh.vertex_buffer.as_ref().unwrap().clone(),
                        )
                        .unwrap()
                        .draw_indexed(dynamic_mesh.indices.len() as u32, 1, 0, 0, 0)
                        .unwrap();
                }
                let cmb = sec_builder.build().unwrap();
                (cmb, now.elapsed().as_millis())
            });
            threads.push(thread);
        }
        for thread in threads {
            let (cmb, time) = thread.join().unwrap();
            println!("  Secondary: {}ms", time);
            builder.execute_commands(cmb).unwrap();
        }
    });
    println!(" Dynamic meshes: {}ms", now.elapsed().as_millis());

    let now = Instant::now();
    builder.end_render_pass(Default::default()).unwrap();
    let cmb = builder.build().unwrap();
    println!(" Build: {}ms", now.elapsed().as_millis());
    cmb
}

You can use a profiler and get at the bottom of what the actual slowdown is due to. However I think it would be more optimal in any case if you recorded one draw command instead of 4k.

Yeah I might have to do that, I was just surprised that it causes my game to run at 20fps

Didn't expect the secondary command buffers. Can't say that I recommend them, certainly not with this here inefficient command buffer, but nothing immediately stands out as super inefficient here. I can only recommend to again profile if you must know, but either way a single draw_indexed_indirect_unchecked command is going to be the most efficient with this or the new command buffer.

Thanks, I guess its time for me to learn indirect