The following tasks/users did not reset the watchdog in time: IDLE (CPU 0)

Question

The following tasks/users did not reset the watchdog in time: IDLE (CPU 0)

Closed this issue 8 months ago · 23 comments

I'm using this code to read from my HX711/load sensor:

use esp_idf_svc::hal::{delay, gpio::PinDriver, peripherals::Peripherals};
use loadcell::{hx711::HX711, LoadCell};

fn main() {
    esp_idf_svc::sys::link_patches();
    esp_idf_svc::log::EspLogger::initialize_default();

    let peripherals = Peripherals::take().unwrap();
    let dt = PinDriver::input(peripherals.pins.gpio1).unwrap();
    let sck = PinDriver::output(peripherals.pins.gpio10).unwrap();
    let mut load_sensor = HX711::new(sck, dt, delay::Ets);

    load_sensor.tare(16);
    load_sensor.set_scale(1.0);

    loop {
        if load_sensor.is_ready() {
            let reading = load_sensor.read_scaled();
            log::info!("Last Reading = {:?}", reading);
        }

        delay::FreeRtos::delay_ms(1000u32);
    }
}

ESPMonitor shows me this error (the code hangs on load_sensor.tare(16);):

E (75386) task_wdt: Task watchdog got triggered. The following tasks/users did not reset the watchdog in time:
E (75386) task_wdt:  - IDLE (CPU 0)
E (75386) task_wdt: Tasks currently running:
E (75386) task_wdt: CPU 0: main
E (75386) task_wdt: Print CPU 0 (current core) registers

What could be the issue?

Answer 1 · 2024-02-25T08:38:46.000Z

You should probably define a critical section implementation that disables all interrupts during the reading of the scale. Check out my implementation here https://github.com/beeb/coffee-scale-app/blob/main/rs/src/critical_section.rs

Answer 2 · 2024-02-25T12:04:55.000Z

@beeb Oh, I've been studying that crate. I'm glad the author is here.

I made my Cargo.toml mimic yours:

[dependencies]
log = { version = "0.4", default-features = false }
esp-idf-svc = { version = "0.47" }
loadcell = "0.2.0"
critical-section = "1.0"

Then added your implementation of a critical section:

use critical_section::{set_impl, with};
use esp_idf_svc::hal::interrupt::{IsrCriticalSection, IsrCriticalSectionGuard};
use esp_idf_svc::hal::{delay, gpio::PinDriver, peripherals::Peripherals};
use loadcell::{hx711::HX711, LoadCell};
use std::sync::Mutex;

// Define the critical section implementation
static CS: IsrCriticalSection = IsrCriticalSection::new();
static CS_GUARD: Mutex<Option<IsrCriticalSectionGuard>> = Mutex::new(None);

pub struct EspCriticalSection {}

unsafe impl critical_section::Impl for EspCriticalSection {
    unsafe fn acquire() {
        let mut guard = CS_GUARD.lock().unwrap();
        *guard = Some(CS.enter());
    }

    unsafe fn release(_token: ()) {
        let mut guard = CS_GUARD.lock().unwrap();
        *guard = None;
    }
}

fn main() {
    esp_idf_svc::sys::link_patches();
    esp_idf_svc::log::EspLogger::initialize_default();

    // Set the critical section implementation
    set_impl!(EspCriticalSection);

    let peripherals = Peripherals::take().unwrap();
    let dt = PinDriver::input(peripherals.pins.gpio2).unwrap();
    let sck = PinDriver::output(peripherals.pins.gpio3).unwrap();
    let mut load_sensor = HX711::new(sck, dt, delay::Ets);

    // Use the critical section around sensitive operations

    log::info!("The program stops here then runs again");

    with(|_| {
        load_sensor.tare(16);
    });
    load_sensor.set_scale(1.0);

    loop {
        if load_sensor.is_ready() {
            // Also protect the reading operation
            let reading = with(|_| load_sensor.read_scaled());
        }

        delay::FreeRtos::delay_ms(1000u32);
    }
}

I don't get the error anymore but now the program gets stuck here: log::info!("The program stops here then runs again");. And it runs again and again.

Answer 3 · 2024-02-25T12:23:47.000Z

Hi @weiying-chen , thank you for the feedback.

I see a few ways around your issue:

Disable the Watch Dog Timer. This is not really a permanent solution, but it does get you there. The problem here is that this single function (tare) blocks the whole CPU0 for ~ 160ms while reading 16 samples sequentially.
Read these sections of the docs, and change the delay provider to Delay:
FreeRtos Delay
Generic Delay

By passing Ets, you are starving the IDLE task.

By the way, I hope you don't mind that I yoink your implementation as the new example code for std usage of the library.

Answer 4 · 2024-02-25T12:42:25.000Z

Also you should not use with as the loadcell crate handles entering the critical section at the right moment.

Answer 5 · 2024-02-25T13:37:29.000Z

@beeb Thanks for your help. I got my example working thanks to your repository.

@DaneSlattery this code runs (thanks for the suggestion):

use esp_idf_svc::hal::{delay, gpio::PinDriver, peripherals::Peripherals};
use loadcell::{hx711::HX711, LoadCell};

fn main() {
    esp_idf_svc::sys::link_patches();
    esp_idf_svc::log::EspLogger::initialize_default();

    let peripherals = Peripherals::take().unwrap();
    let dt = PinDriver::input(peripherals.pins.gpio1).unwrap();
    let sck = PinDriver::output(peripherals.pins.gpio10).unwrap();
    let mut load_sensor = HX711::new(sck, dt, delay::FreeRtos);

    load_sensor.tare(16);
    load_sensor.set_scale(1.0);

    loop {
        if load_sensor.is_ready() {
            let reading = load_sensor.read_scaled();
            log::info!("Last Reading = {:?}", reading);
        }

        delay::FreeRtos::delay_ms(1000u32);
    }
}

However, the execution gets stuck as soon as I connect my HX711's DT to my ESP32-C3's GIOP (I'm using 1).

I (14376) rust_esp32_bme280: Last Reading = Ok(0.0)
I (15916) rust_esp32_bme280: Last Reading = Ok(0.0)
I (17456) rust_esp32_bme280: Last Reading = Ok(0.0) 
[I made the connection here. Now it's stuck.]

Maybe my HX711 is broken?

Answer 6 · 2024-02-25T13:42:32.000Z

I would connect the pins before powering on the device. If it still gets stuck, please check your ground and VCC is connected.

Answer 7 · 2024-02-25T13:44:18.000Z

Which board are you using? Sometimes gpio1 is shared with some other function of the board like a built-in button or so. Have you tried with another gpio pin? Also which amplifier are you using? Some of them need the VCC and VDD pins to be bridged together.

Answer 8 · 2024-02-25T13:56:18.000Z

The problem here is that this single function (tare) blocks the whole CPU0 for ~ 160ms while reading 16 samples sequentially.

Maybe not a problem with the idf-svc crate. FreeRTOS can probably yield execution to another higher priority task if needed. Although the C3 is single core so not entirely sure. In any case the critical section is only entered for the required duration so interrupts are probably allowed to trigger during the tare execution in-between reading bits.

Answer 9 · 2024-02-25T13:58:52.000Z

@DaneSlattery My HX711's VCC and GND are connected to my ESP32-C3's 3V3 and GND. My ESP32-C3 is also connected to a breadboard power supply (without it, I get a broken pipe error).

@beeb I'm using a ESP32-C3-MINI-1. If I use gpio 2 and 3, the program is always stuck (even without connecting the jumper wires). If I use 1 and 10, it only gets stuck when I put the jumper wire in 1 (DT). If I use 0 and 10, it's the same result as using 1 and 10.

I have this HX711 amplifier. As you can see, it has VCC but it doesn't have VDD.

Note: Sometimes MY ESP32-C3 disconnects if I put a jumper wire in 0. Maybe there are some power issues.

Answer 10 · 2024-02-25T14:03:43.000Z

Do you use cargo espflash monitor to check for any errors? What does it say when the board hangs or restarts?

Which esp board do you have?

Also yes, connect all wires prior to powering on the device.

A picture of your wiring could also help ruling out any other issues if you can.

Answer 11 · 2024-02-25T14:09:02.000Z

@beeb Yeah, I'm using cargo espflash monitor. When I connect my HX711's DT to my ESP32-C3's 0 giop or 1 giop, I don't see any error messages. It just hangs.

I (70556) rust_esp32_bme280: Last Reading = Ok(524287.0)
I (72096) rust_esp32_bme280: Last Reading = Ok(524287.0)
[It's hanging right here.]

Sometimes when I put the jump wire in 0 (DT), it disconnects:

I (151846) rust_esp32_bme280: Last Reading = Ok(0.0)
I (153386) rust_esp32_bme280: Last Reading = Ok(0.0)
Device disconnected; exiting

I have the ESP32-C3-MINI-1. This one.

Note: when I was first testing my HX711, I heard some sparkling noise (like tiny bonfire crackling). Maybe I damaged my HX711?

Answer 12 · 2024-02-25T14:18:13.000Z

https://docs.espressif.com/projects/esp-idf/en/stable/esp32c3/hw-reference/esp32c3/user-guide-devkitm-1.html mentions that the 3 ways to power the board are mutually exclusive. You should use the USB only without powering the 5v or 3.3v rails. Also, like said before it's probably not a good idea to connect/disconnect things after powering the board.

Answer 13 · 2024-02-25T14:41:27.000Z

@beeb If I only power the board with the USB and my mini computer and if I don't connect/disconnect things after powering the board, I still have the same issue.

I guess I'll buy a new HX711 and load cell tomorrow and see if that fixes it.

Answer 14 · 2024-02-25T18:32:05.000Z

@weiying-chen , I highly recommend you keep the hardware connected the entire time while debugging. The pin is floating when not connected (while the circuit is live). When you connect it while the circuit is live, you create a lot of noise on that GPIO input into the esp32, which can surely confuse the library into believing the HX711 is not ready.

I think it will be important to add a reset mode to the library to recover from situations like this.

Answer 15 · 2024-02-26T00:22:14.000Z

@DaneSlattery Thanks for the advice. And you can use my example . I'll be honored.

Answer 16 · 2024-02-26T05:36:25.000Z

To expand on the critical section topic and to put in writing what I learned while chatting with the good folks over on the esp-rs matrix chat:

The esp-idf crate does provides a critical section implementation, but it's more of a "thread-local" one which doesn't prevent higher priority tasks from triggering interrupts. If an interrupt triggers during the reading of a bit, then it could lead to the wrong value being read. As for the watchdog problem, using with effectively blocks all interrupts from triggering (just like the IsrCriticalSection however it does so for too long when used in combination with tare. This leads to a timeout in the watchdog.

Answer 17 · 2024-02-26T14:03:52.000Z

@DaneSlattery @beeb

I bought a new HX711 and load sensor. It seems to be running now (it's all 0.0, though, but at least it's not stuck):

I (26316) rust_esp32_hx711: Last Reading = Ok(0.0)
I (27316) rust_esp32_hx711: Last Reading = Ok(0.0)
I (28316) rust_esp32_hx711: Last Reading = Ok(0.0)

I'm trying to apply pressure to the metallic rectangle (I don't have anything attached to it), and the reading sometimes changes to this:

I (10566) rust_esp32_hx711: Last Reading = Ok(7340032.0)
I (11566) rust_esp32_hx711: Last Reading = Ok(8388607.0)
I (12566) rust_esp32_hx711: Last Reading = Ok(8388607.0)

Maybe I need these things to see predictable changes?

Answer 18 · 2024-02-28T06:59:31.000Z

@DaneSlattery @beeb It works now. I had to solder the load cell wires directly to the HX711 instead of soldering them to the jump wires.

I (8936) rust_esp32_bme280: Last Reading = Ok(-10.0)
I (9036) rust_esp32_bme280: Last Reading = Ok(-3.0)
I (9136) rust_esp32_bme280: Last Reading = Ok(28.0)
I (9236) rust_esp32_bme280: Last Reading = Ok(44.0)
I (9336) rust_esp32_bme280: Last Reading = Ok(83.0)
I (9436) rust_esp32_bme280: Last Reading = Ok(64.0)
I (9536) rust_esp32_bme280: Last Reading = Ok(67.0)
I (9636) rust_esp32_bme280: Last Reading = Ok(449.0) [I put something on the load cell here]
I (9736) rust_esp32_bme280: Last Reading = Ok(4367.0)
I (9836) rust_esp32_bme280: Last Reading = Ok(7652.0)

The readings are a little crazy, but I guess I have to calibrate the scale.

By the way, this is the final code:

use esp_idf_svc::hal::{
    delay::{Delay, FreeRtos},
    gpio::PinDriver,
    peripherals::Peripherals,
};

use loadcell::{hx711::HX711, LoadCell};

fn main() {
    esp_idf_svc::sys::link_patches();
    esp_idf_svc::log::EspLogger::initialize_default();

    let peripherals = Peripherals::take().unwrap();
    let dt = PinDriver::input(peripherals.pins.gpio2).unwrap();
    let sck = PinDriver::output(peripherals.pins.gpio3).unwrap();
    let delay = Delay::new_default();
    let mut load_sensor = HX711::new(sck, dt, delay);

    load_sensor.tare(16);
    load_sensor.set_scale(1.0);

    loop {
        if load_sensor.is_ready() {
            let reading = load_sensor.read_scaled();
            log::info!("Last Reading = {:?}", reading);
        }

        FreeRtos::delay_ms(100u32);
    }
}

Thanks for all the help!

Answer 19 · 2024-02-28T07:47:22.000Z

Glad you could figure it out! Yeah indeed the wires need to be directly soldered to the amplifier, as it's trying to measure minute differences in voltage. Likewise, having the loadcell mounted rigidly with screws on a solid base and putting the weight on a solid plate that's also rigidly screwed to the load cell is important. I also found that immobilizing all wires with a bit of tape inside the enclosure of the scale helps to get more stable readings.

Answer 20 · 2024-02-28T08:44:36.000Z

@beeb I'm suddenly getting 0.0 again ... It's as if my setup damages the load cells or the HX711 ... or maybe something else is going on?

Answer 21 · 2024-02-28T08:47:09.000Z

Sounds like a hardware issue, make sure you didn't damage one of the very fragile loadcell wires, check your solder joints etc. I'm marking this issue as closed however because I don't think the code is at fault!

Answer 22 · 2024-02-28T08:47:37.000Z

Okay, thanks again for all the help!

Answer 23 · 2024-02-29T13:52:34.000Z

I bought another HX711. It's bigger.

It's been working great so far. Perfect readings (same code as before but with load_sensor.set_scale(0.0027)):

I (5636) rust_esp32_hx711: Last Reading = Ok(0.0162)
I (5736) rust_esp32_hx711: Last Reading = Ok(0.0513)
I (5836) rust_esp32_hx711: Last Reading = Ok(0.0216)
I (5936) rust_esp32_hx711: Last Reading = Ok(37.6245)
I (6036) rust_esp32_hx711: Last Reading = Ok(79.2396)
I (6136) rust_esp32_hx711: Last Reading = Ok(66.1203)
I (6236) rust_esp32_hx711: Last Reading = Ok(60.936302)

I think there's something wrong with the smaller HX711 (or it doesn't like my setup). So I think there's nothing wrong with your crate. It works perfectly. Keep up the good job guys.

Update: Dammit, the readings became 0.0 again ...

Update 2: What the ... I used 5V and it's running again ...