Question: How to exec multiple things at once?

[Furry]

In my project I'm using Lua Scripts to handle text-parsing & collecting logic for multiple public proxy website. Each script is for it's own website, where it invokes an HTTP request, and does some logic. My Rust code calls each of these lua scripts by reading the files, and invokes them one by one, though some take a little bit to execute (5-10 seconds). I'd like to figure out how to make them process all at the same time.

for entry in fs::read_dir("./sources").unwrap() {
    let entry = entry.unwrap();
    let path = entry.path();
    let filename = path.file_name().unwrap().to_str().unwrap();
    if filename.ends_with(".lua") {
        let contents = fs::read_to_string(path).unwrap();
        lua.context(|ctx| {
            let r = ctx.load(&contents).eval::<Table>().unwrap();
            let x: Vec<String> = r.get("addresses").unwrap();
            let name: String = r.get("name").unwrap();
            LOGGER.log(format!(
                "Loaded {} proxies from {}",
                x.len().to_string().blue(),
                name.blue()
            ));
            addrs.extend(x);
        });
    }```
    
I'd like to run all the scripts at once, but Context & Lua don't impl Send, so I can't spawn Tokio threads. How would I do something like this?

[jugglerchris]

Hi,
Lua is Send.
Unfortunately Lua the language implementation is not thread safe (therefore Lua is not Sync), so it is not possible to run Lua code in the same state simultaneously on different threads. However you can have a separate Lua for each thread (or have a pool of them protected with Mutex<> etc.) if that works for your use case.

[Furry]

Thank you for the quick reply!

(or have a pool of them protected with Mutex<> etc.)

I like the idea of this, and am a bit curious on how this would work. If I have a pool of Lua instances, how would I know if one is already actively executing something, or if it's ready to process?

[jugglerchris]

I like the idea of this, and am a bit curious on how this would work. If I have a pool of Lua instances, how would I know if one is already actively executing something, or if it's ready to process?

I'm afraid I don't have a ready-made solution, but it seems like something similar to r2d2 does for a database connection pool would work.

Other options might include just create a new one as needed (if the initialisation time is acceptable), keep instances in a thread local variable, etc. - at this point it's not Lua/rlua specific.

[Furry]

Alright, thank you for the information, that helps a lot! I'll see what I can work out.