alexcrichton/tokio-curl

How to perform `transfer`?

Closed this issue · 5 comments

estin commented

Session::perform accept Easy handle.

Transfer (used for non-static livetime for example) borrow Easy handle in private field and Transfer cannot be performed by Session.

By my question on SO

Sorry for my poor English (

Thanks for the report! Currently this library doesn't support usage of Transfer and instead requires configuration directly on Easy. This is done because we need the lifetimes to work out.

Do the Easy methods work out for your use case though?

estin commented

Do the Easy methods work out for your use case though?

Yes. If works with fixed size static buffer, like that

extern crate curl;
extern crate futures;
extern crate tokio_core;
extern crate tokio_curl;

use std::str;
use std::cmp;

use curl::easy::Easy;
use futures::{done, Future};
use tokio_core::reactor::Core;
use tokio_curl::Session;


fn main() {
    // output buffer
    const BUFLEN: usize = 5 * 1024 * 1024;
    static mut pos: usize = 0;
    static mut out: [u8; BUFLEN] = [0; BUFLEN];

    let mut lp = Core::new().unwrap();
    let session = Session::new(lp.handle());
    let mut req = Easy::new();
    req.get(true).unwrap();
    req.url("https://www.rust-lang.org").unwrap();
    req.write_function(|data| {
            unsafe {
                if pos == BUFLEN {
                    return Ok(data.len());
                }
                let src_len = data.len();
                if src_len != 0 {
                    // save data to global buffer
                    let size = cmp::min(src_len, BUFLEN - pos);
                    out[pos..size].clone_from_slice(&data[..size]);
                    pos += size;
                }
                Ok(src_len)
            }
        })
        .unwrap();

    let request = session.perform(req).then(|_| {
        unsafe {
            println!("len: {} out:\n{}", pos, str::from_utf8(&out).unwrap());
        };
        done::<u32, u32>(Ok(1))
    });

    // execute and wait result
    lp.run(request).unwrap();
}

Oh I'd actually recommend using Arc<Mutex> for this instead of a global variable for now. I'd eventually like to make a Easy handle that doesn't require Send (allowing Rc<RefCell>) but for now I think that Arc<Mutex> is likely the best bet for shuffling the data around.

estin commented

Arc<Mutex> is works for me.
Thanks a lot!
It is good lesson and experience for me!

Example how to I use Arc<Mutex> to write data into local Vec

extern crate curl;
extern crate futures;
extern crate tokio_core;
extern crate tokio_curl;

use std::str;
use std::sync::{Arc, Mutex};

use curl::easy::Easy;
use futures::{done, Future};
use tokio_core::reactor::Core;
use tokio_curl::Session;


fn main() {
    let out = Arc::new(Mutex::new(Vec::new()));

    let mut lp = Core::new().unwrap();
    let session = Session::new(lp.handle());
    let mut req = Easy::new();
    req.get(true).unwrap();
    req.url("https://www.rust-lang.org").unwrap();

    let dst = out.clone();
    req.write_function(move |data| {
            let mut dst = dst.lock().unwrap();
            dst.extend_from_slice(data);
            Ok(data.len())
        })
        .unwrap();

    let request = session.perform(req).then(|_| {
        let result = out.lock().unwrap();
        println!("len: {} out:\n{}",
                 result.len(),
                 str::from_utf8(&result).unwrap());
        done::<u32, u32>(Ok(1))
    });

    // execute and wait result
    lp.run(request).unwrap();
}

Looks good to me! Closing as this is handled I believe.