Perform() hangs when passing in false to WRITEFUNCTION

[ikandaswamy]

Hi,

I was trying to write data returned into a buffer. However if there is a write error in the call back function then it looks like easy.Perform() hangs and never returns.

`b := NewBuffer(make([]byte, 0, 10), capacity)

// Callback function to save data instead of redirecting it into stdout.
writeToBufferFunc := func(buf []byte, userdata interface{}) bool {
	if silent == false {
		_, err := b.Write([]byte(buf))
		if err != nil {
			return false
		}
	}
	return true
}
    // the above callback function returns false in my case. Since there is a write error
    // I am trying to restrict the buffer capacity. 

this.myCurl.Setopt(curl.OPT_WRITEFUNCTION, writeToBufferFunc)

this.myCurl.Setopt(curl.OPT_WRITEDATA, b.mybuf)
logging.Infof("\n\n ---------------- Reaches here \n")

    // Hangs here and never reaches 2. 
if err := this.myCurl.Perform(); err != nil {
            logging.Infof("\n\n ---------------- Reaches here 2 \n")
	if show_error == true {
		return nil, err
	} else {
		return nil, nil
	}
}

`

Is there a way for me to fix this in my code ?

Thanks
Isha

[ikandaswamy]

What I was initially trying to do when I came across this error, was to restrict the buffer size that the results of curl are written into. That is possible in c (libcurl). Im wondering if there is an easier way to do this in go-curl.

[andelf]

you can access b.mybuf by convert userdata to []byte.

[ikandaswamy]

Yes, but does that allow us to restrict the write function writing into the buffer of a certain size ?

Thanks
Isha

[andelf]

ref: https://curl.haxx.se/libcurl/c/CURLOPT_WRITEFUNCTION.html

I've wrapped the original size_t. Since it denotes both written size and error.

Maybe we can change write function's type into func([]byte, interface{}) int, error？

[ikandaswamy]

That would be perfect. Then one can use a custom write function, that restricts buffer size that returns length and error.

Since it is a callback function, can it return multiple values ? How will it throw the error ? Can we capture the error ?

Thanks
Isha

[ikandaswamy]

Hi @andelf
Was wondering if there was any update on this ? Since I wanted to restrict the size of the return buffer for curl.

Thanks
Isha

[andelf]

😿

[andelf]

Haven't look into this project for a long time.
Give me some time to recall how I wrote it.

[ikandaswamy]

Found a workaround for this. In order to restrict the return buffer size, we can do the following :

var b bytes.Buffer

// Callback function to save data instead of redirecting it into stdout.
writeToBufferFunc := func(buf []byte, userdata interface{}) bool {
	if silent == false {

		// Check length of buffer b. If it is greater than
		if int64(b.Len()) > responseSize {       == > Check for size here
			// No more writing we are all done
			// If this interrupts the stream of data then we throw not a JSON endpoint error.
			sizeError = true  ==> Set boolean value to handle error post return from  
                                                                OPT_WRITEFUNCTION
			return true  ==> Don't handle error but return true to stop writing into the buffer.
		} else {
			b.Write([]byte(buf))
		}
	}
	return true
}

this.myCurl.Setopt(curl.OPT_WRITEFUNCTION, writeToBufferFunc)
this.myCurl.Setopt(curl.OPT_WRITEDATA, b)

Handle size error here. Since we can't return false in the event of an error, we can do the above and handle error later.

[joaolsilva]

@andelf, what's the status on this? The workaround above isn't really a workaround since the data is still downloaded. go-curl currently pauses the download using CURL_WRITEFUNC_PAUSE when the writeFunc returns false, and will wait until the timeout to actually finish.

EDIT: I've made a workaround, using a Write Func and a Progress Func. Since the progress func does cancel the request when it returns false I use a writeFunc to return false when the download size is exceeded (pausing the download), and cancel the request in the progress func. Then it's just a matter of ignoring the curl.E_ABORTED_BY_CALLBACK that is returned by Perform()