stoewer/go-nakadi

Chunked buffered response is not correctly constructed

Closed this issue · 0 comments

krnkl commented

While trying to cover chunked read with unittests I've discovered that resulted line is not correctly constructed, and as a result - cannot be unmarshalled from json.
I.e. the following part:

        line, isPrefix, err := s.buffer.ReadLine()
	if err != nil {
		return Cursor{}, nil, errors.Wrap(err, "failed to read next batch")
	}

	fmt.Println("first chunk:\t\t", string(line))
	for isPrefix {
		var add []byte
		add, isPrefix, err = s.buffer.ReadLine()
		if err != nil {
			return Cursor{}, nil, errors.Wrap(err, "failed to read next batch")
		}

		fmt.Println("before append:\t\t", string(line))
		line = append(line, add...)
		fmt.Println("after append:\t\t", string(line))
	}

produces the following output(buffer read size is 16 bytes):

first chunk:		 {"cursor":{"part
before append:		 ition":"0","offs
after append:		 ition":"0","offsition":"0","offs
before append:		 ition":"0","offsition":"0","offs
after append:		 ition":"0","offsition":"0","offset":"1","event_t
before append:		 ition":"0","offsition":"0","offset":"1","event_t
after append:		 ition":"0","offsition":"0","offset":"1","event_type":"test-event
before append:		 ition":"0","offsition":"0","offset":"1","event_type":"test-event

Acceptance criteria:

  • fix chunked read from buffer
  • Improve test coverage can be improved for simple_stream.go.