[Question] The line will be truncated during kube-apiserver (another app) is writing in.

[wuxler]

Hi, I try to used nxadm/tail to parse log file of kube-apisever who logs out json lines into a log file with package lumberjack.

But I find that the line will be truncated during kube-apiserver is writing in. Just like the image following shows, and it will cause that failed to call json.Unmarshal to convert a json line string to an Object.

Maybe it because the interval duration tail try to read less than kube-apisever write cost ?

execute code:

func (lt *LogTail) Start() error {
	tailConfig := tail.Config{
		Follow: true,
		ReOpen: true,
		// set start location at the end of file
		Location: &tail.SeekInfo{
			Offset: 0,
			Whence: io.SeekEnd,
		},
		Logger: logrus.StandardLogger(),
	}
	t, err := tail.TailFile(lt.options.Path, tailConfig)
	if err != nil {
		return errors.Wrapf(err, "fail to tail file %q", lt.options.Path)
	}

	ctx, cancel := context.WithCancel(context.Background())
	quit := make(chan struct{})
	go func(ctx context.Context) {
		for {
			select {
			case <-ctx.Done():
				quit <- struct{}{}
				return
			case l := <-t.Lines:
				err := lt.sendLine(l.Text)
				if err != nil {
					logrus.Warnf("%+v", err)
				}
			}
		}
	}(ctx)

	stopCh := make(chan os.Signal, 1)
	signal.Notify(stopCh, syscall.SIGINT, syscall.SIGTERM)
	sigInfo := <-stopCh
	fmt.Println(sigInfo)

	cancel()
	t.Cleanup()

	<-quit
	fmt.Println("Goodbye")

	return nil
}

func (lt *LogTail) sendLine(string) error

func (lt *LogTail) sendLine(logLine string) error {
	// batch send
	if len(lt.buffer) < 200 {
		lt.buffer = append(lt.buffer, logLine)
		return nil
	}
	var items []auditv1.Event
	for _, line := range lt.buffer {
		event := auditv1.Event{}
		if err := json.Unmarshal([]byte(line), &event); err != nil {
			logrus.Errorf("%+v", errors.Wrapf(err, "fail to decode as json format for %q", line))
			continue
		}
		items = append(items, event)
	}
	eventList := &auditv1.EventList{
		Items: items,
	}
	if err := PostAuditToReportServer(eventList); err != nil {
		return err
	}
	lt.cleanBuffer()
	return nil
}

[nxadm]

I marked it as a bug (to be investigated and tested).

[nxadm]

Hi @wuxler,

I don't know your application, but I tried to simulate the flow with examples/03-consumeJSON.

As you can see I create JSON objects in a tight loop, magnitudes faster then the tail consumer that unmarshalls the JSON. Because I can not reproduce the bug in these extreme circumstances, I will close the bug.

If you think the problem still exists in the tail library, can you adapt the example according to your situation and reopen the issue?