riverqueue/river

Intermittent test failure: `Test_Client_Stop/with_continual_insertion,_no_jobs_are_left_running` `i/o timeout`

brandur opened this issue · 0 comments

Another Test_Client_Stop/with_continual_insertion,_no_jobs_are_left_running problem, but this one much more rare, and appears to occur from test related code:

--- FAIL: Test_Client_Stop (0.00s)
    --- FAIL: Test_Client_Stop/with_continual_insertion,_no_jobs_are_left_running (0.35s)
        logger.go:225: time=2024-03-27T10:20:31.72[9](https://github.com/riverqueue/river/actions/runs/8450148836/job/23145728595?pr=288#step:8:10)Z level=INFO msg="Notifier: Listener connecting"
        logger.go:225: time=2024-03-27T[10](https://github.com/riverqueue/river/actions/runs/8450148836/job/23145728595?pr=288#step:8:11):20:31.729Z level=INFO msg="Elector: Listening for leadership changes" client_id=fv-az1024-25_2024_03_27T10_20_31 topic=river_leadership
        logger.go:225: time=2024-03-27T10:20:31.729Z level=INFO msg="Notifier: Listening on topic" topic=river_leadership
        logger.go:225: time=2024-03-27T10:20:31.731Z level=INFO msg="Notifier: Listening on topic" topic=river_insert
        logger.go:225: time=2024-03-27T10:20:31.731Z level=INFO msg="Notifier: Listening on topic" topic=river_leadership
        logger.go:225: time=2024-03-27T10:20:31.731Z level=INFO msg="Elector: Attempting to gain leadership" client_id=fv-az1024-25_2024_03_27T10_20_31
        logger.go:225: time=2024-03-27T10:20:31.731Z level=INFO msg="Notifier: Listening on topic" topic=river_job_control
        logger.go:225: time=2024-03-27T10:20:31.732Z level=INFO msg="River client started" client_id=fv-az1024-25_2024_03_27T10_20_31
        logger.go:225: time=2024-03-27T10:20:31.731Z level=INFO msg="Client: Election change received" client_id=fv-az1024-25_2024_03_27T10_20_31 is_leader=false
        logger.go:225: time=2024-03-27T10:20:31.791Z level=INFO msg="Elector: Gained leadership" client_id=fv-az1024-25_2024_03_27T10_20_31
        logger.go:225: time=2024-03-27T10:20:31.791Z level=INFO msg="Client: Election change received" client_id=fv-az1024-25_2024_03_27T10_20_31 is_leader=true
        logger.go:225: time=2024-03-27T10:20:31.791Z level=INFO msg="Reindexer: Scheduling first run" next_run_at=2024-03-28T00:00:00.000Z
        logger.go:225: time=2024-03-27T10:20:31.807Z level=INFO msg="Notifier: Listener closing"
        logger.go:225: time=2024-03-27T10:20:31.807Z level=INFO msg="Elector: Attempting to resign leadership" client_id=fv-az1024-25_2024_03_27T10_20_31
        logger.go:225: time=2024-03-27T10:20:31.853Z level=INFO msg="Elector: Resigned leadership successfully" client_id=fv-az1024-25_2024_03_27T10_20_31
        logger.go:225: time=2024-03-27T10:20:31.862Z level=INFO msg="River client stopped" client_id=fv-az1024-25_2024_03_27T10_20_31
        client_test.go:526: 
            	Error Trace:	/home/runner/work/river/river/client_test.go:526
            	            				/opt/hostedtoolcache/go/1.21.8/x64/src/runtime/asm_amd64.s:1650
            	Error:      	Received unexpected error:
            	            	write failed: write tcp [12](https://github.com/riverqueue/river/actions/runs/8450148836/job/23145728595?pr=288#step:8:13)7.0.0.1:44238->127.0.0.1:5432: i/o timeout
            	Test:       	Test_Client_Stop/with_continual_insertion,_no_jobs_are_left_running
FAIL
FAIL	github.com/riverqueue/river	[18](https://github.com/riverqueue/river/actions/runs/8450148836/job/23145728595?pr=288#step:8:19).5[25](https://github.com/riverqueue/river/actions/runs/8450148836/job/23145728595?pr=288#step:8:26)s

Sample run:
https://github.com/riverqueue/river/actions/runs/8450148836/job/23145728595?pr=288