laravel/horizon

"You can't write against a read only replica"

Closed this issue · 11 comments

ernst77 commented
  • Horizon Version: 3.4.7
  • Laravel Version: 6.11.0
  • PHP Version: 7.4.0
  • Redis Driver & Version: predis/phpredis 1.1.1
  • Database Driver & Version: N/A

Description:

After some time I get multiple errors:

  • You can't write against a read only replica. {"exception":"[object] (Predis\\Response\\ServerException(code: 0): READONLY You can't write against a read only replica. at /var/www/vendor/predis/predis/src/Pipeline/Pipeline.php:102)

  • ERR Error running script (call to f_230148430cb441a76f6486ae084405a674e00385): @user_script:2: @user_script: 2: -READONLY You can't write against a read only replica.

  • ERR Error running script (call to f_230148430cb441a76f6486ae084405a674e00385): @user_script:2: @user_script: 2: -READONLY You can't write against a read only replica. {"exception":"[object] (Predis\\Response\\ServerException(code: 0): ERR Error running script (call to f_230148430cb441a76f6486ae084405a674e00385): @user_script:2: @user_script: 2: -READONLY You can't write against a read only replica. at /var/www/vendor/predis/predis/src/Client.php:370)

There is no slave Redis instances running (as far as I understand as I only initiate with horizon command). So I am not quite sure why this is happening...

As well queue is empty it seems like it's randomly throwing errors and then stops... and again...

From sentry.io errors I see that it was invoked with:

  • Invoked Artisan command: horizon:work -- input | 'horizon:work' redis --delay=0 --memory=128 --queue=default --sleep=3 --timeout=60 --tries=2 --supervisor='b396ac2bb269-E0QY:supervisor-1'

Strangely enough even after there errors, everything seems to be working just fine... it just randomly spits out a lot of these errors for few seconds...

Steps To Reproduce:

Start horizon with supervisord and leave it for few hours...

driesvints commented

Hey there,

Can you first please try one of the support channels below? If you can actually identify this as a bug, feel free to report back and I'll gladly help you out and re-open this issue.

Thanks!

gauravmak commented

Happening with me too. As per the attached Logs, it seems to be happening via horizon only. When HorizonCommand or SupervisorCommand is handled.

Someone has also tried a Stackoverflow question.

  • 👀1
gauravmak commented

@ErnestStaug Did you manage to solve this?

ernst77 commented

@ErnestStaug Did you manage to solve this?

Hi, I think my problem solved when I shut down supervisor who was running php worker along with horizon, both of them were trying to execute the queue at the same time...

gauravmak commented

Okay, I re-installed the Redis-server and things look better for the time being. Thanks.

ernst77 commented
  • 👍2
mattkenefick commented

Has anyone else had this issue with a better solution?

  • 👍1
thegarlynch commented

Reopen this and let's discuss general answer.

  • 👍1
Deleted user commented

I had this problem too. Any solution?

  • 👍1
AntonLindell commented

I am having the same issue. Caused both from horizon snapshot command and from normal (and totally random from what I can see) requests.

Running redis in docker with appendonly --yes option on. The whole stack is in Docker swarm.

steveoliver commented

Maybe related.

I encountered this message "You can't write against a read only replica" leading up to the discovery that php.ini had disable_functions including pcntl_signal.

Running php artisan horizon from the CLI I got this error:

Horizon started successfully.

In HorizonCommand.php line 50:

  Call to undefined function Laravel\Horizon\Console\pcntl_signal()

Removing pcntl_signal from the list of disabled functions, I was good to go.