300 Second task timeout limit reached

[danshumaker]

Issue
Due to several lengthy acquia post-code-deploy hooks we are using that regenerate the search index for drupal. Our acquiacli jobs are taking longer than 5 minutes (300 seconds). Thus we get a timeout on the job

Notification in-progress                     
 4 mins [⚬⚬⚬⚬⚬⚬⚬⚬⚬⚬⚬⚬⚬⚬⚬⚬⚬⚬➤⚬⚬⚬⚬⚬⚬⚬⚬⚬]   0%I0521 15:03:09.999927      75 log.go:172] (0xc000676000) Data frame received for 5
 I0521 15:03:09.999957      75 log.go:172] (0xc00082a960) (5) Data frame handling
 I0521 15:03:09.999973      75 log.go:172] (0xc00082a960) (5) Data frame sent
  [error]  Task timeout of 300 seconds exceeded. 
 I0521 15:03:10.088292      75 log.go:172] (0xc000676000) (0xc000696000) Stream removed, broadcasting: 3
 I0521 15:03:10.088368      75 log.go:172] (0xc000676000) Data frame received for 1
 I0521 15:03:10.088378      75 log.go:172] (0xc000403f40) (1) Data frame handling
 I0521 15:03:10.088387      75 log.go:172] (0xc000403f40) (1) Data frame sent
 I0521 15:03:10.088394      75 log.go:172] (0xc000676000) (0xc000403f40) Stream removed, broadcasting: 1
 I0521 15:03:10.088646      75 log.go:172] (0xc000676000) (0xc00082a960) Stream removed, broadcasting: 5
 I0521 15:03:10.089299      75 log.go:172] (0xc000676000) Go away received
 I0521 15:03:10.089478      75 log.go:172] (0xc000676000) (0xc000403f40) Stream removed, broadcasting: 1
 I0521 15:03:10.089655      75 log.go:172] (0xc000676000) (0xc000696000) Stream removed, broadcasting: 3
 I0521 15:03:10.089753      75 log.go:172] (0xc000676000) (0xc00082a960) Stream removed, broadcasting: 5
 command terminated with exit code 1

Sollution Proposal & Question
I don't see anywhere that I can adjust the acquiacli ssh timeouts. Is it possible to make the timeout longer? I know I can use the -no-wait flag to make acquiacli return quickly but this is a CI job that needs to wait for a artifact:deploy job to finish before other jobs start acquiacli c:s (for example).

A nice option to have would be '--ssh-timeout'. Or perhaps change the way the task progress is determined and reported so that it doesn't have to keep a single connection open that long. Perhaps opening a shell to check on the task every thirty seconds or something. I have not tried the --progress-delay option. But that option seems to be only output related and not ssh related.

Describe alternatives you've considered
I've tried making the job faster by limiting the io with these options

--no-ansi
-n
--no-lock
-q

But those didn't eliminate the problem.

[typhonius]

Hi there, this isn't an SSH timeout but instead a task timeout. I opted to include one because it has been known in the past for Acquia tasks to hang indefinitely.

There are two ways that you can override this timeout to give you more time for your tasks to complete. The first is in the configuration either at the project level (in your project directory) or at the global level (in your ~/.acquiacli/acquiacli.yml). The key you'll be looking for there is timeout.

A newer (and as yet undocumented) way of achieving this override is to use an environment variable. This was introduced in 2.0.5 of the package though so you may need to either re-clone it or use self:update if you're on the phar version.