bitrise-steplib/steps-amazon-s3-deploy

step fails with install error

Closed this issue · 24 comments

INFO[08:59:33] (awscli) isn't installed, installing...      
ERRO[08:59:37] brew install awscli failed -- out: () err: (exit status 1) 
INFO[08:59:37] Failed to install (awscli) with brew         
ERRO[08:59:37] Step (amazon-s3-deploy) failed, error: Failed to install Step dependency, error: exit status 1 

Does a rebuild help?

no

is it possible to enable debug output for this step?

Does this happen on your Mac or on Bitrise.io ?

Also, can you please attach the full step log, with its header and footer?

for comparison, our sample build which uses this step just completed ~20 mins ago:

+------------------------------------------------------------------------------+
| (10) amazon-s3-deploy                                                        |
+------------------------------------------------------------------------------+
| id: amazon-s3-deploy                                                         |
| version: 3.5.0                                                               |
| collection: https://github.com/bitrise-io/bitrise-steplib.git                |
| toolkit: bash                                                                |
| time: 2016-09-17T09:37:51-07:00                                              |
+------------------------------------------------------------------------------+
|                                                                              |
INFO[09:37:52] (awscli) isn't installed, installing...      
INFO[09:38:19]  * [OK] awscli installed          
INFO[09:38:19]  * [OK] Step dependency (awscli) installed, available. 
... REDACTED ...
  success
|                                                                              |
+----+--------------------------------------------------------------+----------+
| ✅  | amazon-s3-deploy                                             | 38 sec   |
+----+--------------------------------------------------------------+----------+

On bitrise.io. I had 2 failing builds in a row, 3rd is currently running.

On which stack?

8.0

Can you please add a Script step right before this step, with the content:

brew install awscli

Did it work?

Build's still running..

It fails with:

Error: Could not symlink bin/carthage
Target /usr/local/bin/carthage
already exists. You may want to remove it:
  rm '/usr/local/bin/carthage'

To force the link and overwrite all conflicting files:
  brew link --overwrite carthage

To list all files that would be deleted:
  brew link --overwrite --dry-run carthage
Error: Could not symlink bin/carthage
Target /usr/local/bin/carthage
already exists. You may want to remove it:
  rm '/usr/local/bin/carthage'

To force the link and overwrite all conflicting files:
  brew link --overwrite carthage

To list all files that would be deleted:
  brew link --overwrite --dry-run carthage

One of my build steps performs a manual install of carthage. Weird thing is that this used to work, maybe something has changed in homebrew?

Yes, I definitely think so, but the CLI should print the same error automatically.. We'll check why the error is not in the log

And of course, thanks for reporting the issue! We'll close it here and create a new one for the CLI, for the missing brew install error logs

Yes that would be helpful. I'll simply uninstall carthage before installing my own version. Homebrew is great but it's very tricky to get repeatable builds with it, since the versions are so volatile.

You're right, there's no easy way to do that with brew.. Just one note, the stack has a preinstalled Carthage version too, so if you're OK with that, you don't have to install it yourself

Can you please copy paste the script you use, so that we can reproduce this? Might help with the fix

Which script? The carthage installer? It's part of a makefile:

CARTHAGE := /usr/local/bin/carthage
CARTHAGE_VERSION := 0.17.2
CURL_OPTS := --fail --silent -L --retry 3

install_carthage:
    @set -e; \
    if [ ! -f $(CARTHAGE) ] || ([ `$(CARTHAGE) version` != "$(CARTHAGE_VERSION)" ] && [ `$(CARTHAGE) version` != "$(CARTHAGE_VERSION).0" ]); then \
        curl $(CURL_OPTS) https://github.com/Carthage/Carthage/releases/download/$(CARTHAGE_VERSION)/Carthage.pkg > Carthage.pkg; \     
        sudo installer -pkg Carthage.pkg -target / && rm Carthage.pkg; \
    fi

Strange.. We still try to reproduce the issue, for us brew does not fail to install awscli if we install Carthage the same way you did.

bitrise.yml:

---
format_version: 1.3.0
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git
workflows:
  test:
    steps:
    - script:
        inputs:
        - content: |
            #!/bin/bash
            set -ex
            CARTHAGE="/usr/local/bin/carthage"
            CARTHAGE_VERSION="0.17.2"
            CURL_OPTS="--fail --silent -L --retry 3"
            #
            curl $CURL_OPTS https://github.com/Carthage/Carthage/releases/download/$CARTHAGE_VERSION/Carthage.pkg > ./Carthage.pkg
            sudo installer -pkg Carthage.pkg -target /
            rm ./Carthage.pkg
    - script:
        deps:
          brew:
          - name: awscli
            bin_name: aws
        inputs:
        - content: |-
            #!/bin/bash
            set -ex
trigger_map:
- pattern: "*"
  is_pull_request_allowed: false
  workflow: test

output:

+------------------------------------------------------------------------------+
| (0) script                                                                   |
+------------------------------------------------------------------------------+
| id: script                                                                   |
| version: 1.1.3                                                               |
| collection: https://github.com/bitrise-io/bitrise-steplib.git                |
| toolkit: bash                                                                |
| time: 2016-09-17T22:46:51-07:00                                              |
+------------------------------------------------------------------------------+
|                                                                              |
+ CARTHAGE=/usr/local/bin/carthage
+ CARTHAGE_VERSION=0.17.2
+ CURL_OPTS='--fail --silent -L --retry 3'
+ curl --fail --silent -L --retry 3 https://github.com/Carthage/Carthage/releases/download/0.17.2/Carthage.pkg
+ sudo installer -pkg Carthage.pkg -target /
installer: Package name is Carthage
installer: Installing at base path /
installer: The install was successful.
+ rm ./Carthage.pkg
|                                                                              |
+----+--------------------------------------------------------------+----------+
| ✅  | script                                                       | 5.3 sec  |
+----+--------------------------------------------------------------+----------+

                                          ▼

+------------------------------------------------------------------------------+
| (1) script                                                                   |
+------------------------------------------------------------------------------+
| id: script                                                                   |
| version: 1.1.3                                                               |
| collection: https://github.com/bitrise-io/bitrise-steplib.git                |
| toolkit: bash                                                                |
| time: 2016-09-17T22:46:54-07:00                                              |
+------------------------------------------------------------------------------+
|                                                                              |
INFO[22:46:55] (awscli) isn't installed, installing...      
INFO[22:47:15]  * [OK] awscli installed          
INFO[22:47:15]  * [OK] Step dependency (awscli) installed, available. 
|                                                                              |
+----+--------------------------------------------------------------+----------+
| ✅  | script                                                       | 22 sec   |
+----+--------------------------------------------------------------+----------+

Maybe brew has some logic which does not print or perform this link check unless a certain amount of time elapses?..

@jberkel we could finally reproduce the "no error output" issue, although not the way it happened for you (we just uninstalled brew, echo 'hi' > /usr/local/bin/carthage then try to install it again - brew gives an error)

related fix PR: bitrise-io/go-utils#33

@jberkel thanks for reporting the issue!
In the next Bitrise CLI release you'll get a better error output, something like:

INFO[09:16:06] (carthage) isn't installed, installing...
ERRO[09:16:10] brew install carthage failed -- out: (==> Downloading https://homebrew.bintray.com/bottles/carthage-0.18.el_capitan.bottle.tar.gz
Already downloaded: /Users/viktorbenei/Library/Caches/Homebrew/carthage-0.18.el_capitan.bottle.tar.gz
==> Pouring carthage-0.18.el_capitan.bottle.tar.gz
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/carthage
Target /usr/local/bin/carthage
already exists. You may want to remove it:
  rm '/usr/local/bin/carthage'

To force the link and overwrite all conflicting files:
  brew link --overwrite carthage

To list all files that would be deleted:
  brew link --overwrite --dry-run carthage

Possible conflicting files are:
/usr/local/bin/carthage
==> Caveats
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

zsh completion has been installed to:
  /usr/local/share/zsh/site-functions

fish completion has been installed to:
  /usr/local/share/fish/vendor_completions.d
==> Summary
🍺  /usr/local/Cellar/carthage/0.18: 48 files, 12.2M) err: (exit status 1)
INFO[09:16:10] Failed to install (carthage) with brew
ERRO[09:16:10] Step (script) failed, error: Failed to install Step dependency, error: exit status 1

Fix will be on Bitrise CLI master once this PR is merged: bitrise-io/bitrise#411

No such file or directory - /usr/local/Cellar/git/2.25.0
I was trying to install ImageMagick using brew-install. Is there any workaround for that?

Hi @rubaiyat6370
Can you please enable the Support Access for your project and send the build URL?

Thanks
Chaitanya