rockthejvm/spark-essentials

Can not run ./build-images.sh

BaoAn21 opened this issue · 3 comments

When I run ./build-images.sh, I got error
failed to solve: process "/bin/sh -c export PATH="/usr/local/sbt/bin:$PATH" && apt update && apt install ca-certificates wget tar && mkdir -p "/usr/local/sbt" && wget -qO - --no-check-certificate "https://github.com/sbt/sbt/releases/download/v1.9.6/sbt-1.9.6.tgz\" | tar xz -C /usr/local/sbt --strip-components=1 && sbt sbtVersion -Dsbt.rootdir=true" did not complete successfully: exit code: 1
I think this error occur when running Scala instalation step in Dockerfile.
I am using Mac M1 chip, is that relate with this error? Thank for your help.

There's a trailing backslash \ in the URL, which is not present in the dockerfile

Thanks for your reply, a trailing backslash \ appear because my terminal auto add in, but in the dockerfile, everything is normal (I cloned your repo and run right away)
Might I paste full error log in here?. Thank you for your help.

=> ERROR [7/8] RUN export PATH="/usr/local/sbt/bin:/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" && apt upda 6.1s

[7/8] RUN export PATH="/usr/local/sbt/bin:/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" && apt update && apt install ca-certificates wget tar && mkdir -p "/usr/local/sbt" && wget -qO - --no-check-certificate "https://github.com/sbt/sbt/releases/download/v1.9.6/sbt-1.9.6.tgz" | tar xz -C /usr/local/sbt --strip-components=1 && sbt sbtVersion -Dsbt.rootdir=true:
#0 0.253
#0 0.253 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#0 0.253
#0 0.936 Hit:1 http://ports.ubuntu.com/ubuntu-ports jammy InRelease
#0 1.195 Get:2 http://ports.ubuntu.com/ubuntu-ports jammy-updates InRelease [119 kB]
#0 2.424 Hit:3 http://ports.ubuntu.com/ubuntu-ports jammy-backports InRelease
#0 2.730 Get:4 http://ports.ubuntu.com/ubuntu-ports jammy-security InRelease [110 kB]
#0 3.122 Get:5 http://ports.ubuntu.com/ubuntu-ports jammy-updates/universe arm64 Packages [1,208 kB]
#0 4.038 Get:6 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 Packages [1,368 kB]
#0 4.384 Fetched 2,806 kB in 4s (688 kB/s)
#0 4.384 Reading package lists...
#0 5.022 Building dependency tree...
#0 5.141 Reading state information...
#0 5.161 9 packages can be upgraded. Run 'apt list --upgradable' to see them.
#0 5.170
#0 5.170 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#0 5.170
#0 5.197 Reading package lists...
#0 5.782 Building dependency tree...
#0 5.895 Reading state information...
#0 6.013 wget is already the newest version (1.21.2-2ubuntu1).
#0 6.013 ca-certificates is already the newest version (20230311ubuntu0.22.04.1).
#0 6.013 Suggested packages:
#0 6.013 ncompress tar-scripts tar-doc
#0 6.037 The following packages will be upgraded:
#0 6.037 tar
#0 6.040 1 upgraded, 0 newly installed, 0 to remove and 8 not upgraded.
#0 6.040 Need to get 287 kB of archives.
#0 6.040 After this operation, 4,096 B of additional disk space will be used.
#0 6.040 Do you want to continue? [Y/n] Abort.


Dockerfile:30

28 |
29 | #Scala instalation
30 | >>> RUN export PATH="/usr/local/sbt/bin:$PATH" && apt update && apt install ca-certificates wget tar && mkdir -p "/usr/local/sbt" && wget -qO - --no-check-certificate "https://github.com/sbt/sbt/releases/download/v1.9.6/sbt-1.9.6.tgz" | tar xz -C /usr/local/sbt --strip-components=1 && sbt sbtVersion -Dsbt.rootdir=true
31 |
32 | RUN wget --no-verbose https://archive.apache.org/dist/spark/spark-${SPARK_VERSION}/spark-${SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}-scala${SCALA_VERSION_BASE}.tgz && tar -xvzf spark-${SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}-scala${SCALA_VERSION_BASE}.tgz \

ERROR: failed to solve: process "/bin/sh -c export PATH="/usr/local/sbt/bin:$PATH" && apt update && apt install ca-certificates wget tar && mkdir -p "/usr/local/sbt" && wget -qO - --no-check-certificate "https://github.com/sbt/sbt/releases/download/v1.9.6/sbt-1.9.6.tgz\" | tar xz -C /usr/local/sbt --strip-components=1 && sbt sbtVersion -Dsbt.rootdir=true" did not complete successfully: exit code: 1

Oh, I know where is my problem, I should change apt update to apt update -y to automatically accept any prompts.