ERROR: manifest for jupyter/pyspark-notebook:3.0.1 not found:
ronniemo opened this issue · 2 comments
Bringing up the structure on Ubuntu 20.04.2 LTS fails with:
ERROR: manifest for jupyter/pyspark-notebook:3.0.1 not found:
jupyter/pyspark-notebook:3.0.1 .
Sending build context to Docker daemon 4.608kB
Step 1/21 : ARG BASE_CONTAINER=jupyter/scipy-notebook
Step 2/21 : FROM $BASE_CONTAINER
latest: Pulling from jupyter/scipy-notebook
5d3b2c2d21bb: Pull complete
3fc2062ea667: Pull complete
75adf526d75b: Pull complete
72e7cf59352f: Pull complete
e9224c9534ef: Pull complete
ab0ac327f7ab: Pull complete
4f4fb700ef54: Pull complete
bcb880d8578e: Pull complete
248d9913118b: Pull complete
64fa47cb4705: Pull complete
ad93b6933f8d: Pull complete
86523fd9146f: Pull complete
26dc3f57c647: Pull complete
d263dfa0c17e: Pull complete
2462f3e06788: Pull complete
247e6f51cc7c: Pull complete
8b564ff5ca6e: Pull complete
4a1472a804e8: Pull complete
60ef0942b664: Pull complete
8ba85559ca44: Pull complete
6d486a1fdf72: Pull complete
73fe3e6f61ce: Pull complete
Digest: sha256:232ecb8da6abaa47ed41ef9c275ebced38245a1f35367f45b6d1b17e089bfb1e
Status: Downloaded newer image for jupyter/scipy-notebook:latest
---> 3ba02a671982
Step 3/21 : LABEL maintainer="Jupyter Project jupyter@googlegroups.com"
---> Running in 7c7b56655699
Removing intermediate container 7c7b56655699
---> a97ec53e55b4
Step 4/21 : SHELL ["/bin/bash", "-o", "pipefail", "-c"]
---> Running in 3a97dbc73954
Removing intermediate container 3a97dbc73954
---> 203e5063b4f4
Step 5/21 : USER root
---> Running in 85a785394e03
Removing intermediate container 85a785394e03
---> bce4fb5426fb
Step 6/21 : ARG spark_version="3.0.1"
---> Running in af859f2b5cd8
Removing intermediate container af859f2b5cd8
---> 6eee4dc9e7e3
Step 7/21 : ARG hadoop_version="2.7"
---> Running in a9122e4a6e6c
Removing intermediate container a9122e4a6e6c
---> ed747359c872
Step 8/21 : ARG spark_checksum="F4A10BAEC5B8FF1841F10651CAC2C4AA39C162D3029CA180A9749149E6060805B5B5DDF9287B4AA321434810172F8CC0534943AC005531BB48B6622FBE228DDC"
---> Running in 7d7331260a73
Removing intermediate container 7d7331260a73
---> 93c26bd9b9bc
Step 9/21 : ARG openjdk_version="11"
---> Running in b73f66858501
Removing intermediate container b73f66858501
---> c911bce1084a
Step 10/21 : ENV APACHE_SPARK_VERSION="${spark_version}" HADOOP_VERSION="${hadoop_version}"
---> Running in 05450387634f
Removing intermediate container 05450387634f
---> 1745a38fa7dd
Step 11/21 : RUN apt-get -y update && apt-get install --no-install-recommends -y "openjdk-${openjdk_version}-jre-headless" ca-certificates-java && apt-get clean && rm -rf /var/lib/apt/lists/*
---> Running in 6ea34575ff24
Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [109 kB]
Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:6 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1,275 kB]
Get:8 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [682 kB]
Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [197 kB]
Get:11 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [21.6 kB]
Get:12 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [717 kB]
Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Err:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages
Hash Sum mismatch
Hashes of expected file:
- Filesize:11344505 [weak]
- SHA256:200acdc3421757fa8f8759c1cedacae2cf8f5821d7810ca59d8a313c5b3ae71e
- SHA1:28dce64e724849c2296c10a2a3de8334c7829494 [weak]
- MD5Sum:fc2f03a69a8d61b33e29934cbba9e675 [weak]
Hashes of received file: - SHA256:033c734e5cc078888fb803bd2effc12e588c290116e533a53ce455a8f3a395d3
- SHA1:163288eb9dffbdd9f1ecdbd3b9e8be82058e943f [weak]
- MD5Sum:765487cfa301da7eb36c6e4225ebbf7a [weak]
- Filesize:11344505 [weak]
Last modification reported: Thu, 23 Apr 2020 17:34:17 +0000
Release file created at: Thu, 23 Apr 2020 17:33:17 +0000
Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [948 kB]
Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,128 kB]
Get:15 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [229 kB]
Get:16 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [29.6 kB]
Get:17 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [4,305 B]
Fetched 8,932 kB in 12s (731 kB/s)
Reading package lists...
E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal/universe/binary-amd64/by-hash/SHA256/200acdc3421757fa8f8759c1cedacae2cf8f5821d7810ca59d8a313c5b3ae71e Hash Sum mismatch
Hashes of expected file:
- Filesize:11344505 [weak]
- SHA256:200acdc3421757fa8f8759c1cedacae2cf8f5821d7810ca59d8a313c5b3ae71e
- SHA1:28dce64e724849c2296c10a2a3de8334c7829494 [weak]
- MD5Sum:fc2f03a69a8d61b33e29934cbba9e675 [weak]
Hashes of received file:
- SHA256:033c734e5cc078888fb803bd2effc12e588c290116e533a53ce455a8f3a395d3
- SHA1:163288eb9dffbdd9f1ecdbd3b9e8be82058e943f [weak]
- MD5Sum:765487cfa301da7eb36c6e4225ebbf7a [weak]
- Filesize:11344505 [weak]
Last modification reported: Thu, 23 Apr 2020 17:34:17 +0000
Release file created at: Thu, 23 Apr 2020 17:33:17 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.
Removing intermediate container 6ea34575ff24
The command '/bin/bash -o pipefail -c apt-get -y update && apt-get install --no-install-recommends -y "openjdk-${openjdk_version}-jre-headless" ca-certificates-java && apt-get clean && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100
Hi,
The repo was updated to use Spark 3.1.2. This issue should be fixed now.
https://github.com/cordon-thiago/airflow-spark/tree/airflow1.10.7_spark3.1.2