flatcar/flatcar-release-mirror

Images are updated on every script run

Closed this issue · 4 comments

It seems there is some logic to skip updates when image did not change, but it doesn't seem to work for me. Logs from download_file:

+ local url=https://stable.release.flatcar-linux.net/./amd64-usr/./2345.3.0/./flatcar_production_image.bin.bz2
+ local file=./flatcar_production_image.bin.bz2
+ local log=
+ [[ -n flatcar_production_image ]]
++ echo ./flatcar_production_image.bin.bz2
++ grep flatcar_production_image
+ [[ -z ./flatcar_production_image.bin.bz2 ]]
+ [[ -f ./flatcar_production_image.bin.bz2 ]]
++ caddy_etag ./flatcar_production_image.bin.bz2
++ local file=./flatcar_production_image.bin.bz2
+++ stat -c %Y ./flatcar_production_image.bin.bz2
++ local modtime=1589454620
+++ stat -c %s ./flatcar_production_image.bin.bz2
++ local size=20574208
+++ base36enc 1589454620
+++ local input=1589454620
+++ base36=($(echo {0..9} {a..z}))
++++ echo 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z
++++ bc
+++ for i in $(bc <<< "obase=36; $input")
+++ echo -n q
+++ for i in $(bc <<< "obase=36; $input")
+++ echo -n a
+++ for i in $(bc <<< "obase=36; $input")
+++ echo -n b
+++ for i in $(bc <<< "obase=36; $input")
+++ echo -n j
+++ for i in $(bc <<< "obase=36; $input")
+++ echo -n 1
+++ for i in $(bc <<< "obase=36; $input")
+++ echo -n 8
+++ echo
++ local modtime36=qabj18
+++ base36enc 20574208
+++ local input=20574208
+++ base36=($(echo {0..9} {a..z}))
++++ echo 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z
++++ bc
+++ for i in $(bc <<< "obase=36; $input")
+++ echo -n c
+++ for i in $(bc <<< "obase=36; $input")
+++ echo -n 8
+++ for i in $(bc <<< "obase=36; $input")
+++ echo -n z
+++ for i in $(bc <<< "obase=36; $input")
+++ echo -n 5
+++ for i in $(bc <<< "obase=36; $input")
+++ echo -n s
+++ echo
++ local size36=c8z5s
++ printf '"%s%s"\n' qabj18 c8z5s
+ local 'etag="qabj18c8z5s"'
++ curl https://stable.release.flatcar-linux.net/./amd64-usr/./2345.3.0/./flatcar_production_image.bin.bz2 -I -H 'If-None-Match: "qabj18c8z5s"' --location --retry 5 --silent -f -S
+ log='HTTP/2 200
server: nginx/1.17.4
date: Thu, 14 May 2020 11:11:30 GMT
content-type: application/octet-stream
content-length: 489265127
etag: "q6hc7a83andz"
last-modified: Sat, 29 Feb 2020 20:13:58 GMT
accept-ranges: bytes
'
+ [[ 0 != \0 ]]
++ echo 'HTTP/2 200
server: nginx/1.17.4
date: Thu, 14 May 2020 11:11:30 GMT
content-type: application/octet-stream
content-length: 489265127
etag: "q6hc7a83andz"
last-modified: Sat, 29 Feb 2020 20:13:58 GMT
accept-ranges: bytes
'
++ grep '304 Not Modified'
+ local notmodified=
+ [[ ! -z '' ]]
+ echo -n +
++ echo 'Updating https://stable.release.flatcar-linux.net/./amd64-usr/./2345.3.0/./flatcar_production_image.bin.bz2'
Updating https://stable.release.flatcar-linux.net/./amd64-usr/./2345.3.0/./flatcar_production_image.bin.bz2
++ curl https://stable.release.flatcar-linux.net/./amd64-usr/./2345.3.0/./flatcar_production_image.bin.bz2 -R -o ./flatcar_production_image.bin.bz2 --location --retry 5 --silent -f -S
^C++ rm /tmp/mirror-lock
++ trap - SIGTERM
++ kill -- -25007

Either the modtime of the file was changed (or not correctly set) or caddy got a new scheme to calculate the etag.

Check modtime with stat -c %Y FILE

Caddy returns etag: "q6hc7a83andz", but locally I get qabj18c8z5s calculated, as you see in the logs.

Seems the size is also off!? size=20574208

I may have provided wrong log snippet, because of #9 and running script multiple times. Anyway, it seems #6 contains a fix for this.