Running chmod on file results in 'text file busy' when running straight after.
GrahamDumpleton opened this issue ยท 46 comments
Dockerfile contents are:
FROM debian
WORKDIR /app
COPY . /app
CMD [ "/app/a.sh" ]
Contents of 'a.sh' are:
#!/bin/sh
echo 'A'
sh /app/b.sh 1
chmod +x /app/b.sh
/app/b.sh 2
The file a.sh is already set to be executable.
Contents of 'b.sh' are:
#!/bin/sh
echo 'B', $@
The file b.sh is not executable.
When image is built and then run one gets the error:
$ docker run --rm busy-text-file
A
B, 1
/app/a.sh: 5: /app/a.sh: /app/b.sh: Text file busy
What appears to be happening is that the execution of chmod +x b.sh just prior to trying to running b.sh is leaving it in the 'Text file busy' state for long enough that then running it fails.
Docker version information is:
Client version: 1.3.2
Client API version: 1.15
Go version (client): go1.3.3
Git commit (client): 39fa2fa
OS/Arch (client): darwin/amd64
Server version: 1.3.1
Server API version: 1.15
Go version (server): go1.3.3
Git commit (server): 4e9bbfa
Hit by another example of this when doing something entirely different.
chmod a+x /src/squid3-3.3.8/./configure
mkdir -p .
cd . && CFLAGS="-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall" CXXFLAGS="-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security" CPPFLAGS="-D_FORTIFY_SOURCE=2" LDFLAGS="-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now" /src/squid3-3.3.8/./configure --build=x86_64-linux-gnu --prefix=/usr --includedir="\${prefix}/include" --mandir="\${prefix}/share/man" --infodir="\${prefix}/share/info" --sysconfdir=/etc --localstatedir=/var --libexecdir="\${prefix}/lib/squid3" --srcdir=. --disable-maintainer-mode --disable-dependency-tracking --disable-silent-rules --datadir=/usr/share/squid3 --sysconfdir=/etc/squid3 --mandir=/usr/share/man --enable-inline --enable-async-io=8 --enable-storeio="ufs,aufs,diskd,rock" --enable-removal-policies="lru,heap" --enable-delay-pools --enable-cache-digests --enable-underscores --enable-icap-client --enable-follow-x-forwarded-for --enable-auth-basic="DB,fake,getpwnam,LDAP,MSNT,MSNT-multi-domain,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB" --enable-auth-digest="file,LDAP" --enable-auth-negotiate="kerberos,wrapper" --enable-auth-ntlm="fake,smb_lm" --enable-external-acl-helpers="file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,unix_group,wbinfo_group" --enable-url-rewrite-helpers="fake" --enable-eui --enable-esi --enable-icmp --enable-zph-qos --enable-ecap --enable-ssl --enable-ssl-crtd --disable-translation --with-swapdir=/var/spool/squid3 --with-logdir=/var/log/squid3 --with-pidfile=/var/run/squid3.pid --with-filedescriptors=65536 --with-large-files --with-default-user=proxy --enable-linux-netfilter
/bin/sh: 1: /src/squid3-3.3.8/./configure: Text file busy
The first time was with own custom build scripts and could come up with workaround.
This time was when trying to build squid from source code. Because it is a timing issue of sorts, is hit and miss as to whether works or not.
For the case of boot2docker on MacOS X where using Vagrant, this sort of issue appears to be more easily triggered.
This also happens when building containers with docker build.
$ docker --version
Docker version 1.2.0, build fa7b24f
Also happens on
Docker version 1.4.0, build 4595d4f
Seems like a workaround would be to sleep 1 between chmod and execution.
Same issue here:
$ docker version
Client version: 1.2.0
Client API version: 1.14
Go version (client): go1.2.1
Git commit (client): fa7b24f
OS/Arch (client): linux/amd64
Server version: 1.2.0
Server API version: 1.14
Go version (server): go1.2.1
Git commit (server): fa7b24f
The line it fails on looks like:
RUN chmod ugo+rx script.sh && ./script.sh
I tried upgrading docker to the latest available in the PPA repo (1.4.1) and I still have the same issue:
Client version: 1.4.1
Client API version: 1.16
Go version (client): go1.3.3
Git commit (client): 5bc2ff8
OS/Arch (client): linux/amd64
Server version: 1.4.1
Server API version: 1.16
Go version (server): go1.3.3
Git commit (server): 5bc2ff8
Futher details:
Ubuntu 14.10/x86_64 with all of the latest updates as of yesterday.
For some additional data points, I was unable to reproduce this for:
Ubuntu (using btrfs storage driver):
$ docker version
Client version: 1.3.2
Client API version: 1.15
Go version (client): go1.3.3
Git commit (client): 39fa2fa/1.3.2
OS/Arch (client): linux/amd64
Server version: 1.3.2
Server API version: 1.15
Go version (server): go1.3.3
Git commit (server): 39fa2fa/1.3.2
CentOS 7 (using devicemapper storage driver):
$ docker version
Client version: 1.3.2
Client API version: 1.15
Go version (client): go1.3.3
Git commit (client): 39fa2fa
OS/Arch (client): linux/amd64
Server version: 1.3.2
Server API version: 1.15
Go version (server): go1.3.3
Git commit (server): 39fa2fa
After upgrading the Ubuntu machine to 1.4.1 I was still unable to reproduce:
$ docker version
Client version: 1.4.1
Client API version: 1.16
Go version (client): go1.3.3
Git commit (client): 5bc2ff8
OS/Arch (client): linux/amd64
Server version: 1.4.1
Server API version: 1.16
Go version (server): go1.3.3
Git commit (server): 5bc2ff8
$ cat Dockerfile
from centos:centos7
add test.sh /test.sh
run chmod ugo+rx /test.sh && /test.sh
$ cat test.sh
#!/bin/bash
echo 'hello'
$ docker build -t docker-9547 --no-cache=true .
Sending build context to Docker daemon 3.072 kB
Sending build context to Docker daemon
Step 0 : FROM centos:centos7
---> 8efe422e6104
Step 1 : ADD test.sh /test.sh
---> 83dff9221423
Removing intermediate container 585ab447d89b
Step 2 : RUN chmod ugo+rx /test.sh && /test.sh
---> Running in 3480bbef19d7
hello
---> 6488f4d96729
Removing intermediate container 3480bbef19d7
Successfully built 6488f4d96729
I also get the "Text file busy" error message, however, during installation of a newer version of a package.
In my case the message looks like:
dpkg (subprocess): unable to execute old pre-removal script (/var/lib/dpkg/info/package-name.prerm): Text file busy
I belive the root-cause is probably same as described above.
I get this problem for:
distribution: Ubuntu 14.04
kernel: 3.13.0-45-generic
filesystem: ext4
docker version:
Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.4.1
Git commit (client): a8a31ef
OS/Arch (client): linux/amd64
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.4.1
Git commit (server): a8a31ef
and
distribution: Debian wheezy backport
kernel: 3.16.0-0.bpo.4-amd64
filesystem: xfs
docker version:
Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.4.1
Git commit (client): a8a31ef
OS/Arch (client): linux/amd64
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.4.1
Git commit (server): a8a31ef
Same issue:
/bin/sh: 1: /entrypoint.sh: Text file busy
Service 'web' failed to build: The command [/bin/sh -c chmod +x /entrypoint.sh && /entrypoint.sh] returned a non-zero code: 2`
Generated by:
FROM ruby:2.2.0
....
RUN chmod +x /entrypoint.sh && /entrypoint.sh
Version (boot2docker):
Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.4.1
Git commit (client): a8a31ef
OS/Arch (client): darwin/amd64
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.4.1
Git commit (server): a8a31ef
Workaround:
RUN chmod +x /entrypoint.sh; sleep 1; /entrypoint.sh
A (hopefully) better 'workaround' is to use sync:
RUN chmod +x /entrypoint.sh; sync; /entrypoint.sh
I tried syncon several positions in my Dockerfile, unfortunatly it didn't helped.
I was having a similar issue; switching to the 'overlay' storage driver seems to have resolved it.
I have same problem.
Could you add "docker info" in addition to "docker version" output, It shows more relevant details I think.
Containers: 3
Images: 46
Storage Driver: aufs
Root Dir: /mnt/sda1/var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 52
Execution Driver: native-0.2
Kernel Version: 3.18.5-tinycore64
Operating System: Boot2Docker 1.5.0 (TCL 5.4); master : a66bce5 - Tue Feb 10 23:31:27 UTC 2015
CPUs: 4
Total Memory: 1.961 GiB
Name: boot2docker
ID: NHSE:2QWD:3R7W:D6D3:U42I:T72B:N67E:MNRB:FZVZ:ZEFB:6QDU:SRAU
Debug mode (server): true
Debug mode (client): false
Fds: 12
Goroutines: 16
EventsListeners: 0
Init Path: /usr/local/bin/docker
Docker Root Dir: /mnt/sda1/var/lib/docker
Any ideas?
This error seems to happen at random...
docker version
Client version: 1.4.1
Client API version: 1.16
Go version (client): go1.3.3
Git commit (client): 5bc2ff8
OS/Arch (client): linux/amd64
Server version: 1.4.1
Server API version: 1.16
Go version (server): go1.3.3
Git commit (server): 5bc2ff8
docker info
Containers: 7
Images: 151
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Dirs: 165
Execution Driver: native-0.2
Kernel Version: 3.16.0-36-generic
Operating System: Ubuntu 14.10
CPUs: 8
Total Memory: 7.71 GiB
Name: ilustracion
ID: BPXK:DXGQ:UPUX:4RES:7HZK:PXNX:WKKA:SBKU:X7GU:VRRK:DFPF:BQ7C
WARNING: No swap limit support
So the detailed reports all have aufs backend in common.
I can add another output if needed:
$ docker info
Containers: 3
Images: 25
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Dirs: 31
Execution Driver: native-0.2
Kernel Version: 3.13.0-51-generic
WARNING: No swap limit support
$ docker version
Client version: 1.0.1
Client API version: 1.12
Go version (client): go1.2.1
Git commit (client): 990021a
Server version: 1.0.1
Server API version: 1.12
Go version (server): go1.2.1
Git commit (server): 990021a
Ubuntu 14.04LTS.
I've started to run into this problem as well. I'm running a docker build in a Vagrant VM.
I get this error
/bin/sh: 1: phpenv: Text file busy
from this command
INFO[0502] The command [/bin/sh -c sudo php-build -i development --pear 5.3.29 /opt/phpenv/versions/5.3.29 && sudo chown -R root:root /opt/phpenv && phpenv rehash && phpenv global 5.3.29 && ln -s /opt/phpenv/shims/php /usr/bin/php] returned a non-zero code: 2
docker info
Containers: 8
Images: 319
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 335
Dirperm1 Supported: false
Execution Driver: native-0.2
Kernel Version: 3.13.0-52-generic
Operating System: Ubuntu 14.04.2 LTS
CPUs: 4
Total Memory: 3.86 GiB
Name: vagrant-ubuntu-trusty-64
ID: 7XWM:7SE2:SYNI:6M6R:GIFW:5QZU:RDMN:SFEC:VWOR:QGAY:JYKM:S6FF
WARNING: No swap limit support
docker version
Client version: 1.6.2
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): 7c8fca2
OS/Arch (client): linux/amd64
Server version: 1.6.2
Server API version: 1.18
Go version (server): go1.4.2
Git commit (server): 7c8fca2
OS/Arch (server): linux/amd64
Using boot2docker on mac, and using the workaround 'sync' like this bellow, I haven't the 'text file busy error message'.
RUN chmod +x /root/post-install; sync && \
/root/post-install;
I also had this issue.
Using docker 1.7.1 with overlay.
Running sync in between fixed it.
Thanks!
Using sync helped me get passed this issue too. Thanks @dreuter !
I never saw this issue until I switched from boot2docker with VirtualBox to docker-machine with Fusion.
The same Dockerfile that would build without hiccups with the VirtualBoxified boot2docker VM would fail with "Text file busy" after trying to execute an injected bash script immediately after chmod +x'ing it when built against the VMware VM.
Either sleep 1s or a sync is a valid workaround for me ๐ .
sync also worked for me:
RUN chmod +x configure; sync; ./configure
This appears to be intermittent, so reports about sync or sleep fixing it might have been something else. I have a build that hits this problem, but a retry without any change was successful.
Got the error with titpetric/netdata on docker hub a few minutes ago, seems to be a still open issue. Never seen it with local builds.
Client version: 1.7.1
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 786b29d
OS/Arch (client): linux/amd64
All versions the same for server version also.
The same with the latest docker and aufs
Containers: 4
Images: 1026
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 1035
Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.19.0-39-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 8
Total Memory: 15.67 GiB
Name: jenkins-slave-2
ID: 6LLJ:FR3F:XWJE:BM2Q:LEEJ:2Y7S:RMDB:FWL6:QND5:5FGZ:7AXF:22MO
WARNING: No swap limit support
sync worked for me as well, thanks @dreuter
Client:
Version: 1.9.0
API version: 1.21
Go version: go1.4.3
Git commit: 76d6bc9
Built: Tue Nov 3 19:20:09 UTC 2015
OS/Arch: darwin/amd64
Server:
Version: 1.9.0
API version: 1.21
Go version: go1.4.3
Git commit: 76d6bc9
Built: Tue Nov 3 19:20:09 UTC 2015
OS/Arch: linux/amd64
I start to see this with:
Ubuntu 16.04
Linux 585271 4.4.0-22-generic #40-Ubuntu SMP Thu May 12 22:03:46 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
root@585271:~# docker info
Containers: 49
Running: 7
Paused: 0
Stopped: 42
Images: 76
Server Version: 1.11.1
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 439
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge null host
Kernel Version: 4.4.0-22-generic
Operating System: Ubuntu 16.04 LTS
OSType: linux
Architecture: x86_64
CPUs: 24
Total Memory: 125.9 GiB
Name: 585271
ID: N4F2:2LEE:3ZLQ:CVMM:I3MX:HAWA:24HQ:S2VB:KG33:32XP:FJLN:OFRG
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Dockerfile (end):
CMD ["/opt/setup_and_run.sh"]
/opt/setup_and_run.sh looks like:
...
chown -R $EXTERNAL_USER: /opt
exec su - $EXTERNAL_USER -s/bin/bash -c '/opt/solr-4.10.3/bin/solr start -p 8983 -s /opt/etc/solr-home/ -f -a "-Djetty.basedir=/opt/solr-4.10.3/"'
sync fixed it too.
Also I found out that
RUN chmod +x configure && ./configure
will fail whereas
RUN chmod +x configure && sh ./configure
or
RUN chmod +x configure && bash ./configure
... will work.
@nolazybits - chmod +x is redundant in this case as you invoke sh or bash respectively.
+1... Seeing this in docker 1.12.3 with Ubuntu 16.04.1 running in VirtualBox on a Windows 10 host.
$ docker info
Containers: 16
Running: 14
Paused: 0
Stopped: 2
Images: 37
Server Version: 1.12.3
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 96
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge null host overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor seccomp
Kernel Version: 4.4.0-38-generic
Operating System: Ubuntu 16.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 11.73 GiB
What was the official fix?
Step 9/33 : RUN chmod +x /opt/XXXXX.bin && cd /opt && /opt/XXXXX.bin && rm -f /opt/XXXXX.bin
---> Running in 1b078c01f0e7
**/bin/sh: 1: /opt/XXXXX.bin: Text file busy**
ERROR: Service 'XXXXXXXX' failed to build: The command '/bin/sh -c chmod +x /opt/XXXXX.bin && cd /opt && /opt/XXXXX.bin && rm -f /opt/XXXXX.bin' returned a non-zero code: 2
same error even after wait 30 ๐ , for below Docker
ADD XXXXX.bin /opt
RUN chmod +x /opt/XXXXX.bin && cd /opt && /opt/XXXXX.bin && rm -f /opt/XXXXX.bin
RUN wait 30
@nolazybits , bash works ๐ !! below Docker now works
ADD XXXXX.bin /opt
RUN chmod +x /opt/XXXXX.bin && cd /opt && /opt/XXXXX.bin bash && rm -f /opt/XXXXX.bin bash
@usowmyas do you mean
RUN cd /opt && bash /opt/XXXXX.bin && rm -f /opt/XXXXX.bin
(i.e. prefix with bash). In that case you don't need the chmod at all. (wait would need to go between chmod and execution, not after it).
As we are currently only using docker for testing our application (not production use), I would rather not have to change the application just because of a docker bug. I am running the following within the startup script as a workaround, and while it appears to work, it was an intermittent issue for me anyway so I'm not certain it's a good workaround. I would welcome any thoughts.
function chmod { command chmod $@; sync; }
export -f chmod
If anyone else is having the same issue I have found a solution to this and why it's causing issues. To fix it split your command across two RUN statements. In the example above RUN chmod +x configure && bash ./configure becomes
RUN chmod+x configure
RUN ./configure
The reason why this happens is each step in run in an intermediate container which for some reason does file locking causing the Text File Busy Error. Alternatively running in a subshell as above would also work
Hope this helps
@dreuter sync fixed it too. thanks
and the docker info is below:
Server Version: 17.09.1-ce
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 132
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 06b9cb35161009dcb7123345749fef02f7cea8e0
runc version: 3f2f8b84a77f73d38244dd690525642a72156c64
init version: 949e6fa
Kernel Version: 3.16.0-4-amd64
Operating System: Debian GNU/Linux 8 (jessie)
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.963GiB
Name: packer-debian-8-amd64
ID: FJK7:DAID:F5ET:BTV6:4VGP:C3V7:SSCS:ZUZ6:DHFN:N3DO:ANYM:B3E4
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Username: baykier
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No oom kill disable support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
same issue here, docker version 1.12.6 . sync fixed it.
I confirm that problem has not yet been solved, and for me the solution was to switch to overlay2 storage driver.
Docker info:
Containers: 4
Running: 2
Paused: 0
Stopped: 2
Images: 22
Server Version: 18.06.1-ce
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 0
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 468a545b9edcd5932818eb9de8e72413e616e86e
runc version: 69663f0bd4b60df09991c08812a60108003fa340
init version: fec3683
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.4.0-133-generic
Operating System: Linux Mint 18
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 15.16GiB
Name: titanio
ID: GXK2:QHUK:LYKW:2YSX:GI5W:N5MG:2AY4:TXYN:B2VW:MPP4:ZYVQ:O2VM
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Username: angryluke
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit support
Docker version:
Client:
Version: 18.06.1-ce
API version: 1.38
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:24:56 2018
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 18.06.1-ce
API version: 1.38 (minimum version 1.12)
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:23:21 2018
OS/Arch: linux/amd64
Experimental: false
I am seeing this (in RUN chmod a+x /usr/local/bin/script.sh && /usr/local/bin/script.sh) on one machine reliably while on four other machine exactly the same Dockerfile runs through just fine. The thing is that I've upgraded the docker installation from ancient 17.05 to 19.03.8, removed all old images and rebooted the machine and it's still showing that error.
The machines all have Linux 4.4.0, though slightly different builds. There is one significant difference though:
- The non-working machine uses aufs.
- The working machines all use overlay2.
So confirming this is an aufs issue.
If possible, I would recommend switching to the default (overlay2) storage driver. To switch, the easiest way is to "clean" the docker storage that's in use (stopping the docker service, and removing /var/lib/docker), which will remove all local docker-related data (images, containers, volumes). After that docker should pick the default storage driver (unless you manually configured a custom driver in your daemon.json or in the systemd unit file).