[BUG] service can't be used with `extends` as it declare `depends_on`
r3nor opened this issue Β· 47 comments
Description
When trying to bring up a compose project that makes use of extends
feature, I am getting the following error:
service "nginx" can't be used with `extends` as it declare `depends_on`
Steps To Reproduce
- Have a docker-compose.yml project with extends, like this:
nginx:
extends:
file: ./nginx/compose.yml
service: nginx
container_name: "${SERVER_NAME:}-nginx"
env_file: .env
and the nginx/compose.yml:
version: "3.9"
services:
nginx:
image: nginx:1
restart: unless-stopped
depends_on:
- web_app
volumes:
- ./html:/var/www/html
-
docker compose up
-
See error: service "nginx" can't be used with
extends
as it declaredepends_on
Compose Version
2.24.6
Docker Environment
N/A
Anything else?
No response
This is the expected behavior, see https://github.com/compose-spec/compose-spec/blob/master/05-services.md#restrictions
This seems to be a new restriction added in 2.24.6. We have an extends
combined with depends_on
and everything was working fine with 2.24.5 and previous versions. Not sure why this wasn't flagged as a breaking change?
Yeah, same for me. Was working fine on 2.24.5
and broke on this new 2.24.6
release.
This makes reusing blocks much harder, as fragments cannot be used across files. As far as I know, using extends
was the only way to have reusable depends_on
blocks while allowing to separate services in multiple files and have a common file to extend from.
restriction already was implemented in 2.23.x, new compose parser used in 2.24.x prior to latest release was just missing this check.
@meriouma maybe you want to check out include: https://docs.docker.com/compose/compose-file/14-include/
@meriouma maybe you want to check out include: https://docs.docker.com/compose/compose-file/14-include/
That is not sufficient. It is common to do stuff like have an image like "background worker" and then have 3 instances of it - high-pri, medium-pri, and low-pri. You could then have the base depend on stuff like your DB and message bus and such, then just have 3 instances that extend the base. Now you have to repeat those dependencies over and over in the instantiations. This is definitely more degradation in compose.
We have used Compose for development purposes with a Kubernetes cluster for deployment. It deeply feels like Compose has been losing functionality while Kubernetes has been gaining it, and Kind is becoming more and more compelling to replace Compose from these sorts of things.
encountering this too. why is a breaking change being included in a patch release at all?
what's also frustrating here is that everything used to work as-intended before compose 2.24 and its use of a not-even-prod-ready compose-go project, but there wasn't really a clear alert that everything was going to break.
@davidlougheed compose-go/v2 is not "not-even-prod-ready". We released as RC as we anticipated the need for some additional refactoring, and don't want third party library to suffer from API changes.
THAT BEING SAID: as compose-go includes a consistency check, it would be able to detect reference to another service is orphaned after being used by extends
, such a check docker compose v1 didn't implemented (so this rule in the compose file format, inherited by the compose specification). With the updated architecture of compose-go/v2 we can now relax this constraint, and offer more flexibility to users (which still should read the docs before they complain about a feature ). I'll write a proposal in the compose-spec repo for this purpose, so we can get this both fixed in the docs and in the code
This is the expected behavior, see https://github.com/compose-spec/compose-spec/blob/master/05-services.md#restrictions
@ndeloof So, what is the supported way of extending services that depend on other services?
It's possible to merge depends_on "later":
$ find *.yaml -print -exec cat {} \;
compose.yaml
include:
- path:
- defs.yaml
- deps.yaml
defs.yaml
services:
x:
image: x
y:
image: y
# depends_on:
# - x
z:
extends: y
deps.yaml
services:
y:
depends_on:
- x
$ docker compose config
name: deps
services:
x:
image: x
networks:
default: null
"y":
depends_on:
x:
condition: service_started
required: true
image: "y"
networks:
default: null
z:
image: "y"
networks:
default: null
networks:
default:
name: deps_default
... and if you want to "inherit" the depends_on, deps.yaml
could be like this:
services:
y: &idem
depends_on:
- x
z: *idem
Got this error with ubuntu update
from docker-compose-plugin (2.24.6-1ubuntu.22.04jammy)
to (2.24.5-1ubuntu.22.04jammy)
It's quite straightforward to see why is was not supposed to work if you consider such a case:
With a compose.yaml
like this:
services:
service:
extends:
file: file.yaml
service: service
that extends a service defined in a file.yaml
like this:
services:
hidden:
image: hidden
service:
image: image
depends_on:
- hidden
sure, but the same applies to other resources, without such a constraint:
file.yaml
services:
service:
image: image
volumes:
- hidden:/work
volumes:
hidden: ...
about that, extending services defined like this: https://docs.docker.com/compose/compose-file/05-services/#ipv4_address-ipv6_address doesn't work either, but it's not caught by checks if there's a homonym network around
that's my point: consistency check just validate there's a resource for any declared reference, whenever this might not have been declared in the original compose file. As we already do for networks/volumes, I don't see any reason we don't do the same for dependent services
hmm I removed my extends
in favour of includes + merging and now I get bizarre network errors where services are not joining some (but not all) of the declared networks, leading to name resolution errors. maybe related, or maybe not. curious if anyone else encounters this.
Edit: ah no it's unrelated, I'm also encountering a different compose-go bug: #11533
This is the cleanest solution I found for this issue:
services:
app-abstract:
image: nginx
profiles: [abstract]
app-1:
extends: app-abstract
profiles: !reset []
depends_on: ...
app-2:
extends: app-abstract
profiles: !reset []
depends_on: ...
But it's far from ideal having to add and reset profiles
like this, not to mention VS Code does not like the !reset
thing.
This is to say that there is no clean solution for this issue, one that would still allow to extend services within the same compose file without having to create extra compose files.
Perhaps, @ndeloof, you can consider allowing services to be marked as abstract: true
. Such services would not be rendered by docker compose, they'd rather just be useful for other services extend from.
Does the change mean that affected depends_on
never actually worked and can therefore be safely removed? What did compose previously do when it encountered such an depends_on
?
Extending depends_on was working before this change.
I want to reuse another developers service without making them change their code. If I 'include' I am stuck with their implementation. why can't I just overwrite with my own 'depends_on' and move on?
BTW I don't think you were able to overwrite depends_on
before. What you happen is that their depends_on
would be merged with yours.
@ndeloof please take a look how many linked issues this change produced. I vote for reverting this behavior. Πy company also has tens of projects with currently broken depends_on
@dmitryuk already in progress: compose-spec/compose-spec#470
This issue was closed two weeks ago as not planned
.
Shouldn't it be reopened, now that the relevant restriction in compose-spec
has been lifted?
Thanks @ndeloof for fixing the spec.
The change broke my project as I have the base docker-compose.yaml
which declares my API server (which depends_on
a database image) built from the project source (ie: compiling code) and a .devcontainer/docker-compose-dev.yaml
which extends
the base service, but changes the build.target
so that an image is used that has all the developer dependencies and mounts the source files.
This used to work so that the developer image/container would still depend on the DB which worked well for the development team.
Now that the spec has been updated, is there a projected timeline on updating the compose project to meet the revised spec?
will be part of tomorrow's release
Hello there, do we have an ETA for the version 2.25 to be released on Docker Desktop? My company heavily depends on this fix, for now I workaround the issue by download docker-compose instead and working with it, but the entire company is trained on the platform to use docker compose - would be nice to have the fix in place.
@ndeloof does this mean we should stop extending services as this feature will be deprecated in future?
@matthiasPOE no, we relaxed this restriction in the spec compose-spec/compose-spec#470
issue is closed by release v2.26.0
Hello there, do we have an ETA for the version 2.25 to be released on Docker Desktop? My company heavily depends on this fix, for now I workaround the issue by download docker-compose instead and working with it, but the entire company is trained on the platform to use docker compose - would be nice to have the fix in place.
I was looking for the directory where this plugin is located to replace it manually. After several unsuccessful attempts, I replaced docker-compose
in all found directories and one of them worked. I don't know which one, but you can also try it yourself.
https://github.com/docker/compose#linux
Don't rely on the about page, it still shows the old version:
docker compose version
# Docker Compose version v2.26.0
/usr/lib/docker/cli-plugins/docker-compose
@gander are you referring to OSX Docker Desktop? That location does not exist in my version of Sonoma.
@gander are you referring to OSX Docker Desktop? That location does not exist in my version of Sonoma.
No, my Linux Mint location. Check other from https://github.com/docker/compose or use find
or command
to locate your docker-compose
Yeah, tnx. I was able to follow the original install instructions from Docker.
Any idea about when the fix will be released in OSX Docker Desktop?
should happen in a few days
In case someone else is facing issues because Docker Desktop is still running an old version of the docker-compose plugin, I've found this useful video explaining how to easily upgrade the docker-compose plugin used by Docker Desktop.
@beni0888 you didn't link the video π
You're right! Fixed!!