Add support for subdirectories when building from git source

[thaJeztah]

As noted in docker/build-push-action#120 (comment), specifying a sub-directory when building from a git source is not yet supported in BuildKit;

buildkit/source/gitidentifier.go

Lines 46 to 48 in 477b1aa

	if repo.Subdir != "" {
	return nil, errors.Errorf("subdir not supported yet")
	}

The PR that added this feature doesn't clearly mention if there's specific reasons for it not to be supported (perhaps there's technical limitations); #63

The classic builder support this https://github.com/docker/cli/blame/2955ece0244307bdbb626ad2471024a51080ff1a/docs/reference/commandline/build.md#L108-L111, so for feature-parity, we should add support in BuildKit as well

Build Syntax Suffix	Commit Used	Build Context Used
myrepo.git	refs/heads/master	/
myrepo.git#mytag	refs/tags/mytag	/
myrepo.git#mybranch	refs/heads/mybranch	/
myrepo.git#pull/42/head	refs/pull/42/head	/
myrepo.git#:myfolder	refs/heads/master	/myfolder
myrepo.git#master:myfolder	refs/heads/master	/myfolder
myrepo.git#mytag:myfolder	refs/tags/mytag	/myfolder
myrepo.git#mybranch:myfolder	refs/heads/mybranch	/myfolder

[tiborvass]

This would be great indeed, but if we want further optimizations in git protocol, I found git clone --filter available since git 2.19:
https://stackoverflow.com/questions/600079/how-do-i-clone-a-subdirectory-only-of-a-git-repository/52269934#52269934

However, in the code we are not using git clone, but git init + git remote + git fetch. The weird thing is that git fetch --filter is not documented (oversight?) but does exist in the code.

Another caveat is that --filter only works with repos that have explicit support for partial clone, greatly limiting the reach of this optimization. Unclear whether there's a tradeoff or why it's not default.

[turbolent]

Just ran into this, but was confused, because I wasn't using a subdirectory, I had specified :/.

Due to

buildkit/source/gitidentifier.go

Lines 46 to 48 in 477b1aa

	if repo.Subdir != "" {
	return nil, errors.Errorf("subdir not supported yet")
	}

, this is treated as a subdirectory, even though it just indicates the root of the repository. Removing this suffix fixed the problem.

Maybe the subdirectory check should be exteneded to allow "/"?

Hope this helps someone else who runs into this.

[thaJeztah]

Maybe the subdirectory check should be exteneded to allow "/"?

Sounds reasonable to ignore that case (my 0.02c); feel free to open a pull request for that

[kechaoyang01]

any progress for this issues? my apps contain multiple containers and subfolder, need this to be fixed.

[raton465]

busco el doker de tcp y udp

[CraigHead]

Second. Any traction on this? We keep our dockerfiles in a ci directory and this currently is blocking us outside of us moving 200 dockerfiles to the root directory of the repo.

[justincormack]

Sparse checkout now works on github see https://stackoverflow.com/questions/600079/how-do-i-clone-a-subdirectory-only-of-a-git-repository and https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/ - not sure where else though, it is not clear if we should have a fallback or fail if the server does not support it.