Add build info, such as image-builder version, as a file to built image
invidian opened this issue · 31 comments
Is your feature request related to a problem? Please describe.
As part of kubernetes-sigs/cluster-api-provider-aws#3107, while testing newly built images, we found out, that the images were built using older version of image-builder, which did not include some important patches.
It would be great if we could easily identify what version of image-builder was used to build the image. I could not find information or issue about such.
Describe the solution you'd like
It would be nice if built images could include information what version of image-builder were used for building this specific image. This is similar to #70, but I don't think version would have to be included in the image name. It could be a AMI resource tag or a special file on the root FS of the image with the information.
/kind feature
Hmm, it seems most recently published CAPA images does not have it:
$ aws ec2 describe-images --image-ids ami-046f5f86347c8d203 --region ap-northeast-1
{
"Images": [
{
"Architecture": "x86_64",
"CreationDate": "2022-02-18T08:45:21.000Z",
"ImageId": "ami-046f5f86347c8d203",
"ImageLocation": "258751437250/capa-ami-flatcar-stable-1.23.4-00-1645173023",
"ImageType": "machine",
"Public": true,
"OwnerId": "258751437250",
"PlatformDetails": "Linux/UNIX",
"UsageOperation": "RunInstances",
"State": "available",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/xvda",
"Ebs": {
"DeleteOnTermination": true,
"Iops": 3000,
"SnapshotId": "snap-08b8908cc7c84dc7f",
"VolumeSize": 8,
"VolumeType": "gp3",
"Throughput": 125,
"Encrypted": false
}
},
{
"DeviceName": "/dev/xvdb",
"VirtualName": "ephemeral0"
}
],
"Description": "Cluster API base image designed for v1.23.4",
"EnaSupport": true,
"Hypervisor": "xen",
"Name": "capa-ami-flatcar-stable-1.23.4-00-1645173023",
"RootDeviceName": "/dev/xvda",
"RootDeviceType": "ebs",
"SriovNetSupport": "simple",
"VirtualizationType": "hvm"
}
]
}
Neither AMI or snapshot have no tags from what I see.
@sedefsavas do you have some idea why the tags are not present? Perhaps I should move this issue to CAPA repo?
Cc @Ankitasw can this be because you were using the main branch?
The only reasons I know off the top of my head why it wouldn't show up are:
- the machine you are building from doesn't have
gitinstalled (extremely unlikely) - you are building by calling
packerinstead of callingmake
@Ankitasw @sedefsavas if you are available, let's chat about this next week. I'd like to get to the bottom of it, because taking a look again from the describe-images command that @invidian demonstrated, I see that none of the recently published CAPA images have any tags on them at all. This doesn't make any sense as tags are always applied to each build here:
image-builder/images/capi/packer/ami/packer.json
Lines 46 to 57 in 8b6dd4a
It's not an optional config.
So I'd really like to see what's happening and how it's getting skipped. Right now it's a mystery to me! =)
Cc @Ankitasw can this be because you were using the main branch?
I am not sure if anything is changed in the main branch of image-builder.
The only reasons I know off the top of my head why it wouldn't show up are:
the machine you are building from doesn't have git installed (extremely unlikely)
you are building by calling packer instead of calling make
We have git installed in the machine, also I used make command to build the images. Even I am not sure what went wrong, only difference from last time we did is we used main branch to build images.
I am not sure if describe-images command is supposed to give the tags in output or not, but I can see the tags in all AMIs published in the AWS console of the account where AMIs are getting published.
cc @codenrhoden @sedefsavas
I don't think we can rely on describe-images output for tags as it doesn't give any tags info, I have confirmed the same for older images as well.
@Ankitasw do you know if tags are kept private to the user that built the image perhaps?
I built an AMI this weekend, and was able to see all the expected tags for my own AMI, but not for the AMI's published for CAPA. Are you able to see the tags via the AWS console perhaps?
Here's what I built:
$ aws ec2 describe-images --image-ids ami-0478758581673cf71 --region us-west-2 --output json
{
"Images": [
{
"Architecture": "x86_64",
"CreationDate": "2022-03-05T16:37:37.000Z",
"ImageId": "ami-0478758581673cf71",
"ImageLocation": "053033561758/capa-ami-amazon-2-1.21.10-00-1646497937",
"ImageType": "machine",
"Public": false,
"OwnerId": "053033561758",
"PlatformDetails": "Linux/UNIX",
"UsageOperation": "RunInstances",
"State": "available",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/xvda",
"Ebs": {
"DeleteOnTermination": true,
"Iops": 3000,
"SnapshotId": "snap-088f5f889103dc31a",
"VolumeSize": 8,
"VolumeType": "gp3",
"Throughput": 125,
"Encrypted": false
}
}
],
"Description": "Cluster API base image designed for v1.21.10",
"EnaSupport": true,
"Hypervisor": "xen",
"Name": "capa-ami-amazon-2-1.21.10-00-1646497937",
"RootDeviceName": "/dev/xvda",
"RootDeviceType": "ebs",
"SriovNetSupport": "simple",
"Tags": [
{
"Key": "build_timestamp",
"Value": "1646497937"
},
{
"Key": "kubernetes_cni_version",
"Value": "v0.8.7"
},
{
"Key": "build_date",
"Value": "2022-03-05T16:32:18Z"
},
{
"Key": "kubernetes_version",
"Value": "v1.21.10"
},
{
"Key": "distribution_version",
"Value": "2"
},
{
"Key": "source_ami",
"Value": ""
},
{
"Key": "distribution",
"Value": "Amazon Linux"
},
{
"Key": "distribution_release",
"Value": "Amazon Linux 2"
},
{
"Key": "containerd_version",
"Value": "1.6.0"
},
{
"Key": "image_builder_version",
"Value": "v0.1.11-97-g8b6dd4ae7"
}
],
"VirtualizationType": "hvm"
}
]
}Are you able to see the tags via the AWS console perhaps?
Yes I am able to see tags in console as you can see in snapshot attached in previous comment.
@Ankitasw do you know if tags are kept private to the user that built the image perhaps?
I dont think tags are kept private to anyone, but atleast in any of old CAPA AMIs as well, I don't see any tags attached in the describe-images output.
Thanks @Ankitasw. Apologies that I missed your screenshot -- when I was scrolling through I thought it was mine. 🤦
It's really interesting that the tags show up in some cases and not in others. Right now I don't have an explanation for this.
Hmm, from https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html#tag-restrictions:
When you tag public or shared resources, the tags you assign are available only to your AWS account; no other AWS account will have access to those tags. For tag-based access control to shared resources, each AWS account must assign its own set of tags to control access to the resource.
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
- After 90d of inactivity,
lifecycle/staleis applied - After 30d of inactivity since
lifecycle/stalewas applied,lifecycle/rottenis applied - After 30d of inactivity since
lifecycle/rottenwas applied, the issue is closed
You can:
- Mark this issue or PR as fresh with
/remove-lifecycle stale - Mark this issue or PR as rotten with
/lifecycle rotten - Close this issue or PR with
/close - Offer to help out with Issue Triage
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
/remove-lifecycle stale
This would still be super useful.
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
- After 90d of inactivity,
lifecycle/staleis applied - After 30d of inactivity since
lifecycle/stalewas applied,lifecycle/rottenis applied - After 30d of inactivity since
lifecycle/rottenwas applied, the issue is closed
You can:
- Mark this issue or PR as fresh with
/remove-lifecycle stale - Mark this issue or PR as rotten with
/lifecycle rotten - Close this issue or PR with
/close - Offer to help out with Issue Triage
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
/remove-lifecycle stale
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
- After 90d of inactivity,
lifecycle/staleis applied - After 30d of inactivity since
lifecycle/stalewas applied,lifecycle/rottenis applied - After 30d of inactivity since
lifecycle/rottenwas applied, the issue is closed
You can:
- Mark this issue or PR as fresh with
/remove-lifecycle stale - Mark this issue or PR as rotten with
/lifecycle rotten - Close this issue or PR with
/close - Offer to help out with Issue Triage
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
/remove-lifecycle stale
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
- After 90d of inactivity,
lifecycle/staleis applied - After 30d of inactivity since
lifecycle/stalewas applied,lifecycle/rottenis applied - After 30d of inactivity since
lifecycle/rottenwas applied, the issue is closed
You can:
- Mark this issue as fresh with
/remove-lifecycle stale - Close this issue with
/close - Offer to help out with Issue Triage
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
/remove-lifecycle stale
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
- After 90d of inactivity,
lifecycle/staleis applied - After 30d of inactivity since
lifecycle/stalewas applied,lifecycle/rottenis applied - After 30d of inactivity since
lifecycle/rottenwas applied, the issue is closed
You can:
- Mark this issue as fresh with
/remove-lifecycle stale - Close this issue with
/close - Offer to help out with Issue Triage
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
/remove-lifecycle stale
@invidian I'm not sure what more image-builder can do here. The image-builder version is being published along with the AMI as an AWS tag but as you pointed out from the documentation, there's no way of accessing that for public images. For any images you build yourself, the image-builder version is there and can be accessed both via the web UI and via describe-images but that isn't possible for the CAPA provided AMIs.
The only other thing I can think of is if CAPA sets the ami_description to include the ib_version when they build their AMIs. (cc @Ankitasw @wyike)
Do you mean add something more here https://github.com/kubernetes-sigs/image-builder/blob/master/images/capi/packer/ami/packer.json#L5 @AverageMarcus
Specifically I mean
I'm not sure how you currently build the AMIs but if you're already providing a PACKER_VARS_FILE you can update it to include a new ami_description property with something like:
Cluster API base image designed for {{user `kubernetes_semver`}} built with image-builder {{user `ib_version`}}
Alternatively, the image-builder version could be included into the AMI name but that feels messy to me unless you plan to build the same Kubernetes versions against different image-builder versions.
@invidian I'm not sure what more image-builder can do here.
For example, as I mentioned while opening this issue:
or a special file on the root FS of the image with the information.
🤦 Sorry, I completely missed that while focussing on the tagging discussion.
Yeah, I think it makes sense to include a build-info file or something similar that includes various details about the built image.
I'm going to update the issue title to be more specific to that if you don't mind.
/retitle Add build info, such as image-builder version, as a file to built image