Enable e2e tests for Azure
feiskyer opened this issue · 25 comments
We should set up an e2e tests for this repo
- Setup e2e tests infra on Azure
- Enable e2e tests for PRs
- Switch to test-infra for presubmit e2e tests (presubmit tests are running for pull requests in cloud-provider-azure repo) @ritazh
- Add e2e steps to setup test-infra via Makefile
- Update presubmit tests to the new way
- Remove current perl scripts
- Enable full periodic e2e tests for stable releases (periodic tests are running in backgroud for each release braches)
- Add more test scenarios, e.g. conformance, correctness, alpha-features, autoscaling, scalability, multi-zone, slow and serial
- Enable e2e for more releases, e.g. v1.13 and v1.12
- Setup prow jobs for new test scenarios
- Add Azure features e2e testings #7
- LoadBalancer service tests with various annotations
- ACR image pulling tests without docker secrets setting explicitly
- Multi availability zones tests
- Cloud provider configuration options tests (e.g. Standard LoadBalancer and VMSS)
- Upgrading tests (e.g. cloud-provider version updates)
- Documentation for e2e @ritazh
/assign @karataliu
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale
/remove-lifecycle stale
/lifecycle frozen
What's the current status of this?
The e2e tests have been enabled for this repo, but there's no periodly e2e tests yet. We should enable periodical tests for master branches as well.
/remove-lifecycle frozen
/assign @feiskyer @ritazh
/unassign @karataliu
@feiskyer: GitHub didn't allow me to assign the following users: ritazh.
Note that only kubernetes members and repo collaborators can be assigned and that issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide
In response to this:
/assign @feiskyer @ritazh
/unassign @karataliu
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
I see the e2e job ci-cloud-provider-azure-master is part of testgrid: https://testgrid.k8s.io/sig-azure-master#azure-master-conformance as defined here: https://github.com/kubernetes/test-infra/blob/master/testgrid/config.yaml#L7578-L7582
There are two e2e jobs for cloud-provider-azure in prow:
presubmit job pull-cloud-provider-azure-e2e: https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/cloud-provider-azure/cloud-provider-azure-config.yaml#L50
periodic job ci-cloud-provider-azure-master : https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/cloud-provider-azure/cloud-provider-azure-config.yaml#L95
@feiskyer Can you please update the checklist in this issue's description to make sure they are up-to-date and clarify what is the gap? Thanks!
@ritazh presubmit jobs in for new pull requests, while periodic jobs are for running backgroud for each branches. Would update the description.
@feiskyer I would like to contribute to this project as a part of my GSOC candidacy. Please let me know something I can pick as a good first issue. Thanks in advance.
UPDATE: The task to update the e2e test infra to use kubetest directly is currently WIP.
@feiskyer I would like to contribute to this project as a part of my GSOC candidacy. Please let me know something I can pick as a good first issue. Thanks in advance.
@jaybatra26 Thanks for the interest. Are you familiar with Go and Azure? Most of works here are coupled with Azure functionalities.
@feiskyer , I am familiar with go, but not a lot of azure. I can try setting it up in my system. If there is any document I can follow to set up in local and test, I will sincerely appreciate it.
@jaybatra26 Could you start with #101? It's for changing the vendor tools to dep and not related to Azure.
@feiskyer Sure. Trying it.
not done yet.
/reopen
@feiskyer: Reopened this issue.
In response to this:
not done yet.
/reopen
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
Hey , I am familiar with Golang , kubernetes and went throught he code base. Earlier I contributed to updating AZURE INSTANCES. So, @feiskyer Can I take up this issue?
@Abhik1998 sure, feel free to claim the item you'd like to work on
I started up with the Makefile @feiskyer , its similiar to something like this.
shell:
ifeq ($(CMD_ARGUMENTS),)
# no command is given, default to shell
docker-compose -p $(PROJECT_NAME)$(HOST_UID) run --rm $(SERVICE_TARGET) sh
else
# run the command
docker-compose -p $(PROJECT_NAME)$(HOST_UID) run --rm $ (SERVICE_TARGET) sh -c "$(CMD_ARGUMENTS)"
endif
Regular Makefile part for buildpypi itself
help:
@echo ''
@echo 'Usage: make [TARGET] [EXTRA_ARGUMENTS]'
@echo 'Targets:'
@echo ' build build docker --image-- for current user:
@echo ' rebuild rebuild docker --image-- for current user:
@echo ' test test docker --container-- for current user:
@echo ' service run as service --container-- for current user:
@echo ' login run as service and login --container-- for current user:
@echo ' clean remove docker --image-- for current user:
@echo ' prune shortcut for docker system prune -af. Cleanup inactive containers and cache.'
@echo ' shell run docker --container-- for current user:
@echo ''
@echo 'Extra arguments:'
@echo 'cmd=: make cmd="whoami"'
@echo '# user= and uid= allows to override current user. Might require additional privileges.'
@echo 'user=: make shell user=root (no need to set uid=0)'
@echo 'uid=: make shell user=dummy uid=4000 (defaults to 0 if user= set)'
rebuild:
# force a rebuild by passing --no-cache
docker-compose build --no-cache $(SERVICE_TARGET)
service:
# run as a (background) service
docker-compose -p
login: service
# run as a service and attach to it
docker exec -it
build:
# only build the container. Note, docker does this also if you apply other targets.
docker-compose build $(SERVICE_TARGET)
clean:
# remove created images
@docker-compose -p
&& echo 'Image(s) for "$(PROJECT_NAME):$(HOST_USER)" removed.'
|| echo 'Image(s) for "$(PROJECT_NAME):$(HOST_USER)" already removed.'
prune:
# clean all that is not actively used
docker system prune -af
test:
# here it is useful to add your own customised tests
docker-compose -p
echo "I am whoami. My uid is id -u." && echo "Docker runs!"'
&& echo success
@Abhik1998 Could you send out a PR? By the way, we don't use docker-compose, docker is enough for building the images and running the tests.
Yes sure am sending a PR. Well, I sent that Docker type because I used that file earlier
Would be tracked in smaller issues.
/close
@feiskyer: Closing this issue.
In response to this:
Would be tracked in smaller issues.
/close
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.