devopshq/artifactory-cleanup

Formatting Question - KeepLatestNVersionImagesByProperty

NFerrovia opened this issue · 3 comments

Thank you for developing this wonderful tool, it has been very useful.
Having said that, i am having some issues getting some of the docker rules to work.

For example, my yaml file looks like this:

artifactory-cleanup:
  server: $SERVER
  user: $USER
  password: $PASS

  policies:
     - name: Test
       rules:
         - rule: RepoList
           repos:
             - docker-repo
         - rule: KeepLatestNVersionImagesByProperty
           count: 2

and im getting the following error after the result queries:

********************************************************************************
Found 1457 artifacts
Filter artifacts - rule: RepoList - Apply the policy to a list of repositories.
Filter artifacts - rule: KeepLatestNVersionImagesByProperty - Leaves ``count`` Docker images with the same major.
Traceback (most recent call last):
  File "/Users/nferrovia/Library/Python/3.8/bin/artifactory-cleanup", line 8, in <module>
    sys.exit(ArtifactoryCleanupCLI())
  File "/Users/nferrovia/Library/Python/3.9/lib/python/site-packages/plumbum/cli/application.py", line 177, in __new__
    return cls.run()
  File "/Users/nferrovia/Library/Python/3.9/lib/python/site-packages/plumbum/cli/application.py", line 634, in run
    retcode = inst.main(*tailargs)
  File "/Users/nferrovia/Library/Python/3.9/lib/python/site-packages/artifactory_cleanup/cli.py", line 121, in main
    for summary in cleanup.cleanup(
  File "/Users/nferrovia/Library/Python/3.9/lib/python/site-packages/artifactory_cleanup/artifactorycleanup.py", line 53, in cleanup
    artifacts_to_remove = policy.filter(artifacts)
  File "/Users/nferrovia/Library/Python/3.9/lib/python/site-packages/artifactory_cleanup/rules/base.py", line 284, in filter
    artifacts = rule.filter(artifacts)
  File "/Users/nferrovia/Library/Python/3.9/lib/python/site-packages/artifactory_cleanup/rules/docker.py", line 235, in filter
    grouped = pydash.group_by(artifacts, iteratee=_groupby)
  File "/Users/nferrovia/Library/Python/3.9/lib/python/site-packages/pydash/collections.py", line 397, in group_by
    key = cbk(value)
  File "/Users/nferrovia/Library/Python/3.9/lib/python/site-packages/artifactory_cleanup/rules/docker.py", line 232, in _groupby
    return self.get_version(artifact)[: self.number_of_digits_in_version]
  File "/Users/nferrovia/Library/Python/3.9/lib/python/site-packages/artifactory_cleanup/rules/docker.py", line 222, in get_version
    raise ValueError(f"Can not find version in '{artifact}'")
ValueError: Can not find version in '{'repo': ............

the error leads me to believe it is some mistake on my part defining the versions, but im not sure what im doing wrong.
Our Artifactory directory looks like this:

Artifactory
     docker-repo
           microservice1
                  v2.31.0 (dockerimage)
                        manifest.json
                        layer1
                        layer2
                        layer....
                  v2.32.0
                       ....
                       ....
                  v2.....
           microservice2
           microservice....
     other-repo1
     other-repo2

It help me a lot if i could get some guidance regarding the proper syntax! ty.

Hi! I think the supported version right now is with no v at the start - 2.32.0

Have a look at the tests https://github.com/devopshq/artifactory-cleanup/blob/master/tests/test_rules_docker.py

Try to add v in the custom_regexp or play with regexp a bit if it doesn't help

- rule: KeepLatestNVersionImagesByProperty
  count: 1
  custom_regexp: "v[^\\d][\\._]((\\d+\\.)+\\d+)"

Thank you! i've tried now with both and it seems like the problem lies possibly with repos that have repos with multiple denominations like some of them being v2.1... and some of them being flag 2.1... Ill test some more.

Same error here. Extending the regex to include a ^v makes it break a couple of lines further, since version is converted into a tuple of ints:

  File "/usr/local/lib/python3.9/site-packages/artifactory_cleanup/rules/docker.py", line 224, in get_version
    version = tuple(map(int, version_str.split(".")))
ValueError: invalid literal for int() with base 10: 'v1'