CTFd/ctfcli

ctf challenge test

ColdHeat opened this issue · 2 comments

We should have a ctf challenge test command. Some ideas:

  • Test that Dockerfiles build
  • Test the created image
  • Test chal is solveable

I'm working on something like this at the moment, it needs a lot of testing and Docker support needs to be finished off (plus some of my additions to challenge.yml could be improved), but would you be interested in a PR and if so are there any specific requirements?
Right now what mine does is:

  • Builds the docker image
  • Runs the docker image
  • For each test defined:
    • Copies all files in the files field of challenge.yml into a temp folder
    • Copies the script into the folder as well
    • Runs the script (depending on the test type, either the script should print a valid flag, or return status code 0)

It does kind of have the caveat that you need to be able to trust the test scripts though. That hasn't really been a major problem in my case because I was always going to virtualise the process anyway

pve commented

Can you add an example of a challenge.yml file that you wanted tested, in particular with a docker image to be build? I have some ideas and resources.