Testing, CI?
EivindArvesen opened this issue · 10 comments
It would be great to have some sort of testing, so that contributors don't have to manually perform every action/combination that could be affected by their changes, or worse yet, don't test at all.
It would probably be best to set up integration with Travis CI, or something, but I have no idea what options exist for testing with bash and zsh – except for shUnit2, which I recently stumbled upon.
I will look into writing some basic tests. Should they be in bash or is perl/python fine too ?
That would be great!
I would prefer bash or python.
On the one hand, keeping everything in the same language would be tidy (and ensure that contributors can deal with all parts), but on the other hand python has some well established frameworks.
What are your thoughts?
And what approach were you thinking of?
It makes sense to keep them in bash, at least for now.
The approach I am thinking of is:
- Keep all the tests divided logically into files (for example one file that tests start, one file that tests stop, one file that tests the environment variables and so on)
- Have a script that will run all of the files containing the tests.
How does it sound ?
It sounds great!
Don't be afraid to make commits and PRs often with regards to this issue, though.
It looks like I'll have some spare time this weekend, so I might have time to help out with this after fixing some other issues, if you'd like.
@IamBC : What is your current status on this?
I'm thinking about dedicating some time to writing tests over the weekend.
Sorry for the delay, I was caught up with some personal issues. I am going to write some tests to start things up tonight. I will make the pull request either tonight or Saturday morning latest.
I am very sorry, but my contributions will be delayed indefinitely. I will try to keep up with the progress of the project and at least try to give feedback on the new features.
No problem at all. I hope things work out.
I will certainly appreciate any feedback!
Just started out with shUnit2.
It works great, and both bash and zsh tests smoothly on my machine.
Travis CI, however, is another story. Here, zsh keeps getting an "Unknown failure" with shUnit2.
I'd be very grateful for any suggestions from someone with experience regarding zsh (and/or shUnit2) on Travis.
Migrated the tests from shUnit2 to BATS and made a few scripts.
Things finally work.
Testing should probably be mentioned in the wiki as well.