system and integration testing

Question

system and integration testing

Opened this issue 11 years ago · 8 comments

While unit tests can detect breakage in individual units, it would be nice to know whether the whole system works together.

I think we can set up some virtual machines and do automated tests on testnet or testnet-in-a-box.

I'm not familiar with best practices in this area, so I'd like to know which options are available. (Just some bash/python scripts?)

Answer 1 · 2013-12-10T13:45:24.000Z

Squish seems like a decent possibility: http://stackoverflow.com/questions/4129430/qt-automated-testing

http://www.froglogic.com/squish/gui-testing/index.php

I can start on this after I finish up some more unit tests.

Answer 2 · 2013-12-10T13:50:08.000Z

Hm, it looks like we can get pretty far with just python unit tests: http://www.voom.net/pyqt-qtest-example

Answer 3 · 2013-12-10T15:16:50.000Z

Actually I'd rather test command-line (or RPC) versions first. GUI testing can be cumbersome,

Answer 4 · 2013-12-29T16:34:04.000Z

Can we close this?

Answer 5 · 2013-12-29T16:55:52.000Z

If I understand correctly, we have unit tests now, but they are no substitute for system tests (i.e. checking it live incl. interactions with bitcoind and so on).

Answer 6 · 2013-12-29T17:05:41.000Z

color-explorer-vagrant sets up a VM, downloads the blockchain, and I can have it do some simple color transactions. The problem is, to do some actual live color transactions, you'd have to feed it some test-coins every time you want to do a test. Wouldn't that be hard to automate?

Answer 7 · 2013-12-29T17:06:26.000Z

I guess it could feed the test coin back when it's done testing.

Answer 8 · 2013-12-29T19:34:30.000Z

test_wallet_model.py does actual integration testing using testnet and bitcoind.

p2p has mostly integrated tests in test_ewctrl and test_agent. let me know if you feel like that's not enough and I can make more stuff.