Testing elixir_ale applications, test harness mode?
Closed this issue · 10 comments
I was chatting with @GregMefford on slack#nerves this morning about the best may to test applications which use elixir_ale to communicate with hardware. He suggested I talk to you about adding a test mode to elixir_ale so that when running MIX_ENV=test mix test in my project I could compile it on my development system and test the functions and APIs of my application absent any attached hardware.
I haven't thought about this in detail but I was thinking that perhaps in a test environment elixir_ale GenServers could respond to messages by logging them or invoking some mock action. I'd love to hear your thoughts on this topic and perhaps I can explore implementing such a thing.
I started doing something similar for the GPIO module. Figured it was worth commenting to say I would use something like this!
Edit: what I cobbled together (...not saying it's pretty) - https://gist.github.com/loeschg/65621b2e3e699754cfff4b13bd748add
I haven't had a chance to review the slack channel yet today, but this need comes up frequently since it's so hard to test. Could one or both of you fork the elixir_ale repo and come up with a start? Maybe base it off of @loeschg's gist?
@adkron - since you've been thinking about unit testing with the grovepi repo, do you have thoughts on this?
I'd very much like to get something available here to help out testing, but haven't had a chance recently. Thank you all for posting about the topic.
What platforms are you using? I don't have elixir_ale compiling on OS X. I haven't tried that hard. I started to replace the I2C module with one I defined locally. In test I don't load the elixir_ale package. I then inject a replacement. I'm not sure if a great generic solution but we can talk about it. I have the idea spiked out on https://github.com/BinaryNoggin/grovepi/tree/button_real. Take a look in the test directory and the configs. This is rough work too. I'm just trying to figure it all out. Let me know if you have any questions. I'd love to solve this.
@adkron right now I'm stubbing out the elixir_ale API and only compiling it as a dependancy for :prod environment. There are clearly two things involved: 1) testing elixir_ale itself, and 2) testing an application which uses it.
In the second case I think stubbing out the API functions to return a positive response is the right way to go. There is no need to have elixir_ale trigger interrupts or send data to the application as elixir_ale has no knowledge of that domain. What matters, IMHO, is only that an API function was called and the arguments it was passed. For example, GPIO.start_link should return something like {:ok, :pid}. About the only thing we might want to add is an assertion or throwing an exception when the API is used incorrectly, like passing a string to GPIO.write/2
if Mix.env != :prod do
def ElixirALE.GPIO.start_link(_), do: {:ok, :pid}
...
end
That's how I'm handling it right now and it works well. Does elixir_ale need to provide this or would a simple documentation change with examples of how to stub things out be sufficient?
Other ideas?
I don't think we can do it in elixir_ale since we are both ignoring it in everything except prod. We could make a separate package with "fake_ale" that provides reasonable stubs to load?
If we include the conditional as part in elixir_ale then we wouldn't need to ignore it and those who already are won't break anything. Adding another separate package dependancy seems wrong.
I have something that is working for me for now. I'll give it a bit more thought about how I might make the dev/test/prod situation more seamless and allow for testing the application.
My only concern is if people are using different environments outside those three. I've seen this is the wild and I don't want to require these names if they have more or a different convention.
I guess we can assume prod as the only one that runs the GPIO, but we still have to be able to compile locally, right?
Since the Mix.env is only available at compile time I don't think we have a great solution to this. I propose adding some documentation about stubbing out your own version of ALE.