I'm not able to mock a method in the same script
Ginxo opened this issue · 8 comments
Expected Behavior
To mock a method in the same groovy script
So I have this methods in util.grooy file
def method1() {
method2()
}
def method2(){
sh "ls"
}
I tried to mock method2 execution when method1 is called like
import com.homeaway.devtools.jenkins.testing.JenkinsPipelineSpecification
import org.jenkinsci.plugins.workflow.cps.CpsScript
class UtilSpec extends JenkinsPipelineSpecification {
def groovyScript = null
def setup() {
groovyScript = loadPipelineScriptForTest("vars/util.groovy")
}
def "[util.groovy] method1"() {
setup:
getPipelineMock("CpsScript").method2() >> null // ?????
when:
groovyScript.method1()
then:
1 * getPipelineMock("method2")()
0 * getPipelineMock("sh")('ls')
}
}
As a result I expect to execute method2 but sh tool
Actual Behavior
The sh tool is executed in any case
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.934 s <<< FAILURE! - in UtilSpec
[ERROR] [util.groovy] method1(UtilSpec) Time elapsed: 0.757 s <<< ERROR!
java.lang.IllegalStateException:
There is no pipeline step mock for [method2].
1. Is the name correct?
2. Does the pipeline step have a descriptor with that name?
3. Does that step come from a plugin? If so, is that plugin listed as a dependency in your pom.xml?
4. If not, you may need to call explicitlyMockPipelineStep('method2') in your test's setup: block.
at UtilSpec.[util.groovy] method1(UtilSpec.groovy:17)
Additional Information
How is it possible to mock either in the same script file or in a different one, like util.method2()?
I see it's possible to do it from another script like
1 * getPipelineMock('util.method2')() >> null
but I can't find how to do it from the same class/script
I'm having the same problem; attempting to mock a function within the same script, a solution for this would be very nice to have.
Don't Do That
"Mocking a method in the code unit under test" is not something that is normally done in unit tests. Typically, Mocking exists to simplify dealing with external / 3rd-party code. Here, "external" and "3rd-party" is from the perspective of the piece of code you are currently testing.
Most Mocking frameworks will either not support, or struggle with mocking a "real" method in the code unit under test... because you're not supposed to do that!
Spies Do This (but not for us)
There is a technique to mock/stub your "real" code under test, though; it's called Spying, instead of Mocking. Spock does support Spying. A Spy is a real object (not a mock) with the stubbing and expectation capabilities of a Mock wrapped around it. So, all method calls on a Spied object will actually call through to the real implementation in the real object... unless you stub them to intercept them and do something else.
Again, though:
"
Mockingstubbing a method in the code unit under test" is not something that is normally done in unit tests.
Spying is ideologically best-used when your code-under-test has a dependency that cannot easily be Mocked (for whatever reason), but you need to stub interactions with it or you need to verify interactions with it. "Need" here does not mean "it's more convenient," but means "a correct test literally cannot be written or run otherwise."
So, in the example util.groovy provided in this issue, ideologically, you SHOULD NOT do the following when writing a test. But, you could almost stub the response to method2 in this way:
import com.homeaway.devtools.jenkins.testing.JenkinsPipelineSpecification
class UtilSpec extends JenkinsPipelineSpecification {
def groovyScript = null
def spyScript = null
def setup() {
groovyScript = loadPipelineScriptForTest("vars/util.groovy")
spyScript = Spy( groovyScript )
}
def "[util.groovy] method1"() {
setup:
groovyScript.method2() >> null
when:
groovyScript.method1()
then:
1 * groovyScript.method2()
0 * getPipelineMock("sh")('ls')
}
}This still won't actually work, though, because all of Spock's Mocking & Spying depends on copying the interface of the target object or class. In this case, groovyScript is an anonymous subclass of groovy.lang.Script, created at runtime in the Groovy scripting engine, when util.groovy is loaded. There is no util.class bytecode object on the classpath for Spock to find and copy, so Spock cannot create a Spy for the "util" type.
Here There Be Dragons
Never fear, though; you can still do it... but you really, really shouldn't do this unless you understand enough about Groovy, Spock, Jenkins, and Groovy Metaprograming to explain in your own words why this approach is necessary, and why it's necessary for your use-case:
import com.homeaway.devtools.jenkins.testing.JenkinsPipelineSpecification
class UtilSpec extends JenkinsPipelineSpecification {
def groovyScript = null
def setup() {
// load pipeline script
groovyScript = loadPipelineScriptForTest("vars/util.groovy")
// create a mock for uitl.method2. The name actually does not matter.
explicitlyMockPipelineStep( "util.method2" )
// create a new runtime-expandable metaClass for method dispatch for the groovyScript
def newMetaClass = new ExpandoMetaClass(Script.class, true, true)
newMetaClass.initialize()
// copy all new methods from the groovyScript over to the new metaClass
groovyScript.metaClass.getMethods().each { _method ->
if(
( _method.getDeclaringClass().getName() == groovyScript.getClass().getName() ) &&
! ( newMetaClass.getMetaMethods().any{ it.getName() == _method.getName() } ) &&
! ( newMetaClass.getMethods().any{ it.getName() == _method.getName() } )
) {
// the method doesn't exist
newMetaClass."${_method.getName()}" = groovyScript.&"${_method.getName()}"
}
}
// define "method2" on the new metaClass to call the pipeline mock for util.method2
// the .method2 is the name that matters
newMetaClass.method2 = { -> getPipelineMock("util.method2")() }
// set the new metaClass & method definitions on the loaded pipeline script
groovyScript.metaClass = newMetaClass
// re-add the pipeline mocks to the Script; they had been on the old metaClass whicih we threw out.
addPipelineMocksToObjects(groovyScript)
}
def "[util.groovy] method1"() {
when:
groovyScript.method1()
then:
1 * getPipelineMock("sh")("id")
1 * getPipelineMock("util.method2")()
0 * getPipelineMock("sh")("ls")
}
}The above example assumes a modified util.groovy that adds a pipeline step to method1 to verify that the "regular" mocking behavior of Jenkins-Spock still works:
def method1() {
sh "id"
method2()
}
def method2(){
sh "ls"
}You can see this working in action on the https://github.com/ExpediaGroup/jenkins-spock/tree/issue-78/examples/shared-library/test branch.
Required Reading
In order to determine if you actually need the above, and to be able to defensibly incorporate it into your software, I would recommend reading & understanding at least the following:
@awittha Thank you for the thorough explanation and for providing the examples above (there definitely be dragons in that one).
I do want to provide the background as to why I felt this feature was nice to have, as well as get your feedback on our approach.
Within our shared libraries we (try to) make a deliberate effort to develop small, functionally cohesive methods that contribute to a single well-defined task. We find doing so, results in methods and pipelines that are easier to understand and (most of the times) are easier to unit test. However, this approach results in shared libraries comprised of methods that have multi-layer call graphs; i.e. a call tree where one method calls another method (within the same library), calls another, etc.
Here is an example of one:
https://github.com/edgexfoundry/edgex-global-pipelines/blob/master/vars/edgeXReleaseGitTag.groovy#L130
Call graph:
Obviously, its been our experience that the methods representing the leaves of the call tree are simple to write unit tests for (because the mocking/stubbing is straight-forward), where the methods that represent branches are a bit more challenging due to amount of mocking that needs to be done (for the reason above).
In light of your response, I'm tempted to calibrate our development approach to one that results in a more flattened call graph where we try to limit the number of methods that have internal calls. I think ultimately its a balancing act between maintaining smaller tightly cohesive methods on one side and larger (possibly more complicated) methods but that facilitate ease of creating (maintaining) good unit-tests on the other. I for one tend to lean on always having good unit tests/coverage. Ideally we would have both. Your thoughts?
a more flattened call graph where we try to limit the number of methods that have internal calls.
Given the situation you've described and the example linked to, I think that seems reasonable. Even if this were just a Java library (yes, you'd be able to use Spies there (probably) to "solve" the problem) the real problem is, as you identified: the complexity/depth of the control flow of the top- and higher-level methods.
Looking at https://github.com/edgexfoundry/edgex-global-pipelines/blob/master/vars/edgeXReleaseGitTag.groovy, I don't actually see any global state in that script. It looks like every method takes in all of the necessary input as parameters (or from Jenkins' environment)!
Each step could be its own file in the shared library, then, right? And then any calls between those units suddenly become external calls, for which the basic mocking tools work without issue.
Yes, moving functions to their own files could be a better approach, the majority of our libraries don't contain global state.
In fact, its a practice we have already been implementing; moving functions to our utils shared library for the sake of being able to unit test them. The only thing we need to be careful of is creating a conglomerated utils library.
Another approach would be to implement a sort of name-spacing using file names (poor man's namespace):
utils.groovy -> contain methods shared by all other shared libraries
[functional-area].groovy -> shared library for the respective functional area
[functional-area]Utils.groovy -> util methods for the functional area shared library (to facilitate unit testing)