Moq to NSubstitute migration tool
mikocot opened this issue · 33 comments
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Regardless future changes of Moq library it's quite clear that its reputation and trust is gone, hence we'll see tons of projects moving to alternatives. This will be much easier if a migration tool was available to at least majority of the work.
Describe the solution you'd like
A clear and concise description of what you want to happen.
An external tool to duplicate existing test projects and rewrite them using NSubstitute calls instead of Moq
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Script the change for most common calls, do the rest of the work manually,
Additional context
Add any other context or screenshots about the feature request here.
As it mostly applies to tests themselves.... it should be pretty easy to test the results
If NSubstitute wants to "benefit," so to speak, from Moq's recent mistakes, this would all but ensure it, I reckon.
Question: Are you willing to pay for this tool? Or free again?
As a starting point, I found this which might be useful. I used it on a project. It didn't solve everything (for exemple I had to remove many .Object, some Setup weren't replaced if not inlined, there were some Arg.Any in concrete method calls...), But it saved me a lot of time.
I'm not sure it really needs to be a tool right now? Just start with a before/after of Moq/NSubstitute code. This could be documentation.
Hi @mikocot , thanks for raising this. I understand your reasoning but I don't think the NSub team will take this on. It is probably better as a separate community project.
If NSubstitute wants to "benefit," ...
Speaking on behalf of NSubstitute (but this view seems shared by Moq and FakeItEasy devs I've spoken with), we don't mind what testing tools anyone uses. If people are testing then that's great! We just want to encourage testing; if our library helps people do that then awesome! If another library works better for you then also awesome!
I'm not sure it really needs to be a tool right now? Just start with a before/after of Moq/NSubstitute code. This could be documentation.
The Azure SDK has some documentation that includes NSub and Moq examples. That might be a reasonable start?
I'm interested in exploring this, and have made migration tools before (https://github.com/samsmithnz/AzurePipelinesToGitHubActionsConverter)
I'm brand new here and only used moq. Is this even feasible?
Moq have also Moq.Protected to access protected methods by name (in a string).
To mimic that in NSubstitute one can use this simple helper class/method:
public static class MockHelper
{
public static object CallProtectedMethod(this object obj, string methodName, params object[] callParams)
{
var method = obj.GetType()
.GetMethod(methodName, BindingFlags.NonPublic | BindingFlags.Instance);
var result = method.Invoke(obj, callParams);
return result;
}
}
Code sample where Moq.Protected was in use:
mockedObject.Protected().Verify("ExecuteMethodProtected", Times.Once(), ItExpr.IsAny<string>());
can be replaced with this:
mockedObject.Received(1).CallProtectedMethod("ExecuteMethodProtected", Arg.Any<string>());
We use Moq everywhere at my work, so I’m currently building a Powershell tool to run over test files and using regex to search Moq code and replace with NSubstitute. I Will share it here if nothing turns up in the meantime.
I've just put together a dotnet tool that runs these regex over test files, real dirty at the moment but I don't think it needs to be much more than this. pretty good results but still needs a bit of work, particularly around Setup and Verify - hopefully get it done today.
If anyone wants to collab, let me know & I'll add you on the repo https://github.com/dylan-asos/moq-to-nsub/tree/main
For anyone who is curious, Nick Chapsas has indicated he will be making a video guiding people on how to migrate from Moq to NSubstitute.
Throwing these in, I see some of them were already mentioned but regardless.
https://gist.github.com/AlbertoMonteiro/daeab549df57727ddaa7
https://www.garethrepton.com/Unit-Testing-async-methods/
https://blog.krusen.dk/nsubstitute-make-async-method-throw
This is the kinda stuff I use ChatGPT for, you can just have it do the translation for you.
Also here is a small guide ChatGPT made.
1. Install NSubstitute
First, you'll need to add the NSubstitute NuGet package to your project.
Install-Package NSubstitute2. Replace Moq Syntax with NSubstitute Syntax
Let's go through some common examples:
a) Creating a Mock
Moq:
var mock = new Mock<IYourInterface>();NSubstitute:
var substitute = Substitute.For<IYourInterface>();b) Setting Up Return Values
Moq:
mock.Setup(x => x.Method()).Returns(42);NSubstitute:
substitute.Method().Returns(42);c) Verifying Calls
Moq:
mock.Verify(x => x.Method(), Times.Once);NSubstitute:
substitute.Received().Method();d) Arguments Matching
Moq:
mock.Setup(x => x.Method(It.IsAny<int>())).Returns(true);NSubstitute:
substitute.Method(Arg.Any<int>()).Returns(true);3. Replace All Instances
Now you'll need to go through your code and replace all instances of Moq syntax with NSubstitute syntax. You might find the Find & Replace tool in your IDE helpful.
4. Run Your Tests
Make sure to run all your unit tests to ensure everything is working as expected.
Also if you used Moq's .Callback to do work with the input arguments of a function, you can migrate like this:
Moq:
int? capturedArg = null;
mock
.Setup(x => x.MyMethod(It.IsAny<int>(), It.IsAny<string>()))
.Returns(true)
.Callback((int i, string s) => capturedArg = i);NSubstitute:
int? capturedArg = null;
substitute
.MyMethod(Arg.Do<int>(i => capturedArg = i), Arg.Any<string>())
.Returns(true);or maybe a wrapper library with moq's public api on top of nsub for a drop-in replacement to aid the (huge amount of) existing code
@dtchepak Please please, I urge you and the team to reconsider this decision. I can imagine the reasons of being hesitant to work on a migration tool in this crazy situation, but this isn't just about promoting NSubstitute. A serious blow has been dealt to the .NET Ecosystem as a whole and there really isn't a comparable testing framework elsewhere. There is also a clear difference between the library maintainers providing a canonical migration tool versus a community driven one that people are going to have a hard time discovering, let alone trusting it.
As you can imagine, people and major corporations are actively migrating away from Moq. Those that do not have the capacity are considering to drop .NET due to broken trust. If the team can't work on this tool, please at least consider accepting a contribution.
Hi @rzn34 ,
There are a few reasons I want to avoid this. Please excuse the unstructured brain dump:
- I don't know Moq well at all. As such I would probably be the least well suited person to work on a script for migrating Moq tests.
- AFAICT a script would be very hard to verify. I am not prepared for a bug in a migration script to invalidate a test and result in a bug slipping through a product's/project's tests and into production.
- AFAICT there is no reason existing projects can't pin their version of Moq to the most recent version they are happy with. If someone is waiting on an update to fix a bug with Moq I think it would be more efficient and effective to replace Moq in that specific test or tests with a hand-coded test double.
- I am confident the Moq team will resolve the situation, and in the event the community isn't happy with the response I'm equally confident members of the community will spin up an alternative fork.
For these same reasons I am not prepared to recommend a specific migration tool as a canonical one.
I can understand your point of view on this but I hope you can understand why it is not practical for the NSubstitute team to take this on.
Question: Are you willing to pay for this tool? Or free again?
@tonyqus to be fair many were willing to pay for moq but most don't like to be held at gunpoint and pay ransom, while still having to go through some dodgy process requiring external apps, getting warnings in build and unknown pieces of code in their software
I'm not sure it really needs to be a tool right now? Just start with a before/after of Moq/NSubstitute code. This could be documentation.
@taspeotis I guess you don't have too many tests to migrate? :)
@dtchepak appreciate your feedback. we were evaluating what needs to be done and the very idea came up to stick with the earlier version of moq for existing code and write new tests with nsub. i still see potential in having a compatibility wrapper (namespace NSubstitute.MoqCompatibility) to remove moq dependency in existing projects. that can be a standalone library. best done by someone who is well-versed in both libs. it will help the quick transition and not seeing moq anywhere in our dependency graph.
Hi @rzn34 , There are a few reasons I want to avoid this. Please excuse the unstructured brain dump:
- I don't know Moq well at all. As such I would probably be the least well suited person to work on a script for migrating Moq tests.
- AFAICT a script would be very hard to verify. I am not prepared for a bug in a migration script to invalidate a test and result in a bug slipping through a product's/project's tests and into production.
- AFAICT there is no reason existing projects can't pin their version of Moq to the most recent version they are happy with. If someone is waiting on an update to fix a bug with Moq I think it would be more efficient and effective to replace Moq in that specific test or tests with a hand-coded test double.
- I am confident the Moq team will resolve the situation, and in the event the community isn't happy with the response I'm equally confident members of the community will spin up an alternative fork.
For these same reasons I am not prepared to recommend a specific migration tool as a canonical one.
I can understand your point of view on this but I hope you can understand why it is not practical for the NSubstitute team to take this on.
I'm actually with you on this, moq is an external library so you guys can't take responsibility for such tool ongoing compatibility nor for supporting all the edge cases. I don't think such tool belongs directly into this project repository, but rather should be a linked project, that maybe you guys could oversee to ensure it does the right things.
The reason I've created this issue, is because I expect that many will face that problem and it obviously makes sense to make it a shared effort than dozens of quasi working scripts individually, plus I expect people are going to look for such tool over here. At the same time I guess most will take the most pragmatic approach which is to fix the previous version of moq and see the situation progresses, which is what we've decided for our projects so far, and which is why I haven't started developing such tool myself yet.
Cool to see several solutions already showing up.
I've taken a first pass at a (community) conversion tool: https://MoqToNSubstitute.azurewebsites.net/
The source handles most of the scenarios listed above, but has a number of scenarios to that still need to be resolved, and I'll continue to iterate, but wanted to share, so that others can provide feedback (and code samples that may not work).
Isn't it part of the job for a software engineer to learn what ever tool or API they need to learn, to be able to do the job? This just sounds like laziness. You may very well have hundreds if not thousands of tests. I have 6.5k tests to update which will take no more than several days of work.
@ColinM9991 when you are dealing with tests for multiple domains (read: different teams / ownership etc.) the least invasive approach (switching the package and replacing namespaces etc.) is encouraged over 50k lines search-and-replace operation which everyone reviewing the change would need to wrap their heads around. and having a community-driven effort is much better than going solo for this, if robust solution interests you.
Isn't it part of the job for a software engineer to learn what ever tool or API they need to learn, to be able to do the job?
Agreed
This just sounds like laziness.
Also agreed. But I consider lazyness a trait of a software engineer since his/her job is mostly to automate repetitive processes so computers can do what lazy humans don't want to do.
You may very well have hundreds if not thousands of tests. I have 6.5k tests to update which will take no more than several days of work.
If only that could be automated so you can avoid it.
@kasperk81 and @Jopie64 there's every possbility I spoke out of turn without first making the correct argument.
I consider lazyness a trait of a software engineer since his/her job is mostly to automate repetitive processes so computers can do what lazy humans don't want to do.
Absolutely, I agree 100%. Repeatedly performing a task manually, that can be automated, is time wasted.
Getting together as a community and streamlining the migration process isn't an issue. It's something we'll all go through at some point, with the ongoing and bizarre conversations around Moq's invasive billing model, so nothing is more efficient than working together and sharing easier methods of achieving this goal.
The issue, in my view, is asking or expecting the NSubstitube team to develop this tool for the community thus taking time away from developing NSubstitute. With that said, there aren't actually that many people here making the direct ask of the NSub development team do work on such a tool, there are only a few people asking this.
- AFAICT a script would be very hard to verify. I am not prepared for a bug in a migration script to invalidate a test and result in a bug slipping through a product's/project's tests and into production.
At least for verifying, you would need a bunch of failing, and passing tests, and verify they have the same result after the migration. You could do what Rust did, and have a program download random repos, do the migration, and test if the results are the same over, and over.
I'm not saying that is what you should do, I just thought it was interesting.
Another approach that could be taken is re-implementing Moq's API, but with NSubstitute under the hood
AFAICT there's nothing stopping the Moq to NSubstitute migration tool to be made by a third-party instead of trying to get the NSubstitute team to sign on. Be the change
I've taken a first pass at a (community) conversion tool: https://MoqToNSubstitute.azurewebsites.net/
The source handles most of the scenarios listed above, but has a number of scenarios to that still need to be resolved, and I'll continue to iterate, but wanted to share, so that others can provide feedback (and code samples that may not work).
^^^ I created a tool. Please try it. It seems to be working for most scenarios (except callbacks, that I'm still chipping away at)
@samsmithnz Thank you for your efforts. I suggest you put a donation link on your website since your efforts is a lifesaver for a few developers after the companies decide they wanna get rid of Moq.
Question: Are you willing to pay for this tool? Or free again?
@tonyqus to be fair many were willing to pay for moq but most don't like to be held at gunpoint and pay ransom, while still having to go through some dodgy process requiring external apps, getting warnings in build and unknown pieces of code in their software
It's kzu's fault to create damage by binding Sponsorlink to Moq release although I can understand the goal of kzu. However, the guy who creates Moq to NSubstitute migration tool are good guys and wanna help you. You should donate because they saves huge efforts for big companies who are heavy Moq users. Compared with homelesses who do nothing but just beg, these developers are much better and worth a donation.
Writing a conversion tool is no trivial task, I know this because I just finished writing one.
I was able to completely replace Moq in one of my unit test projects with over 600 lines of Moq specific code using Visual Studio and a combination of text replacements and regular expression replacements. There were a few challenges that had to be resolved manually but the conversion was completed in about 4 days.
I took these replacement expressions and created a console application that iterates through a project and replaces the most common Moq to NSubstitute statements. It is still a work in progress but new replacements could easily be added to the code. I am sharing it on github for anybody to use (no sponsor link):
https://github.com/SilverLiningComputing/MoqToNSubstituteConverter
The limitations are mentioned in the readme and it will always need some manual clean-up once the process is complete. The default run does analysis only on the current folder and there is a log file to see how well it performs before modifying any code.
I've taken a first pass at a (community) conversion tool: https://MoqToNSubstitute.azurewebsites.net/
The source handles most of the scenarios listed above, but has a number of scenarios to that still need to be resolved, and I'll continue to iterate, but wanted to share, so that others can provide feedback (and code samples that may not work).^^^ I created a tool. Please try it. It seems to be working for most scenarios (except callbacks, that I'm still chipping away at)
@samsmithnz Your site is down.
@MisinformedDNA Back! .NET 8 upgrade hiccup. :)
I created a Visual Studio extension that helps with the Moq Framework convention for NSubstitute if you want to give it a try. I hope it helps with this process.
https://marketplace.visualstudio.com/items?itemName=RicardoPignone.ricardopignone-Moq2NSubstitute