Mocking frameworks in .Net

A few month ago I played with some mocking frameworks in .Net. There are already some comparisons available (here, here or here). In this blog post I want to show which frameworks are available and which one fits best for agile development.

You could download the source code from github.com.

Software under Test (SUT)

To demonstrate and evaluate all the different mocking framework I had to create a scenario where it is appropriate to use a mocking framework. So I chose an user entity which I want to validate and in a successful case also to persist in a database.

namespace Mocking
{
 public class User
 {
  public IUserGateway Gateway
  {
   get; set;
  }

  public User()
  {
   this.Gateway = new UserGateway();
  }

  public User(IUserGateway gateway)
  {
   this.Gateway = gateway;
  }

  public bool Persist(IUserValidator validator)
  {
   bool bValid = validator.Validate(this);

   if(bValid) bValid = this.Gateway.Persist(this);

   return bValid;
  }
 }
}

This is the User entity class which implements the active record pattern. As you see, there are three injection patterns realized, the constructor injection and setter injection for the gateway and the parameter injection for the validator.

namespace Mocking
{
 public class UserGateway : IUserGateway
 {
  public virtual bool Persist(User user)
  {
   return true;
  }
 }
}

The UserGateway class is responsible to persist the entity. In my scenario it is just a dummy, because I don’t want to setup a database for this demo.

namespace Mocking
{
 public class UserValidator : IUserValidator
 {
  public bool Validate(User user)
  {
   if(user == null) return false;

   return true;
  }
 }
}

The validator is also more or less just a dummy for this demo.

Both interfaces, IUserGateway and IUserValidator, has the same methods, properties as the classes which implement them.

NMock2

One of the first mocking frameworks in .Net which I was looking at was NMock2. Specially thecheat sheet make it really easy to learn this framework.

namespace MockingTest
{
 [TestClass]
 public class NMock2Test
 {
  [TestMethod]
  public void TestPersist()
  {
   Mockery mocks = new Mockery();

   //Create mocks
   IUserGateway mockGateway = mocks.NewMock();
   IUserValidator mockValidator = mocks.NewMock();

   //Create user
   User user = new User();

   //Expectations
   using(mocks.Ordered)
   {
    Expect.Once.On(mockValidator).Method("Validate").With(user).
Will(Return.Value(true));
    Expect.Once.On(mockGateway).Method("Persist").With(user).
Will(Return.Value(true));
   }

   //Assign Gateway
   user.Gateway = mockGateway;

   //Test method
   Assert.AreEqual(true, user.Persist(mockValidator));

   mocks.VerifyAllExpectationsHaveBeenMet();
  }
 }
}

You could use the AAA-Syntax (Arrange-Act-Assert). Also more extended features like the ordered calls with the using syntax is very nice.

But, there is are also negative points: NMock2 uses “magic strings”. In your Expectations you have to declare the called method with a strings, which makes your tests very brittle when your refactor your code. For example if you have to rename a method, even the compiler could not help you. An other negative point is, that you can only mock interfaces. When you have to mock a class, you have to change your design by adding an interface just for mocking.

NMock3

Recently I discover, that there is already a follower of NMock2 on codeplex: NMock3.

namespace NMock3Test
{
    [TestClass]
    public class NMock3Test
    {
        [TestMethod]
        public void TestPersist()
        {
            MockFactory factory = new MockFactory();

            //Create mocks
            Mock mockGateway = factory.CreateMock();
            Mock mockValidator = factory.CreateMock();

            //Create user
            User user = new User();

            //Expectations
            using(factory.Ordered)
            {
                mockValidator.Expects.One.MethodWith(m => m.Validate(user)).WillReturn(true);
                mockGateway.Expects.One.MethodWith(m => m.Persist(user)).WillReturn(true);
            }

            //Assign gateway
            user.Gateway = mockGateway.MockObject;

            //Test method
            Assert.AreEqual(true, user.Persist(mockValidator.MockObject));

            factory.VerifyAllExpectationsHaveBeenMet();
        }
    }
}

The migration is quite easy: Replace the reference from your test project from the NMock2.dll with a new reference to the NMock3.dll. After that you could use the new approach with the MockFactory and the lambda expressions in your Expectations.

NMock3 introduce type safety and readiness for refactorings combined with the easiness of NMock2. They also keep the tradition of a good cheat sheet.

With NMock3 I couldn’t mock a class (even when I followed the exceptions…"). The creation of stubs should be possible. The project is still a young one and so the documentation isn’t that good as it should be. But NMock3 remain an interesting mocking framework.

Simple.Mocking

When I looked for mocking frameworks I also found this new and not very well known framework Simple.Mocking on codeplex. Its also avoid “magic strings” and you could also use the AAA-Syntax. Simple.Mocking has only one committer, Mikael Waltersson.

namespace MockingTest
{
    [TestClass]
    public class SimpleNetTest
    {
        [TestMethod]
        public void TestPersist()
        {
            //Create mocks
            var expectationScope = new ExpectationScope();
            IUserGateway mockGateway = Mock.Interface(expectationScope);
            IUserValidator mockValidator = Mock.Interface(expectationScope);

            //Create user
            User user = new User();

            //Expectations
            Expect.Once.MethodCall(() => mockValidator.Validate(user)).Returns(true);
            Expect.Once.MethodCall(() => mockGateway.Persist(user)).Returns(true);

            //Assign gateway
            user.Gateway = mockGateway;

            //Test method
            Assert.AreEqual(true, user.Persist(mockValidator));

            AssertExpectations.IsMetFor(expectationScope);
        }
    }
}

The ExpectationsScope is used to avoid to call AssertExpectations.IsMetFor for each mock. This construct is special and I’m (currently) not a big fan of it.

With Simple.Mocking you could just mocking interfaces and delegates, you couldn’t mock classes.

Rhino.Mocks

Rhino.Mocks is one of the famous .Net mocking frameworks. As Simple.Mocking there is also only one committer to this framework, Oren Eini (alias Ayende Rahien).

namespace MockingTest
{
 [TestClass]
 public class RhinoMockTest
 {
  [TestMethod]
  public void TestPersistWithRecordingReplay()
  {
   MockRepository mocks = new MockRepository();

   IUserGateway gateway = mocks.StrictMock();
   IUserValidator validator = mocks.StrictMock();

   //Create user
   User user = new User();

   //Expectations
   using(mocks.Ordered())
   {
    Expect.Call(validator.Validate(user)).Return(true);
    Expect.Call(gateway.Persist(user)).Return(true);
   }

   //Stop recording, start replay
   mocks.ReplayAll();

   //Assign gateway
   user.Gateway = gateway;

   //Test method
   Assert.AreEqual(true, user.Persist(validator));

   mocks.VerifyAll();
  }

  [TestMethod]
  public void TestPersistWithArrangeActAssert()
  {
   var gateway = MockRepository.GenerateMock();
   var validator = MockRepository.GenerateMock();

   //Create user
   User user = new User();

   //Expectations
   validator.Expect(x => x.Validate(user)).Return(true);
   gateway.Expect(x => x.Persist(user)).Return(true);

   //Assign gateway
   user.Gateway = gateway;

   //Test method
   Assert.AreEqual(true, user.Persist(validator));

   //Asserts
   validator.VerifyAllExpectations();
   gateway.VerifyAllExpectations();
  }
 }
}

Rhino.Mocks supports two ways to use the framework: the record-replay and the AAA-syntax. My favorite is clearly the AAA-syntax, because there is much less code needed (reducedaccidental complexity). Rhino.Mocks uses also Lambda-Expressions to avoid “magic strings”, so refactoring shouldn’t be a problem.

Moq

Moq is another mocking framework which takes advantage of the .Net 3.0 language features like lambda expressions. So Moq doesn’t use “magic strings” and it enables also the AAA-syntax. There are two ways how to create mocks. You could create mocks directly with the Mock class or you could use the MockFactory class. For both scenarios I created a test.

namespace MockingTest
{
 [TestClass]
 public class MoqTest
 {
  [TestMethod]
  public void TestPersist()
  {
   var gateway = new Mock();
   var validator = new Mock();

   User user = new User();

   //Expectations
   validator.Setup(x => x.Validate(user)).Returns(true).AtMostOnce();
   gateway.Setup(x => x.Persist(user)).Returns(true).AtMostOnce();

   //Assign gateway
   user.Gateway = gateway.Object;

   //Test method
   Assert.AreEqual(true, user.Persist(validator.Object));

   validator.VerifyAll();
   gateway.VerifyAll();
  }

  [TestMethod]
  public void TestPersistWithFactory()
  {
   MockFactory factory = new MockFactory(MockBehavior.Strict);

   //Class works, but methods have to be virtual -> not nice
   var mockGateway = factory.Create();
   var mockValidator = factory.Create();

   User user = new User();

   //Expectations
   mockValidator.Setup(x => x.Validate(user)).Returns(true).AtMostOnce();
   mockGateway.Setup(x => x.Persist(user)).Returns(true).AtMostOnce();

   //Assign gateway
   user.Gateway = mockGateway.Object;

   //Test method
   Assert.AreEqual(true, user.Persist(mockValidator.Object));

   factory.VerifyAll();
  }
 }
}

I prefer the MockFactory approach, because you don’t need to verify each mock (its reduce the test code which is a good thing). Be sure, that in this scenario you don’t forget the parameter MockBehavior.Strict which makes your mock objects real mocks. Without this parameter they are more like Stubs, which don’t fail and just return a default value.

But there is a little problem: If you want to mock a class, then all the methods have to be virtual, but Moq has no problem to mock an interface.

Typemock Isolator

Unfortunately Typemock Isolator isn’t free, but you could get a trial to try this great mocking framework. As you could see in the TestPersistWithReflectiveMocking Typemock Isolator could mock classes, also non-virtual methods. Great. It also doesn’t use “magic strings”. But wait, you could not mock interfaces? With the new AAA-syntax it is possible and it looks very nice.

namespace MockingTest
{
 [TestClass]
 public class TypeMockTest
 {
     [TestMethod]
     public void TestPersistWithNaturalMocking()
     {
         MockManager.Init();

         User user = new User();

         // Natural mocking
         using(RecordExpectations recorder = new RecordExpectations())
         {
             IUserGateway mockGateway = new UserGateway();
             IUserValidator mockValidator = new UserValidator();
    
             //Expectations
             recorder.ExpectAndReturn(mockValidator.Validate(user), true);
             recorder.ExpectAndReturn(mockGateway.Persist(user), true);
         }

         //Create instances
         IUserGateway gateway = new UserGateway();
         IUserValidator validator = new UserValidator();
   
         //Assign gateway
         user.Gateway = gateway;

         //Method
         Assert.AreEqual(true, user.Persist(validator));

         MockManager.Verify();
     }
  
     [TestMethod]
     public void TestPersistWithReflectiveMocking()
     {
         MockManager.Init();

         //Create mocks
         Mock mockGateway = MockManager.Mock();
         Mock mockValidator = MockManager.Mock();

         //Create user
         UserGateway gateway = new UserGateway();
         IUserValidator validator = new UserValidator();
         User user = new User(gateway);

         //Expectations
         mockValidator.ExpectAndReturn("Validate", true).Args(user);
         mockGateway.ExpectAndReturn("Persist", true).Args(user);

         //Method
         Assert.AreEqual(true, user.Persist(validator));

         MockManager.Verify();
     }

  [TestMethod]
  [Isolated]
  public void TestPersistWithAAA()
  {
   //Arrange
   //Create mocks
   UserGateway gateway = Isolate.Fake.Instance();
   IUserValidator validator = Isolate.Fake.Instance();
            //Create user
   User user = new User(gateway);
            //Expectations
   Isolate.WhenCalled(() => validator.Validate(user)).WillReturn(true);
   Isolate.WhenCalled(() => gateway.Persist(user)).WillReturn(true);

   //Act
   bool bPersist = user.Persist(validator);

   //Assert
   Assert.AreEqual(true, bPersist);
  }
 }
}

Typemock Isolator also supports the record-replay approach (demonstrated in the TestPersistWithNaturalMocking test method). But as you already know, I prefer the AAA-syntax.

The question about Typemock Isolator is, if it is worth to pay for a mocking framework if there are so much other free opportunities?  Typemock Isolator is very powerful and for some companies it is very important to get support and also guarantee that the product will be continued for example ten years. In such a scenario Typemock Isolator would be definitely an option. In reaction of such scenarios, also other providers of mocking frameworks offer commercial support for their frameworks, for example Rhino.Mocks.

The undocumented – NUnit Mocking

It is not very known that there exists also in NUnit a mocking framework. It is possible to mock interfaces and classes if the inherit from MarshalByRefObject.

namespace MockingTest
{
 [TestClass]
 public class NUnitMockTest
 {
  [TestMethod]
  public void TestPersist()
  {
   //Gateway
   DynamicMock mockGateway = new DynamicMock(typeof(IUserGateway));
   IUserGateway gateway = (IUserGateway) mockGateway.MockInstance;
   
   //Validator
   DynamicMock mockValidator = new DynamicMock(typeof(IUserValidator));
   IUserValidator validator = (IUserValidator)mockValidator.MockInstance;

   //User
   User user = new User(gateway);

   //Expectations
   mockValidator.ExpectAndReturn("Validate", true, user);
   mockGateway.ExpectAndReturn("Persist", true, user);

   Assert.AreEqual(true, user.Persist(validator));
   mockValidator.Verify();
   mockGateway.Verify();
  }
 }
}

The mocking framework is easy to use and if you already use NUnit, then you don’t need an additional framework.

As I know, it isn’t official and there is no documentation. You have also to use “magic strings” for methods and the limitation for classes also reduce the benefit.

Conclusion

For a modern mocking framework it is essential that its enable refactoring by not using “magic strings”. It is also essential for me that it allows to use the AAA-syntax. A third essential thing is that it allows you to mock classes. You don’t want to create interface just for mocking, for example for your domain model. Avoid accidental complexity is very important, so you shouldn’t add unnecessary code just for mocking. That doesn’t mean that “design for testability” is wrong, “design for testability” means, that you design your code in a way, that you have the possibility to test your code, for example you avoid huge classes (god object anti-pattern) with a lot of private methods or you avoid creation of instances in methods (unable to use dependency injection).

So, which framework fulfill all the requirements? NMock3, which inherit the easiness of NMock2 combined with the elimination of the “magic strings”? Or Typemock Isolator which also eliminate the “magic strings” and could even mock classes?

For me, there isn’t currently a clear winner visible, maybe Typemock makes the race. But there are frameworks which I would no longer use, as NMock2 or NUnit mocking. They don’t fulfill my requirements clearly. That doesn’t mean they aren’t suitable for your scenarios.

Also to be clear: There are other topics that should be considered: mocking events, Stubs, etc. The list of the mocking frameworks isn’t complete, for example Moles is a framework which could be interesting too.