ploeh blog

testmethod Code Snippet

This is an installment in my Zero-Friction TDD series.

If you are a regular reader of this blog, you may have noticed a certain pattern in my unit test examples (like this one). This is because I always follow the Four-Phase Test pattern (which is a superset of the more well-know Triple-A (Arrange Act Assert) pattern), with a few code comments to denote the different phases (incidentally, in this case, I don't regard these code comments as Apologies).

Writing the skeleton of such a test (the TestMethod attribute, the method declaration, the four comments) is very repetitious work, and I got tired of it around the third or fourth time I had to do it. Therefore, I wrote a code snippet that, in essence, just generates this template:

[TestMethod]

public void Test()

{

    // Fixture setup

    // Exercise system

    // Verify outcome

    Assert.Inconclusive();

    // Teardown

}

From there, you are ready to go ahead and fill in the blanks. It saves me a bit of typing for every test I write and allows me to get started faster with formulating the actual value-adding code, while the structure reminds me to write good and maintainable tests.

I've been using this code snippet for more than a year now, and although I've considered many enhancements to it, I've always felt comfortable with the versatility that this simple implementation provides.

For your convenience, I've attached the code snippet to this post: Just download it and put it in your <Documents>\Visual Studio 2008\Code Snippets\Visual C#\My Code Snippets folder. Now you can just type testmethod and expand the snippet, and you have a structured Four-Phase Test ready to be implemented.

Once more, it saves you a bit of mental context switching, and you can focus on writing the test code that really matters.

Ignore Irrelevant Return Values

This is an installment in my Zero-Friction TDD series.

Sometimes, you don't care about the return value from a particular operation. The simplest example is if you want to check that creating a new instance of a specific type will throw an exception if supplied with wrong parameter values:

[ExpectedException(typeof(ArgumentNullException))]

[TestMethod]

public void CreateWithNullTextWillThrow()

{

    // Fixture setup

    int anonymousNumber = 8;

    string nullText = null;

    // Exercise system

    new Plop(anonymousNumber, nullText);

    // Verify outcome (expected exception)

    // Teardown

}

In this example, even though the new operation is supposed to return a value, we don't care about it, since we only want to test that the correct exception type is being thrown. To save myself from having to stop and think about a variable name (even though in this case, I'd just have used sut), I prefer to not declare and assign the variable at all.

In general, I believe that APIs should follow the principle of Command-Query Separation, but sometimes it makes sense to break this rule. When you have an operation that both return a value and have side-effects, testing the side-effect via state-based testing doesn't require the return value.

Imaging that you have a method with this signature:

public object CommandQuery(int number)

Testing the side-effect may look like this:

[TestMethod]

public void CommandQueryWillUpdateNumberProperty()

{

    // Fixture setup

    int expectedNumber = 89;

    int anonymousNumber = 11;

    string anonymousText = "Anonymous text";

    Plop sut = new Plop(anonymousNumber, anonymousText);

    // Exercise system

    sut.CommandQuery(expectedNumber);

    // Verify outcome

    Assert.AreEqual<int>(expectedNumber, sut.Number, "CommandQuery");

    // Teardown

}

Notice that I've ignored the return value of the CommandQuery method.

Save yourself the time and effort it takes to declare and assign variables if you don't use them anyway.

Microsoft Dynamics Mobile 1.5 Released

If you are wondering about what I'm doing these days, we (the Microsoft Dynamics Mobile Team) just released a new version of Microsoft Dynamics Mobile!

And there was much rejoicing :)

Anonymous Variables

This post is an installment in my series on Zero-Friction TDD.

Often when writing a unit test, the SUT's API will force you to create objects that you really don't care about. As an example, take a look at the signature of this constructor:

public Plop(int number, string text)

To create an instance of the Plop class, you must supply both parameters, but you may not care about their values. You can't just pass in null as the text parameter, because the constructor checks for this and throws an ArgumentNullException if you do. In other words, you are forced to create some non-null text to pass to the constructor, even if its specific value is totally irrelevant to the test at hand.

That sounds a lot like the description of a Dummy, except that such variables are taxonomically different because they aren't Test Doubles. In xUnit Test Patterns, Gerard Meszaros also discusses Anonymous Creation Methods as one way of creating variables like these, so I've lifted this terminology to call them Anonymous Variables.

When naming such variables, I prefix the variable name with anonymous to indicate that I don't really care about the value.

[TestMethod]

public void CreateWillExposeNumberAsProperty()

{

    // Fixture setup

    int expectedNumber = 4;

    string anonymousText = "Anonymous text";

    Plop sut = new Plop(expectedNumber, anonymousText);

    // Exercise system

    int result = sut.Number;

    // Verify outcome

    Assert.AreEqual<int>(expectedNumber, result, "Number");

    // Teardown

}

Naming the text variable anonymousText instead of just text or something similar probably doesn't save me much context switching, but it's a good help for the Test Reader because it clearly indicates that we don't really care about this value.

Even though it's redundant, I also type in Anonymous text as the string value, although this could really be anything. However, it saves me the trouble of coming up with something else.

The same pattern can be used if we want to test the text instead of the number

[TestMethod]

public void CreateWillExposeTextAsProperty()

{

    // Fixture setup

    int anonymousNumber = 3;

    string expectedText = "Anonymous text";

    Plop sut = new Plop(anonymousNumber, expectedText);

    // Exercise system

    string result = sut.Text;

    // Verify outcome

    Assert.AreEqual<string>(expectedText, result, "Text");

    // Teardown

}

With numbers, the value itself can contain no hint of its origin and usage (like a string can by saying I don't matter), so the name anonymousNumber is the only clear indicator of the number's purpose. It also helps clarify what may otherwise look like a magic number: Why is the value 3? The name gives us an indication that it doesn't matter, and it might as well have been some other value. Without that hint, the Test Reader might have to spend more time before determining whether or not the number 3 carries any particular connotation in this context.

Notice that even though the text variable isn't an Anonymous Variable any more, I've kept the text Anonymous text as the string value, since I don't really care about the explicit value, but only about the relationship between input and output. Again, this is a help for the Test Reader - and remember: That may very well be you six months into the future.

In terms of avoiding context switching when writing tests, this tip falls into the Almost-Not-Worth-It category, but if you combine it with the assistance it provides the Test Reader, it is certainly a valuable habit to pick up.

Naming Direct Output Variables

In my series of Zero-Friction TDD tips and tricks, it's time to look at naming Direct Output variables.

[TestMethod]

public void DoStuffWillReturnMessage()

{

    // Fixture setup

    string expectedResult = "ploeh";

    MyClass sut = new MyClass();

    // Exercise system

    string result = sut.DoStuff(expectedResult);

    // Verify outcome

    Assert.AreEqual<string>(expectedResult, result, "DoStuff");

    // Teardown

}

As you can see, I use the name result for the return value of DoStuff, and I always use that word, irrespective of its type or other circumstances. Whether you prefer result or another word is not important; the salient point is that by always using the same word, you save a context switch (because you don't have to stop and think of a good name) and thereby incrementally increase your productivity.

Zero-Friction TDD

Writing good code is difficult. Unit tests are written as code, so a corollary to the first sentence is that writing good unit tests is also difficult.

TDD (particularly if you interpret the last D as Design) carries this challenge in abundance, since you ought to be focusing on designing your SUT, but you might end up spending a lot of energy on ensuring the readability and maintainability of the tests themselves.

The goal is Zero-Friction TDD. To me, this means that you should be able to focus entirely on designing the SUT. Writing the test should present as little productivity friction as possible.

Writing a test will always require some attention to the test itself, but any trick you can use to take your mind off the test will enable you to better focus on the SUT. Every time you have to think about how to write the test instead of how to design the SUT, you make a context switch, and each of these cost you productivity.

This means that even if a given optimization strategy may seem insignificant in itself, a lot of these can go a long way towards keeping you in the zone.

I've already posted one example, where I describe a few arguments for naming SUT test variables sut. The argument I didn't present is that it frees you from thinking about a good name for the variable. In the context of that post, such an argument would have seemed trifling, but in the broader context, it saves you from one context switch and enables you to move on with writing the test.

In future posts, I will share other tricks you can use to focus your mental energy on designing the SUT, while still writing good unit tests.

Update: Posts in the series:

• Naming SUT Test Variables
• Naming Direct Output Variables
• Anonymous Variables
• Ignore Irrelevant Return Values
• testmethod Code Snippet
  

Assert.Throws

xUnit.net provides the Assert.Throws method for testing exceptions, instead of the ExpectedException attribute. In his original announcement, James Newkirk explains why this is a much better approach, and I can only agree.

However, those of us that for one reason or another use MSTest are currently stuck with the ExpectedException attribute, but I've entered a suggestion on Connect that we get a similar method in MSTest. If you would also like to have Assert.Throws in MSTest, go to the suggestion and vote for it!

At PDC

I'm currently at PDC in Los Angeles, so if you are here as well, take a moment to say hello if you see me :)

UAC Privileges Flow Across WCF Boundaries

Today, I just spent an entire afternoon troubleshooting a problem in one of my WCF services. When the solution finally dawned on me, it was so simple that I wanted to kick myself, so I thought that by sharing my experience, I might spare you the agony.

In short, the setup was this: I had just developed a custom ServiceAuthorizationManager that basically just used WindowsPrincipal.IsInRole to test whether the caller (authenticated with Windows Authentication) was a member of .\Administrators.

I had already set it up on three of my four services, and it just worked as it should. On the fourth, however, it didn't.

It was the same code running on all four services, and using the debugger, I could tell that in all four cases, the WindowsPrincipal in question represented my own account. However, in the first three services, IsInRole(".\Administrators") returned true as expected, whereas it returned false in the fourth service.

Even more strange, the fourth service exhibited expected behavior if IsInRole was called against Domain Users, or similar groups.

As far as I could tell, there was no significant differences in the WCF bindings and behaviors for the four services, but since WCF is pretty complex, I spent a lot of time comparing these without finding the reason.

It finally occurred to me that I was running my little test application for the fourth service in non-elevated mode, and since I was running on Windows Server 2008, the process didn't have administrator privileges because of UAC. That I knew, but it had never crossed my mind that this non-elevated mode flows across WCF boundaries, so even though the service wasn't impacted by UAC itself, the Windows credentials that represented my account was.

When I restarted my test tool in elevated mode, everything worked as expected.

It was my luck that I already had three working services when I ran into this issue; had I hit it on the first service, I might have concluded that my entire approach was unsound.

Naming SUT Test Variables

If you are a regular reader of this blog, you may have noticed that for the last couple of months, every test I've posted has shared some similarities; one of which is that I always name my SUT test variable sut. Here's a simple example:

[TestMethod]

public void DoStuffWillReturnMessage()

{

    // Fixture setup

    string expectedResult = "ploeh";

    MyClass sut = new MyClass();

    // Exercise system

    string result = sut.DoStuff(expectedResult);

    // Verify outcome

    Assert.AreEqual<string>(expectedResult, result, "DoStuff");

    // Teardown

}

Notice how the new instance of MyClass is named sut, and not, say, mc.

No matter the name of the class, I always name the variable sut, since it gives me at least two advantages:

1. It Very Clearly Identifies the SUT
In the above example, it may be very apparent that the SUT is an instance of MyClass no matter the name of the variable. However, as test complexity grows, the sut variable may disappear among other variables, as this example illustrates:

[TestMethod]

public void ComplexOperationWillSucceed()

{

    // Fixture setup

    string expectedBar = "bar";

    Fnaah f = new Fnaah();

    Ndøh n = new Ndøh(f);

    MyClass sut = new MyClass();

    // Exercise system

    Foo result = sut.ComplexOperation(n);

    // Verify outcome

    Assert.AreEqual<string>(expectedBar, result.Bar, "ComplexOperation");

    // Teardown

}

Had the SUT variable been mc, it would have been much less apparent which of f, n, or mc was the actual SUT, and which variables just belonged to the Test Fixture. Although the rest of the Four-Phase Test structure (including the comments) helps the Test Reader in figuring out which of the variables is the SUT, the name of the variable is definitely a helpful hint.

2. It Is Robust Against Refactoring
Imagine that instead of sut I had named the MyClass variable mc in all my 200 tests, and that I then decided to rename MyClass to ØhmHvadVedJeg (note to self: Use Danish characters in class names more often!). In that case, I would now have 200 tests where instances of ØhmHvadVedJeg were called mc. That's not very nice to the poor Test Reader.

You could of course manually edit all 200 tests to align the variable names again, but that's a bit of work, and you may even forget that you have to do that, since the renaming probably took place somewhere completely different.

When the SUT variable name is sut, you can rename your types as much as you want; the tests will still be communicative and correct.

Testing Against the Real Event Log

Lots of applications write to the Windows Event Log. When TDD'ing, event logging is part of an application's Indirect Output, so to verify that logging happens correctly, you need to verify the test against an Observation Point. As always, the correct approach to this requirement is to inject some sort of event logging abstraction (such as an interface that models the event log) into your SUT, and then verify the behavior using a Test Spy or Mock Object.

In an old post, I've described how to do this with Enterprise Library. These days, I don't use Enterprise Library, but the principle stays the same, even if the specific API is a bit different.

As such, you can TDD an entire application's event logging functionality without ever touching the real log, but at some point you will need to write an implementation that writes to the real Windows Event Log. In most cases, the implementation should be so simple that you can treat it as a Humble Object, so no testing is necessary.

However, if, for some reason, you'd like to test your Windows Event Log code, you'll need to do the following:

1. In the Fixture Setup phase of the test, get the time of the most recent event log entry, so that you can later verify that the entry you expect to be written was, in fact, written after the test executed.
2. Exercise the SUT.
3. Verify that the entry was written by looking through the event log entries that are more recent than the time you recorded previously.

This pretty much models how you'd perform a manual test by first looking in the Event Log using the MMC snap-in; running your code; and then finally refreshing the view in the Event Log MMC again.

Here's how it can be done using LINQ to Objects:

[TestMethod]

public void DoStuffWillWriteEntyInTheEventLog()

{

    // Fixture setup

    string sourceName = "MyApp";

    EventLog el = new EventLog("Application");

    DateTime latestEntryTime = (from entry in el.Entries.Cast<EventLogEntry>()

                                where entry.Source == sourceName

                                orderby entry.TimeWritten descending

                                select entry.TimeWritten).

                                DefaultIfEmpty(DateTime.MinValue).First();

    MyClass sut = new MyClass();

    // Exercise system

    sut.DoStuff();

    // Verify outcome

    bool entryExists = (from entry in el.Entries.Cast<EventLogEntry>()

                        where entry.Source == sourceName

                        && entry.TimeWritten >= latestEntryTime

                        && entry.Message.Contains("ploeh")

                        select entry).Any();

    Assert.IsTrue(entryExists, "Entry was written");

    // Teardown

}

Following the above algorithm, I first extract the most recent entry from the event source I'm interested, and using DateTime.MinValue if none was ever written - which would be the case the first time I ever run the test on any given system.

After exercising the SUT, a new query to the event log determines whether the entry was written after latestEntryTime.

A word of caution: There are generally tens of thousands of entries in the normal Windows Event log, so these queries are almost guaranteed to lead to Slow Tests. The single test in this example takes almost 15 seconds on my laptop, so don't rely on LINQ to the Event Log as a substitute for abstracting away the Windows Event Log.

In fact, don't even include one of these tests in your normal TDD test suite, since even 15 seconds is way to long to wait when you are following the Red, Green, Refactor rythm.

Such a test belongs in a special integration test suite, and may also be a good candidate to be included in a Build Verification Test suite.

SqlExpressDatabaseInstaller

When unit testing data access components, I prefer to test against run-time components that match the target environment as closely as possible. In other words, if the data access component targets SQL Server 2008, I prefer to test against a SQL Server 2008 database. In a number of earlier posts, I've described this approach, where the database is being automatically created as an Immutable Shared Fixture.

While I prefer this approach, it has a few disadvantages that may or may not force you to look for alternatives:

• The entire test suite must at least have sysadmin privileges on the SQL Server instance, because otherwise it's not possible to create and delete databases. Since unit testing mostly focus on the functional aspects of a SUT, this may not be that important, but it's certainly more desirable if unit tests can execute under least privilege.
• Some run-time environments where the test suite will execute may not have SQL Server installed - e.g. my team's current build server doesn't have SQL Server installed, so even though my machines have, that's not going to help me if I want my tests to be running as part of the Build Verification Tests.

A reasonable alternative to SQL Server is SQL Server Express, and particularly its User Instance feature, which allows you to create a file-based database on the fly and still have full sysadmin privileges on just that database. Besides, in my particular case, our build server doesn't have SQL Server, but it does have SQL Server Express.

Since SQL Server Express uses the same relational data engine as SQL Server, it's a pretty decent alternative.

When I set up the database as part of my Immutable Shared Fixture, I want to create a one-off User Instance that can be deleted afterwards. An .mdf file in my temp folder does the trick.

As usual, I will set up and tear down the Immutable Shared Fixture using Installers. To do this, I need an Installer than can create and delete a SQL Server Express User Instance. As usual, I create a dedicated Installer for this purpose:

public class SqlExpressDatabaseInstaller : Installer

{ 

    public override void Install(IDictionary stateSaver)

    {

        this.ExecuteNonQuery("CREATE DATABASE [{0}] ON PRIMARY (NAME='{1}', FILENAME='{2}')");

        base.Install(stateSaver);

    }

    public override void Uninstall(IDictionary savedState)

    {

        SqlConnection.ClearAllPools();

        string dropStatement =

            "IF EXISTS (SELECT [name] FROM sys.databases WHERE [name] = '{0}') BEGIN DROP DATABASE [{0}] END";

        this.ExecuteNonQuery(dropStatement);

        base.Uninstall(savedState);

    }

    private void ExecuteNonQuery(string sqlStatement)

    {

        string databaseFileName = this.Context.Parameters["DatabaseFileName"];

        string logicalFileName = Path.GetFileName(databaseFileName);

        SqlConnectionStringBuilder scsb = new SqlConnectionStringBuilder();

        scsb.DataSource = @".\SQLEXPRESS";

        scsb.IntegratedSecurity = true;

        scsb.UserInstance = true;

        using (SqlConnection conn = new SqlConnection(scsb.ConnectionString))

        {

            using (SqlCommand cmd =

                new SqlCommand(

                    string.Format(sqlStatement, databaseFileName, logicalFileName, databaseFileName),

                    conn))

            {

                conn.Open();

                cmd.ExecuteNonQuery();

            }

        }

    }

}

If you compare it with SqlDatabaseInstaller, you will probably notice a lot of similarities. They both use SQL DDL statements to create and drop the database, but since SQL Express User Instances are file-based, we need to specify the file name explicitly in the CREATE statement.

Another difference from SqlDatabaseInstaller lies in the way the connection string is being built. First of all, SqlExpressDatabaseInstaller expects a file name from the Context instead of a full connection string. Subsequently, it specifies SQL Express as the DataSource and sets the UserInstance property to true.

Because of the similarities between SqlDatabaseInstaller and SqlExpressDatabaseInstaller, the obvious next step would be to merge the functionality of these to classes into a single class, but I haven't really had the need for that yet, although I may do so in the future.

Before dropping the database, I invoke SqlConnection.ClearAllPools to clear all connection pools; if I don't do that, the file will be locked and cannot be deleted. In my original post on data access component testing, I recommended disabling connection pooling for testing, but as one of my readers was so kind to point out, SqlConnection.ClearAllPools is a much better alternative, so I'm now using that exclusively. Had I posted SqlDatabaseInstaller today, I would have included a call to SqlConnection.ClearAllPools in its Uninstall method as well, so these two database Installers really are quite similar.

N-Tier Synchronization With The Sync Framework And WCF ChannelFactory (Part 2)

In my previous post, I discussed using ServerSyncProviderProxy with ChannelFactory<T>, and explained how you can get up and running with that combination. While the described modification is definitely necessary, there's at least one other caveat that I've encountered so far.

ChannelFactory<T> implements both ICommunicationObject and IDisposable, and it's necessary to properly close an instance. A ChannelFactory<T> instance and any proxies that it creates are connected, and closing the ChannelFactory<T> instance will also close the proxies. However, that's not true the other way around.

ServerSyncProviderProxy itself implements IDisposable, and it properly disposes of the proxy passed to it, but since it has no knowledge of the ChannelFactory used to create the proxy, it's not going to dispose of that.

This causes the ServerSyncProviderProxy to hang (and ultimately time out) when you try to dispose of it.

Lesson #2

When using proxies created by ChannelFactory<T>.CreateChannel, you must manually ensure that the ChannelFactory is properly disposed when the SyncAgent disposes. Fortunately, that's not very hard to do; it just requires you to hold a reference to it in your derived SyncAgent and create it like this:

this.syncServiceFactory = 

    new ChannelFactory<ISyncService>(binding, serviceAddress.ToString());

ISyncService proxy = this.syncServiceFactory.CreateChannel();

this.RemoteProvider = new ServerSyncProviderProxy(proxy);

Subsequently, you need to ensure that it's being properly disposed together with the SyncAgent itself:

protected override void Dispose(bool disposing)

{

    base.Dispose(disposing);

    if (disposing)

    {

        this.syncServiceFactory.Close();

    }

}

This will allow the SyncAgent and, implicitly, the ServerSyncProviderProxy to close immediately.

N-Tier Synchronization With The Sync Framework And WCF ChannelFactory (Part 1)

Sync Services for ADO.NET 2.0 allows a proper n-tier architecture where you can synchronize a client with a back-end data store via a web service. To do this, your web service must expose a simple interface consisting of four operations. As described in the documentation, one of these operations must have the following signature:

[OperationContract]

SyncSchema GetSchema(Collection<string> tableNames, SyncSession syncSession);

When used in a SyncAgent, a proxy is used with ServerSyncProviderProxy like this:

this.RemoteProvider = new ServerSyncProviderProxy(proxy);

One thing to notice about the ServerSyncProviderProxy constructor is that it takes as input any object (that is, an instance of System.Object). Obviously, you can't just pass in any object and expect it to work at run-time, so there's actually a sort of duck typing in play here.

As far as I can tell, ServerSyncProviderProxy uses Reflection to invoke the expected methods, such as the GetSchema method shown above.

That probably works well for a WCF proxy generated by Visual Studio (I don't know, because I haven't tried), but fails if you try to reference the service interface (let's call it ISyncService) directly and create a proxy using ChannelFactory<T>.CreateChannel.

ChannelFactory<ISyncService> cf = 

    new ChannelFactory<ISyncService>(binding, serviceAddress.ToString());

ISyncService proxy = cf.CreateChannel();

this.RemoteProvider = new ServerSyncProviderProxy(proxy);

If you do that, you will get a run-time exception. What happens is that ServerSyncProviderProxy expects the GetSchema method to take a string[], not a Collection<string>. Although I have no authoritative explanation for this, it's not that surprising, since the default proxy code generator for WCF (svcutil, Visual Studio) will create proxies with lists of elements represented by arrays - you have to explicitly state that you want collections instead, if that's the case.

Lesson #1

If you want to create a proxy instance using ChannelFactory<T>, you will have to change the signature of the GetSchema method to

[OperationContract]

SyncSchema GetSchema(string[] tableNames, SyncSession syncSession);

That will enable you to use a proxy created by ChannelFactory<T>, but will obviously break for proxies generated by svcutil or Visual Studio. If you must support both scenarios, you will need to derive from ServerSyncProviderProxy and override the GetSchema method to do something more intelligent.

Update (2008-08-27): Part 2 is now available here.

An Overview of Unit Testing Duplex WCF Services and Clients

In the last couple of posts, I've demonstrated how to isolate implementation from WCF contract definition and behavior in a duplex communication scenario. These posts have been rather detailed, so it occurred to me that you might benefit from an overview.

The main goal was to ensure testability of implementations of both service and client. While the contracts reference WCF (System.ServiceModel), the implementations themselves only reference the contracts, and not WCF.

This diagram illustrates dependencies between the various libraries. The left side contains server-side components, while the right side contains client-side components. The only shared components are WCF and the Contract library.

Note that neither StuffServer nor StuffAgent has any dependency on System.ServiceModel, although they have a dependency on Contract. The same is true for both unit test projects.

The top boxes on both sides (MyServer and MyClient) are hypothetical top-level executables such as I've discussed before, although never shown any code from. These Humble Executables tie everything together by referencing all relevant libraries, but if you prefer, you can just as well use a configurable Dependency Injection container - Oran Dennison explains how to do this on Spring.NET, while Ayende has a Windsor ServiceHostFactory.

Here are the main posts on the subject:

• Unit Testing Duplex WCF Services
• Unit Testing Duplex WCF Clients

If you would like to take a closer look at my sample code, I've attached it to this post. As usual, the standard disclaimers apply.