Another reason why you can't prove code through testing

Why isn’t it possible to prove that a program is correct by testing it? To use testing to prove that a program works, you’d have to test every conceivable input value to the program and every conceivable combination of input values.

-- Steve McConnell, Code Complete, 2nd edition, Microsoft Press

It may be impossible to try every combination of inputs, but testing ‘normal’ input and edge cases is usually a safe way of avoiding a for() loop from -INT_MAX to INT_MAX. There’s another reason you can’t be sure that your tests have proved the code correct, however: the code you are calling may retain state.

C++ unit testing framework

I’ve been trying to find a decent unit testing framework for C++. My main requirements:

As the test writer, you should have to write as little code as possible in order to get keep the framework happy.

No external tools needed, or libraries to link to, if possible. I’m not against dependencies, but I’d like to use the framework anywhere there’s a C++ compiler.

It should be possible to run the tests in a variety of ways, i.e. you can just grab the built test suite and run it with whatever front end you fancy.

Liberal license. I like the MIT license, myself, but anything close is ok.

After I’d hacked something up myself, I was pointed to tut, which looks quite decent. It does everything with templates, which is cleaner than my cpp stuff. I’m no template guru, though, so I couldn’t figure out how to give names to tests. With tut, tests are numbered, which looks a bit annoying. Test 6 failed? What was test 6?

I carried on with my own hacked up framework, which seems to be approximately complete now. I’ll ask the tut author if he knows whether it’ll be possible to add names to tests, and to compile all the tests into a shared library for running with a separate front-end.

Quick attempt at framework may be found here.

With my framework, tests look like this:

#include "UnitTest.h"
#include "BankAccount.h"

UNIT_TESTS

TEST_DATA(BankAccount)

  BankAccount account;

TEST_END_DATA(BankAccount)

TEST_SET_UP(BankAccount)
{
  account.setBalance(3);
}

TEST_TEAR_DOWN(BankAccount)
{
  account.setBalance(0);
}

TEST(BankAccount, InitialBalance)
{
  ASSERT_EQUAL(account.balance(), 3);
}

TEST(BankAccount, Credit)
{
  account.credit(100);
  ASSERT_EQUAL(account.balance(), 103);
}

TEST(BankAccount, Debit)
{
  account.debit(100);
  ASSERT_EQUAL(account.balance(), -97);
}

TEST(BankAccount, Bogus)
{
  account.debit(100);
  ASSERT_EQUAL(account.balance(), -10000);
}

There are console-based and Qt-based test runners. The Qt version isn’t quite complete yet. It needs to show what went wrong in which test, and allow navigating to the source position, but it shows the general idea of what I’m trying to do.

C# parameter passing rules

This page is a guide to how parameters should be passed in C#.

Rules for value types

valueTypeName variableName	The normal way of passing.
ref variableName	Callee is invited to modify in-place.
out variableName	Callee must modify in-place.

Rules for reference types

ReferenceTypeName variableName	Callee may modify in place
ref ReferenceTypeName variableName	Callee is invited to modify in place or change what is referred to.
out ReferenceTypeName variableName	Callee must modify in place.

Examples

There follows a sketch of the declaration of a class with many methods, each handling different parameter types, passed in different ways. Look at each example below and refer back to the declaration of class G, to see the parameter declaration relevant to the call being made.

class G
{
 ...
 void useValueOf(string s);
 void modifyIfDesired(ref string s);
 void assignTo(out string s);

 void modifyIfDesired(object o);
 void modificationsWillBeIgnored(object copyOfO);
 void modifyIfDesiredOrPointToADifferentObject(ref object o);
 void assignTo(out object o);
}

Pass primitive type

g.useValueOf("hello");
string s = "world";
g.useValueOf(s);
// s must still be "world"

---

Pass primitive type when you want to allow it to be modified

string s = "bunnies";
g.modifyIfDesired(s);
// s might have changed

---

Pass primitive type, requiring that it is assigned to

string s;
g.modifyOrDie(s);
// Compiler ensures that callee assigns to s - gives error if not.

---

Pass reference type, callee may modify

Note that it is impolite for the callee to modify the argument. You should only modify an argument when it is passed as ‘ref’ or ‘out’. People don’t expect you to modify arguments which are passed ‘by value’, even though they really are references.

MyClass obj = new MyClass();
g.modifyIfDesired(obj);

---

Pass reference type

We really don’t want to see any changes the callee makes.

MyClass obj = new MyClass(); // implements ICloneable.
g.modificationsWillBeIgnored(obj.Clone());

---

Pass reference type

We are telling the callee that modifications are expected, though it doesn’t have to make any if it doesn’t want to. The callee may point our reference at a different object! Doing so is not a normal thing to do, however, and should be used only when absolutely necessary for some trickery - preferably with a mention in the method documentation.

MyClass obj = new MyClass();
g.modifyIfDesiredOrPointToADifferentObject(obj);

---

Pass reference type, callee must assign to it

MyClass obj;
g.assignTo(obj);
// obj has been assigned a value. Of course, it might have been assigned null!

Breadth-first directory traversal without recursion

Update 2011-05-01: This should probably be rewritten to use the new IEnumerable-returning directory listing method.

public const uint DefaultMaxDirectoryDepth = 32;

public delegate void DirectoryVisitedDelegate(string path, IFileSystem fileSystem);

public static void Traverse
(
    string rootPath,
    DirectoryVisitedDelegate visitedDelegate,
    IFileSystem fileSystem
)
{
    Traverse(rootPath, visitedDelegate, fileSystem, DefaultMaxDirectoryDepth);
}

public static void Traverse
(
    string rootPath,
    DirectoryVisitedDelegate visitedDelegate,
    IFileSystem fileSystem,
    uint maxDepth
)
{
    char[] directorySplitChars = new char[] { Path.DirectorySeparatorChar };
    Queue<string> directoryQueue = new Queue<string>();

    directoryQueue.Enqueue(rootPath);

    while (directoryQueue.Count > 0)
    {
        string path = directoryQueue.Dequeue();

        try
        {
            foreach (string subDirectoryPath in fileSystem.GetDirectories(path))
            {
                if (subDirectoryPath.Split(directorySplitChars).Length < maxDepth)
                {
                    directoryQueue.Enqueue(subDirectoryPath);
                }
            }
        }
        catch (UnauthorizedAccessException)
        {
            // We can't look in that directory. Never mind.
        }
        catch (FileNotFoundException)
        {
            // The directory has gone away. Never mind.
        }

        visitedDelegate.Invoke(path, fileSystem);
    }
}

Boolean parameters

When I first started developing software, one thing that annoyed me immediately was when people used boolean parameters where their meaning wasn’t immediately obvious.

This is ok:

void setUserIsDeadFlag(bool)

This is not:

User findUserByName(string, bool)

In the documentation, the parameters will have names and possibly further info pertaining to their use, but when I come to write code, I’m forced to add a comment explaining what the parameter is, so that I can read my own code later:

User bob = findUserByName("bob", false /* Don’t include dead users */);

I can’t even trim my comment down, because the parameter name is something like include_dead_users. If I simply used the name in my comment, I’d make it look like I was passing false because I wanted to include dead users, which would be confusing:

User bob = findUserByName("bob", false /* include_dead_users */);

The best solution to this problem (in all the languages I know) is to use enumerations, or at least constants. Here’s C++:

enum DeadUserInclusionPolicy { Include, Exclude };

Before we start rushing off though, what’s wrong with the above? Let’s have a look at a method signature and a call:

void findUserByName(string name, DeadUserInclusionPolicy deadUserInclusionPolicy)
User bob = findUserByName("bob", Exclude);

The enumeration is still no help, because ‘Exclude’ means nothing to the reader of our code. Let’s try again:

namespace DeadUsers { enum InclusionPolicy { Include, Exclude } };

void findUserByName(string name, DeadUsers::InclusionPolicy policy)
User bob = findUserByName("bob", DeadUsers::Exclude);

Finally we have readable ‘user’ code. In VB.NET and C# you don’t need to bother with a namespace and creative naming, because when you pass a value from an enumeration, you have to prefix it with the name of the enumeration, e.g. DeadUsersInclusionPolicy.Exclude

A large proportion of code that I come across has the boolean parameter problem. When I’m using C++ or C#, I add a comment to remind myself what I’m passing - and try to remember to update the comment if I change the value I pass. When I’m using VB.NET, I get to use a feature of the language to make things explicit: named parameters.

IO.Directory.Delete("c:\", Recursive:=True)

That’s better! Not only does this help where I have to call a method with boolean parameters, it also allows me to call methods with multiple parameters with less worry that I’ll get them mixed up:

Dim Bob As New User(Name:="Bob", Surname:="Holness", Whereabouts:="Unknown")

Adding a doctype to System.Xml.XmlDocument

Doc.AppendChild _
( _
    Doc.CreateDocumentType _
    ( _
        name:="html", _
        publicID:="-//W3C//DTD XHTML 1.0 Strict//EN", _
        systemID:="http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd", _
        internalSubset:=Nothing _
    ) _
)

Allowing anonymous access to an IIS virtual directory through code (programmatically)

var entry = new System.DirectoryServices.DirectoryEntry(@"IIS:\\127.0.0.1\W3SVC\1\Root\" + VirtualDirectoryName);
entry.Properties["AuthAnonymous"][0] = true;

entry.CommitChanges();