Another reason why you can't prove code through testing

Why isn’t it possible to prove that a program is correct by testing it? To use testing to prove that a program works, you’d have to test every conceivable input value to the program and every conceivable combination of input values.

-- Steve McConnell, Code Complete, 2nd edition, Microsoft Press

It may be impossible to try every combination of inputs, but testing ‘normal’ input and edge cases is usually a safe way of avoiding a for() loop from -INT_MAX to INT_MAX. There’s another reason you can’t be sure that your tests have proved the code correct, however: the code you are calling may retain state.

C++ unit testing framework

I’ve been trying to find a decent unit testing framework for C++. My main requirements:

As the test writer, you should have to write as little code as possible in order to get keep the framework happy.

No external tools needed, or libraries to link to, if possible. I’m not against dependencies, but I’d like to use the framework anywhere there’s a C++ compiler.

It should be possible to run the tests in a variety of ways, i.e. you can just grab the built test suite and run it with whatever front end you fancy.

Liberal license. I like the MIT license, myself, but anything close is ok.

After I’d hacked something up myself, I was pointed to tut, which looks quite decent. It does everything with templates, which is cleaner than my cpp stuff. I’m no template guru, though, so I couldn’t figure out how to give names to tests. With tut, tests are numbered, which looks a bit annoying. Test 6 failed? What was test 6?

I carried on with my own hacked up framework, which seems to be approximately complete now. I’ll ask the tut author if he knows whether it’ll be possible to add names to tests, and to compile all the tests into a shared library for running with a separate front-end.

Quick attempt at framework may be found here.

With my framework, tests look like this:

#include "UnitTest.h"
#include "BankAccount.h"

UNIT_TESTS

TEST_DATA(BankAccount)

  BankAccount account;

TEST_END_DATA(BankAccount)

TEST_SET_UP(BankAccount)
{
  account.setBalance(3);
}

TEST_TEAR_DOWN(BankAccount)
{
  account.setBalance(0);
}

TEST(BankAccount, InitialBalance)
{
  ASSERT_EQUAL(account.balance(), 3);
}

TEST(BankAccount, Credit)
{
  account.credit(100);
  ASSERT_EQUAL(account.balance(), 103);
}

TEST(BankAccount, Debit)
{
  account.debit(100);
  ASSERT_EQUAL(account.balance(), -97);
}

TEST(BankAccount, Bogus)
{
  account.debit(100);
  ASSERT_EQUAL(account.balance(), -10000);
}

There are console-based and Qt-based test runners. The Qt version isn’t quite complete yet. It needs to show what went wrong in which test, and allow navigating to the source position, but it shows the general idea of what I’m trying to do.

C# parameter passing rules

This page is a guide to how parameters should be passed in C#.

Rules for value types

valueTypeName variableName	The normal way of passing.
ref variableName	Callee is invited to modify in-place.
out variableName	Callee must modify in-place.

Rules for reference types

ReferenceTypeName variableName	Callee may modify in place
ref ReferenceTypeName variableName	Callee is invited to modify in place or change what is referred to.
out ReferenceTypeName variableName	Callee must modify in place.

Examples

There follows a sketch of the declaration of a class with many methods, each handling different parameter types, passed in different ways. Look at each example below and refer back to the declaration of class G, to see the parameter declaration relevant to the call being made.

class G
{
 ...
 void useValueOf(string s);
 void modifyIfDesired(ref string s);
 void assignTo(out string s);

 void modifyIfDesired(object o);
 void modificationsWillBeIgnored(object copyOfO);
 void modifyIfDesiredOrPointToADifferentObject(ref object o);
 void assignTo(out object o);
}

Pass primitive type

g.useValueOf("hello");
string s = "world";
g.useValueOf(s);
// s must still be "world"

---

Pass primitive type when you want to allow it to be modified

string s = "bunnies";
g.modifyIfDesired(s);
// s might have changed

---

Pass primitive type, requiring that it is assigned to

string s;
g.modifyOrDie(s);
// Compiler ensures that callee assigns to s - gives error if not.

---

Pass reference type, callee may modify

Note that it is impolite for the callee to modify the argument. You should only modify an argument when it is passed as ‘ref’ or ‘out’. People don’t expect you to modify arguments which are passed ‘by value’, even though they really are references.

MyClass obj = new MyClass();
g.modifyIfDesired(obj);

---

Pass reference type

We really don’t want to see any changes the callee makes.

MyClass obj = new MyClass(); // implements ICloneable.
g.modificationsWillBeIgnored(obj.Clone());

---

Pass reference type

We are telling the callee that modifications are expected, though it doesn’t have to make any if it doesn’t want to. The callee may point our reference at a different object! Doing so is not a normal thing to do, however, and should be used only when absolutely necessary for some trickery - preferably with a mention in the method documentation.

MyClass obj = new MyClass();
g.modifyIfDesiredOrPointToADifferentObject(obj);

---

Pass reference type, callee must assign to it

MyClass obj;
g.assignTo(obj);
// obj has been assigned a value. Of course, it might have been assigned null!