Just a Thought...: Tag test

Test Driven Development 101

Berin Loritsch — Thu, 03 Jul 2008 11:47:00 +0000

I’m back after a long hiatus, and I’m probably talking to the air right now, but that’s OK. For the two or three of you actually listening to me, listen on. We have a number of projects of different sorts at the company I work for, and you’d be surprised at how few do test driven development—and how few even write unit tests. In this day and age, not writing tests that can be automated at all is inexcusable. At the very least, the tricky stuff should be thoroughly tested. This article is for anyone who is either skeptical about or interested in Test Driven Development. That includes managers as well as developers.

What’s the Problem, Man?

We all ( at least should ) want to write quality software, and we will take this as the “given” in geometry. Since we want to write quality software, how do we do it? In the early cowboy days of software engineering, it was painful to write code. First you did all your planning, wrote what you hoped would work, printed the stacks of punch cards, and then loaded it on the main frame. Which explains all the texts on the huge design up front methodology. However, that was before my time. When I first started, we wrote a little, ran the software, and debugged. Which explains why debuggers were so important at that time. However, times have changed. We now have a plethora of unit testing frameworks and maintainable build scripts to incorporate the tests into the build process.

As soon as we ran into some hard problems that really couldn’t be completely designed up front (like TCP/IP stacks), we developers had to figure out a way to make sure things worked properly. I mean, the spec is pretty complete, but there are a lot of details and practical limitations imposed by hardware that you won’t find in the spec. If you attempted to alter the spec to include all these corner cases, you would also lose any way of understanding how it is supposed to work. This is how the automated unit test was born. All these corner cases had to be accounted for so that future changes to the code won’t break the existing functionality. I think everyone recognizes the importance of testing. It’s just how, when, and where testing happens that people disagree on.

Another problem that is often brushed under the table is code slipping in that doesn’t need to be there. How often have you tracked down a problem only to discover code that was never supposed to exist be the culprit. You remove the offending code, and violá it works! The code could have been there from old legacy requirements that no longer apply, or it could be a developer without experience over thinking a problem. Nevertheless the code is there and it is causing problems.

If you adopt a clean as you go philosophy to developing software, you had better make sure you don’t accidentally breaking anything as you refactor your code. Even with “safe” refactoring, there is an inherent risk that you can inadvertently break something you didn’t think you would. If only there was a way to make sure all the important functionality keeps working…. Oh yeah, you do have a full test suite don’t you? Oh, it didn’t get written because you were under the gun and you were on a roll? Too bad.

Test Driven Development (TDD) was designed to not only address these issues, but to instill a discipline so that your unit tests would get written. Let’s face it, all people are lazy. It’s in our nature to not do anything we think is a waste of time. Sure we’ll do a little work now to avoid a bunch of work later, but we need to know it really is going to avoid a bunch of work later. What some people fail to realize about TDD is that the time will be spent somewhere. Either you write your tests up front or you spend time in a debugger later. Either you prove that your approach will work now, or you do it later. Either you break up your work into testable chunks now, or you attempt to do it later and fail at it.

OK, So How Does It Work?

At its heart, TDD is pretty simple—you simply perform a prove, fix, prove cycle. First you prove it doesn’t work . Then you fix the problem. Finally, you prove your fix worked . If you are lucky, your first “prove” step will prove the code already handles what you were thinking about. Write the test anyway, because you need to make sure that any changes will still support that test case. It’s fairly easy to see how to perform the mechanics of TDD, but less about it’s design implications without a little more explanation.

One of the side-effects of writing your tests before you write code is that it makes your code easier to test. Remember, people are lazy? Since a proper unit test sets up the environment and checks the effect of the call after, we make it easy to set up the environment. In fact, the less we rely on network connectivity or environment variables, the better. If a bit of code can be tested just by passing objects into it and examining the return value, then you aren’t going to be overly clever in your implementation. Easily testable code, also happens to be more modular. If you can pass in a mock object for something that would normally talk to the network so that you can have the mock imitate the situations you would encounter there, you can more easily test just the small unit you are working on.

Unit Tests vs. Integration Tests

In the best of all worlds, a project would have both. Unit tests make sure that the smallest unit (such as a method on a class) is doing what we expect it to do. A proper unit test also tests only one aspect of that method. It’s not uncommon to have several tests for the same method to make sure that all the corner cases are taken care of. Integration tests make sure all the different units work together like we thought they would. A unit test uses mock objects to isolate the thing we are testing from everything else. An integration test sets up the complete environment and runs the test against that environment. They are different tools for different problems. To do TDD, unit tests are required, but additional testing is a plus.

More often than not, code that is properly unit tested will work just fine. However, there are some issues that only crop up when the whole system is put together. Perhaps there is some race condition, or some complicated event loop that only appears in certain conditions. Once you track down the cause of the problem, you can write a unit test to reproduce the condition that caused the mess in the first place. With the new unit test, you fix the code, and everything should work in integration again.

Many times, the integration tests are all done manually. The challenge is that it is hard to set up a complete environment, test the user interface or system messaging, and evaluate the results automatically. The problem is that over time the number of tests that people have to do for a release becomes daunting. People are less picky than machines. If there is a tool to support integration testing like Selenium or some other testing framework, use it to at least catch regression issues. You know those issues where you accidentally break something that used to work when you add a new feature? Anything that was working and gets broken needs to be tested every time. Machines are good at doing rote repetition like that.

Battle Rhythm

When you sit down and start doing TDD, you’ll develop a battle rhythm. Most IDEs these days have support for running your unit tests without going through the whole build process. It’s pretty convenient, and it makes TDD a lot easier to do. So, you’ve got a new requirement and you need to get it working. It’s good to have a general idea of where you want to go, or how it is supposed to work, but don’t be a slave to that idea. We start writing the unit test at the easiest place it is to begin—whatever that may be. I personally find it is best to start with the happy path (the path where everything works as expected). For example, look at the pseudo code below:

Test String is a URL
--------------------
1. Use the string "http://bloritsch.d-haven.net" 
2. Pass the string to the String Utility "isURL" method
3. Assert the response is "true"

Of course, it’s pretty easy to make this one test pass. All we have to do is write the StringUtility.isURL() method to simply return true. No evaluation or anything. When we run the test, we prove that solution is good enough for now. So we need to start thinking about the next test. What if the string is not a URL? So we add the next test:

Test String is NOT a URL
------------------------
1. Use the string "I'm not a URL, ignoramus!" 
2. Pass the string to the String Utility "isURL" method
3. Assert the response is "false"

Now we’ve proven we have some work to do. So we change StringUtility to return true if the string starts with “http:”. It’s the simplest thing, right? But what if we want to include SSL encrypted URLs, or mailto URLs? So you add tests for them, and make them pass—without breaking the other tests you’ve written. All these tests are cumulative, so while they may have been extra work at the beginning, they can save your bacon later. Don’t forget about those corner cases, what if the string is null ? Etc.

By the time you are done with this method, you’ll have done the following things without even realizing it:

Documented what you consider a URL and what is not a URL (design documentation side-effect)
Proven the design works (design proof side-effect)
Tested the implementation (implementation proof)
Provided a safety net to do refactoring (supporting implementation malleability)

Sure the method is just a part of the overall design, but you’ve thought about and decided how the method is going to be used from the perspective of someone using the method. It’s a shift in thinking from being the “implementer” of the method, which usually results in code that is easier to use elsewhere. You’ve also introduced a level of trust in this method that you wouldn’t have if you just reached for that regular expression you found on the net to determine if something is a URL or not. You’ve also introduced a boundary where the implementation can be as simple or complex as you want—but code that uses the method won’t care about those details.

The battle rhythm of proving, fixing, and proving actually improves your development speed. It may not seem like it at first, but by testing all along the way we’ve minimized the time we will have to spend in a debugger. The ramp up time is a little slower, but as you get your battle rhythm going, you stay at a constant pace. Without TDD, I find myself working with bursts of productivity interrupted by long periods of finding out exactly where I went wrong in a debugger. With TDD, I find myself working at a steady pace, and those occasions where I missed something I spend much less time in the debugger. Once I’ve discovered the culprit, I add the test case that reproduces the error condition and then make it work. Now, when I refactor code I can make sure I don’t reintroduce the problem accidentally.

Silver Bullet?

There is no silver bullet, and no golden hammer that will make things work perfectly the first time. There are only tools that help you get closer to the ideal. TDD is a tool that helps improve quality from the start. Most detractors of TDD look at the claims of documenting your design as being false—or at least unreadable to normal people. I may give them that argument, but TDD isn’t about documenting design, it’s about building a better quality product with the minimal amount of investment. It’s about improving your productivity over the course of a software project. It’s about reducing the number of “doh!” bugs to virtually none saving your brain cells for the more complex problems. Finally, it’s about minimizing the risk involved in refactoring or even rewriting your software.

All of these benefits are things that the text books say are a good thing. It’s also done in a way that is less painful to developers. Writing documentation is a pain in the butt, however, writing test cases is something that directly benefits the developer. It benefits the project over it’s period of performance. Bottom line? More bang for the buck.

Using Java Enums for Finite State Machines

Berin Loritsch — Mon, 23 Jul 2007 12:57:00 +0000

Finite state machines are useful design constructs for a number of situations, although they seem to be fewer and farther between these days. Currently, the only place I tend to use them is when I have to write a parser by hand. Sure there are BNF parser generators around, but not all parsing requirements fit those restrictions. I have to parse legacy message formats which predate BNF parser theory (i.e. from the 1950s), so this is a useful tool to ensure that the message is properly formatted and all the information is pulled out properly.

According to Design Patterns by the GoF, the way to do an object oriented finite state machine is to use objects to represent each state, each with a common interface. My first introduction was a C++ program that was written with the old C mindset. That meant that the states were represented by an enum and all actions were taken with large switch or if/else hierarchies. I can see how using objects can clean things up, because the conditional logic would be decided by the state object. Of course, the state pattern in the GoF book required keeping the state in an external object and neglected to dictate how to change the state properly.

The Problem We are Solving

For my purposes, I have to populate a message object with all the information from a text message. That includes things like security markings, addresses, tags, captions, etc. The message format is distinct enough so that each line means something. Of course, there is a proper order of processing, but I can parse one line at a time. That simplifies things a whole bunch. I can have the parser work with an interface that looks like this:

public interface Partial {
    public State parse(Message message, String line)
        throws ParseException;
}

In Java 7 we will likely be able to use partials for this approach, which will clean up a lot of the code clutter. In that case the interface would look like this:

{Message, String => State throws ParseException}

Either approach you take will allow you to write the enumeration delegating the actual processing to anonymous classes. I’m sure you’re thinking, “My God! This guy’s off his rocker! I thought he liked elegant code!” Trust me, you’ll see the elegance in a minute. You aren’t going to do this all the time, but when the situation calls for it, you’ll appreciate it. The real benefit comes from testing.

Enums and Finite State Machines

Java enums are objects which is really useful. I’ve used this fact to associate sort order SQL snippets with an enum for the different sorting algorithms supported in a system, along with other uses. I decided to do an experiment with rewriting a parser we have. The parser works for the most part, but it leaves out some important information we want to support, and more importantly it is difficult to change. It needs some major rework beyond the scope of a bunch of refactorings. That is why I chose to rewrite it.

We have to start writing the State instances, and it really helps to have a starting point. For this blog, I’ll only split a message into header and body sections. There’s a whole lot more going on, but I just want to show how things work. The marker that splits the message header from the body will be a line that has “BODY” on it with nothing else. First, let’s look at our State enum. The important thing here is that we are not

public enum State implements Partial {
    HEADER(null),
    BODY(null);

    private final Partial partial;

    private State(Partial parser) {
        partial = parser;
    }

    public State parse(Message message, String line)
            throws ParseException {
        return partial.parse(message, line);
    }
}

The implementations of HEADER and BODY are null right now because we will get into it a bit later. First, you’ll notice a couple things about the enum. We are passing something that does work into the constructor, which also means that our enums can do work. For consistency sake we used the same interface for the enum as we did for the interface we pass into the constructors. If we were to use the closures spec, we wouldn’t have an interface to implement, so the method we provided would be how we access the blocks passed into the constructor. The spirit of the design is the same, it’s just that there is less extra code to type. Just so you can see what it looks like (assuming I have a better understanding of the spec), here you go:

public enum State {
    HEADER(null),
    BODY(null);

    private final {Message,String=>State throws ParseException}
           partial;

    private State(
            {Message,String=>State throws ParseException} parser) {
        partial = parser;
    }

    public State parse(Message message, String line)
            throws ParseException {
        return partial.invoke(message, line);
    }
}

In either case, the base design is identical. The only way to have the functionality of enum values change based on the value is to use the delegate approach. In short, we are passing in an object that does the work that is specific to that state in the constructor and calling it later when we call the parse method. It’s also important to note that enums are singletons by definition. There is one and only one State.HEADER enum value in the system, as there is one and only one State.BODY enum value in the system. That means the implementation has to be re-entrant. As long as you don’t attempt to keep any state in the objects you should be fine.

For the rest of the article, I’ll be focusing on the anonymous class approach (i.e. the first version). I’m assuming you can do the translation into closures later. Besides, I’m not sure if it would be legal to use the control invocation syntax for the constructor or not. This is a question for Neal Grafter, would this be legal syntax for the constructor of an object (it would be better if that’s the case)?:

HEADER(Message message, String line:) {
    HEADER
}

The State Implementations

The implementation of the state is very simple. We are providing an anonymous class (or closure declaration). The header is going to add the line of text to the header provided until we hit the “BODY” line. We aren’t going to copy that line at all. The important thing is that we can easily test these conditions in isolation from any other state. Let’s write some tests to make sure our implementation does what it is supposed to do (we are skipping the boiler plate JUnit code):

public void testCopiesLineToMessageHeader_and_ReturnsHEADERstate()
        throws ParseException {
    String line = "test line";
    Message message = new Message();

    State state = State.HEADER.parse(message, line);

    assertEquals( State.HEADER, state );
    assertEquals( line, message.getHeader() );
}

Currently our implementation compiles but only throws NullPointerExceptions because we haven’t given it anything to do yet. Let’s at least get this to pass. We have to rewrite the HEADER constructor in the enum:

HEADER(new Partial() {
    public State parse(Message message, String line)
            throws ParseException {

        message.addLineToHeader( line );
        return HEADER;
    }
});

That’s all well and good, but we need to make sure that we switch to the BODY state eventually. So let’s add a new test:

public testBodyLineReturnsBodyState_and_DoesNotWriteToMessage()
        throws ParseException {
    String line = "BODY" 
    Message message = new Message();

    State state = State.HEADER.parse(message, line);

    assertEquals( State.BODY, state );
    assertEmpty( message.getHeader() );
}

Now, all we have to do is make this pass in the HEADER enum:

HEADER(new Partial() {
    public State parse(Message message, String line)
            throws ParseException {
        if ( line.equals("BODY") ) return BODY;

        message.addLineToHeader( line );

        return HEADER;
    }
});

Now, if you are using Java closures, you simply cannot have multiple exit points. It’s easy enough to alter the logic. Some people don’t like what I did as a matter of principle. That’s OK. We know it works and its tested. We can refactor it later to our heart’s content. For the purpose of brevity, it’s up to you to do the same thing with the State.BODY implementation.

Using our Finite State Machine

Using the Finite State Machine we just created (granted it has only two states) is really easy. We know that we need a message object, and we need to iterate over the lines to a message. We’ll assume that you got it from some Reader. Here is a method that does the hard work for you:

public Message parseMessage(Reader in)
        throws ParseException, IOException
{
    Message message = new Message();
    State state = State.HEADER;
    BufferedReader reader = new BufferedReader(in);

    String line = null;

    while((line = reader.readLine()) != null) {
        state = state.parse(message, line);
    }

    reader.close();

    return message;
}

I left all the error handling code as an exercise for you, dear reader. If you want to provide accurate line numbers for your ParseException objects, you can surround the thing in a try/catch and use the following construct:

catch(ParseException pe) {
    // Rewrite the line number where the problem occurred.
    ParseException npe =
        new ParseException(pe.getMessage(), lineNumber);
    npe.setStackTrace(pe.getStackTrace());
    throw npe;
}

You have to keep track of the line number yourself. The lineNumber variable was incremented in the while loop in my code. I also wrapped the IOException in a ParseException keeping track of the line number so the signature was simplified. Of course, you’ll want close the reader in a finally clause.

In Conclusion

You aren’t going to write a finite state machine every day, but when you do you’ll want to keep things as simple as possible. The approach I outlined is very handy in the sense that you can completely test each state in isolation from the others. Because it is its own object, you don’t have to worry about testing the whole of the application to ensure your states are doing what they were designed to do. I typically have each State instance tested in its own TestCase object. This makes it particularly clear what each state is supposed to do, and how/when transitions take place.

I find that FSM are much easier to understand when you think about one state at at time. Trying to keep the whole “if this, then that, or is it the other thing” reasoning can be avoided that way. Doing it the old procedural approach is really not very useful these days. Don’t query data and keep control. Ask your objects to do work for you. Delegate.