Just a Thought...

Process . It’s a dirty word around some people. All it means is “A way to get something done”. Nothing more, nothing less. However, we have things like ISO 9001, CMM, Agile, XP, Scrum, etc. all presenting themselves as the way to develop your software. Each has it’s benefits and drawbacks, and each works with a different kind of client. If you hear anyone tell you that their way to do things is better, faster, etc. than anything you are doing without knowing the details you know you are dealing with a snake oil salesman. Even within the CMMI framework, there is a lot of leeway for you to have a heavy process or an agile process.

Process is just a way to get things done.

I’ve heard of some people’s software development process as “write a little code, run the app, write a little more code, and so on.” That process can work for some people, but not for everyone. I happen to be a fan of Test Driven Development coupled with Continuous Design—a fairly Agile approach. However, I know there is a lot of people who just can’t think that way. That’s fine. We just probably won’t be working on the same projects.

Manual Process vs. Automatic Process

The first aspect software development process is finding the right ratio between things you do manually, and things that the process takes care of for you—as a side effect of just doing the process. We programmers tend to be lazy, and so we do whatever is easiest on us to do. Ironically, we are so lazy sometimes that we would rather not invest the time to set up the automation unless there is a clear reason to do so. It’s actually a good thing because too much automation can get in the way of making changes. Of course, manual processes don’t scale very well.

Anything you do manually is something you have to remember to tell someone else.

When you are working alone, you can take all the shortcuts you want because you don’t have to explain what you are doing. The problem comes when you add someone to your team. All the things you were doing you now have to explain to someone else and get them to do it, or it won’t get done. Heck, sometimes you either forget a step even when you are by yourself. It’s for that reason we even consider automation. The places where we have the best benefit from using tools include configuration management and creating builds. I wouldn’t imagine writing any code without version control software of some sort even if I’m by myself. The versions are managed for you so you can go back to a release to reproduce issues if necessary, the code is backed up and managed beyond your own hard drive, and as soon as you add more people to the team it takes care of merging code for you. The benefits far out way the alternative. Creating a build might be easy or difficult depending on the structure of the complete application. The build process will create your distributables and run any tests you have included. Tools include Make, Rake, Ant, etc. and have differing levels of complexity. The important aspect is that you aren’t relying on someone else to make the identical configuration settings in their IDE to get the same executable. It’s all taken care of by the script.

There are other areas where automation can help out, but it does depend on the culture of the project, and the ease of setting it up. How do you manage your issues? Many tools generate reports just from how you use the application. You can show how quickly you are burning through your issues, and you can see how fast or slowly you are approaching the completed release. Sometimes you can add flags to the tool to notify you if values are outside a certain set of parameters.

Testing your Process

In any software project there are number of things you have to keep track of, and your process (the way you get things done) should make it easier. So how do we know if the process is working or not? You have to start by identifying what’s important. What is going to make your client lose confidence in you? If you do nothing else, what has to be done? Do you really need full Earned Value Management (EVM) or can you get away with proving you are making progress at the expected rate. Many times it’s enough just to know when you are falling behind. One thing that’s a must in any process is that you produce the deliverables.

If anything important slips through the cracks, your process is broken. Period.

Once you’ve identified the important stuff you can’t miss, you keep track of it. So how do I do that? You do your process. If you can ensure your process takes care of tracking the important stuff, all the better. Keeping track of something is what some people call “metrics”. The thing I don’t like about metrics is that you are essentially associating numbers to parts of your process. Numbers people like metrics because they have something to play with. The reality is that numbers don’t mean anything at all. Trends mean something, numbers don’t. If you are getting better, (i.e. your trend is going in the right direction), your process is working. If you are getting worse, (i.e. your trend is going in the wrong direction), your process needs to be fixed.

So how do you track trends, and when does a change really mean something? The tricky thing with trends is that if your sample is too short you might make adjustments that are not warranted, but if the sample is too long you won’t make the adjustments soon enough. There is no substitute for your gut feeling when you are just starting to track how well your process is working. The trick is not to be alarmist, and make proper adjustments. It’s like learning to drive for the first time. You either make too many large adjustments and overcompensate, or you make too many small adjustments. The good news is that you aren’t going to crash right away.

There are a few things that process people (you know those heavy process proponents) forget to look at while they are testing their process. How much work is needed to feed the process, vs. how much work is spent actually making deliverables? If you need 5 people that do nothing other than track the project’s progress when you have a team of 3 developers generating code, there’s something a little off-kilter. Notice that I said the 5 people are feeding process. Ensuring you have a quality product is work that’s done for the deliverable. Ensuring you have a quality code review is not. That’s “meta-work” or work about work. It’s stuff done to feed the process. Even the discipline of code reviews can be brought into question as long as you gain the benefit another way. A cod review is merely a means to both mentor other team members and ensure the quality of the product. If your process handles those benefits in other ways, then you don’t need a formal code review.

Fixing a Broken Process

If you are like me, then you really won’t wonder about how you fix a broken process. You just make the changes on the fly and the process evolves. You know these “lessons learned” meetings people have that dig up all the surprises and issues that we ran across? Are you merely identifying lessons, or are you actually learning them? If a technology choice turned out to be bad, how can you make sure the same mistake doesn’t happen again?

You haven’t really learned a lesson unless you did something to correct it.

If you discover that something you are doing isn’t lending any value whatsoever, you shouldn’t be doing it anymore. The advantage of changing your ways vs. logging issues is that it actually fixes the cause of your heartburn. Logging an issue helps you vent and get things off your chest, but it’s still there until you actually do something about it. That’s why the traditional “lessons learned” meetings don’t work. They let people vent, and then nothing is done about it. “Yeah, that was painful. We’ll be doing it again next go ‘round too.”

One of the biggest lessons I learned is that waiting until a “lessons learned” meeting to vent is waiting too long. If you are experiencing some heartburn now, fix it now. If you spend more time collecting numbers than you do using those numbers (um, I mean metrics), then just how important are those numbers in the grand scheme of things? Can you find another way of collecting those numbers? It’s funny, but machines are much better about collecting them than people are. There’s probably some way of making those numbers be a byproduct of how you do things. In essence, if you can make your tools support the way you do things and keep track of the numbers for you, you are better off. As long as you are depending on people to actually log numbers, you are depending on the most unreliable source for numbers. It’s a pain to keep track of them. Even if we only need to type them into a form, it’s still a pain to keep track of them. Let your tools do all the data entry for you.

The bottom line is prove you need to change, change, and then prove your change is helping. It’s TDD for process. If you see a bad trend and the process is the culprit, prove it. Then decide what a good trend would be. Make your changes and see how close you are to the results you want. You may find that your change is “close enough”, in which case you are done. Of course, you may find that you just introduced more work than the way of doing things, in which case you revert back or make a different change.

I’d like to tell you I get everything right the first time, but then I’d be lying. However, what if I told you I got it wrong on purpose? Would you believe me? That said, the fundamental premise of YAGNI (You Ain’t Gonna Need It) is that you do the simplest thing that can work, and refactor later. The principle behind continuous design is that you put in enough structure to work, and refactor and modify as you go on. As you learn how your application’s structure is growing, you find places where you’ll need to help it out a bit.

One such example is the Controller base class in the Java ActionPack code. I started out with a base structure that I need an instance of a controller, populated with the current request and response objects—similar to how JSPs work. Then I started adding some helper methods to make working with controllers easier. Today, I have a mixture of an object factory (and its supporting structure) and helper methods and attributes. The Controller code is getting pretty big, and it may be hard to draw the line between support and factory related code.

So what do you do? The answer is obvious, you refactor. Now is the time to separate the factory code from the Controller base class. According to the GoF Design Patterns book, the proper way to do a factory is to have a factory class whose whole responsibility is to create instances of a different class. In short one objects makes a whole bunch of another object. So why didn’t I start there? In the beginning the Controller class was primarily a factory itself. We really didn’t need to separate things. YAGNI. Now that the code has grown, and we’ve added a lot more support, the time has come to use the factory pattern as specified in the GoF book.

Designing for Change

It should be obvious, but one of the chief principles of object oriented programming is to hide the details of how to do things, and you just tell objects to do a job. You only make available what you intend users of the object to use. That way we can hide changes to the algorithm without breaking compatibility. For example, now that I’ve identified a weak point in the Java ActionPack code, I need to be free to fix it without breaking every web application using the software. Part of the reason for having the Dispatcher and Controller classes in the same package is to provide package access to the factory methods on the Controller class. That means I can safely make the change because that part of the API was never made available to the average user.

Both public and protected methods are usually exposed to the user in some way. Either they are helper methods such as “redirectToAction”, or they are meant to be accessed from anywhere. Package protected (a Java unique feature) and private methods are typically where the guts of things live. They are hidden APIs that can be adjusted as you wish without fear of affecting users.