Posted on

Software Estimates Are Hard

Hopefully we are better than this…

I suck at software estimates. And I bet you do too. So many software projects get delivered late. I am getting better at them, but it is going to take time. Let us consider a few reasons why they are so challenging.

Not Enough Time

This is a problem that is often overlooked. Worse still, managers often do not understand that this a problem in the first place. There seems to be this idea in some of their heads that developers know how they are going to solve the problem as soon as it is given to them.

The reality is that developers are often given tasks so poorly defined that they barely even understand what the problem is. They have to dig through unfamiliar code. What snares lie in wait for them there only God knows. Even after digging though that code and hounding people for better requirements, they only have a faint idea of how they are going to approach the problem. And the approach will likely change tomorrow.

So if a meeting gets called to introduce a brand new project and get a rough estimate of how long it would take, that estimate is going to be wrong. It is just guess work at that point. Coming up with a reasonably accurate estimate takes work, and steps cannot be skipped.

Sometimes we don’t even have time for that’s what she said jokes.
Not Breaking Things Down

This is such a simple concept, yet I am guilty of neglecting it all the time. How many of these lines do you catch yourself saying?

Ummm it would maybe take a sprint or two?
I should have that done in a few days.
I don’t see it taking more than a week.

You can hear the underlying message. The person uttering phrases like these has not broken down the development task at all. Does this developer know which methods he is going to write? How the different components will interact with each other? How the code behaves today? What automated tests need to be written? The only effective way I’ve found to produce reasonable software estimates is to estimate small things and add them together. The bigger the task is the worse we are at estimating it because by its very nature we are omitting important details. It is much easier to accurately estimate “Write a method that checks if a string is a guid”, than it is to accurately estimate “Write a point of sale system”. The further you can break down your tasks the better off you will be.

Forgetting About Other Stuff

We are all more than our jobs. We have a life outside of work, and sometimes we forget about that when we are making our estimates. Vacations and doctor’s appointments should be considered. If you are a leader on your team you also need to consider how your absence will affect the more junior members of your team. Will you need to leave more concrete direction for them? How much time will that take? You also need to think about what a typical work week looks like for you. The truth is, on any given day we often find ourselves working on just about everything except what we thought we were going to. Those disruptions and urgent requests can be really unpredictable, but they must be considered in order to produce realistic estimates.

How can we get better?

The first step in getting better at software estimates is to communicate well with your manager or product owner. Ask for some time to come up with an estimate. You probably won’t get as much time as you want, but it is way better to put a little thought into your estimate than to put in no thought at all. This is crucial because you can’t break down the task instantaneously. This step must happen for any other advice here to be helpful. With this accomplished you can begin to plan the work and start uncovering the time consuming tasks that are currently shrouded by the veil of your ignorance. If you can identify these up front you are really helping yourself out. That is really the key to it all: gathering more information about the future. The more information you have the more accurate your estimation will likely be.

It is really a shame my manager won’t let me make my estimates after the project is complete. I would be perfectly accurate!

Posted on

How A Web Application Works: An Epic Tale of Courage and Sacrifice

how a web application works featured image

The smell of blood fills the air. At Camp Client, the battle rages on. The Empire of Users have been relentless in their assault against the Browser Alliance. While the Chrome and Firefox Battalions have been handling the swarm with commendable bravery, the Internet Explorer Battalion is suffering heavy casualties. Leadership is sorely lacking. As pressure from the Users mounts, the regiment desperately needs instruction and supplies from Server Headquarters so that a swift response can be made.

Unfortunately, the geography of the surrounding countryside impedes communication efforts. Server Headquarters is not far away, but it is situated on the other side of the deep, dark Internet Chasm. The only way across is a narrow, rickety collection of rope and wood called the HTTP Bridge. This bridge is unstable, goes through a wall of fire, and is really only useful for sending messages and a handful of supplies back and forth. But it is an extremely important and strategic link. Without it, communication would be lost as would any hope of victory.

A young squire is sent with a request in hand across the bridge to bring word of the camp’s needs to Server Headquarters. The journey is not for the faint of heart, but when you’re a grunt you have no choice but to be courageous. After making the perilous journey across the chasm he presents the request to Officer Web Server.

Officer Web Server has the noble task of accepting requests as they come across the bridge and delivering them to the right people. However, he is a little socially awkward, and most of the time he gets his charismatic best friend, Officer Application Server, to do the heavy lifting.

Officer Application Server rubs shoulders with all of the Java Generals, and knows which ones he should pass on the requests to. Sometimes this dynamic duo has trouble finding the correct general, but that is usually only because the request was lacking in information. The communication protocol has been well established, so you can hardly blame them for throwing out bad requests from careless soldiers. When these two are given a message that is not malformed they are a shining example of discipline and reliability. Luckily for the Browser Alliance, the message they receive from the young squire is crystal clear. They deliver the message to the correct Java General with great haste.

Now the Java Generals did not rise to their station in life by accident. They are smart, capable planners with deeply strategic minds. They are intelligent, but also wise. Few of these generals make any decisions without first consulting the Great Oracle™. Only then will they decide what to provide their men back at Camp Client. After ending their session with the wise old master they know what they must do. Unfortunately, details are not their strong suit and they tend to be a little verbose. If they were the ones who had to communicate the message and package up the supplies the war would surely be lost.

Fortunately, this is what Officer Template Engine was born to do. He knows what the Java General wants to say and communicates this information articulately. He drafts the final response and gathers the supplies that get delivered back to Officer Web Server and sent with another poor young squire across the HTTP Bridge.

Camp Client is overjoyed upon receiving the response. Much needed supplies of HTML, CSS, JavaScript, and memes have been brought to the front lines. The soldiers survey the bounty that lies before them. “Just wait until those Users see this”, one combat veteran remarks with satisfaction. The other soldiers nod in agreement. They will live to fight another day. Many soldiers have been lost in this great war, and many more will be cut down before it is over. Though their names may change, the legend of their bravery will never fade away. This same story will be played out again and again as long as good men stand against the forces of darkness.

Posted on

Getting Bit By BigDecimal

I recently wrote some Java code to a port an existing feature into a newer architecture. In a nutshell, what this code did was take a fixed total price and recalculate what the unit price and discount should be at different quantities. Being the well meaning, responsible developer that I am, I decided to refactor the logic to make it easier to read for the next poor sap who would have to come behind me. As is so often the case, that poor sap turned out to be me.

One of the things I decided to do was put the data in a real object. Like much of the data in the legacy application that I battle with on a daily basis, this data was “Stringly typed”. The whole app is littered with List<String>, Map<String,String>, and the abomination of desolation that is the List<Map<String,String>>. This code was no better, and half the logic was just converting between Strings and BigDecimal objects. So I decided to be the hero that Gotham deserves and replaced these horrid beasts with real data types and real objects.

I confidently released my code into production. Unfortunately, I also released a bug into production. In the original implementation, an error was thrown if the amount was less than $0.01, but not if the unit price was $0.00. In my buggy implementation, an error was being thrown when the unit price was $0.00. I was confused. I was presented with the holy grail of unchanging, well-defined requirements. All I had to do was make the code do what it did before. How could this happen? After digging into the code the problem was obvious. I had gotten bit by BigDecimal.

First, the fix. I had to replace the red line with the green line like so :

At first this had me scratching my head. Why didn’t zero equal zero? Well, according to the documentation , the equals() method considers two BigDecimal objects equal only if they are equal in value and scale. The compareTo() method, on the other hand, will not consider scale when evaluating equality.

Luckily the fix was simple, but I was frustrated that I didn’t catch it before release. The problem reminded me of a few valuable lessons in crafting software.

The first is to read the manual. I’m a big believer in writing self-documenting code and using comments sparingly to document ideas that you can’t express in code. However, if you are going to use code that somebody else wrote, you better know what it does. In my case that was the Java standard library, but the same principle applies to any third-party dependencies that you are leveraging. Unlike most closed source internal software, major libraries and frameworks have nice documentation that is kept up to date. Reading the documentation for those libraries is how you learn to use them effectively. On top of that, read the code itself.

The second is to write automated tests. I didn’t do it because I was working with legacy code and it would have been a pain in the butt. I should have done it anyway.

The third is that building software is rarely as simple as it seems. Doing it professionally for a few years will pound humility into you pretty quickly. Still, as you see the same problems repeated over and over, it is easy to become cocky. The devil is in the details, and we often stumble over the simplest stuff because we don’t pay close enough attention. On many occasions I have seen large, risky projects released with a few minor bugs and small projects released with SLA violations and data loss. I think this is because we do a better job of testing and code reviewing when we feel the risk is high. We would do well to remember this phenomenon when we begin to think we are God’s gift to programming.