So what was a wonderful project to start with became a nightmare, but I finally see the light at the end of the tunnel with it. Because this project started off so innocently, and became such a disaster, I feel like I should share some of the mistakes I made along the way, and some of the insights that I have gained in both 1) extracting myself from the problem and 2) not allowing myself to get sucked into issues like this again. So lets start with a little bit of history of the project and the situation and go from there.
The development company, which I will refer to as “E” in this post is one that I have worked with for a while. They are good, skillful and friendly, they also do have a history of not generating sufficient (or any) spec’s for projects before they sign deals. This is a known problem, and something that both developers and designers have been complaining about for a while.
They brought me a project, which another development group within their company had been working on, and which had gone woefully over time and was now about to go over budget. This project was needed completed in a rush and the original developers had walked away in frustration. I agreed to see what I could do about getting things up and running that weekend, and though the code base was a horror beyond compare, I did, though I will be the first to admit/point out that most of the sight still did not work at all. (at least the core functionality now ran)At this point the client, hereafter called Mr. X, asked “what will it take to make this right?” So I talked with the original development team, who provided a unified and consistent answer of “redo from scratch”, and I looked at the database carefully, which was a complete disaster, and I looked at the page-flow / usability which was also a complete disaster, and I thought about it. Now I have a long developed standing principal that a complete rewrite is almost never the answer. On the other hand the original development team was saying the same thing, and the code was _really_ bad and crufty, so I thought “hey, this is probably the case that makes it almost”. So I returned my answer: I think you need a rewrite, and it will provide you with a working site, it will take 80 – 120 hours.
This was the single largest block of mistakes, and I’m going to try to enumerate some of them.
1) Never rewrite from scratch. This was not the exception. I’m beginning to wonder if there ever actually is one
2) If the original dev team has thrown up their hands and stormed off, its probably for a serious reason, usually either management or the client. I know management here and they are kind, so I should have suspected Mr. X
3) “Provide you with a working site” is not a spec. This one is really really bad
4) “use the old site as a spec” is actually a very solid spec, but also one you don’t want to get involved with, particularly if the old site did not work.
5) Don’t let consensus view make you feel ok about your choices. Just because all the other dev’s say it needs a rewrite doesn’t mean it needs a rewrite
6) Lawyers, by and large, are very very detail oriented clients, Any client that you anticipate being detail oriented, such as Mr. X who is a lawyer, should get an *estimate* not a price, and should understand that they are paying time and materials, to go thru as many iterations as they like before launch.
I spent the next two weeks grinding out a complete rewrite of the site. It worked quite well. (in all the time I have spent reviewing it, I have only found 3 significant bugs, as well as a slew of very minor ones & generally weak security, which is pretty damn good for a project on this timeline) I used cake to write it. I fixed the page-flow and usability issues, I did triage some of the extraneous features to get it done in budget and on the new schedule. And I pulled it in at a cool 99 hours. (this was also my first cakePhp project, and I now have an overall positive reaction to cake)
So we take it to get skinned, and the designer puts some templates & CSS on the thing, and shows it to the client. She doesn’t like the design now, she wants it to look _exactly_ like the old site. This should have been a huge red-flag, and honestly was, but we didn’t react to it properly. It was noted and nothing was done about it. So the designer pulls out a bunch of HTML from the old site and slaps it into the template files, and I take several hours and reconnect everything. Now there are some bugs, because of all the retro-fitting.
Next the client wants all of the items that got triaged back. They come up one at a time, mixed in with other requests. This, though it sucks hard, is reasonable. I thought Mr. X was on a tight time-table, but apparently he isn’t and is more than happy let the project be late in order to get all his features. If he had said that in the first place I would have gone ahead and put them in originally. I grumble and do it.
Next comes a slew of 1) minor textual tweaks, 2) minor graphical tweaks, 3) Major user interface tweaks, and 3) minor functionality tweaks. I can’t really comment on the textual tweaks or the graphical ones, but the UI changes are steadily bringing the UI back to match the first, highly dysfunctional one. It turns out that this was not broken as I had originally thought, but was actually the way that the client wanted things. Well, that was a communication’s disaster. At this point we are creeping up on a month past the date I gave for completion (so much for the rush) and the code is starting to look pretty manhandled. Frustration so running very, very, high in the team. The designer gets a “last list of changes” from Mr. X, who points out that we are on the home stretch. The last list of changes goes into a spreadsheet and we spend another week cranking them out.
At this point I’m not billing anymore, I’m way over estimate, and have resigned myself to just getting things done. I’m frustrated that we don’t have anything in writing that says that when we finish this “last list of changes” we are done, and so is our designer. Sadly that is something that the client just wasn’t willing to agree to (apparently)
The last list of changes gets down to two items which need some clarification, mostly because they are UI / biz logic and they make no sense at all. The designer goes back to clarify, and Mr. X keeps repeating at him “I need everything just like the old site”, as well as generating more and more additional requests. At this point, the designer breaks (on the inside). He get’s himself out of the phone conversation with Mr. X politely, and calls us back to say that he can no longer work with Mr. X and that if he ever has to talk to him again he’s going to flip out and do something violent. To say our designer was agitated is something of an understatement. I have also been working with this designer for over a year, and he has consistently been both levelheaded and laid back in every situation that I have been with him in.
This is the best thing that could have happened. Until this point no one was willing to play hardball, seeing our designer so angry drove home just how out of control this project had become. Like a slowly boiled frog, we were perishing but hadn’t noticed.
So I call the owner, we talk about what’s going on, he understands the situation, and then I call the client. Mr. X starts by apologizing for upsetting our designer, and reiterates that we are in the home stretch. I explain that there was some sort of communication failure with the designer, and ask him if he can walk me through what he talked about. On every page that we touched on Mr. X discovered a few changes that he felt needed to be made. It became readily apparent that he was discovering this as it went. This is the part where I’m proud of myself.
I stopped the conversation and explained that we needed a definitive list from him of what had to be changed in order for the project to be complete. He said, “no, I can’t do that, there are parts that I haven’t seen yet, and things I need you to change dramatically, I need to see those big changes after they are done to make sure that they match up with expectations”. This is where I finally stood up for the team and said “No, we can’t do any more development until we have an exact point by point list of what needs to be done for this project to be finished”. Mr. X expressed his dismay at being asked for something written in blood, and pointed out that he might find small formatting / textual changes after the fact, and that those would need to be addressed. I responded that we did need something “written in blood” and that the problem was that those small formating changes, while individually not a problem, in aggregate could and had been devastating. After some time discussing how the project got out of control, and what development typically looks like, I finally got Mr. X to agree. He is now working on a spec, written in blood, and though I anticipate it being long and hard, it does mean that the light at the end of the tunnel will be in sight.
The deeper moral of the story is: never work on anything until you have defined what “done” looks like. If you don’t know what success is, you can’t know when you will get there.
A corollary to this is: Every major player in a project needs to agree about what “done” looks like
and a second corollary is: It is never to late in a project to stop, and define what “done” will look like.
I hope you all learn from my cautionary tale.
Dude, have I been there.
The truest thing I ever saw in a production trade magazine is that “It’s the jobs that a company turns DOWN that determine whether a company is successful or not. Some jobs and clients are just so #$%#ed up that they bust budgets, drive staff away and make everybody wish they were pulling espressos at the Starbucks for a living.”
I remember a client who came to me, they had worked with a small company, and had done two prototypes with them. They wanted our company to do something quite different from either of the prototypes. Itold my boss, “These people can’t make up their mind. They drove the other people crazy and they wouldn’t work for them. Do not take this work!”
My boss took the work anyway. We lost over $200,000 on the project before we refused to work for them anymore (I refused to be the project manager.)
Anyway, I always say “Wisdom comes from having made all possible mistakes.”
Hi, Harry. Finally got around to fully reading this, and figured I’d weigh in.
I learned this lesson, too, the hard way. A year ago I was tasked with creating custom reports within our application for clients, who get to write their own specs for how the report should look and function. I learned VERY quickly which clients were problem clients from the vague specifications they provided and how some of those specifications didn’t quite fit with the underlying business logic of our product (which betrayed a basic misunderstanding on the client’s part of how the product works).
After one particular instance in which we deployed and redeployed one client’s changes at least 4 times before they stopped sending us additional “changes”, my managers and I sat down and wrote a formal procedure to ensure the specs were confirmed by the client and that our asses were covered once we unveiled the “final result”. First, the client requesting a custom report sends us their desired layout and specs, which we verify with the client where vaguaries and questions exist. Then, we create the report, but don’t check it into source control – first, we send a sample to the client to confirm with them that the product is finished, before any code check-ins or deployment to our production environment happens. If they send additional changes, we simply bill them more hours – and since the code isn’t yet checked in nor deployed, it’s trivially easy to fix and resend a sample. Since each round of changes costs more billable hours, clients generally get their specs straight by the first round of changes.
Thankfully, these custom reports usually take no more than 12 hours to complete in general, so I didn’t have to go through the hell you went through – but Harry, I feel your pain.
–Jeff