The reviewers had clearly spent some time with the application, reviewing both the code and the interface. (I asked about this after the review, and one reviewer said he had spent 2-3 hours reviewing what was a relatively short and simple application. It is telling, I think, that attendance was voluntary–the reviewers in attendance were keen reviewers.) They got down to the level of detail where they brought up the date format (e.g. why did the developer use dd-mmm-yyyy instead of mm/dd/yyyy?).

The review was relatively structured while remaining casual in tone. They discussed CFCs, followed by other ColdFusion code, followed by user interface concerns and SQL. Each suggestion was backed up with a reason, e.g. avoid using evaluate() because it can lead to performance issues, or use mm/dd/yyyy because ~60% of the intended audience is most familiar with that format. In general, the topics covered fell into these categories: security, speed, standards/best practices, readability/maintainability, and user interface concerns.

It really seemed like a useful process to me. True, it added 12-18 developer hours to the project, but that is a small price to pay to avoid a possible security breach or server crash (not that any such vulnerabilities or flaws were found). The key, it seemed to me, was in the preparedness of the reviewers, who each came with comments in hand. Their preparedness, in turn, was helped by having basic guidelines as to what issues to look for.

I started by setting a group “instant unarchive” on the first two-hundred emails that I have. Then I did the select all action, and clicked the link that appears at the top of your email list when you select all to “select all 70 billion emails” and archived the bunch. I then clicked on the left the “instant unarchive” group, and moved them to my inbox. Ta-Da now I’m down to my latest 200 emails in my inbox. As some of these are still relevent & outstanding I’m going to have to archive them by hand.

The concept of having an empty inbox is pretty darned appealing to me. I’ll give you updates as I go. (oh by the way, the command key to archive an email is “e”)

The development company, which I will refer to as “E” in this post is one that I have worked with for a while. They are good, skillful and friendly, they also do have a history of not generating sufficient (or any) spec’s for projects before they sign deals. This is a known problem, and something that both developers and designers have been complaining about for a while.

This was the single largest block of mistakes, and I’m going to try to enumerate some of them.
1) Never rewrite from scratch. This was not the exception. I’m beginning to wonder if there ever actually is one
2) If the original dev team has thrown up their hands and stormed off, its probably for a serious reason, usually either management or the client. I know management here and they are kind, so I should have suspected Mr. X
3) “Provide you with a working site” is not a spec. This one is really really bad
4) “use the old site as a spec” is actually a very solid spec, but also one you don’t want to get involved with, particularly if the old site did not work.
5) Don’t let consensus view make you feel ok about your choices. Just because all the other dev’s say it needs a rewrite doesn’t mean it needs a rewrite
6) Lawyers, by and large, are very very detail oriented clients, Any client that you anticipate being detail oriented, such as Mr. X who is a lawyer, should get an *estimate* not a price, and should understand that they are paying time and materials, to go thru as many iterations as they like before launch.

I spent the next two weeks grinding out a complete rewrite of the site. It worked quite well. (in all the time I have spent reviewing it, I have only found 3 significant bugs, as well as a slew of very minor ones & generally weak security, which is pretty damn good for a project on this timeline) I used cake to write it. I fixed the page-flow and usability issues, I did triage some of the extraneous features to get it done in budget and on the new schedule. And I pulled it in at a cool 99 hours. (this was also my first cakePhp project, and I now have an overall positive reaction to cake)

So we take it to get skinned, and the designer puts some templates & CSS on the thing, and shows it to the client. She doesn’t like the design now, she wants it to look _exactly_ like the old site. This should have been a huge red-flag, and honestly was, but we didn’t react to it properly. It was noted and nothing was done about it. So the designer pulls out a bunch of HTML from the old site and slaps it into the template files, and I take several hours and reconnect everything. Now there are some bugs, because of all the retro-fitting.

Next the client wants all of the items that got triaged back. They come up one at a time, mixed in with other requests. This, though it sucks hard, is reasonable. I thought Mr. X was on a tight time-table, but apparently he isn’t and is more than happy let the project be late in order to get all his features. If he had said that in the first place I would have gone ahead and put them in originally. I grumble and do it.

Next comes a slew of 1) minor textual tweaks, 2) minor graphical tweaks, 3) Major user interface tweaks, and 3) minor functionality tweaks. I can’t really comment on the textual tweaks or the graphical ones, but the UI changes are steadily bringing the UI back to match the first, highly dysfunctional one. It turns out that this was not broken as I had originally thought, but was actually the way that the client wanted things. Well, that was a communication’s disaster. At this point we are creeping up on a month past the date I gave for completion (so much for the rush) and the code is starting to look pretty manhandled. Frustration so running very, very, high in the team. The designer gets a “last list of changes” from Mr. X, who points out that we are on the home stretch. The last list of changes goes into a spreadsheet and we spend another week cranking them out.

At this point I’m not billing anymore, I’m way over estimate, and have resigned myself to just getting things done. I’m frustrated that we don’t have anything in writing that says that when we finish this “last list of changes” we are done, and so is our designer. Sadly that is something that the client just wasn’t willing to agree to (apparently)

The last list of changes gets down to two items which need some clarification, mostly because they are UI / biz logic and they make no sense at all. The designer goes back to clarify, and Mr. X keeps repeating at him “I need everything just like the old site”, as well as generating more and more additional requests. At this point, the designer breaks (on the inside). He get’s himself out of the phone conversation with Mr. X politely, and calls us back to say that he can no longer work with Mr. X and that if he ever has to talk to him again he’s going to flip out and do something violent. To say our designer was agitated is something of an understatement. I have also been working with this designer for over a year, and he has consistently been both levelheaded and laid back in every situation that I have been with him in.

This is the best thing that could have happened. Until this point no one was willing to play hardball, seeing our designer so angry drove home just how out of control this project had become. Like a slowly boiled frog, we were perishing but hadn’t noticed.

So I call the owner, we talk about what’s going on, he understands the situation, and then I call the client. Mr. X starts by apologizing for upsetting our designer, and reiterates that we are in the home stretch. I explain that there was some sort of communication failure with the designer, and ask him if he can walk me through what he talked about. On every page that we touched on Mr. X discovered a few changes that he felt needed to be made. It became readily apparent that he was discovering this as it went. This is the part where I’m proud of myself.

I stopped the conversation and explained that we needed a definitive list from him of what had to be changed in order for the project to be complete. He said, “no, I can’t do that, there are parts that I haven’t seen yet, and things I need you to change dramatically, I need to see those big changes after they are done to make sure that they match up with expectations”. This is where I finally stood up for the team and said “No, we can’t do any more development until we have an exact point by point list of what needs to be done for this project to be finished”. Mr. X expressed his dismay at being asked for something written in blood, and pointed out that he might find small formatting / textual changes after the fact, and that those would need to be addressed. I responded that we did need something “written in blood” and that the problem was that those small formating changes, while individually not a problem, in aggregate could and had been devastating. After some time discussing how the project got out of control, and what development typically looks like, I finally got Mr. X to agree. He is now working on a spec, written in blood, and though I anticipate it being long and hard, it does mean that the light at the end of the tunnel will be in sight.

The deeper moral of the story is: never work on anything until you have defined what “done” looks like. If you don’t know what success is, you can’t know when you will get there.

A corollary to this is: Every major player in a project needs to agree about what “done” looks like

and a second corollary is: It is never to late in a project to stop, and define what “done” will look like.

I hope you all learn from my cautionary tale.

• TimeGadget Project Tracker
• Activity Tracker

The former offers an Excel report, the latter a graph.

Since a number of developers will be using it, it would be nice if we had something we could all use and then run reports from one place. Any ideas?

The import utility was developed by a guy with a masters degree in computer science, so it just goes to show you that no amount of expertise can prevent such an error (or, quite possibly in this case, a typo).

It was an easy fix, of course, but fixing all the data that had been incorrectly imported in the past took a bit longer. It cost me a few hours, but that’s better than causing a credit crisis. Either way, it reinforced in my mind the idea that we could use some unit testing around here. Sure, it might take a little extra time to write the tests, but probably less than the amount of time I spent hunting down the pesky little gator and fixing the mess it left in its wake.

The secret is: Write sample code first

This is not the same thing as writing test cases first, its about figuring out how you want to USE your lib, before you sit down and write the lib. you should write sample code that makes use of your library, and if it’s ugly, then you should re-write it. Try not to worry about how you are going to actually make the library do what it will have to at all, just think about writing code exactly the way you want to, in your dream scenario, when you apply the library. Go through it, are there Param’s that can be left out most of the time because 90% of the cases they will always be the same? (set rational defaults)

Now write some more sample code, this time come up with the most absurd, obtuse, edge case applications of your library. Things that it was just not meant to do properly. Write some sample code that is able to do them. Unlike the previous sample code, this does not need to be pretty, you just need to make sure that its possible. I strongly suggest making use of optional closures in order to keep yourself out of boxes.

Now go back over your ideal sample code, polish it even more. You have probably come up with some things that could be made simpler, easier and more obvious after writing the really evil edge cases.

And then finally, start writing the actual library. Figure out how to make it leap through whatever hoops you need to in order to make it fit the sample code that you have written. Try hard not to compromise your original sample code vision. If you must, make sure that you do it in a way that leaves the sample code still graceful. (sometimes it makes sense to change several things if you have to change just one)

I have found that this leaves me with two advantages for actually re-using my libraries later:
1) When I want to use them, they tend to work just how I want them to. This is a result of writing them to an ideal use case
2) I have two big lumps of sample code so I can remember how the hell I was supposed to use them without re-reading the whole body of code. Often I actually paste the simple sample into what I’m doing and then just transform it into what I need.

The question is, would I do much better to try to wear these hats one at time? would code refactoring first, without changing any features, followed by a round of feature changes result in a more manageable process? Or am I just deluding myself, thinking that something else looks better because I’m not elbows deep in it at the moment?

Next, I discover that I can’t user the PEAR auto-installer, because it can’t find the damn modules directory, and neither can I. I have even tried making new ones in every likely place. What’s worse, I seem to have also lost traditional MySQL support as well.

I’m getting pretty annoyed with OSX here, It’s a damn nice client OS, but it’s driving me nuts on the server side. (Still better than Windows, it’s just a bit like the The Twilight Zone, things are close enough to where they should be to make you think that you know what’s going on, but when you try to actually do something….)

I guess I’ll try MAMP now. I have been meaning to look at that anyway.

Although we have had a CVS server set up since at least 2005 (it was there when I started), it is used inconsistently at best.And using a version control system inconsistently is worse, in some ways, than no version control system at all: you can’t rely on the repository to have the most recent information, so you have to compare the repository to other likely sources of updated information (the development and production environments).

What are the barriers to adoption for version control? In part, it is unfamiliarity. In part, it is just one more step between the developer and a quick, harmless edit to a file. I’d group both of those as ease-of-use issues. The other barrier is a lack of understanding about the benefits version control provides.

The ease-of-use issue is exacerbated by a lack of good interface tools. We are using TortoiseCVS, and another developer and I are using the CVS features built in to Eclipse. I think that using Eclipse works well, and I’ve even created a couple simple ANT scripts to roll files out to the development environment once I’ve tested them in my sandbox.

The majority of other developers are using Dreamweaver, and tend to work directly in the development environment (rather than a sandbox). It may not seem like a big deal to switch applications for the benefits of CVS, but numerous times a day it could get tedious:

1. Checkout module to sandbox in Windows Explorer using TortoiseCVS.
2. Switch to Dreamweaver.
3. Switch to Windows Explorer to Update/Commit.
4. Copy files from sandbox to development environment.

I have been evaluating CVS for Dreamweaver. Its UI could stand a bit of spit-and-polish, but I think it might work for us. (I am open to suggestions.)

How do I convince the other developers of the benefits of using version control? That might be even trickier. The other developers seem not to experience my frustration because the files they are working with on dev are usually the most current versions. And although the low-tech versioning schema of storing dozens of backup files is messy and risky, it hasn’t actually presented any problems. A single developer tends to work on a single application, so no one, to my knowledge, has ever overwritten someone else’s updates. In this environment, do we really need CVS?

I think that we do. Although we haven’t had any problems with overwriting another developer’s code, or found out-dated pages with security issues floating around on the production server, why take those risks? Version control can solve those problems for us, as well as providing other benefits (such as accessing any previous version of the code).

I know developers who use version control at home for their personal ego web sites. Overzealous? Not really. I think any shop worth its salt uses a formal version control system.