Category: Process

ANT deployment script and SFTP

My development team is moving away from developing on mapped drives/file shares to using cloud-hosted servers on Amazon Web Services (AWS). This is introducing a change to our usual workflow, as our access to the remote servers is limited to SSH and SFTP.

Although I previously used Apache Ant scripts through Eclipse to facilitate deploying application updates, the scripts were generally unpopular with the rest of the development team. (Many of them do not use Eclipse and preferred just to drop-and-drag files from their development sandboxes to the development or production servers.) Additionally, my original Ant scripts relied on the sync command to synchronize folders on the file shares.

Here is a revised Ant script that uses SCP (Secure Copy)–not SFTP but achieves the same goal–to deploy application files from a developer sandbox to the development or production server:

<project name="Deploy myapp" default="Sandbox to Dev">
  <input message="Username:" addproperty="username" />
  <input message="Password:" addproperty="passwd" />
  <property name="applicationFolder" value="myapp"/>
  <property name="site" value="osric.com"/>
  <property name="sandboxRoot" value="${basedir}"/>
  <property 
    name="development" 
    value="${username}:${passwd}@dev.osric.com:/home/web/${site}/${applicationFolder}"/>
  <property 
    name="production" 
    value="${username}:${passwd}@osric.com:/home/web/${site}/${applicationFolder}"/>
  <target name="Sandbox to Dev">
    <scp todir="${development}" trust="true">
      <fileset dir="${sandboxRoot}">
        <exclude name="**/build.xml"/>
        <exclude name="**/.*"/>
      </fileset>
    </scp>
  </target>
  <target name="Sandbox to Production">
    <scp todir="${production}" trust="true">
      <fileset dir="${sandboxRoot}">
        <exclude name="**/build.xml"/>
        <exclude name="**/.*"/>
      </fileset>
    </scp>
  </target>
</project>

There are a couple issues with this script to be aware of:

  • SCP is not included with Ant. The script produced the error “Problem: failed to create task or type scp”. I needed to:
    1. Download JSCH
    2. Place the file in Eclipse’s plugins/[ant folder]/lib folder
    3. Add the JAR file to the Ant build path (via Window–Preferences–Ant Home Entries (default)–Add External JARs…–select the jsch .jar file)
  • The password input is in plain text. Hiding password input in Ant provides a solution for Ant, but one that does not work from Eclipse. I have seen other possible solutions, so I’ll update this once I implement once and confirm that it works.

Joes Goals Runs on Cold Fusion

I use a pretty nifty little website called joe’s goals to track some of the things in my life. It ties in rather nicely with the google personal homepage, and has served me well for more than two years. It’s crashing today, and from the look of it, it runs on cold fusion. I must admit, I’m a little surprised. (Not that I have anything against cold fusion, you just don’t see it that often these days)

Code Reviews

I recently attended a code review at Wharton. We’ve put together some code reviews at my job, but I know that Wharton has been doing formal code reviews for years and I wanted to check out their process to compare notes. For this particular code review they were looking at a small ColdFusion application developed using Squidhead.
Continue reading Code Reviews

Keeping your inbox empty in gMail

So a friend of mine recently convinced me to try using gmail in a different way. His suggestion was that I try archiving all of the mail that I have that no longer requires direct attention, or has some action I must take hanging on it. I have to admit its a pretty appealing idea, since the mail will still show up in searches and whatnot. Shortly following this suggestion I got put on a very high volume mailing list that I also wanted to read many of the emails on. So I decided to give it a try.

I started by setting a group “instant unarchive” on the first two-hundred emails that I have. Then I did the select all action, and clicked the link that appears at the top of your email list when you select all to “select all 70 billion emails” and archived the bunch. I then clicked on the left the “instant unarchive” group, and moved them to my inbox. Ta-Da now I’m down to my latest 200 emails in my inbox. As some of these are still relevent & outstanding I’m going to have to archive them by hand.

The concept of having an empty inbox is pretty darned appealing to me. I’ll give you updates as I go. (oh by the way, the command key to archive an email is “e”)

Writing A Spec Way Too Late

So what was a wonderful project to start with became a nightmare, but I finally see the light at the end of the tunnel with it. Because this project started off so innocently, and became such a disaster, I feel like I should share some of the mistakes I made along the way, and some of the insights that I have gained in both 1) extracting myself from the problem and 2) not allowing myself to get sucked into issues like this again. So lets start with a little bit of history of the project and the situation and go from there.

The development company, which I will refer to as “E” in this post is one that I have worked with for a while. They are good, skillful and friendly, they also do have a history of not generating sufficient (or any) spec’s for projects before they sign deals. This is a known problem, and something that both developers and designers have been complaining about for a while.

Continue reading Writing A Spec Way Too Late

Time Tracking Software

The development team where I work is going to start tracking the time spent working on each project. Although in some ways this feels like a bureaucratic hassle, it will be useful as far as justifying our existence and provide some ammo for why we shouldn’t tackle every suggestion that crosses our desks (e.g. the “trivial tweak” that actually take 12 hours to implement).
Clock
Continue reading Time Tracking Software

The Greater-Than Gator

I lost a lot of time yesterday thanks to tracking down an alligator in an import utility. There was one itsy-bitsy little function that wreaked big havoc, thanks to a confused gator that didn’t know which way to look.
Greater Than Alligator
That’s right, there was a < when there should have been a >. (Or since it was ColdFusion code, there was a LT where there should have been a GT.)

The import utility was developed by a guy with a masters degree in computer science, so it just goes to show you that no amount of expertise can prevent such an error (or, quite possibly in this case, a typo).

It was an easy fix, of course, but fixing all the data that had been incorrectly imported in the past took a bit longer. It cost me a few hours, but that’s better than causing a credit crisis. Either way, it reinforced in my mind the idea that we could use some unit testing around here. Sure, it might take a little extra time to write the tests, but probably less than the amount of time I spent hunting down the pesky little gator and fixing the mess it left in its wake.

Building your own libraries

About 4 months ago I finally got the hang of building my own libraries, and having them consistently come out as useful bits of software that I found myself reusing over and over. More than anything else I stumbled across the secret to this simply as the result of some random flailing, but it’s a very good principle, and I would like to share it because I think that it’s easily adaptable.

The secret is: Write sample code first

This is not the same thing as writing test cases first, its about figuring out how you want to USE your lib, before you sit down and write the lib. you should write sample code that makes use of your library, and if it’s ugly, then you should re-write it. Try not to worry about how you are going to actually make the library do what it will have to at all, just think about writing code exactly the way you want to, in your dream scenario, when you apply the library. Go through it, are there Param’s that can be left out most of the time because 90% of the cases they will always be the same? (set rational defaults)

Now write some more sample code, this time come up with the most absurd, obtuse, edge case applications of your library. Things that it was just not meant to do properly. Write some sample code that is able to do them. Unlike the previous sample code, this does not need to be pretty, you just need to make sure that its possible. I strongly suggest making use of optional closures in order to keep yourself out of boxes.

Now go back over your ideal sample code, polish it even more. You have probably come up with some things that could be made simpler, easier and more obvious after writing the really evil edge cases.

And then finally, start writing the actual library. Figure out how to make it leap through whatever hoops you need to in order to make it fit the sample code that you have written. Try hard not to compromise your original sample code vision. If you must, make sure that you do it in a way that leaves the sample code still graceful. (sometimes it makes sense to change several things if you have to change just one)

I have found that this leaves me with two advantages for actually re-using my libraries later:
1) When I want to use them, they tend to work just how I want them to. This is a result of writing them to an ideal use case
2) I have two big lumps of sample code so I can remember how the hell I was supposed to use them without re-reading the whole body of code. Often I actually paste the simple sample into what I’m doing and then just transform it into what I need.

Code refactoring VS feature addition.

I’m currently working on a pretty major overhaul to a custom Content Management System that I wrote for some clients, and I have found that I am both reworking the interface, and doing code refactoring at the same time. I notice that I often end up performing these two tasks simultaneously, partially because its hard to sell clients on pure code refactoring but it does have to happen, and partly because feature creep is an ever-present evil.

The question is, would I do much better to try to wear these hats one at time? would code refactoring first, without changing any features, followed by a round of feature changes result in a more manageable process? Or am I just deluding myself, thinking that something else looks better because I’m not elbows deep in it at the moment?

Totally Hosed My Dev Environment

So I have been trying to play with the Zend Framework, which by all accounts is pretty cool, and trying to get it set up on the dev environment that I have here on my cute little macbook. Sadly, for some reason the version of os X 10.5 that ships with it doesn’t have pdo_mysql support installed (though it has pdo_mysqllite ?? I mean, come on, seriously?) So it’s off to the recompile mines. Sadly, once you recompile PHP you need to recompile Apache, which wouldn’t be a big deal at all other than my not knowing about it and it not giving me anything like a coherent error message regarding this. Fine, whatever….

Next, I discover that I can’t user the PEAR auto-installer, because it can’t find the damn modules directory, and neither can I. I have even tried making new ones in every likely place. What’s worse, I seem to have also lost traditional MySQL support as well.

I’m getting pretty annoyed with OSX here, It’s a damn nice client OS, but it’s driving me nuts on the server side. (Still better than Windows, it’s just a bit like the The Twilight Zone, things are close enough to where they should be to make you think that you know what’s going on, but when you try to actually do something….)

I guess I’ll try MAMP now. I have been meaning to look at that anyway.