Thinking, Fast and Slow

For example, programmers have been arguing since the 1970s about the placement of curly braces. Some people prefer:

if ( true ) {
doSomething();
}

Others prefer:

if ( true )
{
doSomething();
}

Crockford says that if the compiler treats these 2 forms as equivalent, then there is really no difference (so long as you are consistent). These are Gut decisions. However, people will use their Head to try to rationalize their Gut decisions and come up with some ridiculous rationalizations.

OK, fine. What does that mean in practical terms, i.e. writing code?

Crockford's advice is:

Prefer forms that are error-resistant.

This is aligned closely with my own advice:

Prefer explicit forms to implicit forms.

One of Crockford's illustrations (which I have not verified but sounds plausible enough) is that the space character was introduced into Latin writing after the prior practice of running all the words together led to ambiguity and transcription errors. The space characters made the text more explicit, and therefore less prone to transcription error.

Here are 3 of the Javascript examples Crockford provided during his talk:

Curly Braces & Conditional Statements and Loops
In Javascript, a conditional statement or a loop containing only one statement can be written without curly braces:
if ( condition ) doSomething();

This form is error-prone. Include the curly braces:
if ( condition ) {
doSomething();
}

Some people might complain and say that this is less concise and requires extra typing. Crockford's response was: less concise is better, and the majority of the time it takes to write a program is in thinking, not typing. If adding a few characters is slowing you down, you either need to take a typing class or spend more time thinking!

Double-Equals versus Triple-Equals
Crockford said that using double-equals (==) to compare objects is always a bad idea because it does implicit type conversion. For example:
var n = 5;
n == 5; //true
n == "5"; // true
n === "5"; // false
n === 5; // true

The triple-equals also helps developers know that a number is expected, not a character or other data type.

As Crockford emphasized:

If a feature is sometimes problematic, and a more reliable feature exists, use it!

Some people might try to justify the use of implicit type conversion: "But I'm doing it intentionally!" Crockford said that relying on implicit type conversion means you don't know what you're doing.

Incrementing Values
Crockford said he never uses the prefix or postfix ++ operator, because many people don't understand the difference and it can lead to confusion. For example:
var x = 5;
var y = x++; // y is now 5 and x is 6
var z = ++x; // z is now 7 and x is 7
x += 1; // x is now 8. This is Crockford's preferred method to increment a variable

Crockford provides several resources to help people write better, more error-resistant Javascript:

• Code Conventions for the JavaScript programming language
• JSLint, which alerts you to errors in your code or error-prone code ("JSLint will hurt your feelings")
• JSMin, which "minifies" (removes unnecessary characters) from Javascript files. This should be largely unnecessary if your web server uses Gzip compression.

I’ve been interested in unit testing and test-driven development for several years now, but have never put it into practice. It seemed like a good idea in theory, but it required picking a testing framework, installing it, configuring it, and, of course, writing good tests. Without any hands-on experience, it’s a difficult practice to adopt. Getting introduced to unit tests this way lifts nearly all those burdens. On top of that, the advantages are clear: the tests will tell you when you have it right.

I was a little surprised to run into unit testing via coursework, because I’d been under the impression that computer science education focuses a lot on theory, and skips over a lot of the practice. I’ve met a lot of people coming out of computer science programs who, while probably excellent theorists, don’t know heads from tails when you sit them at a terminal. I was happy to see that my coursework included practical knowledge as well.

All the unit tests are provided by the professor. I assume we may write some tests of our own later on–and certainly nothing is stopping us from writing our own tests now–but I do wonder how many students will be prepared to take that next step. From what I understand, writing good tests is the better part of successful unit testing.

Some things I’ve noticed about the provided unit tests:

• There are a lot of them. The unit tests are half as long the code that passes them.
• There are many sample values to test the same function, mostly to test specific edge cases. Failure at a specific edge case helps to identify where the code went wrong.
• In some cases, the tests are within a loop that generates random test data. Although there’s nothing quite like real human input to break your programs, 1000 random tests might help.

In practice, do programmers tend to write their own unit tests? I imagine it would be ideal if you partnered with someone, and you wrote their unit tests, and they wrote yours. It might be easy to overlook an edge case or dismiss something as impossible if you are too involved with the specifications. At the same time, it is probably difficult to write a unit test without spending some time reading the specifications and understanding exactly what it is supposed to do.

I had always heard that, unless the image conveys important information (e.g. a graphic of text used as navigation, or a chart or graph) that the alt text should be left blank:
<img src="myimage.png" alt="" />

A screen-reader passes such an image over without saying anything. This makes sense to me. When I’ve closed my eyes and tried navigating the web using a screen-reader like JAWS, anything non-essential was a distraction and just got in my way. Knowing that a page contained an image of, say, a corporate headquarters in no way helped me understand the page content.

However, a colleague of mine suggested that the alt text should describe, briefly, the image. He offered a compelling use case: you are browsing on your mobile device with images disabled, due to bandwidth. How would you know if there was an image that you did want to view if no description was provided? This is a case where the alt text can provide a better user experience for users without vision impairments. But does it make the experience worse for users with vision impairments using assistive technology?

(A case could be made that vision-impaired users should also know there is an image on the page for orientation purposes: “The link to the annual report is underneath the photo of the corporate headquarters.” However, what may appear above or below on screen may not make sense when page content is read linearly.)

I looked to the WCAG 2.0 section on text alternatives, which states that images used for decoration or formatting should be implemented in such a way that they “can be ignored by assistive technology.” That’s a good case for using an empty alt attribute. If the image is sensory (WCAG 2.0 has been criticized for being vague–obviously anything visual is sensory), then the item should “at least provide descriptive identification of the non-text content.”

What about that photo of the corporate headquarters then? It’s decorative, but not in the same way as a fleuron or a border. It may not be an inspiring image, but maybe it should have associated alt text.

I decided to check 4 sites that I thought might demonstrate best-practice, but found little consistency across these examples:

• The National Federation for the Blind – they use rather extensive alt text for the main image on their homepage: “Graphic consisting of two photos. On left is a group of children with white canes on a hayride. Right is a close-up of a finger reading Braille.” However, they fail to use the alt attribute for their menu divider graphic, which is clearly a decorative element.
• Freedom Scientific’s JAWS Screen Reading Software – the alt text “A student uses JAWS to do work on a desktop computer” accompanies a photo of a man at a computer. Decorative images (menu dividers, stars) use an empty string for the alt text.
• WAIM – Web Accessibility in Mind – they avoid the issue on their services page by inserting the pictures as CSS background images. These would not appear at all to a screen reader, to a text-based browser, or to a user agent with images disabled. This would be functionally equivalent to using empty alt text.
• The Social Security Administration’s Disabilities Benefits – this page gets it completely wrong, including alt text for images that do not even appear visible to users with normal vision (e.g. a tracking image with the alt text “DCSIMG” and a spacer image with the alt text “blank space”).

MIT’s general web-accessibility guidelines offer some additional guidance:

ALT tags are often misused, mostly people overuse them. It’s better to leave the ALT tag blank (ALT=”") then to enter a text description that’s not useful or is redundant. For example an image with a caption below it does not need alt text that matches the caption, leave the alt text blank to avoid redundancy.

The University of Michigan’s Accessibility Quick Guide suggests using empty alt text for non-informative images.

Unfortunately, we’re still left with a rather vague recommendation: use a description when useful or informative. How do we decide when a description is useful or informative? My gut feeling is to agree with MIT: it should be left blank in most cases (such as with the hypothetical photograph of a corporate headquarters), but I think no great harm is done if a brief description is included.

The best and easiest way I’ve found to do this is via Cygwin:
touch -t 201109201145 *.pdf

When I searched for how to accomplish this, I found all sorts of sites advising me to download a $20 shareware application, or write a C# application to do it. Fortunately, in a *nix type environment, it’s a one-line command.

There are other ports of Unix/Linux/*nix commands for Windows, some of which may be more lightweight than Cygwin. (I rely on Cygwin quite a bit, so I have it installed on all my Windows machines.) I have not used any of the following, but here are links to some alternatives, in case you are interested:

• Core Utils
• GNU Win32 Core Utils
• Unx Utils
• Touch for Windows

Sys.WebForms.PageRequestManagerServerException: An unexpected error has occurred

I found a couple blog posts about this suggesting a permissions issue (which made sense, as users in other groups could successfully edit pages):

• SP2010/Wiki Pages: “Sys.WebForms.PageRequestManagerServerErrorException: An unexpected error occured.” when editing a page
• Sys.WebForms.PageRequestManagerServerException error when editing pages

Both sites suggested that an Access Denied message should appear in the ULS logs. However, no such message appeared in relation to my errors.

ULS entry:
System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.SharePoint.Publishing.WebControls.MediaWebPart.get_WebPartAdderId()
at Microsoft.SharePoint.Publishing.WebControls.InsertMediaRibbonButton.RegisterRequiredScripts()
at Microsoft.SharePoint.Publishing.WebControls.InsertMediaRibbonButton.OnPreRender(EventArgs e)
at System.Web.UI.Control.PreRenderRecursiveInternal()
at System.Web.UI.Control.PreRenderRecursiveInternal()
at System.Web.UI.Control.PreRenderRecursiveInternal()
at System.Web.UI.Control.PreRenderRecursiveInternal()
at System.Web.UI.Control.PreRenderRecursiveInternal()
at System.Web.UI.Control.PreRenderRecursiveInternal()
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint... 1317dffd-b607-4f1c-9145-b4adc1d364ba

The MediaWebPart. SharePoint 2010 added a video/audio insertion feature on the ribbon, and the error occurred when trying to add the button.

It turns out, the permissions to the Web Part Gallery had been customized (i.e. broken inheritance from the parent site collection), and some groups who had permission to edit wiki pages did not have read permission for the Web Part Gallery (and, consequently, the MediaWebPart). Adding Restricted Read access for the group in question (to either the MediaWebPart or to the Web Part Gallery) solved this problem.

I don’t imagine this will be a common problem for other people, but I thought it worth mentioning.

• 2,001
• 1,998
• 2,011

This is confusing to the viewer, as values no longer look like years.

The most frequent suggestion is to create an additional field that calculates a value based on the Number field and ignores all non-numeric parts. This is relatively straightforward, and can be accomplished entirely within the browser:

=TEXT([MyNumericData], "0")

Or, in a case where you’d like to retain two digits after the decimal:
=TEXT([MyNumericData], "0.00")

SharePoint appears to store the data in its numeric format in a another column, hidden to the browser but visible in SharePoint Designer. It’s [column name] followed by a period. For example, I created a field called MyNumericData, so the hidden field is MyNumericData. Again:

• MyNumericData
• MyNumericData.

Adding this to a view via SharePoint Designer is non-obvious. Here are the steps I used:

1. Click on a data item in Design mode until you see a tab labeled xsl:value-of:
   
   

   SharePoint Designer 2010 Design Mode

2. Right-click the item and select Edit Formula
3. Remove the current XPath expression
4. Add the MyNumericData. row from the fields pane
5. Add the format-number function from the Math / Number functions
6. Add the appropriate format pattern (e.g. ‘#’)

The appropriate format pattern depends on how you wish to display the data:

• Display integer value:
  format-number($thisNode/@MyNumericData.,'#')
• Always display two digits after the decimal:
  format-number($thisNode/@MyNumericData.,'0.00')
• Display up to 2 digits after the decimal:
  format-number($thisNode/@MyNumericData.,'#.##')

If you look at the code view after making these changes, you will see that SharePoint Designer added a hefty chunk of XSL to the XsltListViewWebPart.

Is there any advantage to this method over adding a calculated field? Not as far as I can tell–but it’s interesting to note that there is more than one overly-complicated solution to what seems like a trivial problem.

I couldn’t change the page layout via the browser. I couldn’t change the page layout via SharePoint Designer.

Fortunately, I found Patrick Lamber’s helpful post, How to programmatically change the page template in Moss 2007, which basically described the same error and described applying a new page layout via the object model.

For whatever reason, I thought I could do it just as easily via PowerShell. I started following Jason Grimme’s post, Update SharePoint List Item(s) with Powershell, but I was unable to check-out/check-in the page or access the PublishingPageLayout property.

Liam Cleary’s reply on a TechNet post (Using PowerShell to Checkout and Checkin a file) tipped me off that I needed to use SPFile (rather than SPListItem) in order to perform the necessary operations. Here are the commands I ended up using:

$spWeb = Get-SPWeb("http://currentsitename/webname")
$spFile = $spWeb.GetFile("http://currentsitename/webname/Pages/Test-Page.aspx")
$spFile.CheckOut("Online",$null)
$spFile.Properties["PublishingPagelayout"] = "/_catalogs/masterpage/Block.aspx, Block"
$spFile.Update()
$spFile.CheckIn("Update page layout via PowerShell",[Microsoft.SharePoint.SPCheckinType]::MajorCheckIn)
$spWeb.Dispose()

I visited the page and…another error! This one telling me there wasn’t a valid page layout. Funny, it worked in development.

It turns out, when I compared it to the PublishingPageLayout property of a working page, I had missed the space character after the comma.

Doesn’t work:
"/_catalogs/masterpage/Block.aspx,Block"

Works:
"/_catalogs/masterpage/Block.aspx, Block"

Once I applied the proper path, I was also able to change the page layout via the browser again.

This is bad. I’m not always visiting my own web site, so it could be down for days without me knowing. Although I really need to identify what is using all the RAM, in the meantime I’ll settle for a monitoring system that will notify me when the server is down.

#!/bin/bash
if curl -s --head http://osric.com/ | grep "200 OK" > /dev/null
  then
    echo "The HTTP server on osric.com is up!" > /dev/null
  else
    echo "The HTTP server on osric.com is down!"
fi

cURL will let you retrieve a URL via the command line, and provides more options than Wget for a single URL. In this case, I used the silent switch to eliminate the status/progress output, and the head switch to retrieve only the document headers. The document header is then piped to Grep, which searches for the string “200 OK” (the HTTP status message for a successful request).

I send the result of that to /dev/null so that the output doesn’t appear on the screen.

If grep does find 200 OK, then I send a success message to /dev/null. This is largely unnecessary, but it is nice to leave in to test the script in a successful case–just remove the > /dev/null. If it doesn’t find 200 OK, then there is a problem. It might not mean, necessarily, that the web server is down, but it definitely indicates there is a problem that needs to be identified.

I added a call to this script to a crontab to run every 5 minutes. If there is no output, nothing happens. If there is output, the output is sent to me via e-mail, which, assuming I am checking my e-mail religiously, should reduce server downtime.

Here is my sample code:

<cfftp action="open"
    connection="test"
    server="ftp.osric.com"
    username="chris"
    password="********************"
    timeout="60"
    stoponerror="yes">
<cfftp
    connection = "test"
    action = "getFile"
    name = "downloadFile"
    transferMode = "binary"
    localFile = "S:\chris\handlebar-moustache.jpg"
    remoteFile = "handlebar-moustache.jpg"
    timeout="60">

Here’s the error message it produced:
An error occurred during the FTP getFile operation.
Error: getFile operation exceeded timeout.

However, the local file was still created (as a zero-byte file).

The solution, in my case, was to turn on passive mode (add attribute passive="yes" to the cfftp tag).

Active FTP vs. Passive FTP, a Definitive Explanation has a brief explanation of the differences between active and passive FTP.

It also gave me a good opportunity to put fakenamegenerator.com to the test. The site quickly provided me with a list of 6000 random names.

Now, how to get them into a SharePoint list?

First, I tried creating an empty SharePoint project in Visual Studio, and adding a feature receiver to insert the items. The only issue is that I wasn’t sure how to parse the file from within the feature receiver, so I just created an array with 6000 members by copying-and-pasting a slightly modified CSV file into the constructor. But when I tried deploying it, I got this error:

Error:
Error occurred in deployment step 'Activate Features': Unhandled exception was thrown by the sandboxed code wrapper's Execute method in the partial trust app domain: An unexpected error has occurred.

I checked the list, and 1368 of 6000 list items had been added. I tried deploying it a couple more times, and it added 1682 items and 1629 items. What was causing the error?

I took a look at the associated ULS log entries:

• Resource "IdlePercentProcessorTime" for monitored process "ipc://5a551cc1-06e5-4241-8c77-55a3a93f82db:7000 (PID=2764" has value: 71.21212 which is above absolute resource limit: 10.
• The worker process "ipc://5a551cc1-06e5-4241-8c77-55a3a93f82db:7000", SHIM/PROXY PID 2764/1100 has been terminated because resource "IdlePercentProcessorTime" is above hard threshold value
• Stopping shim process. Shim process name: "SPUCWorkerProcess" Shim PID: "0x0ACC" Shim service url: "ipc://5a551cc1-06e5-4241-8c77-55a3a93f82db:7000"
• Stopping proxy process. Proxy process name: "SPUCWorkerProcessProxy" Proxy PID: "0x044C" Proxy service url: "ipc://00657311-1135-413b-bda0-839e7794fae6:7000"

I saw the IdlePercentProcessorTime and focused on ProcessorTime–I thought the server was working too hard. I tried putting the thread to sleep for 100 milliseconds between items, but that time it only added 219 items!

Right: IdlePercentProcessorTime. The process had even more idle time, because it was sitting around with a sleeping thread. Apparently, sandboxed solutions can’t just tie up processor threads. (Check out Quotas and Locks for Sandboxed Solutions for more info.)

I could deploy the data to the list in 6 batches of 1000, but that’s an inelegant solution. Besides, what if I wanted to insert 100,000 records instead?

Rather than mess with any of that, why not just bypass the sandbox restrictions and make it a farm solution? This time I got a different error:

Error occurred in deployment step 'Add Solution': failed to load receiver assembly "AddLotsOfNames...."

There were zero items in the feature, so there was no assembly. I addressed this by adding an empty module. The 6000 names were added to the list.

Using a feature receiver to add all the names is less than ideal, though. A console application would be better. I built a console app that would take a string as a command line argument and use that as the filename for the input data. (If I wanted to re-use the file, I could take other command line arguments, such as the SharePoint site and list address into which the data should be inserted.)

Visual Studio notified me of errors right away. My code from the feature receiver relied on a reference to Microsoft.SharePoint. I tried to add the reference to my console app, but the reference wasn’t listed. Why not? The console app was set to target .NET Framework 4, but the Microsoft.SharePoint component is in .NET Framework 3.5. Once I targeted the right framework, I was able to add the reference.

The console application compiled, but produced a run-time error:
An unhandled exception ('System.IO.FileNotFoundException') occurred in AddPeopleConsole.exe [5184]

This was a little confusing, because my first thought was to check the name and path of the data file, which was correct. It turns out that the console app is set, by default, to compile for an x86 (or 32-bit) platform. My development environment has a 64-bit processor, so I changed the platform target to Any CPU.

Here’s the working code:

namespace AddPeopleConsole
{
  class Program
  {
    static void Main(string[] args)
    {
      string filename = args[0];
      var reader = new StreamReader(File.OpenRead(@filename));

      using (SPSite siteCollection = new SPSite("http://sp2010dev/"))
      {
        using (SPWeb web = siteCollection.RootWeb)
        {
          SPList list = web.GetList("/Lists/Contacts/");
          while (!reader.EndOfStream)
          {
            var person = reader.ReadLine();
            // Separate the person data into discrete values
            string[] values = person.Split(',');
            // Add an empty item to the list
            SPItem item = list.AddItem();
            // Assign values to the item
            item["Title"] = values[0];
            item["FirstName"] = values[1];
            item["WorkPhone"] = values[2];
            item["Email"] = values[3];
            // Save the item, i.e. put the person in the list
            item.Update();
          }
        }
      }
    }
  }
}

Testing in my development environment isn’t very useful, in this case. The 5000 item limit on a list view is to protect the system from performance degradation, as a query of over 5000 items will cause the entire table to lock (for the duration of the query). On a single-user dev environment (or even a multi-user dev environment), these table locks would probably go unnoticed.

If I ever need to insert a huge number of records from a text file, though, I’m prepared!