Category: Uncategorized

Applying per directory X-Frame-Options headers in Apache

To help prevent against click-jacking, I had applied the following to my Apache 2.2 configuration based on the suggestions described in OWASP’s Clickjacking Defense Cheat Sheet and Mozilla Developer Network’s The X-Frame-Options response header:

Header always append X-Frame-Options SAMEORIGIN

However, my site has certain pages that are included in an iframe on another site, for the purpose of displaying content on digital signage devices. After I added that header, those pages would no longer load in an iframe on the digital signage devices’ browsers.

I thought I might be able to change SAMEORIGIN to ALLOW-FROM and list both the URI of my site and the URI of the digital signage page. However, the HTTP Header Field X-Frame-Options RFC indicates:

Wildcards or lists to declare multiple domains in one ALLOW-FROM statement are not permitted

The pages I wanted to exempt from the X-Frame-Options restriction exist in their own directory, /digitalsignage, so I tried to override the X-Frame-Options header in a .htaccess file:

Header always append X-Frame-Options ALLOW-ACCESS http://example.com

That caused a 500 Server Error. This message appeared in the error logs:

.htaccess: error: envclause should be in the form env=envar

The Header directive must be malformed, but I’m am not sure how. I did not determine how to properly format the statement so as not to produce that error, although several sites have pointed out that some browsers (Chrome, Safari) do not support ALLOW-ACCESS.

I changed the .htaccess file back to SAMEORIGIN, to match what was in the main site configuration:

Header always append X-Frame-Options SAMEORIGIN

I then noted that the response header sent by the server included SAMEORIGIN twice:

Header: SAMEORIGIN, SAMEORIGIN

That’s the expected behavior when using append. It appeared only once after I changed append to set:
Header always set X-Frame-Options SAMEORIGIN

I tried using set instead of append with ALLOW-ACCESS:

Header always set X-Frame-Options ALLOW-ACCESS http://example.com

But it still produced the same 500 Server Error.

After reading the documentation for Apache’s mod_headers, I realized that unset would allow me to remove the X-Frame-Options header from the /digitalsignage directory:
Header always unset X-Frame-Options

That worked, and the pages were successfully included as iframes in a page on the digital signage company’s site.

Canvas enrollments.csv and add_sis_stickiness

I have an enrollments.csv file for Instructure’s Canvas LMS, and I want all of the enrollments in it to “stick”–that is, to survive a batch mode SIS import. These are primarily course designers, and so they have no official standing in the class–and therefore are not in our database, and therefore are not included with regular updates to enrollments.

According to the Canvas documentation for SIS imports:

add_sis_stickiness – Boolean

This option, if present, will process all changes as if they were UI changes. This means that “stickiness” will be added to changed fields. This option is only processed if ‘override_sis_stickiness’ is also provided.

Source: https://canvas.instructure.com/doc/api/sis_imports.html#method.sis_imports_api.create

However, experience tells me otherwise. An inquiry to Instructure’s support confirms that add_sis_stickiness does not apply to enrollments. Enrollments added this way will be deleted following the next enrollments batch import.

The choices to preserve these course designer enrollments are basically to add each one manually using the web UI, or add them via the API. Either option will make the enrollments “stick.”

I opted to use the API. Since I already had a formatted input file, I wrote a short BASH script (with the help of several man pages and a couple StackOverflow pages) that reads the CSV and processes each row, adding the enrollment via the API:

headerrow=1
while read row; do
    if [ $headerrow -eq 0 ]
    then
        # get the SIS course ID
        cid="$(echo $row | cut -d',' -f1)"
        # get the SIS user ID
        uid="$(echo $row | cut -d',' -f2)"
        # get the role / enrollment type
        type="$(echo $row | cut -d',' -f3)"
        # reformat the enrollment type
        tid="$(echo $type | cut -c1 | tr [[:lower:]] [[:upper:]])""$(echo $type | cut -c2-)"Enrollment
        echo course is $cid
        echo user is $uid
        echo type is $tid
        result="$(curl https://[yourcanvassite].instructure.com/api/v1/courses/sis_course_id:$cid/enrollments -H 'Authorization: Bearer [REDACTED]' -X POST -F enrollment[type]=$tid -F enrollment[user_id]=sis_user_id:$uid -F 'enrollment[enrollment_state]=active' -F 'enrollment[notify]=false')"
        echo $result
    fi
    headerrow=0
done <enrollments.csv

Yammer Enterprise Block Users appears to be limited to 70 users per submission

This is just based on trial-and-error testing, but when I submit over 70 e-mail addresses at:

https://www.yammer.com/[my domain]/admin/blocked_email_addresses

I get one of the following errors:

  • Nginx 502 Bad Gateway
  • We’re sorry, but something went wrong.
    We’ve been notified about this issue and we’ll take a look at it shortly.

When I submit 70 addresses or fewer, it works as expected.

I have not yet discovered a way to block addresses via PowerShell.

Error retrieving Glacier vault inventory via AWS CLI: Unknown options: inventory-retrieval}’

Amazon’s Glacier service is great, but low-cost storage has other costs. For example, you might have a vault but not know what archives it contains. You can retrieve the vault inventory–essentially listing the contents of a directory–but like any Glacier retrieval, it may take several hours.

Using Downloading Vault Inventory using the REST API, I put this together:
C:\>aws glacier initiate-job --account-id - --vault-name my_poorly_named_vault --job-parameters '{"Type": "inventory-retrieval"}'

Which returned:

Unknown options: inventory-retrieval}'

I had taken the command directly from AWS’s example:
http://docs.aws.amazon.com/cli/latest/reference/glacier/initiate-job.html.

(Keep in mind that I had already followed the steps at Installing the Amazon Web Services Command Line Interface and Configuring the Amazon Web Services Command Line Interface.)

According to the documentation for the job-parameters option (http://docs.aws.amazon.com/sdkforruby/api/Aws/Glacier/Types/JobParameters.html#type-instance_method),
valid values are “archive-retrieval” and “inventory-retrieval”.

But the error message says inventory-retrieval}’. Why is it picking up the trailing curly brace and the apostrophe?

I formatted the job-parameters JSON in a file named aws-json.txt, with the curly braces on separate lines:

{
    "Type": "inventory-retrieval"
}

I tried this variation on the initiate-job command:
C:\>aws glacier initiate-job --account-id - --vault-name my_poorly_named_vault --job-parameters file://aws-json.txt

That worked!

The results returned:

{
    "jobId": "y8ugyoNzzusaf6Lv72G3hsjAA6O7nw5bJQ2u6J9TDnJ82_qx-lxnqrhSxIcGvOU1iiXoUhZboiojxsDu8gLQOfiJ7hR2",
    "location": "/123456789011/vaults/my_poorly_named_vault/jobs/y8ugyoNzzusaf6Lv72G3hsjAA6O7nw5bJQ2u6J9TDnJ82_qx-lxnqrhSxIcGvOU1iiXoUhZboiojxsDu8gLQOfiJ7hR2"
}

Online Advertising Click-Thru Rates, Revisited

A couple years ago, I wrote Online Advertisements and Statistical Analysis, in which I did my best to show that a past study of online advertising click-thru rates (CTRs) wasn’t worth the pixels it was printed on.

About a week ago, my wife and I were visiting friends, and I found myself in a room with 3 neuroscientists. The topic of statistics came up, and I managed to insert into conversation my small triumph in analyzing the click-thru study and determining both a confidence interval and the number of tests that would need to be run in order to have a meaningful confidence interval. “Sure,” one of the scientists says, “but what you should really do instead is a chi-square test for goodness-of-fit.”
Continue reading Online Advertising Click-Thru Rates, Revisited

Online Advertisements and Statistical Analysis

Quite a few years ago, I was in the online advertising business. My team and I created banner ads to run alongside web site content, to entice viewers to click on ads and find out more about advertiser offers. We scheduled ads to run alongside specific content. We targeted ads towards users in specific geographic regions, thanks to a browser cookie that told us their ZIP code. And we constantly managed inventory.

Although we made animated ads, we avoided anything that blinked. There were no monkeys to punch.

Click-Through Rate (or Click-Thru Rate or CTR) was a key measurement of an ad’s success. Although at first we would sometimes see click CTRs between 1-2% (meaning that an ad was clicked between 10 and 20 out of every 1000 views, or impressions), as online advertising proliferated, and as our systems got better at filtering out false impressions and clicks from various robots, crawlers, and spiders, CTRs trended much lower: 0.25% suddenly looked good, and 0.10% was not uncommon in some cases. That’s 1 click for every 1000 impressions.

That’s why we were insanely interested in a blog post we found, now presumably lost to the ages, that ran a set of 6 banner ads, which varied only slightly, and analyzed the results to determine what aspects of the ads could improve CTRs. Did including the phrase “click here” really help? If the words “click here” were in blue and underlined, like a typical web link, would that improve the CTR?
Continue reading Online Advertisements and Statistical Analysis

Apache Install and Ambiguous Errors

I installed Apache 2.2.11 on the Windows XP portion of my desktop workstation for development purposes, but I got a lot of ambiguous errors when starting from the Apache Service Monitor or the Windows start menu.

Finally, when I started Apache from the command line I got a more informative error:
(OS 10048) Only one usage of each socket address (protocal/network address/port) is normally permitted. : make_sock: could not bind to address 127.0.0.1:80 no listening sockets available, shutting down

It turns out, I had Skype running, which by default binds to ports 5520, 80, and 443. There are several solutions:
Continue reading Apache Install and Ambiguous Errors

Installing Adobe AIR and Tweetdeck on an Asus eee 701

Tweetdeck is an Adobe AIR application that is a twitter client, and recently also a Facebook client.

My attempts to install Adobe AIR on the Asus eee 701 (running the default Xandros distro) were foiled several times in spite of following the instructions:

  1. Download Adobe AIR
  2. Make the AdobeAIRInstaller.bin file executable
  3. Run the .bin file as a superuser

I got a nice friendly fail message from the Adobe AIR installer every time.

I found a few relevant forum posts, e.g. Adobe Air Linux won’t install on Eee PC, that suggested memory was an issue. Sure enough, running in Full Desktop Mode with 1440×900 screen resolution (on an external display), I only had about 90MB of 500MB free.

I restarted the eee in Easy Mode and then immediately ran AdobeAirInstaller.bin. Success! (I later found these same instructions on the eee user forums.)

Installing Tweetdeck was trivial at that point: download the .air file, find it in the File Manager, and double-click it. However, when I ran it, it didn’t do anything. At one point I got a message that I was running an unknown desktop, and that Tweetdeck required Gnome or KDE.

I restarted in Full Desktop Mode, and was surprised to find a Tweetdeck icon already on the desktop. I ran it and was prompted to use KWallet, a KDE password manager. I canceled out of that, and found that Tweetdeck opened, but still didn’t do anything.

I tried again, activated the KWallet password manager, and then it worked! Tweetdeck prompted me for my twitter login, I additionally logged in to Facebook, and now I have a mean, lean, social networking machine.

One of my projects is about to be licenced

A web development company that I have worked with before has decided that they are interested in licensing a web-framework that I have developed. They are probably going to go forward and use it on a _lot_ of projects (well that’s certainly what I hope) and I will get a small cut each time they deploy it. I’m really super excited about seeing my code deployed this way, as well as by the prospect of passive income.

I also feel as if recently my code has taken a big step forward in quality. I’m not sure exactly what did it, but I suspect that it was either trying to write a good written word piece for the first time, or starting to do daily meditation. Either way my code quality has gone up in a visible and measurable way. I feel good about this because I just did another pass through the code base and feel like it’s getting much closer to something that could be licensed with real value.

Brochure Ape 3.0 (alpha)

Over the Last several days, have been pounding away on a CMS that really takes it’s inspiration from some of the work that Chris did for U-Penn. (It is also partially inspired by cake, and by joomla.) The idea is for the app to be as easy to install and use as is humanly possible. At the same time I expect that some more impressive things will need to be done by more advanced users, so I have sub-divided usage patterns out into 5 levels.

Installation: I’m hoping to get this as simple as possible. I took the time to write up a custom script “installer_helper.php” that drags the user kicking and screaming through the installation process. Hopefully this is enough, though a few steps still seem to hard. (Rather horrifically, the first one is the worst.)

Publisher: Hopefully this will be so easy that an ape could do it. You can either be logged in as an admin, or visiting the page. If you are logged in and visit a page that doesn’t exsist, you get the option to create the page, by selecting from a list of templates. The new page is automatically added to the master navigation. Each of the templates has areas of text, Images, and other sundry that can be edited by clicking on the little “edit” button next to them. Publishers will never need to touch the web server, or HTML.

Designer: New templates useing the smarty mark-up rules. Their are a few additional calls that can be made to put in editable fields, but for the most part it just straight HTML. The new template is simply deposited in the “view” directory, and then becomes an avalible option for the Publishers. Javascript, Images and CSS can all be put in their respective directories in “webroot”

Developer: If you need to add some actual functionality to the site, Brochure Ape can act as a rough and ready framework as well. Custom PHP code can be written, and put in files in the “controler” directory. At any point if a page is created that matches the name of a controller, it will run both the CMS code, and the controler code. There is also a pretty cool table-object that can be used to deal with specific database tables.

Extender: The ultimate idea is for the core code to be broken out into modules, which have been divided up sanely. Something a little bit more like a micro-kernal approch. At this point it’s just not true, and the extender roll would be a hard and unsupported one. I do however intend to keep on cleaning this up.