super lightweight cms

We wanted to roll something out that would allow us to update some areas of our site more easily, and make the content a little more dynamic. Specifically, the News and Announcements page and the Behind the Scenes page.

We've all had experiences with big crazy cms solutions, that do a million things, none of them all that well, and never exactly what you want.

Also, we're a really small team, so we didn't want to set something up that would require a lot of maintenance, or even require a new service or machine.

For "Behind the Scenes", we're already using twitter for company updates. I think this is a great solution for posting short news items, or links, for any site.

The next step was to integrate with Flickr for our Around the office photos. We set up a public, invite only Bizo group. Anyone at Bizo can post photos to this group, and they'll show up on our site. Flickr handles the authentication/authorization, content storage and hosting, and a decent API for getting content on the site.

For "News and Announcements", we wanted a way to be able to easily update any news or press releases, quickly, and without requiring a site release.

We ended up creating a simple blogspot blog to host the content. Blogs/RSS seemed like a pretty close fit. You already have the concept of multiple published items, with dates, titles, and content. Using the excellent rome java library, we pull the content in on the backend, do a little content parsing, then render the results.

The "content parsing" is a little hacky. We wanted slightly more structure around our entries than you get from an RSS feed. So, we expect all of the posts to be formatted like "image (optional), text, read more link". Blogger lets you set up Post Templates, so, it's actually not that bad. When you go "super lightweight", you have to expect to make some tradeoffs, and this seemed like one worth making.

The other nice thing about externalizing your content like this is that you can start pulling it into other places. Last night we launched a new version of our homepage with a flash ticker, that pulls in these news items from the same place.

So there you go: a free, lightweight, zero infrastructure CMS recipe.

Contradictions

So I've got a need to do some work in Rails. Since I've not really used it before, I decided to go through the nice tutorial at Rubyonrails.org. I'm running merrily along when:

% rake db:create
Rails requires RubyGems >= 1.3.1 (you have 1.2.0). Please `gem update --system` and try again.

Okay, fine. I try it...

% sudo gem update --system
Updating RubyGems
Nothing to update

Huh. Really?

I found this helpful tidbit about updating RubyGems on Mac OS X.

% sudo gem install rubygems-update
Successfully installed rubygems-update-1.3.1
1 gem installed
% sudo update_rubygems
Installing RubyGems 1.3.1
[... a bunch of stuff ...]
------------------------------------------------------------------------------
RubyGems installed the following executables:
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/gem

If `gem` was installed by a previous RubyGems installation, you may need
to remove it by hand.

Well, that definitely looks more promising!

% gem --version
1.3.1

Sweet.

java keyword challenge

Found this C++ coding exercise on twitter this morning: Write a standards-conforming C++ program containing a sequence of at least ten different consecutive keywords not separated by identifiers, operators, punctutation characters, etc.

Pretty fun read.

I thought it would be interesting to try and come up with something similar in Java. It seems like I'm constantly adding "final" everywhere, so it can't be that hard, right?

Here's the list of the Java Language Keywords.

Unfortunately, in Java, we don't have anything we can repeat, like sizeof. Also, some of the java keywords require non-keyword tokens ({, }) to be used (do, while), where their c++ equivalents do not.

Anyway, let's give it a shot:

private static transient volatile boolean b;
private static transient final boolean b2 = true;

5 sequential keywords in a member declaration.

Okay, method declarations:

public static synchronized final native void a();
public static synchronized final strictfp void b() { }

I can't say I've ever used native or strictfp, but that gets us to 6 sequential keywords.

There aren't as many sequential keywords we can use for class declarations:

private static final class c { }
private abstract static class d { }

That's only 4 sequential keywords.

So 6 sequential java keywords. That's my max. Can anyone do better?

hadoop job visualization

Last week we had our first hack day at bizo.

We run a ton of hadoop jobs here, many of them multi-stage and completely automated, including machine startup/shutdown, copying data to/from s3, etc.

I wanted to put something together that would help us to visualize the job workflow. I was hoping that by doing this it would give us some insight into how the jobs could be optimized in terms of time and cost.

Here's a generated graph (scaled down a bit, click for full size) for one of our daily jobs:



The program runs through our historical job data, and constructs a workflow based on job dependencies. It also calculates cost based on the machine type, number of machines, and runtime. Amazon's pricing is worth mentioning. Each instance type has a per hour price, and there is no partial hour billing. So, if you use a small machine for 1 hour it's 10 cents, if you use it for 5 minutes, it's also 10 cents. If you use it for 61 minutes, that's 20 cents (2 hours).

As we can see above, a run of this entire workflow cost us $5.40. You can also see that there are a number of steps where we're spending considerably less than an hour, so if we don't care about total runtime, we can save money by reducing the machine size (here we're using all medium machines), or running with fewer machines.

I think the workflow visualization is interesting because it shows that you don't really care about total runtime at all. Since a number of the tasks can run in parallel, it's really the runtime of the longest path along that graph that drives how long you are waiting for the job to complete. In this example, even though we're using 176 minutes of machine runtime, we're only really waiting for 137 minutes.

This means that there are cases where we can spend less money and not affect the overall runtime. You can see that pretty clearly in the example above. "ta_dailyReport" only takes 16 minutes, but on the other branch we're spending 90 minutes on "ta_monthlyAgg" and "ta_monthlyReport." So, if we can spend less money on ta_dailyReport by using fewer machines or smaller machines, as long as we finish in less than 90 minutes we're not slowing down the process.

Yesterday, I decided to play around with the machine configurations based on what I learned from the above graph. Here are the results:



For this job we don't care so much about total runtime, within reason, so I made a few small machine changes at a few different steps.

Even though we're spending an extra 45 minutes of machine time, the longest path is only 13 minutes longer. So, the entire job only takes an extra 13 minutes to run, and we save $1.00 (an 18% savings!).

As you can see there are still a number of tasks where we're using less than an hour, so we could cut costs even further even if we didn't care too much about affecting the overall runtime.

All in all, a pretty successful hackday. My co-workers also worked on some pretty cool stuff, which hopefully they will post about soon.

A quick note on the graphs, for those who are interested. My graphing program simply generated Graphviz files as its output. Graphviz is a totally awesome simple text graphing language for constructing graphs like these. For a really quick introduction, check out this article. Finally, if you're interested, you can check out my raw .gv files here and here.

Is a Square a Rectangle?

Is a Square a Rectangle?

Really great breakdown of thinking about "is a square a rectangle" in the OO sense, and thinking about designing with Inheritance.

From the productivity improvement department...

As a software developer, one of the things I rely upon to get the job done is autocomplete.  It is a feature I use every day whether that be in the bash shell or Eclipse.   The bottom line is that autocomplete saves keystrokes, and, therefore helps me get the job done just a little bit faster.  Autocomplete is a close second to sliced bread.

Now, something that is actually better than sliced bread is Capistrano.  If you've followed our blog, you know that we use Capistrano to deploy our code to EC2 and for other system administration tasks.

So, what if we could take the utility of autocomplete and marry it with the power of Capistrano?  Well, I came across a site that provides a ruby script that does just this.  The script supports Capistrano namespaces and utilizes a bash capability know as programmable completion.  The result?  Pure joy for yours truly.  

Here's an example of the autocomplete capability in action:

In the terminal window, I typed cap common:s[TAB] and was shown a list of possible completions.

If you are a Capistrano user you should definitely check it out.  I highly recommended it.

Open Source Simple DB Firefox Plugin: SDB Tool

We are huge fans of Amazon Web Services here at Bizo. We run our entire infrastructure on EC2, S3, SQS and SimpleDB.

We are also fans of some of the Firefox Plugin tools that make it easier to work with AWS: ElasticFox and S3 Organaizer.

So when we needed a GUI tool that would let us work better with SimpleDB and figured we'd extend the "Firefox Plugin Suite" and build a Firefox Plugin of our own. Today we've officially released our Simple DB Firefox plugin called "SDB Tool". You can install it by clicking here on your Firefox 3.0+ browser.

The code is release under the Apache2 license and available on GitHub. We are using Google Code for issue management.

We hope you find it useful and would love to hear how you are using it.

-Donnie (email me)