I recently was setting up a new VPS on Linode and I got the following error when trying to create a new database:

ERROR: new encoding (UTF8) is incompatible with the encoding of the template database (SQL_ASCII) 
HINT: Use the same encoding as in the template database, or use template0 as template.

This error is related to the locale of the system when Postgres was installed. If you run the commands in my previous post before installing Postgres, you should avoid these errors altogether.

If Postgres has already been installed, however, you can run the following commands (assuming Postgres version 8.4) to reinitialise the database with the correct encoding:

pg_dropcluster --stop 8.4 main
pg_createcluster --start -e UTF-8 8.4 main

Note: this will erase all databases and reset your postgres configuration so it is only really useful when you are first setting up your database, or have taken appropriate measures to be able to restore your data.

Have you ever gotten error messages such as the following when running commands such as apt-get?

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_CTYPE = "en_US.UTF-8",
	LANG = (unset)
    are supported and installed on your system.

I’ve had this problem quite a lot on my VPS boxes on Linode and SliceHost. The solution is simple, simply run the following:

locale-gen en_US.UTF-8
update-locale LANG=en_US.UTF-8

Throughout Mac OS X, almost every input field and text area have consistent keyboard shortcuts to navigate and edit the text. However, in both Terminal and iTerm, some of these don’t work like you might be used to. Furthermore, both iTerm and Terminal respond differently to each other, as well as to hour the rest of the system does it.

Three shortcuts I’ve especially missed are: Option-Left to move left one word, Option-Right to move right one word, and Option-Backspace to delete one word to the left.

The ideal solution to get these working as I’d like would be to configure an inputrc file. However, after playing around for a few hours, I still couldn’t get this working reliably using the Option key. So, my fallback is to configure the terminal apps to do this themselves.

If you set up the terminal applications to use Option as the Meta key, some remote systems will already work with the keyboard shortcuts I want. However, locally only the Option-Backspace one works and even then it only works out of the box in Terminal.

So, here’s how to get all three working in both Terminal and iTerm, locally and on remote systems:

Terminal

First, enable Option as meta key option by going to the Preferences > Settings > Your Profile > Keyboard and checking “Use option as meta key” check box. This will get Option-Backspace working.

Next, while still in the Keyboard settings from the last step, press the + button. In the popup, choose “cursor left” for the Key, “option” for the Modifier, and “send string to shell:” for the Action. In the text field, type Esc then lowercase B key and press OK. This will get the Option-Left shortcut working.

To enable the Option-Right shortcut, repeat the previous step, instead choosing “cursor right” for the Key, and typing Esc then lowercase F in the text field.

If you switch Terminal themes, you will have to repeat these steps for the new theme.

iTerm

For the iTerm configuration, I will simply point you to this excellent article which help me get up and running. The sections that correspond to the shortcuts I set up are 2.1, 2.2, and 2.4.

When writing new Cucumber scenarios, developers will usually start with a failing scenario, incrementally making it pass, step by step. If this is a common workflow for you, you’ll probably have noticed a trend of having to re-run it each time a new step is failing, adding something like Then show me the page after the last passing step to see why the story is currently failing.

Chendo and I came up with a nifty little solution in just a few lines that really helps this process of seeing what webrat sees when a scenario fails. Throw this snippet of code in a file in your features/support/ directory:

After do |scenario|
  if scenario.status == :failed
    save_and_open_page
  end
end

I apologise to those who are uninterested for the rant that follows:

I can not stand nor hardly believe that the current government of my otherwise awesome country would choose to completely ignore the advice and expertise of industry professionals and the interests of the voters.

Today Steven Conroy, the Australian Minister of Communications, managed to get a piece of legislation passed introduced that will require all Australian ISPs to participate in the mandatory filtering of content delivered to Australian citizen. Welcome to the likes of Iran and China.

The legislation has been disguised as a measure to protect children from porn and the likes, despite repeated advice from industry professionals that it will do little to curb illegal content, cost the tax payers millions of dollars and ultimately make legal browsing a slower and inferior experience than it already is.

Furthermore, the government has made a effort to actively suppress the contents of the block list, keeping those affected (read: everyone) completely in the dark at to how it will affect their experience. The block list was leaked and shown to include content about which the government should have no say in how or why it is viewed such as some youtube videos, topics on Euthanasia, a gay dating site, flash and downloadable games, and sexual education material (contraceptives, etc).

To the Rudd government:

This is not good enough. I am a voter and a tax payer and I will not stand for this. I will actively participate in removing this ridiculous action and I will march the streets to show my support for an unrestricted flow of information like the rest of the world will continue to enjoy. I will show other tax payers how you have wasted $40 million (or more) of their money on a piece of technology no one wants and that will not work and remind them that this is done in a time of financial uncertainty instead of focusing on real issues like our carbon footprint as a nation. I have the knowledge and the will to undermine this pointless act, and I will use them to do so.

In the process of going through the redesign of
http://mocra.com, I keep finding better ways to make the HTML more semantic.

Jack Chen and I have been turning the design into HTML over the last few days and we’ve been striving to be as semantic as possible by taking advantage of new HTML5 elements and by separating the content further from the design code with all the new CSS3 selectors.

Here’s an example of something we just did to make the HTML cleaner and more meaningful, whilst still maintaining the look and style.

Here’s what we are styling:

Usually your HTML (or HAML in this case) would look something like this:

%footer
  %aside
    = link_to 'Dr Nic Williams', 'http://drnicwilliams.com/'
    ,
    = link_to 'Bodaniel Jeanes', 'http://bjeanes.com/'
    ,
    = link_to 'Jack Chen', 'http://chendo.net/'
    ,                      
    = link_to 'Ryan Bigg', 'http://frozenplague.net/'

However, we can make this list of people more semantic by using an <ul> element and some CSS magic.

Here’s what the new HTML (HAML here) would look like:

%footer
  %aside
    %ul
      %li= link_to 'Dr Nic Williams', 'http://drnicwilliams.com/'
      %li= link_to 'Bodaniel Jeanes', 'http://bjeanes.com/'
      %li= link_to 'Jack Chen',       'http://chendo.net/'
      %li= link_to 'Ryan Bigg',       'http://frozenplague.net/'

And the CSS (Sass here):

body
  footer
    aside
      padding-top: 6px
      float: left

      ul
        list-style: none
        margin: 0
        padding: 0
      
        li
          display: inline
          
          &:after
            content: ","
            
          &:last-child:after
            content: ""

The other day we got some HTML from the designer of one of our clients to implement as the new UI for an app we are building at Mocra. Overall the HTML was good but implementing it started to feel a little moist (i.e. not DRY).

For example, around all the apps form the client’s design had a rounded white box, the HTML of which went something like this:

<div class="content-wht">
    <div class="content-wht-inside">
        <fieldset>
            <div class="frow">
                <label>Currency:</label>
                <select class="fs"><option>EUR</option></select>
            </div>

            <div class="frow frlast">
                <label>File:</label>
                <input type="file" name="" />
            </div>
        </fieldset>
    </div>
</div>

<div class="cround" align="center">
    <img src="/images/round-corners-bottom.gif" alt="" />
</div>

It’s a lot of div tags to have to repeat around every form that has this style. Luckily a nice feature of Rails partials can help out. Create a shared partial file that will act as a layout like this:

<!-- app/views/shared/_fieldset.html.erb -->
<div class="content-wht">
    <div class="content-wht-inside">
        <fieldset>
            <%= yield %>
        </fieldset>
    </div>
</div>

<div class="cround" align="center">
    <img src="/images/round-corners-bottom.gif" alt="" />
</div>

Then in your views that need to have this white box, simply wrap your content in that layout like so:

<% render :layout => 'shared/fieldset' do %>
  <div class="frow">
      <label>Currency:</label>
      <select class="fs"><option>EUR</option></select>
  </div>

  <div class="frow frlast">
      <label>File:</label>
      <input type="file" name="" />
  </div>
<% end %>

<!-- Or, if the inner content is already in a partial -->

<%= render, :partial => 'form', :layout => 'shared/fieldset' %>

Dependencies

Install the GEOS and PROJ frameworks from here

Postgres

Download and Compile

http://ftp2.au.postgresql.org/pub/postgresql/source/v8.4.1/postgresql-8.4.1.tar.bz2 | tar xjf -
cd postgresql-8.4.1
./configure
make && sudo make install

The files are now all installed in the right places, onwards to making them usable!

Create a postgres User and Group

This is adjusted from the instructions in the comments here).

# Find unused Group ID and User ID:
export GROUPID=`sudo dscl . -list /Groups PrimaryGroupID | ruby -e 'puts STDIN.read.scan(/\d+/m).map{|i|i.to_i}.uniq.sort.last.succ'`
export USERID=`sudo dscl . -list /Users UniqueID | ruby -e 'puts STDIN.read.scan(/\d+/m).map{|i|i.to_i}.uniq.sort.last.succ'`

# Create group:
sudo dscl . -create /Groups/_postgres
sudo dscl . -create /Groups/_postgres PrimaryGroupID $GROUPID
sudo dscl . -append /Groups/_postgres RecordName postgres

# Create user:
sudo dscl . -create /Users/_postgres
sudo dscl . -create /Users/_postgres UniqueID $USERID
sudo dscl . -create /Users/_postgres PrimaryGroupID $GROUPID
sudo dscl . -create /Users/_postgres UserShell /bin/bash
sudo dscl . -create /Users/_postgres RealName "PostgreSQL Server"
sudo dscl . -create /Users/_postgres NFSHomeDirectory /usr/local/pgsql
sudo dscl . -append /Users/_postgres RecordName postgres

That’s the clean way of making a user (so it doesn’t show up in your Accounts preference pane, etc.

Clean up

sudo touch /var/log/psql.log
sudo mkdir /usr/local/pgsql/data
sudo chown -R postgres:postgres /usr/local/pgsql/data /var/log/psql.log

# Put Postgres in your path
export $PATH="/usr/local/pgsql/bin:$PATH"
sudo sh -c 'echo /usr/local/pgsql/bin > /etc/paths.d/pgsql'

Thanks to Apple we have a nice way of setting the PATH across the entire system so every app should now know how to find the Postgres binaries and the data directory is now set up to store the database files — now, we just need to create them.

Initialise the Database

# Become the postgres user
sudo su - postgres

# If this throws next commant throws an error about shared memory, 
# try putting "these lines":http://gist.github.com/224815 into 
# /etc/sysctl.conf, rebooting, and trying again:
initdb -D /usr/local/pgsql/data -E UTF8

# Start the database
postgres -D /usr/local/pgsql/data >/var/log/psql.log 2>&1 &

# Create a test database and check you can connect to it
createdb test
psql test

# Just create a superuser for general dev tasks
createuser postgres

# we don't want to be the postgres user anymore
exit

Your done with the Postgres setup now!

PostGIS

curl http://postgis.refractions.net/download/postgis-1.3.7SVN.tar.gz | tar xzf -
cd postgis-1.3.7SVN
./configure --with-geosconfig=/Library/Frameworks/GEOS.framework/unix/bin/geos-config --with-projdir=/Library/Frameworks/PROJ.framework/unix
make && sudo make install

Phew! That one was easy.

Postgres Ruby Gem

This is the part that everyone cares about the most. If this works your Rails apps should start working like a charm.

sudo env ARCHFLAGS='-arch x86_64' gem install pg

Dr Nic’s latest post, “hash bang cucumber”, reminded me of a piece of hax I whipped up a few weeks ago with Ruby and Fish Shell.

Fish has an event system that allows you to register functions to be auto-run after or during certain events (such as when a particular environment variable is changed). One of this events is called fish_command_not_found. It is triggered whenever you type a non-existant command into the prompt.

With this in mind, you can trigger certain commands to run by matching what fish couldn’t manage to run automatically by catching this event.

For instance:

function __fish_method_missing --on-event fish_command_not_found
  method_missing $argv
end

funcsave method_missing

Given that function is created (simply paste the whole code block into your Fish terminal), I can now create a command called method_missing (or whatever you call inside your __fish_method_missing) and place it somewhere in your $PATH — I like ~/.config/fish/bin.

My method_missing binary is simply something like this:

#!/usr/bin/env ruby

command = ARGV.shift

def run(cmd)
  puts "Running #{cmd.inspect} instead"
  system(cmd)
end

case command
when /^git(@|:\/\/).*\.git$/
  run("git clone #{command.inspect}")
when /^(?:ftp|https?):\/\/.+\.t(?:ar\.)?gz$/
  run("curl #{command.inspect} | tar xzv")
else
  $stderr.puts "No default action defined in #{__FILE__.inspect}"
  abort
end

Each command you want to register just becomes a new when statement. For instance, to implement the functionality Dr Nic was trying to achieve, I simply modify the case block as such:

case command
when /^[a-z0-9_\-\/]+\.feature(:\d+)?$/
  run("cucumber #{command}")
# ...
end

The existing entries I have in my method_missing command will auto-clone the repository of a pasted Git URL and download and expand a URL for a tar file, respectively. Not too shabby, and dead easy to implement

Apologies readers — I just realised that the code formatting is kinda screwed up in RSS feeds because I am using some HTML5 elements to represent my code blocks and using CSS to style them appropriately on the site. The RSS feeds obviously lack these stylesheets and so look pretty crap. I am not sure what the best solution is yet. Perhaps I embed a <style> in each RSS post or simply have the RSS feeds only show an excerpt of the full post.

Any ideas?

For a project I am working on at the moment, I needed a Hash that returned a different default value, i.e. not nil. Specifically, I needed it to return another Hash, and for that internal Hash’s default value to be an Array, like so:

hash = # ...
p hash.class                              #=> Hash
p hash[:non_existant_key].class           #=> Hash
p hash[:another][:non_existant_key].class #=> Array

Naïvely, I threw this into my code, and went along my merry way: entries = Hash.new(Hash.new([])). A few days later, I came back to this code to finish it and realised that something was awry with the values I was getting out of the Hash. It took me a while to figure it out because the values were all integers being summed together and I was never looking or setting the values directly.

However, eventually I saw a pattern — many of the values were the same. For instance, given the following assignments, I should expect p entries[153][:monday] #=> [12.hours]:

entries[153][:monday]  << 12.hours
entries[153][:tuesday] << 3.hours
entries[87][:monday]   << 7.hours
entries[87][:tuesday]  << 2.5.hours

However, what I found was the following:

p entries[153][:monday]  #=> [12.hours, 3.hours, 7.hours, 2.5.hours]
p entries[153][:tuesday] #=> [12.hours, 3.hours, 7.hours, 2.5.hours]
p entries[87][:monday]   #=> [12.hours, 3.hours, 7.hours, 2.5.hours]
p entries[87][:tuesday]  #=> [12.hours, 3.hours, 7.hours, 2.5.hours]

# And in fact:
p entries[:any][:thing] #=> [12.hours, 3.hours, 7.hours, 2.5.hours]

Let that soak in for a second. Hash.new({}) uses the same instance of the internal Array as the default value for each of the inner-most keys. It also doesn’t set the key you want so p entries still printed {}. In retrospect, it is blatantly obvious that it does this, but the ramifications are still huge.

The way to get the intended effect is of course to use the block syntax of Hash.new which, while much uglier, definitely works as expected:

entries = Hash.new {|hash, key| hash[key] = Hash.new {|h, k| h[k] = [] }}
entries[153][:monday]  << 12.hours
entries[153][:tuesday] << 3.hours
entries[87][:monday]   << 7.hours
entries[87][:tuesday]  << 2.5.hours

p entries #=> {87=>{:monday=>[7.hours], :tuesday=>[2.5.hours]}, 153=>{:monday=>[12.hours], :tuesday=>[3.hours]}}

Wow. Obvious, but good to remember. I am sure most of you have had this issue before, but I think it is still worthy of mention…

Today I casually tweeted a hypothesis that @chendo drained my battery. The image below shows the amusing responses that followed:

This article isn’t named ideally so if you have a better title, tell me.

This is the scenario: we have a simple HTTP resource on the internal network at Mocra. This resource is a closed source tool which is great except that it has one drawback — there are only two security modes: basic authentication on, or basic authentication off. There is only one username/password and you can’t control when or by whom it is required.

Now, my preferred behaviour here is to be able to give out multiple login/passwords and also to only require authentication outside of our local network. I set out to get this working.

I devised this solution a few months ago (which is why it’s using Apache at all) but I thought it might be of some use to someone. I created an Apache VirtualHost to act as a proxy between this resource and its clients. It looks like this:

<VirtualHost *:80>
  # This virtual host proxies requests for the internal resource
  ServerName external.resource.com

  # Set auth header to base64 encoded 'username:password' 
  # where username and password are the credentials for internal resource
  RequestHeader set Authorization "Basic dXNlcm5hbWU6cGFzc3dvcmQ="

  ProxyPass /resource_path/ http://192.168.1.123:1337/resource_path/
  ProxyPassReverse /resource_path/ http://192.168.1.123:1337/resource_path/

  <Location />
    Order allow,deny
    Allow from 192.168.1.0/24 # our local subnet
    Allow from 127.0.0.1 # duh
    Allow from 1.3.3.7 # our external router IP

    AuthType basic
    AuthName "Our Internal Resource"
    # where all the external login/passwords are
    AuthUserFile /var/www/.htpasswd 
    Require valid-user

    # this is the secret sauce that allows EITHER
    # local access or Basic Auth access
    Satisfy Any 
  </Location>

  RewriteEngine on
  RewriteRule ^/?$ http://external.resource.com:80/resource_path/ [R]
</VirtualHost>

A few weeks ago, I was modifying my clone of Enki (my blogging platform) to use Ultraviolet instead of CodeRay, as it supports using both TextMate themes and syntaxes. This means I can extend it fairly limitlessly and use TextMate itself to modify the themes an syntaxes.

However, Ultraviolet doesn’t come with the best tools to turn the TextMate syntax and theme files into files Ultraviolet can use, and the ones that it does provide puts those translated files INTO the gem on your system. This means that you can’t easily deploy or share them.

So, to remedy this I made a small gem Ultraviolet Tools that provides two command line tools: uv-create-syntax and uv-create-theme.

They aren’t very complicated, nor very documented, but they are straight-forward to use:

Install

Install Ultraviolet Tools with sudo gem install bjeanes-ultraviolet-tools -s http://gems.github.com.

Usage

Themes

Run uv-create-theme path/to/theme.tmTheme

Syntaxes

Run uv-create_syntax path/to/syntax.plist or uv-create_syntax path/to/bundle/with/syntaxes.tmBundle

Using the resulting files

Use my fork of ultraviolet as it has an option to change the load path for themes and syntaxes so that you can load them out of your application files.

Just a quick post so show you how fields_for form helper can cause invalid HTML when used in combination with accepts_nested_attributes_for in your views.

If you are for any reason editing/creating your child objects in a tabular form, your view might look something like this:

<% form_for(@todo_list) do |f| %>
  <%= f.label      :name %>
  <%= f.text_field :name %>

  <h3>To Do Items:</h3>
  <table>
    <thead>
      <tr>
        <th>To Do Item</th>
        <th>Due Date</th>
        <th>Done?</th>
      </tr>
    </thead>
    <tbody>
      <% f.fields_for :items do |item| %>
        <tr>
          <td><%= item.text_field :content %></td>
          <td><%= item.text_field :due_date %></td>
          <td><%= item.check_box  :done %></td>
        </tr>
      <% end %>
    </tbody>
  </table>
  
  <%= f.submit 'Save' %>
<% end %>

At first glance this looks fine. However, when the ToDoList model has accepts_nested_attributes_for :items, the fields_for helper also outputs a hidden field for each existing Item instance with it’s ID.

This means that we get the following HTML:

<form action="/to_do_lists/1" class="edit_to_do_list" id="edit_to_do_list_1" method="post">
  <div style="margin:0;padding:0;display:inline">
    <input name="_method" type="hidden" value="put" />
    <input name="authenticity_token" type="hidden" value="CE2jyRoewbwsqu4eX8AFWFqsgrMfCN35Jzy6b43MhsA=" />
  </div>
  <label for="to_do_list_name">Name</label>
  <input id="to_do_list_name" name="to_do_list[name]" size="30" type="text" value="Sup" />

  <h3>To Do Items:</h3>
  <table>
    <thead>
      <tr>
        <th>To Do Item</th>
        <th>Due Date</th>
        <th>Done?</th>
      </tr>
    </thead>
    <tbody>
      <input id="to_do_list_items_attributes_0_id" name="to_do_list[items_attributes][0][id]" type="hidden" value="1" />
        <tr>
          <td><input id="to_do_list_items_attributes_0_content" name="to_do_list[items_attributes][0][content]" size="30" type="text" value="arst" /></td>
          <td><input id="to_do_list_items_attributes_0_due_date" name="to_do_list[items_attributes][0][due_date]" size="30" type="text" value="2009-07-30" /></td>
          <td><input name="to_do_list[items_attributes][0][done]" type="hidden" value="0" /><input id="to_do_list_items_attributes_0_done" name="to_do_list[items_attributes][0][done]" type="checkbox" value="1" /></td>
        </tr>
      <input id="to_do_list_items_attributes_1_id" name="to_do_list[items_attributes][1][id]" type="hidden" value="2" />
        <tr>
          <td><input id="to_do_list_items_attributes_1_content" name="to_do_list[items_attributes][1][content]" size="30" type="text" value="arst" /></td>
          <td><input id="to_do_list_items_attributes_1_due_date" name="to_do_list[items_attributes][1][due_date]" size="30" type="text" value="2009-07-30" /></td>
          <td><input name="to_do_list[items_attributes][1][done]" type="hidden" value="0" /><input id="to_do_list_items_attributes_1_done" name="to_do_list[items_attributes][1][done]" type="checkbox" value="1" /></td>
        </tr>
    </tbody>
  </table>
  
  <input id="to_do_list_submit" name="commit" type="submit" value="Save" />
</form>

The keen eye will notice the hidden <input> tags that are direct children of the tbody:

    <tbody>
      <input id="to_do_list_items_attributes_0_id" name="to_do_list[items_attributes][0][id]" type="hidden" value="1" />
        <tr><!-- clip! --></tr>
      <input id="to_do_list_items_attributes_1_id" name="to_do_list[items_attributes][1][id]" type="hidden" value="2" />
        <tr><!-- clip! --></tr>
    </tbody>

This is not a valid place for an <input> tag.

I’ve put an example project demonstrating this on my GitHub