Entries Tagged 'Webgeeking' ↓

On Saturday CII played host to the 2nd Christchurch Barcamp, only 2.5 years after the first one. As with the first one, we probably didn’t publicize it as well as we should have, but it turned into a great event nonetheless.

There were many excellent talks of both a technical and general nature – leaning heavily towards the technical, but that more or less suited the participants (I won’t say ‘audience’ because at an unconference there is no real seperation between organizers, presenters and audience!)

At some point we all gathered ourselves together to show solidarity with the 120 people in Wellington hashing out the PublicACTA declaration that day, via twitter. It was unfortunate that we’d already announced the date for the barcamp by the time PublicACTA was announced, as there were certainly several people who would have liked to have gone to both events (including moi!).

#bcchch showing their support for #PublicACTA

As one of the nominal unorganizers of this particular unconference, I’d particularly like to thank CII for hosting the event, my co-unorganizer Pete for doing much of the publicity, getting the ball rolling, and buying the beer, Stephen for organizing the Pizza and doing much of the MC work once things got going, and everyone who showed up, got into the spirit of things, and shared a great day of talks with us, despite it being nice and sunny outside!

A few highlights of the barcamp for me included:

Rus Werner from CrowdFusion gave us a quick demo of how to put together a website using the CrowdFusion framework, which looks to be a highly advanced content management system targeted at large group blogs, but capable of supporting all sorts of content management scenarios. Rus is the only NZ based developer, the rest of the team were in LA that day to put their latest client live – TMZ – which is one heck of a big site.

We were very pleased to have CJ down from Wellington and she joined her fellow Hitlab member Rob Ramsay to talk about a really fascinating idea they’ve had for combining augmented reality and the DigitalNZ APIs into a new kind of mobile-web game. Actually it appears to have got a long way past just being an ‘idea’ and we’re eagerly looking forward to seeing this come to life later this year.

Later on in the day Pete wowed us with his demonstration of what’s possible with HTML5 and Webkit, culminating with a live demo of the Quake engine running at 30fps *in the browser* – I had already heard about this awesome demo (done by some Google guys to show off what’s possible with HTML5 these days), but it’s another thing to actually see it running.

It was great to hear that we have a Camino developer in Christchurch, and Chris gave us an update on the Camino project, entitled ‘Not dead yet!’ or something along those lines. I asked about what it might take to get extensions into Camino, and he seemed open to the concept. My feeling is that it will be best be done using the new Mozilla Jetpack framework, as that does not use any XUL, which Camino avoids in favour of native Mac ‘Cocoa‘ componentry.

My old friend Neil from Screaming Duck software talked about his new lightweight browser plugin (plugin as in something like Flash, rather than an extension like Interclue) ‘thefbi‘ that allows for a subset of native x86 code to run inside a sandbox in the browser. I think there may well be a niche there despite the fact that HTML5 may make most plugins irrelevant in the long run (and many of us would be very happy about that).

Andrew from Morningstar Security gave an overview of the huge scan of NZ based ‘websites’ he originally presented at the last Kiwicon. Apparently lots of really crazy stuff has public facing content on Port 80 these days – Printers, Voip-phones, drilling equipment, you name it. Much of this stuff should only be available behind a firewall or at least a login, but as you would expect, this is not always the case.

Personally I did a presentation about “Personal Idea Management” and although it was ok, I put it together in a bit of a rush and didn’t really get in all the “ideas about ideas” that I wanted to. I got some good feedback afterwards and I look forward to reworking the presentation and giving it another go sometime in the future. Maybe as a Pecha Kucha talk, or at another unconference. At the very least I’ll make a blog post on the subject at some stage! For this talk I was mainly talking about the special class of ideas that could form the core of new tech projects (or even new startups), which is a type I am particularly prone to. This is a pretty tiny subset of the general field of “ideas”, and although in an ideal world one might have software custom designed to help out with it, it’s probably going to be handled mainly within the standard note-taking / brainstorming apps such as Mindmappers, Hierarchical Outliners, and Personal Wikis. I have tried a bunch of these and have never quite settled on one I’m totally happy with, or a system that was in any way ideal, but working on this presentation and the feedback I got afterwards has definitely given me some ideas and I look forward to trying them out. Since the presentation I’ve had two new ideas for iPad apps, and they will join the backlog of 50+ things we could be building if we had any spare capacity right now. Certainly I look forward to the day when some of that backlog hits the front burner!

Many of the folks at the barcamp said we should have another one soon – which hopefully means “within the next year”! I may or may not still be living in Christchurch at the time, but I will try to be there regardless! Remember there’s nothing stopping anyone from getting another barcamp ball rolling at any time, anywhere. Come up with the nugget of a plan, make a post to barcamp.org about it, tell your friends, and you’re on your way! I may make another “lessons learned” post with a few hints about what went right for us and what went wrong this time around, but really all the info you need is at barcamp.org, and if I do make that post I’ll try to find a place for those thoughts there as well.

Anyone who missed Barcamp but would like to present a tech talk in an informal setting, I encourage you to come along to a Spacecraft gathering, and let people know in advance what you’ll be talking about. There are also the monthly TVIC tech dinners – the last one was very good with possibly as many as 20 geeks in attendance.

Following up on my previous post about the Christchurch Digital NZ hackfest, I did find the bug in my search plugin script, it was just one of those minor typos that takes ages to find because the error message you got when you tried to use it was completely uninformative. Sigh. However, since I spent most of my time on this during “Mozilla Service Week” I chalked up a few hours there to add to their total. Kudos to Mozilla for organising that and I’ll be sure to take part in a more serious way should they do it again.

Digital NZ has created a Custom Search Builder, and it seems to me that they could add Search Plugin generation to this reasonably easily. All they need to do is take a copy of my sample and use it to create a template the swaps out the content of the ShortName and Description tag, and also the template attribute in the <url> tag – everything else can stay the same. Then they need to add a javascript install link to the search result pages similar to the one here.

Heres what my sample search plugin file looks like (NB: First bit of code I can recall posting in this blog. I promise not to make it a habit.)


<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/"
xmlns:moz="http://www.mozilla.org/2006/browser/search/">
<ShortName>Digital NZ</ShortName>
<Description>Digital NZ Archive Search</Description>
<InputEncoding>UTF-8</InputEncoding>
<Image width="16" height="16" type="image/x-icon">data:image/x-icon;base64,[data string goes here]</Image>
<Url type="text/html" method="get" template="http://search.digitalnz.org/en/search">
<Param name="search_text" value="{searchTerms}"/>
</Url>
</OpenSearchDescription>


The Webserver has to serve up that XML file with a MIME type of application/opensearchdescription+xml

eg using apache, in a directory where .xml isn’t reserved for anything else, one could put this in a .htaccess file:

AddType application/opensearchdescription+xml .xml

It’s also relatively trivial to add something to the headers of a webpage to enable autodiscovery of one or more search plugins. I’ve done that on my example page. Once you’ve done so it’ll appear in the search plugin manager for the browser, like this:

For more details, and how to do other cool things like enabling search completion, see:

OpenSearch Specification
Mozilla article on creating OpenSearch Plugins for Firefox

Fellow Kiwifoo camper Jo Eaton was in town on the weekend, spreading the good word about their mission to index all New Zealand’s digital content, and make the data available via their new developer API’s, with a travelling “Hackfest”. Fortunately It wasn’t too far for me to walk to take part – it was upstairs at CII, where Interclue is located.

Among the various hacks there was an iPhone app and a Drupal module, and I got most of the way through building a Search Plugin for Firefox and IE, which is a relatively trivial hack in theory but I’d never built one before so it was a useful learning experience.

Unlike fully blown browser extensions such as Interclue or Lazarus, search plugins are just an xml file that when loaded using a special javascript method (only available in certain browsers, such as Firefox 2+ and IE7+) will cause your browser to create another search provider for the search box, which by default in Firefox only has a few general purpose search engines such as Google and Yahoo available, and a few site specific ones such as Wikipedia. But anyone can create a new search plugin for the search on their website, and getting users to install it can mean that they come back to your website more often.

I found a few little niggles, such as that the xml file had to be served up with the right MIME type by the webserver, and that the best way to provide the icon was using a “Data URI” – essentially a way of encoding an image using text. Fortunately Hixie has a kitchen for that.

My attempt is here, and in theory it should install fine in IE7+ and Firefox2+ by clicking the link on this page, but so far, it doesn’t, and I’m not quite sure why. I’ll update this post once I’ve fixed it! [update: fixed]

DigitalNZ has a “roll your own search engine” system set up for their growing collection of Digital Kiwiana, and it should be simple enough to extend that system to build a search plugin for each derived engine, since they will share the same pattern apart from the target URLs. There are also standards for search completion (guessing what you want to search for) and autodiscovery. I’ll make another post in a couple of days once I’ve had a chance to figure it out properly.

[updated because I forgot their mission was only to index the metadata, the digitizing and putting online bit is up to the contributors and partner organizations]

As I mentioned, Lazarus is now a Mozilla recommended add-on. That seemed to be good for about 3,000 downloads a day last month. But on Sunday night I glanced at our stats and noticed we’d had 10,000 downloads. Gosh.

Provisionally, I am inclined to blame this tweet, which eventuated shortly after this post. Cheers Fred. :-)

I expected the download rate to tail off after that. However, it’s actually steadily increasing, thanks to a spate of retweets and blog posts from people like Rick Broida, who is syndicated all over the place. We might clock over 100,000 installs for the week at this rate. That would put Lazarus in the the top 20 Firefox addons, maybe even top 10. Unfortunately I can’t tell right now because the AMO stats are broken – the last 5 days of traffic haven’t been processed for some reason. So everyone’s weekly numbers are currently way off. That may be fixed by the time you read this post, of course.

A few weeks ago I was talking to my advisors about whether we should be focused on Lazarus or Interclue. I argued that Interclue had better long term monetization prospects, and showed them my marvelous J-curves. Nat pointed out that Lazarus had the more obvious value proposition and some incredible user feedback. I threw a few ideas into the ring on how we could extend and monetize Lazarus, and it was mooted that it would be best to get at least one of those ideas into action before we started to actively promote it. However, it appears that active promotion wasn’t exactly required in this case. We appear to have entered a cycle where every time someone complains in a tweet, on a forum, or in their blog that they just lost a pile of text they typed in, someone else chimes in and says “Got Firefox? Get Lazarus.” or words to that effect. Hence we have some pretty impressive word of mouse going on right now.

[UPDATE: It appears there was a bit of a stats glitch! We were never getting more than ~5k installs / day for Lazarus. However, ~5k/day is still very impressive and a considerable boost over what we were getting before that]

Some good news. Lazarus Form Recovery, our little side project, has risen through the ranks and joined Interclue on the AMO “recommended list” – probably the highest accolade available in the world of browser add-ons, apart from perhaps a glowing review in the Mossberg column, as our colleagues at Surf Canyon recently achieved.

The AMO directory (addons.mozilla.org) is linked directly from the Firefox Tools|Add-ons menu, and add-ons from the recommended list are even featured within the browser itself, so the 30-40 add-ons on that list do get a lot of exposure, and having two in there at once really is a great honor, given how many they have to choose from.

I’ve been asked a few times how we managed it. The short version is in both cases I wrote to Mozilla and explained how our addon met their criteria for recommendation, which you can read at the bottom of this page, and not long after that they were recommended. So it’s really about having the right sort of add-on and the right sort of reviews, rather than doing much in the way of lobbying or cajoling.

Most of the credit goes to Karl, who put a lot of effort into making Lazarus almost flawless. My only significant contributions were the original concept, a few innovative implementation ideas (eg asymmetric encryption to get around having to enter a password), and letting him avoid our Interclue todo list for a month or three. It took a while longer to get Lazarus right than we expected, there are a bunch of edge cases where form recovery is hard, but we felt it was worth chasing them all down so we could honestly say “Never lose anything you type into a web form again”.

Part of the Interclue Manifesto says “We will never stop looking for more ways to increase the value of the time people spend online.”, and certainly being able to recover hours of typing that otherwise would have been lost has increased the value of my time online, and from the ecstatic reactions we’ve gotten from Lazarus users, I’d say we haven’t strayed too far from our core mission.

Here are some snippets from Lazarus reviews on AMO:

I guess that means they like it :-)

It’s also gotten good feedback from tech bloggers who picked up on it. Not a lot of mainstream attention so far, probably because I haven’t contacted any of them, but hopefully that will come with time. Hey Walt, about that column of yours….