Journal tags: apis

Yesterday when I mentioned my paranoia of third-party dependencies on The Session, I said:

I’ve built in the option to switch between multiple geocoding providers. When one of them inevitably starts enshittifying their service, I can quickly move on to another. It’s like having a “go bag” for geocoding.

(Geocoding, by the way, is when you provide a human-readable address and get back latitude and longitude coordinates.)

My paranoia is well-founded. I’ve been using Google’s geocoding API, which is changing its pricing model from next March.

You wouldn’t know it from the breathlessly excited emails they’ve been sending about it, but this is not a good change for me. I don’t do that much geocoding on The Session—around 13,000 or 14,000 requests a month. With the new pricing model that’ll be around $15 to $20 a month. Currently I slip by under the radar with the free tier.

So it might be time for me to flip that switch in my code. But which geocoding provider should I use?

There are plenty of slop-like listicles out there enumerating the various providers, but they’re mostly just regurgitating the marketing blurbs from the provider websites. What I need is more like a test kitchen.

Here’s what I did…

I took a representative sample of six recent additions to the sessions section of thesession.org. These examples represent places in the USA, Ireland, England, Scotland, Northern Ireland, and Spain, so a reasonable spread.

For each one of those sessions, I’m taking:

• the venue name,
• the town name,
• the area name, and
• the country.

I’m deliberately not including the street address. Quite often people don’t bother including this information so I want to see how well the geocoding APIs cope without it.

I’ve scored the results on a simple scale of good, so-so, and just plain wrong.

• A good result gets a score of one. This is when the result gives back an accurate street-level result.
• A so-so result gets a score of zero. This when it’s got the right coordinates for the town, but no more than that.
• A wrong result gets a score of minus one. This is when the result is like something from a large language model: very confident but untethered from reality, like claiming the address is in a completely different country. Being wrong is worse than being vague, hence the difference in scoring.

Then I tot up those results for an overall score for each provider.

When I tried my six examples with twelve different geocoding providers, these were the results:

Geocoding providers

Provider	USA	England	Ireland	Spain	Scotland	Northern Ireland	Total
Google	1	1	1	1	1	1	7
Mapquest	1	1	1	1	1	1	7
Geoapify	0	1	1	0	1	0	3
Here	1	1	0	1	0	0	3
Mapbox	1	1	0	1	1	-1	3
Bing	1	0	0	0	0	0	1
Nominatim	0	0	0	0	-1	1	0
OpenCage	-1	1	0	0	0	-1	-1
Tom Tom	-1	-1	0	0	-1	1	-2
Positionstack	0	-1	0	-1	1	-1	-2
Locationiq	-1	0	-1	0	0	-1	-3
Map Maker	-1	0	-1	-1	-1	-1	-5

Some interesting results there. I was surprised by how crap Bing is. I was also expecting better results from Mapbox.

Most interesting for me, Mapquest is right up there with Google.

So now that I’ve got a good scoring system, my next question is around pricing. If Google and Mapquest are roughly comparable in terms of accuracy, how would the pricing work out for each of them?

Let’s say I make 15,000 API requests a month. Under Google’s new pricing plan, that works out at $25. Not bad.

But if I’ve understood Mapquest’s pricing correctly, I reckon I’ll just squeek in under the free tier.

Looks like I’m flipping the switch to Mapquest.

If you’re shopping around for geocoding providers, I hope this is useful to you. But I don’t think you should just look at my results; they’re very specific to my needs. Come up with your own representative sample of tests and try putting the providers through their paces with your data.

If, for some reason, you want to see the terrible PHP code I’m using for geocoding on The Session, here it is.

I’ve written quite a bit about how I’d like to see a declarative version of the Web Share API. My current proposal involves extending the type attribute on the button element to support a value of “share”.

Well, maybe a different attribute will end up accomplishing what I want! Check out the explainer for the “invokers” proposal over on Open UI. The idea is to extend the button element with a few more attributes.

The initial work revolves around declaratively opening and closing a dialog, which is an excellent use case and definitely where the effort should be focused to begin with.

But there’s also investigation underway into how this could be away to provide a declarative way of invoking JavaScript APIs. The initial proposal is already discussing the fullscreen API, and how it might be invoked like this:

button invoketarget="toggleFullsceen"

They’re also looking into a copy-to-clipboard action. It’s not much of a stretch to go from that to:

button invoketarget="share"

Remember, these are APIs that require a user interaction anyway so extending the button element makes a lot of sense.

I’ll be watching this proposal keenly. I’d love to see some of these JavaScript cowpaths paved with a nice smooth coating of declarative goodness.

I’ve been banging the drum for a button type="share" for a while now.

I’ve also written about other potential button types. The pattern I noticed was that, if a JavaScript API first requires a user interaction—like the Web Share API—then that’s a good hint that a declarative option would be useful:

The Fullscreen API has the same restriction. You can’t make the browser go fullscreen unless you’re responding to user gesture, like a click. So why not have button type=”fullscreen” in HTML to encapsulate that? And again, the fallback in non-supporting browsers is predictable—it behaves like a regular button—so this is trivial to polyfill.

There’s another “smell” that points to some potential button types: what functionality do browsers provide in their interfaces?

Some browsers provide a print button. So how about button type="print"? The functionality is currently doable with button onclick="window.print()" so this would be a nicer, more declarative way of doing something that’s already possible.

It’s the same with back buttons, forward buttons, and refresh buttons. The functionality is available through a browser interface, and it’s also scriptable, so why not have a declarative equivalent?

How about bookmarking?

And remember, the browser interface isn’t always visible: progressive web apps that launch with minimal browser UI need to provide this functionality.

Šime Vidas was wondering about button type="copy” for copying to clipboard. Again, it’s something that’s currently scriptable and requires a user gesture. It’s a little more complex than the other actions because there needs to be some way of providing the text to be copied, but it’s definitely a valid use case.

• button type="share"
• button type="fullscreen"
• button type="print"
• button type="bookmark"
• button type="back"
• button type="forward"
• button type="refresh"
• button type="copy"

Any more?

A while back I wrote a blog post called Web Audio API weirdness on iOS. I described a bug in Mobile Safari along with a hacky fix. I finished by saying:

If you ever find yourself getting weird but inconsistent behaviour on iOS using the Web Audio API, this nasty little hack could help.

Recently Jonathan Aldrich posted a thread about the same bug. He included a link to my blog post. He also said:

Thanks so much for your post, this was a truly pernicious problem!

That warms the cockles of my heart. It’s very gratifying to know that documenting the bug (and the fix) helped someone out. Or, as I put it:

Yay for bugblogging!

Forgive the Germanic compound word, but in this case I think it fits.

Bugblogging doesn’t need to involve a solution. Just documenting a bug is a good thing to do. Recently I documented a bug with progressive web apps on iOS. Before that I documented a bug in Facebook Container for Firefox. When I documented some weird behaviour with the Web Share API in Safari on iOS, I wasn’t even sure it was a bug but Tess was pretty sure it was and filed a proper bug report.

I’ve benefited from other people bugblogging. Phil Nash wrote Service workers: beware Safari’s range request. That was exactly what I needed to solve a problem I’d been having. And then that post about Phil solving my problem helped Peter Rukavina solve a similar issue so he wrote Phil Nash and Jeremy Keith Save the Safari Video Playback Day.

Again, this warmed the cockles of my heart. Bugblogging is worth doing just for the reward of that feeling.

There’s a similar kind of blog post where, instead of writing about a bug, you write about a particular technique. In one way, this is the opposite of bugblogging because you’re writing about things working exactly as they should. But these posts have a similar feeling to bugblogging because they also result in a warm glow when someone finds them useful.

Here are some recent examples of these kinds of posts—tipblogging?—that I’ve found useful:

• Eric wrote about flexibly centering an element with side-sligned content using CSS.
• Rich documented how to subset a variable font on a Mac.
• Stephanie wrote about a CSS technique for animating in a newly added element.

All three are very handy tips. Thanks, Eric! Thanks, Rich! Thanks, Stephanie!

I feel like it’s high time I revived some interest in my proposal for button type="share". Last I left it, I was gathering use cases and they seem to suggest that the most common use case for the Web Share API is sharing the URL of the current page.

If you want to catch up on the history of this proposal, here’s what I’ve previously written:

• A declarative Web Share API
• A polyfill for button type=”share”
• The reason for a share button type

Remember, my proposal isn’t to replace the JavaScript API, it’s to complement it with a declarative option. The declarative option doesn’t need to be as fully featured as the JavaScript API, but it should be able to cover the majority use case. I think this should hold true of most APIs.

A good example is the Constraint Validation API. For the most common use cases, the required attribute and input types like “email”, “url”, and “number” have you covered. If you need more power, reach for the JavaScript API.

A bad example is the Geolocation API. The most common use case is getting the user’s current location. But there’s no input type="geolocation" (or button type="geolocation"). Your only choice is to use JavaScript. It feels heavy-handed.

I recently got an email from Taylor Hunt who has come up with a good litmus test for JavaScript APIs that should have a complementary declarative option:

I’ve been thinking about how a lot of recently-proposed APIs end up having to deal with what Chrome devrel’s been calling the “user gesture/activation budget”, and wondering if that’s a good indicator of when something should have been HTML in the first place.

I think he’s onto something here!

Think about any API that requires a user gesture. Often the documentation or demo literally shows you how to generate a button in JavaScript in order to add an event handler to it in order to use the API. Surely that’s an indication that a new button type could be minted?

The Web Share API is a classic example. You can’t invoke the API after an event like the page loading. You have to invoke the API after a user-initiated event like, oh, I don’t know …clicking on a button!

The Fullscreen API has the same restriction. You can’t make the browser go fullscreen unless you’re responding to user gesture, like a click. So why not have button type="fullscreen" in HTML to encapsulate that? And again, the fallback in non-supporting browsers is predictable—it behaves like a regular button—so this is trivial to polyfill. I should probably whip up a polyfill to demonstrate this.

I can’t find a list of all the JavaScript APIs that require a user gesture, but I know there’s more that I’m just not thinking of. I’d love to see if they’d all fit this pattern of being candidates for a new button type value.

The only potential flaw in this thinking is that some APIs that require a user gesture might also require a secure context (either being served over HTTPS or localhost). But as far as I know, HTML has never had the concept of features being restricted by context. An element is either supported or it isn’t.

That said, there is some prior art here. If you use input type="password" in a non-secure context—like a page being served over HTTP—the browser updates the interface to provide scary warnings. Perhaps browsers could do something similar for any new button types that complement secure-context JavaScript APIs.

After I jotted down some quick thoughts last week on the disastrous way that Google Chrome rolled out a breaking change, others have posted more measured and incisive takes:

• Chris Ferdinandi wrote Google vs. the web,
• Rich Harris wrote Stay alert,
• Chris Coyier wrote Choice Words about the Upcoming Deprecation of JavaScript Dialogs, and
• Jim Nielsen wrote I Want To Confirm a Prompt That We Stay Alert

In fairness to Google, the Chrome team is receiving the brunt of the criticism because they were the first movers. Mozilla and Apple are on baord with making the same breaking change, but Google is taking the lead on this.

As I said in my piece, my issue was less to do with whether confirm(), prompt(), and alert() should be deprecated but more to do with how it was done, and the woeful lack of communication.

Thinking about it some more, I realised that what bothered me was the lack of an upgrade path. Considering that dialog is nowhere near ready for use, it seems awfully cart-before-horse-putting to first remove a feature and then figure out a replacement.

I was chatting to Amber recently and realised that there was a very different example of a feature being deprecated in web browsers…

We were talking about the KeyboardEvent.keycode property. Did you get the memo that it’s deprecated?

But fear not! You can use the KeyboardEvent.code property instead. It’s much nicer to use too. You don’t need to look up a table of numbers to figure out how to refer to a specific key on the keyboard—you use its actual value instead.

So the way that change was communicated was:

Hey, you really shouldn’t use the keycode property. Here’s a better alternative.

But with the more recently change, the communication was more like:

Hey, you really shouldn’t use confirm(), prompt(), or alert(). So go fuck yourself.

Safari has been subjected to a lot of ire recently. Most of that ire has been aimed at the proposed changes to the navigation bar in Safari on iOS—moving it from a fixed top position to a floaty bottom position right over the content you’re trying to interact with.

Courage.

It remains to be seen whether this change will actually ship. That’s why it’s in beta—to gather all the web’s hot takes first.

But while this very visible change is dominating the discussion, invisible changes can be even more important. Or in the case of Safari, the lack of changes.

Compared to other browsers, Safari lags far behind when it comes to shipping features. I’m not necessarily talking about cutting-edge features either. These are often standards that have been out for years. This creates a gap—albeit an invisible one—between Safari and other browsers.

Jorge Arango has noticed this gap:

I use Safari as my primary browser on all my devices. I like how Safari integrates with the rest of the OS, its speed, and privacy features. But, alas, I increasingly have issues rendering websites and applications on Safari.

That’s the perspective of an end-user. Developers who have to deal with the gap in features are more, um, strident in their opinions. Perry Sun wrote For developers, Apple’s Safari is crap and outdated:

Don’t get me wrong, Safari is very good web browser, delivering fast performance and solid privacy features.

But at the same time, the lack of support for key web technologies and APIs has been both perplexing and annoying at the same time.

Alas, that post also indulges in speculation about Apple’s motives which always feels a bit too much like a conspiracy theory to me. Baldur Bjarnason has more to say on that topic in his post Kremlinology and the motivational fallacy when blogging about Apple. He also points to a good example of critiquing Safari without speculating about motives: Dave’s post One-offs and low-expectations with Safari, which documents all the annoying paper cuts inflicted by Safari’s “quirks.”

Another deep dive that avoids speculating about motives comes from Tim Perry: Safari isn’t protecting the web, it’s killing it. I don’t agree with everything in it. I think that Apple—and Mozilla’s—objections to some device APIs are informed by a real concern about privacy and security. But I agree with his point that it’s not enough to just object; you’ve got to offer an alternative vision too.

That same post has a litany of uncontroversial features that shipped in Safari looong after they shipped in other browsers:

Again: these are not contentious features shipping by only Chrome, they’re features with wide support and no clear objections, but Safari is still not shipping them until years later. They’re also not shiny irrelevant features that “bloat the web” in any sense: each example I’ve included above primarily improving core webpage UX and performance. Safari is slowing that down progress here.

But perhaps most damning of all is how Safari deals with bugs.

A recent release of Safari shipped with a really bad Local Storage bug. The bug was fixed within a day. Yay! But the fix won’t ship until …who knows?

This is because browser updates are tied to operating system updates. Yes, this is just like the 90s when Microsoft claimed that Internet Explorer was intrinsically linked to Windows (a tactic that didn’t work out too well for them in the subsequent court case).

I don’t get it. I’m pretty sure that other Apple products ship updates and fixes independentally of OS releases. I’m sure I’ve received software updates for Keynote, Garage Band, and other pieces of software made by Apple.

And yet, of all the applications that need a speedy update cycle—a user agent for the World Wide Web—Apple’s version is needlessly delayed by the release cycle of the entire operating system.

I don’t want to speculate on why this might be. I don’t know the technical details. But I suspect that the root cause might not be technical in nature. Apple have always tied their browser updates to OS releases. If Google’s cardinal sin is avoiding anything “Not Invented Here”, Apple’s downfall is “We’ve always done it this way.”

Evergreen browsers update in the background, usually at regular intervals. Firefox is an evergreen browser. Chrome is an evergreen browser. Edge is an evergreen browser.

Safari is not an evergreen browser.

That’s frustrating when it comes to new features. It’s unforgivable when it comes to bugs.

At least on Apple’s desktop computers, users have the choice to switch to a different browser. But on Apple’s mobile devices, users have no choice but to use Safari’s rendering engine, bugs and all.

As I wrote when I had to deal with one of Safari’s bugs:

I wish that Apple would allow other rendering engines to be installed on iOS devices. But if that’s a hell-freezing-over prospect, I wish that Safari updates weren’t tied to operating system updates.

I spent the last couple of weekends rolling out a new feature on The Session. It involves playing audio in a web page. No big deal these days, right? But the history involves some old file formats…

The first venerable format is ABC notation. File extension: .abc, mime type: text/vnd.abc. It’s an ingenious text format for musical notation using ASCII. The metadata of the piece of music is defined in JSON-like key/value pairs. Then the contents are encoded with letters: A, B, C, etc. Uppercase and lowercase denote different octaves. Numbers can be used for note lengths.

The format was created by Chris Walshaw in 1997 when dial-up was the norm. With ABC, people were able to swap tunes on email lists or bulletin boards without transferring weighty image or sound files. If you had ABC software on your computer, you could convert that lightweight text file into sheet music …or audio.

That brings me to the second old format: midi files. File extension: .mid, mime-type: audio/midi. Like ABC, it’s a lightweight format for encoding the instructions for music instead of the music itself.

Think of it like SVG: instead of storing the final pixels of an image, SVG stores the instructions for drawing the image instead. The instructions in a midi file are like “play this note for this long on this instrument.” Again, as with ABC, you need some software to turn the instructions into sound.

There was a time when lots of software could play midi files. Quicktime on the Mac, for example. You could even embed midi files in web pages. I mean literally embed them …with the embed element. No Geocities page was complete without an autoplaying midi file.

On The Session, people submit tunes in ABC format. Then, using the amazing ABCJS JavaScript library, the ABC is turned into SVG on the fly! For years I’ve also offered midi files, generated on the server from the ABC notation.

But times have changed. These days it’s hard to find software that plays midi files. Quicktime doesn’t do it anymore. And you’d need to go to the app store on iOS to find a midi file player. It’s time to phase out the midi files on The Session.

I still want to provide automatically-generated audio though. Fortunately ABCJS gives me a way to do this. But instead of using the old technology of midi files, it uses a more modern browser feature: the Web Audio API.

The end result sounds like a midi file, but the underlying technique is more like a synthesiser. There’s a separate mp3 file for each note. The JavaScript figures out how long each “sample” needs to be played for, strings them all together, and outputs them with Web Audio. So you’ve got cutting-edge browser technology recreating a much older file format. Paul Rosen—the creator of ABCJS—has a presentation explaining how it all works under the hood.

Not only is there a separate short mp3 file for each note in seven octaves, but if you want the sound of a different instrument, you need samples for all seven octaves in that instrument. They’re called soundfonts.

Paul provides soundfonts for ABCJS. It’s a repo that was forked from this repo from Benjamin Gleitzman. And here’s where it gets small worldy…

The reason why Benjamin has a repo of soundfonts is because he needed to create midi-like audio in the browser. He wanted to do this for a project on September 28th and 29th, 2013 …at Science Hack Day San Francisco!

I was there too—working on my own audio-related hack—and I remember the excellent (and winning) hack that Benjamin worked on. It was called Symphony of Satellites and it’s still online along with the promo video. Here’s Benjamin’s post-hackday write-up from seven years ago.

It’s rare that the worlds of the web and Irish music cross over. When I got to meet Paul—creator of ABCJS—at a web conference a couple of years ago it kind of blew my mind. Last weekend when I set out to dabble with a feature on The Session, I certainly didn’t expect to stumble on a connection to Science Hack Day! (Aside: the first Science Hack Day was ten years ago—yowzers!)

Anyway, I was able to get that audio playback working on The Session. Except for some weirdness on iOS that I had to fix. But that’s a hack for another day.

The latest newsletter from The History Of The Web is a good one: The Browser Engine That Could. It’s all about the history of browsers and more specifically, rendering engines.

Jay quotes from a 1992 email by Tim Berners-Lee when there was real concern about having too many different browsers. But as history played out, the concern shifted to having too few different browsers.

I wrote about this—back when Edge switched to using Chromium—in a post called Unity where I compared it to political parties:

If you have hundreds of different political parties, that’s not ideal. But if you only have one political party, that’s very bad indeed!

I talked about this some more with Brian and Stuart on the Igalia Chats podcast: Web Ecosystem Health (here’s the mp3 file).

In the discussion we dive deeper into the naunces of browser engine diversity; how it’s not the numbers that matter, but representation. The danger with one dominant rendering engine is that it would reflect one dominant set of priorities.

I think we’re starting to see this kind of battle between different sets of priorities playing out in the browser rendering engine landscape.

Webkit published a list of APIs they won’t be implementing in their current form because of security concerns around fingerprinting. Mozilla is taking the same stand. Google is much more gung-ho about implementing those APIs.

I think it’s safe to say that every implementor wants to ship powerful APIs and ensure security and privacy. The issue is with which gets priority. Using the language of principles and priorities, you could crudely encapsulate Apple and Mozilla’s position as:

Privacy, even over capability.

That design principle would pass the reversibility test. In fact, Google’s position might be represented as:

Capability, even over privacy.

I’m not saying Apple and Mozilla don’t value powerful APIs. I’m not saying Google doesn’t value privacy. I’m saying that Google’s priorities are different to Apple’s and Mozilla’s.

Alas, Alex is saying that Apple and Mozilla don’t value capability:

There is a contingent of browser vendors today who do not wish to expand the web platform to cover adjacent use-cases or meaningfully close the relevance gap that the shift to mobile has opened.

That’s very disappointing. It’s a cheap shot. As cheap as saying that, given Google’s business model, Chrome wouldn’t want to expand the web platform to provide better privacy and security.

On Monday, I linked to Tom’s latest video. It uses a clever trick whereby the title of the video is updated to match the number of views the video has had. But there’s a lot more to the video than that. Stick around and you’ll be treated to a meditation on the changing nature of APIs, from a shared open lake to a closed commercial drybed.

It reminds me of (other) Tom’s post from a couple of year’s ago called Pouring one out for the Boxmakers, wherein he talks about Twitter’s crackdown on fun bots:

Web 2.0 really, truly, is over. The public APIs, feeds to be consumed in a platform of your choice, services that had value beyond their own walls, mashups that merged content and services into new things… have all been replaced with heavyweight websites to ensure a consistent, single experience, no out-of-context content, and maximising the views of advertising. That’s it: back to single-serving websites for single-serving use cases.

A shame. A thing I had always loved about the internet was its juxtapositions, the way it supported so many use-cases all at once. At its heart, a fundamental one: it was a medium which you could both read and write to. From that flow others: it’s not only work and play that coexisted on it, but the real and the fictional; the useful and the useless; the human and the machine.

Both Toms echo the sentiment in Anil’s The Web We Lost, written back in 2012:

Five years ago, if you wanted to show content from one site or app on your own site or app, you could use a simple, documented format to do so, without requiring a business-development deal or contractual agreement between the sites. Thus, user experiences weren’t subject to the vagaries of the political battles between different companies, but instead were consistently based on the extensible architecture of the web itself.

I know, I know. We’re a bunch of old men shouting at The Cloud. But really, Anil is right:

This isn’t our web today. We’ve lost key features that we used to rely on, and worse, we’ve abandoned core values that used to be fundamental to the web world. To the credit of today’s social networks, they’ve brought in hundreds of millions of new participants to these networks, and they’ve certainly made a small number of people rich.

But they haven’t shown the web itself the respect and care it deserves, as a medium which has enabled them to succeed. And they’ve now narrowed the possibilites of the web for an entire generation of users who don’t realize how much more innovative and meaningful their experience could be.

In his video, Tom mentions Yahoo Pipes as an example of a service that has been shut down for commercial and idealogical reasons. In many ways, it was the epitome of what Anil was talking about—a sort of meta-API that allowed you to connect different services together. Kinda like IFTTT but with a visual interface that made it as empowering as something like the Scratch programming language.

There are services today that provide some of that functionality, but they’re more developer-focused. Trys pointed me to Pipedream, which looks good but you need to know how to write Node.js code and import npm packages. I’m sure it’s great if you’re into serverless Jamstack lambda thingamybobs but I don’t think it’s going to unlock the potential for non-coders to create cool stuff.

On the more visual pipes-esque Scratchy side, Cassie pointed me to Cables:

Cables is a tool for creating beautiful interactive content.

It isn’t about making mashups, but it does look something that non-coders could potentially use to make something that looks cool. It reminds me a bit of Bret Victor and his classic talk on Inventing On Principle—always worth revisting!

Shockwaves rippled across the web standards community recently when it appeared that Google Chrome was unilaterally implementing a new element called toast. It turns out that’s not the case, but the confusion is understandable.

First off, this all kicked off with the announcement of “intent to implement”. That makes it sounds like Google are intending to, well, …implement this. In fact “intent to implement” really means “intend to mess around with this behind a flag”. The language is definitely confusing and this is something that will hopefully be addressed.

Secondly, Chrome isn’t going to ship a toast element. Instead, this is a proposal for a custom element currently called std-toast. I’m assuming that should the experiment prove successful, it’s not a foregone conclusion that the final element name will be called toast (minus the sexually-transmitted-disease prefix). If this turns out to be a useful feature, there will surely be a discussion between implementators about the naming of the finished element.

This is the ideal candidate for a web component. It makes total sense to create a custom element along the lines of std-toast. At first I was confused about why this was happening inside of a browser instead of first being created as a standalone web component, but it turns out that there’s been a fair bit of research looking at existing implementations in libraries and web components. So this actually looks like a good example of paving an existing cowpath.

But it didn’t come across that way. The timing of announcements felt like this was something that was happening without prior discussion. Terence Eden writes:

It feels like a Google-designed, Google-approved, Google-benefiting idea which has been dumped onto the Web without any consideration for others.

I know that isn’t the case. And I know how many dedicated people have worked hard on this proposal.

Adrian Roselli also remarks on the optics of this situation:

To be clear, while I think there is value in minting a native HTML element to fill a defined gap, I am wary of the approach Google has taken. A repo from a new-to-the-industry Googler getting a lot of promotion from Googlers, with Googlers on social media doing damage control for the blowback, WHATWG Googlers handling questions on the repo, and Google AMP strongly supporting it (to reduce its own footprint), all add up to raise alarm bells with those who advocated for a community-driven, needs-based, accessible web.

Dave Cramer made a similar point:

But my concern wasn’t so much about the nature of the new elements, but of how we learned about them and what that says about how web standardization works.

So there’s a general feeling (outside of Google) that there’s something screwy here about the order of events. A lot discussion and research seems to have happened in isolation before announcing the intent to implement:

It does not appear that any discussions happened with other browser vendors or standards bodies before the intent to implement.

Why is this a problem? Google is seeking feedback on a solution, not on how to solve the problem.

Going back to my early confusion about putting a web component directly into a browser, this question on Discourse echoes my initial reaction:

Why not release std-toast (and other elements in development) as libraries first?

It turns out that std-toast and other in-browser web components are part of an idea called layered APIs. In theory this is an initiative in the spirit of the extensible web manifesto.

The extensible web movement focused on exposing low-level APIs to developers: the fetch API, the cache API, custom elements, Houdini, and all of those other building blocks. Layered APIs, on the other hand, focuses on high-level features …like, say, an HTML element for displaying “toast” notifications.

Layered APIs is an interesting idea, but I’m worried that it could be used to circumvent discussion between implementers. It’s a route to unilaterally creating new browser features first and standardising after the fact. I know that’s how many features already end up in browsers, but I think that the sooner that authors, implementers, and standards bodies get a say, the better.

I certainly don’t think this is a good look for Google given the debacle of AMP’s “my way or the highway” rollout. I know that’s a completely different team, but the external perception of Google amongst developers has been damaged by the AMP project’s anti-competitive abuse of Google’s power in search.

Right now, a lot of people are jumpy about Microsoft’s move to Chromium for Edge. My friends at Microsoft have been reassuring me that while it’s always a shame to reduce browser engine diversity, this could actually be a good thing for the standards process: Microsoft could theoretically keep Google in check when it comes to what features are introduced to the Chromium engine.

But that only works if there is some kind of standards process. Layered APIs in general—and std-toast in particular—hint at a future where a single browser vendor can plough ahead on their own. I sincerely hope that’s a misreading of the situation and that this has all been an exercise in miscommunication and misunderstanding.

Like Dave Cramer says:

I hear a lot about how anyone can contribute to the web platform. We’ve all heard the preaching about incubation, the Extensible Web, working in public, paving the cowpaths, and so on. But to an outside observer this feels like Google making all the decisions, in private, and then asking for public comment after the feature has been designed.

You know what I like? Print stylesheets!

I mean, I’m not a huge fan of trying to get the damn things to work consistently—thanks, browsers—but I love the fact that they exist (athough I’ve come across a worrying number of web developers who weren’t aware of their existence). Print stylesheets are one more example of the assumption-puncturing nature of the web: don’t assume that everyone will be reading your content on a screen. News articles, blog posts, recipes, lyrics …there are many situations where a well-considered print stylesheet can make all the difference to the overall experience.

You know what I don’t like? QR codes!

It’s not because they’re ugly, or because they’ve been over-used by the advertising industry in completely inapropriate ways. No, I don’t like QR codes because they aren’t an open standard. Still, I must grudgingly admit that they’re a convenient way of providing a shortcut to a URL (albeit a completely opaque one—you never know if it’s actually going to take you to the URL it promises or to a Rick Astley video). And now that the parsing of QR codes is built into iOS without the need for any additional application, the barrier to usage is lower than ever.

So much as I might grit my teeth, QR codes and print stylesheets make for good bedfellows.

I picked up a handy tip from a Smashing Magazine article about print stylesheets a few years back. You can the combination of a @media print and generated content to provide a QR code for the URL of the page being printed out. Google’s Chart API provides a really handy shortcut for generating QR codes:

https://chart.googleapis.com/chart?cht=qr&chs=150x150&chl=http://example.com

Except that there’s no telling how long that will continue to work. Google being Google, they’ve deprecated the simple image chart API in favour of the over-engineered JavaScript alternative. So just as I recently had to migrate all my maps over to Leaflet when Google changed their Maps API from under the feet of developers, the clock is ticking on when I’ll have to find an alternative to the Image Charts API.

For now, I’ve got the QR code generation happening on The Session for individual discussions, events, recordings, sessions, and tunes. For the tunes, there’s also a separate URL for each setting of a tune, specifically for printing out. I’ve added a QR code there too.

I’ve been thinking about another potential use for QR codes. I’m preparing a new talk for An Event Apart Seattle. The talk is going to be quite practical—for a change—and I’m going to be encouraging people to visit some URLs. It might be fun to include the biggest possible QR code on a slide.

I’d better generate the images before Google shuts down that API.

I like the robustness that comes with declarative languages. I also like the power that comes with imperative languages. Best of all, I like having the choice.

Take the video and audio elements, for example. If you want, you can embed a video or audio file into a web page using a straightforward declaration in HTML.

<audio src="..." controls><!-- fallback goes here --></audio>

Straightaway, that covers 80%-90% of use cases. But if you need to do more—like, provide your own custom controls—there’s a corresponding API that’s exposed in JavaScript. Using that API, you can do everything that you can do with the HTML element, and a whole lot more besides.

It’s a similar story with animation. CSS provides plenty of animation power, but it’s limited in the events that can trigger the animations. That’s okay. There’s a corresponding JavaScript API that gives you more power. Again, the CSS declarations cover 80%-90% of use cases, but for anyone in that 10%-20%, the web animation API is there to help.

Client-side form validation is another good example. For most us, the HTML attributes—required, type, etc.—are probably enough most of the time.

<input type="email" required />

When we need more fine-grained control, there’s a validation API available in JavaScript (yes, yes, I know that the API itself is problematic, but you get the point).

I really like this design pattern. Cover 80% of the use cases with a declarative solution in HTML, but also provide an imperative alternative in JavaScript that gives more power. HTML5 has plenty of examples of this pattern. But I feel like the history of web standards has a few missed opportunities too.

Geolocation is a good example of an unbalanced feature. If you want to use it, you must use JavaScript. There is no declarative alternative. This doesn’t exist:

<input type="geolocation" />

That’s a shame. Anyone writing a form that asks for the user’s location—in order to submit that information to a server for processing—must write some JavaScript. That’s okay, I guess, but it’s always going to be that bit more fragile and error-prone compared to markup.

(And just in case you’re thinking of the fallback—which would be for the input element to be rendered as though its type value were text—and you think it’s ludicrous to expect users with non-supporting browsers to enter latitude and longitude coordinates by hand, I direct your attention to input type="color": in non-supporting browsers, it’s rendered as input type="text" and users are expected to enter colour values by hand.)

Geolocation is an interesting use case because it only works on HTTPS. There are quite a few JavaScript APIs that quite rightly require a secure context—like service workers—but I can’t think of a single HTML element or attribute that requires HTTPS (although that will soon change if we don’t act to stop plans to create a two-tier web). But that can’t have been the thinking behind geolocation being JavaScript only; when geolocation first shipped, it was available over HTTP connections too.

Anyway, that’s just one example. Like I said, it’s not that I’m in favour of declarative solutions instead of imperative ones; I strongly favour the choice offered by providing declarative solutions as well as imperative ones.

In recent years there’s been a push to expose low-level browser features to developers. They’re inevitably exposed as JavaScript APIs. In most cases, that makes total sense. I can’t really imagine a declarative way of accessing the fetch or cache APIs, for example. But I think we should be careful that it doesn’t become the only way of exposing new browser features. I think that, wherever possible, the design pattern of exposing new features declaratively and imperatively offers the best of the both worlds—ease of use for the simple use cases, and power for the more complex use cases.

Previously, it was up to browser makers to think about these things. But now, with the advent of web components, we developers are gaining that same level of power and responsibility. So if you’re making a web component that you’re hoping other people will also use, maybe it’s worth keeping this design pattern in mind: allow authors to configure the functionality of the component using HTML attributes and JavaScript methods.

It’s the afternoon of the second day of An Event Apart Seattle and Jason is talking about Designing Progressive Web Apps. These are my notes…

Jason wants to talk about a situation you might find yourself in. You’re in a room and in walks the boss, who says “We need a progressive web app.” Now everyone is asking themselves “What is a progressive web app?” Or maybe “How does the CEO even know about progressive web apps?”

Well, trade publications are covering progressive web apps. Lots of stats and case studies are being published. When executives see this kind of information, they don’t want to get left out. Jason keeps track of this stuff at PWA Stats.

Answering the question “What is a progressive web app?” is harder than it should be. The phrase was coined by Frances Berriman and Alex Russell. They listed ten characteristics that defined progressive web apps. The “linkable” and “progressive” characteristics are the really interesting and new characteristics. We’ve had technologies before (like Adobe Air) that tried to make app-like experiences, but they weren’t really of the web. Progressive web apps are different.

Despite this list of ten characteristics, even people who are shipping progressive web apps find it hard to define the damn thing. The definition on Google’s developer site keeps changing. They reduced the characteristics from ten to six. Then it became “reliable, fast, and engaging.” What does that mean? Craigslist is reliable, fast, and engaging—does that mean it’s a progressive web app.

The technical definition is useful (kudos to me, says Jason):

1. HTTPS
2. service worker
3. manifest file

If you don’t have those three things, it’s not a progressive web app.

We should definitely use HTTPS if we want make life harder for the NSA. Also browser makers are making APIs available only under HTTPS. By July, Chrome will mark HTTP sites as insecure. Every site should be under HTTPS.

Service workers are where the power is. They act as a proxy. They allow us to say what we want to cache, what we want to go out to the network for; things that native apps have been able to do for a while. With progressive web apps we can cache the app shell and go to the network for content. Service workers can provide a real performance boost.

A manifest file is simply a JSON file. It’s short and clear. It lists information about the app: icons, colours, etc.

Once you provide those three things, you get benefits. Chrome and Opera on Android will prompt to add the app to the home screen.

So that’s what’s required for progressive web apps, but there’s more to them than that (in the same way there’s more to responsive web design than the three requirements in the baseline definition).

The hype around progressive web apps can be a bit of a turn-off. It certainly was for Jason. When he investigated the technologies, he wondered “What’s the big deal?” But then he was on a panel at a marketing conference, and everyone was talking about progressive web apps. People’s expectations of what you could do on the web really hadn’t caught up with what we can do now, and the phrase “progressive web app” gives us a way to encapsulate that. As Frances says, the name isn’t for us; it’s for our boss or marketer.

Jason references my post about using the right language for the right audience.

Should you have a progressive web app? Well, if you have a website, then the answer is almost certainly “Yes!” If you make money from that website, the answer is definitely “Yes!”

But there’s a lot of FUD around progressive web apps. It brings up the tired native vs. web battle. Remember though that not 100% of your users or customers have your app installed. And it’s getting harder to convince people to install apps. The average number of apps installed per month is zero. But your website is often a customer’s first interaction with your company. A better web experience can only benefit you.

Often, people say “The web can’t do…” but a lot of the time that information is out of date. There are articles out there with outdated information. One article said that progressive web apps couldn’t access the camera, location, or the fingerprint sensor. Yet look at Instagram’s progressive web app: it accesses the camera. And just about every website wants access to your location these days. And Jason knows you can use your fingerprint to buy things on the web because he accidentally bought socks when he was trying to take a screenshot of the J.Crew website on his iPhone. So the author of that article was just plain wrong. The web can do much more than we think it can.

Another common objection is “iOS doesn’t support progressive web apps”. Well, as of last week that is no longer true. But even when that was still true, people who had implemented progressive web apps were seeing increased conversion even on iOS. That’s probably because, if you’ve got the mindset for building a progressive web app, you’re thinking deeply about performance. In many ways, progressive web apps are a trojan horse for performance.

These are the things that people think about when it comes to progressive web apps:

1. Making it feel like a app
2. Installation and discovery
3. Offline mode
4. Push notifications
5. Beyond progressive web app

Making it feel like a app

What is an app anyway? Nobody can define it. Once again, Jason references my posts on this topic (how “app” is like “obscenity” or “brunch”).

A lot of people think that “app-like” means making it look native. But that’s a trap. Which operating system will you choose to emulate? Also, those design systems change over time. You should define your own design. Make it an exceptional experience regardless of OS.

It makes more sense to talk in terms of goals…

Goal: a more immersive experience.

Possible solution: removing the browser chrome and going fullscreen?

You can define this in the manifest file. But as you remove the browser chrome, you start to lose things that people rely on: the back button, the address bar. Now you have to provide that functionality. If you move to a fullscreen application you need to implement sharing, printing, and the back button (and managing browser history is not simple). Remember that not every customer will add your progressive web app to their home screen. Some will have browser chrome; some won’t.

Goal: a fast fluid experience.

Possible solution: use an app shell model.

You want smooth pages that don’t jump around as the content loads in. The app shell makes things seem faster because something is available instantly—it’s perceived performance. Basically you’re building a single page application. That’s a major transition. But thankfully, you don’t have to do it! Progressive web apps don’t have to be single page apps.

Goal: an app with personality.

Possible solution: Animated transitions and other bits of UI polish.

Really, it’s all about delight.

Installation and discovery

In your manifest file you can declare a background colour for the startup screen. You can also declare a theme colour—it’s like you’re skinning the browser chrome.

You can examine the manifest files for a site in Chrome’s dev tools.

Once you’ve got a progressive web app, some mobile browsers will start prompting users to add it to their home screen. Firefox on Android displays a little explainer the first time you visit a progressive web app. Chrome and Opera have add-to-homescreen banners which are a bit more intrusive. The question of when they show up keeps changing. They use a heuristic to decide this. The heuristic has been changed a few times already. One thing you should consider is suppressing the banner until it’s an optimal time. Flipkart do this: they only allow it on the order confirmation page—the act of buying something makes it really likely that someone will add the progressive web app to their home screen.

What about app stores? We don’t need them for progressive web apps—they’re on the web. But Microsoft is going to start adding progressive web apps to their app store. They’ve built a site called PWA Builder to help you with your progressive web app.

On the Android side, there’s Trusted Web Activity which is kind of like PhoneGap—it allows you to get a progressive web app into the Android app store.

But remember, your progressive web app is your website so all the normal web marketing still applies.

Offline mode

A lot of organisations say they have no need for offline functionality. But everyone has a need for some offline capability. At the very least, you can provide a fallback page, like Trivago’s offline maze game.

You can cache content that has been recently viewed. This is what Jason does on the Cloud Four site. They didn’t want to make any assumptions about what people might want, so they only cache pages as people browse around the site.

If you display cached information, you might want to display how stale the information is e.g. for currency exchange rates.

Another option is to let people choose what they want to keep offline. The Financial Times does this. They also pre-cache the daily edition.

If you have an interactive application, you could queue tasks and then carry them out when there’s a connection.

Or, like Slack does, don’t let people write something if they’re offline. That’s better than letting someone write something and then losing it.

Workbox is a handy library for providing offline functionality.

Push notifications

The JavaScript for push notifications is relatively easy, says Jason. It’s the back-end stuff that’s hard. That’s because successful push notifications are personalised. But to do that means doing a lot more work on the back end. How do you integrate with preferences? Which events trigger notifications?

There are third-party push notification services that take care of a lot of this for you. Jason has used OneSignal.

Remember that people are really annoyed by push notifications. Don’t ask for permission immediately. Don’t ask someone to marry you on a first date. On Cloud Four’s blog, they only prompt after the user has read an article.

Twitter’s progressive web app does this really well. It’s so important that you do this well: if a user says “no” to your push notification permission request, you will never be able to ask them again. There used to be three options on Chrome: allow, block, or close. Now there are just two: allow or block.

Beyond progressive web apps

There are a lot of APIs that aren’t technically part of progressive web apps but get bundled in with them. Like the Credentials Management API or the Payment Request API (which is converging with ApplePay).

So how should you plan your progressive web app launch? Remember it’s progressive. You can keep adding features. Each step along the way, you’re providing value to people.

Start with some planning and definition. Get everyone in a room and get a common definition of what the ideal progressive web app would look like. Remember there’s a continuum of features for all five of the things that Jason has outlined here.

Benchmark your existing site. It will help you later on.

Assess your current website. Is the site reasonably fast? Is it responsive? Fix those usability issues first.

Next, do the baseline. Switch to HTTPS. Add a manifest file. Add a service worker. Apart from the HTTPS switch, this can all be done on the front end. Don’t wait for all three: ship each one when they’re ready.

Then do front-end additions: pre-caching pages, for example.

Finally, there are the larger initiatives (with more complex APIs). This is where your initial benchmarking really pays off. You can demonstrate the value of what you’re proposing.

Every step on the path to a progressive web app makes sense on its own. Figure out where you want to go and start that journey.

See also:

• Torre’s notes from this talk.

The supersmart Scott Jenson just gave a talk at The Web Is in Cardiff, which was by all accounts, excellent. I wish I could have seen it, but I’m currently chilling out in Florida and I haven’t mastered the art of bilocation.

Last week, Scott wrote a blog post called My Issue with Progressive Enhancement (he wrote it on Google+, which is why you might not have seen it).

In it, he takes to task the idea that—through progressive enhancement—you should be able to offer all functionality to all browsers, thereby foregoing the use of newer technologies that aren’t universally supported.

If that were what progressive enhancement meant, I’d be with him all the way. But progressive enhancement is not about offering all functionality; progressive enhancement is about making sure that your core functionality is available to everyone. Everything after that is, well, an enhancement (the clue is in the name).

The trick to doing this well is figuring out what is core functionality, and what is an enhancement. There are no hard and fast rules.

Sometimes it’s really obvious. Web fonts? They’re an enhancement. Rounded corners? An enhancement. Gradients? An enhancement. Actually, come to think of it, all of your CSS is an enhancement. Your content, on the other hand, is not. That should be available to everyone. And in the case of task-based web thangs, that means the fundamental tasks should be available to everyone …but you can still layer more tasks on top.

If you’re building an e-commerce site, then being able to add items to a shopping cart and being able to check out are your core tasks. Once you’ve got that working with good ol’ HTML form elements, then you can go crazy with your enhancements: animating, transitioning, swiping, dragging, dropping …the sky’s the limit.

This is exactly what Orde Saunders describes:

I’m not suggesting that you try and replicate all your JavaScript functionality when it’s disabled, above all that’s just not practical. What you should be aiming for is being able to complete the basics - for example adding a product to a shopping cart and then checking out. This is necessarily going to be clunky as judged by current standards and I suggest you don’t spend much time on optimising this process.

Scott asked about building a camera app with progressive enhancement:

Snarky Question: How are you supposed to ‘progressively enhance’ an HTML camera app? Show puppies? Not everything devolves to simple markup

— Scott Jenson (@scottjenson) October 25, 2014

Here again, the real question to ask is “what is the core functionality?” Building a camera app is a means to an end, not the end itself. You need to ask what the end goal is. Perhaps it’s “enable people to share photos with their friends.” Going back to good ol’ HTML, you can accomplish that task with:

<input type="file" accept="image/*">

Now that you’ve got that out of the way, you can spend the majority of your time making the best damn camera app you can, using all the latest browser technologies. (Perhaps WebRTC? Maybe use a canvas element to display the captured image data and apply CSS filters on top?)

Scott says:

My point is that not everything devolves to content. Sometimes the functionality is the point.

I agree wholeheartedly. In fact, I would say that even in the case of “content” sites, functionality is still the point—the functionality would be reading/hearing/accessing content. But I think that Scott is misunderstanding progressive enhancement if he think it means providing all the functionality that one can possibly provide.

Mat recently pointed out that there are plenty of enhancements on the Boston Globe site that require JavaScript, but the core functionality is available to everyone:

@jgarber @mjacksonw Right. Lots of cool features on the Boston Globe don’t work when JS breaks; “reading the news” is not one of them.

— mat marqNOVEMBER (@wilto) September 25, 2014

Scott again:

What I’m chaffing at is the belief that when a page is offering specific functionality, Let’s say a camera app or a chat app, what does it mean to progressively enhance it?

Again, a realtime chat app is a means to an end. What is it enabling? The ability for people to talk to each other over the web? Okay, we can do that using good ol’ HTML—text and form elements—with full page refreshes. That won’t be realtime. That’s okay. The realtime part is an enhancement. Use Web Sockets and WebRTC (in the browsers that support them) to provide the realtime experience. But everyone gets the core functionality.

Like I said, the trick is figuring out what’s core functionality and what’s an enhancement.

Ethan provides another example. Let’s say you’re building a browser-based rich text editor, that uses JavaScript to do all sorts of formatting on the fly. The core functionality is not the formatting on the fly; the core functionality is being able to edit text:

@scottjenson That was how we approached @geteditorially’s rich editor, anyway: start with a textarea, layer on functionality from there.

— Ethan Marcotte (@beep) October 25, 2014

If progressive enhancement truly meant making all functionality available to everyone, then it would be unworkable. I think that’s a common misconception around progressive enhancement; there’s this idea that using progressive enhancement means that you’re going to spend all your time making stuff work in older browsers. In fact, it’s the exact opposite. As long as you spend a little bit of time at the start making sure that the core functionality works with good ol’ fashioned HTML, then you can spend most of your time trying out the latest and greatest browser technologies.

As Orde put it:

What you are going to be spending the majority of your time and effort on is the enhanced JavaScript version as that is how the majority of your customers will be experiencing your site.

The other Scott—Scott Jehl—wrote a while back:

For us, building with Progressive Enhancement moves almost all of our development time and costs to newer browsers, not older ones.

Progressive Enhancement frees us to focus on the costs of building features for modern browsers, without worrying much about leaving anyone out. With a strongly qualified codebase, older browser support comes nearly for free.

Approaching browser support this way requires a different way of thinking. For everything you’re building, you need to ask “is this core functionality, or is it an enhancment?” and build accordingly. It takes a bit of getting used to, but it gets easier the more you do it (until, after a while, it becomes second nature).

But if you’re thinking about progressive enhancement as “devolving” down—as Scott Jenson describes in his post—then I think you’re on the wrong track. Instead it’s about taking care of the core functionality quickly and then spending your time “enhancing” up.

Scott asks:

Shouldn’t we be allowed to experiment? Isn’t it reasonable to build things that push the envelope?

Absolutely! And the best and safest way to do that is to make sure that you’re providing your core functionality for everyone. Once you do that, you can go nuts with the latest and greatest experimental envelope-pushing technologies, secure in the knowledge that you don’t even need to worry about the fact that they don’t work in older browsers. Geolocation! Offline storage! Device APIs! Anything you can think of, you can use as a powerful enhancement on top of your core tasks.

Once you realise this, it’s immensely liberating to use progressive enhancement. You can have the best of both worlds: universal access to core functionality, combined with all the latest cuting-edge technology too.

There are many reasons to go to South by Southwest Interactive: meeting up with friends old and new being the primary one. Then there’s the motivational factor. I always end up feeling very inspired by what I see.

This year, that feeling of inspiration was front and centre. First off, I tried to impart some of it on the How to Rawk SXSW panel, which was a lot of fun. Mind you, I did throw some shit at the fan by demonstrating how wasteful the overstuffed schwag bags are. I hope I didn’t get MJ into trouble.

My other public appearance was on The Heather Gold Show which was bags of fun. With a theme of Get Excited and Make Things, the topic of inspiration was bandied about a lot. It was a blast. Heather is a superb host and the other guests were truly inspirational. I discovered a kindred spirit in fellow excitable geek, Gina Trapani.

The actual panels and presentations at SXSW are the usual mixture of hit and miss, although the Cooking For Geeks presentation was really terrific. Any presenter who hacks the audience’s taste buds during a presentation is alright with me.

But by far the most inspirational thing I’ve seen was a panel hosted by Tantek on Open Science. The subject matter was utterly compelling and the panelists were ludicrously articulate and knowledgeable:

• Dr. Kiki,
• Ariel,
• Jessy Cowan-Sharp and
• Natalie Villalobos

The URLs were flying thick and fast: the Signtific thought experiment game, the collaborative Galaxy Zoo—now joined by Moon Zoo—and the excellent Spacehack directory.

I was struck by the sheer volume of scientific data and APIs out there now. And yet, we aren’t really making use of it. Why we aren’t we making mashups using Google Mars? Why haven’t I built a Farmville-style game with Google Moon?

Halfway through the panel, I turned to Riccardo and whispered, We should organise a Science Hack Day.

I’m serious. It would probably be somewhere in London. I have no idea where or when. I have no idea how to get a venue or sponsors. But maybe you do.

What do you think? Everyone I’ve mentioned the idea to so far seems pretty excited about the idea. I’ll try to set up a wiki for brainstorming venues, sponsors, APIs, datasets and all that stuff. In the meantime, feel free to leave a comment here.

I got excited. Now I want to make things …with science! Are you with me?

The mighty Zeldman has written a thought-provoking piece called The Vanishing Personal Site which chronicles the changing nature of personal publishing. Where once we had a central URL that defined our online presence, people are increasingly publishing in fragments distributed across services like Twitter, Pownce, Flickr and Magnolia. It was this fragmentation that spurred my first dabblings with APIs to produce Adactio Elsewhere which I did three years ago to the day.

Jeff takes a different approach by incorporating all of those other publishing points directly back into his site rather than a separate aggregation area. This approach seems to be gaining ground.

One of the comments to Jeffrey’s post points to the newly launched website of the architect Denna Jones built in part by Jon Tan who describes the thinking behind it. The site is driven entirely by third-party services like Tumblr, Del.icio.us and Flickr. Jon, by contrast, has his third-party publishing aggregated on a page called Asides, similar to Adactio Elsewhere.

I think most people, even if they are micro-publishing in many places, still have one URL that they consider as their online representation. It might be a blog, it might be a Flickr profile, or for many people, it might be a Facebook account.

It will be interesting to watch these trends develop. Something else I’m going to watch is Jon Tan’s website. It’s dripping with gorgeous typography wrapped in an elastic layout. How is that I haven’t come across this site before? Why wasn’t I informed?

I’ve used del.icio.us for quite a while now. I’m storing 1159 bookmarks, each one of them tagged. It works just fine but it also feels a little, I don’t know …stale. There is supposedly a redesign in the works but I’m not sure that I want to wait around any longer to find out if they’re finally going to put some microformats in the markup.

Instead, I’m moving over to Magnolia. I’ve had a Magnolia account for years but I’ve never really used it. I didn’t see the point while I had a del.icio.us account. But whereas de.icio.us appears stagnated, Magnolia seems to be constantly innovating. Also, it uses microformats. There’s also the fact that I know Larry and I’ve briefly met Todd (lovely gents, both) but I don’t know Joshua Schachter. That shouldn’t matter but it kind of does.

Moving from del.icio.us to Magnolia is very straightforward. But that alone wasn’t going to be enough for me. I’m also accessing my del.icio.us bookmarks through the API. It turns out that Magnolia provides an ingenious way to ease my pain. As well as providing their own fully-featured API, Magnolia also provides a mirror of the infamously sucky del.icio.us API. All I had to do was change some URL endpoints and I had Adactio Elsewhere switched over in no time. Other services take note: providing mirrored versions of your competitors’ APIs eases the pain of migration.

I’ve updated my feedburner RSS feed to point it at my Magnolia links instead of my del.ious.us links. If you were subscribed to my del.icio.us feed separately, you’ll probably want to update your feedreader to point to my Magnolia links instead.

It remains to be seen whether I’ll stay at Magnolia. Even though it is functionally and cosmetically superior to del.icio.us, that might not be enough. After all, Jaiku is superior to Twitter in almost every way—design,markup, reliability—but Twitter still wins. That’s mostly because that’s where all my friends are. Right now my bookmarking friends are split fairly evenly between del.icio.us and Magnolia. Then again, I’ve never really made much use of the “social” part of “social bookmarking”.

So who knows? Maybe I’ll end up moving back to del.icio.us at some stage. It’s reassuring to know that moving my data around between these services is pretty straightforward: I can export from Magnolia and import into del.icio.us any time I want.

Hackday is almost upon us. Tomorrow, I—along with hundreds of other geeks—will be converging on Alexandra Palace in North London for two days of dev fun.

I’ve got an idea for what I want to do but I think I’ll need lots of help. At XTech, Reboot, @media and other recent geek gatherings I’ve been asking who’s coming and who fancies helping me out. I’ve managed to elicit some interest from some very smart people so I’m hoping that we can hack something fun together.

Here’s the elevator pitch for my idea: online publishing is hacking and slaying.

Inspired by Justin Hall’s idea of Passively Multiplayer Online Games and Gavin Bell’s musings on provenance, I want to treat online publishing as an ongoing way of building up a character. In Dungeons and Dragons or World of Warcraft, you acquire attributes like stamina, strength, dexterity and skill over time. Online, you publish Flickr pictures, del.icio.us links, Twitter updates and blog posts over time. All of this published material contributes to your online character and I think you should be rewarded for this behaviour.

It’s tangentially related to the idea of a lifestream which uses RSS to create a snapshot of your activity. By using APIs, I’m hoping to be able to build up a much more accurate, long-term portrait.

I’m going to need a lot of clever hackers to help me come up with the algorithms to figure out what makes one person a more powerful Flickrer or Twitterer than another. Once the characteristics have been all figured out, we can then think about pitching people against each other. Maybe this will involve a twenty-sided die, maybe it will more like Top Trumps, or maybe it could even happen inside Second Life or some other environment that has persistent presence (the stateless nature of the Web makes it difficult to have battles on a Web site). I have a feel that good designers and information architects would be able to help me figure out some other fun ways of representing and using the accumulated data. Perhaps we can use geo data to initiate battles between warriors in the same geographical area.

Sound like fun? Fancy joining in? Seek me out on the day or get in touch through my backnetwork profile.

Of course, if you want to do something really cool at hackday, you’ll probably be dabbling with arduino kits, blubber bots and other automata. When I was San Francisco a few weeks ago, nosing around the Flickr offices, Cal asked me what I was planning for Hackday. “Well” I said, “it involves using APIs to…” “Pah!” he interrupted, “APIs are passé. Hardware is where it’s at.”

One of the highlights of Refresh Edinburgh for me was listening to Dan Champion give a presentation on his new site, Revish. He talked through the motivation, planning and production of the site. This was an absolute joy to listen to and it was filled with very valuable practical advice.

Revish is a book review site with a heavy dollop of social interaction. Even in its not-quite-finished state, it’s pushing all the right buttons with me:

• The markup is clean, semantic and valid.
• The layout is uncluttered and flexible.
• The URL structure is logical.
• The data is available through microformats, RSS and an API.

There’s some really smart stuff going on with the sign-up process. If your chosen username matches a Flickr username, it automatically grabs the buddy icon. At the sign-up stage you also have the option of globally disabling any Ajax on the site—an accessibility option that I advocate in my book. Truth be told, there isn’t yet any Ajax on the site but the availability of this option shows a lot of forethought.

Also at the sign-up stage, there’s a quick’n’dirty auto-discovery of contacts wherever there’s overlap with Revish usernames and your Flickr contacts. This is very cool—one small step toward portable social networks.

One of the features dovetails nicely with Richard’s recent discussion about machine tags ISBNs. If you tag a picture of a book on Flickr with book:isbn=[ISBN number], that picture will then show up on the corresponding Revish page. You can see it in action on the page for Bulletproof Ajax.

Oh, and don’t worry about whether a book has any reviews on Revish yet: the site uses Amazon’s API to pull in the basic book info. As long as a book has an ISBN, it has a page on Revish. So the Revish page for a book can effectively become a mashup of Amazon details and Flickr pictures (just take a look at the page for John’s new microformats book).

I like this format for machine tagging information related to books. As pointed out in a comment on Richard’s post, this opens up the way for plenty of other tagging like book:title="[book title]" and book:author="[author name]".

I’ve started to implement this machine tag format here. If you look at my last post—which has a whole list of books—you’ll see that I’ve tagged the post with a bunch of machine tags in the book:isbn format. By making a quick call to Amazon, I can pull in some information on each book. For now I’m just displaying a small cover image with a link through to the Amazon page.

That last entry is a bit of an extreme example; I’m assuming that most of the time I’ll be just adding one book machine tag to a post at most, probably to accompany a review.

Machine tags (or triple tags) is still a relatively young idea. Most of the structures so far have been emergent, like Upcoming and Last.fm’s event tags and my own blog post machine tags. There’s now a site dedicated to standardising on some namespaces—MachineTags.org has a blog, a wiki and a mailing list. Right now, the wiki has pages for existing conventions like geo tagging and drafts for events and book tagging. This will be an interesting space to watch.