Webmention

From IndieWeb

Webmention is a simple way to notify any URL when you link to it on your site. From the receiver's perspective, it's a way to request notifications when other sites link to it. Webmention is a modern update to Pingback, using only HTTP and x-www-urlencoded content rather than XMLRPC requests. Webmention supersedes Pingback.

Status
This is an Editor's Draft yet mature enough to encourage implementations and feedback.
Participate
Wiki (Feedback, Open issues)
IRC: #indiewebcamp on Freenode
Concept and v0.1
Sandeep Shetty
Contributors
See revision history for contributors
License
Per CC0, to the extent possible under law, the editor(s) and contributors have waived all copyright and related or neighboring rights to this work. In addition, as of 2024-12-16, the editor(s) and contributors (2015-04-07 onward) have made this specification available under the Open Web Foundation Agreement Version 1.0.

Protocol Summary

If Aaron's blog supports receiving webmentions, and Barnaby's blog supports sending webmentions, then this is how they interact:

  1. User Aaron posts a blog post on his blog
  2. User Barnaby writes post on his blog that links to Aaron's post.
  3. After publishing the post (i.e. it has a URL), Barnaby's server notices this link as part of the publishing process
  4. Barnaby's server does webmention discovery on Aaron's post to find its webmention endpoint (if not found, process stops)
  5. Barnaby's server sends a webmention to Aaron's post's webmention endpoint with
    • source set to Barnaby's post's permalink
    • target set to Aaron's post's permalink.
  6. Aaron's server receives the webmention
  7. Aaron's server verifies that target (after following redirects) in the webmention is a valid permalink on Aaron's blog (if not, processing stops)
  8. Aaron's server verifies that the source (when retrieved, after following redirects) in the webmention contains a hyperlink to the target (if not, processing stops)

Unmentioned but implied (and vaguely mentioned in the pingback spec):

  • Aaron's server displays the information about Barnaby's post somewhere on Aaron's post.

See comment: how to accept a comment for more details on accepting comments in general and displaying them.

Protocol

How to send webmentions

Sender discovers receiver endpoint

The webmention endpoint is advertised in the HTTP Link header or a <link> or <a> element with rel="webmention" . If more than one of these is present, the HTTP Link header takes precedence, followed by the <link> element, and finally the <a> element. Clients MUST support all three options and fall back in this order. 

GET /post-by-aaron HTTP/1.1
Host: aaronpk.com

HTTP/1.1 200 OK
Link: <http://aaronpk.com/webmention-endpoint>; rel="webmention"

<html>
<head>
...
<link href="http://aaronpk.com/webmention-endpoint" rel="webmention" />
...
</head>
<body>
....
<a href="http://aaronpk.com/webmention-endpoint" rel="webmention" />
...
</body>
</html>

Sender notifies receiver

The sender posts form-encoded source and target, where source is the URL of the page containing a link, and the target is the URL of the page being linked to.

202 Accepted is the recommended status code to return indicating that the request SHOULD be queued and processed asynchronously to prevent DoS attacks. The response body SHOULD include a URL that can be used to monitor the status of the request.

See Error Responses for what to do when the webmention is not successful. 

POST /webmention-endpoint HTTP/1.1
Host: aaronpk.com
Content-Type: application/x-www-url-form-encoded

source=https://waterpigs.co.uk/post-by-barnaby&
target=https://aaronpk.com/post-by-aaron

HTTP/1.1 202 Accepted

https://aaronpk.com/webmentions/222

How to receive a webmention

Upon receipt of a POST request containing source and target, 202 Accepted is the recommended status code to return indicating that the request SHOULD be queued and processed asynchronously to prevent DoS attacks. The response body SHOULD include a URL that can be used to monitor the status of the request.

If you choose to process the request and perform the verification step synchronously, you can respond with a 200 OK status on success.

Verification

1. The receiver SHOULD check that target is a valid resource belonging to it and that it accepts webmentions.

2. The receiver SHOULD perform a HTTP GET request on source to confirm that it actually links to target (note that the receiver will need to check the Content-type of the entity returned by source to make sure it is a textual response).

3. At this point the receiver can choose to publish information about this webmention along with any other data it picks up from source.

Error Responses

If the webmention was not successful because of something the sender did, you SHOULD return a 400 Bad Request status code and MAY include a description of the error in the response body.

Possible sender related errors (from the Pingback specification):

  • Source URL not found.
  • Specified target URL not found.
  • Source URL does not contain a link to the target URL.
  • Specified target URL does not accept webmentions.
  • Receiver Error

If the webmention was not successful because of an error on the receivers server, it SHOULD return a 500 Internal Server Error status code and MAY include a description of the error in the response body.

Updating existing webmentions

If receiver had received a webmention in the past with the same source and target then,

  • If both the verification steps are successful, it SHOULD update any existing data it picked from source for the existing webmention.
  • If it received a 410 on step 2 (performing a GET request on source) or does not find a link to target on source, it SHOULD delete the existing webmention.

Security and Privacy

Preventing Abuse

  • The verification process SHOULD be queued and processed asynchronously to prevent DDoS attacks.
  • Receivers SHOULD moderate Webmentions, and if a link is displayed back to the source, SHOULD link to source with rel="nofollow" to prevent spam.
  • Receivers MAY periodically re-verify webmentions and update them.
  • If a receiver chooses to publish data it picks up from source, it should ensure that the data is encoded and/or filtered to prevent XSS and CSRF attacks.

Why

Why webmentions (e.g. instead of Pingback) and why you should implement them.

Webmentions are:

  • Simpler. By dropping XML-RPC, webmention is simpler than Pingback. This means:
    • Less work to implement
    • Easier to test
    • Both of which combine to enable more reliable, interoperable implementations, sooner.
  • Re-using only HTTP enables easier testing and UI. By using only HTTP, it's possible to construct simple HTML forms that exercise the protocol, which is a good design principle for web protocols in general[1]. This enables simpler/easier testing (with just a static HTML file), and the ability to provide simple (no JS needed) webmention submit forms on blog posts for others to paste URLs to their replies (some IndieWeb participants are already doing this, e.g. http://adactio.com/ and http://aaronparecki.com/ posts).

How to test webmentions

Sending Webmentions

The beauty of simple protocols is that you can do most of this manually very easily.

  • Given a target to which you want to send a webmention, you need to first discover it's webmention endpoint:
curl -i -s $target | grep 'rel="webmention"'
    • Note: this is only an approximation, your implementation MUST accept a "webmention" value among the space separated set of terms in the 'rel' attribute. Older implementations may use rel="http://webmention.org/" and some may use both: rel="webmention http://webmention.org/" neither of which are discovered by the grep above. Some may use single quotes (or no quotes at all) to surround the webmention rel value. Still others may only support discovery with an HTTP Link header (e.g. Taproot). Compliant implementations must check the HTTP Link headers for the rel value first, and if not found then parse for the rel value on a <link> or <a> element in the <head> per the HTML specification on parsing rel values.
  • Now to send the webmention:
curl -i -d "source=$your_url&target=$target_url" $targets_webmention_endpoint

One-liner webmentions

This will send the webmention in a single command: 

curl -i -d "source=$your_url&target=$target_url" `curl -i -s $target_url | grep 'rel="http://webmention.org/"' | sed 's/rel="webmention"//' | grep -o -E 'https?://[^ ">]+' | sort | uniq`

See also this gist for sending pingbacks manually: https://gist.github.com/aaronpk/5744879

Accepting Webmentions

Currently the easiest way to test consuming webmentions is to implement sending them, then send yourself mentions.

Once you’re ready to test integration with another site, you can either ask someone in IRC to send you a comment, or mention your page in IRC and send a webmention from IRC log URL for that line to your URL.

For example this IRC line links to http://caseorganic.com/notes/2014/02/18/1/. Pasting the URL for the IRC line into the indiewebify.me webmention sender will send webmentions for all the links on the page. Then you can check your page or your server logs to see if the webmention was successful — in this case the comment successfully showed up on the target page.

Implementation Details

There are some interesting implementation details regarding receiving webmentions. E.g. when to follow redirects and when not to.

endpoint discovery

In step 4 in the protocol summary above, when performing webmention endpoint discovery, implementations MUST check for the webmention and SHOULD check for http://webmention.org/ (for backward compatiblity) as rel values which may be present among a space separated set of rel values inside a rel attribute.

Also note that endpoints may be advertised in both <link> and <a> elements. Supporting <a> is important for sites like WordPress.com that don't allow link elements at all.

must follow redirects

In steps 7 and 8 above, implementations MUST follow redirects of the target and source URLs.

checking target validity

In step 7 above, when the webmention receiver checks the target URL for validity, follow any HTTP redirects and use the resulting URL when checking for validity.

For example, if your posts visibly show permashortlinks for others to copy/cite/link to and someone sends a webmention to a shortlink, be sure to resolve any redirects before checking if it’s on your site, or matching it up to a post on your site.

The webmention receiver should then check if the target is valid and supported. A valid target is one that is recognized by the receiver.

In simple cases, this is just a check that the target URL exists on the receiver's domain. The meaning of "valid" can be extended to also include:

  • mentions of pages that don't exist on your site (in case you want to track possible lost URLs)
  • mentions of POSSE URLs

use cases

  • Webmention proxy servers such as webmention.io and webmention.herokuapp.com are able to accept webmentions for other people's sites
  • Enables indieweb-friendly silos to send backfeed webmentions for comments and responses to the original post of a POSSE copy on that silo.
    • i.e. if you POSSE to a silo, you should accept webmentions where the target is your POSSE copy
    • This requires indieweb-friendly silos send webmentions to the original site's webmention endpoint, so the silo will need to know that one of its posts is a POSSE copy


You should also support receiving webmentions where the target is one of your POSSE copies.

verifying source links to target

In step 8 above, as the webmention receiver, to verify the source URL:

  • retrieve it following redirects,
  • when checking the resultant HTML, look for the target URL as literally provided in the webmention request (i.e. NOT redirects from target)

For example, if you get a webmention with source=http://short.me/t100 target=http://example.com/notes/5:

sync or async response status code

Webmention receivers can choose to verify the webmention synchronously (process immediately) or asynchronously (add it to a queue for processing later). High level difference:

Synchronously:

  • upon successful verification the receiver MUST return a HTTP 200 Success response

Asynchronously:

  • receiver MUST return HTTP 202 Accepted.

See the Brainstorming section for some additional thinking on this.

types of mentions

In addition to just a link (mention) there are several special types of mentions, each which have markup for detection and specific implementation recommendations:

accessing endpoints in the browser

Some implementations now display a page that briefly explains webmentions when the webmention endpoint is accessed in a browser.

Implementations

IndieWeb Examples

The following users/sites have implemented sending and or receiving webmentions, in rough chronological order:

  • Aaron Parecki using p3k on aaronparecki.com as of 2013-05-26. Details:
    • All posts send webmentions (as of 2013-05-26 first (semi-)automatic webmention sent on 2013-03-31) upon posting (asynchronously with posting UI, no results reported).
    • Events, notes (including RSVPs), replies accept webmentions (as of 2013-05-26)
    • Discovery with both: rel="webmention" and rel="http://webmention.org/" links in head, and HTTP Link: rel="http://webmention.org/ header
    • Notes (including RSVPs), replies have an input form for pasting in your comment URL to conveniently send a manual webmention (as of 2013-10-12)
  • Ben Werdmuller using idno on werd.io as of 2013-05-31. Details:
    • All(?) posts send webmentions as of 2013-05-31 and accept as of 2013-06-??
    • Discovery with both: rel="http://webmention.org/" and rel="webmention" links in head, and with HTTP Link: <http://werd.io/webmention/>; rel="webmention"
  • Barnaby using Taproot on waterpigs.co.uk as of 2013-06-11 (per git). Details:
    • All notes and articles accept (as of 2013-06-20) and send webmentions (as of 2013-06-11) upon posting (posting UI returns response content to the browser, then synchronously sends webmentions)
    • Incoming webmentions for notes and articles are parsed and stored, displayed for notes (comment updating as of 2013-06-23) but not yet articles
    • Discovery only with HTTP Link: <http://waterpigs.co.uk/mentions/webmention/>; rel="http://webmention.org/"
  • Jeremy Keith on adactio.com as of 2013-09-15. Details:
    • All blog posts accept webmentions
    • Discovery only with: rel="webmention" link in head
    • Blog posts have an input form for pasting in your comment URL to conveniently send a manual webmention
  • Barry Frost on barryfrost.com as of 2013-09-15[2][3]. Details:
    • All(?) posts send and accept webmentions
    • Discovery with: rel="webmention" link in head, and HTTP Link: <http://barryfrost.com/webmention>; rel="webmention" in header
    • Posts have a hyperlink in footer "How to comment on this post" to instructions detailing posting an h-entry with h-card and how to send a webmention.
  • Christian Weiske receiving webmentions on cweiske.de as of 2013-10-22. Details:
    • All posts accept webmentions using stapibas
    • Discovery only with HTTP Link: <http://cweiske.de/stapibas/xmlrpc.php>; rel="webmention"
    • Approach for sending webmentions (with fallback sending of pingbacks)
      • https://github.com/pear2/Services_Linkback/blob/master/src/PEAR2/Services/Linkback/Client.php#L236
      • check self Atom feed, if there are updates then proceed with discovery/sending for each update, for each link in the update: do a HEAD request first, do discovery for webmention endpoint, if none found, do discovery for pingback endpoint, if none found and the resource is 'application/xhtml+xml' or 'text/html', do a GET request, do discovery for webmention endpoint, if none found, do discovery for pingback endpoint. send webmention if an endpoint was found, otherwise send pingback if an endpoint was found.
  • Bear on bear.im as of 2013-12-01.
    • All posts send and accept webmentions
    • Discovery with both: rel="webmention" and rel="http://webmention.org/" links in head, and HTTP Link: rel="http://webmention.org/ header
    • Incoming webmentions are validated and stored, but not yet added to the appropriate articles
  • Kartik Prabhu on kartikprabhu.com as of 2013-12-03.
    • All articles & notes accept webmentions
    • Articles have an input form for sending a manual webmention inspired by adactio.com. The form prompt links to the webmention endpoint.
    • Webmentions are saved as responses and appear below the article classified according to response type.
    • Webmentions are sent manually for all posts.
  • Jeena Paradies using a home grown Rails app on https://jeena.net as of 2013-12-06. Details:
    • All homepage, blog posts and notes, photos, comments and events accept webmentions
    • Discovery with: rel="webmention" Link: in HTTP header
    • Sends webmentions on create, update and delete, no salmentions yet
    • Has a manual form
  • gRegor Morrill sending and receiving webmentions on gRegorLove.com as of 2014-02-04.
    • Using a custom plugin I developed for Nucleus CMS
    • All articles can receive webmentions
    • Webmention endpoint is broadcast via Link: HTTP header and <link> element
    • Incoming webmentions validate the target is a valid URL on gregorlove.com and the post ID in the URL is valid.
    • Webmentions are parsed and further validated asynchronously. Replies/mentions are displayed interleaved with "local" blog comments..
    • More implementation notes
  • Glenn Jones using node.js module github.com/glennjones/webmentions on glennjones.net as of 2014-02-19.. Details:
    • All notes accept webmentions
    • Discovery only with: rel="webmention" link in head
    • The notes editor has an input for the in-reply-to URL
    • The webmentions module allows pinging of webmention between any sites
    • Yet to implement: full authorship parsing, updates, deletes.
  • Ben Roberts sending and receiving webmentions on ben.thatmustbe.me as of 2014-04-24
    • Parsing replies, mentions, and likes. Rsvps are handled as just mentions currently.
    • Discovery with both: rel="webmention" and rel="http://webmention.org/" links in head, and HTTP Link: rel="http://webmention.org/ header
    • Currently uses vouch and a whitelist while all others are sent to moderation.
    • Processing happens asynchronously by cron job.
    • Submissions return a link for queue record that returns a status as a Webmention-Status header as well as a minor note in the body.
    • Attempting to keep implementation notes as a modified spec at webmetion0.2-b1
    • Sending is done automatically when post is made.
  • Will Norris using webmention.herokuapp.com on willnorris.com as of 2014-07-31. Details:
    • All posts accept webmentions, but none are displayed (yet)
    • Webmentions sent on-demand via command line application that handles the discovery bits
    • Discovery only with rel="webmention" link in head
    • Advertised webmention endpoint is on-site, but POSTs are proxied to webmention.herokuapp.com. (documented here)
    • Previously used webmention plugin for WordPress as of 2013-06-23
  • Joel Dueck receiving webmentions on thelocalyarn.com as of 2014-09-23.
    • All posts accept webmentions as well as traditional comments.
    • Webmentions (if valid) are logged to a text file and sent to Joel via email. Those that pass moderation are manually added to the target post as a comment (which on this site means it is included in the blog and sent to email and RSS subscribers as well as being appended to the target post).
    • Endpoint is a modified version of Jeremy Keith's webmention.php.
    • Webmentions sent manully. More to implement. To-do list is long. Site is running Textpattern CMS, so a complete webmentions plugin is both feasible and ideal.
  • Dan Lyke receiving webmentions on Flutterby.com blog entries as of 2014-10-10.
    • Webmention endpoint is broadcast via <link> element and <indie-action> element.
    • Currently only displays links on title text back to source URLs
  • Johnny Oskarsson receiving webmentions on joskar.se as of 2014-10-23.
    • Webmention endpoint is broadcast via Link: HTTP header only.
    • Pages marked up with h-entry in-reply-to show up as comments, others won't show up at all. (but they will be logged)
    • Webmentions are sent manually for now.
  • Stuart Langridge sending and receiving webmentions on kryogenix.org as of 2014-11-29.
    • Write up linking to various bits of code for Pelican, the Python static site generator, at "Enabling Webmentions".
    • Webmention endpoint is declared in the HTML with a <link> element, because HTTP headers are a pain in the behind to set up by comparison.
    • Uses Pelle Wessman's webmention Heroku endpoint as endpoint, and then some Python to fetch and bake in webmentions on site regeneration, and some JS to pull them in live.
  • Giovanni T. Parra receiving webmentions on fiatjaf.alhur.es as of 2015-02-28.
    • Wrote Jekmentions, a small service that works as a webmention endpoint for Jekyll sites hosted on GitHub that sends the webmentions received as files in commits to the GitHub repository.
    • Webmentions are sent manually for now. Some are being sent, but only through brid.gy to twitter.
  • Amy Guy sending webmentions automatically for all links on all posts on rhiaro.co.uk as of 2015-05-?.

Libraries

Sending

For webmention endpoint discovery:

  • link_rel_parser : http_rels($h) & head_http_rels($url) - HTTP header string parser in PHP for RFC5988 Link: rels (including X-Pingback) & function to curl a HEAD request and parse it all in one.
  • phpish/link_header - Link header (RFC 5988) parser in PHP
  • PEAR: HTTP2 (documentation) - Link header (RFC 5988) parser in PHP
  • ronkyuu - Python client library and command-line tools

Handling

Handling receiving webmentions:

For parsing the mentions, the following may come in handy:

Notifications

The following tools show how a notification could be sent upon receiving a webmention

Publishing Software

Some open source publishing software supports webmentions.

  • Known sends webmentions and accepts webmention comments
  • p3k sends webmentions for all posts and accepts webmention comments on events, notes, replies, RSVPs
  • phorkie sends and accepts webmentions to notify remote instances about forks
  • Taproot

Plugins exist for some open source publishing software:

Services

Some services one can use to receive or send webmentions.

webmention.herokuapp.com

webmention.herokuapp.com receives webmentions for any registered page and allows them to be embedded through javascript.

Bridgy

Main article: Bridgy

brid.gy is a service that sends webmentions for comments/replies, likes, and reposts on Facebook, Twitter, Google+, and Instagram posts. It uses original post discovery to find target links for the webmentions. GitHub repo here.

Bridgy Publish also supports webmention as a mechanism to trigger POSSEing to Facebook, Twitter, and Instagram.

stapibas

stapibas is a self-hosted service to send and receive webmentions for websites and blogs.

It can be used to send out webmentions and pingbacks for new posts on static sites.

webmention.io

Main article: webmention.io

webmention.io is an open-source project and hosted service for receiving webmentions and pingbacks on behalf of your indieweb site.

jekmentions

Main article: Jekmentions

jekmentions is a service that works as webmention endpoint and saves the received webmentions in a GitHub repository, so that they can be used and shown in a Jekyll blog.

checkmention

https://checkmention.appspot.com/ lets you test your webmention implementation on your indieweb site, and whether it robustly detects certain types of XSS attacks. It also allows you to test for authorship spoofing.

node-webmention-testpinger

node-webmention-testpinger is a tool to ping your site with a variety of webmentions markup. Contains copies of a couple of real world examples of mentions that it enables you to ping locally to a development copy of your site.

node-webmention-testendpoint

node-webmention-testendpoint is tool to test your webmentions client. Generates a demo-post and a demo-endpoint to test if your client parses the webmention-endpoint correctly and to check if the ping body is transmitted correctly.

Tools

Tools you can install to send webmentions

Firefox Addon

Wish List

  • A tiny, no-dependencies js file which can fetch indiecomment data from webmention.io and dump it into an element, enabling 2-step indiecomment support:
    • Add a <link> to webmention.io
    • include this script and add some <div data-indiecomments data-url="blah"> element where you want them to appear

FAQ

See Webmention FAQ. Please read this before filing an issue or adding to brainstorming.

Issues

See Webmention Issues.

Brainstorming

See Webmention Brainstorming.

Articles

Blog posts and articles about webmention:

See Also

Retrieved from "https://indieweb.org/wiki/index.php?title=Webmention&oldid=23247"