Twitter Introduces Annotations; Hash Tags become Obsolete

One of the new developments announced at the recent Chirp Twitter developers’ conference was ‘Annotations’ — a way of adding additional metadata to your tweets. Annotations provide a simple way to take those tiny extra bits of data that usually take up some of your 140 characters of tweet space, and put them in a logical, organized and search enabled place for developers to access. Twitter has very specifically not defined exactly what annotations are for or how they should be used to encourage innovation in their use, but provided a few examples in their announcement. Their main example centered around a tweet about a book, and the additional information an Annotation could provide, such as the isbn number and an Amazon.com link.

'annotations':
{
  'iso':
  {
    'isbn': '030759243X'
  },
  'amazon':
  {
    'url': 'http://www.amazon.com/Although-Course-You-Becoming-Yourself/dp/030759...
  }
}

How Will This Work?

The additional data is in a namespace/key/value setup, the example above containing an ISO namespace with an isbn key and value, and will be returned from the Twitter API along with the tweet. Annotations must be provided when creating a new tweet, and cannot be added or altered at a later stage. Twitter is not defining how developers should use the new data, so it will be up to them to create the rules and structure in their apps to create and view annotations, although I fear this could lead to problems (read below). If you’ve worked with the Twitter API before, accessing this new feature will not be difficult as it will be provided along with the main tweet payload in whatever format you have requested. At first annotations will be limited to 512 bytes, but Twitter has admitted they’re flexible on this and it could end up being as large as 2Kb. This is huge considering that 512 bytes could contain up to 512 extra characters of information; which is three times the limit for the actual tweet.

The end user experience will eventually be seamless, as developers build these new features into their apps in a transparent way. Expect to see a different field for your urls, hash tags and other information — and have them displayed back to you next your tweet.

What Can This Be Used For?

The possibilities for annotations are really huge, with the removal of the commonly used hash tags one of the first implementations I expect to see. If you’re not familiar with hashtags, they are values added to the end of a tweet in the form of #topic used to provide a way for people to search Twitter for tweets on a similar topic, an informal way of categorizing Twitter; for example recent tweets have used the ironic hash tag #ashtag regarding the huge ash cloud cancelling flights all over Europe at the moment, e.g. “British govt: British airspace to stay closed at least another 24 hrs #ashtag”.

The problem with hashtags has always been that they eat into your already limited 140 character allotment, and it’s not uncommon for hashtags to start to take over a tweet! By providing your hashtags as part of the annotation for a tweet, it frees it up for what matters: the content.

Other uses I could think of for these annotations

Links: The large size of urls has created a plethora of url shortening services, designed to create the smallest character url to fit into your 140 character tweets. Annotations can now be used to attach these links and keep them out of the content. Links could also be categorized depending on the content of the link destination.

'annotations':
{
  'links':
  {
    'web': 'http://www.xyz.com',
    'image' : 'http://www.abc.com',
    'video' : 'http://www.foo.com/'
  }
}

Extended Tweets: With a limit of 140 characters for the tweet and 512 for the annotation I wouldn’t be surprised to see the extra data used as a way to extend the information of your tweet, kind of like what ‘Read More’ is to an article, making the tweet itself just an excerpt of the total content.

'annotations':
{
  'extended':
  {
    'continued': 'and that is why I will never arm wrestle a goat again.'
  }
}

Languages: Imagine being able to produce a tweet in multiple languages! I think this is one of the more exciting uses of the new Annotations feature, as you can provide multiple versions of a tweet based on contextual information, for example; language. Imagine seeing a tweet with various flags underneath it and being able to change the language simply by clicking on a flag.

'annotations':
{
  'languages':
  {
    'en' : 'Has anyone seen my crepe maker? I last saw it at Easter.'
    'fr': 'N'importe qui a vu mon fabricant de crêpe ? Je dernier l'a vu aux Pâques. ',
    'de' : "Hat jemand meinen crepe Hersteller gesehen? Ich habe zuletzt es an Ostern gesehen. "
  }
}

Code: For the geekier types, you could even try to fit in an entire program or function.

'annotations':
{
  'code':
  {
    'javascript': 'function(){var d=0;setInterval(function() {document.body.style['-webkit-transform']= 'rotate('+ d +'deg)';d+=1},10)};
  }
}

Other examples that I have seen floating around include the weather, your location, currently listening music / watching tv/ film and your Facebook / social media profile information.

Problems I Can See Arising

I fear that, while incredibly empowering, this new feature could lead Twitter and Twitter apps into an era akin to the Wild West. There is no formal structure, or presiding body, over the format of these new annotations and app developers are going to have to be very careful not to work against each other. Having one app implement a way of adding links to a tweet, only to have another app do it in an entirely different way will not benefit anyone as the additional information will only be available to an app with that reads it correctly.

The end user shouldn’t have to be responsible for knowing about the ‘compatibility’ of their metadata. If I’m typing a tweet and I annotate in some information about a book; I shouldn’t have to know if every user is going to actually get that additional information in their chosen Twitter reader; and this is even worse if you’re referring to, or relying on, information in the metadata in your tweet. A Tweet that says “Just finished this book, check it out” and assumes that a client has the capability to read the annotation with book information will lead to a large amount of confusion in clients that don’t support that format.

I fear that by essentially creating a system for developers to create new features, Twitter is going to be making the lives of other developers increasingly difficult as they have to build feature support to their applications as they become widely adopted; and make the lives of consumers difficult as they have to understand that some users may not have access to these new features that they want to use.

Call me old fashioned, but…

The thing I love about Twitter is its simple eloquence, the way that all you have is 140 characters to get your message across. The intricacies of retweeting and hash tags were fun because there was no formal structure for it, you only had to add in some text here, or there, and voila; people will know what you mean! Squeezing the last character out of a tweet often means culling off other information, which ultimately leads to a tweet that is as straight to the point and relevant as possible. Twitter has already formalized the way we retweet, but I don’t use it; preferring instead to simply add my ‘RT @username’ in front of a tweet; and I think that I will still continue to use hashtags as the world moves to annotated categories.

In conclusion

Twitter has created a very extensible way for developers to create new features in Twitter and innovate in ways that have not been possible before. What do you think about the new annotation feature, what would you use it for?

Read the entire announcement at the Twitter Developer Google Group.

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • R_macdonald

    Good post Mal, you are on top of a remarkably extensible chance for substantive innovation on the Twitter platform.

    I think there are amazing opportunities in Twitter’s new annotation feature.

    I am awestruck by the potential of developers applying Linked Data / Semantic Web schema and using RDF in Twitter annotations. It would massively scale search effectiveness and distribution opportunities while allowing sophisticated on-the-fly analysis of Twitter’s firehose & other feeds.

    The Twitter ecosystem is particularly suited to applying principles of the Semantic Web in that machine interpolated meaning could be continually refined the more humans tweet & retweet about the same and related topics (resolved by semantic entity detection; coupled with author, location, hashes, platform, temporal and other information.

    The vision of Semantic Web is occasionally dismissed as long on dream and short on practice. However, early implementations are gaining significant traction and offering substantive value to early adopters. Thompson Reuters, no slouch in the business of information, was quite visionary in acquiring the now @OpenCalais service. The mostly free service is currently identifying semantic entities in over 5 million documents submitted to it per day. @zemanta is also finding a significant user base among bloggers for its related services.

    Twitter annotations may well be a turning point in the early practical application of Tim Berners-Lee’s 1999 “dream for the Web [in which computers] become capable of analyzing all the data on the Web – the content, links, and transactions between people and computers.”

    It is all about the metadata!

    PS For those those considering Twitter Annotations: Map of a Twitter status object from @raffi – tech lead of @twitterapi http://post.ly/bQYz

  • kuyratowa

    I for one am leery of the concept of annotated program being attached to a Tweet – that seems to be a hacker or spam developer’s dream addition to the platform.

  • KevinCole509

    I hate the sound of what I’m about to say, but, though I’ve never feared a learning curve on new things, I’m getting a little weary of being expected to continually relearn things I already work well with.

    I don’t think “hash tags cutting too far into the 140″ was a significant problem – in fact it was more of a nice little feedback mechanism that put some nice, relatively organic limits on some things.

  • http://www.fishoutoforder.net fishoutoforder

    I would love to be able to see these used to pull out even more interesting data from the tweet stream. Running a search to show [books] in the [whodunnit] genre that the twit [recommend]s. We could maybe even count how many people have tweeted about a particular book over a period of time.
    Yes, this could be used for evil, but IT IS JUST SO COOL. And my keyboard can be used for evil too, but mostly I just store small coffee spillages in it.
    Good Reads and similar sites could start sending more detailed data to Twitter, and perhaps individual users would have the options to filter out certain kinds of tweets from their streams via metadata.

    You could opt out of all book recommendation tweets if reading isn’t your thing. Or opt out of any tweeted events that are tagged with location metadata that does not match the range you are interested in. Or opt out of tweets being sent through particular websites.
    This opens up so many possibilities for future growth it is just mind boggling. The idea of having separate [image link] + [URL] + [event] metadata from my 140 characters is exciting, but it is what external websites do with this (and how the Twitter team redesign the web interface to make this metadata easy to view and manage) is of most interest to me.
    Wow. Just wow. Amazing work, Twitter dev team!

  • Mal Curtis

    @fishoutoforder “small coffee spillages”. Classic.

  • Greg

    Love the “translation” usage you are predicting. Also, using the metadata to make tweets look more like what HTML anchor tags do would be great. “You should read Smith’s latest book, The Winnings” with Smith a clickable link to his bio, and The Winnings a clickable link to a book review, or its listing on Amazon, or whatever.
    The message stays the same, but the information is presented more efficiently.
    However, as you say, it’s likely to be used in such a way that the original “purity” of the 140-character tweet is eventually lost. Which WOULD be a shame.

  • Anthony Mitchell

    Yet another role for CADNA/NAF, as SEO juice can be generated by the use of competitors’ brands in annotations. How will C&Ds work? They probably won’t.

  • Marat

    Gosh, if some people with malicious intents can input a virus into annotations, it’s gonna be a big security issue!

  • Andreas Habicher

    The beauty of twitter in the first era was its sheer simplicity. Just as you said: RT, @, #, and the creative uses one could put these signs to. The new Retweet-Function is already a step towards a more complicated twitter where the reason why some blocked users words pop up in the timeline is only recognizable on second glance.

    Call me old fashioned, either, but I much prefer a text-only-tweet to any kind of fancy add-on info, let alone add-on info that takes up more space than the tweet itself.

  • http://www.tyssendesign.com.au Tyssen

    Gosh, if some people with malicious intents can input a virus into annotations, it’s gonna be a big security issue!

    That’s what I’m thinking too. Some of the other examples you’ve outlined do sound very cool though.

  • Zackatoustra.com

    Metadata are already amongst us, guys!

    Realize that “pieces of information” like “id of the tweet, location, date, this tweet is an answer to that other tweet”,etc.. are already meta data about any tweet, even the simplest.
    I think a point Mal has made is that there is only ONE way these annotations will keep Twitter “Universal” : Twitter’s people have to explicitly specify them, and not let the developpers of third-party apps do it, otherwise mess assured.

    I only hope I won’t ever need a “Twitter editor” to integrate metadata into my tweets…
    Hyper Text Twitter Markup Language, anyone?

  • bobbykjack

    I seem to be one of the few that thinks this is actually a really good idea. Let’s not forget that twitter essentially spawned a whole number of ‘URL shortening’ services that have many drawbacks (security, longevity) and are really completely unnecessary. Having an additional ‘URL’ field is a godsend.

    Yes, there’s the potential of the whole namespace problem, but that will always be a problem when trying to label data semantically – until standards emerge. Hopefully, ‘url’ will be the standard name for a link – although I’m not quite sure if there’s a global/default namespace, or if one will have to be agreed upon.

    And to those worrying about security: it’s probably not a good idea to blindly eval() the contents of ANY user input, whether it’s twitter annotations or anything else ;-)

  • the14thdisciple

    … All this effort, when the simple approach would be to just make the tweet-space bigger than 140 characters …

  • Mal Curtis

    @bobbykjack The security issue could be as simple as someone on a website using Javascript and the Twitter api. Imagine a meta refresh placed in the URL space? An incorrectly rendering browser *cough* ie *cough* could well end up send the user off. Imagine if people were just doing this in JS mydiv.innerHTML = '<a href="' + tweet.annotation.url.web + '">A Link</a>';.

    If your annotation payload had a web url of the example below then you’re up sh*t creek! This isn’t twitter client developers I’m talking about here, this is everyday web devs (and front end peeps have no idea about injection techniques :D )!

    Example:
    "></a><meta http-equiv="refresh" content="0;url=http://www.hackedupsite.com" /><a href="

    NOTE: That works! I forgot to escape this comment and ended up at http://www.hackupsite.com :D

    @Zackatoustra.com agreed, that's almost exactly my point. Twitter should at least be defining the annotation features that EVERY client will be implementing.

    @the14thdisciple if you give people a larger tweet space, they'll simply use it for more tweet content, which isn't necessarily a good thing and would definitely not resolve the issues we have. The great thing about annotations is that we're separating the metadata from the content, and since it's in a defined format it's far more searchable / usable.

  • http://www.patricksamphire.com/ PatrickSamphire

    Twitter’s success is based on its simplicity. 140 characters, that’s it. Anyone can use it, and it’s very, very easy and quick. The more that gets added, the less quick and easy it becomes. I think this is a poor move, and I will certainly be sticking with the 140 characters.

  • R_macdonald

    ReadWriteWebs’s Co-Editor, Marshall Kirkpatrick @ marshallk, suggested 4/19/01 that Twitter intends to leave the annotation classification system to be determined by the market. http://bit.ly/csK8Od

    Although I appreciate that Twitter values keeping the annotation ecosystem open for innovation and adaptation, I hope the conversation on Linked Data metadata standards within Twitter annotations is just beginning.

    It could be an historic lost opportunity if the hard driving Twitter team doesn’t step back and consider soliciting the counsel of the W3C, Sir TB-L, Nigel Shadbolt and other thoughtful public interest-oriented folks in the Linked Data community. After all, Metaweb’s Freebase team is just 3 blocks away. From Twitter’s San Francisco headquarters.

    I suspect the conversation on Linked Data metadata standards within Twitter annotations is just beginning.

    It is all about the metadata.

  • bobbykjack

    @Mal: I recognise the problem, but I don’t see how it applies specifically to annotations, as opposed to the content of tweets. If a naive app is currently injecting the content of tweets into its markup, without sanitising it first, that’s just as much of a problem.

  • Mal Curtis

    @bobbykjack I guess you’re right on that one, but I do think that with the more complex payloads we’re likely to see, there’s a higher risk; people not sanitising as stringently because they’re expecting a certain amount of code or variants that aren’t necessarily human readable.

  • http://www.virvo.com Web-Development

    i dont think this makes hashtags obsolete at all – as suggested in the title. This is API level only – its great to have meta data – but i think its an addition to hashtags, not a replacement.