Bittorrent’s Maelstrom: Using Torrents to Host Websites
In December of 2014, Bittorrent announced Project Maelstrom – a way to power the internet by means of torrents. A truly decentralized internet in which the websites you view are hosted by other users of the internet – not a central server.
Other portals have already reposted their most recent announcement – a public beta for Windows users, so I won’t dwell on that. Rather, I’d like to focus on the technical side – actually using the browser, learning how it works, and developing for it. Note that a Mac beta is coming soon.
How does it work?
When using torrents to share files, you get file fragments from various other machines who already have those fragments (we call these machines seeders). The torrent client (software like Deluge, Transmission, or uTorrent, etc) knows where to find these fragments with the help of a special
.torrent file, and pulls in data from all those various locations by comparing hash signatures. What’s more, it compares the various fragments coming from one source to their counterparts coming from another source, so you know you’re always getting the file you wanted. Additionally, many torrents include a hash string via which you can verify their integrity manually, if you so choose.
With Maelstrom, which is nothing but a heavily customized fork of Chromium (so it works with regular websites, too), you browse the Maelstrom-specific internet via the exact same means as torrenting. To host a website, all you need is a
.torrent file for your content and once people click or input its Magnet link or Bittorrent link (more on that later), the contents will be fetched from everyone who already has them. Naturally, this means the younger your content (web page) is, the slower the fetching – because at that point very few people have it. The more it spreads, the more people can be used as seeders, and the faster a new visitor will download it, thus becoming a new seeder as well.
In short, Maelstrom reads in websites from people who already downloaded them, bypassing hosting companies and central servers. It’s distributed, decentralized, free hosting – but only for static sites, for obvious reasons.
To install Maelstrom, you (currently) need to be on a Windows platform (this post will be updated accordingly once the Mac version is out) and download the app.
Note that if you have a running version of Chromium installed, it’s highly likely that the installation will fail and instead summon the installed version of Chromium when executed. I hope Bittorrent will fix their client installation soon, but in the meanwhile, you’ll need to remove Chromium before installing Maelstrom. Also note that Maelstrom, as it is right now, is a severely outdated version of Chromium (37 at the time of writing), so don’t think you can replace your main browser with it – it’s experimental tech still, at best.
Once opened, the browser greets you with a home page which is, in itself, a torrent hosted page, and contains links to other such websites (inspecting the links will reveal that they begin with the
In the past, when you wanted to torrent something, you needed to download a
.torrent file, open it in a client, and the fetching process would begin after your clients parsed the data inside it. Nowadays, it’s possible to use Magnet links. Magnet links contain all the metadata a
.torrent file usually would, avoiding the need for centralized trackers (torrent sites) to host them, like ThePirateBay. This helps these trackers avoid legal trouble, as they no longer actually host anything, and it helps the distribution effort – all you need to do is share a Magnet link with someone and they can hook up with the sharing network immediately, without downloading any additional files.
You can read more about Magnet links here.
The Maelstrom project uses Magnet links to navigate from site to site. When you click on a tile in the greeting page of the browser, you are, in fact, clicking on a magnet link which reads in the data of a
.torrent file present on Amazon’s S3 hosting.
Try and install the browser if you’re on Windows, click a link or two, see it at work. Here’s a gif of how it works for me:
You can see it’s a bit slow on first load, but all subsequent ones are faster because it’s, in essence, just a static site that you now have on your machine.
During regular torrenting, you download files into a folder from which they are then served until they are deleted and you effectively stop being a seeder. This folder takes space, of course, and can grow to immense sizes, which is why few people are long-term seeders.
When Maelstrom downloads websites, it has its own internal cache into which it saves the data and from which it seeds for others. The size of this cache can be modified in the Settings, under Torrents, and even cleared. Naturally, the more stuff there is, the slower things get, so one cannot expect to have the entire web fetched in there eventually – the storage amount is limited. This also means that:
- Bigger sites with streaming content like images, movies and music will take this space up much faster than the rest.
- Eventually, some pages will drop off the grid as they lose popularity and thus, seeders. Their caches will be cleared, the less used sites will go first, and they’ll stop being requested. This torrent-powered “internet” will obviously be highly dynamic and volatile in content.
To develop static sites for use in Maelstrom, one can follow these instructions. They’re quite specific, but that’s what a beta is for – ironing out the kinks and improving the process to make it more user friendly.
Unfortunately, Python 2.7 is required, but it can be installed easily. Then, you build a static website as usual, with all the CSS and JS bells and whistles. For our example, we’ll use this simple two-page “site”.
If you’d like to follow along, clone the torrent web tools and the simple site, each into its own folder. The site is nothing but the default HTML 5 boilerplate with the HTML and CSS slightly edited to provide a different color scheme and links.
Once this is done, we run the
generator.py script on the entire folder which contains the site. It takes over from there and in less than a second, the
.torrent file is ready in the folder where the script was run.
Note that the other arguments you can use with the generator are explained in their Github README file.
Note that you need to point the torrent to the location of the files, rather than your typical download location, so that it automatically enters seed mode.
The website will be accessible once enough people have it downloaded. To speed this process up, you can send your friends the torrent file or its Magnet URI while you’re seeding it, and tell them to select “skip hash check” when adding it into their torrent client to speed things up – that way their client won’t wait to get 4 or more peers on board to verify the site’s contents hash before downloading it. This is unsafe in already popular torrents, but necessary when you’re just starting to spread yours. Note that this is only necessary if you want your friends to help you spread your site – visitors to the actual site through Maelstrom won’t have to add any torrents into their clients or select any options. They’ll be able to access it as usual.
We can try to access the website through Maelstrom, using the link that
generator.py output for us:
bittorrent://b1d5524d5ff428ccd50449691e1b740660b6a2de?dn=simplesite. The docs say the magnet link
magnet:?dn=simplesite&xt=urn%3Abtih%3Ab1d5524d5ff428ccd50449691e1b740660b6a2de should work, too, but for me it didn’t work immediately – I had to wait until at least 4 people had become seeders of the site through uTorrent.
Again, note that Maelstrom works just like a regular client in that it also verifies the hash. Thus, it needs more than one or two seeders. To actually be able to access the site in your browser, you’ll have to spread it around somehow as noted above – ask a friend or two who have uTorrent installed to help you seed. After several people have begun seeding, the website should become publicly accessible.
Is this a fad? Maybe, but only in as much as Bitcoin is. It all depends on people’s adoption rates.
One important thing to note is that Maelstrom is very useful for net neutrality. The government fatcats are already at work on new appeals to the recent net neutrality victory, but this approach would, at least for static websites, alleviate that concern – if we’re sharing websites among each other, they can pound on servers all they want – we’re not there.
Have you tried Maelstrom? How do you feel about this project? Let us know!