BigchainDB: Blockchain and Data Storage

Since I wrote this post, a lot has changed for BigchainDB and in the blockchain space broadly, so it seemed high time for a revisit and refresh of what effect blockchain can have on one of the more fundamental parts of the traditional computing space: data storage.

Originally built as a technology to replace the Bitcoin blockchain in the Ascribe digital artwork tracking project, BigchainDB expanded into a component of the abandoned IPDB and now as a storage layer for the bold Ocean protocol.

This change in use caused changes to the underpinnings and implementation of BigchainDB, as did the closure of RethinkDB, forcing the team to switch the storage engine to the stalwart MongoDB. The blockchain layer on top of the database that provides the transactional support that helps guarantee a database change has taken place and adds additional control and security remains, but has gained maturity, with BigchainDB set to hit 2.0 in 2018.

All these changes now mean that BigchainDB encourages you to use their public network instead of deploying your instances. This approach is somewhat counter to traditional distributed database practice, but is more in-fitting with the evolution of Blockchain-based projects over the past few years, helps BigchainDB monetize their platform (with an ICO or SaaS model) and is an interesting change. Time will tell if customers are comfortable with storing data in a public network, but with access tokens ensuring security and privacy, it’s conceptually not too dissimilar from using a cloud-hosted database.

If you want, you can still install your own BigchainDB instances, but I feel the company will increasingly encourage you not to.

Whichever option you take, you can then use the official Python, JavaScript or community drivers. For example, with JavaScript, install the package with npm install bigchaindb-driver, create a connection and write and read assets to the database using appropriate keys for the writer and reader.

You can read full documentation for the driver here and the database here, but the example below creates an article asset for author and then assigns it to assignee:

const driver = require('bigchaindb-driver')
const author = new driver.Ed25519Keypair()
const assignee = new driver.Ed25519Keypair()

console.log('Author: ', author.publicKey)
console.log('Assignee: ', assignee.publicKey)

const assetdata = {
    'article': {
        'title': 'Blockchain DBs',
        'body': 'Article body',
    }
}

const txCreateAuthorSimple = driver.Transaction.makeCreateTransaction(
    assetdata,
    [driver.Transaction.makeOutput(
        driver.Transaction.makeEd25519Condition(author.publicKey))
    ],
    author.publicKey
)

const txCreateAuthorSimpleSigned = driver.Transaction.signTransaction(txCreateAuthorSimple, author.privateKey)

let conn = new driver.Connection('https://test.bigchaindb.com/api/v1/', {
    app_id: '<APP_ID>',
    app_key: '<APP_KEY>'
})

conn.postTransactionCommit(txCreateAuthorSimpleSigned)
    .then(retrievedTx => console.log('Transaction', retrievedTx.id, 'successfully posted.'))

    .then(() => {
        const txTransferAssignee = driver.Transaction.makeTransferTransaction(
            [{tx: txCreateAuthorSimpleSigned, output_index: 0}],
            [driver.Transaction.makeOutput(driver.Transaction.makeEd25519Condition(assignee.publicKey))],
            {price: '100 dollars'}
        )

        let txTransferAssigneeSigned = driver.Transaction.signTransaction(txTransferAssignee, author.privateKey)
        console.log('Posting signed transaction: ', txTransferAssigneeSigned)

        return conn.postTransactionCommit(txTransferAssigneeSigned)
    })
    .then(res => {
        console.log('Response from BDB server:', res)
        return res.id
    })
    .then(tx => {
        console.log('Is Assignee the owner?', tx['outputs'][0]['public_keys'][0] == assignee.publicKey)
        console.log('Was Author the previous owner?', tx['inputs'][0]['owners_before'][0] == author.publicKey)
    })
    // Search for asset based on the serial number of the bicycle
    .then(() => conn.searchAssets('Blockchain DBs'))
    .then(assets => console.log('Found assets with title:', assets))

Other Alternatives

BigchainDB is no longer the only blockchain-based database in town, depending on your definition, and what you want to accomplish.

FlureeDB wraps a graph-style database with a blockchain layer, which — when you consider the nature of Blockchain — makes some sense. Thanks to the Graph underpinnings, it integrates well with GraphQL and React. It’s still in active development, and follows familiar funding models for databases, with a limited community version, and extra capacity, security and support for premium users. FlureeDB’s involvement with Blockchain technology seems to be using tokens as a replacement for money and some form of consensus mechanism. The project is not open source, so it’s hard to tell what’s under the hood.

From memory, OrbitDB has existed for about as long as BigchainDB, but designed for simpler application needs. While it uses IPFS for storage (which some might claim as a database of sorts), it doesn’t claim to be a “blockchain database”, but rather a choice for decentralized apps.

TiesDB makes a lot of bold claims on its site, but with little detail about how it accomplishes them, and with a sparse repository and sparser documentation on how to run the database, it’s hard to confirm if it delivers. There’s a handful of whitepapers in the repository readme that you can sift through, but they still mostly cover theory rather than practice. Intriguingly, it also allows you to delete data, which, while a fundamental part of traditional databases, somewhat contradicts blockchain ideals. There’s no wrong or right in this decision; some developers have to consider compromises to push blockchain technologies into the mainstream.

Swarm is an Ethereum component that’s the default storage mechanism for distributed apps (Dapps). It doesn’t offer such a seamless way to get started, but if you’re already investigating Ethereum for its other components, then read the documentation for more details.

Filecoin does something different. It offers a mechanism for tracking transactions between blocks of spare storage around data centers and the Internet. It allows you to use traditional storage, but via a blockchain layer that lets users bid for space you offer and tracks their usage of it.

Both of these technologies are described in more detail in this post.

Part of a Decentralized Future

Ignoring its Blockchain heritage for a moment, BigchainDB supplies functionality missing from current NoSQL and distributed databases. That fact alone may be a reason to try it and may provide a valid business/use case.

For the blockchain aficionados among you, BigchainDB and other alternatives also complete the puzzle for a complete decentralized application stack — with Ethereum for applications, IPFS as a file system and BigchainDB for data storage. The pieces are in place for a different way of developing, deploying and maintaining applications, leading to a fascinating future that I would love to hear your opinions on.