Abstract File Systems with Flysystem

Reading and writing files is an integral aspect of any programming language, but the underlying implementation can vary enormously. For example, the finer details of writing data to the local filesystem compared to uploading over FTP is very different – yet conceptually, it’s very similar.

In addition to old warhorses like FTP, online storage is increasingly ubiquitous and inexpensive – with scores of services available such as Dropbox, Amazon’s S3 and Rackspace’s Cloud Files – but these all use subtly different methods for reading and writing.

That’s where flysystem comes in. It provides a layer of abstraction over multiple file systems, which means you don’t need to worry where the files are, how they’re stored, or need be concerned with low-level I/O operations. All you need to worry about are the high-level operations such as reading, writing and organizing into directories.

Such abstraction also makes it simpler to switch from one system to another without having to rewrite vast swathes of application code. It also provides a logical approach to moving or copying data from one storage system to another, without worrying about the underlying implementation.

You can use Dropbox, S3, Cloud Files, FTP or SFTP as if they were local; saving a file becomes the same process whether it’s being saved locally or transferred over the network. You can treat zip archives as if they were a bunch of folders, without worrying about the nitty gritty of creating and compressing the archives themselves.

Installation and Basic Usage

As ever, Composer is the best way to install:

"league/flysystem": "0.2.*"

Now you can simply create one or more instances of League\Flysystem\Filesystem, by passing in the appropriate adapter.

For example, to use a local directory:

use League\Flysystem\Filesystem;
use League\Flysystem\Adapter\Local as Adapter;

$filesystem = new Filesystem(new Adapter('/path/to/directory'));

To use an Amazon S3 bucket, there’s slightly more configuration involved:

use Aws\S3\S3Client;
use League\Flysystem\Adapter\AwsS3 as Adapter;

$client = S3Client::factory(array(
    'key'    => '[your key]',
    'secret' => '[your secret]',
));

$filesystem = new Filesystem(new Adapter($client, 'bucket-name', 'optional-prefix'));

To use Dropbox:

use Dropbox\Client;
use League\Flysystem\Adapter\Dropbox as Adapter;

$client = new Client($token, $appName);
$filesystem = new Filesystem(new Adapter($client, 'optional/path/prefix'));

(To get the token and application name, create an application using Dropbox’s App Console.)

Here’s an example for SFTP – you may not need every option listed here:

use League\Flysystem\Adapter\Sftp as Adapter;

$filesystem = new Filesystem(new Adapter(array(
    'host' => 'example.com',
    'port' => 21,
'username' => 'username',
'password' => 'password',
'privateKey' => 'path/to/or/contents/of/privatekey',
'root' => '/path/to/root',
'timeout' => 10,
)));

For other adapters such as normal FTP, Predis or WebDAV, refer to the documentation.

Reading and Writing to a File System

As far as your application code is concerned, you simply need to replace calls such as file_exists(), fopen() / fclose(), fread / fwrite and mkdir() with their flysystem equivalents.

For example, take the following legacy code, which copies a local file to an S3 bucket:

    $filename = "/usr/local/something.txt";
    if (file_exists($filename)) {
        $handle = fopen($filename, "r");
        $contents = fread($handle, filesize($filename));
        fclose($handle);

        $aws = Aws::factory('/path/to/config.php');
        $s3 = $aws->get('s3');

        $s3->putObject(array(
            'Bucket' => 'my-bucket',
            'Key'    => 'data.txt',
            'Body'   => $content,
            'ACL'    => 'public-read',
        )); 
    }

Using flysystem, it might look a little like this:

    $local = new Filesystem(new Adapter('/usr/local/'));
    $remote = new Filesystem(
        S3Client::factory(array(
            'key'    => '[your key]',
            'secret' => '[your secret]',
        )),
        'my-bucket'
    );

    if ($local->has('something.txt')) {
        $contents = $local->read('something.txt');
        $remote->write('data.txt', $contents, ['visibility' : 'public']);
    }

Notice how we’re using terminology such as reading and writing, local and remote – high level abstractions, with no worrying about things like creating and destroying file handles.

Here’s a summary of the most important methods on the League\Flysystem\Filesystem class:

Method Example
Reading $filesystem->read('filename.txt')
Writing $filesystem->write('filename.txt', $contents)
Updating $filesystem->update('filename.txt')
Writing or updating $filesystem->put('filename.txt')
Checking existence $filesystem->has('filename.txt')
Deleting $filesystem->delete('filename.txt')
Renaming $filesystem->rename('old.txt', 'new.txt')
Reading Files $filesystem->read('filename.txt')
Getting file info $filesystem->getMimetype('filename.txt')
$filesystem->getSize('filename.txt')
$filesystem->getTimestamp('filename.txt')
Creating directories $filesystem->createDir('path/to/directory')
Deleting directories $filesystem->deleteDir('path/to/directory')

Automatically Creating Directories

When you call $filesystem->write(), it ensures that the directory you’re trying to write to exists – if it doesn’t, it creates it for you recursively.

So this…

$filesystem->write('/path/to/a/directory/somewhere/somefile.txt', $contents);

…is basically the equivalent of:

$path = '/path/to/a/directory/somewhere/somefile.txt';
if (!file_exists(dirname($path))) {
    mkdir(dirname($path), 0755, true);
}
file_put_contents($path, $contents);

Managing Visibility

Visibility – i.e., permissions – can vary in implementation or semantics across different storage mechanisms, but flysystem abstracts them as either “private” or “public”. So, you don’t need to worry about specifics of chmod, S3 ACLs, or whatever terminology a particular mechanism happens to use.

You can either set the visibility when calling write():

$filesystem->write('filename.txt', $data, ['visibility' : 'private']);

Prior to 5.4, or according to preference:

$filesystem->write('filename.txt', $data, array('visibility' => 'private'));

Alternatively, you can set the visibility of an existing object using setVisibility:

$filesystem->setVisibility('filename.txt', 'private');

Rather than set it on a file-by-file basis, you can set a default visibility for a given instance in its constructor:

$filesystem = new League\Flysystem\Filesystem($adapter, $cache, [
'visibility' => AdapterInterface::VISIBILITY_PRIVATE
]);

You can also use getVisibility to determine a file’s permissions:

if ($filesystem->getVisibility === 'private') {
    // file is secret
}

Listing Files and Directories

You can get a listing of all files and directories in a given directory like this:

$filesystem->listContents();

The output would look a little like this;

[0] =>
  array(8) {
  'dirname' =>
    string(0) ""
    'basename' =>
    string(11) "filters.php"
    'extension' =>
    string(3) "php"
    'filename' =>
    string(7) "filters"
    'path' =>
    string(11) "filters.php"
    'type' =>
    string(4) "file"
    'timestamp' =>
    int(1393246029)
    'size' =>
    int(2089)
}
[1] =>
array(5) {
    'dirname' =>
    string(0) ""
    'basename' =>
    string(4) "lang"
    'filename' =>
    string(4) "lang"
    'path' =>
    string(4) "lang"
    'type' =>
    string(3) "dir"
}

If you wish to incorporate additional properties into the returned array, you can use listWith():

$flysystem->listWith(['mimetype', 'size', 'timestamp']);

To get recursive directory listings, the second parameter should be set to TRUE:

$filesystem->listContents(null, true);

The null simply means we start at the root directory.

To get just the paths:

$filesystem->listPaths();

Mounting Filesystems

Mounting filesystems is a concept traditionally used in operating systems, but which can also be applied to your application. Essentially it’s like creating references – shortcuts, in a sense – to filesystems, using some sort of identifier.

Flysystem provides the Mount Manager for this. You can pass one or more adapters upon instantiation, using strings as keys:

$ftp = new League\Flysystem\Filesystem($ftpAdapter);
$s3 = new League\Flysystem\Filesystem($s3Adapter);

$manager = new League\Flysystem\MountManager(array(
    'ftp' => $ftp,
    's3' => $s3,
));

You can also mount a file system at a later time:

$local = new League\Flysystem\Filesystem($localAdapter);
$manager->mountFilesystem('local', $local);

Now you can use the identifiers as if they were protocols in URI’s:

// Get the contents of a local file…
$contents = $manager->read('local://path/to/file.txt');

// …and upload to S3
$manager->write('s3://path/goes/here/filename.txt', $contents);

It’s perhaps more useful to use identifiers which are generic in nature, e.g.:

$s3 = new League\Flysystem\Filesystem($s3Adapter);
$manager = new League\Flysystem\MountManager(array(
    'remote' => new League\Flysystem\Filesystem($s3Adapter),
));

// Save some data to remote storage
$manager->write('remote://path/to/filename', $data);

Caching

Flysystem also supports a number of technologies for caching file metadata. By default it uses an in-memory cache, but you can also use Redis or Memcached.

Here’s an example of using Memcached; the cache adapter is passed to the Filesystem constructor as an optional second parameter:

use League\Flysystem\Adapter\Local as Adapter;
use League\Flysystem\Cache\Memcached as Cache;

$memcached = new Memcached;
$memcached->addServer('localhost', 11211);
$filesystem = new Filesystem(new Adapter(__DIR__.'/path/to/root'), new Cache($memcached, 'storageKey', 300));
// Storage Key and expire time are optional

As with the filesystem adpaters, if you wished you could create your own – simply extend League\Flysystem\Cache\AbstractCache.

Summary

In this article I’ve introduced flysystem, a package which provides a layer of abstraction over a variety of forms of file storage. Although I haven’t covered every available adapter, the documentation should help fill in the gaps, introduce a couple of other features and document any new adapters in the future.

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • twistacz

    Thanks for an article. Maybe it wold be better, if sources are highlighted :)

    • http://www.bitfalls.com/ Bruno Skvorc

      Yep, they will be again soon enough.

  • http://nortaox.com/ Francisco Luz

    This is cool.

  • Xandi Ostermeyr

    great article – thank you.

  • http://anorgan.com/ anorgan
    • http://www.bitfalls.com/ Bruno Skvorc

      Cheers for the heads up

  • Frank de Jonge

    @brunoskvorc:disqus: Redis and Memcached are used not to act as a filesystem but as a meta-data caching layer preventing the application to hit the (more expensive) filesystem when retrieving file information. Other than that, this is a really nice write-up! Thanks a lot for the support!

    • http://www.bitfalls.com/ Bruno Skvorc

      They are still used in the same way, no? Or would you like to introduce some changes to the article? I’m open to suggestions for improvement if you have any!

      • Frank de Jonge

        They’re the second constructor argument for a Filesystem instance. So they’re used *with* a filesystem, not instead of. This suggests you can use redis as a filesystem which is totally possible but doesn’t come with the default adapters. I’ve already heard people creating adapters for Flickr and Reddit (which is freaky but cool).

        • http://www.bitfalls.com/ Bruno Skvorc

          Thanks, some updates were made to the article!

  • http://www.orion-techs.com Oluwafemi William Babalola

    very useful :) thanks