Abstract File Systems with Flysystem

Lukas White
Lukas White
Share

Reading and writing files is an integral aspect of any programming language, but the underlying implementation can vary enormously. For example, the finer details of writing data to the local filesystem compared to uploading over FTP is very different – yet conceptually, it’s very similar.

In addition to old warhorses like FTP, online storage is increasingly ubiquitous and inexpensive – with scores of services available such as Dropbox, Amazon’s S3 and Rackspace’s Cloud Files – but these all use subtly different methods for reading and writing.

That’s where flysystem comes in. It provides a layer of abstraction over multiple file systems, which means you don’t need to worry where the files are, how they’re stored, or need be concerned with low-level I/O operations. All you need to worry about are the high-level operations such as reading, writing and organizing into directories.

Such abstraction also makes it simpler to switch from one system to another without having to rewrite vast swathes of application code. It also provides a logical approach to moving or copying data from one storage system to another, without worrying about the underlying implementation.

You can use Dropbox, S3, Cloud Files, FTP or SFTP as if they were local; saving a file becomes the same process whether it’s being saved locally or transferred over the network. You can treat zip archives as if they were a bunch of folders, without worrying about the nitty gritty of creating and compressing the archives themselves.

Installation and Basic Usage

As ever, Composer is the best way to install:

"league/flysystem": "0.2.*"

Now you can simply create one or more instances of League\Flysystem\Filesystem, by passing in the appropriate adapter.

For example, to use a local directory:

use League\Flysystem\Filesystem;
use League\Flysystem\Adapter\Local as Adapter;

$filesystem = new Filesystem(new Adapter('/path/to/directory'));

To use an Amazon S3 bucket, there’s slightly more configuration involved:

use Aws\S3\S3Client;
use League\Flysystem\Adapter\AwsS3 as Adapter;

$client = S3Client::factory(array(
    'key'    => '[your key]',
    'secret' => '[your secret]',
));

$filesystem = new Filesystem(new Adapter($client, 'bucket-name', 'optional-prefix'));

To use Dropbox:

use Dropbox\Client;
use League\Flysystem\Adapter\Dropbox as Adapter;

$client = new Client($token, $appName);
$filesystem = new Filesystem(new Adapter($client, 'optional/path/prefix'));

(To get the token and application name, create an application using Dropbox’s App Console.)

Here’s an example for SFTP – you may not need every option listed here:

use League\Flysystem\Adapter\Sftp as Adapter;

$filesystem = new Filesystem(new Adapter(array(
    'host' => 'example.com',
    'port' => 21,
'username' => 'username',
'password' => 'password',
'privateKey' => 'path/to/or/contents/of/privatekey',
'root' => '/path/to/root',
'timeout' => 10,
)));

For other adapters such as normal FTP, Predis or WebDAV, refer to the documentation.

Reading and Writing to a File System

As far as your application code is concerned, you simply need to replace calls such as file_exists(), fopen() / fclose(), fread / fwrite and mkdir() with their flysystem equivalents.

For example, take the following legacy code, which copies a local file to an S3 bucket:

    $filename = "/usr/local/something.txt";
    if (file_exists($filename)) {
        $handle = fopen($filename, "r");
        $contents = fread($handle, filesize($filename));
        fclose($handle);

        $aws = Aws::factory('/path/to/config.php');
        $s3 = $aws->get('s3');

        $s3->putObject(array(
            'Bucket' => 'my-bucket',
            'Key'    => 'data.txt',
            'Body'   => $content,
            'ACL'    => 'public-read',
        )); 
    }

Using flysystem, it might look a little like this:

    $local = new Filesystem(new Adapter('/usr/local/'));
    $remote = new Filesystem(
        S3Client::factory(array(
            'key'    => '[your key]',
            'secret' => '[your secret]',
        )),
        'my-bucket'
    );

    if ($local->has('something.txt')) {
        $contents = $local->read('something.txt');
        $remote->write('data.txt', $contents, ['visibility' : 'public']);
    }

Notice how we’re using terminology such as reading and writing, local and remote – high level abstractions, with no worrying about things like creating and destroying file handles.

Here’s a summary of the most important methods on the League\Flysystem\Filesystem class:

Method Example
Reading $filesystem->read('filename.txt')
Writing $filesystem->write('filename.txt', $contents)
Updating $filesystem->update('filename.txt')
Writing or updating $filesystem->put('filename.txt')
Checking existence $filesystem->has('filename.txt')
Deleting $filesystem->delete('filename.txt')
Renaming $filesystem->rename('old.txt', 'new.txt')
Reading Files $filesystem->read('filename.txt')
Getting file info $filesystem->getMimetype('filename.txt')
$filesystem->getSize('filename.txt')
$filesystem->getTimestamp('filename.txt')
Creating directories $filesystem->createDir('path/to/directory')
Deleting directories $filesystem->deleteDir('path/to/directory')

Automatically Creating Directories

When you call $filesystem->write(), it ensures that the directory you’re trying to write to exists – if it doesn’t, it creates it for you recursively.

So this…

$filesystem->write('/path/to/a/directory/somewhere/somefile.txt', $contents);

…is basically the equivalent of:

$path = '/path/to/a/directory/somewhere/somefile.txt';
if (!file_exists(dirname($path))) {
    mkdir(dirname($path), 0755, true);
}
file_put_contents($path, $contents);

Managing Visibility

Visibility – i.e., permissions – can vary in implementation or semantics across different storage mechanisms, but flysystem abstracts them as either “private” or “public”. So, you don’t need to worry about specifics of chmod, S3 ACLs, or whatever terminology a particular mechanism happens to use.

You can either set the visibility when calling write():

$filesystem->write('filename.txt', $data, ['visibility' : 'private']);

Prior to 5.4, or according to preference:

$filesystem->write('filename.txt', $data, array('visibility' => 'private'));

Alternatively, you can set the visibility of an existing object using setVisibility:

$filesystem->setVisibility('filename.txt', 'private');

Rather than set it on a file-by-file basis, you can set a default visibility for a given instance in its constructor:

$filesystem = new League\Flysystem\Filesystem($adapter, $cache, [
'visibility' => AdapterInterface::VISIBILITY_PRIVATE
]);

You can also use getVisibility to determine a file’s permissions:

if ($filesystem->getVisibility === 'private') {
    // file is secret
}

Listing Files and Directories

You can get a listing of all files and directories in a given directory like this:

$filesystem->listContents();

The output would look a little like this;

[0] =>
  array(8) {
  'dirname' =>
    string(0) ""
    'basename' =>
    string(11) "filters.php"
    'extension' =>
    string(3) "php"
    'filename' =>
    string(7) "filters"
    'path' =>
    string(11) "filters.php"
    'type' =>
    string(4) "file"
    'timestamp' =>
    int(1393246029)
    'size' =>
    int(2089)
}
[1] =>
array(5) {
    'dirname' =>
    string(0) ""
    'basename' =>
    string(4) "lang"
    'filename' =>
    string(4) "lang"
    'path' =>
    string(4) "lang"
    'type' =>
    string(3) "dir"
}

If you wish to incorporate additional properties into the returned array, you can use listWith():

$flysystem->listWith(['mimetype', 'size', 'timestamp']);

To get recursive directory listings, the second parameter should be set to TRUE:

$filesystem->listContents(null, true);

The null simply means we start at the root directory.

To get just the paths:

$filesystem->listPaths();

Mounting Filesystems

Mounting filesystems is a concept traditionally used in operating systems, but which can also be applied to your application. Essentially it’s like creating references – shortcuts, in a sense – to filesystems, using some sort of identifier.

Flysystem provides the Mount Manager for this. You can pass one or more adapters upon instantiation, using strings as keys:

$ftp = new League\Flysystem\Filesystem($ftpAdapter);
$s3 = new League\Flysystem\Filesystem($s3Adapter);

$manager = new League\Flysystem\MountManager(array(
    'ftp' => $ftp,
    's3' => $s3,
));

You can also mount a file system at a later time:

$local = new League\Flysystem\Filesystem($localAdapter);
$manager->mountFilesystem('local', $local);

Now you can use the identifiers as if they were protocols in URI’s:

// Get the contents of a local file…
$contents = $manager->read('local://path/to/file.txt');

// …and upload to S3
$manager->write('s3://path/goes/here/filename.txt', $contents);

It’s perhaps more useful to use identifiers which are generic in nature, e.g.:

$s3 = new League\Flysystem\Filesystem($s3Adapter);
$manager = new League\Flysystem\MountManager(array(
    'remote' => new League\Flysystem\Filesystem($s3Adapter),
));

// Save some data to remote storage
$manager->write('remote://path/to/filename', $data);

Caching

Flysystem also supports a number of technologies for caching file metadata. By default it uses an in-memory cache, but you can also use Redis or Memcached.

Here’s an example of using Memcached; the cache adapter is passed to the Filesystem constructor as an optional second parameter:

use League\Flysystem\Adapter\Local as Adapter;
use League\Flysystem\Cache\Memcached as Cache;

$memcached = new Memcached;
$memcached->addServer('localhost', 11211);
$filesystem = new Filesystem(new Adapter(__DIR__.'/path/to/root'), new Cache($memcached, 'storageKey', 300));
// Storage Key and expire time are optional

As with the filesystem adpaters, if you wished you could create your own – simply extend League\Flysystem\Cache\AbstractCache.

Summary

In this article I’ve introduced flysystem, a package which provides a layer of abstraction over a variety of forms of file storage. Although I haven’t covered every available adapter, the documentation should help fill in the gaps, introduce a couple of other features and document any new adapters in the future.

Frequently Asked Questions (FAQs) about Abstract File Systems and Flysystem

What is the main purpose of using Flysystem?

Flysystem is a PHP package that provides a simple and intuitive way to handle file systems. It is designed to simplify the process of working with files across multiple platforms, including local file systems, Amazon S3, and FTP. It provides a unified API for different storage options, allowing developers to switch between different file systems without changing their code. This makes it easier to manage and manipulate files, regardless of where they are stored.

How does Flysystem handle file visibility?

Flysystem handles file visibility using two primary visibility settings: public and private. The ‘public’ visibility means that the file is publicly accessible, while ‘private’ visibility means the file is not publicly accessible. You can set the visibility when writing or updating a file, and you can also check or change the visibility of an existing file using the ‘setVisibility’ and ‘getVisibility’ methods.

How can I handle exceptions in Flysystem?

Flysystem uses exceptions to handle errors. When an operation fails, Flysystem will throw a ‘FilesystemException’. You can catch this exception in your code to handle the error gracefully. This allows you to control what happens when an error occurs, such as logging the error or displaying a user-friendly error message.

How can I integrate Flysystem with Laravel?

Laravel provides out-of-the-box support for Flysystem. You can configure Flysystem as a disk in the Laravel filesystem configuration file. Once configured, you can use Laravel’s filesystem facade to interact with your Flysystem disk. This allows you to use all of Flysystem’s features in your Laravel application.

How can I use Flysystem to handle file uploads?

Flysystem provides several methods for handling file uploads. You can use the ‘write’, ‘writeStream’, ‘update’, and ‘updateStream’ methods to upload a file. These methods take the path of the file and the file content or resource as arguments. You can also use the ‘put’ method, which will create a new file or update an existing file.

How does Flysystem handle file metadata?

Flysystem provides methods to retrieve file metadata, such as the file size, timestamp, and mimetype. You can use the ‘getSize’, ‘getTimestamp’, and ‘getMimetype’ methods to retrieve this information. Flysystem also provides a ‘getMetadata’ method that returns an array of all available metadata.

Can I use Flysystem with Amazon S3?

Yes, Flysystem provides a dedicated adapter for Amazon S3. This allows you to use all of Flysystem’s features with Amazon S3. You can configure the S3 adapter with your S3 credentials and bucket name, and then use it as you would use any other Flysystem adapter.

How can I delete files with Flysystem?

Flysystem provides a ‘delete’ method to remove files. This method takes the path of the file as an argument. If the file does not exist, Flysystem will throw a ‘FileNotFoundException’. You can catch this exception to handle the error gracefully.

Can I use Flysystem to list files in a directory?

Yes, Flysystem provides a ‘listContents’ method to list the files in a directory. This method returns an array of file metadata. You can also use the ‘listPaths’ and ‘listWith’ methods to list file paths or files with specific metadata.

How can I read files with Flysystem?

Flysystem provides several methods to read files. You can use the ‘read’ and ‘readStream’ methods to read a file. These methods return the file content or a resource, respectively. You can also use the ‘has’ method to check if a file exists before reading it.