Containerized PHP Development Environments with Vagga

Key Takeaways

• Vagga is a container engine designed to streamline the creation of development environments, offering faster load times and lower memory usage compared to Vagrant.
• Unlike Docker, Vagga operates entirely in userspace, facilitating quicker setup of applications in different environments without the need for full virtualization.
• The installation process for Vagga, particularly on Linux, involves enabling user namespaces and executing a simple script, though it currently requires running inside Vagrant on Windows and Mac OS X.
• Vagga utilizes a configuration file named `Vagga.yaml` to define containers and commands, allowing for detailed customization and control over the development environment setup.
• Key features of Vagga include the ability to run multiple services with different configurations simultaneously, quick provisioning of boxes, and automatic rebuilding of containers when project dependencies change.
• Despite its benefits, Vagga’s current limitations include lack of cross-platform support outside Linux, a complex installation process, and the necessity for manual configuration and setup of software packages within containers.

It happens to all of us once in a while.

We clone a project, and then we try to run it. However, something doesn’t work.

It may be our version of NGINX or Apache. It might be that npm isn’t doing something right. Maybe the project needs an extension, and we don’t have it installed, and now we have to build the extension from source because the dependency does not exist in the repositories for our distribution. No matter the reason, the more complex the setup, the higher the probability of failure.

When I first got to know Vagrant, it was like heaven: there was finally a chance to break free of the shortcomings of Windows, while not having to deal with the accessibility issues present in Linux.

I was happy. For a while. And then the limitation of having virtual machines as development environments hit me. Hard.

Imagine this scene: it’s 4:30 PM. The developer working on the Asterisk system has left the company. Something in the payment logic (written in PHP using the deprecated PHPagi project) doesn’t work, and you have to fix it, fast. You and a colleague have been urging everyone toward a service-oriented architecture, and so, to be able to bring up a development environment, you have to have the following:

• An instance of the Asterisk machine which handles calls. It must have Asterisk and PHP 5.5.X running.
• An instance of another Asterisk machine which actually handles the payment. Since the bank you are working with has a policy of closing all access to its APIs coming from outside the building, this is a copy of the machine above, but with different code running on it.
• The PHP back-end API that does the heavy lifting, written in PhalconPHP.

So you tape together some puppet scripts to provision your boxes, you vagrant up three machines, and prepare to enter the zone!

If this was a movie, the scene would zoom into your host machine at this point, dramatically showing the large amounts of data traveling at light speed between different parts of your system, filling up your memory and CPU, and then coming to a grinding halt. However, since this is not a movie, suffice it to say that your development machine might be able to handle one or two boxes with 8 GB of memory on Windows, but when the third machine comes up, your development environment is going to be slow to the point that you will not be able to launch an IDE.

Of course, you are very clever and resourceful, and know how this stuff works. You’d lower your memory and CPU in your vagrantfile, but that leads to Asterisk complaining about a small memory size, and when you manually call your phone system with a softphone (equivalent to a browser in the online world) to test it, the audio breaks up and distorts. You need a lot more memory than you already have, and you don’t have access to it, and now it’s 10:30 PM.

So what happens to you?

Nothing good. You fail to fix the problem that night, have to deal with an angry boss who doesn’t have the technical background to understand what you are talking about, come back to work tomorrow, tell two of your colleagues to run one of the needed machines from the list above on their own machine, and you string these virtual machines together to get the code to run. Oh, and let’s not forget the problems you have with building all the dependencies because they weren’t documented anywhere. That’d just be too ugly.

But all problems give rise to new opportunities: enter Vagga, a way to set up your project and its dependencies (usually) with a single command, with far less resource usage.

What is Vagga?

Vagga is a container engine, like Docker, which has been created to make it easier to build development environments. Since it is a fully-userspace container engine, it loads much faster than Vagrant, takes much less memory, and allows you to do awesome things like run your application in different environments in just a few seconds, without waiting for a completely virtualized machine to boot up.

Vagga is in beta stage right now and it is likely you might face some problems, but the concept shows great promise, and that’s why we have decided to take it for a spin.

Wait! What’s wrong with Docker and Docker-compose?

Docker-compose is a tool to configure and create multiple Docker containers through a configuration file. I personally don’t have any experience with Docker and Docker-compose. However, Vagga’s documentation has a page comparing Vagga and Docker. You can check it out if you’re interested. If you have experience using Docker, tell us about the differences and similarities in the comments!

The Requirements

Since Homestead improved is something we recommend here at SitePoint, it would be great if we could replicate that setup with Vagga to have a real-world way of assessing its ease of use, strengths and shortcomings. So, what we are going for in this article is a webserver (NGINX), coupled with PHP-fpm to serve an index.php file that is made up of just three lines:

<?php
phpinfo();
?>

Installation

Vagga is very new and does not have the time and manpower behind Vagrant and Docker. You can see that in the installation steps, outlined below. I urge you to grit your teeth and go through it though; I promise that if you are used to using Vagrant like me, the rewards will be surprising.

First, we create a directory for our project. It could be anywhere, but for our example we’ll name it vagga.

Linux

Note: PHP and composer support has been added to Vagga very recently, so we will be using the *-testing packages. When this makes it to the stable branch, you may want to use the URLs below without the *-testing suffix.

If you’re on Linux, you will get the full benefit of Vagga in terms of speed and features. All you need to do is to enable user namespaces, which, depending on your distribution, may or may not be enabled by default.

If you are running Ubuntu 14.04 (Trusty) or higher, installing Vagga is as simple as running this in your terminal:

curl http://files.zerogw.com/Vagga/Vagga-install-testing.sh | sh

If you are using another distribution, I suggest you use the installation page of the Vagga manual to troubleshoot issues, or to get more up-to-date and in-depth steps to proceed.

Windows and Mac OS X

If you are using Windows or Mac, you actually have to have Vagrant installed, because Vagga currently works in Linux. This might seem ironic to you based on the real story I said at the beginning of this article. However, keep in mind that you can still use Vagga for the below reasons, even though you have to do it inside a virtual machine:

• Running different software simultaneously (databases, webservers, PHP-fpm instances) with different configuration, without taking up huge amounts of memory and CPU
• Quickly provisioning your boxes without being familiar with orchestration software (like puppet or chef), or having to deal with container daemons like Docker
• Documenting the steps necessary to bring up a development environment
• Rebuilding containers if project dependencies change (such as adding a new package to your composer.json, or changing the version of an already existing package)

Here are the steps to prepare an environment to install Vagga inside Vagrant:

• Install Vagrant and virtualbox.
• Install the vagrant-Vagga plugin by opening a command prompt and running vagrant plugin install Vagrant-Vagga.
• Create a new folder to house your project, and run vagrant init ubuntu/wily64 to create a vagrantfile, using ubuntu/wily64 as the base image.

Installing vagrant-Vagga has added a new provisioner to your Vagrant installation. Without it, you will not be able to use Vagga because of limitations in vboxfs, the virtual filesystem virtualbox uses to share files between your host and guest machines (if you have no idea what I am talking about, check out this article to get more background).

The next step is to tell Vagrant to run this provisioner every time you run vagrant up to bring up your virtual machine. How? Simply open up your vagrantfile, move to the end, and add these two lines just above the line that reads end:

  config.Vagga.testing = true
  config.vm.provision :Vagga, run: 'always'

The first line is telling Vagrant-Vagga to install the testing version of Vagga. This is required because currently, the feature to install composer and PHP is in the testing binaries only.

The second line is quite self-explanatory; it simply runs the Vagga provisioner, which links your .Vagga directory to another directory inside the virtual machine, preventing the issues that come with having that folder on a vboxfs synced folder.

Phew! You’re done setting up your virtual machine! You can now run vagrant up to bring up the environment you will use to run Vagga containers. Vagrant will take a while to download the box image, so grab yourself a cup of coffee and pat yourself on the back for getting so far – it gets only easier from here!

Getting Ready: Vagga.yaml

Just like we have composer.json, vagrantfile, Dockerfile and similar files of that sort, we also have Vagga.yaml which contains definitions for containers and commands.

Setting Up Our containers

Containers are lightweight sandbox environments, which allow you to install and run applications without touching the other parts of the system. What this means that you can have PHP in a machine and when you create a container with Vagga, not only will they not conflict, but they will exist independently from each other, allowing you to have different versions of a package, with different configurations, living side by side in their own isolated containers.

With Vagga, it is usually better to create one container per service. That is why in our example, we are going to define one container for Nginx, and another for PHP.

Create a file called Vagga.yaml in the vagga directory, which is the root of our project, and let’s define a first sketch

containers:
  nginx:
    setup:
      - !Ubuntu trusty
      - !Install [software-properties-common]
      - !Sh add-apt-repository ppa:nginx/stable -y && apt-get update
      - !Install [nginx]

  php:
    setup:
      - !Ubuntu trusty
      - !Install [software-properties-common]
      - !Sh add-apt-repository ppa:ondrej/php -y && apt-get update
      - !Install [php7.0, php7.0-fpm]
      - !ComposerConfig
        install_runtime: false
      - !ComposerInstall

Before we tell Vagga to build these containers, let’s take a look at what we have included in this file.

Note: if you are not familiar with YAML, take a look at this somewhat outdated article called Using YAML in Your PHP Projects. All you need from that article is how YAML syntax is mapped to PHP data types.

As evident at a glance, there is a first-level key called containers, which is an array of key-value pairs. The key is our container name (nginx and php in this example) and the value is the configuration for each container.

Each container’s configuration must have a setup key defined. It is made up of build steps, using which you can set up the container and get it ready to run your software. There are three types of build steps used in the Vagga.yaml example above:

• Bootstrapping commands: these build steps come first, and install an operating system onto the container. In our example, we have used the !Ubuntu build step to install a predefined version (14.04, AKA trusty) onto our container.
• Distribution commands: these are commands that Vagga converts to a distribution-specific command. For example, our !Install build step above is converted to the apt-get install -y <package name> command when it is being run inside an Ubuntu container.
• Generic commands: these are commands you can run on any distribution you like. These are low-level commands, such as !Sh to run a command with shell, !Download to download a file into the container, !tar and !TarInstall for extracting and installing a tar archive, and so on. We have used the !Sh command in the Vagga.yaml example above to add the nginx/stable and ondrej/PHP PPAs.

So, as you can probably tell, to set up our containers, we:

1. Install Ubuntu Trusty
2. Install software-properties-common to get access to the add-apt-repository command
3. Add nginx/stable to the NGINX container, and ondrej/php to the PHP container, and then run apt-get update to fetch the list of packages from the newly added repositories
4. Install the packages: nginx in the NGINX container, and php7.0 and php7.0-fpm in the php container
5. Configure composer using the !ComposerConfig to not install the PHP package itself (if you don’t do this, it will install the php package, which as of the time of this writing is version 5.5.9), and then install composer itself using !ComposerInstall

If you run Vagga _build nginx or Vagga _build php at this point, you will get some errors saying that a file cannot be written to. This is because for security reasons, Vagga doesn’t allow applications inside your container to write to the file system, unless you explicitly allow some directories to be written to. You can do that by adding volumes.

Adding Volumes

Volumes are primarily a way to allow the applications on your containers to write to a file system. The directory that Vagga.yaml resides in is automatically added as /work inside the container (just like Vagrant adds the directory that contains vagrantfile as /Vagrant inside the virtual machine), but apart from that, Vagga adds /tmp and /run and /run/shm as volumes, too.

To know and read more about different volume types, check the volumes page in the Vagga documentation.

If you search the Internet or keep trying to build your containers, you will get a feel for what directories are required to run a package. For our example, NGINX requires /var/lib, /var/lib/nginx, /var/log, and /var/log/nginx. Similarly, PHP requires access to /run/php, and /var/log. Let’s add those to our containers:

containers:
  nginx:
    setup:
      - !Ubuntu trusty
      - !Depends runtime/nginx/default
      - !Install [software-properties-common]
      - !Sh add-apt-repository ppa:nginx/stable -y && apt-get update
      - !Install [nginx]
    volumes:
      /var: !Tmpfs
       mode: 0o766
       subdirs:
        lib:
        lib/nginx:
        log: # default mode is 0o766
        log/nginx: { mode: 0o1777 }
  
  php:
    setup:
      - !Ubuntu trusty
      - !Install [software-properties-common]
      - !Sh add-apt-repository ppa:ondrej/php -y && apt-get update
      - !Install [php7.0, php7.0-fpm]
      - !ComposerConfig
        install_runtime: false
      - !ComposerInstall
    volumes:
      /run: !Tmpfs
       mode: 0o766
       subdirs:
        php:
      /var/log: !Tmpfs
       mode: 0o766

As you can see, we have added first-level directories as volumes, and have defined all the subdirectories in that directory in the subdirs section of each volume. When a mode is set on a parent directory, all subdirs also inherit it by default.

Note: If you can’t understand the mode parameter of each volume, you can think of it like this: the first value (0 or 1) is the Sticky bit, the o means the value coming after it is octal, and the next three numbers (eg 777) are the permissions.

Configuring NGINX and PHP-fpm

As you probably know if you have ever installed NGINX or PHP-fpm by hand, after you install these applications on a Linux machine, you must configure them. Since we want to make this automatic, we can write the configuration file ourselves, and use the !Copy build step to get it into the container at build time. What I usually do in this step is:

• Attach to the container (i.e. log into it) by running Vagga _run <container name> bash. This simply runs bash in the container and allows you to type commands.
• Copy the configuration file I want to edit to the /work directory, so that I can get access to it in my machine. I do this by running a command like cp nginx.conf /work. Remember, the /work directory is shared between your machine and the container.

So, to configure NGINX and PHP-fpm you need to edit the following files:

• /etc/nginx/sites-available/default
• /etc/php/7.0/fpm/php-fpm.conf
• /etc/php/7.0/fpm/pool.d/www.conf

The convention for putting configuration files in your project seems to be the runtime directory inside your project. Navigate to your project folder (vagga in our example) and create a runtime directory. Inside it create a directory called nginx to store your NGINX configuration, and another called php to hold the other two files. You can either copy the configuration files from your container to get a feel for the process, or just copy and paste the following content into each file.

runtime/nginx/default

In this file, we tell NGINX to listen on port 8000, set the server name to example.com and www.example.com, and give the path to our root folder and PHP-fpm instance (which will be listening on 127.0.0.1:9000):

Note: since Vagga runs without root privileges, processes running inside containers cannot listen to ports below 1024. This is a restriction imposed by the Linux operating system.

server {
        listen   8000;

        root /work/;
        index index.php index.html index.htm;
        server_name  example.com www.example.com;

        location / {
                try_files $uri $uri/ /index.html;
        }

        error_page 404 /404.html;
        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
              root /usr/share/nginx/www;
        }

        location ~ \.php$ {
                try_files $uri =404;
                fastcgi_pass 127.0.0.1:9000;
                fastcgi_index index.php;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                include fastcgi_params;
        }
}

runtime/php/php-fpm.conf

The reason we are editing this file is that, since Docker and Vagga monitor the process inside each machine, they require your applications to be run in foreground, meaning that they shouldn’t be a daemon. By default, PHP-fpm runs as a daemon, and we are editing the configuration to set the daemonize configuration value to no.

daemonize = no

Another, perhaps cleaner, way would be to just create a file called no-daemonized.conf, and put the single line above in it, then put it in our runtime/php directory, and copy that instead.

runtime/php/www.conf

The reason we are changing this file is to make it listen on port 9000, instead of creating a socket file. Change the value listen:

listen = 9000

Copying Configuration into the Containers

The last step in building our containers is to copy these configuration files into the proper place. As you might remember, earlier in this article I said that Vagga will rebuild your containers if the dependencies of your project change. So, since we want modifications to these configuration files to take effect every time we run commands, we also need to add them as dependencies using the !Depend command. Here is the final version of our Vagga.yaml:

containers:
  nginx:
    setup:
      - !Ubuntu trusty
      - !Depends runtime/nginx/default
      - !Install [software-properties-common]
      - !Sh add-apt-repository ppa:nginx/stable -y && apt-get update
      - !Install [nginx]
      - !Copy 
        source: /work/runtime/nginx/default
        path: /etc/nginx/sites-available/default
    volumes:
      /var: !Tmpfs
       mode: 0o766
       subdirs:
        lib:
        lib/nginx:
        log: # default mode is 0o766
        log/nginx: { mode: 0o1777 }
  
  php:
    setup:
      - !Ubuntu trusty
      - !Depends runtime/php/php-fpm.conf
      - !Depends runtime/php/www.conf
      - !Install [software-properties-common]
      - !Sh add-apt-repository ppa:ondrej/php -y && apt-get update
      - !Install [php7.0, php7.0-fpm]
      - !ComposerConfig
        install_runtime: false
      - !ComposerInstall
      - !Copy
        source: /work/runtime/php/php-fpm.conf
        path: /etc/php/7.0/fpm/php-fpm.conf
      - !Copy
        source: /work/runtime/php/www.conf
        path: /etc/php/7.0/fpm/pool.d/www.conf
    volumes:
      /run: !Tmpfs
       mode: 0o766
       subdirs:
        php:
      /var/log: !Tmpfs
       mode: 0o766

Note: notice that when copying, both the source and the destination paths are inside the container. This is unlike Docker, where the source address is given inside the host machine.

Success! You can now run Vagga _build nginx and Vagga _run php to build your containers! Downloading packages will take some time, so do the grab-coffee-pat-self-on-the-back routine, and prepare for the last step: adding the run command to tie everything together.

Commands

Commands are what other developers use to tap into the power of this new setup. There are two types of commands:

• Commands tagged with !Command: these commands are run and just exit. Examples of this type of command are ls, php -v, and so on.
• Commands tagged with !Supervise: these commands have some children, and will do an action (defined by their mode attribute) if any of the commands they are watching would exit. Examples of these commands include nginx and PHP-fpm, which is exactly what we will be using.

First, let’s add a command called php. It will pass the file we want to the PHP interpreter installed inside our php container. Add this to the end of your Vagga.yaml:

commands:
  php: !Command
    description: passes the given parameters to the PHP interpreter
    container: php
    run: [php]

As you can probably guess, description is the help text that is printed when you run vagga without an argument, container is the container to run the command in, and run is the command to run. If we hadn’t provided the argument to run as an array, it would not accept arguments and would just run php without them. However, by providing the value as an array, we are allowing the user to pass in arguments to php, such as a file.

To test this out, you can simply do Vagga php -v. If you are using Vagrant, you can vagrant ssh into the Vagrant machine, cd into the /Vagrant directory, and run Vagga php -v. Or you can simply run vagrant Vagga php -v from your host machine outside your Vagrant box.

Finally, we are ready to run our NGINX and PHP stack. First, change the commands section inside Vagga.yaml:

commands:
  php: !Command
    description: passes the given parameters to the PHP interpreter
    container: php
    run: [php]
  run: !Supervise
    description: Run the NGINX and PHP stack
    mode: stop-on-failure
    children:
      nginx: !Command
        container: nginx
        run: nginx -g "daemon off;"
      php: !Command
        container: php
        environ:
          DATABASE_URL: postgresql://Vagga:Vagga@127.0.0.1:5433/test
        run: php-fpm7.0

As you can see, we are not providing the value of the run attribute as an array, effectively preventing users from passing in any arguments.

Finally, let’s run Vagga run (or vagrant Vagga run) and observe the output:

[19-Mar-2016 15:00:22] NOTICE: fpm is running, pid 3
[19-Mar-2016 15:00:22] NOTICE: ready to handle connections
[19-Mar-2016 15:00:22] NOTICE: systemd monitor interval set to 10000ms

Go ahead and create the index.php file in your vagga directory, like this:

<?php
phpinfo();
?>

If you haven’t already, open your vagrantfile and uncomment this line, so that you will be able to reach your virtual machine using a static IP:

  config.vm.network "private_network", ip: "192.168.33.10"

Then, add example.com to the hosts file on your host machine. On Unix systems, this file is located at /etc/hosts, and on Windows, it is at C:\Windows\System32\drivers\etc\hosts. Here is how you would map the example.com domain to 192.168.33.10, which is the IP of your virtual machine:

192.168.33.10 example.com

Now simply open your browser and browse to example.com:8000, or www.example.com:8000. You should see the PHP info page.

Congratulations! You have created a development environment with Vagga, and you can now allow other users to bring up a development environment in a remarkably short amount of time!

Conclusion

Vagga is great for containerization and creating a living document of what a development environment for an application looks like. It also has the benefit of allowing a developer to quickly set up a development environment, without knowing the nitty-gritty details of the process. In my opinion, this is a huge advantage in the world of open-source, since this allows contributors to be productive in minutes.

Of course, Vagga does have its downsides:

• It is not as cross-platform as Docker. Docker itself is not cross-platform either, but its contributors have created tools that make it very easy to set up on Windows.
• The installation process is not as simple as that of its competitors. You may need to muck around with some configuration based on which distribution and version you are using.
• One advantage Puppet, Ansible, Docker and others have over Vagga is that building the development and production environments doesn’t require you to master two languages – you just build the environment with one syntax. There is the possibility of deploying containers built with Vagga, however, the process is not well-documented yet.
• Vagga, like Docker, requires you to know how to install and configure a particular package, whereas orchestration software such as Puppet has modules to configure popular software for you. Examples like NGINX, PHP and PHP-fpm come to mind.
• You have to do everything from scratch. There are no base images to build from, unlike with Docker or Vagrant.

Do you use other software to allow your team or your contributors to quickly spin up a development environments from scratch? Do you think that in the fast-paced world of development environments of 2016, automated ways to set up a development environment are not worth the effort? Tell us your experiences in the comments!