Your First Drupal 8 Migration

Share this article

Migrate is one of the most established modules in the Drupal ecosystem. So much so that with Drupal 8, a decision has been made to get some of its functionality ported and added to Drupal core. An important reason was that the traditional upgrade between major releases was replaced with a migration of Drupal 6 or 7 content and configuration to Drupal 8.

Drupal 8 logo

Not all the functionality of the Migrate module has been moved into core though. Rather, we have the basic framework within the core migrate module and the functionality serving the upgrade path from Drupal 6 and 7 within the core migrate_drupal module. What’s left can be found in a host of contrib modules. The most important is Migrate Tools which contains, among other things, drush commands and the UI for running migrations. Additionally, the Migrate Source CSV, Migrate Source XML and Migrate Source JSON modules provide plugins for the most used types of migration sources.

In this article we are going to look at how migration works in Drupal 8 by migrating some content into node entities. For simplicity, the data we play with resides in tables in the same database as our Drupal installation. If you want to see the final code, you can check it out in this repository.

Drupal 8 Migration Theory

The structure of a migration in Drupal 8 is made up of three main parts: the source, the process and the destination, all three being built using the new Drupal 8 plugin system. These three parts form a pipeline. The source plugin represents the raw data we are migrating and is responsible for delivering individual rows of it. This data is passed to one or more process plugins that perform any needed manipulation on each row field. Finally, once the process plugins finish preparing the data, the destination plugins save it into Drupal entities (either content or configuration).

Creating a migration involves using such plugins (by either extending or directly using core classes). All of these are then brought together into a special migration configuration entity. This is shipped as module config that gets imported when the module is first enabled or can be constructed using a template (a case we won’t be exploring today). The migration configuration entity provides references to the main plugins used + additional metadata about the migration.

Movie Migration

Let us now get down to it and write a couple of migrations. Our data model is the following: we have two tables in our database: movies and movies_genres. The former has an ID, name and description while the latter has an ID, name and movie ID that maps to a movie from the first table. For the sake of simplicity, this mapping is one on one to avoid the complication of a third table. Here is the MySQL script that you can use to create these tables and populate with a few test records (if you want to follow along):

CREATE TABLE `movies` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `name` varchar(255) DEFAULT NULL,
  `description` text,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8;

CREATE TABLE `movies_genres` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `movie_id` int(11) DEFAULT NULL,
  `name` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8;

INSERT INTO `movies` (`id`, `name`, `description`)
VALUES
	(1, 'Big Lebowsky', 'My favorite movie, hands down.'),
	(2, 'Pulp fiction', 'Or this is my favorite movie?');

INSERT INTO `movies_genres` (`id`, `movie_id`, `name`)
VALUES
	(1, 1, 'Comedy'),
	(2, 1, 'Noir'),
	(3, 2, 'Crime');
	

What we want to achieve is migrate the movies into basic Drupal Article nodes and the genres into taxonomy terms of the Tags vocabulary (which the Article nodes reference via a field). Naturally, we also want the migration to mirror the association between the movies and the genres.

Genres

Let us first take care of the genre migration because in Drupal, the movies will depend on them (they will reference them).

Inside the config/install folder of our module, we need the following configuration entity file called migrate.migration.genres.yml:

id: genres
label: Genres
migration_group: demo
source:
  plugin: genres
  key: default
destination:
  plugin: entity:taxonomy_term
process:
  vid:
    plugin: default_value
    default_value: tags
  name: name

The first three elements of this configuration are the migration ID, label and group it belongs to. The following three keys are responsible for the three main parts of the migration pipeline mentioned earlier. Let’s talk about the latter three separately.

Source

Under the source key we specify which plugin the migration should use to represent and deliver the source data (in our case the plugin with the ID of genres). The key key is used to specify which database should our source query use (that is where our data is).

So in the Plugin/migrate/source folder of our module, let’s create our SQL based source plugin for our genres:

Genres.php

namespace Drupal\demo\Plugin\migrate\source;

use Drupal\migrate\Plugin\migrate\source\SqlBase;

/**
 * Source plugin for the genres.
 *
 * @MigrateSource(
 *   id = "genres"
 * )
 */
class Genres extends SqlBase {

  /**
   * {@inheritdoc}
   */
  public function query() {
    $query = $this->select('movies_genres', 'g')
      ->fields('g', ['id', 'movie_id', 'name']);
    return $query;
  }

  /**
   * {@inheritdoc}
   */
  public function fields() {
    $fields = [
      'id' => $this->t('Genre ID'),
      'movie_id' => $this->t('Movie ID'),
      'name' => $this->t('Genre name'),
    ];

    return $fields;
  }

  /**
   * {@inheritdoc}
   */
  public function getIds() {
    return [
      'id' => [
        'type' => 'integer',
        'alias' => 'g',
      ],
    ];
  }
}

Since we are using a SQL source, we need to have our own source plugin class to provide some information as to how the data needs to be read. It’s not enough to use an existing one from core. The query() method creates the query for the genre data, the fields() method defines each individual row field and the getIds() method specifies the source row field that acts as the unique ID. Nothing complicated happening here.

We are extending from SqlBase which, among other things, looks for the plugin configuration element named key to learn which database it should run the query on. In our case, the default one, as we detailed in the migration configuration entity.

And this is all we need for our simple source plugin.

Destination

For the migration destination, we use the default core taxonomy term destination plugin which handles everything for us. No need to specify anything more.

Process

Under the process key of the migration, we list each destination field we want populated and one or more process plugins that transform the source row fields into data for the destination fields. Since we want the genres to be all terms of the Tags vocabulary, for the vid field we use the default_value plugin which accepts a default_value key that indicates the value each record will have. Since all will be in the same vocabulary, this works well for us.

Lastly, for the term name field we can simply specify the source row field name without an explicit plugin name. This will, under the hood, use the get plugin that simply takes the data from the source and copies it over unaltered to the destination.

For more information on how you can chain multiple process plugins in the pipeline or what other such plugins you have available from core, I recommend you check out the documentation.

Movies

Now that our genres are importable, let’s take a look at the movies migration configuration that resides in the same folder as the previous (config/install):

migrate.migration.movies.yml

id: movies
label: Movies
migration_group: demo
source:
  plugin: movies
  key: default
destination:
  plugin: entity:node
process:
  type:
    plugin: default_value
    default_value: article
  title: name
  body: description
  field_tags:
    plugin: migration
    migration: genres
    source: genres
migration_dependencies:
  required:
    - genres

We notice the same metadata as before, the three main parts of the migration (source, process and destination) but also the explicit dependency which needs to be met before this migration can be successfully run.

Source

Like before, let’s take a look at the movies source plugin, located in the same place as the genres source plugin (Plugin/migrate/source ):

Movies.php:

namespace Drupal\demo\Plugin\migrate\source;

use Drupal\migrate\Plugin\migrate\source\SqlBase;
use Drupal\migrate\Row;

/**
 * Source plugin for the movies.
 *
 * @MigrateSource(
 *   id = "movies"
 * )
 */
class Movies extends SqlBase {

  /**
   * {@inheritdoc}
   */
  public function query() {
    $query = $this->select('movies', 'd')
      ->fields('d', ['id', 'name', 'description']);
    return $query;
  }

  /**
   * {@inheritdoc}
   */
  public function fields() {
    $fields = [
      'id' => $this->t('Movie ID'),
      'name' => $this->t('Movie Name'),
      'description' => $this->t('Movie Description'),
      'genres' => $this->t('Movie Genres'),
    ];

    return $fields;
  }

  /**
   * {@inheritdoc}
   */
  public function getIds() {
    return [
      'id' => [
        'type' => 'integer',
        'alias' => 'd',
      ],
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function prepareRow(Row $row) {
    $genres = $this->select('movies_genres', 'g')
      ->fields('g', ['id'])
      ->condition('movie_id', $row->getSourceProperty('id'))
      ->execute()
      ->fetchCol();
    $row->setSourceProperty('genres', $genres);
    return parent::prepareRow($row);
  }
}

We have the same three required methods as before, that do the same thing: query for and define the data. However, here we also use the prepareRow() method in order to alter the row data and available fields. The purpose is to select the ID of the movie genre that matches the current row (movie). That value is populated into a new source field called genres, which we will see in a minute how it’s used to save the Tags taxonomy term reference.

Destination

In this case, we use the node entity destination plugin and we need nothing more.

Process

There are four fields on the Article node we want populated with movie data. First, for the node type we use the same technique as before for the taxonomy vocabulary and set article to be the default value. Second and third, for the title and body fields we map the movie name and description source fields unaltered.

Lastly, for the tags field we use the migration process plugin that allows us to translate the ID of the genre (that we added earlier to the genres source row field) into the ID of its corresponding taxonomy term. This plugin does this for us by checking the migration mapping of the genres and reading these IDs. And this is why the genres migration is also marked as a dependency for the movies import.

Activating and Running the Migration

Now that we have our two migration configuration entities and all the relevant plugins, it’s time to enable our module for the first time and have the configuration imported by Drupal. If your module was already enabled, uninstall it and then enable it again. This will make the config import happen.

Additionally, in order to run the migrations via Drush (which is the recommended way of doing it), install the Migrate Tools module. Then all that’s left to do is to use the commands to migrate or rollback the movies and genres.

To see the available migrations and their status:

drush migrate-status

To import all migrations:

drush migrate-import --all

To roll all migrations back:

drush migrate-rollback --all

Conclusion

And there we have it – a simple migration to illustrate how we can now import, track and roll back migrations in Drupal 8. We’ve seen how the plugin system is used to represent all these different components of functionality, and how the migration definition has been turned into configuration that brings these elements together.

There is much more to learn, though. You can use different source plugins, such as for data in CSV or JSON, complex process plugins (or write your own), or even custom destination plugins for whatever data structure you may have. Good luck!

Frequently Asked Questions (FAQs) about Drupal 8 Migration

What are the prerequisites for migrating to Drupal 8?

Before migrating to Drupal 8, you need to ensure that your current Drupal site is updated to the latest minor version. For instance, if you’re using Drupal 7, update it to the latest Drupal 7.x version. Also, make sure to back up your site’s database and files. This is crucial in case you need to revert to the previous version. Additionally, you should review the contributed modules and themes in your current site and check if they are compatible with Drupal 8.

How do I prepare my Drupal 8 site for migration?

Preparing your Drupal 8 site for migration involves several steps. First, install a fresh Drupal 8 site. Then, enable the Migrate, Migrate Drupal, and Migrate Drupal UI modules. These modules are part of the core in Drupal 8. Also, ensure that the contributed modules and themes you plan to use in Drupal 8 are installed and enabled.

How do I perform the migration process?

The migration process in Drupal 8 is performed through the user interface. Go to the Upgrade page (admin/upgrade) in your Drupal 8 site. Follow the instructions provided on the page. You will be asked to provide the database details and files directory path of your old site. Once you’ve entered these details, the migration process will start.

What happens to my content during the migration process?

During the migration process, your content, including nodes, comments, users, files, and taxonomy terms, will be copied from your old site to the new Drupal 8 site. The original content on your old site will not be modified or deleted.

What should I do if the migration process fails?

If the migration process fails, check the recent log messages (admin/reports/dblog) in your Drupal 8 site. The log messages will provide information about the errors that occurred during the migration process. You can then fix these errors and try the migration process again.

Can I migrate my site’s configuration to Drupal 8?

Yes, you can migrate your site’s configuration, including content types, fields, and views, to Drupal 8. However, this requires using the Drupal 8 Configuration Management system and may involve some manual work.

How do I migrate my site’s theme to Drupal 8?

Migrating a theme to Drupal 8 involves creating a new Drupal 8 theme and then copying the CSS, JavaScript, and template files from your old theme to the new theme. You may also need to update these files to use the new Drupal 8 theming system.

Can I roll back a migration?

Yes, you can roll back a migration in Drupal 8. This can be done using the Drush command ‘drush migrate-rollback’. However, be aware that rolling back a migration will delete the migrated data in your Drupal 8 site.

How do I update my migrated content in Drupal 8?

After migrating your content to Drupal 8, you can update it using the regular content editing features in Drupal 8. If you need to update the migrated content in bulk, you can use the Views Bulk Operations module.

Can I perform a partial migration?

Yes, you can perform a partial migration in Drupal 8. This can be done using the Migrate API, which allows you to select specific types of content to migrate. However, this requires some knowledge of PHP and Drupal’s APIs.

Daniel SiposDaniel Sipos
View Author

Daniel Sipos is a Drupal developer who lives in Brussels, Belgium. He works professionally with Drupal but likes to use other PHP frameworks and technologies as well. He runs webomelette.com, a Drupal blog where he writes articles and tutorials about Drupal development, theming and site building.

BrunoSdatabase migrationdrupaldrupal 8drupal moduledrupal plugindrupal-planetdrupal8drupal8. modulemigrationmigrationsOOPHPPHP
Share this article
Read Next
Get the freshest news and resources for developers, designers and digital creators in your inbox each week