From Request to Response: A Journey into Drupal 8 Internals

Daniel Sipos
Daniel Sipos
Share

In the first article on Drupal 8 module development we looked a bit at the routing aspect of this process. We’ve seen that creating pages with paths is now a matter of declaring routes that match up with controllers. The latter, as we’ve seen, can return a render array that gets interpreted into markup and displayed in the main content area of that page. However, did you know that under the hood, Drupal actually transforms that array into a Response object according to the dictates of Symfony’s HTTPKernelInterface?

drupal8wide

In this article, I would like us to go deeper into the internals of Drupal 8 (and Symfony2) and look at what actually happens (and can happen) from the moment a request is made by a user to the one in which they see something returned in response. The example I mentioned above is just one direction this process can go in, and today we are also going to see other possibilities. The goal is to understand the flexibility of the system which in turn can help us build awesome applications.

Before going into it, I strongly recommend you check out this diagram which does an amazing job at synthesizing what is often referred to as the render pipeline. Though in my opinion it represents more than the name implies because the render system is only part of what’s depicted, albeit a big one.

Front controller (index.php)

It’s no secret that Symfony2 is now an important part of Drupal. The latter uses many of Symfony’s components, most importantly for this article the HTTPKernel and HTTPFoundation ones. Together they are responsible for encapsulating a user request, passing it to the application and then returning whatever comes back to the user in a consistent and OO way.

The HTTPKernelInterface (something you probably heard about also from other contexts) is what glues all of this together by taking in a Request object and always returning a Response one. A very simple but powerful concept.

This process is initiated inside the index.php file which starts by generating said Request object and passing it to the HTTPKernel::handle() method. The latter is then responsible for returning a Response object. At a high level, this is what happens both in a Drupal application as well as in a Symfony one (or any other that leverages the HTTPKernel component).

HTTPKernel and events

HTTPKernel is the heart of any Symfony based application. Its handle() method, as we saw, has a great deal of responsibility in preparing a response and it does so in a workflow driven by events. This makes for a very flexible application where the heavy lifting is always delegated to listeners of these events.

If you look at the diagram from earlier, you can see this workflow depicted in the second column, and it essentially represents the glue between Symfony and the Drupal side of things.

It starts with the first event called kernel.request. Subscribers to this event handle various tasks. But two very important ones in Drupal 8 are the format negotiation and routing. The first determines the type of response that needs to be returned (html, json, image, pdf, etc) while the second determines what the code responsible for handling this is (the _controller key of a route definition inside the routing.yml file). But like in most steps in this event workflow, if a listener returns a response object, the process skips most of the further steps (stops propagation) and goes straight to kernel.response.

The second event is kernel.controller which is called after the application knows which controller is responsible for handling the request. At this point, listeners can still perform some overriding operations on it. Closely following this step, the Kernel is responsible for resolving the arguments that get passed to the controller. One such operation in Drupal is loading objects based on IDs found in the request (for example nodes) and directly providing the controller with them. Then finally the controller gets called with the respective parameters.

The controller is responsible for returning a response of some kind. If it returns a Response object, the process skips to the kernel.response event. Listeners to the latter can perform last minute modifications on the object such as modifying headers or the content itself. And after getting it from the handle() method, the front controller uses the send() method on the Response object to send it back to the user and terminates the process.

symfony event workflow

Going deeper with render arrays

If the controller does not return a Response object, the Kernel fires one last event: kernel.view. Its subscribers are responsible for turning the result of the controller into an actual Response object. So this means that you have the option of returning from your controller any kind of object as long as you couple it with a VIEW event subscriber that turns that into a proper Response.

However, as we’ve seen in the example, most of the time controllers will return a render array. Usually this represents the page’s main content (similar to page callbacks in Drupal 7).

To handle this, Drupal 8 has a MainContentViewSubscriber responsible for transforming this array into proper Response objects. It does so by using a particular MainContentRenderer chosen during the format negotiation phase we’ve talked about before. And although there are a few such renderers already available, the default one used is the HtmlRenderer.

HTMLRenderer

Since this is the most commonly used type of main content renderer, let’s go in a bit deeper and see how this builds the page.

One of the cool things about this step in the process is the concept of page variants.
This means that HTMLRenderer dispatches an event responsible for finding out which type of page is to be used to wrap the main content render array: RenderEvents::SELECT_PAGE_DISPLAY_VARIANT. By default, the SimplePageVariant is used unless the Block module is enabled. In that case the BlockPageVariant kicks in and allows the placement of the blocks in the regions around the main content. If you want, you can subscribe to this event in your own module and provide your own variant.

All of this happens within the prepare() method of the HTMLRenderer which supplies the renderResponse() method with a #type => 'page' render array that wraps the main content one. The latter two get in turn wrapped into a #type => 'html' render array which gets finally rendered using the Renderer class (the equivalent of drupal_render() in Drupal 7). The resulting HTML string gets added to the Response object and gets returned to the front controller.

Although this is a very high level overview of the process, this is basically what happens. Now we have a Response object which means the Kernel can dispatch its kernel.response event. And right after this, the front controller can send the Response back to the user and terminate the process.

Conclusion

In this article we’ve taken a journey into Drupal 8 (and Symfony2) internals by following the pipeline from a user request to the response the server returns. We’ve seen how Drupal 8 leverages the HTTPKernel and HTTPFoundation Symfony2 components and how it basically lives on top of them. Additionally, we’ve seen how the glue between them is made up of the events the Kernel dispatches to which Drupal subscribes for all of its functionality. Finally, we’ve seen how HTML pages are built and returned to the user with the help of the render pipeline.

I believe that understanding what is going on under the hood in a Drupal 8 application will allow you to create awesome applications by knowing exactly which entry points you have into this flow. And I believe that if you take away only one thing from this article, it should be the word flexibility. Because the flexibility for building exactly what we need in Drupal 8 far surpasses anything in Drupal 7. It has truly become modern.

Frequently Asked Questions (FAQs) about Drupal 8 Internals

What are the key differences between Drupal 7 and Drupal 8?

Drupal 8 introduces several significant changes compared to Drupal 7. It has a new theme engine called Twig, which is more secure and flexible. Drupal 8 is also more mobile-friendly and includes more built-in fields. It supports multilingual sites better, with improved language management and translation support. Additionally, Drupal 8 has better integration with third-party platforms, thanks to its extensive use of Symfony components.

How does Drupal 8 handle requests and responses?

Drupal 8 uses the Symfony HttpKernel component to handle requests and responses. When a request is made, Drupal 8 creates a Request object and passes it to the HttpKernel. The HttpKernel then uses the routing system to determine which controller should handle the request. The controller processes the request and returns a Response object, which the HttpKernel sends back to the client.

What is the role of the routing system in Drupal 8?

The routing system in Drupal 8 is responsible for mapping URLs to specific controllers. It uses route definitions provided by modules to determine which controller should handle a given request. The routing system also supports dynamic routes, which can change based on the state of the system.

How does Drupal 8’s theme system work?

Drupal 8’s theme system uses Twig, a flexible and secure template engine. Themes in Drupal 8 consist of a .info.yml file, which provides metadata about the theme, and Twig template files, which define the HTML output. The theme system also supports template inheritance, allowing themes to extend and override templates from other themes or modules.

How can I develop custom modules in Drupal 8?

Developing custom modules in Drupal 8 involves creating a .info.yml file to provide metadata about the module, and a .module file to contain the module’s PHP code. Modules can also include other files, such as routing files to define routes, and service files to define services. Drupal 8’s use of object-oriented programming and Symfony components makes it easier to write reusable and testable code.

What are the benefits of using Drupal 8 for web development?

Drupal 8 offers several benefits for web development. Its use of Symfony components makes it more robust and flexible. It supports responsive design out of the box, making it easier to create mobile-friendly sites. Drupal 8 also has improved multilingual support, better SEO capabilities, and a more user-friendly admin interface.

How does Drupal 8 handle database interactions?

Drupal 8 uses the Database API to handle database interactions. The Database API provides an abstraction layer over SQL, allowing developers to write database queries without needing to know the specifics of the underlying database engine. It also supports dynamic queries, transactions, and schema management.

What is the role of the Event Dispatcher in Drupal 8?

The Event Dispatcher in Drupal 8 is used to manage events and event listeners. When an event occurs, the Event Dispatcher notifies all registered listeners, which can then take action. This allows modules to interact with each other in a decoupled way.

How does Drupal 8 handle caching?

Drupal 8 has a sophisticated caching system that helps improve performance. It supports multiple cache backends, including database, file, and memory-based backends. Drupal 8 also has a cache tagging system, which allows for fine-grained invalidation of cached data.

How can I extend Drupal 8’s functionality?

Drupal 8’s functionality can be extended through modules and themes. Modules add new functionality or modify existing functionality, while themes control the look and feel of the site. Drupal 8 also supports plugins, which allow for swappable pieces of functionality, and services, which provide reusable functionality that can be injected into other parts of the system.