SOA, or Service Oriented Architecture, is often tossed around as an enterprise-only term, used by companies no smaller than Amazon. Fear not, SOA can be used by the Little Guys too! Let’s break the mold.
The goal of any SOA implementation is to segment portions of an application by logical and business functions. It can break down monolithic applications into many smaller, consumable services. SOA can be used by the Little Guys, but it’s generally not something you want to apply until experiencing growing pains. There’s no cut-and-dry answer to when SOA should be used, but database inefficiency, team scalability issues, deployment pains and unmanageable complexity are all key indicators. Smaller teams might want to use SOA in hopes of easier maintenance, faster iterations, more robust tests, and scalability.
In this article, I want to take an introductory look at the architecture of most monolithic Rails apps, compare it with a service-oriented architecture, and then implement a very basic service with Sinatra, Redis and RSpec. To demonstrate SOA, we’ll highlight a (fictitious) music application that connects people across the world with similar musical interests.
A Typical Rails App
Typical Rails apps rely heavily on one database. Usually, we cram all our primary data into the database and build the MVC stack on top. These apps have the following architecture:
This structure works great as apps grow into existence, but it has a number of long-term issues:
- Database Inefficiency. Unfortunately, the databases we’ve grown to love don’t scale. Millions of rows in a table can put a huge burden on the database, especially if that data is joined, manipulated and obfuscated. So we either optimize our queries or tack on a new technology to cache computed values or provide a new interface to the data (e.g. MapReduce). We’re patching the problem.
- Team Scalability. Working on one model often breaks other models. Too many teams are working on the same codebase and doing so has side effects. There’s no separation of concerns amongst the application; it’s all one repo.
- Deployment. You’re redeploying the entire application when small changes are made, potentially requiring downtime.
- Complexity. Since the data is so tightly wound, every new developer on the team has to learn the entire system. It becomes increasingly hard to find developers and the system is much more difficult to understand.
SOA Approach
A Service Oriented Architecture tries to alleviate the issues with monolithic Rails apps. It would be unwise to jump into an SOA when creating new apps, as the nature of the architecture can often add unwanted complexity. It’s usually a late player in the game and becomes useful when a Rails app grows too large. The goal is to take a large application and revive it by breaking down its logical components into individual services. Each service can be relied on atomically to carry out a set of tasks for its respective domain. Compared to the monolithic Rails architecture, here is how a SOA application might look:
Each service has its own database, its own domain logic, and its own HTTP interface. The above diagram displays three different services. These services are then combined into the high-level application which serves the end user. A portion of the domain logic still lives in the high-level application but much of it has been abstracted into services. The high-level application’s primary job is to join data from the services into something which can be rendered to the user.
Reconstructing an application into services can provide a handful of improvements:
- Databases are smaller. Each service can choose what datastore fits its needs. Typically, the service data will be a subset of the original data and stored in a relational database. However, if the service will experience high reads and writes, maybe an in-memory store like Redis will provide faster results. Regardless of the datastore, the complexity of the data is reduced.
- Teams can now work independently. Each service becomes its own deployable app. This means each team can be assigned an individual service and maintain their codebase without disrupting the adjacent teams.
- Each codebase can be deployed in solitude. As long as the HTTP interface remains constant, services can be deployed quickly and rapidly without affecting the rest of the application.
- Complexity has been broken down. Each service is a well-defined, simple way to access data and logic around a specific domain. New developers can work on one service without needing to understand the others. They can also become familiar with the codebase much quicker.
Defining Services
Let’s look at how the social music application can be broken into services. The goal of the application is to enable users to find people across the world with similar musical interests. This social app pulls artists from Last.fm, allows users to express interest in artists, and provides a list of users who like the same music.
The first question that comes to mind is, “How do I break my application into services?” Of course this depends on the application in question but there’s a few key guidelines to follow and a few options at hand.
First, consider constructing services around logical functionality. The music application’s core functionality has already been broken down above: fetch from Last.fm (Artist service), track users’ interest in artists (Interest service), and manage users (User service).
Next, consider where the application changes. Generally, you want your services to remain stable over time so it’s best to keep frequently-changing functionality in the high-level application, disjunct from the services themselves.
Consider the type of data and the frequency of read/write operations. Individual services can be optimized for one or the other, or both. Registering interest in an artist is a high-write function and aggregating similar users is a high-read function, both of which will exist in the Interest service.
Consider how often data is joined. If data is joined frequently, it’s possible this data should reside within the same service and handled in the database. If data is joined sparsely, it can be spread across multiple services and joined with Ruby in the high-level application.
For this example, we’ll define three services, one of which we’ll implement: the Artist service, the User service, and the Interest service. We’ll skip the User and Artist services and go straight to implementing the Interest service. Here’s how each service works:
- The Artist service is mostly comprised of a background worker to poll Last.fm for data. The Artist service stores artist names and their genres in a relational database.
- The User service uses a relational database and a typical User model. The service provides CRUD and authentication operations for users.
- The Interest service provides an interface to register a particular user’s interest in an artist. Given a user, it also provide a set of users with similar interest. To provide this set, we’ll store the join between Artist and User in a Redis SET. Redis SETs give us high read/write capacity and the ability to find intersections between many users’ musical interests.
Test the Interest Service
Writing tests first helps define how your HTTP service will look to the outside world. It’s also the most critical component to test in any SOA implementation. Don’t stub or fake the request to the service, test the real thing. At this point, we assume the User and Artist services have both been implemented. Let’s write some tests for the Interest service using RSpec.
spec/interest_spec.rb
[gist id=”1810537″]
For the sake of brevity, the Services::User and Services::Artist classes are assumed to be simple HTTP wrappers around the User and Artist services. They invoke similar POST requests to their respective HTTP endpoints to create the setup we need to test the Interest service.
This test makes a POST request to the Interest service to register a user’s interest in a particular artist. It creates a JSON string to send as the POST body. Finally, we check to make sure the response is OK.
Like any other HTTP service, it should be wrapped in a native Ruby library. I won’t cover creating the wrapper library but it should be implemented in a similar fashion as the test case above. REST Client, HTTParty, Faraday, ActiveResource and many other HTTP libraries could be used to build a Ruby wrapper.
Implementing Services
Sinatra is a perfect candidate for services. It furnishes a clean DSL and is easily maintained, provided the service remains simple. Let’s look at how we would implement the Interest service method tested previously.
service.rb
[gist id=”1810540″]
To implement this endpoint, we first initialize Redis (likely to be abstracted out of this block), parse the JSON body so we have access to the user and artist IDs, then register the user’s interest in Redis. The key for our Redis set is a combination of the “artists:” namespace and the artist’s id. Similarly, we track all the artists a particular user is interested in. After a number of users have expressed interest in an artists, the Redis sets will look like the following:
[gist id=”1810560″]
I won’t go through creating the service endpoint to extract users with similar interests but it would use Redis intersections to determine this relationship:
[gist id=”1810577″]
To find similar users, we first find all the artists the given user is interested in, then map those to Redis keys. We then find the intersection between the Redis keys (using the splat operator) which provides us with an array users’ IDs who are interested in the same artists.
Wrapping It Up
A Service Oriented Architecture can dramatically change the way an application is managed and developed. It can provide a nice separation based on business and engineering needs. It’s not without its faults, however. Migrating to a SOA can add overall complexity while simplifying individual components. Its vast decoupling can make data joins difficult, force normally arbitrary decisions to become complicated, and potentially become unwieldy. Uncle Bob Martin recently wrote an article in this vein: Service Oriented Agony.
The benefits can certainly outweigh the costs in many situations. Before taking the dive into transitioning your app to a Service Oriented Architecture, continue researching best practices around service segregation.
We’ve only scratched the service of SOA. Service communication, authentication, testing best practices and caching are all good follow-up concepts. This article should be enough to get even the Little Guys moving towards maintainable, scalable solutions.
Happy architecting!
SOA image via Shutterstock
Mike PackHi, I'm Mike Pack. I'm a freelance software engineer based in beautiful Denver, Colorado. I work primarily in Ruby and JavaScript to create highly dynamic applications. I research and write about architecture, testing and software patterns. I'm highly interested in solving team and software scalability problems, code maintainability ailments and performance bottlenecks.
