Table of Contents
While microservices are all the hype, notable experts warn against starting out that way. Instead you might want to build a modular monolith first, a safe bet if consider going into microservices later but do not yet have the immediate need. This article shows you how to build a lean monolith with OSGi, modular enough to be split into microservices without too much effort when that style becomes the appropriate solution for the application’s scaling requirements.
A very good strategy for creating a well-modularized solution is to implement domain driven design (Eric Evans). It already focuses on business capabilities and has the notion of bounded contexts that provide the necessary modularization. In this article we will use OSGi to implement the services as it provides good support for modules (bundles) and lightweight communication between them (OSGi services). As we will see, this will also provide a nice path to microservices later.
This article does not require prior knowledge of OSGi. I will explain relevant aspects as we go along and if you come away from this article with the understanding that OSGi can be used to build a decoupled monolith in preparation for a possible move towards microservices, it achieved its goal. You can find the sources for the example application on GitHub.
Our Domain: A Modular Messaging Application
To keep the business complexity low we will use a rather simple example – a chat application. We want the application to be able to send and receive broadcast messages and implement this in three very different channels:
- shell support
- irc support
- IoT support using Tinkerforge based display and motion detector
Each of these channels uses the same interfaces to send and receive messages. It should be possible to plug the channels in and out and to automatically connect them to each other. In OSGi terms each channel will be a bundle and use OSGi services to communicate with the other channels.
Don’t worry if you do not have Tinkerforge hardware. Obviously the Tinkerforge module will then not work but it will not affect the other channels.
Common Project Setup and OSGi Bundles
The example project will be built using Maven and most of the general setup is done in the parent pom.
OSGi bundles are just JAR files with an enhanced manifest that contains the OSGi specific entries. A bundle has to declare which packages it imports from other bundles and which packages it exports. Fortunately most of this happens automatically by using the bnd-maven-plugin. It analyzes the Java sources and auto-creates suitable imports. The exports and other special settings are defined in a special file bnd.bnd
. In most cases this file can be empty or even left out.
The two plugins below make sure each Maven module creates a valid OSGi bundle. The individual modules do not need special OSGi settings in the pom – for them it suffices to reference the parent pom that is being built here. The maven-jar-plugin defines that we want to use the MANIFEST
file from bnd instead of the default Maven-generated one.
<build>
<plugins>
<plugin>
<groupId>biz.aQute.bnd</groupId>
<artifactId>bnd-maven-plugin</artifactId>
<version>3.3.0</version>
<executions>
<execution>
<goals>
<goal>bnd-process</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.5</version>
<configuration>
<archive>
<manifestFile>
${project.build.outputDirectory}/META-INF/MANIFEST.MF
</manifestFile>
</archive>
</configuration>
</plugin>
<!-- ... more plugins ... -->
</plugins>
</build>
Each of the modules we are designing below creates an OSGi bundle. The poms of each module are very simple as most of the setup is already done in the parent, so we omit these. Please take a look at the sources of the OSGi chat project to see the details.
Declarative Services
The example uses Declarative Services (DS) as a dependency injection and service framework. This is a very lightweight system defined by OSGi specs that allows to publish and use services as well as to consume configuration. DS is very well-suited for OSGi as it supports the full dynamics of OSGi where bundles and services can come and go at any time. A component in DS can offer an OSGi service and depend on other OSGi services and configuration. Each component has its own dynamic lifecycle and will only activate when all mandatory dependencies are present. It will also dynamically adapt to changes in services and configuration, so changes are applied almost instantly.
As DS takes care of the dependencies the developer can concentrate on the business domain and does not have to code the dynamics of OSGi. As a first example for a DS component see the ChatBroker
service below. At runtime DS uses XML files to describe components. The bnd-maven-plugin automatically processes the DS annotations and transparently creates the XML files during the build.
The Chat API
In our simple chat domain we just need one service interface, ChatListener
, to receive or send chat messages. A ChatListener
listens to messages and modules that want to receive messages publish an implementation of ChatListener
as an OSGi service to signal that they want to listen. This is called the whiteboard pattern and is widely used.
public interface ChatListener {
void onMessage(ChatMessage message);
}
ChatMessage
is a value object to hold all information about a chat message.
public class ChatMessage implements Serializable {
private static final long serialVersionUID = 4385853956172948160L;
private Date time;
private String sender;
private String message;
private String senderId;
public ChatMessage(String senderId, String sender, String message) {
this.senderId = senderId;
this.time = new Date();
this.sender = sender;
this.message = message;
}
// .. getters ..
}
In addition we use a ChatBroker
component, which allows to send a message to all currently available listeners. This is more of a convenience service as each channel could simply implement this functionality on its own.
@Component(service = ChatBroker.class, immediate = true)
public class ChatBroker {
private static Logger LOG = LoggerFactory.getLogger(ChatBroker.class);
@Reference
volatile List<ChatListener> listeners;
public void onMessage(ChatMessage message) {
listeners.parallelStream().forEach((listener)->send(message, listener));
}
private static void send(ChatMessage message, ChatListener listener) {
try {
listener.onMessage(message);
} catch (Exception e) {
LOG.warn(e.getMessage(), e);
}
}
}
ChatBroker
is defined as a declarative service component using the DS annotations. It will offer a ChatBroker
OSGi service and will activate immediately when all dependencies are present (by default DS components are only activated if their service is requested by another component).
The @Reference
annotation defines a dependency on one or more OSGi services. In this case volatile List
marks that the dependency is (0..n)
. The list is automatically populated with a thread safe representation of the currently available ChatListener
services. The send
method uses Java 8 streams to send to all listeners in parallel.
In this module we need a bnd.bnd
file to declare that we want to export the API package. In fact this is the only tuning of the bundle creation we do in this whole example project.
Export-Package: net.lr.demo.chat.service
The Shell Module
The shell channel allows to send and receive chat messages using the Felix Gogo Shell, a command line interface (much like bash) that makes for easy communication with OSGi. See also the appnote at enroute for the Gogo shell.
The SendCommand
class implements a Gogo command that sends a message to all listeners when the command send <msg>
is typed in the shell. It announces itself as an OSGi service with special service properties. The scope and function define that the service implements a command and how the command is addressed. The full syntax for our command is chat:send <msg>
but it can be abbreviated to send <msg>
as long as send
is unique.
When Felix Gogo recognizes a command on the shell, it will call a method with the name of the command and send the parameter(s) as method arguments. In case of SendCommand
the parameter message is used to create a ChatMessage
, which is then sent to the ChatBroker
service.
@Component(service = SendCommand.class,
property = {"osgi.command.scope=chat", "osgi.command.function=send"}
)
public class SendCommand {
@Reference
ChatBroker broker;
private String id;
@Activate
public void activate(BundleContext context) {
this.id = "shell" + context.getProperty(Constants.FRAMEWORK_UUID);
}
public void send(String message) {
broker.onMessage(new ChatMessage(id, "shell", message));
}
}
The ShellListener
class receives a ChatMessage
and prints it to the shell. It implements the ChatListener
interface and publishes itself as a service, so it will become visible for ChatBroker
and will be added to its list of chat listeners. When a message comes in, the onMessage
method is called and simply prints to System.out
, which in Gogo represents the shell.
@Component
public class ShellListener implements ChatListener {
public void onMessage(ChatMessage message) {
System.out.println(String.format(
"%tT %s: %s",
message.getTime(),
message.getSender(),
message.getMessage()));
}
}
The IRC Module
This module uses Apache Camel to connect to an IRC channel, to which messages are sent and received from. The IRCConnector
uses type safe configuration as defined in the OSGi metatype spec 1.3. This allows to define names, types, and defaults for configuration values. At run time these are fed by the Felix config admin and configured through .cfg
files in property syntax. The configuration is given to the component in the activate
method.
You might notice that there are two @Reference
dependencies to the Camel components irc
and bean
which are not directly used in the code below. This is a kind of workaround to make sure we wait until the components are active as Apache Camel is not fully integrated with DS.
@Component(name = "connector.irc", immediate = true)
public class IRCConnector implements ChatListener {
@Reference(target="(component=irc)")
org.apache.camel.spi.ComponentResolver irc;
@Reference(target="(component=bean)")
org.apache.camel.spi.ComponentResolver bean;
private OsgiDefaultCamelContext context;
@Reference
private ChatBroker broker;
private ProducerTemplate producer;
private String ircURI;
@ObjectClassDefinition(name = "IRC config")
@interface TfConfig {
String nick() default "osgichat";
String server() default "193.10.255.100";
int port() default 6667;
String channel() default "#osgichat";
}
@Activate
public void activate(BundleContext bc, TfConfig config) throws Exception {
context = new OsgiDefaultCamelContext(bc, new OsgiServiceRegistry(bc));
ircURI = String.format(
"irc:%s@%s:%d/%s",
config.nick(),
config.server(),
config.port(),
config.channel());
context.addRoutes(new RouteBuilder() {
public void configure() throws Exception {
from(ircURI).bean(new ChatConverter()).bean(broker);
}
});
context.start();
producer = context.createProducerTemplate();
}
@Deactivate
public void deactivate() throws Exception {
context.shutdown();
}
public void onMessage(ChatMessage message) {
if (!"irc".equals(message.getSenderId())) {
try {
producer.sendBody(ircURI, message.getMessage());
} catch (CamelExecutionException e) {
System.out.println(e.getMessage());
}
}
}
}
Once the dependencies and configuration are present, the activate
method is called by OSGi. Therein a Camel context with one route for receiving IRC messages using the Camel IRC component is instantiated. This route is defined by from(ircURI)
, where ircURI
defaults to "irc://osgichat@193.10.255.100:6667/#osgichat"
and opens a connection to the given IRC server and channel and will be called for each message received from the channel. Messages are piped into a ChatConverter
, which converts them to our ChatMessage
type and then sent to the ChatBroker
for delivery in our messaging system.
In the other direction we listen for ChatMessages
by offering the usual ChatListener
service. When a message arrives in onMessage
it is sent into another Camel route using the producerTemplate
. In this route the message is simply sent to the same IRC URI, which tells Camel to send an IRC message to the channel.
Tinkerforge Module
Let’s make our application a little more interesting by adding a channel that interacts with IoT devices. For this we use the Tinkerforge system. It allows to experiment with IoT without soldering and also offers a Java library which communicates with a brick daemon, so there is also no need to write native code.
The TinkerConnect
component creates and configures a Tinkerforge IPConnection, which talks to the brick daemon.
@Component(name = "tf",
configurationPolicy = ConfigurationPolicy.REQUIRE,
service = TinkerConnect.class)
@Designate(ocd = TinkerConnect.TfConfig.class)
public class TinkerConnect {
private static Logger LOG = LoggerFactory.getLogger(TinkerConnect.class);
private IPConnection ipcon;
@ObjectClassDefinition(name = "Tinkerforge config")
@interface TfConfig {
String host() default "localhost";
int port() default 4223;
}
@Activate
public void activate(TfConfig config) throws Exception {
ipcon = new IPConnection();
ipcon.connect(config.host(), config.port());
}
@Deactivate
public void deactivate() throws NotConnectedException {
ipcon.disconnect();
}
IPConnection getConnection() {
return ipcon;
}
}
LCDWriter
is another ChatListener
, which uses the TinkerConnect service to connect to an LCD display and writes all ChatMessage
s to the LCD. The full code also supports a message buffer and scrolling through the messages. Here in the article you can find the minimum code to write to the display.
@Component
public class LCDWriter implements ChatListener {
private BrickletLCD20x4 lcd;
private DateFormat df;
@Reference
TinkerConnect tinkerConnect;
@Activate
public void activate() throws TimeoutException, NotConnectedException {
this.df = DateFormat.getTimeInstance(DateFormat.MEDIUM, Locale.ENGLISH);
IPConnection ipcon = tinkerConnect.getConnection();
lcd = new BrickletLCD20x4("rV1", ipcon);
lcd.backlightOn();
lcd.clearDisplay();
lcd.addButtonPressedListener((button) -> buttonPressed(button));
}
public void onMessage(ChatMessage message) {
try {
initlcd();
lcd.clearDisplay();
lcd.writeLine((short)0, (short)0, df.format(message.getTime()));
lcd.writeLine((short)1, (short)0, message.getSender());
lcd.writeLine((short)2, (short)0, message.getMessage());
if (message.getMessage().length() > 20) {
lcd.writeLine((short)3, (short)0, message.getMessage().substring(20));
}
} catch (Exception e) {
// Ignore
}
}
}
As we have no full ASCII input device we use a motion detector to send a message whenever motion is detected.
@Component(immediate = true, service=MotionDetector.class)
public class MotionDetector {
@Reference
TinkerConnect tinkerConnect;
@Reference
ChatBroker broker;
private BrickletMotionDetector motion;
private MotionDetectedListener listener;
@Activate
public void activate() throws Exception {
IPConnection ipcon = tinkerConnect.getConnection();
motion = new BrickletMotionDetector("sHt", ipcon);
listener = () -> {
broker.onMessage(new ChatMessage("sensor", "sensor", "Motion detected"));
};
motion.addMotionDetectedListener(listener);
}
@Deactivate
public void deActivate() throws Exception {
motion.removeMotionDetectedListener(listener);
}
}
Packaging With bndtools
OSGi needs a separate declaration of the packaging for deployment. This is more involved than in a regular Java project as OSGi is all about loose coupling. The POMs of the individual modules often only depend on APIs, so they typically do not contain enough information to declare the full OSGi setup.
For the run-time packaging in OSGi we need a list of bundles as well as additional configuration for the container and the bundles. The list of bundles can be specified “by hand” but this is very tedious and error prone. A better way is to provide a list of candidate bundles in the form of an OSGi Bundle Repository (OBR) index and use the OSGi resolver to determine the actual bundles to be used.
In our case we define the index using a POM file. In this file we define Maven dependencies on bundles and other index POMs. Some OSGi projects like Aries RSA already provide such index POMs, which makes it very easy to add them. During the build, an OBR index is created using the bnd-indexer-plugin, that contains the meta data of all bundles. The most important part of this data is the list of requirements and capabilities of each bundles.
<plugin>
<groupId>biz.aQute.bnd</groupId>
<artifactId>bnd-indexer-maven-plugin</artifactId>
<version>3.2.0</version>
<configuration>
<localURLs>REQUIRED</localURLs>
</configuration>
<executions>
<execution>
<id>index</id>
<goals>
<goal>index</goal>
</goals>
</execution>
</executions>
</plugin>
To resolve the bundles we use bndtools. It is an Eclipse plugin for OSGi development and also supports Maven plugins for tasks like resolving bundles. In bndtools the deployment is defined by a bndrun
file. This file contains some general config like the OSGi framework to use as well as a pointer to an index and an initial list of requirements. The file can be created by hand or using the bndrun
Eclipse editor. The initial list of requirements is simply a list of top level bundles from the index.
-runrequires: \
osgi.identity;filter:='(osgi.identity=org.apache.felix.metatype)',\
osgi.identity;filter:='(osgi.identity=org.apache.felix.fileinstall)',\
osgi.identity;filter:='(osgi.identity=org.ops4j.pax.logging.pax-logging-service)',\
osgi.identity;filter:='(osgi.identity=org.apache.felix.gogo.command)',\
osgi.identity;filter:='(osgi.identity=org.apache.felix.gogo.shell)',\
osgi.identity;filter:='(osgi.identity=net.lr.demo.chat.command)',\
osgi.identity;filter:='(osgi.identity=net.lr.demo.chat.lcd)',\
osgi.identity;filter:='(osgi.identity=net.lr.demo.chat.irc)'
The first 5 requirements describe the basic infrastructure for config management and logging and the Gogo shell. For our actual application we only need to add the bundles for the channels. All their dependencies are resolved from the requirements of the bundles and the backing index.
The automatic resolve is then done by working from these initial bundles and adding all bundles that are needed to fulfill their requirements. The result is a list of runbundles that form a closure around the requirements and so define a valid deployment.
The All-in-one Packaging aka The Modular Monolith
In the project packaging-all we define a bndrun
file with the bundles of all channels. The resolve process will automatically add required libraries to support the channels like the camel-core and camel-irc bundles. As all channels are running in one process we do not need any remoting – the channels can communicate using plain OSGi services.
In this deployment there is almost no overhead in communication and debugging can be done like for any other java application. As this deployment is very easy to manage you should stay with it as long as feasible.
Running the Application
First we build the project – this will also create runnable JARs. Then we can launch the chat-all.jar
.
mvn clean install
cd packaging/all
java -jar target/chat-all.jar
This runs the all in one packaging. The chat-all.jar contains the OSGi framework as well as all the runtime bundles. Configuration is placed in the etc
directory.
The JAR should start without errors and spawn a Felix Gogo shell that shows the prompt. Parallel to the running application open the #osgichat channel on freenode irc. After a little while a user osgichat should join.
Now we first send a message from the OSGi application to IRC.
g! send Hi there
The message should show on the shell and the IRC channel.
Now send a message on the IRC channel. This message should likewise show on the Gogo shell. To end the process and shell type Ctrl-D
.
What Did We Learn?
During this article we learned how to build an application in a modular way without introducing the complexity of microservices right from the start. We defined individual bundles based on bounded contexts (in this case different chat technologies) and coupled them loosely with OSGi services. Then we created a single deployment unit (deployment monolith), which is easy to deploy and debug.
In the second article we will show how to split this application into two microservices and deploy these.
Frequently Asked Questions (FAQs) about Building a Lean, Modular Monolith with OSGi
What is the main advantage of using OSGi for building a lean, modular monolith?
The primary advantage of using OSGi (Open Service Gateway Initiative) for building a lean, modular monolith is its dynamic module system. This system allows applications or components, known as bundles in OSGi, to be remotely installed, started, stopped, updated, and uninstalled without requiring a reboot. This feature enhances the modularity of applications, making them easier to manage, maintain, and scale. It also reduces the complexity of the system, making it leaner and more efficient.
How does OSGi compare to other modular systems?
OSGi stands out from other modular systems due to its dynamic nature and service-oriented architecture. Unlike static module systems, OSGi allows for the dynamic installation, updating, and uninstallation of modules. This means that changes can be made to the system without requiring a restart, which can significantly reduce downtime. Additionally, OSGi’s service-oriented architecture promotes loose coupling between modules, enhancing the system’s flexibility and scalability.
What are the challenges of building a modular monolith with OSGi?
While OSGi offers many benefits, it also presents certain challenges. One of the main challenges is the learning curve associated with understanding and effectively using the OSGi framework. It requires a deep understanding of the OSGi specifications and best practices. Additionally, debugging in OSGi can be complex due to the dynamic nature of the system. Lastly, while OSGi promotes modularity, it requires careful design and planning to ensure that the system remains maintainable and scalable.
How does OSGi contribute to developer productivity?
OSGi can significantly enhance developer productivity by promoting modularity and reducing system complexity. By allowing for the dynamic management of modules, OSGi makes it easier for developers to work on individual components without affecting the entire system. This can reduce the time and effort required for debugging and testing. Additionally, OSGi’s service-oriented architecture promotes loose coupling, which can make the system easier to understand and modify.
How does a lean, modular monolith compare to a microservices architecture?
A lean, modular monolith and a microservices architecture both aim to break down a system into manageable components. However, they differ in how these components are deployed and interact with each other. In a lean, modular monolith, all components are part of a single deployable unit, while in a microservices architecture, each component is independently deployable. While a microservices architecture can offer greater flexibility and scalability, a lean, modular monolith can be simpler to manage and maintain, especially for smaller systems.
What is the role of the OSGi framework in a lean, modular monolith?
The OSGi framework plays a crucial role in a lean, modular monolith by providing the infrastructure for creating and managing modules. It provides a dynamic module system that allows for the remote installation, updating, and uninstallation of modules. This can make the system more manageable and maintainable. Additionally, the OSGi framework’s service-oriented architecture promotes loose coupling between modules, enhancing the system’s flexibility and scalability.
How can I get started with building a lean, modular monolith with OSGi?
To get started with building a lean, modular monolith with OSGi, you first need to understand the OSGi specifications and best practices. You can then start by designing your system’s architecture, identifying the modules and their interactions. Once the design is in place, you can use the OSGi framework to create and manage these modules. It’s also important to continuously test and refine your system to ensure it remains efficient and maintainable.
What are some best practices for building a lean, modular monolith with OSGi?
Some best practices for building a lean, modular monolith with OSGi include designing a clear and scalable architecture, promoting loose coupling between modules, and continuously testing and refining the system. It’s also important to effectively manage dependencies between modules to prevent issues such as circular dependencies. Lastly, it’s crucial to keep up-to-date with the latest OSGi specifications and best practices to ensure your system remains efficient and maintainable.
Can OSGi be used with other programming languages besides Java?
OSGi is primarily designed for use with Java. However, it can also be used with other JVM languages such as Kotlin and Scala. This is because these languages compile to JVM bytecode, which can be executed by the OSGi framework. However, using OSGi with non-JVM languages can be more challenging and may require additional tools or libraries.
What are some resources for learning more about OSGi and building a lean, modular monolith?
There are many resources available for learning more about OSGi and building a lean, modular monolith. The official OSGi website provides comprehensive documentation and tutorials. Additionally, there are many books, online courses, and tutorials available that cover OSGi and modular system design. It’s also helpful to join online communities or forums where you can ask questions and learn from others’ experiences.