Getting Started with Web Application Monitoring

Share this article

This article was sponsored by Monitis. Thank you for supporting the sponsors who make SitePoint possible.
For better or for worse, our jobs as developers don’t end with that last line of code, the final commit or by hitting the “deploy” button. Even the best-engineered web application isn’t bullet-proof, the most expensive hosting environments still aren’t one hundred percent reliable; and, ultimately, there’s always something that can go wrong. We can plan for failure, have processes in place in the event of a problem arising and even contingencies for a genuine disaster, but in between we can monitor. Monitoring allows us to be reactive; to take action in the event of a problem, as well as proactive; taking preventative action before an issue arises. In this article we’re going to take a look at monitoring, in the context of a website or a web application. Along the way we’ll be taking a detailed look at Monitis, an all-in-one monitoring platform that is one of the leaders in its field, and how it can help make sure that once you’ve launched your app, it remains running and keeps performing.

What can go wrong?

In order to know how and what we should monitor, it helps to have an understanding of what could potentially go wrong. The short answer is probably “well, a lot” — and as such, it’s a really tough question to answer definitively. At the same time, though, there are a range of things that we can anticipate might go wrong. Broadly speaking, we can divide these issues into a number of categories:
  • Hitting “hard” limits; for example a full disk, hitting a physical memory limit or reaching the maximum number of processes
  • Network issues; for example a site becoming unreachable, high packet losses, server connections going down or DNS failures
  • Component or service failures; perhaps your database server has gone down, for example.
  • Problems with third-party services; your S3 bucket is unreachable, your mail provider is experiencing issues, or your CDN has gone down
  • Problems with your applications; errors and exceptions, inconsistencies in your data or even bugs in your code
  • Keeping third-party code or operating system components up-to-date; in particular checking for security patches or service packs
  • There are even silly examples of human error which, alas, do still happen; such as forgetting to renew a SSL certificate.
Once you have an idea of what can go wrong, you start to get a feel for what to monitor.

Important Things to Consider

Whilst a lot of monitoring deals in metrics such as server response times, the amount of available memory or the level your CPU is running at, there’s nothing quite as important as the experience your users are having. In all likelihood there will be some correlation between the raw data from your server or application and the wait times your user faces — for example, high network latency probably means slow response times and therefore users waiting around for your application to load — it’s always worth seeing your service through their eyes. To that end, there are a number of tools you can use. But first, let’s look at a few things we need to consider. There are a range of factors which can make a website or application perform differently depending on the user, their device and their circumstances.

Are they using a desktop / laptop or a mobile?

It probably goes without saying that mobile users often face a very different experience to those on fixed or desktop machines. Nevertheless, it’s worth monitoring the extent to which the performance does vary.

What kind of network connection do they have?

As well as mobiles generally performing worse than desktop in terms of hardware and network constraints, network connections also vary enormously — not everyone has high-speed fiber-optic based broadband connections, for example.

Where are they?

At some stage in a project, you’ll probably need to decide where your server — or servers — will be physically located. Whilst some websites or web applications target a very specific geographical location — for example a UK-only online store, a US high street chain or a Tokyo transportation hub — the Web is global, which means people can or will access it from over the World. As a rather oversimplified rule of thumb, the further a visitor is from your servers, the longer it’ll take to travel “over the wires”. It’s worth taking a look at where your visitors are based — the results might surprise you. For example technology blogs are often hugely popular in India — but have you considered how, say, a US-based server copes with the distance?

Monitoring, Data and Alerts

There are two aspects to monitoring; viewing data, be it historical (“Show me average response times and HTTP Status codes 4xx in the last 24 hours”) or in real-time, and also alerts. For example, you might opt to receive an email if the amount of free disk space on one of your servers drops below a certain threshold, so that you can take pro-active action before it becomes full. Alternatively, you can set up an alert for when something goes wrong. For example, you could opt to receive a push notification or an alert via Zapier if your website becomes inaccessible. Deciding the parameters for alerts is a tricky business. How do you know, for example, how high your CPU load should reach before it’s worth sending you an e-mail? A good way to answer this might be to cross-reference various parameters. For example, by monitoring response times and
CPU load, you might find that response times become unacceptably low if your CPU load goes above 80%. There is a difficult balance to be struck. If alarms are too frequent then it’s easy to become de-sensitized to them. Too few, and chances are you have users somewhere who have become frustrated, servers which have gone down or a site that’s become inaccessible and you just don’t know about it. Learning to achieve this balance is a skill in itself.

Introducing Monitis

Monitis is a service for capturing all sorts of data about the health of your servers, the performance of your web and database servers, the responsiveness of your web application and much more. It provides a real-time view of this data through a web interface — your dashboard — which is illustrated below. Monitis dashboard Additionally it allows you to assess the performance of a website or web application from the viewpoint of your users, whatever device they’re using or wherever they are in the World. You can set up alerts to let you know if something goes wrong. Let’s take a quick look at the options.

Server Monitoring with the Monitis Smart Agent

In order to monitor the health of a server with Monitis, we need to install the agent software. This is a small program which runs in the background as a daemon, monitoring the general health of the server and reporting that data back to Monitis so that we can make use of it. See the Linux or Windows documentation for instructions on how to install it. If you’d like to try it out quickly, I’d recommend creating a new Ubuntu droplet with Digital Ocean. You should find that it works out-of-the-box, with all of its dependencies already installed for you. Alternatively you may wish to check out this Chef cookbook. Once the agent software is installed and running, you’ll need to set up monitoring via the Monitis dashboard on the Web. We’ll look at that next.

Application Monitoring

Monitis also allows you to monitor the health of your applications. Monitis provide SDKs for most popular languages – Java, Perl, Python, PHP, Ruby, C# — popular web and application servers such as Apache, Nginx, IIS, Tomcat and Node.js, various database servers such as MySQL, Postgres, MongoDB and SQL Server, and more. You’ll find a comprehensive list here.

Exploring the Monitis Dashboard

The Monitis dashboard is made up of a series of panels showing various types of data depending on what monitoring and alerts you’ve set up. The first time you access your dashboard, it provides a number of prompts and some inline help. Setting up monitoring for data like CPU load, remaining disk space, free memory and system load, as well for alerts, takes just a few minutes. Once you’re set up, you can manipulate the panels by dragging and dropping to arrange them, resizing them or indeed showing and hiding them. Using the drop-down (context) menu at the top-right of each panel you can also maximize them for a better view. From a panel’s context menu, you can also export monitoring data to a number of formats such as PSD or Comma Separated Values (CSV), as well as print it or send it via e-mail. The settings menu item allows you to tweak the parameters, which vary according to the specific metric being monitored. For example you can set various thresholds for monitoring CPU levels, the level of free memory, the frequency of HTTP requests to attempt, and so on. You can also create new “pages” (tabs), allowing you to divide up your monitors by whatever criteria is most appropriate to your needs. For example, you could set up a tab per-server, with key metrics across all of your services on one “front page”, or summary tab.

Other Tools for Monitoring

There are a number of tools you can use to further understand how your web application is performing or how it might react to certain events.

Full Page Load Monitors

Many web monitoring systems will provide you with response times; for example, “how long does a particular GET request take?”. As useful as that might be, this doesn’t provide the whole picture. Full Page Load Monitors go one step further, giving you an indication of how long a page takes not just to download, but to render in a real browser. This gives you a better understanding of how your website or application is performing from a user’s perspective, not just in terms of HTTP response times. In order to better understand how it affects the user experience, full page monitors often provide Apdex (Application Performance Index) scores. These are less concerned with hard numbers — although they do provide a numeric value between 0 and 1 — but rather, a qualitative measure of quality of experience for real users — for example “Satisfied”, “Tolerating” or “Frustrated”.

Web Stress Testing

Web stress testers allow you to simulate a large and configurable number of requests over a specific duration, to see how your web server copes. This might be useful for trying to determine how your server might handle “The Slashdot effect (which you might also refer to as the Twitter effect, or the Reddit effect) — in other words, how might your web server cope if you suddenly receive a high volume of visits over a short period of time? To give you an idea of what you can expect, you’ll find a sample report right here.

Website Security Scans

It’s also vitally important to monitor security vulnerabilities and, in extreme cases, for injection of malware. You’ll find the option to run a vulnerability scan in the Tools tab, and you’ll find a sample of the output here.


When you have a crisis, you need to have the right information to fix the problem. But even when things are going well, a steady stream of useful data is necessary for understanding what can go wrong with your system and infrastructure, and so where your development priorities should be. The only way to take the right action is with relevant, useful and accurate data, and for that you need a strong monitoring platform like Monitis. The service can support a large range of useful metrics for monitoring the health of your servers, application and network. It allows you to see the state of your services in real-time, examine historical data to help understand them, and alert you when it’s time to take action. Keen to give Monitis a try? Sign up for their 15-day trial today.

Frequently Asked Questions (FAQs) on Web Application Monitoring

What are the key features to look for in a web application monitoring tool?

When choosing a web application monitoring tool, there are several key features to consider. Firstly, the tool should provide real-time monitoring and alerting capabilities to quickly identify and resolve issues. Secondly, it should offer detailed performance metrics, including response time, throughput, error rates, and server CPU usage. Thirdly, the tool should support multiple platforms and languages to ensure compatibility with your web application. Lastly, it should have an intuitive user interface and robust reporting capabilities for easy data analysis and decision-making.

How does web application monitoring improve user experience?

Web application monitoring plays a crucial role in improving user experience. By continuously monitoring the performance of a web application, you can identify and fix issues before they impact users. This ensures that your application remains responsive and reliable, leading to a positive user experience. Additionally, monitoring tools can provide insights into user behavior, helping you optimize your application based on user needs and preferences.

Can web application monitoring help in optimizing server resources?

Yes, web application monitoring can significantly help in optimizing server resources. By monitoring server utilization, you can identify underutilized or overutilized servers and adjust resources accordingly. This not only improves application performance but also reduces costs associated with unnecessary server capacity.

How does web application monitoring aid in troubleshooting?

Web application monitoring aids in troubleshooting by providing detailed insights into application performance. It can identify issues such as slow response times, high error rates, and server downtime. By analyzing these metrics, you can pinpoint the root cause of issues and implement effective solutions. Additionally, some monitoring tools offer predictive analytics, which can help you anticipate and prevent future issues.

What is the role of web application monitoring in DevOps?

In a DevOps environment, web application monitoring plays a critical role in ensuring continuous delivery and integration. It provides real-time feedback on application performance, helping teams identify and fix issues early in the development cycle. This leads to faster, more reliable releases and improved collaboration between development and operations teams.

How does web application monitoring support scalability?

Web application monitoring supports scalability by providing insights into application performance and user behavior. By analyzing these metrics, you can identify bottlenecks and make necessary adjustments to support increased traffic. Additionally, monitoring tools can help you plan for future growth by predicting trends and capacity needs.

Can web application monitoring tools detect security threats?

Yes, some web application monitoring tools can detect security threats. They can identify suspicious activity, such as unusual traffic patterns or attempts to access sensitive data, and alert you in real-time. This allows you to respond quickly and mitigate potential security risks.

How does web application monitoring contribute to business growth?

Web application monitoring contributes to business growth by ensuring optimal application performance, which leads to a positive user experience and increased customer satisfaction. By identifying and resolving issues quickly, you can maintain high service levels and build trust with your customers. Additionally, monitoring tools can provide valuable insights into user behavior, helping you make data-driven decisions to improve your product and drive business growth.

What is the difference between web application monitoring and website monitoring?

While both web application monitoring and website monitoring involve tracking performance, they focus on different aspects. Website monitoring primarily focuses on the availability and uptime of a website, while web application monitoring goes a step further to monitor the performance of specific application functions, such as login, search, and checkout processes.

How can I choose the right web application monitoring tool for my business?

Choosing the right web application monitoring tool depends on several factors, including your business needs, budget, and technical capabilities. Consider tools that offer real-time monitoring, detailed performance metrics, multi-platform support, and an intuitive user interface. Additionally, look for tools with robust reporting capabilities and good customer support. It may also be beneficial to take advantage of free trials or demos to evaluate different tools before making a decision.

Lukas WhiteLukas White
View Author

Lukas is a freelance web and mobile developer based in Manchester in the North of England. He's been developing in PHP since moving away from those early days in web development of using all manner of tools such as Java Server Pages, classic ASP and XML data islands, along with JavaScript - back when it really was JavaScript and Netscape ruled the roost. When he's not developing websites and mobile applications and complaining that this was all fields, Lukas likes to cook all manner of World foods.

Monitismonitoring web applicationssponsoredWeb applicationsweb security
Share this article
Read Next
Get the freshest news and resources for developers, designers and digital creators in your inbox each week