The second article in “Logicify Monitoring Tools” series talks about Graylog, a software we use to manage the logs and analyze structured data received from web applications.
7 min read
At Logicify, we take software monitoring seriously — both while application development and testing and once they are released in production. We usually implement a double-sided system of monitoring (user behavior + performance and technical state) in software released for clients and used internally. This monitoring system has proved effective over years due to three monitoring pillars: Graylog, Grafana, and Google Analytics. Combined with a few other tools and some custom coding, they give us (and our clients) a clear picture of how the whole system and its separate components work in every given time period.
You can find more about Grafana, an open source data visualization tool, in this article. This article focuses on Graylog, an open source software we use for logs storage.
- What is Graylog?
- How to use Graylog when developing apps
- How to use Graylog for publicly released apps
- Advantages of Graylog
- Comparison of Graylog+Grafana to ELK stack
What is Graylog?
As every software system “talks” in logs, they must be constantly monitored. However, the logs are often scattered across multiple servers, and, as the data volume grows, management of the logs becomes more and more time-consuming. To address these challenges, we opted for Graylog, a powerful open source platform for management of both structured and unstructured data along with debugging applications. It is based on Elasticsearch, MongoDB, and Scala. Graylog has a main server, which receives data from its clients installed on different servers, and a web interface, which visualizes the data and allows to work with logs aggregated by the main server.
We use Graylog primarily as the stash for the logs in web applications we build. However, it is also effective when working with raw strings (i.e. syslog): the tool parses it into the structured data. It also allows advanced custom search in the logs using structured queries. In other words, when integrated properly with a web app, Graylog helps engineers to analyze the system behavior on almost per code-line basis.Graylog/Grafana Dashboard Example
How We Use Graylog
The main advantage of Graylog is that it provides a perfect single datastorage for application logs and structured analytical data. This comes in handy if the system infrastructure is large and complex — distributed around multiple places, when not all team members could have immediate access to all components. With Graylog, we tackle these issues and ensure rapid incident response time.
At Logicify, Graylog serves both the applications under development and the ones already released publicly. In both cases, some modes of Graylog application are unique, while some intersect.
Graylog use in development, testing, and staging phases
In non-production environments, Graylog is primarily used as logs storage, making it easy to navigate in logs and structured analytical data and find any information a developer could be interested in — a time-stamp or duration of a web-request, exception or error stack trace.
As Graylog consistently stores all the logs of an application, it allows to track the system’s state for every specific moment of time. This gives developers an efficient mechanism to understand the context of any error and replicate the issue locally (if needed) to fix the bug. Thus, Graylog becomes a powerful helper for the developers to optimize the code and improve the application’s performance.
Graylog use in production phase
In software products that are already released publicly, Graylog is also applied for log storage. However, unlike the case with apps under development, in released apps, these logs are used primarily for the purposes of maintenance and incident response. This is made possible with notifications configured for specific events in the web application, e.g. 5xx errors or performance counter hits. These notifications could be sent via email or chat message every time an issue occurs. Support managers immediately learn what happened from the system itself and react proactively, even before users face and report the issues. Dev people, in turn, could debug the application and fix it ad-hoc in a timely manner.
Moreover, Graylog is crucial for business decisions regarding specific features in a software product. The tool collects custom analytics on user behavior in the system and visualizes the data in the form of pie charts, time bar graph and other graphics. For instance, one could measure the average response time of an application components, or determine the time interval when maximum amount of orders was placed on an eCommerce platform. Based on the data received, product stakeholders could make the decisions on further scaling the application, adding or removing some pieces of functionality. We use Graylog as a collateral tool for data visualization as Grafana is more sophisticated in this regard.Graylog/Grafana Dashboard Example
Interested in setting up a powerful system of monitoring for your web application? We can help! Contact us using the contact form below for any inquiries.
Advantages of Graylog
There are a few advantages of Graylog we have noticed so far, and these are what make the tool perfectly fit our workflow and delivery process.
- Graylog is a free open source software.
- Its trigger actions or notifications immediately inform us when something needs attention, so we constantly keep abreast of the system performance.
- With error stack traces received from Graylog, engineers understand the context of any issues in the source code. This saves time and efforts for debugging/troubleshooting and bug fixing.
- The tool has a powerful search syntax, so it is easy to find exactly what you are looking for, even if you have terabytes of log data. Plus, the search queries could be saved.
- Graylog offers an archiving functionality, so everything older than 30 days could be stored on slow storage and re-imported into Graylog when such a need appears (for example, when the dev team needs to investigate a certain event from the past).
- Python applications could be easily connected with Graylog with an out-of-box library.
Advantages of Graylog+Grafana Compared to ELK Stack
Graylog server (the application and web interface), combined with MongoDB and Elasticsearch as well as Grafana — in our case, is often compared to the so-called ELK stack (Elasticsearch, Logstash, and Kibana). Though both solutions are pretty similar in terms of feature-set, there are a few distinctive differences.
The most important distinction between the two lies in the fact that, from the very beginning, Graylog is positioned as a powerful logging solution, while ELK toolset is a Big Data solution. Graylog can receive structured logs and standard syslog directly from an application through the network protocol. On the contrary, ELK is the solution that analyzes already collected plain text logs using Logstash and then parses and passes them to ElasticSearch.
In ELK, Kibana plays the role of a dashboard to display the data from Logstash. Graylog in this sense is more convenient as it offers a single-application solution (excluding ElasticSearch as a flexible data storage) with almost the same functionality. Besides, the time needed to deploy a usable solution is smaller. Graylog has a friendlier GUI right out of the box and superior permissions system compared to ELK. As Elasticsearch fans, we prefer Graylog to ELK stack as it perfectly meets our needs for log managing.
Graylog is an important component in Logicify double-sided software monitoring system. Combined with Grafana, this Elasticsearch-based tool is responsible for logging and monitoring the technical state of our web applications. For both non- and production environments, Graylog is a nice single silo for the logs. The tool has an intuitive GUI and offers alerting, reporting and custom analyzing features. Most importantly, it brings terabytes of data across multiple log sources and geographies. Based on these advantages, we preferred Graylog over another popular stack with a similar functionality, ELK.
Over the years, Graylog proved effective and user-friendly, so we include it into the standards delivery kit for our web products. We deploy Graylog in the same environment where the web application is hosted, so all the data remain confidential and secured. Interested in more details about or monitoring solution? Drop us a line in the contact form below.
Inquiries about Logicify software monitoring system?
Thank you for contacting us! We would get back to you shortly (usually, within 24 hours on weekdays and 48 hours over weekends).
- Grafana as Yet Another Tool for Technical Monitoring of Software Products
- Logicify Double-Sided System of User Behavior and System Condition Monitoring
- Datawise Recipe for Effective Software Monitoring: App Logs, Infrastructure Telemetry, Health-Check Reports