Monitoring tools are essential today in keeping the infrastructure on its toes so as to cater to the demand. Old monitoring tools, which functioned well in silos and were fit to purpose still faces problems of an integrated view of infrastructure and application. They fall short when faced with new architecture styles of Microservices and Containers and they grind to a halt when faced with elastic infrastructure requirements such as a cloud, containers or hybrid deployment.
The challenges with current state of monitoring infrastructure are many, and the interaction of these challenges with each other make the situation a gridlock.
One of the foremost challenges lies in visualization of data being monitored. Cross platform metrics correlation between application, container and virtual machine are virtually impossible in a typical setup where an organization has multiple monitoring tools. Furthermore, lack of availability of a metrics data model which supports slicing and dicing of the data across multiple dimensions, affects this aspect, which is the need of the day for Microservices architecture.
Usability and user experience is another challenge faced while using contemporary monitoring tools. Large number of metrics databases and dashboards make the collection of tools unwieldy in next generation loads of cloud and container. They are also not easy to setup and may not have a decentralized architecture. This takes away the flexibility to incrementally add or remove the applications being monitored. Furthermore, older solutions scrape all the metrics from a system using single agent. This is very inefficient considering the next generation of elastic loads.
Adding to the above challenges, lack of capability to run queries on the time series data and the inability to both statically and dynamically onboard various entities such as applications, nodes and containers affect proactive management which needs to arise out of monitoring.
A panacea to all these challenges is the need of the hour - but what would be the guiding principles of building such a solution? The future in this case, belongs to an architecture driven by the core principle of being flexible and scalable. A user is not tied to a closed solution, but is provided the flexibility to develop their own monitoring agents, add custom data to metrics, develop new visualization & finally export the metrics data to your favorite systems. A solution guided by this architecture should be able to expose an API which can be consumed by a rich Graphical User Interface to plot dashboards, list availability status and have more functions as need arises.
This architecture's structure can be based on five critical pillars. They are Collection of data, Aggregation of data, Visualization, Alerts/Notifications and Exportation. Many of the challenges mentioned in the beginning of the article are remedied by this approach, and should be present in the ideal next generation monitoring tool.
The challenges faced by visualization are taken care of a well thought out UI which consumes the API exposed by the solution and provides a single integrated dashboard and a single metrics store for the solution. It should also provide a single set of APIs for monitoring host, virtual machines, containers and applications. The five pillars also improve the usability in an ideal solution, which should come packaged with scripts that make it easy to set up on a simple workstation, and also have a rich set of APIs for on-boarding, catalogue, labels, availability, metrics, dashboard and exporter.
For tackling challenges brought on by next-gen technologies such as containers, support must be extended to agents for host, containers and applications. Next generation of elastic infrastructure is continuously moving hence it is imperative to have a solution which can continuously discover new deployment units, which requires a service discovery mechanism. Capability to statically as well dynamically on-board nodes, containers and applications and an out of the box support for regular expressions which can be utilized by end user to get information from a range of servers, containers and applications lend a futuristic outlook to the solution.
Capability to correlate metrics across nodes, VMs, containers and applications, capacity planning and proactive monitoring, ability to generate alerts, notifications and reports are must haves in a monitoring solution of such caliber, and any solution with all the above capabilities will surely see widespread adoption. Today, Wipro is proud to be building a tool that satisfies many of these requirements and along with integrations with best-of-breed solutions addresses the needs of enterprises in the present and future.