Monitoring and management of blockchain networks

Introduction

Blockchain technology is based on a decentralized model, in which peers collaborate and build trust over a business network. Each peer organization can be represented by one or more nodes and this network of nodes is used to broadcast transactions and reach consensus for each transaction submitted. Each node is supposed to be self-sufficient with the ability to serve any of the distributed applications and/or existing enterprise applications. However, the health of each blockchain node and the entire blockchain network needs to be monitored to ensure truly decentralized and robust operations.

What kind of monitoring does blockchain need?

A typical blockchain network comprises of a set of interconnected nodes that act as peers. These nodes usually are hosted on cloud/on premise infrastructure where the blockchain runtime is set up natively on a virtual machine (VM) or by using containerization technologies such as Docker. Transactions submitted to the blockchain network are broadcast to all peers and the new blocks created are propagated, so that all peers have an updated copy of the shared ledger.

To gain insight to the block, its transaction related events and associated metadata, monitoring of any one of the peers is sufficient. And that is usually done by using blockchain explorer, which listens to the events and provides some visualization of the number of transactions received, queued, processed and finally grouped into a new block. However, this level of monitoring does not provide any clue to the usage of resources on that node or the health of other nodes or the latency experienced within the blockchain network.

Another key element that needs to be monitored to gain end-to-end visibility of a blockchain based solution is the off-chain components that comprises of the dApp (decentralized application) layer. The dApp layer comprises of user interface, storage and API (Application Program Interface) SDK (Software Development Kit) components, through which the interaction with a blockchain node is enabled.

Blockchain monitoring framework

Effective monitoring and management of a blockchain network requires a framework, which can integrate the data, assimilate the events generated and provide effective visualization of the blockchain related matrices. This framework should be modular and should support deployment topologies, which can enable monitoring at both an individual node level and the entire blockchain network level.

The diagram above depicts a proposed blockchain monitoring framework, which comprises of the following:

• A monitoring agent, which gets deployed on each blockchain node and associated dApp infrastructure, and can read the logs generated as a part of the transaction process and relay the CPU, memory and I/O usage data
• A log collection engine that handles the streaming log information and assimilates it for further processing
• The elastic nodes cluster, which processes a large amount of log data to organize and index it into matching documents, which are shared and stored as replicas
• A visualization platform, consumes the data collated by elastic nodes and provides effective insight into the blockchain nodes and network statistics
• Enables stakeholders to perform analytical research and generate reports

Leveraging the proposed monitoring framework will help:

• Analyze how the blockchain transaction processing and consensus mechanism utilizes the underlying infrastructure resources
• Provide visibility on a business transaction—end to end—as it gets initiated by a user from the dApp and is captured into blockchain
• Combine and correlate the block and transaction related events from each node and determine the performance and throughput of the blockchain network
• Setup a non-invasive monitoring solution that can be dynamically enabled for each onboarded peer and also support a common network provider model

Conclusion

While there is no dearth of monitoring solutions, the technique for effectively leveraging the existing mechanisms for monitoring a blockchain network is not thought through. The primary reason is that not many enterprise use cases have translated to production grade systems on blockchain yet. Also, the decentralized nature of blockchain poses the question—is monitoring of the whole blockchain network really required?

To maintain, analyze and improve an enterprise blockchain based solution, a holistic monitoring solution is required. This can further be coupled with DevOps tooling to enable maximum uptime of the blockchain network and ensure business continuity.

About the Author

Hitarshi Buch
Lead Architect, CTO Office, Wipro Limited

Hitarshi has 19 years of experience in IT architecture, consulting, design and implementation using blockchain, API, SOA, BPM and Java/J2EE technologies. He has experience in IT transformation and modernization initiatives and has provided enterprise-wide SOA-based solutions. In his current role, as a Lead Architect in Service Transformation at Wipro, he leads the Center of Excellence initiatives as part of the Blockchain practice.