Deploying Hyperledger Fabric in Production

In recent years, a number of economic sectors have seen the great promise of blockchain and are already using or piloting it in different use cases. But proof-of-concept projects are often led by evangelists, developed in R&D, and always in controlled environments. Moving to production requires stakeholder buy-in and can be a real challenge, especially at the current time with blockchain.

The Hyperledger Hyderabad Meetup organized a meetup to demonstrate the potential of Hyperledger technologies and discussed the challenges related to the production deployment of Hyperledger Fabric. This session was headed by a startup team out of the Hyperledger Hyderabad community itself, called ChitMonks, who have set a benchmark in their use case in the chit fund industry. 

The chit fund industry, with estimated assets under management (AUM) of ₹1.50 lakh crore, involves a great deal of paperwork. In some instances, it forms the backbone of the informal economy in developing countries like India. This results in drudgery for both the foremen (who organizes the chits) and the regulators (the State governments). 

Administration of chit funds through companies are happening manually. Every time a company wants to start a chit, they have to submit a huge pile of documentation on a monthly basis to the registration and stamps department, which is the regulator. Consequently, there is a lack of transparency for subscribers, and audits need to be done to ensure the government and chit fund company have matching documentation. The information gap between the chit fund company and the regulators produces a loophole for frauds, which are hard to trace back.

The approach of ChitMonks is to place a private blockchain-based, consortium-driven infrastructure and establish a process-based trust, by bringing all the stakeholders involved in the process to the same platform of blockchain.

Actors and Key activities

ActorTypeRoleKey Activities
ForemanROSCA Business OrganizerParticipant / Blockchain Node Maintainer Initiates ROSCA transactions in the systemResponds to queries related to ROSCA transactionsResponds to queries related to Health of company operationsOptionally maintains the node in the network. 
RegulatorState Government EntityRegulator / Blockchain Node MaintainerApproves or Monitors the ROSCA operations of the companiesMonitors the health of the company financialsAddresses Subscriber GrievancesMonitors the payments made to the subscribers and compliances related to it
SubscriberROSCA ParticipantParticipant Approves transactions that involve his participation.Provides consent to the information-seeking queries 
BankTransaction validatorParticipant and/or ValidatorProvides validation to payment transactionsProvides services to ROSCA companies like FD, automatic money sweeping, etc
AuditorTransaction validatorValidatorProvides audit approvals of the company financials to fulfill the  ROSCA regulations.
Non-Banking Financial services CompaniesOther Business ProviderOther Business Provider/ Blockchain Node MaintainerProvides other financial services to ROSCA organizers and their subscribers
Insurance CompanyOther Business ProviderOther Business ProviderProvides insurance service  to ROSCA SubscribersProvides insurance service to ROSCA organizers / Companies
Auction ServiceService ProviderService Provider / Blockchain Node MaintainerProvides auction services to smoothen and automate the ROSCA auction services
Many new service providers to join….………….

Technical Architecture 

The three major entities involved in the process are:

  • Foremen: it’s a consortium of all the chit fund companies.
  • Regulators: it’s a consortium for all the regulatory entities (generally the local government bodies). 
  • Subscribers: they are the end beneficiaries of the chit fund companies.

The foremen and regulators act as the two organizations in the network. And each of the organizations has two peers each. The organizations also have their respective C.A. (Certificate Authority) for letting any new member join the consortium.

  • A channel is established between the two organizations to keep data in synch between regulators and the foremen.
  • The chaincode is framed around the Chit Funds Act, 1982, passed by Indian legislative. This chaincode takes care of all the guidelines like commissions by foremen, a form of chit agreement, filing of chit agreement, minimum capital requirements for the commencement of a chit, creation of a reserve fund by a company and many more aspects of chit fund.
  • The orderers follow Raft ordering service, after being maintained on Kafka ordering service for quite some-time.
  • The state DB chosen here is couch DB, after testing it with Go level DB, which can handle complex data and adds rich querying practices. 
  • Here the ledger holds only relevant data for both the organization’s, keeping a separate off-chain database for specific purposes.
  • The infrastructure is designed as such where a foremen organization can own and maintain organization nodes. Typically, a big organization can own the infrastructure, medium and small organization can be part of consortium org/nodes. This makes the blockchain network more self-sustaining and more decentralized.  

So, here the blockchain system is used in a very optimized way by keeping only the shared data over the system. To make this complex system easily accessible for both organizations, each of the entities has different applications for their purposes.

As already mentioned, ChitMonks serves its platform to more than 700+ chit fund companies. Data privacy was the biggest concern for them, since the consortium of all the chit fund companies are structured as an organization here, and they all have access to the same state DB. To ensure data privacy, ChitMonks used private data collection introduced by Hyperledger Fabric in its 1.4 release. In this case, each transaction is encrypted, and the keys to check data are shared between the regulator and chit fund company over the private data collection provided by Hyperledger Fabric’s latest release.    

Challenges Encountered

Asset Modeling – Asset modeling helps to define the entities and assets involved in the process of business.  Challenges include the incipient regulatory framework and the relatively limited experience of managing large amounts of data with respect to entities (i.e., who will own what data and hierarchy involved each level of the process). While there haven’t been verifiable data breaches of blockchain backbones to date, there have been deficiencies in the implementation of smart contracts, mostly due to failures in coding. But a well-defined set of terms and conditions from the government authorities (which is also the regulatory and one of the organizations in our Blockchain infrastructure) for  the chit fund business helped ChitMonks to encode an ultimate and sustainable asset modeling. 

Testing chain code – Hyperledger Fabric provides us an immutable infrastructure. So, it’s important to test the chaincode well before deploying it. ChitMonks took the approach to perform unit testing at every change to follow up on potential bugs.   

Docker infrastructure – Containers might thus move from one host to another, especially while scaling the number of containers up or down or during container redeployments. There is no static relation between hosts and services they are running anymore! For troubleshooting, this means one must first figure out which host is running which containers. And, vice versa, when a host exhibits poor performance, it’s highly valuable to be able to isolate which container is the source of the problem and if any containers suffer from the performance issue.

Continuous Integration tools (CI) such as Jenkins can build and deploy Docker images across application environments very easily. Builds can be automated and also can be monitored efficiently using CI administrator console.

Maintaining the Kafka cluster – Initially, ChitMonks established its Hyperledger infrastructure with Kafka-based ordering services. Kafka provides a scalable multi-orderer infrastructure and poses a viable fault-tolerant solution for Hyperlуdger Fabric infrastructure, but it required us to maintain zookeepers along with it in production, which added a layer of complexity. So, ChitMonks have shifted to Hyperledger Fabric 1.4, which allows raft ordering services.  

The Raft consensus is a mature consensus algorithm and is easier to implement on a large scale than Kafka. It can extend a network with new organizations, as we don’t need to rely on a third-party Kafka cluster. In the previous versions of the platform, it was impossible to change a consensus type of a blockchain network without full redeployment. However, Hyperledger Fabric v1.4.2 introduced a mechanism that makes it possible to migrate a network from a Kafka consensus to a Raft-based one. And Hyperledger has provided a very easy step-by-step guide for migrating from Kafka to raft consensus.

Conclusion

The meetup was received with a very enthusiastic response. Many of the attendees were very happy to see a local startup deploying production-grade applications. The ChitMonks engaged well with the audience to help them understand the difference between a POC and production level applications and shared their perspective on scaling a blockchain-based product!

Kartikey is a co-organizer of the Hyperledger Hydrabad Meetup and works as a Software Engineer at MSR Cosmos and is a team member of TEDxHyderabad. You can reach out to me via TwitterFacebook or LinkedIn.