Category

Hyperledger Sawtooth

Apr 30
Love0

Identity Applications in Action & Powered by Hyperledger

By Blog, Hyperledger Fabric, Hyperledger Indy, Hyperledger Sawtooth

Digital identity is gaining a lot of traction as a driver for blockchain adoption. There is growing recognition across many markets that reshaping how digital information is managed and verified can simultaneously increase online trust and privacy. The Hyperledger community is working hard to develop and deploy blockchain-enabled identity technologies and solutions with an eye towards decentralizing control of information and creating new models for verifying identities. 

To illustrate where this technology can take us, we are showcasing some applications where it’s already being put into action:

Known Traveller Digital Identity

Known Traveller Digital Identity, or KTDI, is a World Economic Forum initiative with Accenture that brings together a global consortium of individuals, governments, authorities and the travel industry to enhance security in world travel. The pilot leverages cryptography, blockchain technology and biometrics and aims to allow cross-border travel without presenting physical documents, accelerating the flow of passengers through airports, improving passenger experience, and enabling authorities to better focus limited resources on security improvements.

QDX™ HealthID platform 

QDX™ HealthID platform is a service from Quantum Materials Corp that leverages self-sovereign identity technology to provide end-to-end visibility to support testing and immunization for infectious diseases, including COVID-19, at scale. The platform enables multiple methods of authenticating the individuals being tested, those who are administering the test, as well as the test kits themselves, whether to detect the presence of the virus itself or associated antibodies. 

The QDX™ HealthID platform uses Hyperledger Sawtooth as the backing distributed ledger technology, which is deployed and managed using Blockchain Technology Partner’s platform Sextant for Sawtooth. The platform is also readied for integration with other distributed ledger applications via DAML, the smart contract language open sourced by Hyperledger member, Digital Asset.

MemberPass™

Provided by CULedger, MemberPassTM is a digital credential held by credit union members that protects credit unions and their members from identity theft and fraud in all banking interactions, from call center authentication to lending to new account opening. MemberPass is a simple, secure replacement for user IDs and passwords, and supplants the traditional knowledge-based interrogation contact centers employ today to authenticate members calling for telephone banking services. MemberPass seamlessly authenticates both, the member and the credit union to each other, in any call-in, log-in or walk-in exchange, providing a consistent, frictionless experience across all channels. 

Sovrin Network

Operated by independent Stewards, the Sovrin Network uses the power of a distributed ledger to give every person, organization, and thing the ability to own and control their own permanent digital identity. With recent advancements in digital identity standards, Sovrin provides a secure and private network for identity holders to collect, manage and share their own verifiable digital credentials.

The Sovrin Network is governed by The Sovrin Foundation, a nonprofit organization established to administer the Governance Framework for this open source decentralized global public network enabling self-sovereign identity on the internet. 

Trust Your Supplier

Trust Your Supplier is a production Hyperledger Fabric blockchain network, running on the IBM Blockchain Platform, that provides suppliers with a trusted digital passport to streamline on-boarding with their customers.

The Trust Your Supplier network is a cross-industry source of supplier information and identity helping to simplify and accelerate the on-boarding and lifecycle management process. TYS was built to be a cross-industry blockchain network to facilitate procurement functions within an organization. The network’s first use case is focused on supplier onboarding and validation.

VerifiedMe

Verified.Me is a service offered by SecureKey Technologies Inc., in conjunction with a consortium of seven of Canada’s major financial institutions – BMO, CIBC, Desjardins, National Bank of Canada, RBC, Scotiabank and TD. Verified.Me is a privacy-respecting digital identity and attribute sharing network. The service simplifies identity verification processes by allowing individuals (subjects) to share identity and attribute information from trusted sources (including financial institutions, mobile operators, credit bureau, and government) with the services that they wish to access.

The network is based on permissioned distributed ledgers operated by the consortium. It is built using the IBM Blockchain Platform which is based on Linux Foundation’s open source Hyperledger Fabric  and is aligning with W3C decentralized identity standards, to enable interoperability with other networks. SecureKey’s Triple Blind® approach means that no network participant alone, including SecureKey, can have a complete view of the user journey – the subject can’t be tracked.

The service is free for consumers to use, either using their web browser, or by downloading the mobile app through the App Store (iOS) or Google Play (Android).

Join the conversation about blockchain-based identity technologies and solutions with #HyperledgerIdentity this month on social channels. Also, Hyperledger has an Identity Working Group that is open to all. Learn how to get involved via this video.

Coverage image by PublicDomainPictures from Pixabay

Oct 31
Love0

Announcing Sawtooth PBFT 1.0

By Blog, Hyperledger Sawtooth

The Sawtooth community is excited to announce the official release of Sawtooth PBFT 1.0 along with the Hyperledger Sawtooth 1.2 release. PBFT 1.0 is the result of substantial engineering effort from Bitwise IO and Cargill to provide a fast and practical consensus algorithm for consortium-style Sawtooth networks.

An earlier blog post, Introduction to Sawtooth PBFT, presented the details of the consensus algorithm. The next post, Sawtooth PBFT, Part 2: Extensions and Changes, described the enhancements that made the algorithm more efficient and easier to implement for Sawtooth. Now, we’d like to describe the advantages of Sawtooth PBFT and give practical tips for how and when to use PBFT consensus.

What Makes PBFT Different?

Those familiar with Sawtooth already know about the Proof of Elapsed Time (PoET) consensus algorithm, a lottery-style (Nakamoto) algorithm that is designed for large, open-enrollment blockchain networks. Until now, PoET was the only production-ready consensus option for Sawtooth. While PoET is useful for certain kinds of networks, it isn’t a universal solution. In particular, PoET is not well suited for smaller networks because of its slower performance.

To make Sawtooth a more versatile blockchain platform, we introduced the dynamic consensus interface in Sawtooth 1.1, which provides a new consensus API that simplifies the process of adding new consensus algorithms. We started with Raft, a simple leader-based consensus algorithm designed for small networks. Raft provided a consensus type that we didn’t have with PoET, but because Raft isn’t Byzantine fault tolerant (BFT), it wasn’t a complete solution.

This is where Sawtooth PBFT comes in. PBFT has properties that make it well suited for small networks — it’s leader-based, non-forking, and fast. It also provides the safety and liveness guarantees that are necessary for operating a blockchain network with adversarial trust. This makes PBFT an excellent option for most consortium-style networks. Consortium networks are small (typically 5-20 nodes), have controlled membership (only administrators can add and remove nodes), and typically require finality (non-forking behavior). Once a block is committed, it is committed forever, which eliminates any uncertainty about the permanence of state.

How Well Does PBFT Work?

Sawtooth PBFT works very well! We’ve tested it extensively for safety, reliability, and performance. The code includes a large number of unit tests that validate PBFT’s functionality, and we’ve run a number of long-running PBFT networks to work out the kinks.

How does PBFT Perform?

Part of the extensive testing of PBFT was determining how well it performs. A full Sawtooth network consisting of 10 “m5.large” AWS instances running with PBFT consensus performed 2.5x faster than the same network running with PoET consensus.

Exact results may vary depending on factors like system resources and network size, but the big picture is pretty clear: PBFT can provide a big performance boost compared to PoET.

When Should I Use PBFT?

The short answer is that PBFT should be considered the first choice for small to medium Sawtooth networks that do not require open enrollment. If your use case requires a large network (more than 20 nodes, for instance) or open enrollment, PoET may be a better choice because it is designed to perform well with many nodes and is less strict about membership in the network. But, in general, we recommend using PBFT because of its finality and performance advantages for most networks.

How Do I Use PBFT?

Sawtooth PBFT is easy to configure and use. We designed Sawtooth’s dynamic consensus interface to make choosing consensus simple. The Sawtooth 1.2 documentation includes instructions for setting up a Sawtooth network with PBFT using Docker and Kubernetes, which are designed for application development environments and Proof of Concept evaluations. For production networks and do-it-yourself developers, there’s also a procedure for setting up a Sawtooth network on Ubuntu.

See For Yourself

Sawtooth PBFT is an exciting new addition to the Hyperledger Sawtooth ecosystem. If you’d like to learn more:

About the Author

Logan Seeley is a Software Engineer at Bitwise IO. He has been a part of the Hyperledger Sawtooth community since May 2018, working in a variety of areas including consensus, Hyperledger Sawtooth, and Hyperledger Transact.

Oct 17
Love0

Announcing Hyperledger Sawtooth 1.2

By Blog, Hyperledger Sawtooth

Today, we are pleased to announce that Hyperledger Sawtooth release 1.2 is available! Since the 1.1 release, Sawtooth has continued to grow in capability, diversity, and adoption, thanks to the involvement of many organizations and open source community members. The release of Sawtooth 1.2 shows that growth with the active contribution of features and improvements by an engaged community of developers.

The latest version of Sawtooth provides exciting new benefits — namely, full support for the PBFT consensus engine and for mobile application development with new SDKs for iOS and Android. 

Additionally, this release contains transaction family compatibility with Sawtooth Sabre, enhanced performance & stability, improved documentation, better support for consensus algorithms, and overall platform refinements for a better developer experience.  

PBFT 1.0 Consensus in Sawtooth

The dynamic consensus interface, introduced in Sawtooth 1.1, allows for easy integration with consensus engines that meet a variety of use cases. Sawtooth 1.1 included support for PoET, PBFT, and Raft consensus. Now, Sawtooth PBFT 1.0 is the preferred consensus for  small-to-medium networks — it’s leader-based, non-forking, and fast. PBFT also provides the safety and liveness guarantees that are necessary for operating a blockchain network with adversarial trust. This makes PBFT an excellent option for smaller consortium-style networks.

Complete procedures are provided for configuring PBFT consensus on a Sawtooth node and network. For more information, see Creating a Sawtooth Test Network in the Application Developer’s Guide or Setting Up a Sawtooth Network in the System Administrator’s Guide.

Mobile Support in Sawtooth

Release 1.2 ushers in support for mobile development in Sawtooth with the inclusion of a new Swift SDK for iOS and improved Java SDK for Android. New tutorials and SDK reference documentation for Swift and Java help developers write native mobile client applications for Sawtooth. 

Transaction Family Compatibility with Sabre

All core transaction families are now compatible with Sawtooth Sabre release 0.4.0, a WebAssembly smart contract engine for Hyperledger Sawtooth. This compatibility is a major step towards allowing the default transaction families to be managed as on-chain smart contracts. 

Improved Documentation

We are making strong efforts to support developers by continuing to improve the content and quality of Sawtooth documentation. In this release, you will find Swift and Java tutorials, procedures for configuring a consensus engine, and improved summaries of the supported consensus algorithms for PBFT, PoET, Raft, and Devmode. There are also numerous technical corrections, bug fixes, and general improvements throughout the documentation.

Refinements

This release includes a number of refinements designed to improve performance & stability, allow quicker builds, enhance support for consensus algorithms, and provide development options such as access to raw transaction headers through a new API. For details, see the Release 1.2 (Chime) release notes.

Try Hyperledger Sawtooth 1.2 Today!

Sawtooth version 1.2 is available now, so there’s no need to wait. To get your hands on it, use the links below. If you are new to Sawtooth and would like to get involved, take a look at the community links.

About the Hyperledger Sawtooth Team

Hyperledger Sawtooth 1.2 is the result of the collaboration and dedication of many people. Significant contributions were provided by Cargill, Bitwise IO, Intel, and others. Special thanks to Peter Schwarz, Darian Plumb, Anne Chenette, Shawn Amundson, Ryan Beck-Buysse, Richard Berg, Arun S M, Logan Seeley, Andrea Gunderson, Shannyn Telander, and Eloá França Verona.

Oct 14
Love0

Spotlighting Supply Chain Use Cases

By Blog, Hyperledger Fabric, Hyperledger Sawtooth

It’s #HyperledgerSupplyChain month so we wanted to spotlight some of the exciting ways Hyperledger technologies are improving traceability, adding efficiencies and building trust in supply chains around the world.

The role of open source enterprise blockchain is well laid out by Target as a mechanism for “ensuring multiparty trust across enterprises doing business together” in this article about the benefits of the technology. 

To help boost the #HyperledgerSupplyChain conversation, below are some noteworthy use cases. Chime in on social with your own examples of Hyperledger powering supply chain solutions. 

Walmart’s use of Hyperledger Fabric to track food for better safety: When an outbreak of a food-borne disease happens, it can take days, if not weeks, to find its source. Better traceability could help save lives by allowing companies to act faster and protect the livelihoods of farmers by only discarding produce from the affected farms. Using a system powered by Hyperledger Fabric, Walmart can now trace the origin of over 25 products from five different suppliers. The company plans to roll out the system to more products and categories in the near future. 

ScanTrust’s use of Hyperledger Sawtooth to bring transparency to the supply chain: To help their client Cambio Coffee bring more transparency to their ethical trade business, ScanTrust used Hyperledger Sawtooth to build a blockchain-enabled traceability function to enhance its supply chain application. Cambio Coffee implemented ScanTrust’s unique QR codes on their packs in May 2018 to an enthusiastic response from customers. Currently, the roaster and the delivery company enter data onto the blockchain. Future plans call for the roll out the feature to the shipping company and, eventually the farmers, to cover the whole supply chain. Customers are also interested in using the platform other blockchain-supported initiatives, like “Tip your farmer.”

Circulor’s first-ever mine-to-manufacturer traceability of a conflict mineral with Hyperledger Fabric: The African country of Rwanda is the world’s biggest supplier of tantalum: a rare mineral used to make capacitors found in devices like smartphones and laptops. To prove beyond doubt that every bag of tantalum ore from Rwanda was mined, transported, and processed under OECD-approved conditions, without any child or slave labor, Circulor created a Hyperledger Fabric-based system to trace tantalum from three mines and an ore-sorting facility in Rwanda. The system is designed to slash the high cost for compliance, satisfy regulators, reassure consumers, and build revenues for Rwanda.

To see the full line up of supply chain use cases in the Hyperledger Blockchain Showcase, head here. If you have a Hyperledger-powered supply chain project you’d like to see on this list, please submit it.

We also have an active community for those interested in participating and contributing to supply chain solutions. The Hyperledger Supply Chain Special Interest Group (SIG) represents a global membership of logistics and supply chain professionals united in advancing the state of the supply chain industry through the implementation of enterprise-grade solutions based on Hyperledger technologies.

Oct 01
Love0

All The Farms: An agricultural supply chain story

By Blog, Hyperledger Sawtooth

“Where are the local farms around me, what are they growing, and how can I buy their stuff?”

Those are pretty much the questions that my team and I were interested in answering, and we felt that, if we were wondering these things, so were others. That simple question (or string of them) is also what led us to joining the Hyperledger community and building our product in Hyperledger Sawtooth.

To answer the “farms question” we needed to 1) collect accurate information on farm and ranch locations, 2) build a database with this data, and 3) create an interface that allows people to answer these questions. With these three tasks in mind, we created All The Farms.

Fortunately, my cofounder, Chris Styles, and I had experience with collecting and standardizing data, as well as building APIs and tools to share this information. (Previously, I had cofounded the political database Run For Office, and Chris had led projects like Vuhaus). We still needed help to share the engineering load and provide UX/UI (thanks Szabi and Whitney), but we were able to fill those gaps with people who were equally as passionate about helping people connect with their local food economy. It’s been very personally gratifying to work on this site/database, and we will continue to work on improving it – because it’s a passion of ours.

So, why blockchain for this project? Well, our interest was kicked into gear last September by the IBM/Walmart announcement of their adoption of the Global Food Trust blockchain platform. As we thought about our role in this ecosystem – as a provider of data for local food options – we saw a changing landscape in logistics and traceability. We also recognised the importance of finding the right format for sharing the data we were collecting.

Adding the task of applying our data in a blockchain was daunting, but we were able to recruit an experienced blockchain developer to head up that effort. This is an area where being a smaller company has its advantages – no one expects you to have money, so you can hire with equity. Over the summer, we were fortunate to bring along someone who wanted this project, and our entire team has been able to learn about Hyperledger along with him. (Hi, Dan.)

Participating in something that creates greater transparency in the food system is what drives our team. We see the food consumer as being at a classic disadvantage due to the informational asymmetry in the food chain. Essentially, the consumer has less knowledge than just about everyone else in their transaction.

We were heartened to see Intel and Oregon State University collaborating on a traceability project for Oregon blueberries using  Hyperledger Sawtooth. The fact of the matter is that Intel has a big presence in Oregon and at OSU, so it was easy for us to connect directly with people on this project. They gave us valuable feedback on our plans to put the data we were collecting into blockchain. 

The logistical applications for blockchain are well known and important, but we also saw lateral applications of the technology that could help farms outside of traceability. One use case we are working on is using the value in blockchain for farm/ranch compliance record keeping.This application helps all the farms, not just the ones with the budgets for the full scale traceability use case. We also feel that producers can benefit from blockchain applications for Water Rights and have found a solid use case for tracking industrial hemp seed. Both of those projects we have underway with Oregon State University.

Most of our team has a personal and professional background with farms. Many come from farming families. Others have worked with state agencies in agriculture and ranching. Some have both of those experiences. I think that this type of understanding has allowed a flexibility in our thinking about blockchain and has led us to consider other projects that we feel can be widely beneficial for our farming communities.

What we want the Hyperledger community to know is that our data in Hyperledger Sawtooth is the starting point for other projects. It is meant to be used by logistics companies and institutional buyers that are looking to source locally. It is meant to have applications that help farmers and ranchers consolidate their records for compliance purposes. It is meant to connect our Hyperledger community directly with the real food producers and to figure out ways that we can best work together.

Lastly, climate change: it is real and the way we currently eat emits too many greenhouse gases and degrades our soil as well. A key piece of our work in data collection is identifying farms that use regenerative farming practices. We hope you use the regenerative agriculture filter for your personal shopping, and we also hope you’ll consider putting your technology skills towards a solution with regenerative agriculture. We invite you to collaborate with us and other like-minded allies in our Hyperledger community.

Jim Cupples is a cofounder at Terrapin Data, Inc. which has launched All The Farms. The All The Farms blockchain, built on Hyperledger Sawtooth, is a permissioned blockchain. Please contact jim@allthefarms.com if you are interested in access or have comments or questions.

Sep 19
Love0

2019 Summer Mentee Project Update: Developing Explorer Capabilities for Hyperledger Sawtooth

By Blog, Hyperledger Sawtooth, Hyperledger Summer Mentorship Program

Blockchain itself is, first of all, a back-end technology. That produces a lack of transparency: one can’t see a list of transactions, blocks and signers’ public keys without use of CLI or some node’s API. Both ways usually only let you see raw data with no handy navigation. That’s why there’s often an explorer for certain blockchains.

My project was to develop a Hyperledger Sawtooth explorer. I wanted not only to allow users to explore the blockchain data provided by the API through a nice graphical interface, but also to add some extra features to enhance Hyperledger Sawtooth transparency even more.

To build this app, I used the stack I’m most familiar with: Express.js with MongoDB for the back end and Vue.js with Vuetify for the front end. The main algorithm behind my explorer’s work can be described as this:

1. On launch, back end requests the node of an address that was put to an environment variable. The request is to subscribe for standard Sawtooth events: state-delta and block-commit. In that request, it asks node to send back all events that occurred after the last one known to it. On first launch this means all events.

2. Back end parses all events and saves received data in Mongo.

3. Back end also provides an API for the front end to fetch this data from the database.

4. Front end fetches all the needed data and shows it in according places also providing navigation between transactions/blocks/signers an so on.

As for the general impact of my project, I really hope it would make Hyperledger Sawtooth more transparent both for those who already are in Hyperledger community and for those who are yet to consider choosing Sawtooth as a tool for their projects.

The most vivid impression I have after working on a project related to Hyperledger Sawtooth is the responsiveness of the community members. Some issues I encountered took literally days to solve until I asked a question in Sawtooth RocketChat. That’s my advice for other new members of the community – do not hesitate to ask. If you’re polite, you’ll most probably get a helpful answer.

To sum up, I’d like to say that I was really happy to take part in this internship. It opened the world of open source to me and allowed to really improve as a developer. For that I’d like to thank Hyperledger and my mentors Andrew Backer and Ricardo Garcia from a company called ScanTrust. In short, they provide consumer-oriented businesses with smart blockchain-secure QR codes to stick on every selling piece to enable buyers to see whether this single item was produced, where it’s said to be or it is a counterfeit. If you are a business and looking for such kind of software, go check them out.

Thanks for reading! Find out more about my project here.

Jun 06
Love1

Sawtooth PBFT, Part 2: Extensions and Changes

By Blog, Hyperledger Sawtooth

The upcoming release of Hyperledger Sawtooth 1.2 includes Sawtooth PBFT—a new, Cargill-sponsored, production-ready consensus algorithm. Sawtooth PBFT is much more than a basic implementation of the PBFT consensus algorithm that was originally defined in 1999. Sawtooth PBFT has all the core features of the original definition—it provides the same safety and liveness guarantees—but we have adapted the algorithm to take advantage of features unique to blockchains, and extended it to provide more flexibility and robust guarantees.

In two earlier blog posts, Making Dynamic Consensus More Dynamic and Introduction to Sawtooth PBFT, we introduced Sawtooth’s dynamic consensus interface and explained how the PBFT consensus algorithm works from a high level. In this post, I’ll describe the adaptations and enhancements that helped us bring a robust PBFT consensus algorithm to Hyperledger Sawtooth:

  • Sequence number simplications (no more watermarks)
  • An idle timeout to guarantee more liveness
  • Regular view changes for fairness
  • A consensus seal as proof of commit
  • No checkpoints for garbage collection
  • A catch-up procedure for slow and new nodes
  • Membership changes

Sequence Number == Block Number

In traditional PBFT, the primary node assigns a sequence number for each request when it broadcasts a pre-prepare message for the request. In Sawtooth PBFT, the “requests” are actually blocks that are created by a Sawtooth validator. Blocks in Sawtooth are ordered; we can take advantage of this ordering to simplify the sequence numbering in PBFT. Instead of assigning a sequence number for each block, the primary node will just use the block number as the sequence number.

This simplification makes the algorithm easier to understand. It also means that secondary nodes don’t need to check that the primary node picks reasonable sequence numbers. Originally, nodes would have to make sure that the sequence number was between a low and high watermark, in case the primary node picked a number that was so high that it exhausted all of the possible sequence numbers. Now that the sequence number is automatically determined by the block number, a secondary node only needs to check the pre-prepare message to make sure that the sequence number matches the block number of the block in the message. It’s as simple as that!

More Liveness with an Idle Timeout

One liveness problem that traditional PBFT does not solve is the “silent leader” problem. The leader is responsible for building and sharing blocks with the rest of the network to start a new round of consensus. Even if transactions are being submitted to the network, the leader could stay silent by never sharing a block with the network; this “silent leadership” would allow the leader to halt progress indefinitely.

To solve this problem, we implemented an idle timeout as a new way to trigger a view change. After a block is committed, each node starts its idle timer. The primary node must publish the next block and broadcast a pre-prepare message for the block before the timer expires. Otherwise, a secondary node will initiate a view change.

The idle timeout provides more robust liveness by ensuring that a faulty primary node can’t stall the network indefinitely. This timeout can lead to unnecessary view changes, though; if the network really is idle (no transactions are being submitted), then the primary node isn’t faulty when it doesn’t publish a block, since the Sawtooth validator does not publish a block when there are no transactions. The cost of these extra view changes is negligible, however; when the network is idle, a view change will not hinder the performance of the network.

Pseudo-Fairness with Regular View Changes

Determining if a leader is acting fairly is a very complex and challenging problem; there is still a lot of research being done to solve it.

To get us a step closer to fairness, we introduced a regular view change mechanism to Sawtooth PBFT. This mechanism is fairly simple: after every nth block is committed, the network will “force” a view change by incrementing its view by one. The number of blocks between forced view changes is called the forced view change interval, which is a configurable setting.

Regular view changes reduce unfairness by not allowing any node to control block publishing for too long; instead, every node gets a chance to be the leader for a period of time. While this doesn’t completely solve the fairness problem, it limits unfairness in a way similar to Tendermint and lottery-style consensus algorithms. This solution is also inexpensive: since the view change is “forced” (the view number is directly incremented) at a consistent point across the network, we don’t need the expensive view changing procedure.

Consensus Seal

The original PBFT algorithm does not define a reliable way to check if a block was committed validly (that is, when 2f + 1 nodes agreed to commit the block). The only way to check this would be to request each node’s commit messages and count them. The problem with this approach is that old messages might be unavailable if the messages were removed (garbage collected) or if a node is no longer part of the network.

To overcome this limitation, we implemented a consensus seal for each block that is committed. The consensus seal is a signed data structure that contains a block ID and 2f signed commit messages for that block ID. This seal proves that the network agreed to commit the block and that the block was committed validly.

The seal is stored on the blockchain itself, which means that the seal is immutable and can be checked at a later point. A seal can’t be inserted directly into the block that the seal is for, however, due to the way blocks are constructed by the Sawtooth validator. Instead, the seal is added to the next block that is committed to the blockchain; thus, each block contains the consensus seal for the previous block. This approach has a limitation—the most recently committed block can’t be validated using a consensus seal, since there isn’t a subsequent block yet. However, all previous blocks in the chain can be validated to ensure that they were committed validly, which is a useful guarantee.

Catching Up

In the real world, distributed networks face failures, segmentations, dropped messages, and many other hiccups. When these issues arise, nodes may fall behind, so they need a way to catch up to the rest of the network.

Sawtooth PBFT uses a special catch-up procedure that takes advantage of the consensus seal extension. The catch-up procedure is best illustrated by an example.

Let’s say the network as a whole has committed block 100, but a node has fallen behind and has only committed blocks up to block 90. When the slow node eventually gets block 91, it waits for a pre-prepare message from the primary node. Because the network has already moved past this block, though, the node never gets this old pre-prepare message. Instead, the slow node gets block 92, which contains a consensus seal for block 91. The node examines the seal from block 92 to make sure it’s valid, then commits block 91. Our node knows that it can skip the usual consensus procedure because the consensus seal in block 92 proves that the network agreed to commit block 91.

The node follows this procedure until it commits block 99. At this point, it needs to do something different. Because there is no block 101 yet, the node doesn’t have a seal for block 100. So the node broadcasts a request to the rest of the network for a consensus seal for block 100. If another node has already committed block 100, that node builds the seal and sends it directly to the node that requested it. When the node receives the seal, it will verify the seal and commit block 100. The node that fell behind is now up to date!

Who Needs Checkpoints?

As originally defined, PBFT uses a checkpoint procedure to perform garbage collection for each node’s log of consensus messages. In this procedure, the nodes exchange messages to come to consensus on the current version of state. When the nodes have agreed on the current state, they save these messages as proof of the agreement, then discard all previous consensus messages.

Sawtooth PBFT does not need this checkpoint procedure. Because each block contains a consensus seal that provides proof of the validity of the previous block, and each block is immutable and permanent by nature of a blockchain, every block contains a checkpoint that will be kept forever. This means that the nodes can clean up old log messages any time they wish, while the state can still be verified using the consensus seal.

Membership Changes

Another fact of the real world is that network membership can change. This happens for a variety of reasons: a node may need to be replaced or a new company might want to join a consortium’s network. The original PBFT algorithm does not define a procedure to change membership of a PBFT network, so we implemented our own.

In Sawtooth PBFT, membership in the network is determined by an on-chain Sawtooth setting, sawtooth.consensus.pbft.members, which is a list of the public keys of all nodes in the network. Since this list is on the blockchain itself, it’s shared and agreed on by the whole network, which makes changing membership easy.

An administrator can update the on-chain setting to add, remove, and reorder nodes by submitting the change as a transaction. Each time a block gets committed, all PBFT nodes check this setting to see if it changed because of a transaction in that block, then update their local state if necessary.

Like forced view changes, a membership change is done at a consistent point across the network: when a block is committed. This consistency makes the local view of membership on each node easy to update in an atomic way. This approach to membership also makes it easy to check the historical membership throughout the life of a Sawtooth blockchain managed by PBFT consensus.

Summary

Sawtooth PBFT has a lot to offer. It not only includes all the functionality you would expect from PBFT, but it adds features and optimizations for real-world use. We’ve developed new solutions to overcome limitations of traditional PBFT. Plus, we’ve been able to take advantage of the properties of a blockchain, such as immutability and ordering, to make the algorithm more efficient and robust.

Want to Learn More?

These features are all new to PBFT, and some of the implementations are unique among published consensus algorithms. If you’re interested in learning more about our PBFT extensions, read the regular view changes RFC, consensus seal RFC, and PBFT catch up RFC. Also, check out the Sawtooth PBFT documentation and source code on GitHub.

Do you have questions? Do you have comments or concerns about how the extensions work? Let us know in the #sawtooth-consensus-dev channel on Hyperledger Chat!

Stay tuned for more news and info about the upcoming Sawtooth PBFT 1.0 release.


About the Author

Logan Seeley is a Software Engineer at Bitwise IO. He has been a part of the Hyperledger Sawtooth community since May 2018,


May 10
Love0

When Hyperledger Sawtooth Met Kubernetes – Simplifying Enterprise Blockchain Adoption

By Blog, Hyperledger Sawtooth

Blockchain Technology Partners has teamed up with Digital Asset to deliver DAML Smart Contracts for Hyperledger Sawtooth deployed and managed on Kubernetes using its blockchain management platform Sextant.

One of the great things about being a member of the Linux Foundation is that you are part of a vibrant global community that is home to open source projects in a wide range of ecosystems and host to an impressive array of events worldwide.

  • A global community where you are actively encouraged to participate in meetups and events specific to your ecosystem as well as contribute to broader open source summits
  • A global community where there is the opportunity to collaborate not just within your own ecosystem but with ecosystems that complement yours – this is at the heart of the open source ethos

A case in point is Blockchain Technology Partners (BTP) which, like a number of our Hyperledger members, is also an active participant in the CNCF, aka Cloud Native Computing Foundation. Its founding team recognized early on that the only way to make blockchain technology more accessible and easier to adopt was to address the operational challenges that they could see lying in wait for enterprises. With their team’s  background in open source, operations and cloud, BTP quickly realised that CNCF’s Kubernetes was the perfect match for Hyperledger Sawtooth. This led to the creation of Sextant, their blockchain management platform that radically simplifies the development, deployment and ongoing management of blockchain-based enterprise applications.

The power of open source collaboration doesn’t end there as BTP have now teamed up with Digital Asset to bring DAML Smart Contracts to Hyperledger Sawtooth – integrating their open source DAML runtime with Sawtooth, similar to the way the Sawtooth Ethereum (SETH) project integrates the Burrow EVM with Sawtooth, and handling its deployment and management with Sextant.

For the full story behind the creation of Sextant to deploy and manage Hyperledger Sawtooth on Kubernetes and BTP’s collaboration with Digital Asset to bring DAML to Sawtooth, read our case study.

Apr 23
Love0

Hyperledger Sawtooth: Improving the Devops CI Workflow with Kubernetes

By Blog, Hyperledger Sawtooth

Every devops engineer knows the importance of continuous integration (CI) testing. It’s vital to prevent regressions, as well as maintain performance, security, and supportability. At Bitwise IO, we are experimenting with Kubernetes as an automated CI deployment tool. We like the simplicity of extending tests with deployments on Kubernetes. We think Kubernetes has compelling potential for use in the CI workflow.

Figure 1: The main tools in our CI workflow

This blog post explains how Kubernetes fits into our CI workflow for Hyperledger Sawtooth. We don’t go into detail, but we provide links so you can learn more about Kubernetes and the other tools we use.

Building Sawtooth

Hyperledger Sawtooth uses Jenkins to automate builds of around 20 GitHub repositories. Each new or merged pull request in the master branch initiates a build that contains project-specific tests. The next logical step is to deploy into a test environment.

We have two deployment methods: Debian packages and Docker images. We install the Debian packages inside the Docker deployment image to ensure both that the binaries are tested and that the packages are installable.

Using Docker’s multi-stage build capability, an intermediate build container makes the deployment image smaller and narrows its exposed attack surface (possible vulnerabilities).

Handing off Docker Images with Docker Registry

Jenkins is a great place for build artifacts, but Docker has its own way to easily retrieve images. Docker Registry allows you to push your newly created images and easily retrieve them with Docker, Kubernetes, or anything else that uses the Docker Registry model.

This example shows how to tag an image with the URL of an internal registry, then upload the image to that registry.

$ docker build -f Dockerfile -t registry.url/repo/image_name:${tag} .
$ docker push registry.url/repo/image_name:${tag}

We also use Portus, because Docker Registry does not provide user and access management on its own. The Portus project makes it simple to place an authentication layer over Docker Registry. Now, any authenticated user can pull and deploy the same images that are being deployed into the test environment.

Kubernetes: Simulating Scaled Deployments

Kubernetes excels at creating deployments within and across abstracted infrastructures. We have done our experiments on local (“on-prem”) hardware with a cluster of small Kubernetes nodes dedicated to Sawtooth deployments. Each deployment consists of several pods partitioned in a namespace, which allows us to run multiple networks based on the same deployment file. (Without namespaces, Kubernetes would think we are updating the deployment.) A pod represents a Sawtooth node and contains several containers, each running a specific Sawtooth component: validator, transaction processor, REST API, and so on. Each namespace can have independent quotas for resources such as CPU time, memory, and storage, which prevents a misbehaving network from impacting another network.

Figure 2: Containerized services grouped together in pods.

Because we use Kubernetes, these deployments are portable. We can use them on any cloud infrastructure that supports Kubernetes.

Kubernetes also allows us to scale the number of CI test deployments. With an elastic cloud infrastructure, Kubernetes provides effortless testing on a large number of virtual systems (limited only by the cost of cloud hosting). This solves the issue of limited hardware resources, where each additional deployment will stress existing deployments when they share a node’s resources.

Workload Generation: Deploying Changes Under Load

Deploying Sawtooth is the first step, but you need to give it something to do—better yet, lots to do. Sawtooth includes several workload generators and corresponding transaction processors. In our Kubernetes environment, we deploy intkey_workload and smallbank_workload at rates slightly above what we think the hardware can handle for shorter runs.

Modifying workload rates is as simple as editing the deployment file, changing the rate settings, and reapplying with kubectl. When Kubernetes detects that the pod’s configuration has changed, it terminates the existing workload pod and creates a new one with the changed settings.

This example shows a container definition for an intkey workload pod.

containers:
  - name: sawtooth-intkey-workload
    image: registry.url/repo/sawtooth-intkey-workload:latest
    resources:
      limits:
        memory: "1Gi"
    command:
      - bash
    args:
      - -c
      - |
         intkey-workload \
           --rate 10 \
           --urls ...

Retaining Data with Kubernetes: Logging and Graphing

All this testing isn’t much use if you can’t troubleshoot issues when they arise. Kubernetes can streamline deployments, but it can also frustrate your attempts to gather logs and data embedded inside Docker containers after a pod has failed or stopped. Luckily, Sawtooth provides real-time metrics (which we view with Grafana) and remote logging through syslog. We actively collect logs and metrics from the Sawtooth networks, even down to syslog running on the hardware, then carefully match the logging and metrics artifacts to the testing instance. In the end, we can provide a comprehensive set of log data and system metrics for each code change.

Try It Out!

The Sawtooth documentation can help you get started: See Using Kubernetes for Your Development Environment and Kubernetes: Start a Multiple-node Sawtooth Network.

To configure Grafana, see Using Grafana to Display Sawtooth Metrics.

See these links for more information about each tool in our CI workflow:

About the Authors

Richard Berg is a Senior Systems Engineer at Bitwise IO. He has several years’ experience in sysadmin and devops roles. When not behind a terminal, Richard can be found in the woods with his snow-loving adventure cat.

Ben Betts is a Senior Systems Engineer at Bitwise IO. Ben has lots of experience with deploying, monitoring, coordinating, and supporting large systems and with writing long lists of experiences. He only uses Oxford commas because he has to.

 

 

Apr 16
Love0

DAML smart contracts coming to Hyperledger Sawtooth

By Blog, Hyperledger Sawtooth

It has been a busy two weeks at Digital Asset… first, we announced that we have open-sourced DAML under the Apache 2.0 license and that the DAML SDK is available to all. Five days later, ISDA (the standards body for the derivatives market) announced DAML as an available smart contract language for their Common Domain Model, and we open-sourced a reference library and application. Next up, we announced that we’ve been working with the team at VMware to integrate DAML with their enterprise blockchain platform, VMware Blockchain.

Today, we’re delighted to share that we have been working with fellow Hyperledger members, Blockchain Technology Partners (BTP), to integrate the DAML runtime with Hyperledger Sawtooth! In this blog post, I’ll describe why we believe it’s important to architect a DLT application independently of the platform, why a new language is needed for smart contracts, and why we are working with BTP to integrate it with Hyperledger Sawtooth.

“Following the recent announcement that DAML has been open-sourced, we are delighted that work is already underway to integrate the DAML runtime with Hyperledger Sawtooth. This demonstrates the power of the open source community to enable collaboration and give developers the freedom required to truly move the industry forward.”

Brian Behlendorf, Executive Director of Hyperledger

One language for multiple platforms

As you all know, the enterprise blockchain space is fairly nascent and highly competitive. There are multiple platforms and protocols battling it out to be the “one true blockchain,” each with their own version of maximalists. Hyperledger alone has six distinct frameworks, each tailored to different needs, making necessary trade-offs to solve different problems. The field is rapidly evolving and we are all learning from the contributions of others to better the industry as a whole. One thing all these platforms have in common: Their purpose is to execute multi-party business processes. The differences arise in how a given platform deals with data representation and privacy, transaction authorization, progressing the state of an agreement, and so on.

And so each platform has its own patterns for writing distributed ledger applications, typically in a general-purpose language such as Java, JavaScript, Kotlin, Go, Python, and C++. The result of this is that developers must pick which framework they want to use and then develop their application specifically for that platform. Their application is now tightly coupled to the underlying architecture of that ledger and if a better alternative arises for their needs, that likely results in a wholesale rewrite.

One of the primary goals of DAML was to decouple smart contracts, the business logic itself, from the ledger by defining an abstraction over implementation details such as data distribution, cryptography, notifications, and the underlying shared store. This provides a clean ledger model accessible via a well specified API. With a mapping between this abstraction layer and the specifics of a given platform, as BTP is developing for Hyperledger Sawtooth, DAML applications can be ported from platform to platform without complex rewrites.

Why do smart contracts need a new language?

DAML’s deep abstraction doesn’t just enable the portability of applications—it greatly improves the productivity of the developer by delivering language-level constructs that deal with boilerplate concerns like signatures, data schemas, and privacy. Blockchain applications are notoriously difficult to get right. Libraries and packages can help improve productivity in some cases, but the application will remain bound to a given platform. Even Solidity, the language of choice for writing to an Ethereum Virtual Machine (EVM), exposes elements of the Ethereum platform directly to the developer. And we’ve seen several examples of how damaging a bug in a smart contract, or even the language itself, can be.

Abstracting away the underlying complexities of blockchains allows you to focus only on the business logic of your project and leave lower-level issues to the platform.

For example, when a contract involves many parties and data types it can be extremely difficult to define fine-grained data permissions in a general-purpose language. DAML allows you to define explicitly in code who is able to see which parts of your model, and who is allowed to perform which updates to it.

As a very simple illustration, consider the model for a cash transfer. DAML’s powerful type system makes it easy to model data schemas—even far more complex schemas than this—directly in the application.

DAML data model

Built-in language elements simplify the specification of which party or parties need to sign a given contract, who can see it, and who is allowed to perform actions on it. These permissions can be specified on a very fine-grained, sub-transaction basis. For example, the issuer of cash does not need to know who owns that currency or what they do with it.

DAML permissions

DAML provides a very clean syntax for describing the actions available on a contract, together with their parameters, assertions, and precise consequences.

DAML business logic

What you won’t find in DAML are low-level, platform-specific concerns like hashing, cryptography, and consensus protocols. You define the rules in DAML, and the runtime enforces the rules that you set out.

If you refer to the examples in the DAML SDK documentation, or the open source code for the set of complete sample applications we’ve provided, you’ll really come to appreciate the full richness of DAML and the simplifying effect it can have on complicated workflows.

Why Hyperledger Sawtooth?

Digital Asset has a long history with Hyperledger, being founding premier members and serving as both the Chairs of the Governing Board and Marketing Committees. In fact, we donated code to the initial implementation of Hyperledger Fabric and the trademark “Hyperledger” itself! I personally worked with Jim Zemlin and team at the Linux Foundation to establish the project and co-founded the company Hyperledger with my colleague Daniel Feichtinger back in early 2014.

We have clearly always believed in the need for an organization such as Hyperledger to exist, to create an open source foundation of common components that can serve as the underlying plumbing for the future of global commerce.

Hyperledger Sawtooth has quickly been emerging as an enterprise-grade platform that exemplifies the umbrella strategy that Brian laid out in his first blog post after joining as executive director. It has an extremely modular architecture that lends itself well to the plug-and-play composability that Hyperledger set out to achieve.

An example of this is that Hyperledger Sawtooth originally only offered support for the Proof of Elapsed Time, or PoET, consensus algorithm; consensus is now a pluggable feature. This modularity is accompanied by a very clean separation of business logic from platform logic, offering developers a degree of ‘future-proofing’ by limiting the amount of code that needs to be changed should a core component such as consensus be replaced.

Modularity also makes Hyperledger Sawtooth very amenable to plugging in new language runtimes. We’ve already seen this in action with Hyperledger Burrow, which integrates an Ethereum Virtual Machine into Hyperledger Sawtooth to support contracts written in Solidity. Incorporating the DAML runtime into Hyperledger Sawtooth similarly enables support for contracts written in DAML as an enterprise-grade alternative to Solidity.

Finally, from a ledger model point of view, many of the Hyperledger Sawtooth characteristics already map well to what DAML expects. Hyperledger Sawtooth’s Transaction Processor has a very flexible approach towards roles and permissions, for example, and is based on a very natural DLT network topology of fully distributed peers. DAML is based on a permissioned architecture and Hyperledger Sawtooth can be configured to be permissioned without requiring special nodes.

What comes next?

Digital Asset and BTP will soon be submitting the DAML Integration to the upstream Hyperledger Sawtooth framework, fully open sourcing our work.

The integration will also be commercially supported by BTP’s blockchain management platform, Sextant, which provides application developers with a cloud-ready instance of Hyperledger Sawtooth. Sextant is already available on the AWS Marketplace for Containers, and DAML support for Sextant will be added in July. BTP expects to support Sextant on other cloud provider support soon thereafter.

BTP is one of Digital Asset’s first partners to use the DAML Integration Toolkit, a new tool designed to enable developers and partners to easily integrate our open source DAML runtime with their own products, immediately offering the benefits of the best in class smart contract language to their end customers. We look forward to any collaboration that brings DAML to even more platforms, including the other frameworks in the Hyperledger family!

To learn more, download the DAML SDK today and start building your applications for Hyperledger Sawtooth!