Meet Sawtooth Lake

By Blog, Hyperledger Sawtooth

By Dan Middleton, Sawtooth Lake Maintainer*

When I was asked to post about Sawtooth Lake, an obvious topic was “What is Sawtooth?”  Well, Sawtooth Lake is a distributed ledger with novel consensus and transaction handling, incubated under the Hyperledger umbrella.

But maybe a more interesting question is “Why is Sawtooth?”  Given the profusion of bitcoin forks and other blockchains why create another system?  In a nutshell, there are usages that shouldn’t be contorted on top of a currency or on top of a centralized database – even if it’s replicated.

1-mrhyperledger-300x94 Originally, Sawtooth Lake was designed to explore scalability, security, and privacy questions prompted by the original distributed ledgers. That mandated a certain modularity that was lacking at the time. Starting from scratch allowed us to employ lessons from those pioneering systems and branch into usages that the original currency ledgers weren’t intended to address. PoET, the new consensus hits scalability, while Transaction Families, our contract logic, narrow the attack surface for contracts while simultaneously broadening the functionality. We also have a keen interest in trusted execution environments and what role that can play in private transactions.

In branching into new business cases, we felt it was important that the system preserve certain tenants of a distributed ledger. That is, in an enterprise deployment, the concept of a distributed ledger shouldn’t collapse into a replicated database. Enterprise participants need autonomy and they have the right to run their own nodes. The set of participants will also be dynamic and the system – particularly consensus – must accommodate that volatility. It is not clear, for example, whether an O(n2) protocol with fixed membership like PBFT can support the scale or volatility of a distributed ledger at production levels. Finally, we observed that, “public” and “private” define a spectrum of authorization policies – not a binary option for a distributed ledger.

Since open sourcing earlier this year, we’ve learned a lot from industry trials. Sawtooth Lake’s pluggable Transaction Families have enabled its use in exploratory projects across several industries, including recent public announcements of a global deployment in finance and as the reference platform for a music industry initiative. Likewise pluggable consensus allows Sawtooth Lake to be used in small tightly held networks or large broadly open networks. If there’s a sweet spot for Sawtooth Lake, though, it’s the Consortium Ledger.


Before getting into the novelties of Sawtooth lake, it’s worth motivating the problem space.  There are plenty of replicated databases.  We want to build a truly distributed ledger.  The business models in this space involve tens to hundreds of business entities.  That puts some unique demands on the underlying distributed system.

2-pluribus-300x113 Consortiums fit the bill for Distributed Ledger Technology (DLT) as a collection of related, but not fully trusting organizations, that can benefit from a shared view of data. One of the promises of DLT is that it creates win-win opportunities for organizations to open up silos. Eliminating intermediaries is an often cited benefit, but to me, DLT isn’t just about efficiencies, it’s about unlocking new ways to grow.

Consider a motivating example from supply chain, let’s say you want your product to use only ethically sourced minerals, but your company is several steps removed from mining the raw materials, refining them, etc. A shared ledger recording each transfer from materials harvesting to product assembly would let you trace the provenance and gain confidence that the end product is ethically sourced. To work well, that shared database needs input from a variety of companies. Without a shared ledger, it’s more difficult for any of the suppliers or retailers to advertise and capture the value of the ethical sourcing. While it might be possible to accomplish that with a single, centralized database managed by a governing entity, a DLT offers better and perhaps necessary features.

In too simple of an example, the supply chain has a single chain of relationships or perhaps a radial network with a center – which, importantly, could be served by a traditional, centralized database.

3-supplychain-768x174Overly Simplistic Supply Chain.

A real ecosystem, however, has a web of relationships. By looking at only one node in that network, its direct connections form a natural group that suggests it could be supported with a centralized database. But by observing that each of these nodes has a nexus of relationships – it’s own centrality – it’s clear there’s no single, central database that makes sense. Consider also the kinds of data that are relevant and not relevant to each peer.

ecosystem_comparisonA DLT is ideal for this kind of scenario where relationships and dependencies are many and varied.

We believe that a truly distributed ledger should provide autonomy to each participant, and that participants should be allowed to run their own nodes. A consortium may develop rules for authorizing participants, but centralizing the nodes and the control defeats the purpose of distribution and fails to satisfy the actual requirements of the participants.

Realistically, participating companies will be dynamic, entering and leaving the system as their business dictate. This mandates a truly distributed system, not just a replicated database.


Realizing this distributed ledger vision requires technology choices that satisfy requirements for autonomy and dynamic membership. In DLT the choice of consensus has a dramatic effect on the system as a whole. Sawtooth employs an experimental consensus algorithm we call Proof of Elapsed Time, or PoET. It’s a form of random leader election, wherein each validator node waits a random amount of time before trying to claim a block. In other random leader election algorithms like Proof of Work, that randomness is enforced by searching for partial hash collisions. PoET replaces that work with trusted computing.

6-nantucket-300x96The benefit of this is a more economical and greener algorithm (as compared to ASIC mining rigs). Rather than chips churning on hash functions, the trusted computing functions allow each node to get on with other work until its timer expires, putting computing cycles into useful operations without wasted electricity. PoET is a return to the Satoshi intent of “one-CPU-one-vote”.

In its original form PoET exists near the unpermissioned end of the policy spectrum. Any host that can provide certain cryptographic proofs about the correctness of its execution can publish a block. The sawtooth-core maintainers have begun adding scaffolding to support the more direct authorization models that consortia may require. The first policies in this new framework include basic requirements of a validator such that it is running the correct version of the consensus algorithm and that it has a verifiably unique identity on the network (an anti-sybil feature of PoET). The network itself validates authorization of each node as it joins the system. Once validated, those nodes and their authorization data are committed to a table on the ledger, called the Validator Registry. Building on these new features, participants in a network can define arbitrarily elaborate (or simple) authorization policies. This authorization framework then lets the underlying PoET consensus algorithm be utilized across a wide swath of the public to private ledger spectrum.

Family Ties

Transaction Families are unique to Sawtooth Lake. Transaction logic is, of course, another area of intense exploration in DLT development. This logic is akin to the stored procedures your grandmother may have written for her old-fashioned steam-powered database. The two most pressing questions about this so called “smart contract” layer are (1) where is the proper architectural divide in data control logic versus application logic, and (2) how expressive should that layer be?opscode_head

With regard to expressiveness, Bitcoin started with too much power and has pared back over its evolution. Ethereum sought to provide more expressiveness but control it with economics. In both cases the appropriate divide between application logic and data control logic has been unclear and the allowed ledger operations have been exploited.

8-contrast-smartcontract-txnfamilies_conjoined-768x453Transaction Families create a smaller surface tailored for each domain.

Sawtooth Lake’s Transaction Family architecture is meant to address the questions of architectural boundaries and transaction expressiveness. Rather than trying to create a new language or define the perfect set of opcodes for all domains, Transaction Families decompose the logic problem by domain. If we tease apart the logic layer into two layers we get a set of rules for a domain and a composition layer for creating versatile transactions within that domain.

Transaction families are at once more powerful and more restrictive than other approaches of logic. Transaction Family code is deployed natively within the validator. That embedded validator code defines a data model and a set of updates associated with the model. The DApps (Distributed apps) author can then compose new transaction logic out of the available update methods. We intend that this be a more secure approach than providing all logic for all domains as a single interface to the ledger. Transaction families more tightly bound the code that must be vetted allowing for more tractable analysis. Consider if you will, how the complexity of the DAO code led to this summer’s Ethereum problems.


Sawtooth deployed with two transaction families: Trading and Authorization

Returning again to the consortium ledger, the consortium can define the extent of flexibility on the data model for the domain they participate in. Take for instance some recent work where Sawtooth Lake was used to prototype a bond and settlement market. For this prototype, a transaction family was created specifically to execute only the required bond settlement operations, eliminating the possibility of executing, unallowed, unnecessary, and illegal operations. The consortium can vet and approve a family before it is deployed to the network. The consortium participants can then still differentiate and establish competitive contracts with reduced risk that unwanted operations ensue.

Similarly domains often have services important to that domain which might be inappropriate or unnecessary in others. Again, in the case of bonds, a time service is important to address payment terms for maturity. The granularity of that time is probably coarser than a supply chain  domain focused on delivery of perishable goods like fish, or where the time source for that perishable good is provided by some external authority.

Finally there is a lot of energy surrounding private transactions. The Sawtooth-Core team is experimenting with private transactions using all the tools available including pure cryptographic techniques such as NIZK as well as trusted computing like Intel® SGX. This is an ill-defined space right now. It’s common to hear explicitly contradictory requirements like, the desire to discover assets on the ledger and settle the transfer of assets between parties while at the same time not listing the assets or parties on the ledger. Proposed solutions include recording only opaque identifiers for transactions but not the transactions or the assets themselves. A database that only has transaction IDs but no transactions or assets probably loses a lot of value. However, it’s possible that more interesting solutions exist using computationally expensive means like zero-knowledge relations or using trusted hardware with private enclaves.

Until next time…

I hope this has been an informative look into Sawtooth Lake. We are making a number of significant improvements at the time of this writing from our persistence mechanism up through transaction family API. Next time I’ll bring an update on those activities. Meanwhile you can find the work in progress in our github repo and docs. We are active on slack and there’s always someone around to help ramp a new contributor.


* Dan Middleton is an employee of IntelⓇ Corporation. The views expressed here are in the context of his role as an open source maintainer of Sawtooth Lake and do not necessarily represent those of IntelⓇ Corporation.

Hyperledger Welcomes Iroha

By Blog, Hyperledger Iroha


We’re pleased to announce that the distributed ledger project, Iroha, has been accepted into incubation status under Hyperledger. Originally developed by Hyperledger member company, Soramitsu, Iroha was inspired by the Fabric architecture and aims to provide a development environment where C++, web, and mobile application developers can contribute to the Hyperledger Project.

What is Iroha?

Iroha seeks to complement Fabric, Sawtooth Lake, and other potential projects, by creating reusable components in C++ that can be called from languages such as Go. In this way, Iroha is additive to existing projects and the long term goal is to realize a robust library of reusable components that can be selected and used freely by those running distributed ledgers on Hyperledger technology.

“Iroha allows even more developers to interact with Hyperledger to build infrastructural projects and applications requiring distributed ledger technology,” said Brian Behlendorf, Executive Director of Hyperledger. “It is encouraging to see member companies actively contributing to a diverse and sustainable open source blockchain ecosystem built on cooperation.”

The design and architecture of Iroha is greatly inspired by Fabric, in that blockchain and chaincode services form the overall architecture. Where possible, APIs were made to be similar to Fabric and, rather than competing with Fabric, the goals of Iroha are to:

  1. Provide an environment for C++ developers to contribute to Hyperledger
  2. Provide infrastructure for mobile and web application support
  3. Provide a framework to experiment with new APIs and consensus algorithms that could potentially be incorporated into Fabric in the future.

Why Iroha?

Currently, the Hyperledger Project lacks an infrastructure project written in C++, thus limiting the potential developers who can contribute. Also, there is not currently a strong focus on user interaction or mobile applications, though both are necessary for the realization of the widespread use of distributed ledger technology. Iroha aims to rectify both of these points, bringing in more developers while providing libraries for mobile user interface development.

Iroha is a distributed ledger project that was designed to be simple and easy to incorporate into infrastructural projects requiring distributed ledger technology. Iroha features:

  1. Simple construction
  2. Modern, domain-driven C++ design
  3. Emphasis on mobile application development
  4. New, chain-based Byzantine fault tolerant consensus algorithm, called Sumeragi

Although turing complete smart contracts are available via chaincode in Java (running a sandboxed JVM), users do not need to write chaincode in order to define digital assets in Iroha. Common use cases, such as deploying new currencies and sending text messages, are available as part of the core framework. Iroha is composed of the following:

  • Iroha core
  • Iroha Native iOS Library
  • Iroha JavaScript Library
  • Iroha Native Android Library

Iroha core provides the distributed ledger infrastructure comprising the data membership services, consensus algorithm, peer-to-peer network transmission, data validation, and chaincode infrastructure. The iOS, Android, and JavaScript libraries provide convenience functions for performing common operations, such as digitally signing transactions. Future work will expand these common functions to interoperate with the Fabric ledger.

Who will work on Iroha?

Soramitsu has committed several full time engineers to the project. Makoto Takemiya of Soramitsu is the initial project maintainer, along with six other engineers at Soramitsu. Besides Soramitsu, the co-sponsors of the proposal and other Hyperledger members are considering committing resources to work on Iroha including Toshiya Cho of Hitachi, Takahiro Inaba of NTT Data, and Mark Smargon of Colu.

Soramitsu is also doing collaborative research with The University of Tokyo, The University of Aizu, and Center for Global Communications (GLOCOM, below) of International University of Japan. From the University of Tokyo, Hideyuki Tanaka will consider economics use cases with Iroha. From The University of Aizu, Yasushi Fujii will explore business use cases with Iroha. From GLOCOM, Soichiro Takagi will consider economics and scientific research using Iroha.

“We’re pleased that Iroha has been accepted for incubation into  Hyperledger,” said Makoto Takemiya at Soramitsu. “By creating C++, mobile, and web development environments for Hyperledger, new developers can join the project and help contribute not only to Iroha, but to other sub projects, such as Fabric and Sawtooth Lake.”

Learn more about Iroha

Working with community members and use case partners, we would like to continue to improve upon Iroha and have it reach and active project stage in the future. The end goal is to realize a suite of components that can be freely interoperable with other Hyperledger projects.

The following repositories on github have been created to manage Iroha resources:


You can learn more about Iroha in this whitepaper or other incubation projects under Hyperledger here. Iroha documentation can be found at For those interested in additional information, please reach out to:

Hyperledger at Money20/20

By Blog, Finance

We’re pleased to share the details of Hyperledger Project’s significant presence at the upcoming Money20/20 conference taking place Oct. 23-26, at The Venetian in Las Vegas. If you will be at the event, we encourage you to attend the Hyperledger activities detailed below.

Member Lightning Talks

On Tuesday, Oct. 25, we encourage you to stop by the Hyperledger booth #425 from 12:30 – 1:00pm for short and engaging lightning talks from our members who will showcase their Hyperledger-powered applications built to address financial market requirements. Stop by during lunch and you’ll hear from the following:

  • IBM: Blockchain Explained: Insights From the First Year
  • IntellectEU: Direct Payments Platform
  • Accenture: Editing the Uneditable Blockchain
  • Intel: Embedded TEE Smart Blockchain Wallet
  • ConsenSys: How ConsenSys uses Hyperledger
  • MonetaGo: Interbank Blockchains for Payments

While you’re visiting the Hyperledger booth, be sure to collect your swag, including lapel pins, adapters, pens and stickers.

Hyperledger Sessions

Brian Behlendorf

Brian BehlendorfOn Tuesday, Oct. 25 from 5:00-5:50pm in Lando, Level 4, Hyperledger Executive Director, Brian Behlendorf, will speak on behalf of the project in a blockchain panel titled, “All Together Now: How Bank & Tech Partnerships Are Shaping the Future of BlockChain.”

Blockchain tech has come a long way. By now, most of the largest banks and some notable tech giants are participating in industry-wide blockchain standard-setting activities, partnering with startups and taking different approaches to determining which use cases would improve upon existing technology. Hear some of the key participants in these cross-industry partnerships discuss how collaboration is shaping the future of blockchain technology. How will these partnerships be successful? And what are the key challenges to overcome? Attend the panel to hear why.


Member and Industry Sessions

Several Hyperledger members and industry thought leaders will also feature in the week’s agenda. Below are confirmed sessions.

Sunday, Oct. 23

1:25–1:50pm | Marcello, Level 4 | Part2: Identity Solutions, Blockchain & Financial Inclusion in Emerging Markets | Economic Inclusion & Financial Health Track | Hyperledger Member: Gem

Lack of ownership of a secure and unique digital identity is a fundamental reason that too many are not able to benefit from a growing middle class in many emerging markets across Latin America, Africa and Asia. Several startups are applying blockchain technology towards self-sovereign identity. This session will explore two very different approaches with the respective CEOs of Banqu and Gem, and explore the potential paths they see towards universal identities, the potential partnerships required, and the friction and pitfalls they are encountering as they implement their solutions in market.

4:10–5:00pm | Murano, Level 3 | Panel–Balancing Blockchain Public Policy: Thoughtful Regulation Without Stifling Innovation | Legal & Regulatory: Risk & Compliance track | Hyperledger Members: DTCC and IBM

In only a few short years, a technology that began as an alternative digital currency has managed to capture the imaginations of thousands of innovators and investors around the globe. This session will explore the public policy landscape for blockchain tech, including the key question of how regulators and operators can balance the fine line between protecting consumers and stifling innovation.

Monday, Oct. 24

11:40am–12:05pm | Lando, Level 4 | Part 1: Open-Sourced: How New Business Models Will Disrupt the Disrupters | Entrepreneurship & Investing track | Industry Thought Leader: Zoë Keating

Platform-based businesses like Facebook and Airbnb have built wildly successful companies over the last decade, but what are the innovations that will disrupt the disruptors? In this session, leaders in the venture capital, blockchain and entrepreneurship space discuss how the next big business model disruption might be found in cryptocurrencies.

4:00–4:50pm | Lando, Level 4 | Part 2: Consumer Protection & Blockchain: Building the New Financial Future | Blockchain Tech track | Hyperledger Member: Blockstream

Digital currencies like Bitcoin and Ethereum promise a more secure and efficient backbone for the financial system—so why do they keep getting stolen? What, if anything, can be done to prevent these thefts? This July, Consumers’ Research convened regulatory, legislative and industry leaders in Bretton Woods, NH to address these concerns. In this session, participants will unveil findings from the Bretton Woods conference and share perspectives on how to realize the promise of digital currencies while also protecting consumers.

4:00–5:50pm | Palazzo Ballroom J, K, L, Level 5 | StartupPitch180–Earlier Stage Companies | StartupPitch180 track | Hyperledger Member: Loyyal

Expanded and enhanced for 2016, StartupPitch180 will feature 30 diverse and innovative startup companies presenting their businesses in 180 seconds. Participating companies will compete for cash prizes and a spot on the main stage in between the keynote speakers later in the conference. The action is fast paced, interactive and fun, as judges from leading VC firms and audience voting will determine the 4 winners from our curated list of startup companies.

Tuesday, Oct. 25

5:00–5:50pm | Lando, Level 4 | Panel–All Together Now: How Bank & Tech Partnerships Are Shaping the Future of Blockchain | Blockchain Tech track | Hyperledger Members: ConsenSys and Digital Asset

Blockchain tech has come a long way. By now, most of the largest banks and some notable tech giants are participating in industrywide blockchain standard-setting activities, partnering with startups and taking different approaches to determining which use cases would improve upon existing technology. Hear some of the key participants in these cross-industry partnerships discuss how collaboration is shaping the future of blockchain technology. How will these partnerships be successful? And what are the key challenges to overcome?

Booth Exhibition

While onsite at Money20/20, we encourage you to take advantage of the opportunity to meet and greet with your fellow Hyperledger members. Below is a listing of the Hyperledger member exhibition booth stand numbers at Money20/20.

  • Hyperledger Project: 425
  • Accenture: 3601
  • ConsenSys: SR18
  • Fujitsu: 1031
  • IBM: 1033 and MC131
  • Intel Corporation: 3606, 3607, and the lounge in 219
  • Intuit: 837
  • J.P.Morgan Chase: 1622, 1626, 3805 and 3806
  • NTT DATA: 1227
  • SWIFT: 1146
  • Broadridge and Wells Fargo will also be onsite Money20/20.

We look forward to seeing you in Vegas!

Hyperledger Hackathon and Hackfest Recap

By Blog

The first ever Hyperledger Hackathon took place October 1-2 in Amsterdam. More than 120 developers (20 teams) gathered for the event organized by ABN AMRO, IBM, Holland Fintech and Hyperledger.


During the Hackathon, teams of students, fintechs, start-ups, and staff from IBM and ABN AMRO had 36 hours to build their own application on Hyperledger. As part of their efforts, they were allowed to tap the expertise of  visiting experts from the Hyperledger Technical Steering Committee and others. At the end of the 36-hour period, a jury assessed the teams’ proposals and applications with a focus on application value to clients and business as well as its technical implementation.

The energy and enthusiasm of the teams was incredible to watch. Each team had a deep appreciation of the technology and how it could be applied. A few of the teams even slept at the venue, working through the night to polish their implementation and pitch!

The winning team (pictured below) built a practical application for the storage of medical information, controlled and updated by medical experts and patients as owners of the data. Each team member won an Oculus Rift.


Following the Hackathon was the Hyperledger Hackfest, which took place on October 3-4 in the ABN AMRO Innovation Centre. 60 developers attended. This marked the first time the bi-monthly international Hackfest was held in Europe.

Some of the agenda items addressed at the event included:

  • Overview of consensus model evolution
  • Data confidentiality enhancements (selective distributed ledger)
  • Future support for other chaincode languages
  • Starter kit tutorial
  • Brian and Chris held a session for our ABN AMRO hosts on what the Hyperledger project is all about
  • Brian held a session on new potential projects
  • There were a number of ad hoc breakouts with various maintainers present (Chris, Gari, Binh, Murali, Tamas and Gabor)
  • Fabric maintainers held a meeting to discuss release process
  • Hyperledger Architecture WG also held a session at the Hackfest, led by Ram Jagadeesan from Cisco


If you are interested in further engaging with the technical community, sign up for one or more of the mailing lists at


Hyperledger Announces the Hyperledger Healthcare Working Group

By Blog, Healthcare
Today, Hyperledger is announcing the formation of the Hyperledger Healthcare Working Group (HLHC Working Group).

Health data sharing, like much of health IT, has traditionally lagged other industries in adoption of new technologies. This is not for bad reason – people’s lives and health privacy are at stake, as is 20% of GDP. The potential of blockchain technology, applied to healthcare, is a shared platform that decentralizes health data without compromising the security of sensitive information. This model lifts the costly burden of maintaining patient’s medical histories away from the hospitals: eventually cost savings will make it full cycle back to the patient receiving care.

The HLHC Working Group’s mission is to house and foster technical and business-level conversations about appropriate applications for blockchain technology in the healthcare industry. These conversations will be broad and educational, but will eventually focus on identifying opportunities for near-term collaboration between participants on common software to implement a given application. If appropriately scoped and resourced, these conversations could lead to one or more proposals for new software development efforts to be hosted at Hyperledger.

“There are two really important questions to focus on. First, since there are so many use cases where blockchain could be used in healthcare, where will blockchain (plus smart contracts) provide capabilities not readily enabled by pre-existing approaches? Second, as blockchain capabilities and variants continue to evolve rapidly, how do we support parallel initiatives that don’t create forking that introduces friction into the flow of data and services?” said Dr. John Mattison, Assistant Medical Director, Kaiser Permanente. “We are truly sitting on a ‘TCP/IP moment’, and while it’s impossible to declare ‘what’ the solution is to avoiding that forking, we can come together in this workgroup to define ‘how’ we collaborate to minimize the risks of inadvertently introducing friction through forking.”

The HLHC Working Group, which does not require Hyperledger membership, will center around discovery and exploration of healthcare-related blockchain use cases that address real world problems. Initially, the group will focus on fundamental Distributed Ledger applications, such as establishing registries, interoperability and identities. Technical opportunities will also be discussed. In time, the discussion will expand to more advanced topics, such as smart contracts and process automation.

“The power of Open Source is the power of collaborative ideas. The Hyperledger Healthcare Working Group enables Healthcare enterprises, providers and users to focus resources on a scalable open source project to leverage collaboration, said Dr. Merve Unuvar, IBM Blockchain Product Leader. “The beating heart of the emerging Healthcare Blockchain will be powered by Hyperledger enabling data sharing, privacy and interoperability.”

“The healthcare industry needs to be organized in order to realize the potential for blockchain technology,” said John Bass, Founder and CEO, Hashed Health. “The Hyperledger Healthcare Working Group is a destination for those who want to contribute to the development of technical standards necessary for blockchain innovation to be meaningful.  The team at Hashed Health is extremely excited to join the Group as a founding member driving development efforts with our peers.”

The HLHC Working Group already includes participants from organizations including Accenture, Gem, Hashed Health, Kaiser Permanente and IBM. If you are interested in participating in the HLHC Working Group, please visit the Healthcare industry page to learn more and join the mailing list.

Meet Hyperledger: An “Umbrella” for Open Source Blockchain & Smart Contract Technologies

By Blog

It’s hard to believe I’ve been working at The Linux Foundation on Hyperledger for four months already. I’ve been blown away by the amount of interest and support the project has received since the beginning of the year. As things really start to take off, I think it’s important to take a step back to reflect and recapitulate why and what we’re doing with Hyperledger. Simply put, we see Hyperledger as an “umbrella” for software developer communities building open source blockchain and related technologies. In this blog post, I’m going to try to define what we mean by “umbrella,” that is, the rationale behind it and how we expect that model to work towards building a neutral, foundational community.

The Hyperledger Project was initially seeded with various blockchain-supporting commercial members, some of whom had interesting internal or nascent open source efforts that needed the kind of home that the Linux Foundation could provide. It emerged at a time when it was clear that three points needed to be made to the market:

  1. Open, transparent governance of the software development process for blockchain technologies matters
  2. Intellectual property provenance and safeguards of the software matters
  3. Key use cases are driving permissioned or “consortium” chain models

Out of that set of initial principles, the Technical Steering Committee determined that the two initial projects should be delivered into the project in an “incubation” state: Fabric and Sawtooth Lake. These two projects differ significantly in many ways, and thus could perhaps provide different, yet also sometimes overlapping purposes.  In the spirit of putting all the wood behind the tip of the arrow, attention and efforts began to focus on those two.

While still in a pre-alpha state, Fabric is attracting growing developer attention and significant mindshare around its unique approach. We are seeing additional interest in new projects that build directly upon Fabric, but exist separately and may have different release schedules and priorities. For example, the chaincode management tools, or the Hyperledger Explorer (initially). We would expect to see further exploration of work that has already been demonstrated in concept, such as hooking up the Ethereum virtual machine to Fabric. And Sawtooth Lake’s unique approach to consensus – Proof of Elapsed Time – is garnering attention, too.  Both Fabric and STL just cut another developer preview release.

As both Fabric and Sawtooth Lake have picked up new contributions and developer momentum, and the commercial interest has continued to grow, we are now seeing interest in applying this model to additional technology efforts, potentially leading to new projects at Hyperledger. In some cases those will be a useful spin-out of an existing project and community; other times they will bring in a new community of developers, both on the new project and spilling over onto existing projects.

The Rationale: Why an “Umbrella”?

Blockchain and smart contracts are still in the early stages of a 20-year, if not a 50-year, adoption and maturation cycle. Some have compared it to 1994 and the Web (MIT’s Joi Ito sees it as 1989). There are clear examples of efforts that have seen widespread adoption and scale: Bitcoin and Ethereum. There are clear examples of commercial blockchain stacks, running in production. There is clear momentum around Fabric and Sawtooth Lake. Yet by no means is this a mature industry – we are still seeking better consensus mechanisms for both permissioned and permissionless chains, a better range of choices for smart contract platforms, and still exploring the right identity models. We have no idea if one specific package of software can serve all these needs at the same time, or if the approaches are as divergent as, say, ACID-compliant SQL databases and eventually-consistent NoSQL databases. Furthermore, some needs may span multiple existing efforts – a graphical user interface, for example, that could just as easily span Fabric and Sawtooth Lake.

What we do know is that there are no software development resources to spare. There is a global talent shortage for developers who understand not only cryptocurrency and blockchain engineering challenges, but who also understand distributed systems. The guts of these platforms are not unlike the complex balancing acts that operating system kernels or hardcore database efforts can reflect. Debugging multi-threaded applications was a challenge when all we knew were single-threaded applications; now debugging distributed applications is that much harder. Software that wouldn’t benefit from a rewrite doesn’t exist (or fresh thinking on old problems). Given the amount of duplication of effort we see today on the same core functions, we need to constantly be looking for opportunities for developers to be working on common code and roadmaps whenever possible. We’re not only building consensus mechanisms, we need to live them as a development priority.

In this environment, the most valuable role the Hyperledger Project can play is to serve as a trusted source of innovative, quality-driven open source software development community, creating modular, open source components and platforms. The optimal focus of Hyperledger is to advance industry goals of distributed ledger and smart contracts. Hyperledger will forge a brand that will be seen widely to reflect the accepted default “safe” deployment platform for enterprise teams, and be seen as a great home for active collaboration around new technologies, only then our mission will be accomplished.

Another way to think of this is to consider Hyperledger’s potential role in the emerging landscape of public blockchain technologies. Most of today’s open source blockchain efforts outside of Hyperledger are focused on permissionless chains, necessarily implementing a cryptocurrency as a means to fund mining and participation in consensus. This has tremendous challenges, and not all of them are technical, as the debates over the Bitcoin blocksize or the DAO demonstrate. What may appear to be technical debates at first glance often are really about different visions for the roles these platforms should play in society and who should govern them. We’ve crossed this bridge before, though. The Domain Name System was fortunate enough to rise to ubiquity long before anyone outside the early Internet architects realized how important it was – how important having a widespread network of root nameservers all consistently serving up the same answers for domain names would be. To get there, we might have started with a few developers building a better replacement for “/etc/hosts”, but which eventually evolved into a three-part structure: standards bodies (IETF), implementers (sendmail, postfix, qmail, etc), and global governance (ICANN).

Hyperledger Pie Chart

Mapping this to today, there is no reason why a particular cryptocurrency needs an entirely novel technology stack. The scaling needs of a particular cryptocurrency has tended to drive the technology roadmap for a given stack, but the code behind ETH and ETC (Ethereum and Ethereum “Classic” tokens) is almost entirely the same. The configuration settings, mining community management, trademark questions, and regulatory agency and law enforcement relationships will likely differ, and are really a matter for the global governance of ETC and ETH as cryptocurrencies.

“The most valuable role the Hyperledger Project can play is to serve as a trusted source of innovative, quality-driven open source software development community; creating modular, open source components and platforms; all focused on distributed ledger and smart contract technologies. If Hyperledger can forge a brand that is widely seen as the accepted default ‘safe’ deployment platform for enterprise teams, and be seen as a great home for active collaboration around new technologies, then I think we can say ‘mission accomplished’.”

If Hyperledger could help not only forge common ground between different software development efforts, but also encourage a gradual detachment between standards, implementations, and global governance (whether that’s around currencies or other use cases), then we will also accelerate adoption of blockchain tech widely and further reduce needlessly duplicated engineering and hardening efforts.

Perhaps most importantly, we can directly address what many have observed as a major challenge with the existing open source blockchain efforts – tremendous levels of tribalism amongst developers. While invigorating, it can also make sharing code between efforts, or talking about common challenges and how to meet them, notoriously difficult. This is true even when the payoff would be less duplicated code and more eyes looking for security holes and other issues.  Multiply that rivalry with the effects of holding fungible currency whose value can be tied directly to the software in question, or open source project brands tightly associated with commercial brands in which developers own equity, and incompatible copyright license paradigms, and working together can be nearly impossible.

At Hyperledger we believe we can provide an answer to this.  Let’s bring these different implementation efforts within the same “home”, with a consistent approach to intellectual property, community collaboration standards, overall branding (“Hyperledger ____”) and an encouragement to either work together or usefully differentiate.  If we do this, it will remove barriers to collaboration, encourage developers to find opportunities to work on common code, and address the potential for confusion and wasted duplication of efforts without requiring a top-down single architecture or personality to dominate.

The Model

What does being a project under the Hyperledger banner actually mean? Consider the Apache Software Foundation. Within the ASF, after nearly 20 years of existence as a community of communities, there are nearly 300 different “top-level” software projects. Each has its own charter, developer community, roadmap, development process, etc. They mostly use the same collaboration tools, the same IP framework (the Apache license, and contributor license agreements), they use the same reporting process to the Board of Directors, and they must demonstrate the same sort of community-driven development as established and evaluated by the Apache Incubator. There are several competing projects at the ASF; this is not a problem, as projects that fail to get or sustain a critical level of participation and responsiveness to the needs of its users can simply be retired by the Board when it’s clear they’ve lost it.

There are many parts of this we would not copy verbatim, but it’s a model with proven success at turning new efforts into home runs (see Hadoop, Spark, and countless others), and a clear template understood and trusted by a broad community. In our case we can be a bit different. The Linux Foundation can provide many services that Apache depends upon volunteers to provide, from project coordination and developer collaboration tooling, to hosting and conducting virtual and physical meetings, to worrying about developer contributor agreements and trademarks and other legal issues, and more.

With this as inspiration, and through discussions at the Technical Steering Committee, we’ve come to define a  Hyperledger project as consisting of:

  1. An identified set of software developer “maintainers” who are responsible for the development process, culture, and general technical direction of the project, and engaging the public. The developers can vote in new members and people may retire, but any active project needs at least a few who maintain a heartbeat of activity, if not more.
  2. A bounded set of artifacts, including one or more Git repositories, a bug tracking/issue database, a wiki, a set of mailing lists, and other developer resources (e.g. forum, mailing lists, IRC/Slack channels, etc) tied together as part of the same particular project, and which the maintainers directly and the broader community indirectly are responsible for keeping updated and active.
  3. Dedicated space within Hyperledger to describe the project and the community, with a clear delineation on the relationship and differences with other efforts at Hyperledger (or elsewhere), and an indication that each is equally important to the overall Hyperledger effort.

That’s it. From that, all else can be derived. Ultimately, we want to increase the impact of Hyperledger across the open source blockchain landscape. The increased level of traffic to our community, and the ability to serve many different points across the map, can only enhance our ability to serve the broader blockchain community.

In my next blog post, I will go over what it means to be a part of Hyperledger; the benefits, responsibilities, and the process for launching new projects under Hyperledger.