Category

Blog

Reducing Government Red Tape: British Columbia Creates New Business Identity Model with Hyperledger Indy

By | Blog, Hyperledger Indy

In Canada, starting a new business can be like navigating a maze with three levels: local, provincial, and federal. A helpful checklist on starting a business in Canada offers 60+ links to explore. All this red tape frustrates entrepreneurs and governments alike.

To lighten that burden, teams from the governments of British Columbia (BC), Ontario and Canada have started an ambitious project to fight red tape using decentralized identities and trusted credentials. They are jointly building the Verifiable Organizations Network (VON), to provide an open source software stack that helps business people establish trusted and enduring digital identities and speed up applying for permits and licenses from government agencies.

The first project to emerge is OrgBook BC, an online directory that makes finding authentic and authoritative data about companies faster and easier. Launched in January of 2019, it is designed to reduce the time to do due diligence on a new supplier or client from hours down to a few seconds.

Of course the path to this launch included a number of key development challenges, strategic technology choices and partnership across governments and the larger Hyperledger community. For more on all of these steps, check out this detailed case study.

Now that OrgBook BC is live, the VON goals are expanding the applications, growing the footprint and encouraging more jurisdictions to use the software stack until the network effect takes hold. To get the larger developer and business community engaged, the team is hosting BootCampBC as part of #BCTechSummit. The Honourable Jinny Sims, Minister of Citizens’ Services and MLA for Surrey-Panorama, will be opening the event, underscoring the government’s commitment to driving the growth and adoption of VON.

BootcampBC, hosted by technology leaders from Hyperledger Indy, Verifiable Organizations Network (VON) and the Sovrin Foundation, will cover the concepts of self-sovereign identity and guide participants through building their own Hyperledger Indy agent to verify and issue verifiable credentials using VON’s technology.

Meet Robert Palatnick: The New Chair of Hyperledger’s Governing Board

By | Blog

We’re excited to learn more about the new chair of Hyperledger’s Governing Board, Rob Palatnick. We asked him a few questions about his role at DTCC, how he’s been involved in the community so far and what he hopes to achieve while he is Chair. Check out what he had to say below!

Describe your current role, background and why you wanted to be chair of Hyperledger’s Governing Board?

I am a Managing Director and Chief Technology Architect for DTCC, where I am responsible for the global IT strategy, architecture, standards and engineering design of the systems and applications that support the firm’s broad range of products and services. I also sponsor our Office of Fintech Strategy in partnership with DTCC Product Management leadership.

I have been working in Fintech since I started my career in 1980. The first 14 years of my career were spent working for small software companies including those in banking and financial services, and the past 26 years have been spent at DTCC helping drive our technology transformations.

It is an honor to accept the position of chair of Hyperledger’s Governing Board in succession to Blythe Masters. Blythe guided the board during a critical time when Hyperledger began, when enterprise Distributed Ledger technology (DLT) was first being developed, was untested and embryonic. Since that time, with Blythe’s oversight and the contributions of the Governing Board and the community, Hyperledger now counts 12 active projects, over 260 members and a partnership with the Enterprise Ethereum Alliance as achievements. This is wonderful growth and a testament to the board’s leadership and oversight.

We have now moved beyond start-up mode, and our focus has increasingly turned to real-world application and operation. The next chapter will center around thoughtful guidance of the Hyperledger project portfolio and driving increased use of Hyperledger’s software, with adoption by global communities, across many industries. Within my role as chair and working closely with other Governing Board members, I hope to leverage my knowledge and experience building critical industry scale platforms to help guide Hyperledger to focus on the best priorities to support enterprise scale production implementations. We will also seek to strengthen confidence that DLT can improve the safety and soundness of any market or industry.

How is DTCC currently using Hyperledger technologies or how do you plan to?

DTCC has been active in the Hyperledger community since its founding in January 2016, hosting the very first Hyperledger meet-up in our New Jersey office.  We have also hosted other meetups in our Tampa office and our office in Chennai, India. We have been very active in the code community, actively leading and contributing code to various Hyperledger projects and leading the Hyperledger Explorer project.

Today, DTCC is leveraging open source from multiple Hyperledger projects. We are leveraging Hyperledger Explorer code to build out administrative software for node governance for a project which re-platforms our credit derivatives Trade Information Warehouse (TIW) onto DLT. We are also using a Hyperledger DLT stack for an internal pilot and are exploring other opportunities with the Hyperledger Caliper, Quilt and Burrow projects.

What are the benefits of Hyperledger’s open community and governance model?

Hyperledger has established itself as an umbrella organization for the open source community to develop code for real-world business transactions, and is open to all industries and vendors, and welcomes multiple protocols and innovative models. This contrasts with other organizations that primarily support an open source version of a single commercial product. The open door to new projects and new models is truly critical in supporting the dramatic innovation in DLT today and vital to the future growth of DLT across the financial services industry and beyond. Unlike early public blockchain models which are specifically designed to address a ‘3rd party trust’ problem through consensus protocols that require mining and incentive models, Hyperledger has focused on industry strength and non-functional requirements such as performance, scale, finality, testability and operations. The Linux Foundation brought a tremendous model for successfully engaging the community, organizing projects, maturing releases, testing to ensure quality and pushing out code.

What’s the most important milestone for Hyperledger to reach by the end of 2019?

If you asked that question over the last three years, my answer probably would have been “create the community, establish the brand, and push out code”. For 2019, I think it will be important to ensure we continue to deliver value and satisfy our membership.

We know that Hyperledger projects are achieving code delivery goals, and we know that there has been tremendous response, in the form of Hyperledger code downloads, from the global community. We also hear that Hyperledger code is forming the platform foundation for many important initiatives across industries around the globe. For 2019 the target should start shifting to member satisfaction and value. It is important that we ensure we are addressing community priorities, are meeting expectations and requirements such as scale and performance as Hyperledger project adoption becomes more widespread and moves from pilot into wider scale production.

What’s the one thing you hope to accomplish by being a part of Hyperledger’s Governing Board?

My main goal as a governing board member has been to bring the perspective and voice of a critical financial industry utility to the development of open source DLT that has the promise of replacing and improving significant portions of the technology support that industry. Industry infrastructure must be high quality, resilient, secure, performant and properly governed to deal with inevitable processing exceptions and unexpected conditions. If Hyperledger’s products are built to meet those requirements, then I have accomplished my goal.

What’s the biggest struggle or challenge you see Hyperledger having to overcome?

The near-term challenge is navigating through, as Gartner states, “the trough of disillusionment” that DLT did not meet expectations and change the world overnight. The reality is that, in the enterprise world, things are proceeding exactly as you would expect, if this technology is going to become a critical foundation for the future. The enterprise strength code had to be built and implemented in significant real-world deployments and every day we read about new pilot deployments. Over the next few years we will see which of those models really provide value and grow their communities. We will also see an emerging challenge of DLT proliferation, and a need for interoperability.

Finally, and perhaps most critical for Hyperledger, is to ensure we are established as the open source model and umbrella organization for enterprise blockchain projects. We will see the inevitable consolidation of technology models, which has occurred in previous technology innovation cycles. I believe in the model of open source, and that distributed, decentralized, ledger should be available through open source and not solely depend on technology ‘owned’ by any one vendor. But I also believe that at this early stage we must continue be open to innovative ideas and proposals for additional and alternative foundational components for the emerging ecosystem. This is the biggest source of opportunity for Hyperledger: to provide the open source building blocks for enterprise capable DLT built on community standards and architectures.

What are you most excited about regarding blockchain and distributed ledger technology in 2019? What do you feel is next for the industry?

It is exciting to play a part in this new technology wave. In 20 years, I hope we will be able to look back on this period similarly to the start of the Internet or the early days of database models – when foundational concepts incubated, security, privacy and trust models were considered appropriately, standards for protocols and interoperability were developed and the right governance models were put in place, all of which allowed the technology to proceed and reach the inflection point of adoption in the best possible way.

Hyperledger, Linux Foundation Open Source Networking and Telecom Community Launch New Special Interest Group at Mobile World Congress

By | Blog

Hyperledger is launching a new cross-industry Special Interest Group (SIG) to facilitate focused technical and business-level conversations about appropriate use cases for blockchain technology in the telecom industry. This news comes as part of the Linux Foundation’s participation at Mobile World Congress.

The telecom industry has one of the most complex operations framework, involving many partners, vendors, customers, distributors, network providers. There are a lot of trust issues and transparency challenges due to the involvement of multiple entities, and no clear mechanism to track end-to-end activities of every entity and blockchain can solve main issues in telecom. Blockchain is a decentralized and distributed ledger technology with the potential to deliver trustless operability, immutable and auditable to the telecom industry

To widen the discussion, this SIG will collaborate with other Hyperledger groups, Linux Foundation Open Source Networking, the TSC, Linux Foundation staff, and the project maintainers. The Telecom SIG is interested in collaborating with other Hyperledger and non-Hyperledger groups that aim to identify and share blockchain solutions that increase impact in global development.

The Hyperledger Telecom Impact Special Interest Group

The Hyperledger Telecom SIG is primarily focused on serving as a platform for exchanging ideas and exploring ways to use blockchain technology to advance development and adoption of telecom-specific blockchain implementations. The Telecom SIG will work with the other working groups and technical teams, especially in the areas of implementation.

The areas of focus include:

  • Identifying related proofs of concepts, current pilot, use cases and functional architecture;
  • Working towards building telecom blockchain consortium;
  • Sharing stories of successes, failures, opportunities and challenges;
  • Identifying conferences or other opportunities to connect face to face, as well as submit talks or present as a group at an event.

The  initial work products will include a set of documents describing telecom use cases, white papers, technical solutions for blockchain technology. Some of the use cases  include: SLA Management, ONAP, OSM, Multi-Domain Orchestrator, 5G, LF Edge, Network Slicing, IIoT, SDN and NFV.

Join us at Mobile World Congress in Barcelona

If you are in Barcelona for Mobile World Congress from February 25 to 28, join us to talk more about this new Telecom SIG.  The Linux Foundation will have a booth at the event at [BOOTH DETAILS] and will be hosting a meetup on February 25 that will feature Hyperledger Executive DIrector, Brian Behlendorf, and interim Telecom SIG Chair, Vipin Rathi from Jawaharlal Nehru University’s School of Computer and Systems Sciences.

How to get involved in Hyperledger Special Interest Groups

SIGs gather community members from an industry segment to work on domain-specific problems and create an environment for open discussion, document co-creation and solution proposals. SIGs help specific vertical markets in their efforts to address problems specific to that particular community. To date, Hyperledger has five SIGs, including ones focused on healthcare, the public sector and social impact.

Participation in any SIG is open to everyone in the community. Each group has an open e-mail list, a Chat channel, and a wiki page. Live meetings are also held regularly via web teleconference. When needed, a task force can also be created within the SIG and have working sessions to discuss specific work items.

If you’re interested in joining the Telecome SIG, please subscribe to mailing list and join the Chat channel where meeting details will be announced. You also can visit https://wiki.hyperledger.org/display/TCSIG or find a list of other community meetings on the Hyperledger Community calendar. We look forward to your participation and contributions!

Introducing the Hyperledger Smart Contract Working Group

By | Blog

Welcome to the Hyperledger Smart Contract Working Group!

The Smart Contracts WG will define concepts regarding smart contracts and produce materials to describe the various aspects and meanings, trying to come up to standards or best practices. The audience for smart contracts is large and spans from researchers, developers, businessmen, decision makers, policy makers, law makers, software users, citizens to governments, banks, financial institutions, insurance providers, etc.

The Smart Contract WG will have two main areas of focus with underlying topics of interest for each.

Technology-oriented focus:

  • Models of and mechanism for computation
  • Formal guarantees on outputs of smart contracts
  • Smart contract packaging, code reuse, and dependency auditing
  • Generation of smart contracts from existing artifacts (natural language, business process, state machines, non smart-contract code)
  • Data structures and state
  • Privacy
  • Tooling and compilers for existing virtual machines
  • Design Patterns for Smart Contracts
  • Upgradeability of smart contracts

Law-oriented focus:

  • Smart contracts as representatives of obligations and fulfillment
  • Smart contracts law enforcement cascading actions

Participation is open to everyone in the community. The WG has an open email list, a chat  channel, and a wiki page. Meeting details, meeting notes, and documentation are accessible to everyone in the community. The following resources are available:

If you’re interested in joining, please subscribe to the mailing list and join the Chat channel where meeting details will be announced. Hyperledger has several other very active community Working Groups, learn about all of them here. You also can find a list of other community meetings on the Hyperledger Community calendar.

Walmart turns to blockchain (and Hyperledger) to take on food traceability and safety

By | Blog, Hyperledger Fabric

In today’s high tech world, the global food supply chain delivers fresh meat, produce and more to us year round. The good news is that this massive supply chain means a consumers have access to a huge diversity of food that is generally safe to eat. Still, occasionally it can make us sick. In 2018 alone, there were 18 outbreaks of foodborne illnesses in the USA, including the E. coli found in romaine lettuce.

For this reason, Walmart has always been interested in enhancing transparency and traceability in the food system. The company has tested many approaches to tackling this challenge before turning its attention to blockchain. Two successful Proof of Concept (POC) projects, one in the U.S. and one in China, solidified Walmart’s commitment to blockchain and Hyperledger Fabric. The company is now leading industry-wide adoption of the technology with a coalition involving some of the most prominent players in the food industry, like Nestle and Unilever.

Walmart’s success working with Hyperledger Fabric and the Hyperledger community to build this large-scale solution and growing industry coalition shows the power of multi-stakeholder, open source approach to blockchain:

  • Since the food traceability system is meant to be used by many parties, including Walmart’s suppliers and even direct competitors, an open and vendor-neutral technology ecosystem was a core requirement from the start.
  • Interoperability with other blockchain systems is also key for a cross-industry solution, making Hyperledger’s collaboration with Ethereum vital for long-term success.
  • The diverse community working together on the development of Hyperledger Fabric meant that solutions were already in the works for many issues the Walmart team identified as they built and scaled their solution.

For details on how Walmart went from a test case with mangoes to tracking more than 25 products from five different suppliers via Hyperledger Fabric, read our case study. It includes the a step-by-step account of Walmart’s rollout, from embracing the general idea of blockchain to selecting a core technology to planning the POC projects and taking the system global with partners. It also featured tips from Frank Yiannas, former Vice President of Food Safety at Walmart, on rolling out an enterprise blockchain project.

HIMSS 2019 – Myndshft Delivers a Recap

By | Blog, Healthcare

HIMSS19 took place in Orlando, FL the week of February 11 and this was the third year Hyperledger had a presence at the conference. HIMSS is the leading Healthcare Information and Technology Conference attracting over 45,000 people from 45+ countries each year. Member companies of the Hyperledger Foundation are invited to spend time working the booth, and I had the opportunity to represent Myndshft interfacing with attendees and answering questions about how blockchain technologies fit into healthcare.

Thoughts on Hyperledger Booth at HIMSS

Most people inquiring about blockchain at HIMSS this year were looking for ideas, thoughts, and examples of how blockchain could and should be implemented in healthcare. HIMSS attendees have been following blockchain topics for the last three years (since Hyperledger first had a presence at HIMSS) and many are savvy to how blockchain provides data immutability, creates a network of members, and the general concept of a distributed ledger. There were a few recurring questions the visitors to the Hyperledger booth asked on a variety of different topics, but they all circled around the idea of where blockchain makes the most sense to be implemented.

*NOTE*: There were quite a few questions about how Hyperledger fit into the world of Bitcoin, Ethereum, and cryptocurrency in general, but most of these discussions ended up at one of the following places below after a basic explanation that Hyperledger focuses on blockchains for business and operates differently than completely public blockchains that utilize cryptocurrency.

The three big questions I heard repeatedly at the booth were:

‘Should I use blockchain to enhance my security?’

‘How am I supposed to integrate my existing technology into a blockchain?’

‘Where is blockchain being used today in healthcare?’

Let’s break them down into each of their parts:

Is blockchain for security?

People still aren’t sure how blockchain security works. Many visitors were conflating the cryptographic properties of blockchain with the idea that they need less security infrastructure to run a blockchain. The question asked at the booth a few times was “Explain to me how a blockchain isn’t hackable?” The discussion that followed typically included an explanation that all existing security measures need to be in place for a blockchain network to be secure, but when you use blockchain, the historical data on the chain isn’t able to be changed.Thus individual records aren’t able to be manipulated without notice, providing organizations with minimal trust to work from a common record of truth.

How can I integrate with my current systems or other blockchains?

Many people wanted to know how data got into and out of a blockchain network and how they were able to continue using their existing systems. The majority of participants had concerns that adopting blockchain would require a rip-and-replace of the IT systems their organization has already spent millions of dollars on. The reality is that Hyperledger blockchains like Fabric and Sawtooth are able to coexist with existing IT systems and provide functionality that their existing systems don’t have and never will have without a network. Listening to the conversations around data integration and interoperability was a good reminder that the SDKs provided for the different blockchain projects within Hyperledger need to be a top priority for development. Without good SDKs and the ability for non-blockchain engineers to use them effectively, it is unlikely that blockchain will gain much adoption beyond some niche use cases.

Where is it being used today?

The last question people had was around how blockchain is being used today in healthcare. There are few companies that already have blockchains up and running for different parts of the healthcare ecosystem today.

Myndshft is a company building a network of payers and providers to bring transparency, speed, and auditability to the prior authorization process. The network  integrates with existing systems of record and use events in those systems to drive execution of “smart contracts” (the rules that govern how a blockchain operates), which automate the prior authorization before writing the results back to the providers system.

Change Healthcare is also running a blockchain for claims processing and monitoring. In conjunction with its clearinghouse, Change is using blockchain to ensure all claims that are processed are auditable and traceable.

Synaptic Health Alliance and the DokChain Alliance are also groups looking into blockchain and how it fits to drive efficiencies and collaboration in healthcare.

Overall, the interest and education around blockchain in the healthcare community is growing rapidly, and nobody is wanting to be left behind or appear as a laggard in their implementation and adoption of blockchain technologies. For more information about what Hyperledger is doing in healthcare or how Myndshft is solving complex administrative problems, please reach out to me via email (tyler@myndshft.com) or join us in the Hyperledger Healthcare-SIG Rocket.Chat. 

Hyperledger Sawtooth events in Go

By | Blog, Hyperledger Sawtooth

This blog post shows how to use the Sawtooth Go SDK for event handling. We can create events in a transaction processor and write a client to subscribe to events via ZMQ. There’s also a web socket based event subscription feature not in scope of the topic discussed here. The article has information useful for developers, technical readers, architects, Sawtooth enthusiasts and anybody interested in blockchain.

Please refer to Sawtooth Go SDK reference for setting up the development environment. It includes information on how to write a client in Go.

Events and Event Subscription In Sawtooth

Thanks to the many open source contributors. The Hyperledger Sawtooth documentation describes in detail about event and event subscription. This article supplements the document. Sawtooth has SDK support in many languages, giving flexibility to the application developer. We will discuss the Go SDK usage in detail.

Sawtooth defines two core events sawtooth/block-commit and sawtooth/state-delta. Both occur when a block is committed. The former carries information about the block. The latter carries information about all state changes at the given address. In addition to the predefined events, Sawtooth allows application developers to define new events. A combination of sophisticated SDK and protobuf definitions abstract the complex network communications. The SDK exposes APIs to add new application-specific events. It also provides the mechanism for clients to subscribe to available events.

Figure 1: Representation of the Validator connection with Transaction Processor and Client via ZMQ.

Event: A savior for real world use cases

  1. Notification events such as state change and block commit help the design of interactive clients. It saves clients from having to poll the validator for the status of submitted batches.
  2. Events can be used  as a debugger for transaction processors. Transaction processors can register application-specific events allowing clients to debug the flow remotely..
  3. In addition to debugging, the asynchronous behavior allows for stateless application designs.
  4. Deferring the processing of events at a later point in time. Sawtooth Supply Chain is one such well known example, which makes use of this feature.
Caution beginners! Do not get carried away with overwhelming application specific use cases. Events are also being used for internal operation of Sawtooth. The horizon is open and our imagination is the only limit.

Adding Application-Specific Event In Go

Creating a custom event involves defining three parts, namely

  1. Define attributes: A list of key-value pairs, which later is used to filter during event subscriptions. There can be many values per attribute key.
import

("github.com/hyperledger/sawtooth-sdk-go/protobuf/events_pb2")

// Create a key-value map to define attribute

attributes := []events_pb2.Event_Attribute{

Key: "MyOwnKey", Value:"ValueIWantToPass",

})

2. Define data payload: Byte information for a specific event type.

payload := []byte{'f', 'u', 'n', '-', 'l', 'e', 'a', 'r', 'n', 'i', 'n', 'g'}

3. Define event type: Used as an identifier when subscribing for the events.

After defining these parameters, we can use the available API in the SDK to create an event and add to the context. Listing 1 shows and example of adding an event. Please note that any change to the context’s events will affect the way subscribed clients work.

import("github.com/hyperledger/sawtooth-sdk-go/processor")

func (self *OneOfMyTPHandler) Apply(

request *processor_pb2.TpProcessRequest,

context *processor.Context)

error {    // -- snip --

context.AddEvent(

"MyEventIdentifier",

attributes,

payload)

// -- snip --
}

Listing 1: Adding an event to the context.

Event-Subscription In Go

Subscribing to an event involves establishing a ZMQ connection with the validator. The event types specify which events are subscribed to. An optional filter attribute can be passed for each event when establishing the subscription. The Sawtooth Go SDK uses protobuf definitions for serializing messages exchanged between the client and the validator. The following four steps show the sample code snippets.

  1. Establish a ZMQ connection: To establish a ZMQ connection from a client as a DEALER. Detailed description can be found in ROUTER-DEALER mechanism in ZMQ. The Sawtooth SDK provides an API for establishing client connection with validator.
import (
"github.com/hyperledger/sawtooth-sdk-go/messaging"
"github.com/pebbe/zmq4"
)
zmq_context, err := zmq4.NewContext()
// Error creating a ZMQ context
if err != nil {
   return err
}

// Remember to replace <VALIDATOR-IP> with hostname
// or the IP where validator is listening on port 4004
zmq_connection, err := messaging.NewConnection(
   zmq_context,
   zmq4.DEALER,
   "tcp://<VALIDATOR-IP>:4004",
   //This specifies a server connection which needs
   //binding or client connection that needs to
   // establish a request to server
   false,
)
// ZMQ connection couldn't be established
if err != nil {
   return err
}
// Remember to close the connection when either done
// processing events processing or an error occursdefer zmq_connection.Close()
// -- snip ––

2. Construct EventFilter, EventSubscription and ClientEventsSubscribeRequest: The Event is said to be subscribed when both event_type and all the EventFilters in EventSubscription field match. EventFilters can be applied on attributes defined earlier. FilterType determines rules for  comparing the match string.

import("github.com/hyperledger/sawtooth-sdk-go/protobuf/events_pb2") 

// Define filters over attributes to be triggered when
// a string matches a particular filter type
filters := []*events_pb2.EventFilter{&events_pb2.EventFilter{   
   Key: "MyOwnKey", 
   MatchString: "MyUniqueString", 
   FilterType:  events_pb2.EventFilter_REGEX_ANY,
}} 
my_identifier_subscription := events_pb2.EventSubscription{   
   EventType: "MyEventIdentifier", 
   Filters: filters,
} 
// -- snip --
// Construct subscription request for the validator
request := client_event_pb2.ClientEventsSubscribeRequest{
   Subscriptions: []*events_pb2.EventSubscription{
     &my_identifier_subscription, 
     &my_another_identifier_subscription,
   },
}
// -- snip ––

3. Send request over ZMQ connection: The client’s event subscription request can be sent through an established ZMQ connection. Note that a correlation id returned by SendMsg() can be used to know if the validator as a ROUTER has response messages. Many events can be subscribed to at once.

import (
 "errors"
"github.com/golang/protobuf/proto"
 "github.com/hyperledger/sawtooth-sdk-go/protobuf/client_event_pb2"
 "github.com/hyperledger/sawtooth-sdk-go/protobuf/validator_pb2"
)
// Get serialized request using protobuf libraries
serialized_subscribe_request, err :=
    proto.Marshal(&request)
if err != nil {
  return err
}
 // Send the subscription request, get a correlation id
// from the SDK
corrId, err := zmq_connection.SendNewMsg(
    validator_pb2.Message_CLIENT_EVENTS_SUBSCRIBE_REQUEST,
    serialized_subscribe_request,
)
// Error requesting validator, optionally based on
// error type may apply retry mechanism here
if err != nil {   return err} // Wait for subscription status, wait for response of
// message with specific correlation id_, response, err :=zmq_connection.RecvMsgWithId(corrId)
if err != nil {
  return err
}
 // Deserialize received protobuf message as response
// for subscription requestevents_subscribe_response :=
    client_event_pb2.ClientEventsSubscribeResponse{}

err = proto.Unmarshal(
    response.Content,
    &events_subscribe_response)
if err != nil {
  return err
}
 // Client subscription is not successful, optional
// retries can be done later for subscription based on
// response cause
if events_subscribe_response.Status !=
  client_event_pb2.ClientEventsSubscribeResponse_OK {
  return errors.New("Client subscription failed")
}
 // Client event subscription is successful, remember to
// unsubscribe when either not required anymore or
// error occurs. Similar approach as followed for
// subscribing events can be used here.
defer func(){
    // Unsubscribe from events
    events_unsubscribe_request :=
      client_event_pb2.ClientEventsUnsubscribeRequest{}
    serialized_unsubscribe_request, err =
      proto.Marshal(&events_unsubscribe_request)
    if err != nil {
      return err
    }
    corrId, err = zmq_connection.SendNewMsg(

      validator_pb2.Message_CLIENT_EVENTS_UNSUBSCRIBE_REQUEST,
      Serialized_unsubscribe_request,
    )
    if err != nil {
      return err
    }
    // Wait for status
    _, unsubscribe_response, err :=
    zmq_connection.RecvMsgWithId(corrId)
    // Optional retries can be done depending on error
    // status
    if err != nil {
      return err
    }
    events_unsubscribe_response :=
      client_event_pb2.ClientEventsUnsubscribeResponse{}
    err =
      proto.Unmarshal(unsubscribe_response.Content, 
      &events_unsubscribe_response)
    if err != nil {
      return err
    }
    // Optional retries can be done depending on error
    // status
    if events_unsubscribe_response.Status !=
      client_event_pb2.ClientEventsUnsubscribeResponse_OK {
        return errors.New("Client couldn't unsubscribe successfully")
    }
}()
// -- snip ––

4. Event handling: The established ZMQ connection will send protobuf messages corresponding to the subscribed events.

import (
    "errors"
    "fmt"
    "github.com/golang/protobuf/proto"
    "github.com/hyperledger/sawtooth-sdk-go/protobuf/validator_pb2"
) 
// Listen for events in an infinite loop
fmt.Println("Listening to events.")
for {
  // Wait for a message on connection 
  _, message, err := zmq_connection.RecvMsg()
  if err != nil {
    return err
  }
  // Check if received is a client event message
  if message.MessageType !=
    validator_pb2.Message_CLIENT_EVENTS {
   return errors.New("Received a message not 
requested for")
  } 
  event_list := events_pb2.EventList{}
  err = proto.Unmarshal(message.Content, &event_list)
  if err != nil {
    return err
  }
  // Received following events from validator   
  for _, event := range event_list.Events {
    // handle event here
    fmt.Printf("Event received: %v\n", *event)
  }
}
// -- snip ––

Try it out!

References:

  1. Subscribing to Events, Using the Go SDK, from the Hyperledger Sawtooth website.
  2. Commits by arsulegai in sawtooth-cookiejar example.
  3. The Sawtooth Go SDK.

Chapter on Router-Dealer, ZMQ protocol.

Hyperledger Adds to Growing Line up of Groups Focused on Cross-Industry Development

By | Blog

In another step towards driving cross-industry cooperation in the development and use of blockchain technologies, Hyperledger has added two new groups to its growing list of Special Interest Groups (SIG). These two new groups, the Hyperledger Social Impact SIG and the Hyperledger Trade Finance SIG, join Healthcare and Public Sector focused SIGs.

SIGs gather community members from an industry segment to work on domain-specific problems and create an environment for open discussion, document co-creation and solution proposals. SIGs help specific vertical markets in their efforts to address problems specific to that particular community.

The Hyperledger Social Impact Special Interest Group

The Hyperledger Social Impact SIG is primarily focused on serving as a platform for exchanging ideas and exploration of ways to use blockchain technology in the context of social good. The Social Impact SIG will work with the other working groups and technical teams, especially in the areas of implementation.

The areas of focus include:

  • Identifying related use cases, current pilots and proofs of concept, production case studies, and the opportunity blockchain presents for social impact;
  • Sharing stories of successes, failures, lessons learned, opportunities and challenges;
  • Exploring cross-cutting concerns like security, privacy, and identity in global South contexts;
  • Identifying existing or needed common critical software components that would serve the particular needs of global social impact;
  • Working towards proposing solutions to the problems identified;
  • Identifying conferences or other opportunities to connect face to face, as well as submit talks or present as a group at an event.
  • Identifying the business and NGO community and building an inclusive platform for early adopters to contribute with their experiences.
  • Facilitating and raising awareness of the opportunities blockchain can bring to the field.

The group is currently working on developing an opportunity matrix to map out the landscape of activity in philanthropy and remittances, supply chain, governance and democracy, financial empowerment, and identity. We welcome your participating in this effort!

We also want to hear about your use cases in using Hyperledger in a social impact context, please let us know if you’d like to present at our next meeting by sending a message on our mailing list.

The Hyperledger Trade Finance Special Interest Group

The Hyperledger Trade Finance Special Interest Group is primarily focused on serving as a platform for exchanging ideas and exploration of ways to use blockchain technology in the context of trade finance.

Areas of focus for the group include:

  • Identifying related reference architectures (for example Trade Finance business and integration architecture, technical and infrastructure architecture), frameworks and models (OSI), use cases, current pilots and proofs of concept, and production case studies;
  • Sharing stories of successes, failures, opportunities and challenges;
  • Exploring and addressing cross-cutting architectural principles, options and decisions like performance and scalability, security, identity management and privacy, and identity in trade finance contexts;
  • Identifying existing or needed common critical software, middleware, and hardware components that would serve the particular needs of trade finance, including data transmission and processing from access point (device), through networks, and cloud computing deployment model options (private, public, hybrid)
  • Working towards proposing solutions to the problems identified;

How to get involved in Hyperledger Special Interest Groups

Participation in any SIG is open to everyone in the community. Each group has an open e-mail list, a Chat channel, and a wiki page. Live meetings are also held regularly via web teleconference. When needed, a task force can also be created within the SIG and have working sessions to discuss specific work items.
If you’re interested in joining a SIG, please subscribe to the SIG’s mailing list and start by saying “hi” to the community by sharing what your interests are on the topic. If you are looking for more information on the focus of an active SIG or proposed SIGs please visit wiki.hyperledger.org. We look forward to your participation and contributions!

Introduction to Sawtooth PBFT

By | Blog, Hyperledger Sawtooth

As of release 1.1, Hyperledger Sawtooth supports dynamic consensus through its consensus API and SDKs. These tools, which were covered in a previous blog post, are the building blocks that make it easy to implement different consensus algorithms as consensus engines for Sawtooth. We chose to implement the Raft algorithm as our first consensus engine, which we describe in another blog post. While our Raft implementation is an excellent proof of concept, it is not Byzantine-fault-tolerant, which makes it unsuitable for consortium-style networks with adversarial trust characteristics.

To fill this gap, we chose the Practical Byzantine Fault Tolerance (PBFT) consensus algorithm. We started work on the Sawtooth PBFT consensus engine in the summer of 2018 and continue to develop and improve on it as we work towards its first stable release. This blog post summarizes the PBFT algorithm and describes how it works in Sawtooth.

What Is PBFT?

PBFT dates back to a 1999 paper written by Miguel Castro and Barbara Liskov at MIT. Unlike other algorithms at the time, PBFT was the first Byzantine fault tolerant algorithm designed to work in practical, asynchronous environments. PBFT is thoughtfully defined, well established, and widely understood, which makes it an excellent choice for Hyperledger Sawtooth.

PBFT is similar to Raft in some general ways:

  • It is leader-based and non-forking (unlike lottery-style algorithms)
  • It does not support open-enrollment, but nodes can be added and removed by an administrator
  • It requires full peering (all nodes must be connected to all other nodes)

PBFT provides Byzantine fault tolerance, whereas Raft only supports crash fault tolerance. Byzantine fault tolerance means that liveness and safety are guaranteed even when some portion of the network is faulty or malicious. As long as a minimum percentage of nodes in the PBFT network are connected, working properly, and behaving honestly, the network will always make progress and will not allow any of the nodes to manipulate the network.

How Does PBFT Work?

The original PBFT paper has a detailed and rigorous explanation of the consensus algorithm. What follows is a summary of the algorithm’s key points in the context of Hyperledger Sawtooth. The original definition is broadly applicable to any kind of replicated system; by keeping this information blockchain-specific, we can more easily describe the functionality of the Sawtooth PBFT consensus engine.

Network Overview

A PBFT network consists of a series of nodes that are ordered from 0 to n-1, where n is the number of nodes in the network. As mentioned earlier, there is a maximum number of “bad” nodes that the PBFT network can tolerate. As long as this number of bad nodes—referred to as the constant f—is not exceeded, the network will work properly. For PBFT, the constant f is equal to one third of the nodes in the network. No more than a third of the network (rounded down) can be “out of order” or dishonest at any given time for the algorithm to work. The values of n and f are very important; you’ll see them later as we discuss how the algorithm operates.

Figure 1 — n and f in the PBFT algorithm

As the network progresses, the nodes move through a series of “views”. A view is a period of time that a given node is the primary (leader) of the network. In simple terms, each node takes turns being the primary in a never-ending cycle, starting with the first node. For a four-node network, node 0 is the primary at view 0, node 1 is the primary at view 1, and so on. When the network gets to view 4, it will “wrap back around” so that node 0 is the primary again.

In more technical terms, the primary (p) for each view is determined based on the view number (v) and the ordering of the nodes. The formula for determining the primary for any view on a given network is p = v mod n. For instance, on a four-node network at view 7, the formula p = 7 mod 4 means that node 3 will be the primary (7 mod 4 = 3).

In addition to moving through a series of views, the network moves through a series of “sequence numbers.” In the context of a Sawtooth blockchain, a sequence number is equivalent to a block number; thus, saying that a node is on sequence number 10 is the same as saying that the node is performing consensus on block 10 in the chain.

Each node maintains a few key pieces of information as part of its state:

  • The list of nodes that belong to the network
  • Its current view number
  • Its current sequence number (the block it is working on)
  • The phase of the algorithm it is currently in (see “Normal-Case Operation”)
  • A log of the blocks it has received
  • A log of all valid messages it has received from the other nodes

Normal-Case Operation

Figure 2 — Messages sent during normal operation of PBFT (Node 3 is faulty)

To commit a block and make progress, the nodes in a PBFT network go through three phases:

  1. Pre-preparing
  2. Preparing
  3. Committing

Figure 2 shows these phases for a simple four-node network. In this example, node 0 is the primary and node 3 is a faulty node (so it does not send any messages). Because there are four nodes in the network (n = 4), the value of f for the network is 4-13=1. This means the example network can tolerate only one faulty node.

Pre-preparing

To kick things off, the primary for the current view will create a block and publish it to the network; each of the nodes will receive this block and perform some preliminary verification to make sure that the block is valid.

After the primary has published a block to the network, it broadcasts a pre-prepare message to all of the nodes. Pre-prepare messages contain four key pieces of information: the ID of the block the primary just published, the block’s number, the primary’s view number, and the primary’s ID. When a node receives a pre-prepare message from the primary, it will validate the message and add the message to its internal log. Message validation includes verifying the digital signature of the message, checking that the message’s view number matches the node’s current view number, and ensuring that the message is from the primary for the current view.

The pre-prepare message serves as a way for the primary node to publicly endorse a given block and for the network to agree about which block to perform consensus on for this sequence number. To ensure that only one block is considered at a time, nodes do not allow more than one pre-prepare message at a given view and sequence number.

Preparing

Once a node has received a block and a pre-prepare message for the block, and both the block and message have been added to the node’s log, the node will move on to the preparing phase. In the preparing phase, the node will broadcast a prepare message to the rest of the network (including itself). Prepare messages, like pre-prepare messages, contain the ID and number of the block they are for, as well as the node’s view number and ID.

In order to move onto the next phase, the node must wait until it has received 2f + 1 prepare messages that have the same block ID, block number, and view number, and are from different nodes. By waiting for 2f + 1 matching prepare messages, the node can be sure that all properly functioning nodes (those that are non-faulty and non-malicious) are in agreement at this stage. Once the node has accepted the required 2f + 1 matching prepare messages and added them to its log, it is ready to move onto the committing phase.

Committing

When a node enters the committing phase, it broadcasts a commit message to the whole network (including itself). Like the other message types, commit messages contain the ID and number of the block they are for, along with the node’s view number and ID. As with the preparing phase, a node cannot complete the committing phase until it has received 2f + 1 matching commit messages from different nodes. Again, this guarantees that all non-faulty nodes in the network have agreed to commit this block, which means that the node can safely commit the block knowing that it will not need to be reverted. With the required 2f + 1 commit messages accepted and in its log, the node can safely commit the block.

Once the primary node has finished the committing phase and has committed the block, it will start the whole process over again by creating a block, publishing it, and broadcasting a pre-prepare message for it.

View Changing

In order to be Byzantine fault tolerant, a consensus algorithm must prevent nodes from improperly altering the network (to guarantee safety) or indefinitely halting progress (to ensure liveness). PBFT guarantees safety by requiring all non-faulty nodes to agree in order to move beyond the preparing and committing phases. To guarantee liveness, though, there must be a mechanism to determine if the leader is behaving improperly (such as producing invalid messages or simply not doing anything). PBFT provides the liveness guarantee with view changes.

Figure 3 — Messages sent for a view change in PBFT (Node 0 is the faulty primary, Node 1 is the new primary)

When a node has determined that the primary of view v is faulty (perhaps because the primary sent an invalid message or did not produce a valid block in time), it will broadcast a view change message for view v + 1 to the network. If the primary is indeed faulty, all non-faulty nodes will broadcast view change messages. When the primary for the new view (v + 1) receives 2f + 1 view change messages from different nodes, it will broadcast a new view message for view v + 1 to all the nodes. When the other nodes receive the new view message, they will switch to the new view, and the new primary will start publishing blocks and sending pre-prepare messages.

View changes guarantee that the network can move on to a new primary if the current one is faulty. This PBFT feature allows the network to continue to make progress and not be stalled by a bad primary node.

Want to Learn More?

This blog post only scratches the surface of the PBFT consensus algorithm. Stay tuned to the Hyperledger blog for more information on PBFT, including a future post about our extensions and additional features for Sawtooth PBFT.

In the meantime, learn more about PBFT in the original PBFT paper, read the Sawtooth PBFT RFC, and check out the Sawtooth PBFT source code on GitHub.

About the Author

Logan Seeley is a Software Engineer at Bitwise IO. He has been involved in a variety of Hyperledger Sawtooth projects, including the development of the consensus API, Sawtooth Raft, and Sawtooth PBFT.

Assembling the Future of Smart Contracts with Sawtooth Sabre

By | Blog, Hyperledger Sawtooth

Is WebAssembly the future of smart contracts? We think so. In this post, we will talk about Sawtooth Sabre, a WebAssembly smart contract engine for Hyperledger Sawtooth.

We first learned about WebAssembly a couple of years ago at Midwest JS, a JavaScript conference in Minneapolis. The lecture focused on using WebAssembly inside a web browser, which had nothing to do with blockchain or distributed ledgers. Nonetheless, as we left the conference, we were excitedly discussing the possibilities for the future of smart contracts. WebAssembly is a stack-based virtual machine, newly implemented in major browsers, that provides a sandboxed approach to fast code execution. While that sounds like a perfect way to run smart contracts, what really excited us was the potential for WebAssembly to grow a large ecosystem of libraries and tools because of its association with the browser community.

A smart contract is software that encapsulates the business logic for modifying a database by processing a transaction. In Hyperledger Sawtooth, this database is called “global state”. A smart contract engine is software that can execute a smart contract. By developing Sawtooth Sabre, we hope to leverage the WebAssembly ecosystem for the benefit of application developers writing the business logic for distributed ledger systems. We expect an ever-growing list of WebAssembly programming languages and development environments.

Unblocking Contract Deployment

The primary mechanism for smart contract development in Hyperledger Sawtooth is a transaction processor, which takes a transaction as input and updates global state. Sound like a smart contract? It is! If you implement business logic in the transaction processor, then you are creating a smart contract. If you instead implement support for smart contracts with a virtual machine (or interpreter) like WebAssembly, then you have created a smart contract engine.

If we can implement smart contracts as transaction processors, why bother with a WebAssembly model like Sabre? Well, it is really about deployment strategy. There are three deployment models for smart contracts:

  • Off-chain push: Smart contracts are deployed by pushing them to all nodes from a central authority on the network.
  • Off-chain pull: Smart contracts are deployed by network administrators pulling the code from a centralized location. Network administrators operate independently.
  • On-chain: Smart contracts are submitted to the network and inserted into state. Later, as transactions are submitted, the smart contracts are read from state and executed (generally in a sandboxed environment).

We won’t discuss off-chain push, other than to note that this strategy—having a centralized authority push code to everyone in the network—isn’t consistent with distributed ledgers and blockchain’s promise of distributing trust.

Off-chain pull is an opt-in strategy for updating software, and is widely used for Linux distribution updates. We use this model to distribute Sawtooth, including the transaction processors. By adding the Sawtooth apt repository on an Ubuntu system, you pull the software and install it via the apt-get command. Each software repository is centrally managed, though it is possible to have multiple software repositories configured and managed independently. This model has a practical problem—it requires administrators across organizations to coordinate software updates—which makes business logic updates more complicated than we would like.

On-chain smart contracts are installed on the blockchain with a transaction that stores the contract into an address in global state. The smart contract can later be executed with another transaction. The execution of the smart contract starts by loading it from global state and continues by executing the smart contract in a virtual machine (or interpreter). On-chain smart contracts have a big advantage over off-chain contracts: because the blockchain is immutable and the smart contract itself is now on the chain, we can guarantee that the same smart contract code was used to create the original block and during replay. Specifically, the transaction will always be executed using the same global state, including the stored smart contract. Because contracts are deployed by submitting transactions onto the network, we can define the process that controls the smart contract creation and deletion with other smart contracts! Yes, this is a little meta, but isn’t it great?

The on-chain approach seems superior, so why did we implement Hyperledger Sawtooth transaction processors with the off-chain model? Because our long-term vision—and a main focus for Sawtooth—has been smart contract engines that run on-chain smart contracts. Smart contract engines are more suitable for off-chain distribution, because they do not contain business logic, and are likely to be upgraded at the same time as the rest of the software.

Our initial transaction processor design reflected our goal for several types of smart contract engines. We later implemented one of them: Sawtooth Seth, a smart contract engine that runs Ethereum Virtual Machine (EVM) smart contracts. For us, Seth was a validation that our transaction processor design was flexible enough to implement radically different approaches for smart contracts. Like Ethereum, Seth uses on-chain smart contracts, so Seth is great if you want Ethereum characteristics and compatibility with tools such as Truffle. However, Seth is limited by Ethereum’s design and ecosystem, and does not expose all the features in our blockchain platform. We knew that we needed an additional approach for smart contracts in Hyperledger Sawtooth.

Crafting a Compatible Path Forward

Sawtooth Sabre, our WebAssembly smart contract engine, is our solution for native, on-chain smart contracts.  

The programming model for Sabre smart contracts is the same as that for transaction processors. A transaction processor has full control of data representation, both in global state and in transaction payloads (within certain determinism requirements). Hyperledger Sawtooth uses a global state Merkle-Radix tree, and the transaction processors handle addressing within the tree. A transaction processor can use different approaches for addressing, ranging from calculating an address with a simple field hash to organizing data within the tree in a complex way (to optimize for parallel execution, for example). Multiple transaction processors can access the same global state if they agree on the conventions used in that portion of state.

Sawtooth Sabre smart contracts use this same method for data storage, which means they can access global state in the same way that transaction processors do. In fact, smart contracts and transaction processors can comfortably coexist on the same blockchain.

The other major feature is SDK compatibility. The Sawtooth Sabre SDK API is compatible with the Hyperledger Sawtooth transaction processor API, which means that smart contracts written in Rust can switch between the Sawtooth SDK and the Sabre SDK with a simple compile-time flag. (Currently, Rust is the only supported Sabre SDK.) The details of running within a WebAssembly interpreter are hidden from the smart contract author. Because Sabre smart contracts use the same API as transaction processors, porting a transaction processor to Sabre is relatively easy—just change a few import statements to refer to the Sabre SDK instead of the Hyperledger Sawtooth SDK.

Now the choice between off-chain and on-chain smart contracts is a compile-time option. We use this approach regularly, because we can separate our deployment decisions from the decisions for smart contract development. Most of the transaction-processor-based smart contracts included in Hyperledger Sawtooth are now compatible with Sawtooth Sabre.

A Stately Approach to Permissioning

Hyperledger Sawtooth provides several ways to control which transaction processors can participate on a network. As explained above, transaction processors are deployed with the off-chain pull method. This method lets administrators verify the the transaction processors before adding them to the network. Note that Hyperledger Sawtooth requires the same set of transaction processors for every node in the network, which prevents a single node from adding a malicious transaction processor. Additional controls can limit the accepted transactions (by setting the allowed transaction types) and specify each transaction processor’s read and write access to global state (by restricting namespaces).

These permissions, however, are not granular enough for Sawtooth Sabre, which is itself a transaction processor. Sabre is therefore subject to the same restrictions, which would then apply to all smart contracts. Using the same permission control has several problems:

  • Sabre smart contracts are transaction-based, which means that a smart contract is created by submitting a transaction. This removes the chance to review a contract before it is deployed.
  • Sabre transactions must be accepted by the network to run smart contracts, but we cannot limit which smart contracts these transactions are for, because this information is not available to validator.
  • Sabre must be allowed to access the same areas of global state that the smart contracts can access.

An “uncontrolled” version of Sabre would make it too easy to deploy smart contracts that are  not inherently restricted to a to the permissions that the publisher of the smart contract selects.

Our solution in Sawtooth Sabre is to assign owners for both contracts and namespaces (a subset of global state). A contract has a set of owners and a list of namespaces that it expects to read from and write to. Each namespace also has an owner. The namespace owner can choose which contracts have read and write access to that owner’s area of state. If a contract does not have the namespace permissions it needs, a transaction run against the smart contract will fail. So, while the namespace owner and contract owner are not necessarily the same, there is an implied degree of trust and coordination between them.

Also, contracts are versioned. Only the owners of a contract are able to submit new versions to Sabre, which removes the chance that a malicious smart contract change could be accepted.

A Final Note About WebAssembly

On-chain WebAssembly isn’t limited to just smart contracts. For example, in Hyperledger Grid, we are using on-chain WebAssembly to execute smart permissions for organization-specific permissioning. Another example is smart consensus, which allows consensus algorithm updates to be submitted as a transaction. There are several more possibilities for on-chain WebAssembly as well.

In short, we think WebAssembly is awesome! Sawtooth Sabre combines WebAssembly with existing Hyperledger Sawtooth transaction processors to provide flexible smart contracts with all the benefits of both a normal transaction processor and on-chain smart-contract execution. Sabre also takes advantage of WebAssembly’s ability to maintain dual-target smart contracts, where the contract can be run as either a native transaction processor or a Sabre contract. And the permission control in Sawtooth Sabre allows fine-grained control over both contract changes and access to global state.

We are incredibly grateful for Cargill’s sponsorship of Sawtooth Sabre and Hyperledger Grid (a supply chain platform built with Sawtooth Sabre). We would also like to thank the following people who help make our blog posts a success: Anne Chenette, Mark Ford, David Huseby, and Jessica Rampen.

About the Authors

Andi Gunderson is a Software Engineer at Bitwise IO and maintainer on Hyperledger Sawtooth and Sawtooth Sabre.

Shawn Amundson is Chief Technology Officer at Bitwise IO, a Hyperledger technical ambassador, and a maintainer and architect on Hyperledger Sawtooth and Hyperledger Grid.