A Problem (High Load) and a Constraint (Uptime)

In my time running Fabric network, I once experienced a high transaction load on my existing peers. One of the ways to address this would have been to vertically scale my peers (which I had done earlier). Vertical scaling of peers has its own problem: it would require downtime for the peer which is getting scaled and, again, it will require downtime when we want to scale back down. The alternative was to increase the number of peers in the network. Now the real problem was how could I achieve this without affecting my running fabric network.

Initially, I had generated crypto material using the cryptogen tool of Hyperledger® Fabric. I thought there would be an option to generate new crypto material with the same tool. I came across a command of said cryptogen tool which is used for generating new crypto material.

./cryptogen extend — input=”crypto-config.yaml”

The above command could help me to generate crypto key material for a new entity. I just had to change the peer template in my crypto-config.yaml file to make it work. But I got intrigued by how this command was able to generate certificates without affecting my existing crypto. There were a few questions that were popping into my mind.

Were all the certificates being regenerated? Or only the certificates for the new entity? So to find the answer to my questions I started digging up the internal code of the cryptogen tool of Hyperledger Fabric.

Diving Into Cryptogen

Every participant who wants to interact with the network needs an identity. The CA provides the means for each entity to have a verifiable digital identity. In Hyperledger Fabric we have a built-in CA component that can be used in the blockchain network for generating identity.

The role of CA in generating the key material is very significant as CA is the default Certificate Authority (CA) for Hyperledger Fabric. Unless replaced, it is responsible for the registration of identities, issuance of Enrollment Certificates (ECerts) and certificate renewal and revocation. So the first time the key material was generated the following things happened:

1. The SignCertificate function of CA got invoked which created a signed certificate based on the template of a crypto-config.yaml file and saved it in the base directory.
2. The base directory by default was set to crypto-config. However, if needed it could be changed.

inputDir = ext.Flag(“input”, “The input directory in which existing network place”).Default(“crypto-config”).String()

The cryptogen tool of Hyperledger Fabric uses the CA and MSP (Membership Service Provider) package to generate the certificates of entities. Initially, the extend function of the cryptogen tool was invoked which loaded the modified crypto-config.yaml file and called the renderOrgSpec function. That function processed all the template nodes from the count field and, following this, it called the function extendPeerOrg.

func extend() {
 config, err := getConfig()
 if err != nil {
 fmt.Printf(“Error reading config: %s”, err)
 os.Exit(-1)
 } for _, orgSpec := range config.PeerOrgs {
 err = renderOrgSpec(&orgSpec, “peer”)
 if err != nil {
 fmt.Printf(“Error processing peer configuration: %s”, err)
 os.Exit(-1)
 }
 extendPeerOrg(orgSpec)
 }}

After figuring this out, still, my question was unanswered as to whether the old certificates would also be regenerated or not? So I moved to the next function which was extendPeerOrg.

Now inside this function, I found out all the things were similar to that of the generatePeerOrg function. The only differences were that when generating, NewCA was invoked and while executing the cryptogen executable binary loads the instance of the already generated CA (created during generation of certificates).

signCA := getCA(caDir, orgSpec, orgSpec.CA.CommonName)
tlsCA := getCA(tlscaDir, orgSpec, “tls”+orgSpec.CA.CommonName

My Question Answered: generateNodes

Then I reached the most fascinating part of the code because it gave the answer to the question I was looking for. The extendPeerOrg function internally calls the function generateNodes. This function decides whether the crypto material would be regenerated or not.

After looking into the code of this function, I found out that on the basis of the path it checks if the certificate material has been previously generated or not. If the path of a new peer is not present in the base directory it calls the GenerateLocalMSP function of MSP.

func generateNodes(baseDir string, nodes []NodeSpec, signCA *ca.CA, tlsCA *ca.CA, nodeType int, nodeOUs bool) { for _, node := range nodes {
 nodeDir := filepath.Join(baseDir, node.CommonName)
 if _, err := os.Stat(nodeDir); os.IsNotExist(err) {
 err := msp.GenerateLocalMSP(nodeDir, node.CommonName, node.SANS, signCA, tlsCA, nodeType, nodeOUs)
 if err != nil {
 fmt.Printf(“Error generating local MSP for %s:n%vn”, node, err)
 os.Exit(1)
   }
  }
 }
}

Now coming to what the GenerateLocalMSP function does. It does the following things:

1. It creates a folder structure for the new entity.
2. Gets the Keystore path and eventually generates the private key of the new peer by using the function GeneratePrivateKey.
3. On the basis of the private key, the public key is extracted by using the function GetECPublicKey.
4. After that, it generates a X509 certificate using signing CA.
5. And lastly, the artifacts are moved to their respective MSP folders.

keystore := filepath.Join(mspDir, “keystore”) // generate private key
 priv, _, err := csp.GeneratePrivateKey(keystore)
 if err != nil {
 return err
 } // get public key
 ecPubKey, err := csp.GetECPublicKey(priv)
 if err != nil {
 return err
 }cert, err := signCA.SignCertificate(filepath.Join(mspDir, “signcerts”),
 name, ous, nil, ecPubKey, x509.KeyUsageDigitalSignature, []x509.ExtKeyUsage{})
 if err != nil {
 return err
 }

After all the above steps, admin certs are copied to the org’s peer’s MSP admin certs folder. So after digging up the code, I figured out that only new certificates are generated and the old ones are kept untouched. To support my findings I am going to share the time log of the extended certificates generation:

drwxr-xr-x 4 deepaks deepaks 4096 Jul 22 10:07 peer0.org1.example.comdrwxr-xr-x 4 deepaks deepaks 4096 Jul 22 10:20 peer1.org1.example.com

It’s quite evident from the logs that once the key material has been generated it won’t tamper with existing certificates when generating new crypto material. The above screenshot depicts when peer1 certificates were generated peer0 crypto material was not altered.

Hence it could be summarised that the cryptogen extend command allows one to extend an existing network, generating all the additional key material needed by the newly added entities without affecting the existing crypto material.

Hyperledger is a registered trademark of the Linux Foundation which hosts the Hyperledger Fabric project. Fabric code excerpts are copyright IBM Corp. “DL” and “DLT Labs” are trademarks of DLT Global, Inc.

About the Author

Deepak Singh, DLT Labs™

Deepak is a blockchain developer at DLT Labs. He likes to explore software codes and algorithms at an elementary level.