Skip to content

Hyperledger Fabric

Prerequisites

  • The OS where the integration via PKCS#11 will be carried out must be compatible with the HSM libraries.
  • Since Fabric defaults to using Alpine Linux, you need to change the dockerfile to use a compatible OS.

Versions

  • This procedure was done with the following version of Hyperledger Fabric, on WSL 2 of Windows 11

    Component Version
    fabric-ca (server/client) 1.5.5
    fabric-tool 2.4.7
    fabric-peer 2.4.7
    fabric-orderer 2.4.7
    S.O. base Ubuntu 20.04 over WSL2 on Windows 11
    S.O. containers Ubuntu 22.04 (jammy)
    Go 1.20
    Client HSM 4.7.35
    HSM firmware 5.0.28.0-243-g5a9cb01

Initial Setup without PKCS#11

  1. Follow the procedure described in Fabric test network. In the Getting started guide only configure the following topics:

    1. Prerequisite software
    2. Install Fabric and Fabric samples

  2. Start a test network from scratch every time you run it. If the environment is not clean, errors will occur at various points in the test network's ascent.

  3. Monitor the logs and outputs of the test network. The logs of the HSM can be seen here, when linked with the option of stdout:

    HSM_LOG_DIR=stdout

Integration with HSM via PKCS#11

  • In the following integration examples, independent HSM users/partitions are used for each component/organization.

    Component User name Observations Where it is used
    Integration test fabric-ca-client caclient Used in the integration test of fabric-ca-client. host
    Fabric-ca ca1 CA of organization 1 container
    Fabric-ca ca2 CA of organization 2 container
    Fabric-ca caorderer Orderer's CA container
    Peer 1 peer1 Peer from organization 1 container
    Peer 2 peer2 Peer from organization 2 container

Initial setup

  1. You must follow the steps in the Initial Setup without PKCS#11 section before proceeding.

    Running on an Ubuntu 20.04 on Windows 11 WSL2.

  2. Set the environment variable GOPATH with the working directory, in this case we will use ~/go. You can define in the ~/.profile for ease of use.

    export GOPATH=$HOME/go

Recompile the fabric and fabric-ca tools with PKCS#11 support

  1. Create Hyperledger folders and clone Hyperledger repository fabric-ca .

    cd $GOPATH
mkdir -p src/github.com/hyperledger/
cd src/github.com/hyperledger
git clone -b v1.5.5 https://github.com/hyperledger/fabric-ca
cd fabric-ca/

  2. Compile the fabric-ca e fabric-ca-client with PKCS#11 support.

    make fabric-ca-server GO_TAGS=pkcs11
make fabric-ca-client GO_TAGS=pkcs11

    The binaries are generated in the fabric-ca/bin.

  3. Copy the binaries fabric-ca e fabric-ca-client folder fabric-samples/bin.

  4. Clone manufacture .

    cd $GOPATH/src/github.com/hyperledger
git clone -b v2.4.7 https://github.com/hyperledger/fabric.git
cd fabric

  5. Compile the tools with PKCS#11 support.

    make tools GO_TAGS=pkcs11

    The binaries are generated in the build/bin.

  6. Copy the generated binaries to the fabric-samples/bin.

Integration with fabric-ca-client

  • There is a fabric-ca-client test in the folder ./fabric-samples/hardware-security-module.
  • The file, ./fabric-samples/hardware-security-module/README.md can be used as a guide, but needs to be adapted.

This test can be run from the machine itself, without the need for a container. In this case it was run on an Ubuntu 20.04 on Windows 11 WSL2.

  1. For the test we need to change the hard-codes.

    ./fabric-samples/hardware-security-module/ca-client-config/fabric-ca-client-config-template.yaml

    In the bccspchange the following fields: - LabelLabel of the P11 token. In the case of Dinamo it is "Dinamo HSM". - PinPassword used to log in to P11. Password of the user configured on P11 Dinamo.

    Do not change the field Library. The placeholder is overwritten by the execution script.

    #############################################################################
# BCCSP (BlockChain Crypto Service Provider) section allows to select which
# crypto implementation library to use
#############################################################################
bccsp:
    default: PKCS11
    PKCS11:
        Library: REPLACE_ME_HSMLIB
        Pin: 12345678
        Label: "Dinamo HSM"
        hash: SHA2
        security: 256

  2. Change the IP and user of HSM in the script below. This script will generate the fabric configuration file that will be used to make the fabric-ca-client to generate a new user.

    #!/bin/bash

#
#   Test PKCS#11 CA client
#

# Add the path to the fabric-ca-client with pkcs11 support
export PATH=./fabric-samples/bin:$PATH
export PKCS11_LIB=/usr/lib/libtacndp11.so # path to the installed pkcs11

# Dinamo pkcs11 configuration
export DFENCE_PKCS11_IP=127.0.0.1     # HSM IP Address
export DFENCE_PKCS11_USER=caclient  # HSM User
export DFENCE_PKCS11_AUTO_RECONNECT=1
#export HSM_LOG_DIR=./
#export HSM_LOG_LEVEL=3

./fabric-samples/hardware-security-module/scripts/generate-hsm-user.sh HSMUser

  3. To run the test, make the changes and run the script above.

    Check that the user configured in test-p11-ca-client.sh exists in HSM.

    ./start-fabric.sh
./test-p11-ca-client.sh
./stop-fabric.sh

    Sample output

    ./test-p11-ca-client.sh
2022/11/24 19:06:08 [INFO] TLS Enabled
2022/11/24 19:06:08 [INFO] generating key: &{A:ecdsa S:256}
2022-11-24 19:06:08.860 -03 [bccsp_p11] generateECKey -> INFO 001 Generated new P11 key, SKI 80feb3f43df12ef5dae75a3dd3502092d702a8dd80e8fcb92335c41f54e372d0
2022/11/24 19:06:08 [INFO] encoded CSR
.
.
.

Integration with fabric-ca-server

  • Versions used:

    Component Version
    S.O. Ubuntu 22.04 (jammy)
    Go 1.20
    Client HSM 3.7.35-1

  • For Hyperledger to work with HSM we need to recompile the container with the following settings:

    • Use a Linux distro that supports the HSM client, in this example we'll use Ubuntu;
    • Enable support for PKCS#11.

Generate image with PKCS#11 support

  1. Change the Dockerfile to compile run with Ubuntu and install the HSM. The file Dockerfile is located in the $GOPATH/src/github.com/hyperledger/fabric-ca/images/fabric-ca. After the changes, the file should look like this.

    #
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
ARG GO_VER

FROM ubuntu:jammy
ARG GO_LDFLAGS
ARG GO_TAGS

# Latest debian url of DINAMO client
ENV HSM_CLIENT_URL "https://downloads.dinamonetworks.io/bin/client/linux/x64/4.7.35/dinamo-4.7.35-1.x64.deb"
ENV GO_URL "https://storage.googleapis.com/golang/go1.19.3.linux-amd64.tar.gz"
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update -y && \
  apt-get install -y \
  apt-utils \
  gcc \
  binutils-gold \
  git \
  curl;

RUN curl -L $GO_URL | tar -v -C /usr/local -xz

ENV GOPATH /go
RUN mkdir -p "$GOPATH/src" "$GOPATH/bin"
ENV PATH $GOPATH/bin:/usr/local/go/bin:$PATH

WORKDIR /tmp
RUN curl -L $HSM_CLIENT_URL -o ./dinamo.deb
RUN apt-get install -y ./dinamo.deb
RUN rm ./dinamo.deb

ADD . /build/fabric-ca
WORKDIR /build/fabric-ca
RUN go install -tags "${GO_TAGS}" -ldflags "${GO_LDFLAGS}" \
  github.com/hyperledger/fabric-ca/cmd/fabric-ca-server \
  && go install -tags "${GO_TAGS}" -ldflags "${GO_LDFLAGS}" \
  github.com/hyperledger/fabric-ca/cmd/fabric-ca-client

RUN apt-get install -y \
  tzdata;
ENV FABRIC_CA_HOME /etc/hyperledger/fabric-ca-server
EXPOSE 7054
CMD fabric-ca-server start -b admin:adminpw

  2. Compile the fabric-ca.

    cd $GOPATH/src/github.com/hyperledger/fabric-ca
export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0
make clean-all
make docker GO_TAGS=pkcs11

Setting up the test network

  1. Changing configuration files fabric-ca-server-config.yaml of each organization to use PKCS#11 of the HSM. These files are located in the paths below:

    • fabric-samples/test-network/organizations/fabric-ca/org1/fabric-ca-server-config.yaml.
    • fabric-samples/test-network/organizations/fabric-ca/org2/fabric-ca-server-config.yaml.


    These settings can also be defined by environment variables as shown in the Fabric HSM documentation.

  2. Change the section bccsp for the following values.

    bccsp:
    default: PKCS11
    PKCS11:
        Library: /usr/lib/libtacndp11.so
        Pin: 12345678 # Senha do usuário do HSM
        Label: "Dinamo HSM"
        hash: SHA2
        security: 256
        Immutable: false

  3. Configure the environment variables of the HSM in the compose file of the compose-ca.yaml. This file can be found at fabric-samples/test-network/compose/compose-ca.yaml.

    In this case, we are configuring the CA of organization 1In this case, you can add the environment variables in their respective section, as below.

    Use a different HSM user for each of the CAs.

    services:

  ca_org1:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal # IP do HSM
      - DFENCE_PKCS11_USER=ca1 # Usuário do HSM
      - DFENCE_PKCS11_AUTO_RECONNECT=1

  ...

  ca_org2:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal # IP do HSM
      - DFENCE_PKCS11_USER=ca2 # Usuário do HSM
      - DFENCE_PKCS11_AUTO_RECONNECT=1

Go to the test network and run a test

  1. Run the test script.

  2. Example output from the basic transfer test

    ./test-basic-transfer.sh
2022-11-14 18:32:53.487 -03 0001 INFO [chaincodeCmd] chaincodeInvokeOrQuery -> Chaincode invoke successful. result: status:200

2022-11-14 18:32:56.826 -03 0001 INFO [chaincodeCmd] chaincodeInvokeOrQuery -> Chaincode invoke successful. result: status:200 payload:"Michel"
{"AppraisedValue":800,"Color":"white","ID":"asset6","Owner":"Michel","Size":15}

  3. Check the remote log of HSM that the user configured (e.g: ca1) was used correctly.

    2022/11/14 21:28:05 00000062 000000B7 995E9769 ca1/1d571851df3d6f v-attr update|172.17.0.1 172.17.0.2:4433 ca1
2022/11/14 21:28:05 00000062 000000B8 995E9769 MANAGE_P11 (03:ca1/1d571851df3d6f) [00000000], c: 23|172.17.0.1 172.17.0.2:4433 ca1
2022/11/14 21:28:05 00000062 000000B9 995E9769 172.17.0.1#23 probe|172.17.0.1 172.17.0.2:4433 ca1
2022/11/14 21:28:05 00000062 000000BA 995E9769 ecc 1d571851df3d6f!0nMFd8VvAX80FxUQ7L+1Sh6eRZ6SSEJSJLuq6BasfL8=, c: 23|172.17.0.1 172.17.0.2:4433 ca1

  4. Check that the keys have been created on the test user.

    Dinamo - Remote Management Console v. 4.7.34.0 2018 (c) Dinamo Networks

HSM 127.0.0.1 e - Engine 5.0.28.0-243-g5a9cb01 (DXP) - TCA0000000  - ID ca1

Keys/Objects - List


Name                                      Type                 T E Label
================================================================================
1d571851df3d6f                            prime256v1           n n e60bde44a80e532e20e5ac0952a093a4659e2736bca5b729a5708e6fbdfb8aee
a8d9d38535ec9a                            prime256v1 pub       n y ea1f764edd0e8187409d61dfd78313d706b1b146fecb8a99b5f69309aced5de8
d6f690620dee1a                            prime256v1           n n ea1f764edd0e8187409d61dfd78313d706b1b146fecb8a99b5f69309aced5de8
d7ac2114025001                            prime256v1 pub       n y e60bde44a80e532e20e5ac0952a093a4659e2736bca5b729a5708e6fbdfb8aee

Total of objects: 4

Integration with orderer, peer and tools (container)

  • The tools container is a container that has the basic tools for interacting with the HSM compiled and executed in the container itself. Some parts of the test network scripts intersperse calls to locally compiled tools (on the local machine) and executions of these tools from the container.

Generate images with PKCS#11 support

The Dockerfiles below are in path $GOPATH/src/github.com/hyperledger/fabric/images/.

  1. Edit the files Dockerfile of peer and orderer to be compiled with Ubuntu and with PKCS#11 support.

    # Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0

ARG GO_VER
ARG ALPINE_VER

FROM ubuntu:jammy as peer-base

# Latest debian url of DINAMO client
ENV HSM_CLIENT_URL "https://downloads.dinamonetworks.io/bin/client/linux/x64/4.7.35/dinamo-4.7.35-1.x64.deb"
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update -y && \
  apt-get install -y tzdata

# Bugfix https://github.com/hyperledger/fabric/issues/3779
RUN if [[ ! -e /etc/nsswitch.conf ]] ; then echo 'hosts: files dns' > /etc/nsswitch.conf ; fi

RUN apt-get install -y \
  bash \
  binutils-gold \
  gcc \
  git \
  make \
  musl-dev \
  curl

RUN curl -L $HSM_CLIENT_URL -o /tmp/dinamo.deb
RUN apt-get install -y /tmp/dinamo.deb
RUN rm /tmp/dinamo.deb

FROM golang:latest as golang
ADD . $GOPATH/src/github.com/hyperledger/fabric
WORKDIR $GOPATH/src/github.com/hyperledger/fabric

FROM golang as peer
ARG GO_TAGS
#Bugfix https://github.com/hyperledger/fabric/issues/3645
RUN make peer GO_TAGS=${GO_TAGS}
RUN make ccaasbuilder
RUN \
    export MARCH=$(go env GOOS)-$(go env GOARCH) && \
    mkdir -p release/linux-platform && \
    cp -r release/${MARCH}/builders release/linux-platform/.

FROM peer-base
ENV FABRIC_CFG_PATH /etc/hyperledger/fabric
VOLUME /etc/hyperledger/fabric
VOLUME /var/hyperledger
COPY --from=peer /go/src/github.com/hyperledger/fabric/build/bin /usr/local/bin
COPY --from=peer /go/src/github.com/hyperledger/fabric/sampleconfig/msp ${FABRIC_CFG_PATH}/msp
COPY --from=peer /go/src/github.com/hyperledger/fabric/sampleconfig/core.yaml ${FABRIC_CFG_PATH}/core.yaml
COPY --from=peer /go/src/github.com/hyperledger/fabric/release/linux-platform/builders/ccaas/bin/ /opt/hyperledger/ccaas_builder/bin/
EXPOSE 7051
CMD ["peer","node","start"]

    # Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0

ARG GO_VER
ARG ALPINE_VER
FROM ubuntu:jammy as base

# Latest debian url of DINAMO client
ENV HSM_CLIENT_URL "https://downloads.dinamonetworks.io/bin/client/linux/x64/4.7.35/dinamo-4.7.35-1.x64.deb"
ENV GO_URL "https://storage.googleapis.com/golang/go1.19.3.linux-amd64.tar.gz"
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update -y && \
  apt-get install -y tzdata

# Bugfix https://github.com/hyperledger/fabric/issues/3779
RUN if [[ ! -e /etc/nsswitch.conf ]] ; then echo 'hosts: files dns' > /etc/nsswitch.conf ; fi

RUN apt-get install -y \
  bash \
  binutils-gold \
  gcc \
  git \
  make \
  musl-dev \
  curl

RUN curl -L $HSM_CLIENT_URL -o /tmp/dinamo.deb
RUN apt-get install -y /tmp/dinamo.deb
RUN rm /tmp/dinamo.deb

FROM golang:latest as golang
ADD . $GOPATH/src/github.com/hyperledger/fabric
WORKDIR $GOPATH/src/github.com/hyperledger/fabric

FROM golang as orderer
ARG GO_TAGS
#Bugfix https://github.com/hyperledger/fabric/issues/3645
RUN make orderer GO_TAGS=${GO_TAGS}

FROM base
ENV FABRIC_CFG_PATH /etc/hyperledger/fabric
VOLUME /etc/hyperledger/fabric
VOLUME /var/hyperledger
COPY --from=orderer /go/src/github.com/hyperledger/fabric/build/bin /usr/local/bin
COPY --from=orderer /go/src/github.com/hyperledger/fabric/sampleconfig/msp ${FABRIC_CFG_PATH}/msp
COPY --from=orderer /go/src/github.com/hyperledger/fabric/sampleconfig/orderer.yaml ${FABRIC_CFG_PATH}
COPY --from=orderer /go/src/github.com/hyperledger/fabric/sampleconfig/configtx.yaml ${FABRIC_CFG_PATH}
EXPOSE 7050
CMD ["orderer"]

  2. Configure the Dockerfile from tools to use the HSM client.

     # Copyright IBM Corp. All Rights Reserved.
 #
 # SPDX-License-Identifier: Apache-2.0

 ARG GO_VER
 ARG ALPINE_VER

 FROM golang:latest as tools
 ARG GO_TAGS

 ADD . $GOPATH/src/github.com/hyperledger/fabric
 WORKDIR $GOPATH/src/github.com/hyperledger/fabric

 #Bugfix https://github.com/hyperledger/fabric/issues/3645
 RUN make tools GO_TAGS=${GO_TAGS}

 FROM ubuntu:jammy

 # Latest debian url of DINAMO client
 ENV HSM_CLIENT_URL "https://downloads.dinamonetworks.io/bin/client/linux/x64/4.7.35/dinamo-4.7.35-1.x64.deb"
 ENV DEBIAN_FRONTEND noninteractive

 # git is required to support `go list -m`
 RUN apt-get update -y && \
   apt-get install -y \
   bash \
   binutils-gold \
   gcc \
   git \
   make \
   jq \
   tzdata \
   curl \
   musl-dev;

 RUN curl -L $HSM_CLIENT_URL -o /tmp/dinamo.deb
 RUN apt-get install -y /tmp/dinamo.deb
 RUN rm /tmp/dinamo.deb

 ENV FABRIC_CFG_PATH /etc/hyperledger/fabric
 VOLUME /etc/hyperledger/fabric
 COPY --from=tools /go/src/github.com/hyperledger/fabric/build/bin /usr/local/bin
 COPY --from=tools /go/src/github.com/hyperledger/fabric/sampleconfig ${FABRIC_CFG_PATH}

  3. Compile the peer, orderer e tools.

    cd $GOPATH/src/github.com/hyperledger/fabric/
make  clean-all
make docker GO_TAGS=pkcs11

Setting up the test network

  1. Create the configuration files fabric-ca-client-config.yaml of peers.

    Use the configuration file generated in fabric-samples/test-network/organizations/peerOrganizations/org1.example.com/fabric-ca-client-config.yaml e fabric-samples/test-network/organizations/peerOrganizations/org2.example.com/fabric-ca-client-config.yaml.

    These files are temporary and are generated with each run. They are removed when you stop the test-network. They can be generated by starting the test-network (even with errors). To speed up the generation process, it is possible to stop with CTRL-C right after generating the identities of the Peer2 or as soon as the Orderer identities are generated.

      ./start-fabric.sh

  /mnt/d/tmp/hyperledger/fabric-samples/test-network /mnt/d/tmp/hyperledger
  Using docker and docker-compose
  Creating channel 'mychannel'.

  ...

  Creating Orderer Org Identities
  Enrolling the CA admin
  + fabric-ca-client enroll -u https://admin:adminpw@localhost:9054 --caname ca-orderer --tls.certfiles /mnt/d/tmp/hyperledger/fabric-samples/test-network/organizations/fabric-ca/ordererOrg/tls-cert.pem

  >>> CTRL-C <<<

  2. Create a folder for each organization with the names of org1 e org2 folder fabric-samples/test-network and make a copy of the fabric-ca-client-config.yaml to the folder of the respective organization.

    cd fabric-samples/test-network
mkdir org1
cp organizations/peerOrganizations/org1.example.com/fabric-ca-client-config.yaml org1
mkdir org2
cp organizations/peerOrganizations/org2.example.com/fabric-ca-client-config.yaml org2
cd ../..
./stop-fabric.sh

    Remember to stop the service with ./stop-fabric.sh after copying the files, in order to finish the test-network correctly.

  3. Change the section bccsp with the PKCS#11 settings, from the file fabric-ca-client-config.yaml of each organization, as follows.

    bccsp:
    default: PKCS11
    PKCS11:
        Library: /usr/lib/libtacndp11.so
        Pin: 12345678
        Label: "Dinamo HSM"
        hash: SHA2
        security: 256
        Immutable: false

  4. Change the key bccsp of the orderer.

    bccsp:
    default: PKCS11
    PKCS11:
        Library: /usr/lib/libtacndp11.so
        Pin: 12345678
        Label: "Dinamo HSM"
        hash: SHA2
        security: 256
        Immutable: false

  5. Configure the environment variables of the HSM in the compose of the compose-ca.yaml.

    This file can be found in path fabric-samples/test-network/compose/compose-ca.yaml.

    In this case we are configuring the ordererto include the variables in their respective section, as below.

    Use a different HSM user per orderer.

    services:

  ca_orderer:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal # IP do HSM
      - DFENCE_PKCS11_USER=caorderer               # Usuário do HSM
      - DFENCE_PKCS11_AUTO_RECONNECT=1

  6. Add the client settings of HSM for the peersin the file fabric-samples/test-network/compose/docker/docker-compose-test-net.yaml.

    Add the environment variables for configuring the HSM for each peer, peer0.org1.example.com e peer0.org2.example.comas in the example below.

    services:
  peer0.org1.example.com:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal
      - DFENCE_PKCS11_USER=peer1
      - DFENCE_PKCS11_AUTO_RECONNECT=1
      ...
peer0.org2.example.com:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal
      - DFENCE_PKCS11_USER=peer2
      - DFENCE_PKCS11_AUTO_RECONNECT=1

  7. Configure PKCS#11 in the file fabric-samples/test-network/compose/compose-test-net.yaml. As in the example below.

    services:
  peer0.org1.example.com:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal
      - DFENCE_PKCS11_USER=peer1
      - DFENCE_PKCS11_AUTO_RECONNECT=1
      ...
peer0.org2.example.com:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal
      - DFENCE_PKCS11_USER=peer2
      - DFENCE_PKCS11_AUTO_RECONNECT=1

  8. Add the client settings and enable PKCS#11 in the file fabric-samples/test-network/network.sh.

    In the role createOrgs include the settings before each call to createOrg1 e createOrg2as follows.

    .
.
.
infoln "Creating Org1 Identities"

    export DFENCE_PKCS11_IP=127.0.0.1
    export DFENCE_PKCS11_USER=peer1
    export DFENCE_PKCS11_AUTO_RECONNECT=1
    export HSM_LOG_DIR=/mnt/d/tmp/hyperledger
    export HSM_LOG_LEVEL=3
    export FABRIC_CA_CLIENT_BCCSP_DEFAULT=PKCS11
    export FABRIC_CA_CLIENT_BCCSP_PKCS11_LIBRARY=/usr/lib/libtacndp11.so
    export FABRIC_CA_CLIENT_BCCSP_PKCS11_PIN=12345678
    export FABRIC_CA_CLIENT_BCCSP_PKCS11_LABEL="Dinamo HSM"
    export FABRIC_CA_CLIENT_BCCSP_PKCS11_IMMUTABLE=false
    export FABRIC_CA_CLIENT_BCCSP_PKCS11_HASH=SHA2
    export FABRIC_CA_CLIENT_BCCSP_PKCS11_SECURITY=256

    createOrg1

    infoln "Creating Org2 Identities"

    export DFENCE_PKCS11_USER=peer2

    createOrg2

    unset DFENCE_PKCS11_IP
    unset DFENCE_PKCS11_USER
    unset DFENCE_PKCS11_AUTO_RECONNECT
    unset HSM_LOG_DIR
    unset HSM_LOG_LEVEL
    unset FABRIC_CA_CLIENT_BCCSP_DEFAULT
    unset FABRIC_CA_CLIENT_BCCSP_PKCS11_LIBRARY
    unset FABRIC_CA_CLIENT_BCCSP_PKCS11_PIN
    unset FABRIC_CA_CLIENT_BCCSP_PKCS11_LABEL
    unset FABRIC_CA_CLIENT_BCCSP_PKCS11_IMMUTABLE
    unset FABRIC_CA_CLIENT_BCCSP_PKCS11_HASH
    unset FABRIC_CA_CLIENT_BCCSP_PKCS11_SECURITY
.
.
.

  9. Create a folder for each organization with the names of org1 e org2 folder fabric-samples/config and make a copy of the core.yaml to the folder of the respective organization.

    cd fabric-samples/config
mkdir org1
cp ../test-network/compose/docker/peercfg/core.yaml org1
mkdir org2
cp ../test-network/compose/docker/peercfg/core.yaml org2
cd ../..
    1. Configure the PKCS#11 options by changing the section bccspin the following files:

    • fabric-samples/test-network/compose/docker/peercfg/core.yaml
    • fabric-samples/config/org1/core.yaml
    • fabric-samples/config/org2/core.yaml
    .
.
.
 bccsp:
    default: PKCS11
    PKCS11:
        Library: /usr/lib/libtacndp11.so
        Pin: 12345678
        Label: "Dinamo HSM"
        hash: SHA2
        security: 256
        Immutable: false
.
.
.

  10. Change the script fabric-samples/test-network/organizations/fabric-ca/registerEnroll.sh to use PKCS#11 and use the necessary configuration files.

    1. Editing functions createOrg1 e createOrg2with the following changes:

      1. Add at the beginning of the function, below the line (export FABRIC_CA_CLIENT_HOME...), the code below to enable PKCS#11.

      2. Copy the configuration file to be used.

      export FABRIC_CA_CLIENT_BCCSP_DEFAULT=PKCS11
cp ${PWD}/org1/fabric-ca-client-config.yaml $FABRIC_CA_CLIENT_HOME

      Change the path of the configuration file fabric-ca-client-config.yaml according to the organization used, e.g. org1/fabric-ca-client-config.yaml e org2/fabric-ca-client-config.yaml.

    2. Use the BCCSP in software for generating the TLS certificate of the peer.

    Fabric does not currently support PKCS#11 for use in TLS.

    Select the provider in software just before generating the TLS certificate and enable PKCS#11 again just after generating the certificate.

    1. Adicionar a linha:
   ```bash
   export FABRIC_CA_CLIENT_BCCSP_DEFAULT=SW
   ```

   logo após a linha:

   ```bash
   infoln "Generating the peer0-tls certificates"
   ```

1. Adicionar a linha:

   ```bash
   export FABRIC_CA_CLIENT_BCCSP_DEFAULT=PKCS11
   ```

   logo após a linha:

   ```bash
   { set +x; } 2>/dev/null
   ```

  11. Change the file fabric-samples/test-network/scripts/createChannel.sh with the PKCS#11 client settings.

    1. Add PKCS#11 settings at the start of the function createChannel.
    export DFENCE_PKCS11_IP=127.0.0.1
export DFENCE_PKCS11_USER=peer1
export DFENCE_PKCS11_AUTO_RECONNECT=1
    1. Add the following line at the beginning of the function joinChannel.
    export FABRIC_CA_CLIENT_BCCSP_DEFAULT=PKCS11

    1. Remove the following line at the beginning of the function joinChannel. 

      FABRIC_CFG_PATH=$PWD/../config/

    2. Add the PKCS#11 settings before calling the function createChannelGenesisBlock.

    export DFENCE_PKCS11_IP=host.docker.internal
export DFENCE_PKCS11_USER=peer1
export DFENCE_PKCS11_AUTO_RECONNECT=1

    1. Add the user configuration and path to the configuration before each call (joinChannel, setAnchorPeer) relating to an organization.

    2. Example, peer1add the following lines before the call to joinChannel 1.

      export DFENCE_PKCS11_USER=peer1
FABRIC_CFG_PATH="${PWD}/../config/org1"

    3. Example, peer2add the following lines before the call to joinChannel 2and so on.

      export DFENCE_PKCS11_USER=peer2
FABRIC_CFG_PATH="${PWD}/../config/org2"

  12. Change the file fabric-samples/test-network/scripts/deployCC.sh with the PKCS#11 settings.

    1. At the beginning of the file, just after the running commands are printed, for example just after the line:
    println "- VERBOSE: ${C_GREEN}${VERBOSE}${C_RESET}"
    1. Overwrite the line:
    FABRIC_CFG_PATH=$PWD/../config/

    With the following settings:

    FABRIC_CFG_PATH=$PWD/../config/org1
export DFENCE_PKCS11_IP=127.0.0.1
export DFENCE_PKCS11_USER=peer1
export DFENCE_PKCS11_AUTO_RECONNECT=1

    1. Add the PKCS#11 user configuration and configuration according to each function call, referring to an organization.

    2. Example, peer1add the lines:

      export DFENCE_PKCS11_USER=peer1
FABRIC_CFG_PATH="${PWD}/../config/org1"

      before the line:

      installChaincode 1

    3. Example, peer2add the lines:

      export DFENCE_PKCS11_USER=peer2
FABRIC_CFG_PATH="${PWD}/../config/org2"

      before the line below, and so on.

      installChaincode 2

  13. Add the PKCS#11 settings to the file fabric-samples/test-network/scripts/setAnchorPeer.sh.

    1. Add the lines below, just before the call to createAnchorPeerUpdate.

    The variable $ORG receives the value 1 or 2 according to the call. In our example, the peer of organization 1 is the peer1 and organization 2 peer2. This name construction made it possible to automatically select the user.

    export DFENCE_PKCS11_IP=host.docker.internal
export DFENCE_PKCS11_USER=peer$ORG
export DFENCE_PKCS11_AUTO_RECONNECT=1

  14. Create basic test script with PKCS#11 support test-p11-basic-transfer.sh. Configure PKCS#11 according to each organization.

    #!/bin/bash

#
#   Execute basic asset transfer test
#

pushd ./fabric-samples/test-network
export PATH=${PWD}/../bin:$PATH
export FABRIC_CFG_PATH=$PWD/../config/org1

# Environment variables for Org1
export CORE_PEER_TLS_ENABLED=true
export CORE_PEER_LOCALMSPID="Org1MSP"
export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=${PWD}/organizations/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
export CORE_PEER_ADDRESS=localhost:7051

# PKCS#11 configurations for Org1
export DFENCE_PKCS11_IP=127.0.0.1   # HSM IP Address
export DFENCE_PKCS11_USER=peer1     # HSM User
export DFENCE_PKCS11_AUTO_RECONNECT=1

peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile "${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem" -C mychannel -n basic --peerAddresses localhost:7051 --tlsRootCertFiles "${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt" --peerAddresses localhost:9051 --tlsRootCertFiles "${PWD}/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt" -c '{"function":"InitLedger","Args":[]}'
peer chaincode query -C mychannel -n basic -c '{"Args":["GetAllAssets"]}'
peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile "${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem" -C mychannel -n basic --peerAddresses localhost:7051 --tlsRootCertFiles "${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt" --peerAddresses localhost:9051 --tlsRootCertFiles "${PWD}/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt" -c '{"function":"TransferAsset","Args":["asset6","Christopher"]}'

# Environment variables for Org2
export CORE_PEER_TLS_ENABLED=true
export CORE_PEER_LOCALMSPID="Org2MSP"
export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=${PWD}/organizations/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp
export CORE_PEER_ADDRESS=localhost:9051

# PKCS#11 configurations for Org2
export DFENCE_PKCS11_USER=peer2  # HSM User
export FABRIC_CFG_PATH=$PWD/../config/org2

peer chaincode query -C mychannel -n basic -c '{"Args":["ReadAsset","asset6"]}'
popd

Go to the test network and run a test

  1. To upload, run the tests and then stop the test net, use the following scripts.

    ./start-fabric.sh
./test-p11-basic-transfer.sh
./test-p11-ca-client.sh
./stop-fabric.sh