Skip to content

Hyperledger Fabric

Prerequisites

  • The OS where the integration via PKCS#11 will be carried out must be compatible with the HSM libraries.
  • Since Fabric defaults to using Alpine Linux, you need to change the dockerfile to use a compatible OS.

Versions

  • This procedure was done with the following version of Hyperledger Fabric, on WSL 2 of Windows 11

    Component Version
    fabric-ca (server/client) 1.5.5
    fabric-tool 2.4.7
    fabric-peer 2.4.7
    fabric-orderer 2.4.7
    S.O. base Ubuntu 20.04 over WSL2 on Windows 11
    S.O. containers Ubuntu 22.04 (jammy)
    Go 1.20
    Client HSM 4.7.35
    HSM firmware 5.0.28.0-243-g5a9cb01

Initial Setup without PKCS#11

  1. Follow the procedure described in Fabric test network. In the Getting started guide only configure the following topics:

    1. Prerequisite software
    2. Install Fabric and Fabric samples

  2. Start a test network from scratch every time you run it. If the environment is not clean, errors will occur at various points in the test network's ascent.

  3. Monitor the logs and outputs of the test network containers. The logs from the HSM client can be viewed here, when connected with the stdout option:

    HSM_LOG_DIR=stdout

Integration with HSM via PKCS#11

  • In the following integration examples, independent HSM users/partitions are used for each component/organization.

    Component User name Observations Where it is used
    Fabric-ca-client integration test caclient Used in the fabric-ca-client integration test. host
    Fabric-ca ca1 CA of organization 1 container
    Fabric-ca ca2 CA of organization 2 container
    Fabric-ca caorderer Orderer's CA container
    Peer 1 peer1 Peer from organization 1 container
    Peer 2 peer2 Peer from organization 2 container

Initial setup

  1. You must follow the steps in the Initial Setup without PKCS#11 section before proceeding.

    Running on an Ubuntu 20.04 on Windows 11 WSL2.

  2. Set the GOPATH environment variable to the working directory, in this case we'll use ~/go. You can set it in the ~/.profile file to make it easier to use.

    export GOPATH=$HOME/go

Recompile the fabric and fabric-ca tools with PKCS#11 support

  1. Create Hyperledger folders and clone fabric-ca repository.

    cd $GOPATH
mkdir -p src/github.com/hyperledger/
cd src/github.com/hyperledger
git clone -b v1.5.5 https://github.com/hyperledger/fabric-ca
cd fabric-ca/

  2. Compile the fabric-ca and fabric-ca-client binaries with PKCS#11 support.

    make fabric-ca-server GO_TAGS=pkcs11
make fabric-ca-client GO_TAGS=pkcs11

    The binaries are generated in the fabric-ca/bin folder.

  3. Copy the fabric-ca and fabric-ca-client binaries to the fabric-samples/bin folder.

  4. Clone fabric repository .

    cd $GOPATH/src/github.com/hyperledger
git clone -b v2.4.7 https://github.com/hyperledger/fabric.git
cd fabric

  5. Compile the tools binaries with PKCS#11 support.

    make tools GO_TAGS=pkcs11

    The binaries are generated in the build/bin folder.

  6. Copy the generated binaries to the fabric-samples/bin folder.

Integration with fabric-ca-client

  • There is a fabric-ca-client test in the ./fabric-samples/hardware-security-module folder.
  • The file, ./fabric-samples/hardware-security-module/README.md can be used as a guide, but needs to be adapted.

This test can be run from the machine itself, without the need for a container. In this case it was run on an Ubuntu 20.04 on Windows 11 WSL2.

  1. For the test we need to change the hard-codes.

    ./fabric-samples/hardware-security-module/ca-client-config/fabric-ca-client-config-template.yaml

    In the bccsp option, change the following fields: - Label: Label of the P11 token. In the case of Dinamo it is "Dinamo HSM". - Pin: Password used to log in to P11. Password of the user configured on the P11 Dinamo.

    Do not change the Library field. The placeholder is overwritten by the execution script.

    #############################################################################
# BCCSP (BlockChain Crypto Service Provider) section allows to select which
# crypto implementation library to use
#############################################################################
bccsp:
    default: PKCS11
    PKCS11:
        Library: REPLACE_ME_HSMLIB
        Pin: 12345678
        Label: "Dinamo HSM"
        hash: SHA2
        security: 256

  2. Change the HSM IP and user in the script below. This script will generate the fabric configuration file that will be used to make the fabric-ca-client calls to generate a new user.

    #!/bin/bash

#
#   Test PKCS#11 CA client
#

# Add the path to the fabric-ca-client with pkcs11 support
export PATH=./fabric-samples/bin:$PATH
export PKCS11_LIB=/usr/lib/libtacndp11.so # path to the installed pkcs11

# Dinamo pkcs11 configuration
export DFENCE_PKCS11_IP=127.0.0.1     # HSM IP Address
export DFENCE_PKCS11_USER=caclient  # HSM User
export DFENCE_PKCS11_AUTO_RECONNECT=1
#export HSM_LOG_DIR=./
#export HSM_LOG_LEVEL=3

./fabric-samples/hardware-security-module/scripts/generate-hsm-user.sh HSMUser

  3. To run the test, make the changes and run the script above.

    Check that the user configured in test-p11-ca-client.sh exists in the HSM.

    ./start-fabric.sh
./test-p11-ca-client.sh
./stop-fabric.sh

    Sample output

    ./test-p11-ca-client.sh
2022/11/24 19:06:08 [INFO] TLS Enabled
2022/11/24 19:06:08 [INFO] generating key: &{A:ecdsa S:256}
2022-11-24 19:06:08.860 -03 [bccsp_p11] generateECKey -> INFO 001 Generated new P11 key, SKI 80feb3f43df12ef5dae75a3dd3502092d702a8dd80e8fcb92335c41f54e372d0
2022/11/24 19:06:08 [INFO] encoded CSR
.
.
.

Integration with fabric-ca-server

  • Versions used:

    Component Version
    S.O. Ubuntu 22.04 (jammy)
    Go 1.20
    Client HSM 3.7.35-1

  • For Hyperledger to work with HSM we need to recompile the container with the following settings:

    • Use a Linux distro that supports the HSM client, in this example we'll use Ubuntu;
    • Enable support for PKCS#11.

Generate image with PKCS#11 support

  1. Change the Dockerfile to compile run with Ubuntu and install the HSM client. The Dockerfile is located in the $GOPATH/src/github.com/hyperledger/fabric-ca/images/fabric-ca folder. After the changes, the file should look like this.

    #
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
ARG GO_VER

FROM ubuntu:jammy
ARG GO_LDFLAGS
ARG GO_TAGS

# Latest debian url of DINAMO client
ENV HSM_CLIENT_URL "https://downloads.dinamonetworks.io/bin/client/linux/x64/4.7.35/dinamo-4.7.35-1.x64.deb"
ENV GO_URL "https://storage.googleapis.com/golang/go1.19.3.linux-amd64.tar.gz"
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update -y && \
  apt-get install -y \
  apt-utils \
  gcc \
  binutils-gold \
  git \
  curl;

RUN curl -L $GO_URL | tar -v -C /usr/local -xz

ENV GOPATH /go
RUN mkdir -p "$GOPATH/src" "$GOPATH/bin"
ENV PATH $GOPATH/bin:/usr/local/go/bin:$PATH

WORKDIR /tmp
RUN curl -L $HSM_CLIENT_URL -o ./dinamo.deb
RUN apt-get install -y ./dinamo.deb
RUN rm ./dinamo.deb

ADD . /build/fabric-ca
WORKDIR /build/fabric-ca
RUN go install -tags "${GO_TAGS}" -ldflags "${GO_LDFLAGS}" \
  github.com/hyperledger/fabric-ca/cmd/fabric-ca-server \
  && go install -tags "${GO_TAGS}" -ldflags "${GO_LDFLAGS}" \
  github.com/hyperledger/fabric-ca/cmd/fabric-ca-client

RUN apt-get install -y \
  tzdata;
ENV FABRIC_CA_HOME /etc/hyperledger/fabric-ca-server
EXPOSE 7054
CMD fabric-ca-server start -b admin:adminpw

  2. Compile the fabric-ca image.

    cd $GOPATH/src/github.com/hyperledger/fabric-ca
export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0
make clean-all
make docker GO_TAGS=pkcs11

Setting up the test network

  1. Change each organization's fabric-ca-server-config.yaml configuration files so that they use the HSM's PKCS#11. These files are located in the paths below:

    • fabric-samples/test-network/organizations/fabric-ca/org1/fabric-ca-server-config.yaml.
    • fabric-samples/test-network/organizations/fabric-ca/org2/fabric-ca-server-config.yaml.


    These settings can also be defined by environment variables as shown in the Fabric HSM documentation.

  2. Change the bccsp section to the following values.

    bccsp:
    default: PKCS11
    PKCS11:
        Library: /usr/lib/libtacndp11.so
        Pin: 12345678 # Senha do usuário do HSM
        Label: "Dinamo HSM"
        hash: SHA2
        security: 256
        Immutable: false

  3. Configure the HSM client's environment variables in the ca compose file compose-ca.yaml. This file can be found in fabric-samples/test-network/compose/compose-ca.yaml.

    In this case, we are configuring the CA of organization 1In this case, you can add the environment variables in their respective section, as below.

    Use a different HSM user for each of the CAs.

    services:

  ca_org1:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal # IP do HSM
      - DFENCE_PKCS11_USER=ca1 # Usuário do HSM
      - DFENCE_PKCS11_AUTO_RECONNECT=1

  ...

  ca_org2:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal # IP do HSM
      - DFENCE_PKCS11_USER=ca2 # Usuário do HSM
      - DFENCE_PKCS11_AUTO_RECONNECT=1

Go to the test network and run a test

  1. Run the test script.

  2. Example output from the basic transfer test

    ./test-basic-transfer.sh
2022-11-14 18:32:53.487 -03 0001 INFO [chaincodeCmd] chaincodeInvokeOrQuery -> Chaincode invoke successful. result: status:200

2022-11-14 18:32:56.826 -03 0001 INFO [chaincodeCmd] chaincodeInvokeOrQuery -> Chaincode invoke successful. result: status:200 payload:"Michel"
{"AppraisedValue":800,"Color":"white","ID":"asset6","Owner":"Michel","Size":15}

  3. Check via the HSM 's remote log that the configured user (e.g. ca1) has been used correctly.

    2022/11/14 21:28:05 00000062 000000B7 995E9769 ca1/1d571851df3d6f v-attr update|172.17.0.1 172.17.0.2:4433 ca1
2022/11/14 21:28:05 00000062 000000B8 995E9769 MANAGE_P11 (03:ca1/1d571851df3d6f) [00000000], c: 23|172.17.0.1 172.17.0.2:4433 ca1
2022/11/14 21:28:05 00000062 000000B9 995E9769 172.17.0.1#23 probe|172.17.0.1 172.17.0.2:4433 ca1
2022/11/14 21:28:05 00000062 000000BA 995E9769 ecc 1d571851df3d6f!0nMFd8VvAX80FxUQ7L+1Sh6eRZ6SSEJSJLuq6BasfL8=, c: 23|172.17.0.1 172.17.0.2:4433 ca1

  4. Check that the keys have been created on the test user.

    Dinamo - Remote Management Console v. 4.7.34.0 2018 (c) Dinamo Networks

HSM 127.0.0.1 e - Engine 5.0.28.0-243-g5a9cb01 (DXP) - TCA0000000  - ID ca1

Keys/Objects - List


Name                                      Type                 T E Label
================================================================================
1d571851df3d6f                            prime256v1           n n e60bde44a80e532e20e5ac0952a093a4659e2736bca5b729a5708e6fbdfb8aee
a8d9d38535ec9a                            prime256v1 pub       n y ea1f764edd0e8187409d61dfd78313d706b1b146fecb8a99b5f69309aced5de8
d6f690620dee1a                            prime256v1           n n ea1f764edd0e8187409d61dfd78313d706b1b146fecb8a99b5f69309aced5de8
d7ac2114025001                            prime256v1 pub       n y e60bde44a80e532e20e5ac0952a093a4659e2736bca5b729a5708e6fbdfb8aee

Total of objects: 4

Integration with orderer, peer and tools (container)

  • The tools container is a container that has the basic tools for interacting with the HSM compiled and executed in the container itself. Some parts of the test network scripts intersperse calls to locally compiled tools (on the local machine) and executions of these tools from the container.

Generate images with PKCS#11 support

The Dockerfiles below are in the path $GOPATH/src/github.com/hyperledger/fabric/images/.

  1. Edit the Dockerfile files of the peer and orderer files to be compiled with Ubuntu and with PKCS#11 support.

    # Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0

ARG GO_VER
ARG ALPINE_VER

FROM ubuntu:jammy as peer-base

# Latest debian url of DINAMO client
ENV HSM_CLIENT_URL "https://downloads.dinamonetworks.io/bin/client/linux/x64/4.7.35/dinamo-4.7.35-1.x64.deb"
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update -y && \
  apt-get install -y tzdata

# Bugfix https://github.com/hyperledger/fabric/issues/3779
RUN if [[ ! -e /etc/nsswitch.conf ]] ; then echo 'hosts: files dns' > /etc/nsswitch.conf ; fi

RUN apt-get install -y \
  bash \
  binutils-gold \
  gcc \
  git \
  make \
  musl-dev \
  curl

RUN curl -L $HSM_CLIENT_URL -o /tmp/dinamo.deb
RUN apt-get install -y /tmp/dinamo.deb
RUN rm /tmp/dinamo.deb

FROM golang:latest as golang
ADD . $GOPATH/src/github.com/hyperledger/fabric
WORKDIR $GOPATH/src/github.com/hyperledger/fabric

FROM golang as peer
ARG GO_TAGS
#Bugfix https://github.com/hyperledger/fabric/issues/3645
RUN make peer GO_TAGS=${GO_TAGS}
RUN make ccaasbuilder
RUN \
    export MARCH=$(go env GOOS)-$(go env GOARCH) && \
    mkdir -p release/linux-platform && \
    cp -r release/${MARCH}/builders release/linux-platform/.

FROM peer-base
ENV FABRIC_CFG_PATH /etc/hyperledger/fabric
VOLUME /etc/hyperledger/fabric
VOLUME /var/hyperledger
COPY --from=peer /go/src/github.com/hyperledger/fabric/build/bin /usr/local/bin
COPY --from=peer /go/src/github.com/hyperledger/fabric/sampleconfig/msp ${FABRIC_CFG_PATH}/msp
COPY --from=peer /go/src/github.com/hyperledger/fabric/sampleconfig/core.yaml ${FABRIC_CFG_PATH}/core.yaml
COPY --from=peer /go/src/github.com/hyperledger/fabric/release/linux-platform/builders/ccaas/bin/ /opt/hyperledger/ccaas_builder/bin/
EXPOSE 7051
CMD ["peer","node","start"]

    # Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0

ARG GO_VER
ARG ALPINE_VER
FROM ubuntu:jammy as base

# Latest debian url of DINAMO client
ENV HSM_CLIENT_URL "https://downloads.dinamonetworks.io/bin/client/linux/x64/4.7.35/dinamo-4.7.35-1.x64.deb"
ENV GO_URL "https://storage.googleapis.com/golang/go1.19.3.linux-amd64.tar.gz"
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update -y && \
  apt-get install -y tzdata

# Bugfix https://github.com/hyperledger/fabric/issues/3779
RUN if [[ ! -e /etc/nsswitch.conf ]] ; then echo 'hosts: files dns' > /etc/nsswitch.conf ; fi

RUN apt-get install -y \
  bash \
  binutils-gold \
  gcc \
  git \
  make \
  musl-dev \
  curl

RUN curl -L $HSM_CLIENT_URL -o /tmp/dinamo.deb
RUN apt-get install -y /tmp/dinamo.deb
RUN rm /tmp/dinamo.deb

FROM golang:latest as golang
ADD . $GOPATH/src/github.com/hyperledger/fabric
WORKDIR $GOPATH/src/github.com/hyperledger/fabric

FROM golang as orderer
ARG GO_TAGS
#Bugfix https://github.com/hyperledger/fabric/issues/3645
RUN make orderer GO_TAGS=${GO_TAGS}

FROM base
ENV FABRIC_CFG_PATH /etc/hyperledger/fabric
VOLUME /etc/hyperledger/fabric
VOLUME /var/hyperledger
COPY --from=orderer /go/src/github.com/hyperledger/fabric/build/bin /usr/local/bin
COPY --from=orderer /go/src/github.com/hyperledger/fabric/sampleconfig/msp ${FABRIC_CFG_PATH}/msp
COPY --from=orderer /go/src/github.com/hyperledger/fabric/sampleconfig/orderer.yaml ${FABRIC_CFG_PATH}
COPY --from=orderer /go/src/github.com/hyperledger/fabric/sampleconfig/configtx.yaml ${FABRIC_CFG_PATH}
EXPOSE 7050
CMD ["orderer"]

  2. Configure the Dockerfile from tools to use the HSM client.

     # Copyright IBM Corp. All Rights Reserved.
 #
 # SPDX-License-Identifier: Apache-2.0

 ARG GO_VER
 ARG ALPINE_VER

 FROM golang:latest as tools
 ARG GO_TAGS

 ADD . $GOPATH/src/github.com/hyperledger/fabric
 WORKDIR $GOPATH/src/github.com/hyperledger/fabric

 #Bugfix https://github.com/hyperledger/fabric/issues/3645
 RUN make tools GO_TAGS=${GO_TAGS}

 FROM ubuntu:jammy

 # Latest debian url of DINAMO client
 ENV HSM_CLIENT_URL "https://downloads.dinamonetworks.io/bin/client/linux/x64/4.7.35/dinamo-4.7.35-1.x64.deb"
 ENV DEBIAN_FRONTEND noninteractive

 # git is required to support `go list -m`
 RUN apt-get update -y && \
   apt-get install -y \
   bash \
   binutils-gold \
   gcc \
   git \
   make \
   jq \
   tzdata \
   curl \
   musl-dev;

 RUN curl -L $HSM_CLIENT_URL -o /tmp/dinamo.deb
 RUN apt-get install -y /tmp/dinamo.deb
 RUN rm /tmp/dinamo.deb

 ENV FABRIC_CFG_PATH /etc/hyperledger/fabric
 VOLUME /etc/hyperledger/fabric
 COPY --from=tools /go/src/github.com/hyperledger/fabric/build/bin /usr/local/bin
 COPY --from=tools /go/src/github.com/hyperledger/fabric/sampleconfig ${FABRIC_CFG_PATH}

  3. Compile the peer, orderer e tools.

    cd $GOPATH/src/github.com/hyperledger/fabric/
make  clean-all
make docker GO_TAGS=pkcs11

Setting up the test network

  1. Create the fabric-ca-client-config.yaml configuration files of the peers.

    Use the configuration file generated in fabric-samples/test-network/organizations/peerOrganizations/org1.example.com/fabric-ca-client-config.yaml and fabric-samples/test-network/organizations/peerOrganizations/org2.example.com/fabric-ca-client-config.yaml as a basis.

    These files are temporary and are generated with each run. They are removed when you stop the test-network. They can be generated by starting the test-network (even with errors). To speed up the generation process, you can stop with CTRL-C just after generating the Peer2 identities or just after starting to generate the Orderer identities.

      ./start-fabric.sh

  /mnt/d/tmp/hyperledger/fabric-samples/test-network /mnt/d/tmp/hyperledger
  Using docker and docker-compose
  Creating channel 'mychannel'.

  ...

  Creating Orderer Org Identities
  Enrolling the CA admin
  + fabric-ca-client enroll -u https://admin:adminpw@localhost:9054 --caname ca-orderer --tls.certfiles /mnt/d/tmp/hyperledger/fabric-samples/test-network/organizations/fabric-ca/ordererOrg/tls-cert.pem

  >>> CTRL-C <<<

  2. Create a folder for each organization with the names org1 and org2 in the fabric-samples/test-network folder and make a copy of the fabric-ca-client-config.yaml file to the folder of the respective organization.

    cd fabric-samples/test-network
mkdir org1
cp organizations/peerOrganizations/org1.example.com/fabric-ca-client-config.yaml org1
mkdir org2
cp organizations/peerOrganizations/org2.example.com/fabric-ca-client-config.yaml org2
cd ../..
./stop-fabric.sh

    Remember to stop the service with ./stop-fabric.sh after copying the files, in order to end the test-network correctly.

  3. Change the bccsp section with the PKCS#11 settings in each organization's fabric-ca-client-config.yaml file as follows.

    bccsp:
    default: PKCS11
    PKCS11:
        Library: /usr/lib/libtacndp11.so
        Pin: 12345678
        Label: "Dinamo HSM"
        hash: SHA2
        security: 256
        Immutable: false

  4. Change the bccsp key in the orderer.

    bccsp:
    default: PKCS11
    PKCS11:
        Library: /usr/lib/libtacndp11.so
        Pin: 12345678
        Label: "Dinamo HSM"
        hash: SHA2
        security: 256
        Immutable: false

  5. Configure the HSM client's environment variables in the ca compose file compose-ca.yaml.

    This file is in the path fabric-samples/test-network/compose/compose-ca.yaml.

    In this case we are configuring the ordererto include the variables in their respective section, as below.

    Use a different HSM user per orderer.

    services:

  ca_orderer:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal # IP do HSM
      - DFENCE_PKCS11_USER=caorderer               # Usuário do HSM
      - DFENCE_PKCS11_AUTO_RECONNECT=1

  6. Add the HSM client settings for the peersin the file fabric-samples/test-network/compose/docker/docker-compose-test-net.yaml.

    Add the HSM client configuration environment variables for each peerpeer0.org1.example.com and peer0.org2.example.com, as in the example below.

    services:
  peer0.org1.example.com:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal
      - DFENCE_PKCS11_USER=peer1
      - DFENCE_PKCS11_AUTO_RECONNECT=1
      ...
peer0.org2.example.com:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal
      - DFENCE_PKCS11_USER=peer2
      - DFENCE_PKCS11_AUTO_RECONNECT=1

  7. Configure PKCS#11 in the fabric-samples/test-network/compose/compose-test-net.yaml file. As in the example below.

    services:
  peer0.org1.example.com:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal
      - DFENCE_PKCS11_USER=peer1
      - DFENCE_PKCS11_AUTO_RECONNECT=1
      ...
peer0.org2.example.com:
    ...
    environment:
      ...
      - DFENCE_PKCS11_IP=host.docker.internal
      - DFENCE_PKCS11_USER=peer2
      - DFENCE_PKCS11_AUTO_RECONNECT=1

  8. Add the client settings and enable PKCS#11 in the fabric-samples/test-network/network.sh file.

    In the createOrgs function, include the settings before each call to createOrg1 and createOrg2, as follows.

    .
.
.
infoln "Creating Org1 Identities"

    export DFENCE_PKCS11_IP=127.0.0.1
    export DFENCE_PKCS11_USER=peer1
    export DFENCE_PKCS11_AUTO_RECONNECT=1
    export HSM_LOG_DIR=/mnt/d/tmp/hyperledger
    export HSM_LOG_LEVEL=3
    export FABRIC_CA_CLIENT_BCCSP_DEFAULT=PKCS11
    export FABRIC_CA_CLIENT_BCCSP_PKCS11_LIBRARY=/usr/lib/libtacndp11.so
    export FABRIC_CA_CLIENT_BCCSP_PKCS11_PIN=12345678
    export FABRIC_CA_CLIENT_BCCSP_PKCS11_LABEL="Dinamo HSM"
    export FABRIC_CA_CLIENT_BCCSP_PKCS11_IMMUTABLE=false
    export FABRIC_CA_CLIENT_BCCSP_PKCS11_HASH=SHA2
    export FABRIC_CA_CLIENT_BCCSP_PKCS11_SECURITY=256

    createOrg1

    infoln "Creating Org2 Identities"

    export DFENCE_PKCS11_USER=peer2

    createOrg2

    unset DFENCE_PKCS11_IP
    unset DFENCE_PKCS11_USER
    unset DFENCE_PKCS11_AUTO_RECONNECT
    unset HSM_LOG_DIR
    unset HSM_LOG_LEVEL
    unset FABRIC_CA_CLIENT_BCCSP_DEFAULT
    unset FABRIC_CA_CLIENT_BCCSP_PKCS11_LIBRARY
    unset FABRIC_CA_CLIENT_BCCSP_PKCS11_PIN
    unset FABRIC_CA_CLIENT_BCCSP_PKCS11_LABEL
    unset FABRIC_CA_CLIENT_BCCSP_PKCS11_IMMUTABLE
    unset FABRIC_CA_CLIENT_BCCSP_PKCS11_HASH
    unset FABRIC_CA_CLIENT_BCCSP_PKCS11_SECURITY
.
.
.

  9. Create a folder for each organization with the names org1 and org2 in the fabric-samples/config folder and make a copy of the core.yaml file to the folder of the respective organization.

    cd fabric-samples/config
mkdir org1
cp ../test-network/compose/docker/peercfg/core.yaml org1
mkdir org2
cp ../test-network/compose/docker/peercfg/core.yaml org2
cd ../..
    1. Configure the PKCS#11 options by changing the section bccspin the following files:

    • fabric-samples/test-network/compose/docker/peercfg/core.yaml
    • fabric-samples/config/org1/core.yaml
    • fabric-samples/config/org2/core.yaml
    .
.
.
 bccsp:
    default: PKCS11
    PKCS11:
        Library: /usr/lib/libtacndp11.so
        Pin: 12345678
        Label: "Dinamo HSM"
        hash: SHA2
        security: 256
        Immutable: false
.
.
.

  10. Change the fabric-samples/test-network/organizations/fabric-ca/registerEnroll.sh script to use PKCS#11 and use the necessary configuration files.

    1. Edit the createOrg1 and createOrg2 functions with the following changes:

      1. Add the following code at the beginning of the function, below the line(export FABRIC_CA_CLIENT_HOME...), to enable PKCS#11.

      2. Copy the configuration file to be used.

      export FABRIC_CA_CLIENT_BCCSP_DEFAULT=PKCS11
cp ${PWD}/org1/fabric-ca-client-config.yaml $FABRIC_CA_CLIENT_HOME

      Change the path of the configuration file fabric-ca-client-config.yaml according to the organization used, e.g. org1/fabric-ca-client-config.yaml and org2/fabric-ca-client-config.yaml.

    2. Use BCCSP in software to generate the TLS certificate of the peer.

    Fabric does not currently support PKCS#11 for use in TLS.

    Select the provider in software just before generating the TLS certificate and enable PKCS#11 again just after generating the certificate.

    1. Adicionar a linha:
   ```bash
   export FABRIC_CA_CLIENT_BCCSP_DEFAULT=SW
   ```

   logo após a linha:

   ```bash
   infoln "Generating the peer0-tls certificates"
   ```

1. Adicionar a linha:

   ```bash
   export FABRIC_CA_CLIENT_BCCSP_DEFAULT=PKCS11
   ```

   logo após a linha:

   ```bash
   { set +x; } 2>/dev/null
   ```

  11. Change the fabric-samples/test-network/scripts/createChannel.sh file with the PKCS#11 client settings.

    1. Add the PKCS#11 settings at the start of the createChannel function.
    export DFENCE_PKCS11_IP=127.0.0.1
export DFENCE_PKCS11_USER=peer1
export DFENCE_PKCS11_AUTO_RECONNECT=1
    1. Add the following line at the beginning of the joinChannel function.
    export FABRIC_CA_CLIENT_BCCSP_DEFAULT=PKCS11

    1. Remove the following line at the beginning of the joinChannel function. 

      FABRIC_CFG_PATH=$PWD/../config/

    2. Add the PKCS#11 settings before calling the createChannelGenesisBlock function.

    export DFENCE_PKCS11_IP=host.docker.internal
export DFENCE_PKCS11_USER=peer1
export DFENCE_PKCS11_AUTO_RECONNECT=1

    1. Add the user configuration and path to the configuration before each call(joinChannel, setAnchorPeer) relating to an organization.

    2. Example, peer1add the following lines before the call to joinChannel 1.

      export DFENCE_PKCS11_USER=peer1
FABRIC_CFG_PATH="${PWD}/../config/org1"

    3. Example, peer2add the following lines before the call to joinChannel 2, and so on.

      export DFENCE_PKCS11_USER=peer2
FABRIC_CFG_PATH="${PWD}/../config/org2"

  12. Change the fabric-samples/test-network/scripts/deployCC.sh file with the PKCS#11 settings.

    1. At the beginning of the file, just after the running commands are printed, for example just after the line:
    println "- VERBOSE: ${C_GREEN}${VERBOSE}${C_RESET}"
    1. Overwrite the line:
    FABRIC_CFG_PATH=$PWD/../config/

    With the following settings:

    FABRIC_CFG_PATH=$PWD/../config/org1
export DFENCE_PKCS11_IP=127.0.0.1
export DFENCE_PKCS11_USER=peer1
export DFENCE_PKCS11_AUTO_RECONNECT=1

    1. Add the PKCS#11 user configuration and configuration according to each function call, referring to an organization.

    2. Example, peer1add the lines:

      export DFENCE_PKCS11_USER=peer1
FABRIC_CFG_PATH="${PWD}/../config/org1"

      before the line:

      installChaincode 1

    3. Example, peer2add the lines:

      export DFENCE_PKCS11_USER=peer2
FABRIC_CFG_PATH="${PWD}/../config/org2"

      before the line below, and so on.

      installChaincode 2

  13. Add the PKCS#11 settings to the fabric-samples/test-network/scripts/setAnchorPeer.sh file.

    1. Add the lines below, just before the call to createAnchorPeerUpdate.

    The $ORG variable is assigned a value of 1 or 2 depending on the call. In our example, the peer of organization 1 is peer1 and from organization 2 peer2. This name construction allowed for automatic user selection.

    export DFENCE_PKCS11_IP=host.docker.internal
export DFENCE_PKCS11_USER=peer$ORG
export DFENCE_PKCS11_AUTO_RECONNECT=1

  14. Create basic test script with PKCS#11 support test-p11-basic-transfer.sh. Configure PKCS#11 according to each organization.

    #!/bin/bash

#
#   Execute basic asset transfer test
#

pushd ./fabric-samples/test-network
export PATH=${PWD}/../bin:$PATH
export FABRIC_CFG_PATH=$PWD/../config/org1

# Environment variables for Org1
export CORE_PEER_TLS_ENABLED=true
export CORE_PEER_LOCALMSPID="Org1MSP"
export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=${PWD}/organizations/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
export CORE_PEER_ADDRESS=localhost:7051

# PKCS#11 configurations for Org1
export DFENCE_PKCS11_IP=127.0.0.1   # HSM IP Address
export DFENCE_PKCS11_USER=peer1     # HSM User
export DFENCE_PKCS11_AUTO_RECONNECT=1

peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile "${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem" -C mychannel -n basic --peerAddresses localhost:7051 --tlsRootCertFiles "${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt" --peerAddresses localhost:9051 --tlsRootCertFiles "${PWD}/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt" -c '{"function":"InitLedger","Args":[]}'
peer chaincode query -C mychannel -n basic -c '{"Args":["GetAllAssets"]}'
peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile "${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem" -C mychannel -n basic --peerAddresses localhost:7051 --tlsRootCertFiles "${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt" --peerAddresses localhost:9051 --tlsRootCertFiles "${PWD}/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt" -c '{"function":"TransferAsset","Args":["asset6","Christopher"]}'

# Environment variables for Org2
export CORE_PEER_TLS_ENABLED=true
export CORE_PEER_LOCALMSPID="Org2MSP"
export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=${PWD}/organizations/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp
export CORE_PEER_ADDRESS=localhost:9051

# PKCS#11 configurations for Org2
export DFENCE_PKCS11_USER=peer2  # HSM User
export FABRIC_CFG_PATH=$PWD/../config/org2

peer chaincode query -C mychannel -n basic -c '{"Args":["ReadAsset","asset6"]}'
popd

Go to the test network and run a test

  1. To upload, run the tests and then stop the test net, use the following scripts.

    ./start-fabric.sh
./test-p11-basic-transfer.sh
./test-p11-ca-client.sh
./stop-fabric.sh