Hyperledger Fabric Smart Contract with private data works in vscode but not in external app. How can I write data to multiple implicit collections? - hyperledger-fabric

Question

Summary: An external app can submit a transaction that writes to a single implicit private collection but fails when writing to 2 implicit collections. Everything works OK when using the vscode blockchain extension instead of the external app.
Details:
I am using the vscode blockchain extension (v2.0.8) When used, it installs microfab version 0.0.11.
I am using a 2-org network created in the vscode extension using the 2-org template.
I have a Smart Contract that writes data to 2 implicit private collections (for Org1 and Org2).
Here is the relevant portion of the smart contract (typescript):
#Transaction()
public async createMyPrivateAsset(ctx: Context, myPrivateAssetId: string): Promise<void> {
const exists: boolean = await this.myPrivateAssetExists(ctx, myPrivateAssetId);
if (exists) {
throw new Error(`The asset my private asset ${myPrivateAssetId} already exists`);
}
const privateAsset: MyPrivateAsset = new MyPrivateAsset();
const transientData: Map<string, Uint8Array> = ctx.stub.getTransient();
if (transientData.size === 0 || !transientData.has('privateValue')) {
throw new Error('The privateValue key was not specified in transient data. Please try again.');
}
privateAsset.privateValue = transientData.get('privateValue').toString();
const collectionName: string = await getCollectionName(ctx, ctx.clientIdentity.getMSPID());
await ctx.stub.putPrivateData(collectionName, myPrivateAssetId, Buffer.from(JSON.stringify(privateAsset)));
}
#Transaction()
public async createMyPrivateAssetMultiple(ctx: Context, myPrivateAssetId: string): Promise<void> {
const exists: boolean = await this.myPrivateAssetExists(ctx, myPrivateAssetId);
if (exists) {
throw new Error(`The asset my private asset ${myPrivateAssetId} already exists`);
}
const privateAsset: MyPrivateAsset = new MyPrivateAsset();
const transientData: Map<string, Uint8Array> = ctx.stub.getTransient();
if (transientData.size === 0 || !transientData.has('privateValue')) {
throw new Error('The privateValue key was not specified in transient data. Please try again.');
}
privateAsset.privateValue = transientData.get('privateValue').toString();
for (const mspid of ['Org1MSP', 'Org2MSP']) {
var collectionName: string = await getCollectionName(ctx, mspid);
await ctx.stub.putPrivateData(collectionName, myPrivateAssetId, Buffer.from(JSON.stringify(privateAsset)));
}
}
#Transaction(false)
#Returns('MyPrivateAsset')
public async readMyPrivateAsset(ctx: Context, myPrivateAssetId: string): Promise<string> {
const exists: boolean = await this.myPrivateAssetExists(ctx, myPrivateAssetId);
if (!exists) {
throw new Error(`The asset my private asset ${myPrivateAssetId} does not exist`);
}
let privateDataString: string;
const collectionName: string = await getCollectionName(ctx, ctx.clientIdentity.getMSPID());
const privateData: Uint8Array = await ctx.stub.getPrivateData(collectionName, myPrivateAssetId);
privateDataString = JSON.parse(privateData.toString());
return privateDataString;
}
createMyPrivateAsset writes to a single implicit collection: everything OK.
createMyPrivateAssetMultiple writes to 2 implicit collections: fails in external app.
Both transactions work perfectly when I use the vscode Transaction View to submit transactions.
For createMyPrivateAssetMultiple, I submit using the Org1 gateway and then call readMyPrivateAsset using the Org1 gateway and also using the Org2 gateway and the private data is returned correctly.
Now, when I use an external app, the transaction createMyPrivateAsset works but createMyPrivateAssetMultiple does not.
Here is the relevant portion of the app (typescript):
// connection
const gateway: Gateway = new Gateway();
const connectionProfilePath: string = path.resolve(__dirname, '..', connectionFile);
const connectionProfile = JSON.parse(fs.readFileSync(connectionProfilePath, 'utf8'));
const connectionOptions: GatewayOptions = { wallet, identity: identity, discovery: { enabled: true, asLocalhost: true } };
await gateway.connect(connectionProfile, connectionOptions);
// Get the network (channel) our contract is deployed to.
const network = await gateway.getNetwork('mychannel');
// Get the contract from the network.
const contract = network.getContract('private-contract');
Here is the transaction submission for createMyPrivateAsset
let transientData = {
'privateValue': Buffer.from(`Private value for asset ${assetId}`)
};
const trans:Transaction = contract.createTransaction('createMyPrivateAsset');
const buffer: Buffer = await trans.setTransient(transientData).submit(assetId);
This works fine in the app.
Here is the code for createMyPrivateAssetMultiple
let transientData = {
'privateValue': Buffer.from(`Private value for asset ${assetId}`)
};
const trans:Transaction = contract.createTransaction('createMyPrivateAssetMultiple');
const buffer: Buffer = await trans.setTransient(transientData).submit(assetId);
For this transaction, the app throws this (using Org1 gateway):
2022-06-07T13:21:50.727Z - warn: [TransactionEventHandler]: strategyFail: commit failure for transaction "4e9921b590a361ae01bba673e1d3d204d106522780c820055cec0345e1e67e6f": TransactionError: Commit of transaction 4e9921b590a361ae01bba673e1d3d204d106522780c820055cec0345e1e67e6f failed on peer org1peer-api.127-0-0-1.nip.io:8084 with status ENDORSEMENT_POLICY_FAILURE
The microfab docker container log includes this:
> WARN 0ff Failed fetching private data from remote peers for dig2src:[map[{b16526e4cd2ac3f431103cda23a6f64adc12acab0550eff18c1f25f1cc0d8bc1 private-contract _implicit_org_Org2MSP 6 0}:[]]], err: Empty membership channel=mychannel
...
[ org2peer] 2022-06-07 13:22:50.794 UTC [gossip.privdata] RetrievePvtdata -> WARN 220 Could not fetch all 1 eligible collection private write sets for block [20] (0 from local cache, 0 from transient store, 0 from other peers). Will commit block with missing private write sets:[txID: 4e9921b590a361ae01bba673e1d3d204d106522780c820055cec0345e1e67e6f, seq: 0, namespace: private-contract, collection: _implicit_org_Org2MSP, hash: 3e0f263d2edcfaf29df346504a40fdbadce0807938f204fe3e6bf753b751d9a3
Also, package.json includes this:
"dependencies": {
"fabric-network": "~2.1.0"
},
Can anyone shed light on this problem?

Answer 1

You should definitely be using fabric-network#2.2.x, not 2.1.x.
I suspect what is happening is the VS Code client is not using service discovery and sending proposals for endorsement to all network peers, whereas the standalone client application is using service discovery by default and only sending proposals to orgs required by the chaincode endorsement policy. To get it to consider the collection endorsement policy you would need to add a chaincode interest to the Contract object before submitting transactions:
https://hyperledger.github.io/fabric-sdk-node/release-2.2/module-fabric-network.Contract.html#addDiscoveryInterest
There is a working example of this in the Fabric samples:
https://github.com/hyperledger/fabric-samples/blob/8ca50df4ffec311e59451c2a7ebe210d9e6f0004/asset-transfer-private-data/application-javascript/app.js#L166-L178
Alternatively you could either:
Disable service discovery in the Gateway connection options.
Explicitly set the endorsing orgs for a given transaction invocation:
https://hyperledger.github.io/fabric-sdk-node/release-2.2/module-fabric-network.Transaction.html#setEndorsingOrganizations
In general it's much better to be using service discovery so I would not recommend option 1.
The best approach would actually be to use Fabric v2.4+ and the Fabric Gateway client API:
https://hyperledger.github.io/fabric-gateway/
With this API the client (generally) does not need to worry about the organizations required for endorsement when using private data collections or state-/key-based endorsement policies. Things just work automagically.