Timeout expired while querying large data on Hyperledger Fabric 1.4.1 - hyperledger-fabric

Question

I am using Hyperledger Fabric 1.4.1, with Kafka ordering service, Couch DB as stateDB, Java chaincode and Java SDK.
I experience timeout expired when I try to send large data to the channel and then query it back.
The data size of the transaction I sent is 512KB, and I can query it from peers.
I tried sending it 7 times with the same id, and I can get all 7 histories with shim function getHistoryForKey. But when I sent it 8 times with the same id, getHistoryForKey failed.
The error message is shown below (from peers):
Error: endorsement failure during query. response: status:500 message:"failed to execute transaction fdfbc5b72efd688136c777f5b114a3cedc62339e42f6c39a27e86ca11b8e3d14: error sending: timeout expired while executing transaction"
I have tried to expand the timeout with following options and their combinations in peer and client:
CORE_CHAINCODE_EXECUTETIMEOUT="120s"
CORE_LEDGER_STATE_COUCHDBCONFIG_REQUESTTIMEOUT="120s"
# (seems this setting is not for Hyperledger Fabric)
CORE_PEER_GRPCOPTIONS_REQUESET_TIMEOUT="120s"
But they didn't help in my case.
I also noticed that the 7 copies of testing data are about 3.5MB, and I tried to query 4MB data (which is about 8 copies) from a peer but failed.
When we check the chaincode runtime docker, we see the following error message:
SEVERE: An error occured on the chaincode stream. Shutting down the chaincode stream.
io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED:
io.grpc.netty.NettyClientTransport$3: Frame size 6294852 exceeds maximum: 4194304.
It seems like there is a grpc limitation on running the java chaincode docker.
We tested with go chaincode, we do not experience the same issue.

Answer 1

We managed to fix that by overriding the ChaincodeBase class
ManagedChannelBuilder<?> newChannelBuilder() throws IOException {
final NettyChannelBuilder builder = NettyChannelBuilder.forAddress(host, port);
logger.info("Configuring channel connection to peer.");
builder.maxInboundMessageSize(104857600);
logger.info("maxInboundMessageSize is 104857600.");
if (tlsEnabled) {
builder.negotiationType(NegotiationType.TLS);
builder.sslContext(createSSLContext());
} else {
builder.usePlaintext(true);
}
return builder;
}
I have submitted a ticket on fabric Jira to support changing the variable without overriding.