ElasticSearch-js { body } is undefined - node.js

Question

I am getting this error when running a AWS lambda function to push data into an Elasticsearch instance
I can get it to run if I manually remove the { body } from the node modules, but I can't find why it keeps erroring on that.
my code
client.helpers.bulk({
datasource: docs,
onDocument(doc) {
return {
index: { _index: index , _id: doc.id },
body: doc.body
}
},
onDrop(doc) {
console.log("failed to index ", doc.key);
},
retries: 5,
flushBytes: 1000000,
wait: 10000
})
error
{
"errorType": "Runtime.UnhandledPromiseRejection",
"errorMessage": "TypeError: Cannot destructure property 'body' of 'undefined' as it is undefined.",
"reason": {
"errorType": "TypeError",
"errorMessage": "Cannot destructure property 'body' of 'undefined' as it is undefined.",
"stack": [
"TypeError: Cannot destructure property 'body' of 'undefined' as it is undefined.",
" at /var/task/node_modules/#elastic/elasticsearch/lib/Helpers.js:679:81"
]
},
"promise": {},
"stack": [
"Runtime.UnhandledPromiseRejection: TypeError: Cannot destructure property 'body' of 'undefined' as it is undefined.",
" at process.<anonymous> (/var/runtime/index.js:35:15)",
" at process.emit (events.js:314:20)",
" at process.EventEmitter.emit (domain.js:483:12)",
" at processPromiseRejections (internal/process/promises.js:209:33)",
" at processTicksAndRejections (internal/process/task_queues.js:98:32)"
]
}

Answer 1

I was also getting this error when referencing #opensearch-project/opensearch (which is an elasticsearch-js client fork) and using the client.helpers.bulk helper. That was in conjunction with aws-elasticsearch-connector for implementing AWS SigV4 signed API requests.
The error message was as follows:
TypeError: Cannot destructure property 'body' of 'undefined' as it is undefined.
at node_modules/#opensearch-project/opensearch/lib/Helpers.js:704:93
It was quite annoying and I was not in the mood for implementing my own OpenSearch client and interacting with the APIs directly, so I dig deeper and found the issue.
How can one reproduce the bug?
I created an isolated test to illustrate the problem. Hopefully it's easily reproducible this way.
import { Client } from '#opensearch-project/opensearch';
import * as AWS from 'aws-sdk';
// My fork of https://www.npmjs.com/package/aws-elasticsearch-connector capable of signing requests to AWS OpenSearch
// #opensearch-project/opensearch is not yet capable of signing AWS requests
const createAwsElasticsearchConnector = require('../modules/aws-oss-connector');
const domain =
'PUT_YOUR_DOMAIN_URL_HERE.es.amazonaws.com';
const index = 'YOUR_TEST_INDEX_NAME';
const bootstrapOSSClient = (): Client => {
const ossConnectorConfig = createAwsElasticsearchConnector(AWS.config);
const client = new Client({
...ossConnectorConfig,
node: `https://${domain}`,
});
return client;
};
const main = async (): Promise<void> => {
try {
console.info('Starting processing');
// TEST DEFINITION
const input = [
{ id: '1', name: 'test' },
{ id: '2', name: 'test 2' },
];
const client = bootstrapOSSClient();
const response = await client.helpers.bulk({
datasource: input,
onDocument(doc: any) {
console.info(`Processing document #${doc.id}`);
return {
index: { _index: index, _id: doc.id },
};
},
});
console.info(`Indexed ${response.successful} documents`);
// END TEST DEFINITION
console.info('Finished processing');
} catch (error) {
console.warn(`Error in main(): ${error}`);
}
};
try {
main().then(() => {
console.info('Exited main()');
});
} catch (error) {
console.warn(`Top-level error: ${error}`);
}
and the result was
$ npx ts-node ./.vscode/test.ts
Starting processing
Processing document #1
Processing document #2
(node:39232) UnhandledPromiseRejectionWarning: TypeError: Cannot destructure property 'body' of 'undefined' as it is undefined.
at D:\Development\eSUB\Coronado\git\platform\node_modules\#opensearch-project\opensearch\lib\Helpers.js:704:93
(Use `node --trace-warnings ...` to show where the warning was created)
(node:39232) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:39232) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Indexed 2 documents
Finished processing
Exited main()
Stepping through the code I am able to intercept a single call to node_modules/#opensearch-project/opensearch/lib/Helpers.js:704:93 where
client.bulk() is called
which calls bulkApi() in \opensearch\api\api\bulk.js and returns successfully
await finish() is called in \opensearch\lib\Helpers.js:559
inside \opensearch\lib\Transport.js a call to prepareRequest() is made, ending with return transportReturn
this ends up in request():177 calling
return p.then(onFulfilled, onRejected)
with p being null at that time. That resulted my callback in the AWS Transport class responsible for signing requests to callback to the Helper.js tryBulk() with second parameter undefined, resulting in the Cannot destructure property 'body' error.
What is the expected behavior?
Transport.js implementation of the request should obviously not result in a null p promise issue when callback is passed in the request() call. I logged a bug in the opensearch-js repository.
Workaround
At least for me, this looks to be a problem only when using custom AWS signed requests connector implementation. If your case is similar, a quick workaround involves modifying that implementation Transport class. Here is a quick and dirty hotfix that's specific to aws-elasticsearch-connector.
You need to modify AmazonTransport.js from
class AmazonTransport extends Transport {
request (params, options = {}, callback = undefined) {
...
// Callback support
awaitAwsCredentials(awsConfig)
.then(() => super.request(params, options, callback))
.catch(callback)
}
to
// Callback support
// Removed .then() chain due to a bug https://github.com/opensearch-project/opensearch-js/issues/185
// .then() was calling then (onFulfilled, onRejected) on transportReturn, resulting in a null value exception
awaitAwsCredentials(awsConfig).then();
try {
super.request(params, options, callback);
} catch (err) {
callback(err, { body: null });
}