I am using AWS Comprehend's "PII" detection tools to spot personally identifying information (PII) in arbitrary text submitted by the public.
This is for a Nodejs Lambda that I mean to run in our AWS account.
const {Comprehend} = require('#aws-sdk/client-comprehend');
const client = new Comprehend({ region: "us-east-2" });
module.exports.handler = async (event, _context) => {
const message = event.data;
var redactedMessage = await getRedactedMessage(message);
return redactedMessage;
}
const getRedactedMessage = async (message) => {
// the Comprehend PII system takes strings up to 4000 characters
// so we split it on word boundaries after 4000 chars or less.
var messageParts = message.match(/.{1,4000}(\s|$)/g);
var redactedString = "";
for (const part in messageParts) {
try {
const checkPart = {
Text : message,
LanguageCode : "en"
};
client.detectPiiEntities(checkPart, function (err, pii) {
console.log("Why does this come last?");
});
} catch (error) {
console.error(error);
} finally {
console.log("Why do I get here first?");
}
console.log("And here? Then suddenly...");
}
}
No matter what, the output is
Why do I get here first?
And here? Then suddenly...
Why does this come last?
I have littered this with every possible combination of "async" and "await" commands. I have rewritten the client.detectPiiEntities(checkPart, function (err, pii)... line like a promise with a .then() but no matter what, I can't make it wait for the results of client.detectPiiEntities block.
I'm sure it's something dumb. (Probably me.) Thanks in advance.
Related
I'm scanning all items from a DynamoDB table - within a Lambda function - with DocumentClient. I'm then looping through each item and extracting the payload that I need. I'll use that item from the payload as a parameter with ExpressionAttributeValues in a new query.
Everything works dandy independently. The issue is with the use of the asynchronous function queryItems when nested within an array forEach() method. I getting a parsing error with the function queryItems. I can query the table when I call the function outside of the loop but how else am I going to query each item independently?
I'm not sure how to handle this.
'use strict';
const aws = require('aws-sdk');
const docClient = new aws.DynamoDB.DocumentClient();
var paramsAll = {
TableName: 'MyTable',
Select: "ALL_ATTRIBUTES"
};
exports.handler = async (event, context) => {
try {
let arr = [];
let sequence = '';
//scan all items in table
docClient.scan(paramsAll, function(err, data) {
if (err) {
//handle error
}
else {
//Loop through each item in the table:
let items = (data.Items);
items.forEach(function(Item) {
let p = (Item.payload);
//Extract sequence from the payload
sequence = (p.seq);
arr.push(sequence);
//perform other function with this array (not listed for brevity)
});
//Here is where I'm having the issue:
arr.forEach(function(Item) {
//Pass these items as a paramater within queryItems function but getting Parsing Error: unexpected token queryItems
const results = await queryItems(Item);
//do something with the results...
})
}
});
}
catch (err) {
return { error: err };
}
};
async function queryItems(p) {
try {
var params = {
TableName: 'MyTable',
KeyConditionExpression: '#seq = :value',
ExpressionAttributeValues: { ':value': p },
ExpressionAttributeNames: { '#seq': 'seq' }
};
const data = await docClient.query(params).promise();
return data;
}
catch (err) {
return err;
}
}
I've definitely run into a similar issue. What I believe is happening is just a Javascript syntax issue, where awaiting queryItems inside the synchronous function provided to forEach will produce an error. (Although, when running the code, I do get the specific error "SyntaxError: await is only valid in async functions and the top level bodies of modules", so there might be something else going on.)
I see nothing wrong with the DynamoDB queries, but hoangdv's suggestions are spot on. Specifically, I'd also suggest using the promise style for scan, and while a for...loop will definitely work, using Promise.all and map will be a lot quicker to complete all the queries. Here's how I'd modify the code:
'use strict';
const aws = require('aws-sdk');
const docClient = new aws.DynamoDB.DocumentClient();
// avoid var unless you specifically require it's hoisting behavior.
const paramsAll = {
TableName: 'MyTable',
Select: "ALL_ATTRIBUTES" // most likely not needed, I'd review this section of the docs: https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html#DDB-Scan-request-Select
};
exports.handler = async (event, context) => {
try {
// unless you need to assign a new array to this variable, it is better practice to use const instead.
const arr = [];
// let sequence = ''; // see line 24 for why I commented this out.
// scan all items in table.
// Destructure Items out of the response.
// You may also need to continue scanning with the LastEvaluatedKey depending on the size of your table, and/or your use case.
// You'd continue scanning in a while loop, for example.
const { Items, LastEvaluatedKey } = await docClient.scan(paramsAll).promise();
// push the sequence to the arr.
// There is most likely a reason you omitted for brevity to have sequence defined above,
// but since this example doesn't need it above, I've omitted it entirely
Items.forEach(Item => {
const p = Item.payload;
arr.push(p.seq);
});
// use a for loop or map here instead. forEach will return undefined, which cannot be await'ed.
// instead, map will return a new array of Promises (since the callback is async).
// Then, you can use Promise.all to await until each Promise in the array is resolved.
// Keep in mind, depending on how many items you are iterating through, you may run into DynamoDB's ThrottlingException.
// You would have to batch the queries (in other words, split the arr into pieces, and iterate over each piece), which would have to be done before using map. Then, sleep for a few milliseconds before starting on the next piece.
// I doubt the queries will be quick enough to cause this when using a for loop, though.
await Promise.all(arr.map(async Item => {
const results = await queryItems(Item);
// do something with the results...
}));
}
catch (err) {
// Again, not sure what the use case is, but just FYI this is not a valid return value if this lambda function is intended for use with using API Gateway.
// See here :) https://docs.aws.amazon.com/lambda/latest/dg/services-apigateway.html#apigateway-types-transforms
return { error: err };
}
};
// Presumably, MyTable has a partitionKey of seq, otherwise this KeyConditionExpression is invalid.
async function queryItems(p) {
try {
var params = {
TableName: 'MyTable',
KeyConditionExpression: '#seq = :value',
ExpressionAttributeValues: { ':value': p },
ExpressionAttributeNames: { '#seq': 'seq' }
};
const data = await docClient.query(params).promise();
return data;
}
catch (err) {
return err;
}
}
Your issue is how you await on the for loop, its best to use Promise.all() with a map to await inside of a loop:
await Promise.all(arr.map(async Item => {
const results = await queryItems(Item);
// do something with the results...
}));
However, I cannot seem to understand your logic really well.
You Scan a table called MyTable, but you do not paginate, meaning you are only getting up to 1MB worth of data.
With the results, you strip out the seq value and then once again read every item from MyTable this time using a Query and seq as the key?
Simple call to ec2 Describing Security groups and returning the security group ID. Using Async / await, but when logging the return value, I get undefined. I fully admit I'm coming from Python and I've tried my hardest to wrap my brain around async calls. I thought I had it nailed, but I'm obviously missing something.
'use strict';
// Load Modules
const AWS = require('aws-sdk')
//Set the region
AWS.config.update({region: 'us-west-2'});
// Call AWS Resources
const ec2 = new AWS.EC2();
// Get Security Group ID From Event
const getSgIdFromEvent = async (event) => {
var ec2params = { Filters: [{Name: 'tag:t_whitelist',Values[event['site']]}]};
await ec2.describeSecurityGroups(ec2params, function (err, response) {
if (err) {return console.error(err.message)}
else {
var sgId = response.SecurityGroups[0].GroupId;
return sgId;
};
});
};
// MAIN FUNCTION
exports.handler = (event, context) => {
getSgIdFromEvent(event)
.then(sgId => {console.log(sgId)});
}
"sgId" should return the security group ID. It does print out fine in the original function before the return.
Typically if it is an async call you want you handle it similar to this way without using a callback
// Load Modules
const AWS = require('aws-sdk')
//Set the region
AWS.config.update({ region: 'us-west-2' });
// Call AWS Resources
const ec2 = new AWS.EC2();
// Get Security Group ID From Event
const getSgIdFromEvent = async (event) => {
var ec2params = { Filters: [{ Name: 'tag:t_whitelist', Values[event['site']]}] };
try {
const securityGroupsDesc = await ec2.describeSecurityGroups(ec2params).promise();
const sgId = securityGroupsDesc.SecurityGroups[0].GroupId;
//do something with the returned result
return sgId;
}
catch (error) {
console.log('handle error');
// throw error;
}
});
};
// MAIN FUNCTION
exports.handler = (event, context) => {
getSgIdFromEvent(event)
.then(sgId => { console.log(sgId) });
}
however if it doesn't support async you just use the callback to handle the returned data or error without using async function.However Reading into AWS docs you can find that the function ec2.describeSecurityGroups() returns an AWS Request
which has a method promise() that needs to be invoked to send the request and get a promise returned.Note that the try catch here is not needed but good to have in case error occurs during the process.
As I said in the comment, chance are that describeSecurityGroups doesn't return a Promise. Try transforming it explictly in a Promise instead:
const promiseResponse = await new Promise((res, rej) => {
ec2.describeSecurityGroups(ec2params, function (err, response) {
if (err) {return rej(err.message)}
else {
var sgId = response.SecurityGroups[0].GroupId;
res(sgId);
};
})
});
// promiseResponse is now equal to sgId inside the callback
return promiseResponse; // this will work because the function is async
Note: You can drop the else keyword
Here is the code that worked using async / await. Thanks to #Cristian Traina I realized ec2.describeSecurityGroups wasn't returning a promise, it was returning an AWS.Event.
// Get Security Group ID From Event
const getSgIdFromEvent = async (event) => {
console.log('Getting Security Group ID')
var params = { Filters: [{Name: 'tag:t_whitelist', Values
[event['site']]}]};
const describeSG = await ec2.describeSecurityGroups(params).promise();
return describeSG.SecurityGroups[0].GroupId;
};
// Get Ingress Rules from Security Group
const getSgIngressRules = async (sgId) => {
console.log(`Getting SG Ingress rules for ${sgId}`)
var params = { GroupIds: [ sgId]};
try{
const ingressRules = await ec2.describeSecurityGroups(params).promise();
return ingressRules;
}
catch (error) {
console.log("Something went wrong getting Ingress Ruls");
}
};
// MAIN FUNCTION
exports.handler = (event, context) => {
getSgIdFromEvent(event)
.then(sgId => {return getSgIngressRules(sgId);})
.then(ingressRules => {console.log(ingressRules);});
}
I submitted this as the answer now since the getSgIdFromEvent function I have, is only 8 lines and still using the async/await like I was desiring.
What I was missing was the .promise() on the end of the function and returning that promise.
Thanks for all the responses!
I have a lambda function that I am playing around with. It inserts very basic information into a DynamoDB table. Here is the code:
'use strict';
const alexaSkillKit = require('alexa-skill-kit');
const AWS = require('aws-sdk');
function binDaySkill(event, context, callback) {
alexaSkillKit(event, context, (message) => {
let params = {
Item: {
user_id: '123',
some_data: 'some data here'
},
TableName: 'my_table'
};
let documentClient = new AWS.DynamoDB.DocumentClient();
documentClient.put(params, function (err, data) {
if (err) {
callback("Error", err);
} else {
callback(null, data);
}
});
});
}
The issue I am having is that it only sometimes saves the data in the DB. I have to click test 5-10 times before it does anything.
Can anyone help with what might be causing this?
The reason this is happening is because alexa-skill-kit takes care of the callback for you.
See the documentation. By passing in the context object you allow the wrapping handler (alexaSkillKit(...)) to manage decoding and encoding the returned objects and payload. The handler alexSkillKit callback function just expects you to return a value.
For your code sample you could do the following:
'use strict';
const alexaSkillKit = require('alexa-skill-kit');
const AWS = require('aws-sdk');
function binDaySkill(event, context, callback) {
alexaSkillKit(event, context, (message) => {
let params = {
Item: {
user_id: '123',
some_data: 'some data here'
},
TableName: 'my_table'
};
let documentClient = new AWS.DynamoDB.DocumentClient();
return documentClient.put(params).promise()
.then((data) => {
// stuff with the data!
return data;
}).catch((err) => {
// stuff with the error
throw err;
});
});
}
n.b. The reason it worked after a few invocations is that lambda re-uses the environments each invocation executes in. It does this by effectively "freezing" the state of the environment and thawing it when it's needed again. This is the basis of a lot of optimisations people make; and it meant that you would sometimes thaw an environment that was midway through calling back when it was frozen by the alexaSkillKit returning first.
I'm trying to update a tool that was created a while ago which uses nodejs (I am not a JS developer, so I'm trying to piece the code together) and am getting stuck at the last hurdle.
The new functionality will take in a swagger .json definition, compare the endpoints against the matching API Gateway on the AWS Service, using the 'aws-sdk' SDK for JS and then updates the Gateway accordingly.
The code runs fine on a small definition file (about 15 endpoints) but as soon as I give it a bigger one, I start getting tons of TooManyRequestsException errors.
I understand that this is due to my calls to the API Gateway service being too quick and a delay / pause is needed. This is where I am stuck
I have tried adding;
a delay() to each promise being returned
running a setTimeout() in each promise
adding a delay to the Promise.all and Promise.mapSeries
Currently my code loops through each endpoint within the definition and then adds the response of each promise to a promise array:
promises.push(getMethodResponse(resourceMethod, value, apiName, resourcePath));
Once the loop is finished I run this:
return Promise.all(promises)
.catch((err) => {
winston.error(err);
})
I have tried the same with a mapSeries (no luck).
It looks like the functions within the (getMethodResponse promise) are run immediately and hence, no matter what type of delay I add they all still just execute. My suspicious is that the I need to make (getMethodResponse) return a function and then use mapSeries but I cant get this to work either.
Code I tried:
Wrapped the getMethodResponse in this:
return function(value){}
Then added this after the loop (and within the loop - no difference):
Promise.mapSeries(function (promises) {
return 'a'();
}).then(function (results) {
console.log('result', results);
});
Also tried many other suggestions:
Here
Here
Any suggestions please?
EDIT
As request, some additional code to try pin-point the issue.
The code currently working with a small set of endpoints (within the Swagger file):
module.exports = (apiName, externalUrl) => {
return getSwaggerFromHttp(externalUrl)
.then((swagger) => {
let paths = swagger.paths;
let resourcePath = '';
let resourceMethod = '';
let promises = [];
_.each(paths, function (value, key) {
resourcePath = key;
_.each(value, function (value, key) {
resourceMethod = key;
let statusList = [];
_.each(value.responses, function (value, key) {
if (key >= 200 && key <= 204) {
statusList.push(key)
}
});
_.each(statusList, function (value, key) { //Only for 200-201 range
//Working with small set
promises.push(getMethodResponse(resourceMethod, value, apiName, resourcePath))
});
});
});
//Working with small set
return Promise.all(promises)
.catch((err) => {
winston.error(err);
})
})
.catch((err) => {
winston.error(err);
});
};
I have since tried adding this in place of the return Promise.all():
Promise.map(promises, function() {
// Promise.map awaits for returned promises as well.
console.log('X');
},{concurrency: 5})
.then(function() {
return console.log("y");
});
Results of this spits out something like this (it's the same for each endpoint, there are many):
Error: TooManyRequestsException: Too Many Requests
X
Error: TooManyRequestsException: Too Many Requests
X
Error: TooManyRequestsException: Too Many Requests
The AWS SDK is being called 3 times within each promise, the functions of which are (get initiated from the getMethodResponse() function):
apigateway.getRestApisAsync()
return apigateway.getResourcesAsync(resourceParams)
apigateway.getMethodAsync(params, function (err, data) {}
The typical AWS SDK documentation state that this is typical behaviour for when too many consecutive calls are made (too fast). I've had a similar issue in the past which was resolved by simply adding a .delay(500) into the code being called;
Something like:
return apigateway.updateModelAsync(updateModelParams)
.tap(() => logger.verbose(`Updated model ${updatedModel.name}`))
.tap(() => bar.tick())
.delay(500)
EDIT #2
I thought in the name of thorough-ness, to include my entire .js file.
'use strict';
const AWS = require('aws-sdk');
let apigateway, lambda;
const Promise = require('bluebird');
const R = require('ramda');
const logger = require('../logger');
const config = require('../config/default');
const helpers = require('../library/helpers');
const winston = require('winston');
const request = require('request');
const _ = require('lodash');
const region = 'ap-southeast-2';
const methodLib = require('../aws/methods');
const emitter = require('../library/emitter');
emitter.on('updateRegion', (region) => {
region = region;
AWS.config.update({ region: region });
apigateway = new AWS.APIGateway({ apiVersion: '2015-07-09' });
Promise.promisifyAll(apigateway);
});
function getSwaggerFromHttp(externalUrl) {
return new Promise((resolve, reject) => {
request.get({
url: externalUrl,
header: {
"content-type": "application/json"
}
}, (err, res, body) => {
if (err) {
winston.error(err);
reject(err);
}
let result = JSON.parse(body);
resolve(result);
})
});
}
/*
Deletes a method response
*/
function deleteMethodResponse(httpMethod, resourceId, restApiId, statusCode, resourcePath) {
let methodResponseParams = {
httpMethod: httpMethod,
resourceId: resourceId,
restApiId: restApiId,
statusCode: statusCode
};
return apigateway.deleteMethodResponseAsync(methodResponseParams)
.delay(1200)
.tap(() => logger.verbose(`Method response ${statusCode} deleted for path: ${resourcePath}`))
.error((e) => {
return console.log(`Error deleting Method Response ${httpMethod} not found on resource path: ${resourcePath} (resourceId: ${resourceId})`); // an error occurred
logger.error('Error: ' + e.stack)
});
}
/*
Deletes an integration response
*/
function deleteIntegrationResponse(httpMethod, resourceId, restApiId, statusCode, resourcePath) {
let methodResponseParams = {
httpMethod: httpMethod,
resourceId: resourceId,
restApiId: restApiId,
statusCode: statusCode
};
return apigateway.deleteIntegrationResponseAsync(methodResponseParams)
.delay(1200)
.tap(() => logger.verbose(`Integration response ${statusCode} deleted for path ${resourcePath}`))
.error((e) => {
return console.log(`Error deleting Integration Response ${httpMethod} not found on resource path: ${resourcePath} (resourceId: ${resourceId})`); // an error occurred
logger.error('Error: ' + e.stack)
});
}
/*
Get Resource
*/
function getMethodResponse(httpMethod, statusCode, apiName, resourcePath) {
let params = {
httpMethod: httpMethod.toUpperCase(),
resourceId: '',
restApiId: ''
}
return getResourceDetails(apiName, resourcePath)
.error((e) => {
logger.unimportant('Error: ' + e.stack)
})
.then((result) => {
//Only run the comparrison of models if the resourceId (from the url passed in) is found within the AWS Gateway
if (result) {
params.resourceId = result.resourceId
params.restApiId = result.apiId
var awsMethodResponses = [];
try {
apigateway.getMethodAsync(params, function (err, data) {
if (err) {
if (err.statusCode == 404) {
return console.log(`Method ${params.httpMethod} not found on resource path: ${resourcePath} (resourceId: ${params.resourceId})`); // an error occurred
}
console.log(err, err.stack); // an error occurred
}
else {
if (data) {
_.each(data.methodResponses, function (value, key) {
if (key >= 200 && key <= 204) {
awsMethodResponses.push(key)
}
});
awsMethodResponses = _.pull(awsMethodResponses, statusCode); //List of items not found within the Gateway - to be removed.
_.each(awsMethodResponses, function (value, key) {
if (data.methodResponses[value].responseModels) {
var existingModel = data.methodResponses[value].responseModels['application/json']; //Check if there is currently a model attached to the resource / method about to be deleted
methodLib.updateResponseAssociation(params.httpMethod, params.resourceId, params.restApiId, statusCode, existingModel); //Associate this model to the same resource / method, under the new response status
}
deleteMethodResponse(params.httpMethod, params.resourceId, params.restApiId, value, resourcePath)
.delay(1200)
.done();
deleteIntegrationResponse(params.httpMethod, params.resourceId, params.restApiId, value, resourcePath)
.delay(1200)
.done();
})
}
}
})
.catch(err => {
console.log(`Error: ${err}`);
});
}
catch (e) {
console.log(`getMethodAsync failed, Error: ${e}`);
}
}
})
};
function getResourceDetails(apiName, resourcePath) {
let resourceExpr = new RegExp(resourcePath + '$', 'i');
let result = {
apiId: '',
resourceId: '',
path: ''
}
return helpers.apiByName(apiName, AWS.config.region)
.delay(1200)
.then(apiId => {
result.apiId = apiId;
let resourceParams = {
restApiId: apiId,
limit: config.awsGetResourceLimit,
};
return apigateway.getResourcesAsync(resourceParams)
})
.then(R.prop('items'))
.filter(R.pipe(R.prop('path'), R.test(resourceExpr)))
.tap(helpers.handleNotFound('resource'))
.then(R.head)
.then([R.prop('path'), R.prop('id')])
.then(returnedObj => {
if (returnedObj.id) {
result.path = returnedObj.path;
result.resourceId = returnedObj.id;
logger.unimportant(`ApiId: ${result.apiId} | ResourceId: ${result.resourceId} | Path: ${result.path}`);
return result;
}
})
.catch(err => {
console.log(`Error: ${err} on API: ${apiName} Resource: ${resourcePath}`);
});
};
function delay(t) {
return new Promise(function(resolve) {
setTimeout(resolve, t)
});
}
module.exports = (apiName, externalUrl) => {
return getSwaggerFromHttp(externalUrl)
.then((swagger) => {
let paths = swagger.paths;
let resourcePath = '';
let resourceMethod = '';
let promises = [];
_.each(paths, function (value, key) {
resourcePath = key;
_.each(value, function (value, key) {
resourceMethod = key;
let statusList = [];
_.each(value.responses, function (value, key) {
if (key >= 200 && key <= 204) {
statusList.push(key)
}
});
_.each(statusList, function (value, key) { //Only for 200-201 range
promises.push(getMethodResponse(resourceMethod, value, apiName, resourcePath))
});
});
});
//Working with small set
return Promise.all(promises)
.catch((err) => {
winston.error(err);
})
})
.catch((err) => {
winston.error(err);
});
};
You apparently have a misunderstanding about what Promise.all() and Promise.map() do.
All Promise.all() does is keep track of a whole array of promises to tell you when the async operations they represent are all done (or one returns an error). When you pass it an array of promises (as you are doing), ALL those async operations have already been started in parallel. So, if you're trying to limit how many async operations are in flight at the same time, it's already too late at that point. So, Promise.all() by itself won't help you control how many are running at once in any way.
I've also noticed since, that it seems this line promises.push(getMethodResponse(resourceMethod, value, apiName, resourcePath)) is actually executing promises and not simply adding them to the array. Seems like the last Promise.all() doesn't actually do much.
Yep, when you execute promises.push(getMethodResponse()), you are calling getMethodResponse() immediately right then. That starts the async operation immediately. That function then returns a promise and Promise.all() will monitor that promise (along with all the other ones you put in the array) to tell you when they are all done. That's all Promise.all() does. It monitors operations you've already started. To keep the max number of requests in flight at the same time below some threshold, you have to NOT START the async operations all at once like you are doing. Promise.all() does not do that for you.
For Bluebird's Promise.map() to help you at all, you have to pass it an array of DATA, not promises. When you pass it an array of promises that represent async operations that you've already started, it can do no more than Promise.all() can do. But, if you pass it an array of data and a callback function that can then initiate an async operation for each element of data in the array, THEN it can help you when you use the concurrency option.
Your code is pretty complex so I will illustrate with a simple web scraper that wants to read a large list of URLs, but for memory considerations, only process 20 at a time.
const rp = require('request-promise');
let urls = [...]; // large array of URLs to process
Promise.map(urls, function(url) {
return rp(url).then(function(data) {
// process scraped data here
return someValue;
});
}, {concurrency: 20}).then(function(results) {
// process array of results here
}).catch(function(err) {
// error here
});
In this example, hopefully you can see that an array of data items are being passed into Promise.map() (not an array of promises). This, then allows Promise.map() to manage how/when the array is processed and, in this case, it will use the concurrency: 20 setting to make sure that no more than 20 requests are in flight at the same time.
Your effort to use Promise.map() was passing an array of promises, which does not help you since the promises represent async operations that have already been started:
Promise.map(promises, function() {
...
});
Then, in addition, you really need to figure out what exactly causes the TooManyRequestsException error by either reading documentation on the target API that exhibits this or by doing a whole bunch of testing because there can be a variety of things that might cause this and without knowing exactly what you need to control, it just takes a lot of wild guesses to try to figure out what might work. The most common things that an API might detect are:
Simultaneous requests from the same account or source.
Requests per unit of time from the same account or source (such as request per second).
The concurrency operation in Promise.map() will easily help you with the first option, but will not necessarily help you with the second option as you can limit to a low number of simultaneous requests and still exceed a requests per second limit. The second needs some actual time control. Inserting delay() statements will sometimes work, but even that is not a very direct method of managing it and will either lead to inconsistent control (something that works sometimes, but not other times) or sub-optimal control (limiting yourself to something far below what you can actually use).
To manage to a request per second limit, you need some actual time control with a rate limiting library or actual rate limiting logic in your own code.
Here's an example of a scheme for limiting the number of requests per second you are making: How to Manage Requests to Stay Below Rate Limiting.
I am currently developing a bot using wit.ai. I am quite new to node.js. Basically, I am following the guide provided by node-wit lib. I construct my wit object by:
const wit = new Wit({
accessToken: WIT_TOKEN,
actions,
logger: new log.Logger(log.INFO)
});
In my actions, I have something like:
const actions = {
send({sessionId}, {text}) {
//my sending action goes here.
},
firstaction({context, entities,sessionId}) {
var data = async_function();
context.receiver = data;
return context;
}
}
The issue is that whatever comes after async_function will be executed first. I tried to let async_function return a promise. However, this wouldn't work since whatever comes after my first action in node-wit library will be executed first without waiting for the context to return. I don't want to modify the node-wit library.
Any idea that would solve my issue is appreciated!
you can use async library for asynchronous call
https://caolan.github.io/async/docs.html
const async = require('async')
const actions = {
send({sessionId}, {text}) {
//my sending action goes here.
},
firstaction({context, entities,sessionId}) {
async.waterfall([function(callback) {
var d = async_function();
// if err pass it to callback first parameter
// return callback(err)
callback(null,d);
}], function(err, result) {
if(err) {
return err;
}
var data = result;
context.receiver = data;
return context;
})
}
}