I am having trouble with my lambda function. This is the current setup:
Lambda makes post request to API. API fetches data from postgres database and returns this data. When I use Postman or a local version of my lambda function, this works. When I use the actual lambda function, the API returns data with null.
Below are some code snippets:
lambda:
const axios = require('axios')
exports.handler = async (event) => {
let rows = await axios.post('http:/server/getData', {
params: {
var: event.var
}
})
if(rows.data.statusCode == 204){
//no data available
}
else{
data = rows.data.info
}
};
API Router Component
var router = express.Router()
const Pool = require('pg').Pool
const mgmtdb = new Pool({ ... })
router.post('/getData', function(req, res){
database.query("SELECT info FROM table WHERE var=$1", [req.body.var], (error, results) => {
const rows = results.rows
if (error) {
throw error
}
let status = rows.length == 0 ? 204 : 200
var responseJSON ={};
responseJSON.statusCode = status;
responseJSON.info= rows[0] ? rows[0].info : null;
res.json(responseJSON);
})
})
module.exports = router
When I call the API from Postman I get statusCode: 200 (data available).
If I call the API with the exact same data from lambda I get statusCode: 204 (no data available).
I believe that this is some async timing problem. I don't know how the responses from the API can differ ..
Is it possible that the API streams the response back to the originator for some time not just an impulse? And starts by streaming "no data available" and then, after a few milliseconds "oh, i found some data, here it is"? And that Postman waits for the stream to finish and the lambda function doesn't?
Thanks in advance!
Looks like your query is returning null because the req.body.var doesn't exist. Can you try changing your lambda function to this:
const axios = require('axios')
exports.handler = async (event) => {
let rows = await axios.post('http:/server/getData', {
var: event.var
})
if(rows.data.statusCode == 204){
//no data available
}
else{
data = rows.data.info
}
};
This basically removes the extra level params and makes the req.body.var work.
Sometimes lambda have issue with async await so i suggest you to write code like this
exports.handler = async (event, callback) => {
axios.post('http:/server/getData', {
var: event.var
})
.then((response) => {
callback(null, {
statusCode: 200,
body: JSON.stringify(response)
})
})
};
Related
I have a GET endpoint, which basically makes some API calls to the Spoonacular API. Essentially, I make two API calls within the endpoint.
The first API call gets the list of recipe ID's for the specific ingredients
The second API calls gets the metadata for each of the recipe ID's.
After the first API call I store all the Id's in an array (recipeArray), and I want to make the second api call for each ID in my array (function recipeTest does this).
When I try to do this and then return my response to the front end, it always returns a response before completing all the API calls in the second step.
Here, is my code. The first API calls works just fine, but the second API call (recipeTest function), is where it messed up. Before that function finishes making all the API calls to the Spoonacular API, my endpoint returns an empty Array (res.send(toSend)). So, I was just wondering if there is any way around this?
Thank you so much in advance, I really appreciate it!
module.exports = (app) => {
app.get('/api/search', async (req, res) => {
console.log("endpoint working");
let ingredientList = "apples,+eggs,+bacon"; // needs to be given from the front end
let ingredientSearchUrl = `https://api.spoonacular.com/recipes/findByIngredients?ingredients=${ingredientList}&number=1&ignorePantry=true&apiKey=${keys.spoonacularKey}`;
try {
const ingredientSearchResult = await axios({
method: 'get',
url: ingredientSearchUrl
});
var recipeArray = ingredientSearchResult.data.map(info => {
return info.id;
});
} catch (err) {
console.log("error in finding recipe ID ", err);
}
let toSend = [];
try {
const check = await recipeTest(recipeArray, toSend);
} catch (err) {
console.log("error in finding recipe information ", err);
}
res.send(toSend);
});
}
const recipeTest = async (recipeArray, toSend) => {
return Promise.all(
_.forEach(recipeArray, async (recipeId) => {
let recipeInfoUrl = `https://api.spoonacular.com/recipes/${recipeId}/information?includeNutrition=false&apiKey=${keys.spoonacularKey}`;
let recipeInfo = {};
const recipeData = await axios({
method: 'get',
url: recipeInfoUrl
});
// console.log("recipeInfo search working", recipeData.data);
recipeInfo['id'] = recipeData.data.id;
recipeInfo['title'] = recipeData.data.title;
recipeInfo['time'] = recipeData.data.readyInMinutes;
recipeInfo['recipeUrl'] = recipeData.data.sourceUrl;
recipeInfo['imageUrl'] = recipeData.data.image;
// console.log('recipe info dict', recipeInfo);
toSend.push(recipeInfo);
console.log('toSend inside', toSend);
})
);
}
_.forEach return collection itself and not all your async handlers.
Use recipeArray.map to get an array of async functions to let Promise.all do its work:
Promise.all(
recipeArray.map(x => async (recipeId) => {
I am wondering how to properly test Azure Functions with Jest. I have read the online documentation provided by MSoft but it's very vague, and brief. There are also some outdated articles I found that don't really explain much. Here is what I understand: I understand how to test normal JS async functions with Jest. And I understand how to test very simple Azure Functions. However I am not sure how to go about properly testing more complex Azure Functions that make multiple API calls, etc.
For example I have an HTTP Function that is supposed to make a few API calls and mutate the data and then return the output. How do I properly mock the API calls in the test? We only have one point of entry for the function. (Meaning one function that is exported module.exports = async function(context,req). So all of our tests enter through there. If I have sub functions making calls I can't access them from the test. So is there some clever way of mocking the API calls? (since actually calling API's during tests is bad practice/design)
Here is a sample of code to show what I mean
module.exports = async function (context, req)
{
let response = {}
if (req.body && req.body.id)
{
try
{
//get order details
response = await getOrder(context, req)
}
catch (err)
{
response = await catchError(context, err);
}
}
else
{
response.status = 400
response.message = 'Missing Payload'
}
//respond
context.res =
{
headers: { 'Content-Type': 'application/json' },
status: response.status,
body: response
}
};
async function getOrder(context, req)
{
//connection to db
let db = await getDb() // <- how to mock this
//retrieve resource
let item = await db.get...(id:req.body.id)... // <- and this
//return
return {'status':200, 'data':item}
}
Consider this (simplified) example.
src/index.js (Azure Function entry point):
const { getInstance } = require('./db')
module.exports = async function (context) {
// assuming we want to mock getInstance and db.getOrder
const db = await getInstance()
const order = await db.getOrder()
return order
}
src/db.js:
let db
async function getInstance() {
if (db === undefined) {
// connect ...
db = new Database()
}
return db
}
class Database {
async getOrder() {
return 'result from real call'
}
}
module.exports = {
getInstance,
Database,
}
src/__tests__/index.test.js:
const handler = require('../index')
const db = require('../db')
jest.mock('../db')
describe('azure function handler', () => {
it('should call mocked getOrder', async () => {
const dbInstanceMock = new db.Database() // db.Database is already auto-mocked
dbInstanceMock.getOrder.mockResolvedValue('result from mock call')
db.getInstance.mockResolvedValue(dbInstanceMock)
const fakeAzureContext = {} // fake the context accordingly so that it triggers "getOrder" in the handler
const res = await handler(fakeAzureContext)
expect(db.getInstance).toHaveBeenCalledTimes(1)
expect(dbInstanceMock.getOrder).toHaveBeenCalledTimes(1)
expect(res).toEqual('result from mock call')
})
})
> jest --runInBand --verbose
PASS src/__tests__/index.test.js
azure function handler
✓ should call mocked getOrder (4 ms)
For a complete quickstart, you may want to check my blog post
I'm building a Slackbot that makes a call to an Express app, which then needs to 1) fetch some other data from the Slack API, and 2) insert resulting data in my database. I think I have the flow right finally using async await, but the operation is timing out because the original call from the Slackbot needs to receive a response within some fixed time I can't control. It would be fine for my purposes to ping the bot with a response immediately, and then execute the rest of the logic asynchronously. But I'm wondering the best way to set this up.
My Express route looks like:
const express = require('express');
const router = express.Router();
const knex = require('../../db/knex.js');
const slack = require('../../services/slack_helpers');
// POST api/slack/foo
router.post('/foo', async (req, res) => {
let [body, images] = await slack.grab_context(req);
knex('texts')
.insert({ body: body,
image_ids: images })
.then(text => { res.send('worked!'); }) // This sends a response back to the original Slackbot call
.catch(err => { res.send(err); })
});
module.exports = router;
And then the slack_helpers module looks like:
const { WebClient } = require('#slack/web-api');
const Slack = new WebClient(process.env.SLACKBOT_TOKEN);
async function grab_context(req) {
try {
const context = await Slack.conversations.history({ // This is the part that takes too long
channel: req.body.channel_id,
latest: req.headers['X-Slack-Request-Timestamp'],
inclusive: true,
limit: 5
});
} catch (error) {
return [error.toString(), 'error'];
}
return await parse_context(context);
};
function parse_context(context) {
var body = [];
context.messages.forEach(message => {
body.push(message.text);
});
body = body.join(' \n');
return [body, ''];
}
module.exports = {
grab_context
};
I'm still getting my head around asynchronous programming, so I may be missing something obvious. I think basically something like res.send perhaps needs to come before the grab_context call? But again, not sure the best flow here.
Update
I've also tried this pattern in the API route, but still getting a timeout:
slack.grab_context(req).then((body, images) => {
knex ...
})
Your timeout may not be coming from where you think. From what I see, it is coming from grab_context. Consider the following simplified version of grab_context
async function grab_context_simple() {
try {
const context = { hello: 'world' }
} catch (error) {
return [error.toString(), 'error']
}
return context
}
grab_context_simple() /* => Promise {
<rejected> ReferenceError: context is not defined
...
} */
You are trying to return context outside of the try block where it was defined, so grab_context will reject with a ReferenceError. It's very likely that this error is being swallowed at the moment, so it would seem like it is timing out.
The fix is to move a single line in grab_context
async function grab_context(req) {
try {
const context = await Slack.conversations.history({
channel: req.body.channel_id,
latest: req.headers['X-Slack-Request-Timestamp'],
inclusive: true,
limit: 5
});
return await parse_context(context); // <- moved this
} catch (error) {
return [error.toString(), 'error'];
}
};
I'm wondering the best way to set this up.
You could add a higher level try/catch block to handle errors that arise from the /foo route. You could also improve readability by staying consistent between async/await and promise chains. Below is how you could use async/await with knex, as well as the aforementioned try/catch block
const express = require('express');
const router = express.Router();
const knex = require('../../db/knex.js');
const slack = require('../../services/slack_helpers');
const insertInto = table => payload => knex(table).insert(payload)
const onFooRequest = async (req, res) => {
try {
let [body, images] = await slack.grab_context(req);
const text = await insertInto('texts')({
body: body,
image_ids: images,
});
res.send('worked!');
} catch (err) {
res.send(err);
}
}
router.post('/foo', onFooRequest);
module.exports = router;
I'm trying to create some cloud functions to:
call Google Directions API using Axios
create a document at Firestore based on the API result
Return the document reference to my iOS App.
(I'm on Blaze plan, pay as you go)
I'm having trouble to create the following functions as my Node / JS knowledge is very basic.
Could someone please have a quick a look and let me know what I'm missing?
Obs.:
The code is deploying to firebase with no warnings and erros. I'm pretty sure that the problem is the way that I'm trying to return my callbacks.
Thanks in advance
EDIT
I've made a few changes on the code, but I'm still receiving nil on my iOS App.
The code is still not creating a document on firestore.
const functions = require('firebase-functions');
const axios = require('axios');
var admin = require("firebase-admin");
admin.initializeApp();
// Func called by iOS App, If user is auth, call google maps api and use response to create a document at firestore
exports.getDistanceAndSavePackage = functions.https.onCall((data, context) => {
if (!context.auth){ return {status: 'error', code: 401, message: 'Not signed in'} }
const userId = context.auth.uid;
const startCoordinates = data.startCoords;
const endCoordinates = data.endCoords;
const pkgDocReference = getGoogleRoute(startCoordinates, endCoordinates, res => {
console.log('google cloud function has returned');
let venueId = userId;
let distance = res.distance.value;
let resultStartAdd = res.start_address;
let resultEndAdd = res.end_address;
const pkgDocRef = createTempPackage(venueId, distance, resultStartAdd, resultEndAdd, resultPkg => {
return resultPkg
})
return pkgDocRef;
})
return pkgDocReference;
});
//Create Package Document
function createTempPackage(venueId, distance, startingAddress, endingAddress, callback){
console.log('Creating temp package');
const docRef = admin.firestore().doc(`/temp_packages/`)
docRef.set({
id: docRef.id,
venue_id: venueId,
distance: distance,
starting_address: startingAddress,
ending_address: endingAddress,
timestamp: admin.database.ServerValue.TIMESTAMP,
status: 0
})
.then(docRef => {
console.log('Doc created')
return callback(docRef);
}).catch(error => {
console.log('Error trying to create document')
return callback(error);
})
}
//Call Google directions API
function getGoogleRoute(startCoords, endCoords, callback){
axios({
method: 'GET',
url: 'https://maps.googleapis.com/maps/api/directions/json',
params: {
origin: startCoords,
destination: endCoords,
key: 'mykey'
},
})
.then(response => {
let legs = response.data.routes[0].legs[0];
return callback(legs);
})
.catch(error => {
console.log('Failed calling directions API');
return callback(new Error("Error getting google directions"))
})
}
I don't know if this is final solution, however there is an error in the code:
const docRef = admin.firestore().doc('/temp_packages/')
This statement should trow error:
Value for argument "documentPath" must point to a document, but was "${documentPath}". Your path does not contain an even number of components.
The error is thrown before docRef.set so it will not be taken into consideration in catch statement. I was trying to test it, but all my tries finished with this error. Maybe this error is somewhere in your logs.
I hope it will help!
For a HTTPS trigger you'd need to return {status: 'OK', code: 200, data: json}.
So that it would actually respond through the web-server.
I'm using the IBM Watson Tone Analyser API with Express.js and React. I have this code which sends some test to the Watson API:
// tone-analyser.js
class ToneAnalysis {
constructor() {
const params = {
username: process.env.USERNAME,
password: process.env.PASSWORD,
version_date: '2018-01-31'
}
this.Analyzer = new ToneAnalyzerV3(params);
}
ToneAnalyser(input) {
let tones = this.Analyzer.tone(input, (err, tone) => {
if (err) console.log(err.message)
let voiceTone = tone.document_tone.tones[0].tone_id;
console.log(voiceTone) // Logs the right value on Node.js console
return voiceTone;
});
return tones;
}
}
module.exports = ToneAnalysis;
I then use this on my Express backend like so:
// server.js
const ToneAnalysis = require('./api/tone-analyser');
const app = express();
const input = {
tone_input: 'I am happy',
content_type: 'text/plain'
}
app.get('/api/tone', (req, res) => {
let tone = new ToneAnalysis().ToneAnalyser(input);
return res.send({
tone: tone
});
});
And I make an API call from React here:
// App.js
componentDidMount() {
this.callApi()
.then(res => {
console.log(res.tone); // Logs the wrong value on Chrome console
})
.catch(err => console.log(err));
}
callApi = async () => {
const response = await fetch('/api/tone');
const body = await response.json();
if (response.status !== 200) throw new Error(body.message);
console.log(body);
return body;
};
I expect the value of res.tone to be a string showing the tone gotten from the tone analysis function (new ToneAnalysis().ToneAnalyser(input);). Instead, I get
{
uri: {...},method: "POST", headers: {...}}
headers: {...},
uri: {...},
__proto__: Object
}
I think this happens because the res.send(...) runs before tone has a value from the API. My question is, how do I make res.send(...) run only after tone has a value?
I tried wrapping the callback function in this.Analyzer.tone(input, [callback]) in an async/await block, but that did not fix the issue. Any ideas on how to fix this will be highly appreciated. Thanks!
If the call to
let tone = new ToneAnalysis().ToneAnalyser(input);
returns a promise then you could do something like
tone.then(res.send.bind(res))
If the call to
let tone = new ToneAnalysis()`enter code here`.ToneAnalyser(input);
returns a promise then you could do something like
tone.then(res.send.bind(res))