I'm trying to create some cloud functions to:
call Google Directions API using Axios
create a document at Firestore based on the API result
Return the document reference to my iOS App.
(I'm on Blaze plan, pay as you go)
I'm having trouble to create the following functions as my Node / JS knowledge is very basic.
Could someone please have a quick a look and let me know what I'm missing?
Obs.:
The code is deploying to firebase with no warnings and erros. I'm pretty sure that the problem is the way that I'm trying to return my callbacks.
Thanks in advance
EDIT
I've made a few changes on the code, but I'm still receiving nil on my iOS App.
The code is still not creating a document on firestore.
const functions = require('firebase-functions');
const axios = require('axios');
var admin = require("firebase-admin");
admin.initializeApp();
// Func called by iOS App, If user is auth, call google maps api and use response to create a document at firestore
exports.getDistanceAndSavePackage = functions.https.onCall((data, context) => {
if (!context.auth){ return {status: 'error', code: 401, message: 'Not signed in'} }
const userId = context.auth.uid;
const startCoordinates = data.startCoords;
const endCoordinates = data.endCoords;
const pkgDocReference = getGoogleRoute(startCoordinates, endCoordinates, res => {
console.log('google cloud function has returned');
let venueId = userId;
let distance = res.distance.value;
let resultStartAdd = res.start_address;
let resultEndAdd = res.end_address;
const pkgDocRef = createTempPackage(venueId, distance, resultStartAdd, resultEndAdd, resultPkg => {
return resultPkg
})
return pkgDocRef;
})
return pkgDocReference;
});
//Create Package Document
function createTempPackage(venueId, distance, startingAddress, endingAddress, callback){
console.log('Creating temp package');
const docRef = admin.firestore().doc(`/temp_packages/`)
docRef.set({
id: docRef.id,
venue_id: venueId,
distance: distance,
starting_address: startingAddress,
ending_address: endingAddress,
timestamp: admin.database.ServerValue.TIMESTAMP,
status: 0
})
.then(docRef => {
console.log('Doc created')
return callback(docRef);
}).catch(error => {
console.log('Error trying to create document')
return callback(error);
})
}
//Call Google directions API
function getGoogleRoute(startCoords, endCoords, callback){
axios({
method: 'GET',
url: 'https://maps.googleapis.com/maps/api/directions/json',
params: {
origin: startCoords,
destination: endCoords,
key: 'mykey'
},
})
.then(response => {
let legs = response.data.routes[0].legs[0];
return callback(legs);
})
.catch(error => {
console.log('Failed calling directions API');
return callback(new Error("Error getting google directions"))
})
}
I don't know if this is final solution, however there is an error in the code:
const docRef = admin.firestore().doc('/temp_packages/')
This statement should trow error:
Value for argument "documentPath" must point to a document, but was "${documentPath}". Your path does not contain an even number of components.
The error is thrown before docRef.set so it will not be taken into consideration in catch statement. I was trying to test it, but all my tries finished with this error. Maybe this error is somewhere in your logs.
I hope it will help!
For a HTTPS trigger you'd need to return {status: 'OK', code: 200, data: json}.
So that it would actually respond through the web-server.
Related
Here is what I want to achieve : I want to get a JSON on a daily basis from a URL and convert it to a cloud firestore collection in order to be able to use it in my Flutter app. Ideally, the script would only add new data to the collection.
I saw that I can use scheduler from Firebase cloud functions to run tasks daily. That's not the problem for now.
However, I don't know how to use Firebase cloud functions properly to get data from URL and convert it to collection. Maybe that's not the point of cloud functions and I misunderstood something. So first question : Can I run classic nodeJS stuff inside cloud functions? I suppose I can
Next, I initialized a cloud function project locally, connected it to my Google account and started to write code into index.js.
const functions = require("firebase-functions");
const admin = require('firebase-admin');
const fetch = require('node-fetch');
const db = admin.firestore();
const collectionToiletRef = db.collection('mycollection');
let settings = { method: "Get" };
let url = "my-url.com"
fetch(url, settings)
.then(res => res.json())
.then((json) => {
print(json);
// TODO for each json object, add new document
});
Second question : How can I run this code to see if it works ? I saw that emulator can be used but how can I check visually my cloud firestore collection ? On this simple example, I only want to print my json to see if I can get the data correctly. Where would the printing be done ?
Maybe cloud functions is not what I need for this task. Maybe my code is bad. I don't know. Thanks for your help.
EDIT
I tried this but the call never ends. I think it's waiting for a promise that never returns or something like that.
const functions = require("firebase-functions");
const admin = require('firebase-admin');
const fetch = require('node-fetch');
admin.initializeApp();
const db = admin.firestore();
exports.tempoCF = functions
.firestore.document('/tempo/{docId}')
.onCreate(async (snap, context) => {
console.log("onCreate");
let settings = { method: "Get" };
let url = "https://opendata.paris.fr/api/records/1.0/search/?dataset=sanisettesparis&q=&rows=-1"
try {
let response = await fetch(url, settings);
let json = await response.json();
// TODO for each json object, add new document
await Promise.all(json["records"].map(toiletJsonObject => {
return db.collection('toilets').doc(toiletJsonObject["recordid"]).set({}); // Only to create documents, I will deal with the content later
}));
}
catch(error) {
console.log(error);
return null;
}
}
);
This code works and create all the documents I want but never return. However, the async (snap, context) => {} passed to onCreate is a Promise. And this promise ends when Promise.all ends. I'm missing something but I don't know why. I'm struggling a lot with async programming with Dart or JS. Not very clear in my mind.
Can I run classic nodeJS stuff inside cloud functions?
Sure! Since the fetch method returns a Promise you can very well use it in a background triggered or a scheduled Cloud Function.
How can I run this code to see if it works?
Your code will work perfectly in the emulator suite, but you will need to trigger the Cloud Function with one of the Firebase services that can run in the emulator. For example you can trigger the Cloud Function by creating a document in the Firestore emulator console.
The following Cloud Function will do the trick: just create a doc in a dummy tempo collection and the CF will add a new doc in a newDocscollection. It's up to you to adapt the fields values for this doc, I've just used the entire JSON object.
exports.tempoCF = functions
.firestore.document('/tempo/{docId}')
.onCreate((snap, context) => {
let settings = { method: "Get" };
let url = "https://..."
return fetch(url, settings)
.then(res => res.json())
.then((json) => {
console.log(json);
// TODO for each json object, add new document
return admin.firestore().collection('newDocs').add(json);
})
.catch(error => {
console.log(error);
return null;
});
});
You could also deploy your Cloud Function to the Firebase backend, and if you want to schedule it, just change the code as follows (change the trigger):
exports.scheduledFunction = functions.pubsub.schedule('every 5 minutes').onRun((context) => {
let settings = { method: "Get" };
let url = "https://..."
return fetch(url, settings)
.then(res => res.json())
.then((json) => {
console.log(json);
// TODO for each json object, add new document
return admin.firestore().collection('newDocs').add(json);
})
.catch(error => {
console.log(error);
return null;
});
});
Edit following your edit:
The following code does work correctly in the emulator, creating docs in the toilets collection.
exports.tempoCF = functions.firestore
.document('/tempo/{docId}')
.onCreate(async (snap, context) => {
console.log('onCreate');
let settings = { method: 'Get' };
let url =
'https://opendata.paris.fr/api/records/1.0/search/?dataset=sanisettesparis&q=&rows=-1';
try {
let response = await fetch(url, settings);
let json = await response.json();
return Promise.all( // Here we return the promise returned by Promise.all(), so the life cycle of the CF is correctly managed
json['records'].map((toiletJsonObject) => {
admin
.firestore()
.collection('toilets')
.doc(toiletJsonObject['recordid'])
.set({ adresse: toiletJsonObject.fields.adresse });
})
);
} catch (error) {
console.log(error);
return null;
}
});
I have a GET endpoint, which basically makes some API calls to the Spoonacular API. Essentially, I make two API calls within the endpoint.
The first API call gets the list of recipe ID's for the specific ingredients
The second API calls gets the metadata for each of the recipe ID's.
After the first API call I store all the Id's in an array (recipeArray), and I want to make the second api call for each ID in my array (function recipeTest does this).
When I try to do this and then return my response to the front end, it always returns a response before completing all the API calls in the second step.
Here, is my code. The first API calls works just fine, but the second API call (recipeTest function), is where it messed up. Before that function finishes making all the API calls to the Spoonacular API, my endpoint returns an empty Array (res.send(toSend)). So, I was just wondering if there is any way around this?
Thank you so much in advance, I really appreciate it!
module.exports = (app) => {
app.get('/api/search', async (req, res) => {
console.log("endpoint working");
let ingredientList = "apples,+eggs,+bacon"; // needs to be given from the front end
let ingredientSearchUrl = `https://api.spoonacular.com/recipes/findByIngredients?ingredients=${ingredientList}&number=1&ignorePantry=true&apiKey=${keys.spoonacularKey}`;
try {
const ingredientSearchResult = await axios({
method: 'get',
url: ingredientSearchUrl
});
var recipeArray = ingredientSearchResult.data.map(info => {
return info.id;
});
} catch (err) {
console.log("error in finding recipe ID ", err);
}
let toSend = [];
try {
const check = await recipeTest(recipeArray, toSend);
} catch (err) {
console.log("error in finding recipe information ", err);
}
res.send(toSend);
});
}
const recipeTest = async (recipeArray, toSend) => {
return Promise.all(
_.forEach(recipeArray, async (recipeId) => {
let recipeInfoUrl = `https://api.spoonacular.com/recipes/${recipeId}/information?includeNutrition=false&apiKey=${keys.spoonacularKey}`;
let recipeInfo = {};
const recipeData = await axios({
method: 'get',
url: recipeInfoUrl
});
// console.log("recipeInfo search working", recipeData.data);
recipeInfo['id'] = recipeData.data.id;
recipeInfo['title'] = recipeData.data.title;
recipeInfo['time'] = recipeData.data.readyInMinutes;
recipeInfo['recipeUrl'] = recipeData.data.sourceUrl;
recipeInfo['imageUrl'] = recipeData.data.image;
// console.log('recipe info dict', recipeInfo);
toSend.push(recipeInfo);
console.log('toSend inside', toSend);
})
);
}
_.forEach return collection itself and not all your async handlers.
Use recipeArray.map to get an array of async functions to let Promise.all do its work:
Promise.all(
recipeArray.map(x => async (recipeId) => {
I'm still very new to coding so bear with me! I have followed a youtube course to build a note app and get a base to work from, but I'm now getting this error at random times when deleting the notes in firebase, hoping someone might be able to spot what's cooking here!
"Unhandled Rejection (FirebaseError): No document to update: projects/speakle-dc94b/databases/(default)/documents/notes/GdWPrQNxR3Z9TFMWmqOZ"
And it's referencing the node modules like so:
screenshot of the error in chrome
The code I have that interacts with firebase looks like this:
componentDidMount = () => {
firebase
.firestore()
.collection('notes')
.onSnapshot(serverUpdate => {
const notes = serverUpdate.docs.map(_doc => {
const data = _doc.data();
data['id'] = _doc.id;
return data;
});
console.log(notes);
this.setState({ notes: notes });
});
}
selectNote = (note, index) => this.setState({ selectedNoteIndex: index, selectedNote: note });
noteUpdate = (id, noteObj) => {
firebase
.firestore()
.collection('notes')
.doc(id)
.update({
title: noteObj.title,
body: noteObj.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
}
newNote = async (title) => {
const note = {
title: title,
body: ''
};
const newFromDB = await firebase
.firestore()
.collection('notes')
.add({
title: note.title,
body: note.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
const newID = newFromDB.id;
await this.setState({ notes: [...this.state.notes, note] });
const newNoteIndex = this.state.notes.indexOf(this.state.notes.filter(_note => _note.id === newID)[0]);
this.setState({ selectedNote: this.state.notes[newNoteIndex], selectedNoteIndex: newNoteIndex });
}
deleteNote = async (note) => {
const noteIndex = this.state.notes.indexOf(note);
await this.setState({ notes: this.state.notes.filter(_note => _note !== note) })
if(this.state.selectedNoteIndex === noteIndex) {
this.setState({ selectedNoteIndex: null, selectedNote: null});
} else {
this.state.notes.lenght > 1 ?
this.selectNote(this.state.notes[this.state.selectedNoteIndex - 1], this.state.selectedNoteIndex - 1) :
this.setState({ selectedNoteIndex: null, selectedNote: null });
}
firebase
.firestore()
.collection('notes')
.doc(note.id)
.delete()
.then(function() {
console.log("Document successfully deleted!");
}).catch(function(error) {
console.error("Error removing document: ", error);
});
}
}
It simply means that there is no document of that name to be uploaded.
you could either use set() or add() to add the document because it is not present.
noteUpdate = (id, noteObj) => {
firebase
.firestore()
.collection('notes')
.doc(id)
.update({
title: noteObj.title,
body: noteObj.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
}
replace the above code with this
noteUpdate = (id, noteObj) => {
firebase
.firestore()
.collection('notes')
.doc(id)
.add({
title: noteObj.title,
body: noteObj.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
}
or
noteUpdate = (id, noteObj) => {
firebase
.firestore()
.collection('notes')
.doc(id)
.set({
title: noteObj.title,
body: noteObj.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
}
I was working with something like this only especially in Cloud Functions and while writing an endpoint for doing some operation I came across the below-quoted error.
I was trying to read a document in a collection and if it existed I was trying to write a new doc into another collection. So it was kind of a nested code.
A piece of my code.
const firstDocRef = db.collection('myFirstCollection').doc('myDocName');
const existDoc = firstDocRef.get()
.then((resDoc)=>{
if(resDoc.exists)
{
db.collection('mySecondCollection').add({
.
.
.
.
.
orderCreatedAt:Firestore.FieldValue.serverTimestamp()
})
.then((new_doc)=>{
return res.status(200);
// return 200 ok what we were trying to achieve has completed.
})
.catch(()=>{
console.log("Log, as write failed somehow");
return res.status(500);
// return a 500 internal server error occurred
});
}
else
{
console.log("My first condition wasn't met, so logged it");
return res.end();
// properly terminate the processing of request
}
});
/*.catch((err)=>{
console.log("Our first doc presence check couldn't complete and hence I arrived here, log it");
res.writeHead(500);
return res.end();
// again give back 500 to client
});*/
UnhandledPromiseRejectionWarning: ReferenceError: Firestore is not defined
UnhandledPromiseRejectionWarning: Unhandled promise rejection.
This error originated either by throwing inside of an async function without a catch block
Now I am also new to Firebase but I came across this and somehow solved it.
So I was not getting the above error if I was putting in a catch block in get() document.
Strange huh!
Removed the catch block by commenting it. Got this error.
Now, this is a haywire error, it says the catch is not there, but we did it on purpose.
So I began searching, came across this question here on stack overflow and saw it's still unanswered. Searched and read the documentation myself.
I would like to tell you that this isn't because of any Firestore Security Rules, or anything else. Because I came across some guesses around these notions too while searching for an answer.
The common thing we all are doing here is that we are trying to achieve the ServerTimeStamp at FireStore
I would like to bring your notice to my imports in my node cloud function code.
const functions = require('firebase-functions');
const express = require('express');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
const db = admin.firestore();
So you see, I am using the new way of getting the permission to use Firestore because I am trying to establish a cloud function.
Now this is the documentation reference provided by Google: Click here!
The right syntax proposed by above API reference is
Firestore.FieldValue.serverTimestamp()
It is the culprit, it isn't providing me any timestamp and if there is not a catch block unhandled promise error is occuring and no error is being shown while debugging, it just doesn't work.
I did this, Solution part:
Even after those imports in my Node Program, I imported this:
const {Firestore} = require('#google-cloud/firestore');
Now all I did was that I used the statement in the timestamp field as
Firestore.FieldValue.serverTimestamp()
Exactly as mentioned and even used a catch block just in case any other problem occurs while at production. That is using db constant to do all the Database Transactional Stuff and just for serverTimeStamp I have to bring in new import.
And it worked, I guess require('#google-cloud/firestore') statement imported as {FireStore} brings in all the things that are required for the FieldValue thing to use as a reference.
I hope it helps any new person looking for it and saves a lot of time which I wasted searching for a solution.
I have verified it by running the cloud function on firebase emulator.
You could simply do it like this
You
Get it
If it exsits: you update it
If it doesn't exist, you set it.
const docRef = this.db.collection("users_holdings").doc(userId);
docRef.get().subscribe((doc) => {
if (doc.exists) {
docRef.update({
stockHoldings: stockHoldings,
});
} else {
docRef.set({
stockHoldings: stockHoldings,
});
}
});
I am having trouble with my lambda function. This is the current setup:
Lambda makes post request to API. API fetches data from postgres database and returns this data. When I use Postman or a local version of my lambda function, this works. When I use the actual lambda function, the API returns data with null.
Below are some code snippets:
lambda:
const axios = require('axios')
exports.handler = async (event) => {
let rows = await axios.post('http:/server/getData', {
params: {
var: event.var
}
})
if(rows.data.statusCode == 204){
//no data available
}
else{
data = rows.data.info
}
};
API Router Component
var router = express.Router()
const Pool = require('pg').Pool
const mgmtdb = new Pool({ ... })
router.post('/getData', function(req, res){
database.query("SELECT info FROM table WHERE var=$1", [req.body.var], (error, results) => {
const rows = results.rows
if (error) {
throw error
}
let status = rows.length == 0 ? 204 : 200
var responseJSON ={};
responseJSON.statusCode = status;
responseJSON.info= rows[0] ? rows[0].info : null;
res.json(responseJSON);
})
})
module.exports = router
When I call the API from Postman I get statusCode: 200 (data available).
If I call the API with the exact same data from lambda I get statusCode: 204 (no data available).
I believe that this is some async timing problem. I don't know how the responses from the API can differ ..
Is it possible that the API streams the response back to the originator for some time not just an impulse? And starts by streaming "no data available" and then, after a few milliseconds "oh, i found some data, here it is"? And that Postman waits for the stream to finish and the lambda function doesn't?
Thanks in advance!
Looks like your query is returning null because the req.body.var doesn't exist. Can you try changing your lambda function to this:
const axios = require('axios')
exports.handler = async (event) => {
let rows = await axios.post('http:/server/getData', {
var: event.var
})
if(rows.data.statusCode == 204){
//no data available
}
else{
data = rows.data.info
}
};
This basically removes the extra level params and makes the req.body.var work.
Sometimes lambda have issue with async await so i suggest you to write code like this
exports.handler = async (event, callback) => {
axios.post('http:/server/getData', {
var: event.var
})
.then((response) => {
callback(null, {
statusCode: 200,
body: JSON.stringify(response)
})
})
};
I've been using firebase functions test to do some testing on my functions. I have some code that is supposed to post a thing to firestore, basically in the same way that the examples show to do in the realtime database examples:
exports.addMessage = functions.https.onRequest((req, res) => {
const original = req.query.text;
admin.firestore()
.collection('messages')
.add({ original })
.then(documentReference => res.send(documentReference))
.catch(error => res.send(error));
});
For my test, I've spoofed some basic functionality using sinon, mocha and chai. Here is my current test, which is failing with the error message: TypeError: firestoreService.snapshot_ is not a function
describe('addMessage', () => {
// add message should add a message to the database
let oldDatabase;
before(() => {
// Save the old database method so it can be restored after the test.
oldDatabase = admin.firestore;
});
after(() => {
// Restoring admin.database() to the original method.
admin.firestore = oldDatabase;
});
it('should return the correct data', (done) => {
// create stubs
const refStub = sinon.stub();
// create a fake request object
const req = {
query : {
text: 'fly you fools!'
}
};
const snap = test.firestore.makeDocumentSnapshot({ original: req.query.text }, 'messages/1234');
// create a fake document reference
const fakeDocRef = snap._ref;
// create a fake response object
const res = {
send: returnedDocRef => {
// test the result
assert.equal(returnedDocRef, fakeDocRef);
done();
}
};
// spoof firestore
const adminStub = sinon.stub(admin, 'firestore').get(() => () => {
return {
collection: () => {
return {
add: (data) => {
const secondSnap = test.firestore.makeDocumentSnapshot(data, 'messages/1234');
const anotherFakeDocRef = secondSnap._ref;
return Promise.resolve(anotherFakeDocRef);
}
}
}
}
});
// call the function to execute the test above
myFunctions.addMessage(req, res);
});
});
My question is how the heck do I fix this?
I previously had a test that was just passing the first snap and fakeDocRef, and my test was passing fine, but as soon as I resolve the promise with the new fake document reference, it fails...
Any help would be appreciated! Thanks!
There are three different types of the calls, that are different:
Operating on the Collections.
Operating on the Documents.
Operating on the results of the query.
They have to be used consistently.
Please refer a documentation to see the difference operation on the collection and the document.