Firebase CloudFunctions: Querying Nodes - node.js

I am exploring Cloud Functions with FCM to do a push notification to my iOS app device. I have the following structure and I am listening for people who registered to an event. I want to extract the eventHost so that I can go to my users node to find the userUID and eventually his deviceID, and send a push notification to him.
{
"events" : {
"XXX" : {
"eventHost" : "YYY", //<-- How do I get this node?
"eventID" : "XXX",
"registered" : { //<-- Listening for this node
"ASKDJHAIUCHA" : {
"name" : "Emma Watson",
"userUID" : "ASKDJHAIUCHA"
}
},
},
},
"users" : {
"YYY" : {
"deviceID" : "1234456",
"name" : "Andrew Garfield",
"userUID" : "YYY"
},
}
}
My code for the Cloud Functions as such so far:
exports.sendNotification = functions.database.ref('/events/{eventId}/registered')
.onWrite(event => {
const register = event.data.val();
const eventHost = functions.database.ref('/events/' + event.params.eventId + '/eventHost')
console.log('sendNotifications', eventHost);
const payload = {
notification: {
title: "Event Registration",
body: "Someone registered to your event!"
}
};
const options = {
priority: "high"
};
return admin.messaging().sendToDevice("theDeviceID", payload, options)
});
And my Swift portion when adding value into the database is as such:
#IBAction func registerButtonDidTap(_ sender: Any) {
let personDict = [FIRConstants.UserInfo.Name: user.userName,
FIRConstants.UserInfo.UserUID: user.userUID]
let registerPerson = [user.userUID!: personDict as AnyObject] as [String: AnyObject]
let values = ["registered": registerPerson]
FIRHelperClient.sharedInstance.checkIfEventHasRegistrants(ref, event.eventID!) { (has, error) in
if let error = error {
print(error.localizedDescription)
} else {
if let has = has {
if has {
self.ref.child("events").child(self.event.eventID!).child("registered").updateChildValues(registerPerson)
} else {
self.ref.child("events").child(self.event.eventID!).updateChildValues(values)
}
}
}
}
}
My cloud functions is definitely not complete as currently I am hardcoding theDeviceID. As I am pretty inexperience with Node.js and am trying to write both the iOS code in Swift and the server side code, so pls pardon me if this question is elementary. Some advice here would be helpful, thanks.

You'll need to read the host, which your code currently doesn't do.
exports.sendNotification = functions.database.ref('/events/{eventId}/registered')
.onWrite(event => {
const register = event.data.val();
const eventHostRef = functions.database.ref('/events/' + event.params.eventId + '/eventHost')
return eventHostRef.once('value', (eventHostSnapshot) => {
const eventHost = eventHostSnapshot.val();
console.log('sendNotifications', eventHost);
const payload = {
notification: {
title: "Event Registration",
body: "Someone registered to your event!"
}
};
const options = {
priority: "high"
};
return admin.messaging().sendToDevice("theDeviceID", payload, options)
});
});
I highly recommend that you spend some time learning how to interact with the Firebase Database through JavaScript before continuing though. This doesn't have to be through Cloud Functions. You could also use the Firebase Database Admin SDK from Node.js on your client or take the Firebase codelab for web. Whichever one you take, it will ensure you're much better prepared for interacting with the database through Cloud Functions.
As a final warning: you're nesting multiple data types under a single list. This is not recommended, since it leads to all kinds of problems down the line. Instead, I would pull the registered users into their own top-level node, so that you get:
{
"events" : {
"XXX" : {
"eventHost" : "YYY",
"eventID" : "XXX"
}
},
"registered" : {
"XXX" : {
"ASKDJHAIUCHA" : {
"name" : "Emma Watson",
"userUID" : "ASKDJHAIUCHA"
}
}
},
"users" : {
"YYY" : {
"deviceID" : "1234456",
"name" : "Andrew Garfield",
"userUID" : "YYY"
}
}
}

Related

Elasticsearch node js point in time search_phase_execution_exception

const body = {
query: {
geo_shape: {
geometry: {
relation: 'within',
shape: {
type: 'polygon',
coordinates: [$polygon],
},
},
},
},
pit: {
id: "t_yxAwEPZXNyaS1wYzYtMjAxN3IxFjZxU2RBTzNyUXhTUV9XbzhHSk9IZ3cAFjhlclRmRGFLUU5TVHZKNXZReUc3SWcAAAAAAAALmpMWQkNwYmVSeGVRaHU2aDFZZExFRjZXZwEWNnFTZEFPM3JReFNRX1dvOEdKT0hndwAA",
keep_alive: "1m",
},
};
Query fails with search_phase_execution_exception at onBody
Without pit query works fine but it's needed to retrieve more than 10000 hits
Well, using PIT in NodeJS ElasticSearch's client is not clear, or at least is not well documented. You can create a PIT using the client like:
const pitRes = await elastic.openPointInTime({
index: index,
keep_alive: "1m"
});
pit_id = pitRes.body.id;
But there is no way to use that pit_id in the search method, and it's not documented properly :S
BUT, you can use the scroll API as follows:
const scrollSearch = await elastic.helpers.scrollSearch({
index: index,
body: {
"size": 10000,
"query": {
"query_string": {
"fields": [ "vm_ref", "org", "vm" ],
"query": organization + moreQuery
},
"sort": [
{ "utc_date": "desc" }
]
}
}});
And then read the results as follows:
let res = [];
try {
for await (const result of scrollSearch) {
res.push(...result.body.hits.hits);
}
} catch (e) {
console.log(e);
}
I know that's not the exact answer to your question, but I hope it helps ;)
The usage of point-in-time for pagination of search results is now documented in ElasticSearch. You can find more or less detailed explanations here: Paginate search results
I prepared an example that may give an idea about how to implement the workflow, described in the documentation:
async function searchWithPointInTime(cluster, index, chunkSize, keepAlive) {
if (!chunkSize) {
chunkSize = 5000;
}
if (!keepAlive) {
keepAlive = "1m";
}
const client = new Client({ node: cluster });
let pointInTimeId = null;
let searchAfter = null;
try {
// Open point in time
pointInTimeId = (await client.openPointInTime({ index, keep_alive: keepAlive })).body.id;
// Query next chunk of data
while (true) {
const size = remained === null ? chunkSize : Math.min(remained, chunkSize);
const response = await client.search({
// Pay attention: no index here (because it will come from the point-in-time)
body: {
size: chunkSize,
track_total_hits: false, // This will make query faster
query: {
// (1) TODO: put any filter you need here (instead of match_all)
match_all: {},
},
pit: {
id: pointInTimeId,
keep_alive: keepAlive,
},
// Sorting should be by _shard_doc or at least include _shard_doc
sort: [{ _shard_doc: "desc" }],
// The next parameter is very important - it tells Elastic to bring us next portion
...(searchAfter !== null && { search_after: [searchAfter] }),
},
});
const { hits } = response.body.hits;
if (!hits || !hits.length) {
break; // No more data
}
for (hit of hits) {
// (2) TODO: Do whatever you need with results
}
// Check if we done reading the data
if (hits.length < size) {
break; // We finished reading all data
}
// Get next value for the 'search after' position
// by extracting the _shard_doc from the sort key of the last hit
searchAfter = hits[hits.length - 1].sort[0];
}
} catch (ex) {
console.error(ex);
} finally {
// Close point in time
if (pointInTime) {
await client.closePointInTime({ body: { id: pointInTime } });
}
}
}

AWS PUT request met with "Provided key element does not match schema."

(Edited to incorporate comments)
So I apologize in advance for the long question. I don't know how else to ask it.
I'm trying to finish up a full-stack web app using React, Node, and DynamoDB. POST and GET requests are working fine, but I'm stuck on PUT. My mock PUT request works fine, but once I try it from the front end in React, I get the error mentioned in the title. I'll show the back end code first, then the mock update, and then the front end.
import handler from "./libs/handler-lib";
import dynamoDb from "./libs/dynamodb-lib";
export const main = handler(async (event, context) => {
const data = JSON.parse(event.body);
const params = {
TableName: process.env.tableName,
Key: {
userId: event.requestContext.identity.cognitoIdentityId,
activityId: event.pathParameters.activityId
},
UpdateExpression: "SET title = :title, activityType = :activityType, activityRoutine = :activityRoutine, activityComment = :activityComment",
ExpressionAttributeValues: {
":title": data.title || null,
":activityType": data.activityType || null,
// ":activityRoutine": data.activityRoutine == '' ? "None" : data.activityRoutine,
// ":activityComment": data.activityComment == '' ? "None" : data.activityComment
":activityRoutine": data.activityRoutine || null,
":activityComment": data.activityComment || null
},
ReturnValues: "ALL_NEW"
};
await dynamoDb.update(params);
return { status: true };
This mock update event works without issue:
{
"body": "{\"title\":\"test\",\"activityType\":\"testing\",\"activityRoutine\":\"\",\"activityComment\":\"\"}",
"pathParameters": {
"activityId": "long-alphanumeric-id"
},
"requestContext": {
"identity": {
"cognitoIdentityId": "us-east-and-so-on"
}
}
}
But this code, which produces the exact same Javascript object as the mock, is not okay with AWS:
function saveActivity(activity) {
try {
return API.put("activities", `/activities/${id}`, {
body: activity
});
} catch(e) {
console.log("saveActivity error:", e);
}
}
async function handleSubmit(event) {
event.preventDefault();
setIsLoading(true)
try {
await saveActivity({
title: title, activityType: activityType, activityRoutine: activityRoutine, activityComment: activityComment
// "key": {userId: userId, activityId: activityId}
// "pathParameters": {"id": activityId},
// "requestContext": {"identity": {"cognitoIdentityId": userId}}
});
} catch(e) {
console.log(e)
setIsLoading(false)
}
}
If anyone needs to see more of the code, I'm happy to share, but I figured this question is already getting very long. Any code you see commented out has been tried before without success.
I'd also be happy if someone could point me in the right direction as far as the AWS documentation is concerned. I've been going off of a tutorial and modifying it where need be.
Any help is appreciated!

How to read unread email with body and attachment using node-ews package

I am able to send email using this node-ews package but I am not able find suitable example to read email from Inbox folder and get the body and attachments from the email.
I have gone through the Microsoft docs e.g. https://learn.microsoft.com/en-us/exchange/client-developer/exchange-web-services/how-to-work-with-exchange-mailbox-items-by-using-ews-in-exchange#get-an-item-by-using-the-ews-managed-api
but the examples are provided in C#, C++ or VB..
But I want to do this with Nodejs.
You can use following code to get emails from Inbox using FindItem function and then read each email using GetItem function
// Read emails from Inbox
var ewsFunction = 'FindItem';
var ewsArgs = {
'attributes': {
'Traversal': 'Shallow'
},
'ItemShape': {
't:BaseShape': 'IdOnly',
't:AdditionalProperties': {
't:FieldURI': {
'attributes': {
'FieldURI': 'item:Subject'
}
}
}
},
'ParentFolderIds' : {
'DistinguishedFolderId': {
'attributes': {
'Id': 'inbox'
}
}
}
};
// Itreate over all the emails and store Id and ChangeKey.
ews.run(ewsFunction, ewsArgs, ewsSoapHeader)
.then(result => {
// Iterate over the result and extract Id and ChangeKey of the messages and pass those to GetItem function to read messages
})
// For reading individual messages returned by FindItem (using Id and ChangeKey)
var ewsFunction = 'GetItem';
var ewsArgs = {
'ItemShape': {
'BaseShape': 'Default',
't:AdditionalProperties': {
't:FieldURI': {
'attributes': {
'FieldURI': 'item:Attachments'
}
}
}
},
'ItemIds' : {
'ItemId': {
'attributes': {
'Id': Id,
'ChangeKey' : ChangeKey
}
}
}
};
await ews.run(ewsFunction, ewsArgs, ewsSoapHeader)
.then(result => {
// Iterate over the result and extract meesage
})

Unable to write item(s) to DynamoDB table utilizing DocumentClient - Nodejs

I'm absolutely brand new to DynamoDb and I'm trying to simply write an object from a NodeJS Lambda. Based on what I've read and researched I should probably be using DocumentClient from the aws-sdk. I also found the following question here regarding issues with DocumentClient, but it doesn't seem to address my specific issue....which I can't really find/pinpoint unfortunately. I've set up a debugger to help with SAM local development, but it appears to be only providing some of the errors.
The code's implementation is shown here.
var params = {
TableName: "March-Madness-Teams",
Item: {
"Id": {"S": randstring.generate(9)},
"School":{"S": team_name},
"Seed": {"S": seed},
"ESPN_Id": {"S": espn_id}
}
}
console.log(JSON.stringify(params))
dynamodb.put(params, (error,data) => {
if (error) {
console.log("Error ", error)
} else {
console.log("Success! ", data)
}
})
Basically I'm scrubbing a website utilizing cheerio library and cherry picking values from the DOM and saving them into the json object shown below.
{
"TableName": "March-Madness-Teams",
"Item": {
"Id": {
"S": "ED311Oi3N"
},
"School": {
"S": "BAYLOR"
},
"Seed": {
"S": "1"
},
"ESPN_Id": {
"S": "239"
}
}
}
When I attempt to push this json object to Dynamo, I get errors says
Error MultipleValidationErrors: There were 2 validation errors:
* MissingRequiredParameter: Missing required key 'TableName' in params
* MissingRequiredParameter: Missing required key 'Item' in params
The above error is all good in well....I assume it didn't like the fact that I had wrapped those to keys in strings, so I removed the quotes and sent the following
{
TableName: "March-Madness-Teams",
Item: {
"Id": {
"S": "ED311Oi3N"
},
"School": {
"S": "BAYLOR"
},
"Seed": {
"S": "1"
},
"ESPN_Id": {
"S": "239"
}
}
}
However, when I do that...I kind of get nothing.
Here is a larger code snippet.
return new Promise((resolve,reject) => {
axios.get('http://www.espn.com/mens-college-basketball/bracketology')
.then(html => {
const dynamodb = new aws.DynamoDB.DocumentClient()
let $ = cheerio.load(html.data)
$('.region').each(async function(index, element){
var preregion = $(element).children('h3,b').text()
var region = preregion.substr(0, preregion.indexOf('(') - 1)
$(element).find('a').each(async function(index2, element2){
var seed = $(element2).siblings('span.rank').text()
if (seed.length > 2){
seed = $(element2).siblings('span.rank').text().substring(0, 2)
}
var espn_id = $(element2).attr('href').split('/').slice(-2)[0]
var team_name = $(element2).text()
var params = {
TableName: "March-Madness-Teams",
Item: {
"Id": randstring.generate(9),
"School":team_name,
"Seed": seed,
"ESPN_Id": espn_id
}
}
console.log(JSON.stringify(params))
// dynamodb.put(params)
// .then(function(data) {
// console.log(`Success`, data)
// })
})
})
})
})
Can you try without the type?
Instead of
"School":{"S": team_name},
for example, use
"School": team_name,
From your code, I can see the mis promise on the dynamodb request. Try to change your lines :
dynamodb.put(params).then(function(data) {
console.log(`Success`, data)
})
to be :
dynamodb.put(params).promise().then(function(data) {
console.log(`Success`, data)
})
you can combine with await too :
await dynamodb.put(params).promise().then(function(data) {
console.log(`Success`, data)
})
exports.lambdaHandler = async (event, context) => {
const html = await axios.get('http://www.espn.com/mens-college-basketball/bracketology')
let $ = cheerio.load(html.data)
const schools = buildCompleteSchoolObject(html, $)
try {
await writeSchoolsToDynamo(schools)
return { statusCode: 200 }
} catch (error) {
return { statusCode: 400, message: error.message }
}
}
const writeSchoolsToDynamo = async (schools) => {
const promises = schools.map(async school => {
await dynamodb.put(school).promise()
})
await Promise.all(promises)
}
const buildCompleteSchoolObject = (html, $) => {
const schools = []
$('.region').each(loopThroughSubRegions(schools, $))
return schools
}
const loopThroughSubRegions = (schools, $) => {
return (index, element) => {
var preregion = $(element).children('h3,b').text()
var region = preregion.substr(0, preregion.indexOf('(') - 1)
$(element).find('a').each(populateSchoolObjects(schools, $))
}
}
const populateSchoolObjects = (schools, $) => {
return (index, element) => {
var seed = $(element).siblings('span.rank').text()
if (seed.length > 2) {
seed = $(element).siblings('span.rank').text().substring(0, 2)
}
var espn_id = $(element).attr('href').split('/').slice(-2)[0]
var team_name = $(element).text()
schools.push({
TableName: "March-Madness-Teams",
Item: {
"Id": randstring.generate(9),
"School": team_name,
"Seed": seed,
"ESPN_Id": espn_id
}
})
}
}
I know this is drastically different from what I started with but I did some more digging and kind of kind of worked to this...I'm not sure if this is the best way, but I seemed to get it to work...Let me know if something should change!
Oh I understand what you want.
Maybe you can see the code above works, but there is one concept you have to improve here about async - await and promise especially on lambda function.
I have some notes here from your code above, maybe can be your consideration to improve your lambda :
Using await for every promise in lambda is not the best approach because we know the lambda time limitation. But sometimes we can do that for other case.
Maybe you can change the dynamodb.put method to be dynamodb.batchWriteItem :
The BatchWriteItem operation puts or deletes multiple items in one or more tables.
Or If you have to use dynamodb.put instead, try to get improve the code to be like so :
const writeSchoolsToDynamo = async (schools) => {
const promises = schools.map(school => {
dynamodb.put(school).promise()
})
return Promise.all(promises)
}

How could I detect an Intent with a context ? (node.js SDK for Dialogflow API V2)

How could I detect an Intent with a context ?
I defined an Intent "Book" with an input Context "Activity", then trained my bot with "Book this activity in my schedule".
I don't want to use dialog flow to manage context flow because it may evolve from other inputs in my app. So I want pass it as a parameter for each intent detection.
I use node.js SDK for Dialogflow API V2.
I precise it works fine with REST API v1 ... I think i'm stuck with API v2 Contexts :)
I'm looking to do exactly the same as the following code with API v2
//code below is working fine
import apiai from 'apiai';// Dialogflow SDK v1
import uuidv4 from 'uuid/v4';
const sessionId = uuidv4();
const api = apiai('myToken');
const lang = 'en-US';
const request = api.textRequest('Book this activity in my schedule', {
sessionId,
lang,
contexts : [{ name : 'activity' }],
});
request.on('response', function(response) {
console.log(response);
});
request.on('error', function(error) {
console.log(error);
});
request.end();
I didn't found documentation or exemple to do that except API documentation so I probably did something wrong
Whenever I pass a context as a queryParams or create a Context it don't work.
It also seems that context creation from API don't work at all.
import dialogflow from 'dialogflow';//Dialogflow SDK v2
import uuidv4 from 'uuid/v4';
const projectId = 'myID';
const sessionId = uuidv4();
const languageCode = 'en-US';
const sessionClient = new dialogflow.SessionsClient();
const contextClient = new dialogflow.ContextsClient();
const sessionPath = sessionClient.sessionPath(projectId, sessionId);
...
First try returned my Default Fallback Intent :
const request = {
session : sessionPath,
queryParams : {
context : {
parent : sessionPath,
name : contextClient.contextPath(
projectId,
sessionId,
'activity'
),
},
},
queryInput : {
text : {
text : 'Book this activity in my schedule',
languageCode : languageCode,
},
},
};
sessionClient.detectIntent(request)
.then(responses => {
const result = responses[0].queryResult;
if (result.intent) {
console.log(result.intent.displayName);
} else {
console.log(' No intent matched.');
}
});
Second try returned my Default Fallback Intent :
contextClient
.createContext({
parent : sessionPath,
context : {
name : contextClient.contextPath(
projectId,
sessionId,
'activity'
),
},
})
.then(() => contextClient.listContexts({ parent : sessionPath }))
.then(contexts => {
console.log(contexts);//returned an empty list
return sessionClient.detectIntent(request);
}).then(responses => {
const result = responses[0].queryResult;
if (result.intent) {
console.log(result.intent.displayName);
} else {
console.log(' No intent matched.');
}
});
Anybody see what's wrong with it ?
Help needed :)
Had same issue.
Found that I need to send "lifespanCount" greated than 0.
{
"session": "projects\/verifier-26084\/agent\/sessions\/1gg5b76c37f7111b",
"queryParams":
{
"contexts": [
{
"name": "projects\/verifier-26084\/agent\/sessions\/1gg5b76c37f7111b\/contexts\/ask-children",
"lifespanCount": 1
}]
},
"queryInput":
{
"text":
{
"text": "\u0442\u0440\u0438",
"languageCode": "ru-RU"
}
}
}

Resources