How to get a specific item from JSON - node.js

I'm new to node.js and DialogFlow. I am using DynamoDB to store data and I'm creating skills on Google. I'm trying to write a code to retrieve a specific item on that table.
I've got it working to show all items where ID is equal = 1, but how would I make it so I can just get attribute 'name'?
My idea is a user provides an id then the code will retrieve name where id was 1 and store that as a variable and use agent.add('hello $(name)'); to display it back as speech to the user.
function readdata(agent){
let dbread = new aws.DynamoDB.DocumentClient();
const id = agent.parameters.id;
let read = function(){
var parameters = {
TableName:"Dynamodb",
Key:{
"id":id
}
};
dbread.get(parameters, function(err,data){
if(err){
console.log("error",JSON.stringify(data,null,2));
}else{
console.log("success",JSON.stringify(data,null,2));
}
});
agent.add(`hello ${name}`);
};
read();
}

Once you have the data back from the get() call, the data object will contain an Item attribute. The value of this attribute will be another object that contains the attribute/value pairs for this record, or be empty if the record isn't found.
The debugging you have in place that shows JSON.stringify(data) should show this.
Assuming you knew all the fields were there, you could do something like
const name = data.Item.name;
a more robust way using current JavaScript would be to make sure everything was assigned, otherwise return undefined at any point. So something like this would work
const name = data && data.Item && data.Item.name;
However - you will have a problem doing this with Dialogflow
You don't show which Dialogflow library you're using, but most of them require you to return a Promise to indicate that it needs to wait for asynchronous calls (such as the call to DynamoDB) to complete. You're using get() with a callback function instead of a Promise. So you need to do one of the following:
Wrap the call in a Promise
Since get() returns an AWS.Request you can use the promise() method of this to get a Promise that you can return and which has then portions that generate the response - similar to how you're doing your callbacks now.
Under this scheme, your call might look something like this (untested):
return dbread.get(parameters).promise()
.then( data => {
console.log("success",JSON.stringify(data,null,2));
const name = data && data.Item && data.Item.name;
if( name ){
agent.add( `Hello ${name}` );
} else {
agent.add( "I don't know who you are." );
}
})
.catch( err => {
console.log("error",JSON.stringify(data,null,2));
agent.add( "There was an error" );
});

Related

Meteor Client calling findOne in Server Method

I have a client-side form that can create a document upon submission. I want to see if one of the input fields already exists on a Document in the DB though. This would then alert the user and ask them if they want to continue creating the record.
Client-side event
Template.createDoc.events({
'click button[type=submit]'(e, template) {
//This particular example is checking to see if a Doc with its `name` property set to `value` already exists
const value = $('#name');
const fieldName = 'name';
const exists = Meteor.call('checkIfFieldExistsOnDoc', fieldName, value);
if (exists) {
if (confirm(`Doc with ${value} as its ${fieldName} already exists. Are you sure you want to continue creating Doc?`) {
//db.Docs.insert....
}
}
}
});
Server-side Meteor Method
'checkIfFieldExistsOnDoc'(field, val) {
if (this.isServer) {
this.unblock();
check(field, String);
check(val, String);
if (!this.userId) {
throw new Meteor.Error('not-authorized', 'You are not authorized.');
}
const findObj = {};
findObj[field] = val;
const fieldsObj = {};
fieldsObj[fieldsObj] = 1;
const doc = Docs.findOne(findObj, {fields: fieldsObj});
return doc;
}
},
My issue is that the client-side code always gets undefined back when calling the Server method. I now understand why, however, I'm not keen on wrapping all of my subsequent client-code into a callback yet.
So - any other ideas on how I can attempt to do this simple feature?
Also - I was thinking of having the client-side page's onCreated do a 1-time server call to get ALL names for all Docs, storing this in memory, and then doing the check upon form submission using this. Obviously, this is inefficient and not-scalable, although it would work
Meteor.call in the client side is always an async call. Then you need implement a callback.
See docs: https://docs.meteor.com/api/methods.html#Meteor-call
Meteor.call('checkIfFieldExistsOnDoc', fieldName, value, function(error, result) {
if (result) {
if (confirm(`Doc with ${value} as its ${fieldName} already exists. Are you sure you want to continue creating Doc?`) {
//db.Docs.insert....
}
}
});
On the client, you can wrap any Meteor.call with a Promise and then use it with async/await. There are some packages on Atmosphere that do this for you to.
I've used this package for years: https://atmospherejs.com/deanius/promise
On the client I often just use await Meteor.callPromise() which returns a response nicely.
Here are a couple of the best write-ups on the many options available to you:
https://blog.meteor.com/using-promises-on-the-client-in-meteor-fb4f1c155f84
https://forums.meteor.com/t/meteor-methods-return-values-via-promise-async/42060
https://dev.to/jankapunkt/async-meteor-method-calls-24f9

I want to return data from dynamodb when the key matches

**I want to return the data key values but it is not working. Please help me. The main concept of this method is when this is invoke dal is the keyword and it fetches that key values from the dynamodb. For that i used global also but not updating that global variable also.
I tried returning the value from the callback, as well as assigning the result to a local variable inside the function and returning that one, but none of those ways actually return the response (they all return undefined or whatever the initial value of the variable result is).**
function getLocation(){
let a=[];
const documentClient = new AWSS.DynamoDB.DocumentClient({region:"us-east-2"});
const params = {
TableName : "ProductLocation1",
Key :{
Product_Name : 'dal'
}
};
documentClient.get(params,(err,data)=>{
if(err){
console.log('error is',err);
}
console.log('data is : ',data);
global.a=Object.keys(data);
});
return global.a;
}
try {
const result = await documentClient.get(params).promise();
} catch (error) {
// handle error
}
You could get the result using the promise & await rather than the callback function. In this way, you don't need to store them in a local variable.

Compare API response against itself

I am trying to:
Poll a public API every 5 seconds
Store the resulting JSON in a variable
Store the next query to this same API in a second variable
Compare the first variable to the second
Print the second variable if it is different from the first
Else: Print the phrase: 'The objects are the same' if they haven't changed
Unfortunately, the comparison part appears to fail. I am realizing that this implementation is probably lacking the appropriate variable scoping but I can't put my finger on it. Any advice would be highly appreciated.
data: {
chatters: {
viewers: {
},
},
},
};
//prints out pretty JSON
function prettyJSON(obj) {
console.log(JSON.stringify(obj, null, 2));
}
// Gets Users from Twitch API endpoint via axios request
const getUsers = async () => {
try {
return await axios.get("http://tmi.twitch.tv/group/user/sixteenbitninja/chatters");
} catch (error) {
console.error(error);
}
};
//Intended to display
const displayViewers = async (previousResponse) => {
const usersInChannel = await getUsers();
if (usersInChannel.data.chatters.viewers === previousResponse){
console.log("The objects are the same");
} else {
if (usersInChannel.data.chatters) {
prettyJSON(usersInChannel.data.chatters.viewers);
const previousResponse = usersInChannel.data.chatters.viewers;
console.log(previousResponse);
intervalFunction(previousResponse);
}
}
};
// polls display function every 5 seconds
const interval = setInterval(function () {
// Calls Display Function
displayViewers()
}, 5000);```
The issue is that you are using equality operator === on objects. two objects are equal if they have the same reference. While you want to know if they are identical. Check this:
console.log({} === {})
For your usecase you might want to store stringified version of the previousResponse and compare it with stringified version of the new object (usersInChannel.data.chatters.viewers) like:
console.log(JSON.stringify({}) === JSON.stringify({}))
Note: There can be issues with this approach too, if the order of property changes in the response. In which case, you'd have to check individual properties within the response objects.
May be you can use npm packages like following
https://www.npmjs.com/package/#radarlabs/api-diff

Using node.js and promise to fetch paginated data

Please keep in mind that I am new to node.js and I am used with android development.
My scenario is like this:
Run a query against the database that returns either null or a value
Call a web service with that database value, that offers info paginated, meaning that on a call I get a parameter to pass for the next call if there is more info to fetch.
After all the items are retrieved, store them in a database table
If everything is well, for each item received previously, I need to make another web call and store the retrieved info in another table
if fetching any of the data set fails, all data must be reverted from the database
So far, I've tried this:
getAllData: function(){
self.getMainWebData(null)
.then(function(result){
//get secondary data for each result row and insert it into database
}
}
getMainWebData: function(nextPage){
return new Promise(function(resolve, reject) {
module.getWebData(nextPage, function(errorReturned, response, values) {
if (errorReturned) {
reject(errorReturned);
}
nextPage = response.nextPageValue;
resolve(values);
})
}).then(function(result) {
//here I need to insert the returned values in database
//there's a new page, so fetch the next set of data
if (nextPage) {
//call again getMainWebData?
self.getMainWebData(nextPage)
}
})
There are a few things missing, from what I've tested, getAllData.then fires only one for the first set of items and not for others, so clearly handling the returned data in not right.
LATER EDIT: I've edited the scenario. Given some more research my feeling is that I could use a chain or .then() to perform the operations in a sequence.
Yes it is happening as you are resolving the promise on the first call itself. You should put resolve(value) inside an if statement which checks if more data is needed to be fetched. You will also need to restructure the logic as node is asynchronous. And the above code will not work unless you do change the logic.
Solution 1:
You can either append the paginated response to another variable outside the context of the calls you are making. And later use that value after you are done with the response.
getAllData: function(){
self.getMainWebData(null)
.then(function(result){
// make your database transaction if result is not an error
}
}
function getList(nextpage, result, callback){
module.getWebData(nextPage, function(errorReturned, response, values) {
if(errorReturned)
callback(errorReturned);
result.push(values);
nextPage = response.nextPageValue;
if(nextPage)
getList(nextPage, result, callback);
else
callback(null, result);
})
}
getMainWebData: function(nextPage){
return new Promise(function(resolve, reject) {
var result = [];
getList(nextpage, result, function(err, results){
if(err)
reject(err);
else{
// Here all the items are retrieved, you can store them in a database table
// for each item received make your web call and store it into another variable or result set
// suggestion is to make the database transaction only after you have retrieved all your data
// other wise it will include database rollback which will depend on the database which you are using
// after all this is done resolve the promise with the returning value
resolve(results);
}
});
})
}
I have not tested it but something like this should work. If problem persists let me know in comments.
Solution 2:
You can remove promises and try the same thing with callback as they are easier to follow and will make sense to the programmers who are familiar with structural languages.
Looking at your problem, I have created a code that would loop through promises.
and would only procede if there is more data to be fetched, the stored data would still be available in an array.
I hope this help. Dont forget to mark if it helps.
let fetchData = (offset = 0, limit= 10) => {
let addresses = [...Array(100).keys()];
return Promise.resolve(addresses.slice(offset, offset + limit))
}
// o => offset & l => limit
let o = 0, l = 10;
let results = [];
let process = p => {
if (!p) return p;
return p.then(data => {
// Process with data here;
console.log(data);
// increment the pagination
o += l;
results = results.concat(data);
// while there is data equal to limit set then fetch next page
// otherwise return the collected result
return (data.length == l)? process(fetchAddress(o, l)).then(data => data) : results;
})
}
process(fetchAddress(o, l))
.then(data => {
// All the fetched data will be here
}).catch(err => {
// Handle Error here.
// All the retrieved data from database will be available in "results" array
});
if You want to do it more often I have also created a gist for reference.
If You dont want to use any global variable, and want to do it in very functional way. You can check this example. However it requires little more complication.

Trying to understand how the node.js programming model works

I've been reading about node.js recently (like many others). I find interesting for some use cases, but am a bit struggling to understand the inner workings, specifically the interaction between closure functions and the process flow of the code.
Let's say I have a function which accepts a key-value array. The function must check that the values follow certain data-quality guidelines (for example some keys must have a value, other keys must have numbers as values etc) before storing the data somewhere (for the purpose of this question let's assume data validation has to be done in the application itself).
In "regular" developments models I'd write something like this:
resultName = validateName(data.name)
resultAddress = validateAddress(data.address)
resultID = validateID(data.id)
if (resultName && resultAddress && resultID) {
store(data)
else {
sendErrorToUser(data)
}
Get the results of the validations, and either explain the error(s) to the user or store data and return some kind of confirmation. The flow is very clear.
The way I understand node.js, the way to do this would be to delegate the validations to a different function (to avoid waiting for each validation to finish), and supply two callback functions to the functions which validate the chunks of data:
* a callback to call when validation is successful
* a callback to call when validation fails
It's easy to now return to the user with a "please wait" message, but I have to wait for all validations to clear (or fail) before storing the data or explaining the problem to the user. As a simple way to figure out if all the validations are done I thought of using a variable that counts the number of functions that called the callback, and emitting a "validation complete" event to store the validated data (or get back to the user with any errors). Or, alternatively, emit an event after each validation is complete and in that event's code check if all validations are complete before emitting the "store" / "error" events.
My question is -- am I approaching this correctly? Or is there a more suitable way to do these kinds of things with node.js (or similar event-based systems).
Thank you!
Alon
Are your validations asynchronous? If they are not you can use the code you posted, the "regular" one.
If the validations are asynchronous (checking uniqueness of an email for instance), you need to provide callbacks:
var validateUniqueEmail = function (data, callback) {
db.find({email: data.email}, function (err, result) {
callback(err, result === null);
})
};
var validateAndStore = function (data, callback) {
asyncValidation(data, function (err, is_valid) {
if (err) {
callback(err, null);
} else if (!is_valid) {
callback('Email is not unique', null);
} else {
db.store(data, callback);
}
});
}
The code above can be simplified a lot by using some validator or ORM modules already existing
example: mongolia validator module.
Let's go. Basically, what you want to do is something along the lines of :
var validate(data, cb){
var allOk = true;
for(var key in data){
allOk = allOk && validate[key](data.key); // validator depends on the key
}
if (allOk) cb(null, data); else cb(new Error "bleh");
}
This could be done the following way (note how we pass the failed keys as the first (error) argument to the callback):
var validate(data, cb){
var status = {true:[], false:[]},
total = Object.keys(data).length,
done = 0;
for (var key in data)
(function(key){
validate[key](data[key], function(ok){
status[ok].push(key);
if (++done == total){
status[false].length ? cb(status[false]) : cb(null);
}
});
})(key);
}
Which you can use this way :
validate(data, function(failures){
if (failures){
// tell the user the input does not validate on the keys stored in failures
} else {
// all ok
store(data);
}
});
Correct me if I'm wrong, but I think what you're asking is how to handle the response from multiple asynchronous calls.
Here's how I do it (using your validation example):
var result = {};
function isAllDataAvailable() {
return result.name !== undefined
&& result.address !== undefined
&& result.id !== undefined;
}
function callback(error) {
if (error) {
showError(error);
// terminate here (?)
return;
}
if (isAllDataAvailable()) {
showOutput();
}
}
validateName(data, callback);
validateAddress(data, callback);
validateEmail(data, callback);
The key here is the result object, which starts out as empty. As each field gets validated, it gets added to the result object (by the validation functions, which I've left out in the above snippet). I've used a single callback method, but you could have multiple, say callbackName, callbackAddress, etc. The validation results are processed only if and when the result object has been fully populated, which is checked in isAllDataAvailable.
Hope this helps.
Consider using: https://github.com/asaf/nodejs-model
It will make your life much easier when dealing with validators.

Resources