NodeJS "synchronous" function call returns [object Promise] - node.js

I wanted to create a platform to learn the power of NodeJS, so I arranged a web application that uses SQLite. I wanted to make the web app show a random post, so I made the web app select one:
var post = db.run("SELECT post_name FROM posts ORDER BY RANDOM() LIMIT 3;")
When I try to use post to get the entire row that it's results are included in:
var postrow = db.all("SELECT * FROM posts WHERE post_name = ${post}") // return object promise
As you can see, it will return [object Promise], causing a database error since there is no column named like that. But this really confuses me because the column is post_name, post is supposed to be the value, not the column. I have tried a lot (if not all) of the methods I could find, and it still would return [object Promise] no matter what.
I am using NodeJS v18.2.
My full code is:
app.get("/home", function(req,res){
var post1 = db.run("SELECT post_name FROM posts ORDER BY RANDOM() LIMIT 3;").then(function(value) {
return value;
});
res.render(__dirname + "/public/index.html", {
post1: post1,
postowned: db.run(`SELECT ownedby FROM posts WHERE post_name = ${post1}`)
})})
Edit: The definition of db is:
dbWrapper
.open({
filename: dbFile,
driver: sqlite3.Database
})
.then(async dBase => {
db = dBase;
try {
if (!exists) {
await db.run(
"CREATE TABLE posts ( post_name VARCHAR(20), post_text TEXT, ownedby VARCHAR(15), cdate TIMESTAMP );"
);
}
console.log(await db.all("SELECT post_name from post"));
} catch (dbError) {
console.error(dbError);
} });

db.run and db.all are both asynchronous functions, so they will return [object Promise]. Nonetheless, it is possible to change the value to the actual response by using db.runs optional callback parameter, The code below selects a random post:
var posts = []
db.run("SELECT * FROM posts ORDER BY RANDOM() LIMIT 3;", function(err, row) {
if (row.post_name != null){
// if the post name is not null then push it to posts
posts.push({ title:row.post_name, text:row.post_text, date:row.cdate, owned:row.ownedby})
}
})
// the columns will be available in posts as 0, 1, it goes on as more posts are selected
It is possible to only select columns that you want by using the comma (for instance SELECT post_name, post_text FROM posts)
Since app.get is a synchronous function, and you can't call it with async, this does not use await, but this would still work as it won't give the promise response.

Related

How to get a specific item from JSON

I'm new to node.js and DialogFlow. I am using DynamoDB to store data and I'm creating skills on Google. I'm trying to write a code to retrieve a specific item on that table.
I've got it working to show all items where ID is equal = 1, but how would I make it so I can just get attribute 'name'?
My idea is a user provides an id then the code will retrieve name where id was 1 and store that as a variable and use agent.add('hello $(name)'); to display it back as speech to the user.
function readdata(agent){
let dbread = new aws.DynamoDB.DocumentClient();
const id = agent.parameters.id;
let read = function(){
var parameters = {
TableName:"Dynamodb",
Key:{
"id":id
}
};
dbread.get(parameters, function(err,data){
if(err){
console.log("error",JSON.stringify(data,null,2));
}else{
console.log("success",JSON.stringify(data,null,2));
}
});
agent.add(`hello ${name}`);
};
read();
}
Once you have the data back from the get() call, the data object will contain an Item attribute. The value of this attribute will be another object that contains the attribute/value pairs for this record, or be empty if the record isn't found.
The debugging you have in place that shows JSON.stringify(data) should show this.
Assuming you knew all the fields were there, you could do something like
const name = data.Item.name;
a more robust way using current JavaScript would be to make sure everything was assigned, otherwise return undefined at any point. So something like this would work
const name = data && data.Item && data.Item.name;
However - you will have a problem doing this with Dialogflow
You don't show which Dialogflow library you're using, but most of them require you to return a Promise to indicate that it needs to wait for asynchronous calls (such as the call to DynamoDB) to complete. You're using get() with a callback function instead of a Promise. So you need to do one of the following:
Wrap the call in a Promise
Since get() returns an AWS.Request you can use the promise() method of this to get a Promise that you can return and which has then portions that generate the response - similar to how you're doing your callbacks now.
Under this scheme, your call might look something like this (untested):
return dbread.get(parameters).promise()
.then( data => {
console.log("success",JSON.stringify(data,null,2));
const name = data && data.Item && data.Item.name;
if( name ){
agent.add( `Hello ${name}` );
} else {
agent.add( "I don't know who you are." );
}
})
.catch( err => {
console.log("error",JSON.stringify(data,null,2));
agent.add( "There was an error" );
});

Multple SQL queries in Node with oracledb

I'm new to Node and am having problems reading from Oracle.
I have the basic examples all set up and can issue basic queries, and process the results etc..
The problem I'm having is that I need to;
Execute one query (Q1)
For each item in the results of Q1 I need to execute a second query (Q2)
I need to combine the results of Q1 and Q2s into an array to return as a promise
I am struggling to find an example where I can perform #2 - call the same query multiple times for each item returned from Q1, using the same connection which was used for Q1.
My code is below - I first perform a read, then iterate through the results storing connection.execute objects which I then run via the Promise.all line - the result of which I just output as I want to get this working before I code the logic to combine the results of Q1 and Q2.
When I run this via mocha, the results of don't contain any data - I see the column headings but no data.
So what am I missing here?
// placeholder for the connection
let conn;
// return case list array
var caseList = [];
var queryList = [];
return new Promise((resolve, reject) => {
// retrieve connection
oracledb.getConnection({
user: dbconfig.user,
password: dbconfig.password,
connectString: dbconfig.connectString
}) // the connection is returned as a promise
.then(connection => {
console.log('Connected to the DB!');
// assign connection
conn = connection;
// execute statement
return connection.execute(
`select caseid, casereference, startdate from caseheader inner join orgobjectlink on caseheader.ownerorgobjectlinkid = orgobjectlink.orgobjectlinkid where orgobjectlink.username = :username`,
[params.username], {
outFormat: oracledb.OBJECT // set the output format to be object
}
);
})
.then(result => {
// iterate around rows
result.rows.forEach(row => {
var caseObj = {
caseID: row.CASEID,
reference: row.CASEREFERENCE,
dateAssigned: moment(row.STARTDATE).format('YYYY-MM-DD'),
username: params.username,
}
caseList.push(caseObj);
console.log(caseObj.caseID)
queryList.push(conn.execute(`select concernroleid, concernrolename from concernrole inner join caseparticipantrole on concernrole.concernroleid = caseparticipantrole.participantroleid where caseparticipantrole.caseid = :caseID and (caseparticipantrole.typecode = 'PRI' or caseparticipantrole.typecode = 'MEM')`,
[caseObj.caseID], {
outFormat: oracledb.OBJECT
}));
});
// build up queries
return Promise.all(queryList).then(results => {
console.log(results);
Promise.resolve(results);
}, err => {
console.log(err);
});
}).then({
if(conn){
console.log("Closing DB connection");
conn.close();
}
}).catch(err => {
console.log('Error', err);
});
});
Promise.all will not work for you as you want to use a single connection and as mentioned previously a connection will only do one thing at a time anyway. To solve this problem using promises, you'd have to build up and unwind a promise chain. I can show you an example, but it's nasty - probably better to just forget I mentioned it.
A better option would be to go into a simple for loop using async/await. I can show you can example of that too but again, I think this is the wrong move. We call this row by row fetching (a.k.a slow by slow).
It's likely the best solution for you will be to take the results from the first query and build up an array. Then execute the second query using one of these options to process the array. https://oracle.github.io/node-oracledb/doc/api.html#sqlwherein
You'll need to include the caseid column in the select clause and perhaps even order by that column so that post-processing of the result set is simplified in Node.js.
This solution has the potential to greatly improve performance and resource utilization, but that has to be balanced against the amount of data you have, the resources, etc. I could probably show you an example of this too, but it will take a bit longer and I'd want to get some more info from you to ensure we're on the right path.
One problem is the Promise.all().then... function doesn't return anything (and doesn't need the additional resolve()). The way to get this sorted is build small, testable, promise returning functions, and test them individually.
Starting simply, write a mocha test to connect to the database...
function connect() {
return oracledb.getConnection({
user: dbconfig.user,
password: dbconfig.password,
connectString: dbconfig.connectString
});
}
Here's one that can run a command on the db. Test this with a simple query that you know will return some results.
function executeCmd(connection, cmd, params) {
return connection.execute(cmd, params, { outFormat: oracledb.OBJECT });
}
With just these two (and one more) we can outline a simple function that does the job: connect to the database, run a select, process each result asynchronously, then disconnect.
function connectAndQuery(username) {
let connection;
return connect().then(result => {
connection = result;
let cmd = `select caseid, casereference, startdate from caseheader inner join orgobjectlink on caseheader.ownerorgobjectlinkid = orgobjectlink.orgobjectlinkid where orgobjectlink.username = :username`;
return executeCmd(connection, cmd, [username]);
}).then(result => {
let promises = result.rows.map(row => processCaseRow(connection, row, username));
return Promise.all(promises);
}).then(result => {
// result should be an array of caseObj's
return connection.close().then(() => result);
});
}
The last thing to build and test is a promise-returning function which processes a row from the main function above.
I had to take some liberty with this, but I think the objective is -- given a row representing a "case" -- build a case object, including a collection of "concernedRoles" that can be queried with the caseID. (that last bit was my idea, but you can build a separate collection if you like)
// return a promise that resolves to an object with the following properties...
// caseID, reference, dateAssigned, username, concernedRoles
// get concernedRoles by querying the db
function processCaseRow(connection, row, username) {
var caseObj = {
caseID: row.CASEID,
reference: row.CASEREFERENCE,
dateAssigned: moment(row.STARTDATE).format('YYYY-MM-DD'),
username: username
}
let cmd = `select concernroleid, concernrolename from concernrole inner join caseparticipantrole on concernrole.concernroleid = caseparticipantrole.participantroleid where caseparticipantrole.caseid = :caseID and (caseparticipantrole.typecode = 'PRI' or caseparticipantrole.typecode = 'MEM')`;
return executeCmd(connection, cmd, row.CASEID).then(result => {
caseObj.concernedRole = result
return caseObj
})
}

MongoDB returning a null, but query works separately

In a post function, I am trying to retrieve the nth activity of a user (since I have a dropdown that return the index number of the activity). When I run the query
collection.find({'local.email':req.user.local.email},
{'local.activities':{$slice : [currActivity,1]}});
I receive the correct activity object in Robo3T.
But, when I call the same query in Node inside a post function, it returns an undefined.
app.post('/addlog',function(req,res){
var currActivity = req.body.curAct;
var score = req.body.score;
var comment = req.body.reason;
mongoose.connect('mongodb://****:****#ds044907.mlab.com:44907/intraspect',function (err, database) {
if (err)
throw err
else
{
db = database;
var collection = db.collection('users');
var retrievedAct = collection.find({'local.email':req.user.local.email},
{'local.activities':{$slice : [currActivity,1]}}).toArray().then(console.log(retrievedAct));
if (retrievedAct.length > 0) { printjson (retrievedAct[0]); }
console.log(currActivity);
console.log(retrievedAct[0]);
// console.log(req.body.newAct);
collection.update({'local.activities.name':retrievedAct[0]},
{$push: {'local.activities.log' : {
comments: comment,
score: score,
log_time: Date.now()
}}})
.then(function(){
res.redirect('/homepage');
})
.catch(function() {
console.log('Error');
});
}
});
});
I checked that the currActivity variable does infact contain the integer value for the nth activity.
If you want the result of collection.find().toArray(), as specified in the docs, you have two options:
Passing a callback to .toArray() like you did with mongoose.connect()
Using the Promise that it returns if you don't pass a callback
Now you are doing neither of them.
Also, you are mixing callback style and Promises in your code. I recommend you unificate your code. If you are using a Node.js version bigger than 8, using async/await could be nice, it makes it simpler.

Google Cloud Datastore, how to query for more results

Straight and simple, I have the following function, using Google Cloud Datastore Node.js API:
fetchAll(query, result=[], queryCursor=null) {
this.debug(`datastoreService.fetchAll, queryCursor=${queryCursor}`);
if (queryCursor !== null) {
query.start(queryCursor);
}
return this.datastore.runQuery(query)
.then( (results) => {
result=result.concat(results[0]);
if (results[1].moreResults === _datastore.NO_MORE_RESULTS) {
return result;
} else {
this.debug(`results[1] = `, results[1]);
this.debug(`fetch next with queryCursor=${results[1].endCursor}`);
return this.fetchAll(query, result, results[1].endCursor);
}
});
}
The Datastore API object is in the variable this.datastore;
The goal of this function is to fetch all results for a given query, notwithstanding any limits on the number of items returned per single runQuery call.
I have not yet found out about any definite hard limits imposed by the Datastore API on this, and the documentation seems somewhat opaque on this point, but I only noticed that I always get
results[1] = { moreResults: 'MORE_RESULTS_AFTER_LIMIT' },
indicating that there are still more results to be fetched, and the results[1].endCursor remains stuck on constant value that is passed on again on each iteration.
So, given some simple query that I plug into this function, I just go on running the query iteratively, setting the query start cursor (by doing query.start(queryCursor);) to the endCursor obtained in the result of the previous query. And my hope is, obviously, to obtain the next bunch of results on each successive query in this iteration. But I always get the same value for results[1].endCursor. My question is: Why?
Conceptually, I cannot see a difference to this example given in the Google Documentation:
// By default, google-cloud-node will automatically paginate through all of
// the results that match a query. However, this sample implements manual
// pagination using limits and cursor tokens.
function runPageQuery (pageCursor) {
let query = datastore.createQuery('Task')
.limit(pageSize);
if (pageCursor) {
query = query.start(pageCursor);
}
return datastore.runQuery(query)
.then((results) => {
const entities = results[0];
const info = results[1];
if (info.moreResults !== Datastore.NO_MORE_RESULTS) {
// If there are more results to retrieve, the end cursor is
// automatically set on `info`. To get this value directly, access
// the `endCursor` property.
return runPageQuery(info.endCursor)
.then((results) => {
// Concatenate entities
results[0] = entities.concat(results[0]);
return results;
});
}
return [entities, info];
});
}
(except for the fact, that I don't specify a limit on the size of the query result by myself, which I have also tried, by setting it to 1000, which does not change anything.)
Why does my code run into this infinite loop, stuck on each step at the same "endCursor"? And how do I correct this?
Also, what is the hard limit on the number of results obtained per call of datastore.runQuery()? I have not found this information in the Google Datastore documentation thus far.
Thanks.
Looking at the API documentation for the Node.js client library for Datastore there is a section on that page titled "Paginating Records" that may help you. Here's a direct copy of the code snippet from the section:
var express = require('express');
var app = express();
var NUM_RESULTS_PER_PAGE = 15;
app.get('/contacts', function(req, res) {
var query = datastore.createQuery('Contacts')
.limit(NUM_RESULTS_PER_PAGE);
if (req.query.nextPageCursor) {
query.start(req.query.nextPageCursor);
}
datastore.runQuery(query, function(err, entities, info) {
if (err) {
// Error handling omitted.
return;
}
// Respond to the front end with the contacts and the cursoring token
// from the query we just ran.
var frontEndResponse = {
contacts: entities
};
// Check if more results may exist.
if (info.moreResults !== datastore.NO_MORE_RESULTS) {
frontEndResponse.nextPageCursor = info.endCursor;
}
res.render('contacts', frontEndResponse);
});
});
Maybe you can try using one of the other syntax options (instead of Promises). The runQuery method can take a callback function as an argument, and that callback's parameters include explicit references to the entities array and the info object (which has the endCursor as a property).
And there are limits and quotas imposed on calls to the Datastore API as well. Here are links to official documentation that address them in detail:
Limits
Quotas

Node.js and Redis

I am trying to link up a redis database with a Node.js application I am building to be able to store comments about items. I'm using the node_redis library to handle the connection. When I attempt to retrieve the comments out of the database however only "[true]" is returned. For testing purposes I have stuffed everything into one method and I have hardcoded the values in, but I still receive "[true]".
exports.getComment = function (id){
var comments = new Array();
rc.hmset("hosts", "mjr", "1", "another", "23", "home", "1234");
comments.push(rc.hgetall("hosts", function (err, obj) {
var comment = new Array();
if(err){
comment.push("Error");
} else {
comment.push(obj);
}
return comment;
}));
return comments;
}
Updated the code according to the tutorial and here is the result:
Retrieving the comment:
exports.getComment = function (id, callback){
rc.hgetall(id, callback);
}
Adding the comment:
exports.addComment = function (id, area, content, author){
//add comment into the database
rc.hmset("comment",
"id", id,
"area", area,
"content", content,
"author" , author,
function(error, result) {
if (error) res.send('Error: ' + error);
});
//returns nothing
};
Code to render:
var a = [];
require('../lib/annotations').addComment("comment");
require('../lib/annotations').getComment("comment", function(comment){
a.push(comment)
});
res.json(a);
Node.js is asynchronous. Which means it asynchronously does the redis stuff, and then gets the result back in the callback function.
I suggest you read this tutorial and fully understand it before getting further: http://howtonode.org/node-redis-fun
Basically, this way won't work:
function getComments( id ) {
var comments = redis.some( action );
return comments;
}
But it has to be this way:
function getComments( id, callback ) {
redis.some( action, callback );
}
This way, you use the API like this:
getComments( '1', function( results ) {
// results are available!
} );
The problem lays within the actual Redis-Node library when the call to addComment is made as it is below.
require('../lib/annotations').getComment("comment", function(comment){
a.push(comment)
});
This call is missing an argument in the callback function. The first argument is the error report which should return null if everything is ok, the second is the actual data. So it should be structured like the call below.
require('../lib/annotations').getComment("comment", function(comment){
a.push(err, comment)
});

Resources