I have this graph:
And I'm trying to write a query with the "SQL Syntax" of OrientDB v3.0 that start from a Client and follow the read path (the X means: and don't have Have relation to the Client).
I can get the Segments, but I don't find how to walk to the Contact.
The docs have many examples but only walking one path.
I have tried without success these queries:
SELECT FROM (TRAVERSE out("Access").out("Contain") FROM (SELECT #rid FROM Client where myId = 30543) MAXDEPTH 1)
SELECT FROM (
TRAVERSE out("Contain") FROM
(TRAVERSE out("Access") FROM (SELECT #rid FROM Client where myId = 30543) MAXDEPTH 1)
MAXDEPTH 1
)
SELECT out('Access').out("Contain") FROM Client WHERE myId = 30543
Do you have any info to accomplish this traverse?
I'm using the Node.js API:
const pool = await orient.getPool();
const session = await pool.acquire();
logger.info('Running query...');
session.command(`SELECT out('Access').out("Contain") FROM Client WHERE myId = 30543`)
.on('data', (data) => {
if (data.out_Contain && data.out_Contain.delegate) {
logger.info('Segment %s contains %o Contact', data['#rid'].toString(), data.out_Contain.delegate.size);
} else if (data['#rid']) {
logger.info('Segment %s contains %o Contact', data['#rid'].toString(), data);
} else {
logger.info('Data %o', data);
}
})
.on('error', (err) => {
logger.error(err);
})
.on('end', () => {
console.timeEnd('query');
logger.info('End of the stream');
process.emit('SIGINT');
});
logger.debug('Registering SIGINT');
process.once('SIGINT', async () => {
await session.close();
await pool.close();
await orient.stop();
});
Please try using this code:
"select out_{edgeclass}.in from (select expand(out_{edgeclass}.in) from {Vetex} where {condition})"
The match is better for this kind of tasks.
With SELECT:
In the first version, the query is more readable, but it does not use indexes, so it is less optimal in terms of execution time. The second and third use indexes if they exist, (on Person.name or City.name, both in the sub-query), but they're harder to read. Which index they use depends only on the way you write the query.
But match:
the query executor optimizes the query for you, choosing indexes where they exist. Moreover, the query becomes more readable, especially in complex cases, such as multiple nested SELECT queries.
Here the right query:
SELECT EXPAND(contatti)
FROM (
match { class: Client, as: user, where : ( myId = 30543)}
.out('Access')
.out('Contain'){ class:Contact, as:contatti, where: (gender= 'M')},
NOT {as:user} -Have-> {as:contatti}
RETURN DISTINCT contatti LIMIT 1000
)
Related
I'm using the mssql npm module (with Tedious driver) to read/write to Azure Sql database from my node Server. https://www.npmjs.com/package/mssql
All the examples I've found provide an hardcoded example of the query whether to read or write records, like this:
var insertRecordIntoTable = function (callback) {
sql.connect(dbConfig).then(pool => {
return pool.request()
.input('ID', sql.Int, 210)
.input('Name', sql.NVarChar, "John Doe")
.input('EmailAddress', sql.NVarChar, "test#test.com")
.query("INSERT INTO Accounts (ID, Name, EmailAddress) VALUES (#ID, #Name, #EmailAddress)")
}).then(result => {
console.dir(result)
callback(result);
}).catch(err => {
// ... error checks
console.log("Error occured: " + err);
callback(err);
});
}
Obviously, I'd like to write one standard method to write records to any table in the database.
Now I can fetch structure of each table and use that to find how what datatype each field should be from the key in jsonRecord and write something like this:
var insertRecordIntoTable = function (jsonRecord, tableName, callback) {
let arrKeys = jsonRecord.allKeys();
let columnNames = getCommaSeparatedColumnNames(arrKeys);
let valuePlaceholders = getValuePlaceholdersForSql(arrKeys);
sql.connect(dbConfig).then(pool => {
return pool.request()
// how do I write something like this so that dynamic number of fields and values get populated in the query inside this promise.
// I'm open to methods without promise as well.
for(let x=0; x < arrKeys.length; x++){
let key = arrKeys[x];
// .input('ID', sql.Int, 210)
.input(key, getTypeForKey(key, tableName), jsonRecord[ key ] )
}
.query("INSERT INTO " + tableName + " (" + columnNames + ") VALUES (" + valuePlaceholders + ")")
}).then(result => {
console.dir(result)
callback(result);
}).catch(err => {
// ... error checks
console.log("Error occured: " + err);
callback(err);
});
}
function getTypeForKey(key){. // looks up table schema and returns keyType }
function getCommaSeparatedColumnNames(arrKeys){ return arrKeys.join(", "); }
function getValuePlaceholdersForSql(arrKeys){ // write code to append '#' before every key and then join using comma's and return that string }
I'm sure node.js writing to SQL is a fairly common functionality and there may be better ways to achieve what I'm trying to do here. Please feel free to go a different route.
P.S. - Although I should say that I prefer mssql over Tedious package. It just seems better in functionality after going through the documentation in the last several hours.
If you want to interact with your database without creating all the queries by yourself, you can use a query builder like knex to manage the data as objects:
knex('Accounts').insert({ID: 210, Name: "John Doe", EmailAddress: "test#test.com"})
Would be similar to:
insert into `Accounts` (`EmailAddress`, `ID`, `Name`) values ('test#test.com', 210, 'John Doe')
Also I see you are checking types. If you need validation, maybe a complete ORM (I like Objection.js) would be a good choice.
I need to render in a single page several query returns, for example:
1) students per neighborhood
2) student average
app.get('/gerais',(req,res) => {
const client = new Client();
client.connect()
.then(() => {
return client.query('SELECT COUNT(name) studentsperneighborhood,'
+' neigh FROM student INNER JOIN adress ON student.adress_id ='
+'adress.id GROUP BY neigh');
})
.then((results) => {
console.log('results?',results);
res.render('general-info',results);
})
.catch((err) => {
console.log('error',err);
res.send('FAIL');
});
});
how could I modify the return, to return another query?
Try running the queries separately and pass the results of both the queries in different objects.
Then render your page and inject these objects there to use them.
for eg. if you are using ejs file:-
then
for eg:-
connect.query('query1',function(err1, data1){
connect.query('query2',function(err2, data2){
res.render('studentsinfo.ejs',{
studentsperneighborhood: data1,
studentaverage: data2
})})})
Now, you use use these 2 objects to display your data.
I'm new to node(express) and pg-promise, and have not been able to figure out how to add the result of each nested query(loop) into the main json array result query.
I have two tables: Posts and comments.
CREATE TABLE post(
id serial,
content text not null,
linkExterno text,
usuario VARCHAR(50) NOT NULL REFERENCES usuarios(alias) ON UPDATE cascade ON DELETE cascade,
multimedia text,
ubicacation VARCHAR(100),
likes integer default 0,
time VARCHAR default now(),
reported boolean default false,
PRIMARY KEY (id) );
CREATE TABLE comment(
id serial,
idPost integer NOT NULL REFERENCES post(id) ON UPDATE cascade ON DELETE cascade,
acount VARCHAR(50) NOT NULL REFERENCES users(alias) ON UPDATE cascade ON DELETE cascade,
content text NOT NULL,
date date default now(),
PRIMARY KEY (id));
So I want to add the result of each comments to each post and return the posts.
I have this, but doesn't work:
con.task(t => {
return t.any('select *, avatar from post, users where user= $1 and user = alias ORDER BY time DESC LIMIT 10 OFFSET $2', [username, pos])
.then(posts => {
if(posts.length > 0){
for (var post of posts){
post.coments = t.any('select * from comment where idPost = $1 ', post.id);
}
}
});
}).then(posts => {
res.send(posts);
}).catch(error => {
console.log(error);
});
Any suggestions?
PD: I think my question is kind of similar to this one:
get JOIN table as array of results with PostgreSQL/NodeJS
ANSWERS:
Option 1 (best choice):
Making a single query through JSON to psql (JSON query)
See answer by #vitaly-t
OR
Getting the nested data asynchronously using ajax.
Option 2:
function buildTree(t) {
return t.map("select *, avatar from publicacion, usuarios where usuario = $1 and usuario = alias ORDER BY hora DESC LIMIT 10 OFFSET $2", [username, cantidad], posts => {
return t.any('select * from comentario where idPublicacion = $1', posts.id)
.then(coments => {
posts.coments = coments;
console.log(posts.coments);
return posts;
});
}).then(t.batch); // settles the array of generated promises
}
router.get('/publicaciones', function (req, res) {
cantidad = req.query.cantidad || 0; //num de publicaciones que hay
username = req.session.user.alias;
con.task(buildTree)
.then(data => {
res.send(data);
})
.catch(error => {
console.log(error);
});
});
Option 3(async):
try{
var posts = await con.any('select *, avatar from post, users where user = $1 and user = alias ORDER BY time DESC LIMIT 10 OFFSET $2', [username, q])
for (var post of posts){
post.coments = await con.any('select * from comment where idPublictcion = $1', post.id);
}
}catch(e){
console.log(e);
}
I'm the author of pg-promise ;)
con.task(t => {
const a = post => t.any('SELECT * FROM comment WHERE idPost = $1', post.id)
.then(comments => {
post.comments = comments;
return post;
});
return t.map('SELECT *, avatar FROM post, users WHERE user = $1 AND user = alias ORDER BY time DESC LIMIT 10 OFFSET $2', [username, pos], a)
.then(t.batch);
})
.then(posts => {
res.send(posts);
})
.catch(error => {
console.log(error);
});
Also see this question: get JOIN table as array of results with PostgreSQL/NodeJS.
UPDATE
In case you do not want to go all the way with the JSON query approach, then the following will scale much better than the original solution, as we concatenate all child queries, and then execute them as one query:
con.task(async t => {
const posts = await t.any('SELECT *, avatar FROM post, users WHERE user = $1 AND user = alias ORDER BY time DESC LIMIT 10 OFFSET $2', [username, pos]);
const a = post => ({query: 'SELECT * FROM comment WHERE idPost = ${id}', values: post});
const queries = pgp.helpers.concat(posts.map(a));
await t.multi(queries)
.then(comments => {
posts.forEach((p, index) => {
p.comments = comments[index];
});
});
return posts;
})
.then(posts => {
res.send(posts);
})
.catch(error => {
console.log(error);
});
See API:
helpers.concat
Database.multi
If you want structured (nested) data, without having to
A) re-write your sql using json function, or split it out into multiple task queries, or
B) refactor your code to use the API of a heavy ORM
you could check out sql-toolkit. It's a node library built for pg-promise which allows you to write regular native SQL and receive back properly structured (nested) pure business objects. It's strictly an enhancement toolkit on top of pg-promise, and does not seek to abstract out pg-promise (you still set up pg-promise and can use it directly).
For example:
class Article extends BaseDAO {
getBySlug(slug) {
const query = `
SELECT
${Article.getSQLSelectClause()},
${Person.getSQLSelectClause()},
${ArticleTag.getSQLSelectClause()},
${Tag.getSQLSelectClause()}
FROM article
JOIN person
ON article.author_id = person.id
LEFT JOIN article_tags
ON article.id = article_tags.article_id
LEFT JOIN tag
ON article_tags.tag_id = tag.id
WHERE article.slug = $(slug);
`;
return this.one(query, { slug });
// OUTPUT: Article {person: Person, tags: Tags[Tag, Tag, Tag]}
}
The select clause uses the business object "getSQLSelectClause" methods to save tedium in typing the columns, as well as ensure no collisions of names (nothing magical going on, and could just be written out instead).
The this.one is a call into sql-toolkits base DAO class. It is responsible for structuring the flat result records into a nice nested structure.
(Also notice that it is "one" which matches our mental model for the SQL. The DAO methods for one, oneOrNone, many, and any ensure their count against the number of generated top level business objects - not the number of rows the sql expression returns!)
Check out the repository for details on how to set it up on top of pg-promise. (Disclamer, I am the author of sql-toolkit.)
You can use await but it will work sync.
return t.any('select *, avatar from post, users where user= $1 and user = alias ORDER BY time DESC LIMIT 10 OFFSET $2', [username, pos])
.then(posts => {
if(posts.length > 0){
for (var post of posts){
post.coments = await t.any('select * from comment where idPost = ', post.id);
}
}
return posts;
});
Actually i recommend you to use orm tools like bookshelf, knex, typeorm
I have two collections, one contains my static items and other collection contains reverse geocode results for that item. They are matched by id property.
I am writing a script that would fill reverse geocode collection with missing items.
This is my current solution which is super slow, it does:
Get total count of static items
Create read stream from static items collection
Uses find one on reverse geocode collection for each item that comes from the read stream
If items exists, increase counter by 1 and ignore it
If item doesn't exist, fetch it from API, save it to collection and increase counter by 1
When counter is equal total count, it means all items are fetched,
therefore resolve function
function fetchMissingData(){
return new Promise((resolve, reject) => {
const staticData = Global.state.db.collection('static_data')
const googleData = Global.state.db.collection('google_reverse_geocode')
staticData.count((countErr, count) => {
if (countErr) return reject(countErr)
let counter = 0
let fetched = 0
function finishIfReady(){
process.stdout.write(`Progress...(${counter}/${count}), (fetched total: ${fetched})\r`)
if (count === counter) {
resolve({ fetched, counter })
}
}
staticData.find()
.on('data', (hotel) => {
googleData.findOne({ id: hotel.id }, (findErr, geocodedItem) => {
if (findErr) return reject(findErr)
if (geocodedItem) {
counter++
finishIfReady()
} else {
GMClient.reverseGeocode({ latlng: hotel.geo }, (err, response) => {
if (err) return reject(err)
googleData.insertOne({
id: hotel.id,
payload: response,
}).then(() => {
fetched++
counter++
finishIfReady()
}).catch(e => reject(e))
})
}
})
})
.on('error', e => reject(e))
})
})
}
Is there more elegant solution using aggregation framework that would allow me same behavior without O(n^{2}) O(nlogn) complexity?
First, the actual complexity is O(nlogn) because findOne on id use binary search. Second, although there is no way to pass the theory complexity O(nlogn) in this case, there is way to help make your code faster in practice. This is what I would do:
function getIdOfAllGeoData() {
// return an array of existing Geo data IDs
return Global.state.db.collection('google_reverse_geocode')
.find().toArray().map(o => o.id);
}
function getStaticDataMissingGeo(existingGeoDataIds) {
const staticData = Global.state.db.collection('static_data');
return staticData.find({
id: {
$nin: existingGeoDataIds
}
}).toArray();
}
function fetchMissingData() {
const existingGeoDataIds = getIdOfAllGeoData();
const staticDataMissingGeo = getStaticDataMissingGeo(existingGeoDataIds);
// staticDataMissingGeo is all the static that need geo data
// you can loop through this array, get each items geo data and insert to database
// ...
}
Finally, you could use bulk operation to speed thing up, it will be much faster. Also, my mongo related code above may not be correct, consider it as an idea.
I am using NodeJS, PostgreSQL and the amazing pg-promise library. In my case, I want to execute three main queries:
Insert one tweet in the table 'tweets'.
In case there is hashtags in the tweet, insert them into another table 'hashtags'
Them link both tweet and hashtag in a third table 'hashtagmap' (many to many relational table)
Here is a sample of the request's body (JSON):
{
"id":"12344444",
"created_at":"1999-01-08 04:05:06 -8:00",
"userid":"#postman",
"tweet":"This is the first test from postman!",
"coordinates":"",
"favorite_count":"0",
"retweet_count":"2",
"hashtags":{
"0":{
"name":"test",
"relevancetraffic":"f",
"relevancedisaster":"f"
},
"1":{
"name":"postman",
"relevancetraffic":"f",
"relevancedisaster":"f"
},
"2":{
"name":"bestApp",
"relevancetraffic":"f",
"relevancedisaster":"f"
}
}
All the fields above should be included in the table "tweets" besides hashtags, that in turn should be included in the table "hashtags".
Here is the code I am using based on Nested transactions from pg-promise docs inside a NodeJS module. I guess I need nested transactions because I need to know both tweet_id and hashtag_id in order to link them in the hashtagmap table.
// Columns
var tweetCols = ['id','created_at','userid','tweet','coordinates','favorite_count','retweet_count'];
var hashtagCols = ['name','relevancetraffic','relevancedisaster'];
//pgp Column Sets
var cs_tweets = new pgp.helpers.ColumnSet(tweetCols, {table: 'tweets'});
var cs_hashtags = new pgp.helpers.ColumnSet(hashtagCols, {table:'hashtags'});
return{
// Transactions
add: body =>
rep.tx(t => {
return t.one(pgp.helpers.insert(body,cs_tweets)+" ON CONFLICT(id) DO UPDATE SET coordinates = "+body.coordinates+" RETURNING id")
.then(tweet => {
var queries = [];
for(var i = 0; i < body.hashtags.length; i++){
queries.push(
t.tx(t1 => {
return t1.one(pgp.helpers.insert(body.hashtags[i],cs_hashtags) + "ON CONFLICT(name) DO UPDATE SET fool ='f' RETURNING id")
.then(hash =>{
t1.tx(t2 =>{
return t2.none("INSERT INTO hashtagmap(tweetid,hashtagid) VALUES("+tweet.id+","+hash.id+") ON CONFLICT DO NOTHING");
});
});
}));
}
return t.batch(queries);
});
})
}
The problem is with this code I am being able to successfully insert the tweet but nothing happens then. I cannot insert the hashtags nor link the hashtag to the tweets.
Sorry but I am new to coding so I guess I didn't understood how to properly return from the transaction and how to perform this simple task. Hope you can help me.
Thank you in advance.
Jean
Improving on Jean Phelippe's own answer:
// Columns
var tweetCols = ['id', 'created_at', 'userid', 'tweet', 'coordinates', 'favorite_count', 'retweet_count'];
var hashtagCols = ['name', 'relevancetraffic', 'relevancedisaster'];
//pgp Column Sets
var cs_tweets = new pgp.helpers.ColumnSet(tweetCols, {table: 'tweets'});
var cs_hashtags = new pgp.helpers.ColumnSet(hashtagCols, {table: 'hashtags'});
return {
/* Tweets */
// Add a new tweet and update the corresponding hash tags
add: body =>
db.tx(t => {
return t.one(pgp.helpers.insert(body, cs_tweets) + ' ON CONFLICT(id) DO UPDATE SET coordinates = ' + body.coordinates + ' RETURNING id')
.then(tweet => {
var queries = Object.keys(body.hashtags).map((_, idx) => {
return t.one(pgp.helpers.insert(body.hashtags[i], cs_hashtags) + 'ON CONFLICT(name) DO UPDATE SET fool = $1 RETURNING id', 'f')
.then(hash => {
return t.none('INSERT INTO hashtagmap(tweetid, hashtagid) VALUES($1, $2) ON CONFLICT DO NOTHING', [+tweet.id, +hash.id]);
});
});
return t.batch(queries);
});
})
.then(data => {
// transaction was committed;
// data = [null, null,...] as per t.none('INSERT INTO hashtagmap...
})
.catch(error => {
// transaction rolled back
})
},
NOTES:
As per my notes earlier, you must chain all queries, or else you will end up with loose promises
Stay away from nested transactions, unless you understand exactly how they work in PostgreSQL (read this, and specifically the Limitations section).
Avoid manual query formatting, it is not safe, always rely on the library's query formatting.
Unless you are passing the result of transaction somewhere else, you should at least provide the .catch handler.
P.S. For the syntax like +tweet.id, it is the same as parseInt(tweet.id), just shorter, in case those are strings ;)
For those who will face similar problem, I will post the answer.
Firstly, my mistakes:
In the for loop : body.hashtag.length doesn't exist because I am dealing with an object (very basic mistake here). Changed to Object.keys(body.hashtags).length
Why using so many transactions? Following the answer by vitaly-t in: Interdependent Transactions with pg-promise I removed the extra transactions. It's not yet clear for me how you can open one transaction and use the result of one query into another in the same transaction.
Here is the final code:
// Columns
var tweetCols = ['id','created_at','userid','tweet','coordinates','favorite_count','retweet_count'];
var hashtagCols = ['name','relevancetraffic','relevancedisaster'];
//pgp Column Sets
var cs_tweets = new pgp.helpers.ColumnSet(tweetCols, {table: 'tweets'});
var cs_hashtags = new pgp.helpers.ColumnSet(hashtagCols, {table:'hashtags'});
return {
/* Tweets */
// Add a new tweet and update the corresponding hashtags
add: body =>
rep.tx(t => {
return t.one(pgp.helpers.insert(body,cs_tweets)+" ON CONFLICT(id) DO UPDATE SET coordinates = "+body.coordinates+" RETURNING id")
.then(tweet => {
var queries = [];
for(var i = 0; i < Object.keys(body.hashtags).length; i++){
queries.push(
t.one(pgp.helpers.insert(body.hashtags[i],cs_hashtags) + "ON CONFLICT(name) DO UPDATE SET fool ='f' RETURNING id")
.then(hash =>{
t.none("INSERT INTO hashtagmap(tweetid,hashtagid) VALUES("+tweet.id+","+hash.id+") ON CONFLICT DO NOTHING");
})
);
}
return t.batch(queries);
});
}),