Equivalent sequelize operation for given sql server query - node.js

SELECT *
FROM (SELECT *, ROW_NUMBER() OVER ( ORDER BY No_ ) AS RowNum
FROM Item) DerivedTable
WHERE RowNum >= 501 AND RowNum <= 501 + ( 5 - 1 );
I think the older sql server versions do no support FETCH ROWS and NEXT ROWS which is equivalent to OFFSET and LIMIT in mysql, the above query seems the only way to apply that logic.
How can sequelize implement the above query, which creates a virtual table "DerivedTable" with a column "RowNum" that is used in the WHERE clause.
Is there any other method to do this in sequelize, maybe including raw query or anything else?

It seems you are not alone with this issue. With SQL Server 2012, you can just use:
Model
.findAndCountAll({
where: {
title: {
$like: 'foo%'
}
},
offset: 10,
limit: 2
})
.then(function(result) {
console.log(result.count);
console.log(result.rows);
});
However since you are on an earlier version it seems you are stick with having to hand write the query.
Something like this:
var theQuery = 'declare #rowsPerPage as bigint; '+
'declare #pageNum as bigint;'+
'set #rowsPerPage='+rowsPerPage+'; '+
'set #pageNum='+page+'; '+
'With SQLPaging As ( '+
'Select Top(#rowsPerPage * #pageNum) ROW_NUMBER() OVER (ORDER BY ID asc) '+
'as resultNum, * '+
'FROM myTableName)'+
'select * from SQLPaging with (nolock) where resultNum > ((#pageNum - 1) * #rowsPerPage);';
sequelize.query(theQuery)
.spread(function(result) {
console.log("Good old paginated results: ", result);
});
});
see this and this

Related

Inserting multiple rows into SQL Server from Node.js

I am working on a project that will upload some records to SQL Server from a node.js program. Right now, this is my approach (inside an async function):
con = await sql.connect(`mssql://${SQL.user}:${SQL.password}#${SQL.server}/${SQL.database}?encrypt=true`);
for (r of RECORDS) {
columns = `([column1], [column2], [column3])`;
values = `(#col1, #col2, #col3)`;
await con
.request()
.input("col1", sql.Int, r.col1)
.input("col2", sql.VarChar, r.col2)
.input("col3", sql.VarChar, r.col3)
.query(`INSERT INTO [dbo].[table1] ${columns} VALUES ${values}`);
}
Where records is an array of objects in the form:
RECORDS = [
{ col1: 1, col2: "asd", col3: "A" },
{ col1: 2, col2: "qwerty", col3: "B" },
// ...
];
This code works, nevertheless, I have the feeling that it is not efficient at all. I have an upload of around 4k records and it takes roughly 10 minutes, it does not look good.
I believe if I can create a single query - instead of wrapping single inserts inside a for loop - with all the record values it will be faster, and I know there is a syntax for reaching that in SQL:
INSERT INTO table1 (column1, column2, column3) VALUES (1, "asd", "A"), (2, "qwerty", "B"), (...);
However I cannot find any documentation from mssql module for node on how to prepare the parameterized inputs to do everything in a single transaction.
Can anyone guide me into the right direction?
Thanks in advance.
Also, very similar to the bulk insert, you can use a table valued parameter.
sql.connect("mssql://${SQL.user}:${SQL.password}#${SQL.server}/${SQL.database}?encrypt=true")
.then(() => {
const table = new sql.Table();
table.columns.add('col1', sql.Int);
table.columns.add('col2', sql.VarChar(20));
table.columns.add('col3', sql.VarChar(20));
// add data
table.rows.add(1, 'asd', 'A');
table.rows.add(2, 'qwerty', 'B');
const request = new sql.Request();
request.input('table1', table);
request.execute('procMyProcedure', function (err, recordsets, returnValue) {
console.dir(JSON.stringify(recordsets[0][0]));
res.end(JSON.stringify(recordsets[0][0]));
});
});
And then for the SQL side, create a user defined table type
CREATE TYPE typeMyType AS TABLE
(
Col1 int,
Col2 varchar(20),
Col3 varchar(20)
)
And then use this in the stored procedure
CREATE PROCEDURE procMyProcedure
#table1 typeMyType READONLY
AS
BEGIN
INSERT INTO table1 (Col1, Col2, Col3)
SELECT Col1, Col2, Col3
FROM #MyRecords
END
This gives you more control over the data and lets you do more with the data in sql before you actually insert.
As pointed out by #JoaquinAlvarez, bulk insert should be used as replied here: Bulk inserting with Node mssql package
For my case, the code was like:
return await sql.connect(`mssql://${SQL.user}:${SQL.password}#${SQL.server}/${SQL.database}?encrypt=true`).then(() => {
table = new sql.Table("table1");
table.create = true;
table.columns.add("column1", sql.Int, { nullable: false });
table.columns.add("column2", sql.VarChar, { length: Infinity, nullable: true });
table.columns.add("column3", sql.VarChar(250), { nullable: true });
// add here rows to insert into the table
for (r of RECORDS) {
table.rows.add(r.col1, r.col2, r.col3);
}
return new sql.Request().bulk(table);
});
The SQL data types have to match (obviously) the column type of the existing table table1. Note the case of column2, which is a column defined in SQL as varchar(max).
Thanks Joaquin! I went down on the time significantly from 10 minutes to a few seconds

How to get all records from the last 7 days sequelize

I am trying to get all records from a mysql Database with sequelize and I have tried following approaches:
shops.findAndCountAll({
where: {
createdAt: {
[Op.gte]: moment().subtract(7, 'days').toDate()
}
}
})
and when I use this, I get the error:
ReferenceError: moment is not defined
So I tried this approach:
shops.findAndCountAll({
where: {
createdAt: {
[Op.gte]: Sequelize.literal('NOW() - INTERVAL "7d"'),
}
}
})
But I get the following error
code: 'ER_PARSE_ERROR',
errno: 1064,
sqlState: '42000',
sqlMessage: "You have an error in your SQL syntax; check the manual that corresponds to
your MySQL server version for the right syntax to use near '' at line 1",
sql: "SELECT count(*) AS `count` FROM `shop` AS `shops` WHERE `shops`.`createdAt` >= NOW() - INTERVAL '7d';"
},
sql: "SELECT count(*) AS `count` FROM `shop` AS `shops` WHERE `shops`.`createdAt` >= NOW() - INTERVAL '7d';"
}
How can I fix this issue. I do not mind which of the approaches I use, as long as I get it to work.
Thank you in advance
You are not importing moment ; therefore moment is not defined.
try
const moment = require('moment') //<es6
or
import moment from 'moment'
Using DATE_ADD() or DATE_SUB()
SELECT * FROM Table_Name
WHERE connect_time >= DATE_ADD(CURDATE(),INTERVAL -7 DAY);
or
SELECT * FROM Table_Name
WHERE connect_time >= DATE_SUB(CURDATE(),INTERVAL 7 DAY);
Without those functions, you can also do
SELECT * FROM Table_Name
WHERE connect_time >= (CURDATE() + INTERVAL -7 DAY);
or
SELECT * FROM Table_Name
WHERE connect_time >= (CURDATE() - INTERVAL 7 DAY);

How to pass order info to sequelize.js raw query?

In sequelize, I can use
my_table.findAll({ order: [['datetime', 'desc']] })
to query data and order by a column. But when I try to use parameterized raw query like:
var input_parameters = {order_column: 'datetime', order: 'desc'};
sequelize.query('select * from my_table order by :order_column :order', { replacements: input_parameters, type: models.sequelize.QueryTypes.SELECT });
It can't return the correct order because
the order info asc/desc is escaped in the query, the final prepared query is like 'select * from my_table order by 'datetime' 'desc''.
Is there a way to pass order info to raw parameterized query?
This might not be the sequelize way, but...what if:
let order_column = 'something';
let order = 'DESC';
sequelize.query(`select * from my_table order by ${order_column} ${order}`, { type: models.sequelize.QueryTypes.SELECT });
UPDATE:
This is the right answer
await sequelize.query(
'SELECT * FROM projects ORDER BY ? ?',
{
replacements: ['something', 'desc'],
type: QueryTypes.SELECT,
}
);
This way sequelize still protects you from sql injection.
I wondered samething. But I think there's no options in raw query.
So I usually define class methods in model to use method much sequelize-like follows.
/**
* usage :
*
* model.findSomething({
* where: whereCondition,
* limit: limit,
* offset: offset,
* order: [
* ['col1', 'asc'],
* ['col2', 'desc']
* ]})
*/
model.findSomething = function ({where, limit, offset, order}) {
let sql = 'SELECT col1, col2 FROM some_table '
... (build where conditions)
// build order conditions
if (order && order.length > 0) {
let orderBy = ' ORDER BY '
for (let i = 0; i < order.length; i++) {
if (order[i].length > 0) { // [column] or [column, asc/desc]
orderBy += (i > 0 ? ', ' : ' ') + order[i].join(' ')
}
}
sql += orderBy
} else {
sql += ` ORDER BY comment_group, comment_depth, comments.comment_id`
}
... (build limit and offset)
}
Before you call sequelize.query, just build the sql statement first.
Too late answer, but I hope this let help you.
let order_column = 'something';
let order = 'DESC';
sequelize.query('select * from my_table order by " + order_column +' ' order, { replacements: input_parameters, type: models.sequelize.QueryTypes.SELECT })

Using nodejs 'cassandra-driver' prepared query with IN query

This is a potential newbie question but I couldn't find an answer out in the wild.
I'm currently building a simple user event log of this form (some attributes elided to keep this simple):
CREATE TABLE events.by_hour (
level tinyint, /* 60 = FATAL, 10 = TRACE */
hour int, /* in YYYYMMDDHH format */
insertion_time timeuuid,
userid TEXT,
message TEXT,
PRIMARY KEY ((type,hour),userid,insertion_time))
WITH CLUSTERING ORDER BY (userid ASC, insertion_time ASC);
I would like to make a prepared query to fetch all events optionally filtered by a given user over some set of levels ideally over the course of several hours.
I can easily build a query that queries a single level and hour:
var cassandra = require('cassandra-driver');
var client = new cassandra.Client({contactPoints: ['127.0.0.1']});
var query
= 'SELECT * FROM events.by_hour
WHERE level IN (?) and hour IN (?) and userid = ?;';
var options = [ 10, 2016032117,'user_uuid' ];
client.execute(query, options, {
prepare: true
}, function (err, result) {
// Do stuff...
next(err);
});
This works fine for a single hour and level. I would love to specify multiple hours and levels in the above query but playing with the code I'm unable to get that to work either by specifying the set as a string or as an array of values.
I think I'm forced to do something like this:
Build the query based upon the number of levels and hours are needed:
// for 1 level and 2 hours:
query = 'SELECT * FROM events.by_hour
WHERE level IN (?) and hour IN (?,?) and userid = ?;';
options = [ 10, 2016032117,2016032118,'user_uuid' ];
client.execute(query, options, {
prepare: true
}, function (err, result) {
// Do stuff...
next(err);
});
// for 2 levels and 2 hours:
query = 'SELECT * FROM events.by_hour
WHERE level IN (?,?) and hour IN (?,?) and userid = ?;';
options = [ 10, 20, 2016032117,2016032118,'user_uuid' ];
client.execute(query, options, {
prepare: true
}, function (err, result) {
// Do stuff...
next(err);
});
I don't love this. BUT I can still get the benefits of prepared queries even here since we can just pass in 'prepared: true'. It feels like there should be a better way... but I'm not sure what I'm trying to do is even possible with single prepared query.
Can anyone lend some wisdom here?
Thanks!
You should use the IN operator followed by the query marker (no parenthesis) and provide the parameter for the query marker as an array:
const query = 'SELECT * FROM events.by_hour WHERE level IN ? and ' +
'hour IN ? and userid = ?';
const params = [ [ 10, 20, 30] , [ 2016032117, 2016032118 ],'user_uuid' ];
client.execute(query, params, { prepare: true }, callback);

How to insert dynamic values to select query (for a date range )in Cassandra using node js

var result = 'select message from people.users where event_time > ? and event_time < ? allow filtering';
client.execute(result,[startdate,enddate], function(err,result)
This is not working its showing :
GET /favicon.ico 500 17.785 ms - 386
{ [ResponseError: Expected 8 or 0 byte long for date (10)]
name: 'ResponseError',
message: 'Expected 8 or 0 byte long for date (10)',
info: 'Represents an error message from the server',
code: 8704,
query: 'select message from people.users where event_time > ? and event_time <
? allow filtering' }
There is a mismatch between the type expected by Cassandra (timestamp) and the value provided (it looks like it is not Ecmascript Date).
It should be something like:
var startDate = new Date('2015-01-01');
var endDate = new Date();
var query = 'select message from people.users where event_time > ?' +
' and event_time < ? allow filtering';
client.execute(query, [startDate, endDate], { prepare: true}, callback);
Also, you should use the prepare flag for best performance and for accurate type mapping between Ecmascript and Cassandra.

Resources