Prepared statements with Node-MSSQL - node.js

I want to store API data in a SQL Server database with Node.js. It works already with a normal INSERT query as you see in the first code below.
server.route({
method: 'POST',
path: '/',
handler: async(request, h) => {
try {
await pool.query("INSERT INTO mytable(device,data,stationId,rssi,unix_timestamp) VALUES('"+request.payload.device+"','"+request.payload.data+"','"+request.payload.station+"',
'"+request.payload.rssi+"','"+request.payload.time+"')");
return h.response('Callback received').code(200);
}
catch (err) {
console.log("SQL Err", err.stack);
return 'Error';
}
}
});
Now my I want to improve the code a little bit to avoid SQL injection and to have a better overview.
Therefore I want to use prepared statements like described in this documentation: https://www.npmjs.com/package/mssql#prepared-statement
So far I've managed to do this here:
server.route({
method: 'POST',
path: '/',
handler: async(request, h) => {
const ps = new sql.PreparedStatement(pool)
try {
ps.input('device', sql.VarChar(10))
ps.input('data', sql.VarChar(24))
ps.input('station', sql.NChar(10))
ps.input('rssi', sql.Float)
ps.input('time', sql.Int)
await ps.prepare('INSERT INTO mytable(device,data,stationId,rssi,unix_timestamp) VALUES(#device,#data,#station,#rssi,#time)');
try {
await ps.execute(
{ device: request.payload.device },
{ data: request.payload.data },
{ station: request.payload.station },
{ rssi: request.payload.rssi },
{ time: request.payload.time }
)
} finally {
await ps.unprepare();
}
return h.response('Callback received').code(200);
}
catch (err) {
console.log("SQL Err", err.stack);
return 'Error';
}
}
});
And the following error occurs:
at Parser.emit (events.js:223:5)
at Parser.<anonymous> (C:\Users\AW\sqltest\node_modules\tedious\lib\token\token-stream-parser.js:37:14)
at Parser.emit (events.js:223:5)
at addChunk (C:\Users\AW\sqltest\node_modules\readable-stream\lib\_stream_readable.js:297:12)
at readableAddChunk (C:\Users\AW\sqltest\node_modules\readable-stream\lib\_stream_readable.js:279:11)
at Parser.Readable.push (C:\Users\AW\sqltest\node_modules\readable-stream\lib\_stream_readable.js:240:10)
at Parser.Transform.push (C:\Users\AW\sqltest\node_modules\readable-stream\lib\_stream_transform.js:139:32)
This is an example of the JSON data I get from the API:
{
"device":"887B53",
"data":"4660000000000062b4a8",
"station":"1B2C"
"rssi":"-123",
"time":"1585258718"
}
I hope someone can help me with that.

It appears that the error is occurring at the following line:
ps.input('station', sql.NChar(10))
I would debug the value that is being assigned here and see if it fits the schema, or if it's undefined.
Additionally on a side note if you are worried about readability and SQL injections, consider using an ORM such as Sequelize.

Related

Adding firebase firestore entry failed : 7 PERMISSION_DENIED

I'm trying to use a Firebase Cloud Function to create a document within the Firestore database from my Node js environment with Express js, but it fails with below error on the function logs.
Error: Process exited with code 16
at process.on.code (/layers/google.nodejs.functions-framework/functions-framework/node_modules/#google-cloud/functions-framework/build/src/invoker.js:275:22)
at process.emit (events.js:198:13)
at process.EventEmitter.emit (domain.js:448:20)
at process.exit (internal/process/per_thread.js:168:15)
at Object.sendCrashResponse (/layers/google.nodejs.functions-framework/functions-framework/node_modules/#google-cloud/functions-framework/build/src/logger.js:37:9)
at process.on.err (/layers/google.nodejs.functions-framework/functions-framework/node_modules/#google-cloud/functions-framework/build/src/invoker.js:271:22)
at process.emit (events.js:198:13)
at process.EventEmitter.emit (domain.js:448:20)
at emitPromiseRejectionWarnings (internal/process/promises.js:140:18)
at process._tickCallback (internal/process/next_tick.js:69:34)
firebase.ts file :
import * as admin from 'firebase-admin'
import * as functions from 'firebase-functions'
admin.initializeApp({
credential: admin.credential.cert({
privateKey: functions.config().private.key.replace(/\\n/g, '\n'),
projectId: functions.config().project.id,
clientEmail: functions.config().client.email
}),
databaseURL: 'https://app-id.firebaseio.com'
})
const db = admin.firestore()
export { admin, db }
controller.ts :
import { Response } from 'express'
import { db } from './config/firebase'
type EntryType = {
title: string,
text: string,
}
type Request = {
body: EntryType,
params: { entryId: string }
}
const addEntry = async (req: Request, res: Response) => {
const { title, text } = req.body
try {
const entry = db.collection('entries').doc()
const entryObject = {
id: entry.id,
title,
text,
}
await entry.set(entryObject).catch(error => {
return res.status(400).json({
status: 'error',
message: error.message
})
})
return res.status(200).json({
status: 'success',
message: 'entry added successfully',
data: entryObject
})
} catch(error) {
console.log(error);
return res.status(500).json(error.message)
}
}
Im receiving below response from this trigger :
{
"status": "error",
"message": "7 PERMISSION_DENIED: Invalid project number: 113102533737774060828"
}
Is this related to the Cloud Firestore rules in the Google cloud? Im fairly new to Google cloud functions.
Any suggestions would be appreciated.
This typically means that the credentials you're using are not for the project you're trying to use them on.
Check your functions.config().private.key to ensure it is indeed for the project you run this code on.

Socket Hangup Error In Node JS On Force API Timeout

I am using request module in Node JS (v8.12) to call a third party API. Since the API is not very reliable and due to lack of better option I am timing out the call after 2 seconds in case if there is no response from the API. But in doing so it creates a socket hang up error. Below is the code used and stack trace
const options = {
url: resource_url,
rejectUnauthorized: false,
timeout: 2000,
method: 'GET',
headers: {
'content-Type': 'application/json',
}
};
return new Promise(function (resolve, reject) {
request(options, function (err, res, body) {
if (!err) {
resolve(JSON.parse(body.data));
} else {
if (err.code === 'ETIMEDOUT' || err.code == 'ESOCKETTIMEDOUT') {
resolve(someOldData);
} else {
resolve(someOldData);
}
}
});
});
Error: socket hang up
at createHangUpError (_http_client.js:331:15)
at TLSSocket.socketCloseListener (_http_client.js:363:23)
at scope.activate (<trace-log-base-path>/dd-trace/packages/dd-trace/src/scope/base.js:54:19)
at Scope._activate (<trace-log-base-path>/dd-trace/packages/dd-trace/src/scope/async_hooks.js:51:14)
at Scope.activate (<trace-log-base-path>/dd-trace/packages/dd-trace/src/scope/base.js:12:19)
at TLSSocket.bound (<trace-log-base-path>/dd-trace/packages/dd-trace/src/scope/base.js:53:20)
at emitOne (events.js:121:20)
at TLSSocket.emit (events.js:211:7)
at _handle.close (net.js:554:12)
at TCP.done [as _onclose] (_tls_wrap.js:356:7)
After doing a bit of reading and research I found this article pointing out a similar issue so I switched to http module as mentioned in one of the solution in the article. But switching to http module also did not resolve the issue. Below is code implementation using http and stack trace.
let responseData;
const requestOptions = {
hostname: resource_host,
path: resource_path,
method: 'GET',
timeout: 2000,
};
return new Promise((resolve, reject) => {
const requestObject = http.request(requestOptions, (responseObj) => {
responseObj.setEncoding('utf8');
responseObj.on('data', (body) => {
responseData = body;
});
responseObj.on('end', () => {
resolve(responseData);
});
});
requestObject.on('error', (err) => {
responseData = someOldData;
resolve(responseData);
});
requestObject.on('timeout', () => {
responseData = someOldData;
requestObject.abort();
});
requestObject.end();
});
Error: socket hang up
at connResetException (internal/errors.js:608:14)
at Socket.socketCloseListener (_http_client.js:400:25)
at <trace-log-base-path>/dd-trace/packages/dd-trace/src/scope/base.js:54:19
at Scope._activate (<trace-log-base-path>/dd-trace/packages/dd-trace/src/scope/async_hooks.js:51:14)
at Scope.activate (<trace-log-base-path>/dd-trace/packages/dd-trace/src/scope/base.js:12:19)
at Socket.bound (<trace-log-base-path>/dd-trace/packages/dd-trace/src/scope/base.js:53:20)
at Socket.emit (events.js:322:22)
at Socket.EventEmitter.emit (domain.js:482:12)
at TCP.<anonymous> (net.js:672:12)
I went through multiple SO post and various other resources over the web, but I am unable to resolve this issue.
Could it be because of the third party, because I also tried to reproduce the issue by creating a dummy server which sleeps for some time after the request is fired and timing out that request but was unable to reproduce the issue.
I'll be very grateful for any help in this regards.
Removing requestObject.abort() in timeout event block when using http module resolves this issue.

Parametized INSERT query with node-mssql

I want to parametrize an insert query with node.js for SQL Server. Unfortunately it will not work and I don't really know if it's a Node module issue or a syntax failure.
Code:
server.route({
method: 'POST',
path: '/',
handler: async (request, h) => {
try {
await pool.query("INSERT INTO sigfoxmessages(device,data,station,rssi,unix_timestamp) VALUES($1,$2,$3,$4,$5))"
[request.payload.device, request.payload.data, request.payload.station, request.payload.rssi, request.payload.time]);
return h.response('Callback received').code(200);
}
catch (err) {
console.log("SQL Err", err.stack);
return 'Error';
}
}
});
Error:
at exports.Manager.execute (C:\Users\A\sqltest\node_modules#hapi\hapi\lib\toolkit.js:60: 33)
at Object.internals.handler (C:\Users\A\sqltest\node_modules#hapi\hapi\lib\handler.js:46 :48)
at exports.execute (C:\Users\A\sqltest\node_modules#hapi\hapi\lib\handler.js:31:36)
at Request._lifecycle (C:\Users\A\sqltest\node_modules#hapi\hapi\lib\request.js:365:68)
at processTicksAndRejections (internal/process/task_queues.js:94:5)
at async Request._execute (C:\Users\A\sqltest\node_modules#hapi\hapi\lib\request.js:274: 9)
Used node modules:
hapi/hapi 19.0.5
mssql: 6.0.1
Does anyone have an idea or or a suggestion?
According to the documentation for mssql you can use es6 template literals in you INSERT statement.
pool.query`INSERT INTO sigfoxmessages (device,data,station,rssi,unix_timestamp) VALUES(${request.payload.device}, ${request.payload.data}, ${request.payload.station}, ${request.payload.rssi}, ${request.payload.time}))`
Docs:
https://www.npmjs.com/package/mssql

Saving data to Postgres from AWS Lambda

I'm building a lambda function that is supposed to save a game feedback, like a performance grade, into my Postgres database, which is in AWS RDS.
I'm using NodeJS typescript and the function is kinda working, but in a strange way.
I made an API Gateway so I can POST data to the URL to the lambda process it and save it, the thing is, when I POST the data the function seems to process it until it reaches a max limit of connected clients, and than it seems to lose the other clients'data.
Another problem is that every time I POST data I get a response saying that there was a Internal Server Error and with a 'X-Cache→Error from cloudfront' header. For a GET request I figured it out that it was giving me this response because the format of the response was incorrect, but in this case I fixed the response format and still get this problem...
Sometimes I get a timeout response.
My function's code:
import { APIGatewayEvent, Callback, Context, Handler } from "aws-lambda";
import { QueryConfig, Client, Pool, PoolConfig } from "pg";
export const insert: Handler = async (
event: APIGatewayEvent,
context: Context,
cb: Callback
) => {
// context.callbackWaitsForEmptyEventLoop = false;
const config: PoolConfig = {
user: process.env.PG_USER,
host: process.env.PG_HOST,
database: process.env.PG_DB,
password: process.env.PG_PASS,
port: parseInt(process.env.PG_PORT),
idleTimeoutMillis: 0,
max: 10000
};
const pool = new Pool(config);
let postdata = event.body || event;
console.log("POST DATA:", postdata);
if (typeof postdata == "string") {
postdata = JSON.parse(postdata);
}
let query: QueryConfig = <QueryConfig>{
name: "get_all_questions",
text:
"INSERT INTO gamefeedback (gameid, userid, presenterstars, gamestars) VALUES ($1, $2, $3, $4);",
values: [
parseInt(postdata["game_id"]),
postdata["user_id"],
parseInt(postdata["presenter_stars"]),
parseInt(postdata["game_stars"])
]
};
console.log("Before Connect");
let con = await pool.connect();
let res = await con.query(query);
console.log("res.rowCount:", res.rowCount);
if (res.rowCount != 1) {
cb(new Error("Error saving the feedback."), {
statusCode: 400,
body: JSON.stringify({
message: "Error saving data!"
})
});
}
cb(null, {
statusCode: 200,
body: JSON.stringify({
message: "Saved successfully!"
})
});
console.log("The End");
};
Than the log from CloudWatch error with max number of clients connected looks like this:
2018-08-03T15:56:04.326Z b6307573-9735-11e8-a541-950f760c0aa5 (node:1) UnhandledPromiseRejectionWarning: error: sorry, too many clients already
at u.parseE (/var/task/webpack:/node_modules/pg/lib/connection.js:553:1)
at u.parseMessage (/var/task/webpack:/node_modules/pg/lib/connection.js:378:1)
at Socket.<anonymous> (/var/task/webpack:/node_modules/pg/lib/connection.js:119:1)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at addChunk (_stream_readable.js:263:12)
at readableAddChunk (_stream_readable.js:250:11)
at Socket.Readable.push (_stream_readable.js:208:10)
at TCP.onread (net.js:607:20)
Can any of you guys help me with this strange problem?
Thanks
Well for one thing you need to put creating a pool above the handler, like so:
const config: PoolConfig = {
user: process.env.PG_USER,
...
};
const pool = new Pool(config);
export const insert: Handler = async (
event: APIGatewayEvent,
context: Context,
cb: Callback
) => {
..etc
The way you have it you are creating a pool on every invocation. If you create the pool outside the handler it gives Lambda a chance to share the pool between invocations.

How should I do to make an elasticSearch , search query from Node.js correctly?

I am stuck in a problem and can not figure out the solution.
I want to use an elasticSearch query from my nodejs.
The problem is, I can make it work from postman, but not from node.
http://user:psd#my_domain:9200/ra_autocomplete/search
And from my nodejs app :
var elasticsearch = require('elasticsearch');
var client = new elasticsearch.Client({
host: 'my_domain',
port : 9200,
protocol : 'http',
auth : 'user:psd',
maxRetries : 2
});
And then,
client.search({
index: "ra_autocomplete",
body: {
query: {
m_prefix : {
r_n : {
query : my_var
}
}
}
}
} , function(err, res) {
console.log(err);
console.log(res);
});
I get this error :
Error: Not Found
at respond (my_path\node_modules\elasticsearch\src\lib\transport.js:307:15)
at checkRespForFailure
(my_path\node_modules\elasticsearch\src\lib\transport.js:266:7)
at HttpConnector.
(my_path\node_modules\elasticsearch\src\lib\connectors\http.js:159:7)
at IncomingMessage.bound
(my_path\node_modules\lodash\dist\lodash.js:729:21)
at emitNone (events.js:111:20)
at IncomingMessage.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1056:12)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickCallback (internal/process/next_tick.js:180:9)
status: 404,
displayName: 'NotFound',
message: 'Not Found',
path: '/roads_autocomplete/_search',
query: {},
body: '{"query":{"m_prefix":{"r_n":{"query":"montexte a analyser"}}}}',
statusCode: 404,
response: '\r\n404 Not Found\r\n\r\n404 Not Found\r\nnginx/1.10.3 (Ubuntu)\r\n\r\n\r\n',
toString: [Function],
toJSON: [Function] }
Any help would be appreciated, the problem is, when I try to make it with postman, it goes well.
Thank you.
I think the problem is that you are not connecting to elastic search node.
Add your port and domain to into one line and then start by running a simple query.
var elasticsearch = require('elasticsearch');
var client = new elasticsearch.Client({
host: 'localhost:9200',
log: 'trace'
});
client.search({
index: 'products',
type: 'product',
body: {
query: {
bool: {
}
}
}
}).then((body) => {
return body;
}, (error) => {
console.trace(error.message);
});

Resources