I'm using node js, express and postgresql as backend.
This is the approach I used to make a rest API:
exports.schema = function (inputs, res) {
var query = knex('schema')
.orderBy('sch_title', 'asc')
.select();
query.exec(function (err, schemas) {
if(err){
var response = {
message: 'Something went wrong when trying to fetch schemas',
thrownErr: err
};
console.error(response);
res.send(500, response);
}
if(schemas.length === 0){
var message = 'No schemas was found';
console.error(message);
res.send(400, message);
return;
}
res.send(200, schemas);
});
};
It works but after a while postgres logs an error and it's no longer working:
sorry, too man clients already
Do I need a close each request somehow? Could not find any about this in the express docs. What can be wrong?
This error only occurs on production server. Not on developing machine.
Update
The app only brakes in one 'module'. The rest of the app works fine. So it's only some queries that gives the error.
Just keep one connection open for your whole app. The docs shows an example how to do this.
This code goes in your app.js...
var Knex = require('knex');
Knex.knex = Knex.initialize({
client: 'pg',
connection: {
// your connection config
}
});
And when you want to query in your controllers/middlewares...
var knex = require('knex').knex;
exports.schema = function (req, res) {
var query = knex('schema')
.orderBy('sch_title', 'asc')
.select();
// more code...
};
If you place Knex.initialize inside an app.use or app.VERB, it gets called repeatedly for each request thus you'll end up connecting to PG multiple times.
For most cases, you don't need to do an open+query+close for every HTTP request.
Related
My backend gets a request to get records from an Azure SQL db. To manage this requests I'm using Express in Nodejs, and Tedious (to connect to DB). When the request to the appropriate route comes in, Tedious opens the connection with db, queries it, and it should send the response back to frontend.
However, the code responds before I have an answer with from the db, and thus when I go to send the real (the actually desired) response, Express tells me it already sent headers back (the dreaded: 'Cannot set headers after they are sent to the client').
After debugging quite a bit (using several console.log(JSON.stringify(resp.headersSent)); ) to see when was the response actually sent, I noticed that it's sent the moment I connect with Azure (see below).
I'm not sure if I'm missing something (though I already checked the documentation for all those programs quite a bit), but how can I control when the response is sent? Or, is there another way of doing this.
I omitted several of the other routes for brevity. Other routes work fine and thus I know code connects well to Azure db, and frontend does query backend correctly. Help is appreciated. Thank you.
const express = require('express');
const cors = require('cors');
const Connection = require('tedious').Connection;
const Request = require('tedious').Request;
const config = {
authentication: {
options: {
userName: "xxxx",
password: "xxxx"
},
type: 'default'
},
server: "xxxx",
options: {
database: "xxxx",
encrypt: true
}
};
const app = express();
app.use(express.json({type: '*/*'}));
app.use(cors({ origin: '*' }));
app.get("/allproj/", function (req, resp) {
const q = `select Title, Report_Date, Project_Number, Phase_Code, Items_No, PId from projec order by PId desc`;
let ansf = [];
const connection = new Connection(config);
connection.on('connect', (err, connection) => {
if (err) {
console.log(err);
} else { //this is the moment the headers are sent,
//seemingly with positive response from connection
queryItems(q);
}
});
queryItems = (q) => {
request = new Request(q, function (err, rowCount) {
if (err) {
console.log(err);
} else {
console.log(rowCount + ' rows pulled');
connection.close();
}
});
request.on('row', function(columns) {
let ans = [];
columns.forEach(function(column) {
ans.push(column.value);
if (ans.length === 6) { // I know each row is 6 cols long
ansf.push(ans);
ans = [];
}
});
console.log('ansf length: ' + ansf.length);
resp.send({ ansf }); // This is the response I would like to return
});
request.on('done', function(rowCount) {
console.log(rowCount + ' rows returned');
connection.close();
});
connection.execSql(request);
};
resp.redirect("/");
});
app.listen(3000, process.env.IP, function() {
console.log("Started OK...");
});
Remove resp.redirect("/");
As it is already transferring your request to "/" and when control come at resp.send({ansf}), It gives you error.
I searched for a good example of Node+Express+Oracle. I need to read values from an Oracle view to create the response to a GET operation.
Most examples does not use "Express" (my choice of framework) and simply log the database information. Here are some of the best I found : from Express, from Oracle 1, from Oracle 2
Here is the one I preferred to base my code on. I like it because it includes Express, and actual manipulation of the response data. Sadly, it still does not work for me. And I am wondering what the issue is.
I think I understand the asynchronous nature of the JavaScript language (ie: callbacks), and I am beginning to suspect my problem might be tied to : A stupid variable name snafu, My version of Oracle (12c) or the fact that the source is a view (replicated from another schema).
I tried the following code in many other forms (including async functions), and also tried to switch from node 10 to 11... no changes.
Any help would be welcomed.
As of now, the code "works", in the sense that the row data from the database is printed on screen, but the response in Postman (my test tool) is empty, with no http error code... just like the connection would have been cut before replying.
Note the commented, misplaced response code, that returns a response if uncommented (and then crashes the program since the header is written twice).
Here is the code.
It is based on this project/file (and simplifed):
var express = require('express');
var oracledb = require('oracledb');
var app = express();
var connAttrs = {
"user": "user",
"password": "pass",
"connectString": "some_host/SCHEMANAME"
}
// Http Method: GET
// URI : /myObj
app.get('/myObj', function (req, res) {
"use strict";
oracledb.getConnection(connAttrs, function (err, connection) {
if (err) {
// Error connecting to DB
res.set('Content-Type', 'application/json');
res.status(500).send(JSON.stringify({
status: 500,
message: "Error connecting to DB",
detailed_message: err.message
}));
return;
}
// THIS works if uncommented. This is obviously the wrong place to put this.
//res.contentType('application/json').status(200);
//res.send("Test.1.2");
connection.execute("SELECT * FROM SCHEMA.SOMEVIEW", {}, {
outFormat: oracledb.OBJECT // Return the result as Object
}, function (err, result) {
if (err) {
res.set('Content-Type', 'application/json');
res.status(500).send(JSON.stringify({
status: 500,
message: "Error getting the user profile",
detailed_message: err.message
}));
} else {
//log first row... THIS WORKS !
console.log(result.rows[0]);
//This does not work, like if "res" could not be touched here !!!
res.contentType('application/json').status(200);
res.send(JSON.stringify(result.rows[0]));
}
// Release the connection
connection.release(
function (err) {
if (err) {
console.error(err.message);
} else {
console.log("GET /myObj: Connection released");
}
});
});
});
});
var server = app.listen(3000, function () {
"use strict";
var host = server.address().address,
port = server.address().port;
console.log(' Server is listening at http://%s:%s', host, port);
});
Here is a representation of the console:
{ field1: 'some data',
field2: 'some more data' }
GET /myObj : Connection released
And here is the behaviour in postman:
ARGGG ! It was a simple timeout ! Postman DOES mention it as a possibility (see image), but I would like the program to make it flash it bright letters !!!
Closing the question. Sorry for the disturbance.
I am using Promised-Mongo to connect MongoDB with Promises from NodeJS backend code. It worked fine, until I enabled MongoDB's client access control. When I run this code, I get "could not authenticate" message":
var pmongo = require('promised-mongo').compatible();
var db = pmongo('myusername:mypassword#localhost/mydb', ['candidates']);
db.candidates.save(req.body)
.then(function () {
// never reached here
})
.catch(function (e) {
// it reached here, where e.message says "could not authenticate"
});
Pure MongoDB code (i.e. no Promises...) works fine:
var mongodb = require('mongodb');
var uri = 'mongodb://myusername:mypassword#localhost/mydb';
mongodb.MongoClient.connect(uri, function (err, db) {
if (err) {
// never reached here
}
var candidates = db.collection('candidates');
candidates.insert(req.body, function(err, result) {
if (err) {
// never reached here
}
res.send('{result: success}');
});
});
Any idea?
Per several issues in the github repository (see here and here) it looks like using this library with authentication is totally broken. Per the second link, most people seem to be wrapping the official library with a promise via something like promisify, bluebird, or a thin custom wrapper.
I've been trying to get a simple nodejs API with CRUD functionality working. I'm using express, and 'monk' package for communicating with mongodb. I've successfully pulled data with a GET request.
I'm unable to get a post() function to work. I'm able to insert new documents to mongo when the insert request is called from a GET request that inserts a doc every time it's called. However, no matter what I do, with or without an actual insert request, my post is returning a 400.
Here's my route file:
var express = require('express');
var router = express.Router();
/* GET hours page. (for users to submit hours) */
router.get('/', function (req, res) {
var db = req.db;
var collection = db.get('entries');
collection.find({}, /*{limit:20}, */function (err, docs) {
if (err) {
console.log('couldn\'t load entries');
}
res.json(docs);
});
});
/* POST hours page. (for users to submit hours) */
router.post('/', function (req, res) {
if (!(req.body.job || req.body.code || req.body.hours)) {
handleError(res, 'Invalid user input', 'Must complete input', 400);
}
var db = req.db;
var collection = db.get('entries');
var newEntry = req.body;
collection.insert(newEntry, function (err, docs) {
if (err){
handleError(res, err.message, 'Failed to create new entry');
}
res.json(docs);
});
});
module.exports = router;
I really don't know why ever single post request is returning a 400. I'm thinking it's a problem with my main file, but it has barely been altered from the initial express generated file.
You if condition is wrong. !(req.body.job || req.body.code || req.body.hours) should be !(req.body.job && req.body.code && req.body.hours).
And are you sure you have really post anything? Check the Content-Type of your requests, which should be application/x-www-form-urlencoded;.
I recommand using supertest to test your app. The usage is very simple.
import request = require("supertest");
import should = require("should");
import app = require("../youApp");
describe("POST /foo", () => {
it("should post something", done => {
request(app)
.post("/foo")
.send({job: "my job", code: "...", "hours"})
.expect(200, done);
});
});
Somthing more
replace var with const .
use arrow function instead of function(req, res).
I am new to Angular JS and node.js/express framework. I am working on a small application which uses angular and express frameworks. I have express app running with couple of end points. One for POST action and one for GET action. I am using node-mysql module to store and fetch from mysql database.
This application is running on my laptop.
angular.js client:
controller
function ItemController($scope, storageService) {
$scope.savedItems = storageService.savedItems();
alert($scope.savedItems);
}
service
myApp.service('storageService', function($resource) {
var Item = $resource('http://localhost\\:3000/item/:id',
{
id:'#id',
},
{
query: {
method: 'GET',
isArray: true
}
);
this.savedItems = function() {
Item.query(function(data){
//alert(data);
return data;
});
}
Express server with mysql database:
...
app.get('/item', item.list);
...
items.js
---------
exports.list = function(req, res) {
var sql = 'select * from item';
connect: function() {
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'localhost',
user : 'admin',
database : 'test'
});
return connection;
},
query: function(sql) {
var connection = this.connect();
return connection.query(sql, function(err, results) {
if (err) throw err;
return results;
});
},
res.send(results);
};
When I send static array of items (json) from server, $scope.savedItems() is getting populated.
but when I access items in database, even though server is returning items, $scope.savedItems in client is empty. Using $http directly did not help either.
I read async nature of $resource and $http from angular.js documentation and I am still missing something or doing something wrong.
Thanks in advance and appreciate your help.
This has to do with the async nature of angular $resource.
$scope.savedItems = storageService.savedItems();
Returns immediately an empty array, which is populated after the data returns. Your alert($scope.savedItems); will therefore show only an empty array. If you look at your $scope.savedItems a little bit later you would see that it has been populated. If you would like to use the data just after it has been returned you can use a callback:
$scope.savedItems = storageService.savedItems(function(result) {alert(result); });
Just as a quick note. You could also watch the savedItems.
function ItemController($scope, storageService) {
$scope.savedItems = storageService.savedItems();
$scope.$watch(function() {
return $scope.savedItems;
}, function(newValue, oldValue) {
if (typeof newValue !== 'undefined') {
// Do something cool
}
},
true);
}
i suspect, node is not returning mysql results. The fact that it works for static files and not for mysql rules out issues with angular. Can you add firebug logs for the http call or chrome developer logs. This can shed more light on the matter