pg-promise UTF connection string - node.js

pg-promise does not understand UTF passwords?.. I can't make it work with them. Tried on linux and osx, postgres 9.3, 9.5 - seems to be not specific to versions. Looked into code. pg-promise uses pg, which uses pg-connect-string which is build based on back pg. Can't find the the root of the problem. Please help.
code to reproduce:
MacBook-Air:js vao$ cat 2.js
var pgp = require("pg-promise")();
var cs = 'postgresql://utf:утф#127.0.0.1:5433/a';
var db = pgp(cs);
db.connect()
.then(function (obj) {
obj.done(); // success, release the connection;
})
.catch(function (error) {
console.log("ERROR:", error.message || error);
});
console.log(cs);
returns:
MacBook-Air:js vao$ node 2.js
postgresql://utf:утф#127.0.0.1:5433/a
ERROR: password authentication failed for user "utf"
Using same connection string with psql:
MacBook-Air:js vao$ psql 'postgresql://utf:утф#127.0.0.1:5433/a'
psql (9.5.3)
Type "help" for help.
a=> \q
Trying bad password deliberately with same connection string:
MacBook-Air:js vao$ psql 'postgresql://utf:утфWrongPassword#127.0.0.1:5433/a'
psql: FATAL: password authentication failed for user "utf"
MacBook-Air:js vao$

As the author of pg-promise, I'd like at least to offer some guidance here on where to look, not the exact answer, as I've never dealt with UTF passwords myself.
pg-promise uses node-postgres, which in turn uses pg-connection-string to parse the connection. If there is an issue, then it is most likely inside pg-connection-string.
I would recommend trying Unicode notation that relies on %. Maybe it will work. Maybe not, I've never tried. Also the following question was never answered: Node.js postgres UTF connect string, which isn't reassuring either.
Sorry, I cannot be of more help at present.

Related

Queries from Node to Postgres DB not in UTF8

For a project, I do have to code a server in JS (Node) which return elements from a database (Postgres) on a Windows computer.
Everything is "working fine" in a way that connections are OK and elements are returned, however, I do have an encoding issue. My elements are in French, hence, some letters, such as "éàç", are not correctly "seen" by the server (I printed the result). The console is working with UTF8 (I tested it) so the problem is elsewhere.
It seems to me that this issue comes from the query to the database (HTML and JS from the client side are OK and support these letters). Here is a part of my code which returns the wrong things.
var client = new pg.Client({
user: 'xx',
password: 'yy',
database: 'phrases',
host: 'localhost',
client_encoding: 'utf8',
port: '5432'
});
client.connect();
client.query('SELECT id,mots from phrase WHERE langue=$1 ORDER BY anno limit 20;', ['fr'])
.then(
function(data){
text = data.rows[0].mots; //<= return the text needed
console.log(text);
}
);
I'm almost sure the problem is not from this part of code but from the database side however I may be wrong (and thus I can't see why). From the PSQL console, I do have:
show CLIENT_ENCODING; > UTF8
show SERVER_ENCODING; > UTF8
And from the database:
Nom | PropriÚtaire | Encodage | Collationnement | Type caract. | Droits d'acc
-----------+-------------+----------+----------------------------+--------------------+-----------------------
phrases | postgres | UTF8 | French_France.1252 | French_France.1252 |
I tried from a python program (print a simple request to the DB) to see if results were the same, they are; letters aren't well printed. Hence I'm sure the problem comes from the DB and are linked with the encoding but no way to find something working.
Does anyone have an idea?
Best,
I know this is an old post and the author is probably never going to see this, but I think that an answer should be written for others with this issue.
I had a very similar problem as the one you describe. Apparently, the encoding is correct, but when you do a SELECT * FROM table query some letters are rendered as gibberish. This might be because Windows does not automatically use UTF-8 encoding.
Before entering your database via cmd you could run the command chcp 65001. This will work but is generally considered inefficient and dangerous. A better solution would be to do the following:
Search for the app Run and input intl.cpl
Go to the Administrative Tab
Click on Change System Locale
Check the box in the bottom left.
Credit to this answer by #mklement0

Python psycopg2 executing select pg_notify doesn't work

This is my very first question at StackOverflow so if i am doing something wrong please be gentle.
I'm struggling with executing SELECT pg_notify from Python script. It seems that it doesn't work at all.
My NodeJS server is listening 'testnotify' channel using pg-promise. I'm putting this just for completeness because it is working.
db.connect({direct: true})
.then(sco => {
sco.client.on('notification', data => {
console.log('Received:', data);
});
return sco.none('LISTEN $1~', 'testnotify');
})
.catch(error => {
console.log('Error:', error);
});
My Python script should rise notification after series of successful db operations.
I'm doing this like that
conn = psycopg2.connect(conn_string)
cur = conn.cursor()
cur.execute("SELECT pg_notify('%s', '%s');" % ('testnotify', 'blabla'))
or like that
query = "SELECT pg_notify('testnotify', 'blabla');"
print(query)
cur.execute(query)
I've tried in similar way with NOTIFY testnotify, 'blabla' and nothing works. Nothing happen at NodeJS side.
But when i copy result of print(query) from Python console and execute it directly from PostgreSQL then it is working like a charm.
I don't understand what's wrong with my code.
I'm using psycopg2 2.7.5, PostgreSQL 10, Node 10 LTS, pg-promise at Windows 10.
Side note: This is not a problem with Node because it is working when pg_notify or notify is raised using trigger at source table in postgresql or when executing notification as regular sql query at db. It is not working only when i'm trying to raise notification from python script.
After two days of juggling with this i think that this is something obvious and stupid but i can't see it. Or maybe it is just impossible...
Please help.
Oh my... i just found solution... even two ways to solve this.
The clue was in psycopg documentation... obvious huh?
to send notification using
cur.execute("SELECT pg_notify('%s', '%s');" % ('testnotify', 'blabla'))
one have to set connection to autocommit like that
import psycopg2
import psycopg2.extensions
conn.set_isolation_level(psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)
or simply if you don't want autocommit then after doing
cur.execute("SELECT pg_notify('%s', '%s');" % ('testnotify', 'blabla'))
you have to commit it
conn.commit()
aaand now Node is receiving notifications from postgresql via python

rabbitmq createConnection throws error [Error: Connection ended: possibly due to an authentication failure.]

Hi I trying to connect with rabbitmq using node.js
but getting error.
[Error: Connection ended: possibly due to an authentication failure.]
var MessageQueue = AMQP.createConnection({'host' => '127.0.0.1', 'port' => '5672', 'login' => 'guest', 'password' => 'guest'});
MessageQueue.on('ready', function () {
console.log('success);
}).on('error', function (e) {
console.log(e);
});
This is the error you have pointed out:
{handshake_error,tuning,0,{exit,{amqp_error,not_allowed,"negotiated channel_max = 0 is higher than the maximum allowed value (2047)",'connection.tune'}
This means that your client library is trying to use 0 as the value for channel_max, which means "infinite". RabbitMQ's limit out of the box is 2047 for safety reasons.
When reporting problems, you must tell people what software you are using, and the versions. In this case, I have no idea what javascript library you are using.
In the case of the amqp.node library, you can specify a non-zero channelMax value.
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
Finally I have fixed the issues. It is working fine now. Thank you guys for helping me out.
I have changed channel_max = 0 to 20 and now it is working fine for me.

Mongo / Mongoose query works in Mongo shell but not in Node (on EC2 Ubuntu image)

I'm having a hard time determining why this query works in the Mongo shell but not when I try to hit the server endpoint from the browser.
Shell command:
db.playlists.find({keywords: {"$in" : ["featured"]}}).limit(5)
Server code:
app.get('/getFeaturedPlaylists', (req, res) => {
let query = {
keywords: { "$in" : ["featured"]}
}
database.Playlist.find({query}).limit(5).exec(function(err, data) {
err ? console.log(err) : res.send({body: JSON.stringify(data)});
})
})
Expected result is an array of playlist objects:
{"body": "[{\"songList\":[{\"songId\":522,\"bpm\":45},{\"songId\":310,\"bpm\":45},{\"songId\":199,\"bpm\":90},{\"songId\":170,\"bpm\":150},{\"songId\":17,\"bpm\":150},{\"songId\":722,\"bpm\":45},{\"songId\":767,\"bpm\":90},{\"songId\":754,\"bpm\":120},{\"songId\":844,\"bpm\":150},{\"songId\":422,\"bpm\":120},{\"songId\":638,\"bpm\":90},{\"songId\":243,\"bpm\":150},{\"songId\":524,\"bpm\":90},{\"songId\":575,\"bpm\":120},{\"songId\":143,\"bpm\":120}],\"shuffledQueue\":[],\"keywords\":[\"bluegrass\",\"featured\"],\"_id\":\"5a76188fde76d97bda5e342e\",\"playlistId\":2,\"playlistName\":\"Playlist2\"},{\"songList\":[{\"songId\":401,\"bpm\":45},{\"songId\":536,\"bpm\":150},{\"songId\":982,\"bpm\":60},{\"songId\":812,\"bpm\":60},{\"songId\":466,\"bpm\":150}],\"shuffledQueue\":[],\"keywords\":[\"featured\",\"bluegrass\"],\"_id\":\"5a76188fde76d97bda5e3432\",\"playlistId\":6,\"playlistName\":\"Playlist6\"},{\"songList\":[{\"songId\":248,\"bpm\":120},{\"songId\":84,\"bpm\":45},{\"songId\":53,\"bpm\":60},{\"songId\":749,\"bpm\":45},{\"songId\":811,\"bpm\":120}],\"shuffledQueue\":[],\"keywords\":[\"featured\",\"hip-hop\"],\"_id\":\"5a76188fde76d97bda5e3433\",\"playlistId\":7,\"playlistName\":\"Playlist7\"},{\"songList\":[{\"songId\":894,\"bpm\":150},{\"songId\":190,\"bpm\":60},{\"songId\":235,\"bpm\":60},{\"songId\":632,\"bpm\":60},{\"songId\":970,\"bpm\":60},{\"songId\":475,\"bpm\":90},{\"songId\":304,\"bpm\":60},{\"songId\":816,\"bpm\":120},{\"songId\":613,\"bpm\":150},{\"songId\":310,\"bpm\":45},{\"songId\":599,\"bpm\":45},{\"songId\":91,\"bpm\":90},{\"songId\":650,\"bpm\":120},{\"songId\":219,\"bpm\":45},{\"songId\":290,\"bpm\":90}],\"shuffledQueue\":[],\"keywords\":[\"hip-hop\",\"featured\"],\"_id\":\"5a76188fde76d97bda5e3436\",\"playlistId\":10,\"playlistName\":\"Playlist10\"},{\"songList\":[{\"songId\":778,\"bpm\":60},{\"songId\":652,\"bpm\":60},{\"songId\":792,\"bpm\":90},{\"songId\":353,\"bpm\":120},{\"songId\":528,\"bpm\":60},{\"songId\":887,\"bpm\":120},{\"songId\":287,\"bpm\":90},{\"songId\":926,\"bpm\":120},{\"songId\":671,\"bpm\":90}],\"shuffledQueue\":[],\"keywords\":[\"featured\",\"reggae\"],\"_id\":\"5a76188fde76d97bda5e3437\",\"playlistId\":11,\"playlistName\":\"Playlist11\"}]"}
Actual result is an empty array:
{"body":"[]"}
Even more peculiar, this issue only arose when I moved the server and database to EC2 instances. Using the exact same code and data set on my local machine, it responds as I expect it to (via Postman or Chrome -- this is how I got the "expected response" snippet above). Additionally, all of the other server endpoints pointing at this database work, but they're only looking for individual playlists.
Any insights into this would be greatly appreciated, and of course I'm happy to provide any more details as needed.
Thanks!
Here's the issue,
database.Playlist.find({query})
You're not passing the query object but rather an empty object { } with {query}.
Do this instead,
database.Playlist.find(query)
Hope this solves the problem.

Error while executing a Ms Access db through nodejs

I am trying to access a Ms Access 2007 db trough nodejs in Windows 7, but even this simple query won't work. I receive the following message in the command prompt (this is a translation, the original is in portuguese): "Operation not allowed when object is closed". Anybody have any answers? The javascript code is written below:
var ADODB = require('node-adodb');
var connection = ADODB.open('Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\\teste\\dbteste.accdb;Persist Security Info=False;');
ADODB.debug = true;
connection
.query('SELECT * FROM [Tabela];')
.on('done', function (data){
console.log('Result:'.green.bold, data);
})
.on('fail', function (data){
});
Thanks!
Try to change the connectionString :
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\\teste\\dbteste.accdb;Persist Security Info=False;
by
Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\\teste\\dbteste.accdb;Persist Security Info=False;
Also, check if the .accdb path is right.
Moreover, add ADODB.encoding = 'iso-8859-15'; for portuguese encoding.
I'm not sure but you may replace [Tabela] by simply Tabela. The ; at the sql query end is not necessary.
I encountered this issue too. This seems to be a bug in node-adodb module.(last seen at 8/31/2015)
In the linked page you can see that two people have encountered the same problem and the developer of node-adodb has asked them to upload the example( on 7/17/2015)

Resources