Queries from Node to Postgres DB not in UTF8 - node.js

For a project, I do have to code a server in JS (Node) which return elements from a database (Postgres) on a Windows computer.
Everything is "working fine" in a way that connections are OK and elements are returned, however, I do have an encoding issue. My elements are in French, hence, some letters, such as "éàç", are not correctly "seen" by the server (I printed the result). The console is working with UTF8 (I tested it) so the problem is elsewhere.
It seems to me that this issue comes from the query to the database (HTML and JS from the client side are OK and support these letters). Here is a part of my code which returns the wrong things.
var client = new pg.Client({
user: 'xx',
password: 'yy',
database: 'phrases',
host: 'localhost',
client_encoding: 'utf8',
port: '5432'
});
client.connect();
client.query('SELECT id,mots from phrase WHERE langue=$1 ORDER BY anno limit 20;', ['fr'])
.then(
function(data){
text = data.rows[0].mots; //<= return the text needed
console.log(text);
}
);
I'm almost sure the problem is not from this part of code but from the database side however I may be wrong (and thus I can't see why). From the PSQL console, I do have:
show CLIENT_ENCODING; > UTF8
show SERVER_ENCODING; > UTF8
And from the database:
Nom | PropriÚtaire | Encodage | Collationnement | Type caract. | Droits d'acc
-----------+-------------+----------+----------------------------+--------------------+-----------------------
phrases | postgres | UTF8 | French_France.1252 | French_France.1252 |
I tried from a python program (print a simple request to the DB) to see if results were the same, they are; letters aren't well printed. Hence I'm sure the problem comes from the DB and are linked with the encoding but no way to find something working.
Does anyone have an idea?
Best,

I know this is an old post and the author is probably never going to see this, but I think that an answer should be written for others with this issue.
I had a very similar problem as the one you describe. Apparently, the encoding is correct, but when you do a SELECT * FROM table query some letters are rendered as gibberish. This might be because Windows does not automatically use UTF-8 encoding.
Before entering your database via cmd you could run the command chcp 65001. This will work but is generally considered inefficient and dangerous. A better solution would be to do the following:
Search for the app Run and input intl.cpl
Go to the Administrative Tab
Click on Change System Locale
Check the box in the bottom left.
Credit to this answer by #mklement0

Related

Text in Bash terminal getting overwritten! Using JS, Node.js (npms are: inquirer, console.table, and mysql)

Short 10sec video of what is happening: https://drive.google.com/file/d/1YZccegry36sZIPxTawGjaQ4Sexw5zGpZ/view
I have a CLI app that asks a user for a selection, then returns a response from a mysql database. The CLI app is run in node.js and prompts questions with Inquirer.
However, after returning the table of information, the next prompt overwrites the table data, making it mostly unreadable. It should appear on its own lines beneath the rest of the data, not overlap. The functions that gather and return the data are asynchronous (they must be in order to loop), but I have tried it with just a short list of standard synchronous functions for testing purposes, and the same problem exists. I have tried it with and without console.table, and the prompt still overwrites the response, as a console table or object list.
I have enabled checkwinsize in Bash with
shopt -s checkwinsize
And it still persists.
Is it Bash? Is it a problem with Inquirer?
I was having this issue as well. In my .then method of my prompt I was using switch and case to determine what to console log depending on the users selection. Then at the bottom I had an if statement checking to if they selected 'Finish' if they didn't I called the prompt function again, which I named 'questionUser'.
That's where I was having the issue, calling questionUser again was overriding my console logs.
What I did was I wrapped the questionUser call in a setTimeout like this:
if(data.option !== 'Finish'){
setTimeout(() => {
questionUser();
}, 1000);
}
This worked for me and allowed my console log data to not get deleted.
Hopefully if anyone else is having this issue this helps, I couldn't find an answer to this specific question anywhere.

cannot get baseboard serial number in nodejs

I am running ubuntu in vmware. I am trying to get the baseboard serial number.
var si = require("systeminformation");
console.log(si.baseboard().serial);
It return undefined. Is the problem in my code? Or the problem is ubuntu is running in vmware?
You can see here in the systeminformation source that it's reading the file /sys/devices/virtual/dmi/id/board_serial (if running dmidecode -t 2 2>/dev/null fails).
If you cat /sys/devices/virtual/dmi/id/board_serial in your shell (if it even exists), what do you get?
If it's empty or non-existent, then the data isn't provided by your environment.
To complete AKX answer, si.baseboard() returns a Promise (since v3), so you have to do something like this:
si.baseboard().then(el => console.log(el.serial))
or, if you want to stick with callback syntax
si.baseboard(el => { console.log(el.serial) })
Note for me, it returns an empty string if I launch the script as a regular user. I can display the serial number only if I launch it as root.
I find that I can use serial-number to get an unique number even on virtual machine.
var serialNumber = require('serial-number');
serialNumber(function (err, value) {
console.log(value);
});

pg-promise UTF connection string

pg-promise does not understand UTF passwords?.. I can't make it work with them. Tried on linux and osx, postgres 9.3, 9.5 - seems to be not specific to versions. Looked into code. pg-promise uses pg, which uses pg-connect-string which is build based on back pg. Can't find the the root of the problem. Please help.
code to reproduce:
MacBook-Air:js vao$ cat 2.js
var pgp = require("pg-promise")();
var cs = 'postgresql://utf:утф#127.0.0.1:5433/a';
var db = pgp(cs);
db.connect()
.then(function (obj) {
obj.done(); // success, release the connection;
})
.catch(function (error) {
console.log("ERROR:", error.message || error);
});
console.log(cs);
returns:
MacBook-Air:js vao$ node 2.js
postgresql://utf:утф#127.0.0.1:5433/a
ERROR: password authentication failed for user "utf"
Using same connection string with psql:
MacBook-Air:js vao$ psql 'postgresql://utf:утф#127.0.0.1:5433/a'
psql (9.5.3)
Type "help" for help.
a=> \q
Trying bad password deliberately with same connection string:
MacBook-Air:js vao$ psql 'postgresql://utf:утфWrongPassword#127.0.0.1:5433/a'
psql: FATAL: password authentication failed for user "utf"
MacBook-Air:js vao$
As the author of pg-promise, I'd like at least to offer some guidance here on where to look, not the exact answer, as I've never dealt with UTF passwords myself.
pg-promise uses node-postgres, which in turn uses pg-connection-string to parse the connection. If there is an issue, then it is most likely inside pg-connection-string.
I would recommend trying Unicode notation that relies on %. Maybe it will work. Maybe not, I've never tried. Also the following question was never answered: Node.js postgres UTF connect string, which isn't reassuring either.
Sorry, I cannot be of more help at present.

get values from client side to server side (node.js + express.js)

I'm still trying to understand the concepts of node.js so please don't blame me if this is a dumb question..
In node.js, is it possible to get a value from index.jade to index.js?
For example:
index.jade
a(href="/bla" name="someName") Blabla
index.js
router.get('/bla', function(req, res){
//get value of name ("someName") or string ("Blabla")
console.log(req.body.name) ??
});
If this is not possible, I would like to know why...
Thanks.
No, it's not possible, for the simple reason that the name attribute in your HTML doesn't get passed to the server (any server, not necessarily a Node-based server).
If you want to pass a value in a GET request, you generally pass it as part of the URL:
a(href="/bla?name=someName") Blabla
This will generate the following HTML:
Blabla
In your server code, you can access the value using req.query.name.
Taking this a step further: if you have a variable available to your template called "name", you can use something similar, but a bit more dynamic:
a(href="/bla?name=" + encodeURIComponent(name)) Blabla
encodeURIComponent makes sure that any "special" characters (that may have a special meaning in URL's) will be encoded properly.

Problems running KeystoneJS without MongoDB

I would like to try running KeystoneJS without MongoDB.
There's a short blog post explaining how to do it at http://ifrederik.com/blog/2014/11/cms-without-db-running-keystonejs-without-mongodb/
Basically, it explains how to replace MondgoDB with TingoDB and using a Tungus driver.
The advice is to put the following into the top of the keystone.js file
global.TUNGUS_DB_OPTIONS = { nativeObjectID: true, searchInArray: true };
var tungus = require('tungus');
var mongoose = require('mongoose');
And later to set mongo database url to TingoDB.
keystone.set('mongo', 'tingodb://'+__dirname+'/data');
By doing this I got KeystoneJS up and running. By inspecting the contect of data/users file in TingoDB I can even see that the default user gets created, but I was not able to log in. It always reports that username / password combination is not ok.
What am I missing? How do I debug the problem to find out what exactly is the problem here?
Ok, to asnwer to myself, the problems seems to be because User.modele.findOne({email: emailRegExp}) doesn't work in TingoDB/Tungus.
When replaced it with lookup.email, without using regex-es, it semms to work.
But who knows if and what else will break because of incompatibility.

Resources