I have a working code to fire query on hive using node module "jdbc", as an alternative i was trying "jshs2" , i am able to connect to hive and fire query but still stuck to retrieve the resultset, can anyone who have used the "jshs2", module can put up an example.
Thanks for any help.
I just started a project where I have to connect to hive from node too. I was able to run a query on database and iterate through the resultset using the following demo code:
const {
Configuration,
HiveConnection,
IDLContainer,
} = require('jshs2');
const options = {
auth: 'NOSASL',
host: 'myServer',
port: myPort,
};
const hiveConfig = new Configuration(options);
const idl = new IDLContainer();
async function main() {
await idl.initialize(hiveConfig);
const connection = await new HiveConnection(hiveConfig, idl);
const cursor = await connection.connect();
const res = await cursor.execute('SELECT * FROM orders LIMIT 10');
if (res.hasResultSet) {
const fetchResult = await cursor.fetchBlock();
fetchResult.rows.forEach((row) => {
console.log(row);
});
}
cursor.close();
connection.close();
}
main().then(() => {
console.log('Finished.');
});
I'm using node v8.x, so I can use ES6 features like destructuring and async/await. If your node version is older you must work with promise chains.
Related
I have an app built with angular and node.js (with pg npm package, version = 8.7.1)
The app divided to microservice . Each server-app have "pg" package installed and have a connection to postgres db.
The problem is that if I run some "update" query and after this I running getList query, then I Got the old value instead the updated object. If I add setTimeout for 5 sec then it works fine
On my localhost all works fine. The issue occur only on heroku (with postgres on cloud) on the srever. sometimes I got the updated data and sometimes not
Here is my code:
Client code (angular) - calling to update func and then getList func with async & await
async filter({ value }) {
const list: any = await this.getList()
const [myData]: any = await this.updateData(this.value)
const list: any = await this.getList() // Here is the issue !!
}
The function calls to API to the server like this:
getList(): Promise<any> {
return this.http.get<any>(`${ENV.BASE_API}/doGetApiCalls`).toPromise();
}
updateData(value: any): Promise<any> {
return this.http.put<any>(`${ENV.BASE_API}/doUpdateApiCalls`, value).toPromise();
}
The server code is:
Bl code
async function updateData(description, id) {
let query = updateDataQuery(description, id);
let results = await postgressQuery(query);
return getDataResults;
}
DEL code
function updateDataQuery(description: string, id:number) {
const query = `UPDATE public.books
SET description='${description}',
WHERE book =${id}
RETURNING *`
return query;
}
And here is the connection to postgres db (BL calling to lib by import this)
const DATABASE_URL = process.env.DATABASE_URL;
const pool = new Pool({
connectionString:DATABASE_URL,
ssl:{rejectUnauthorized: false}
})
let openConnect = async () => {
await pool.connect();
}
let postgressQuery = async (q) => {
try {
const result = await pool.query(q);
return await result.rows;
}
catch (e) {
console.log(e);
}
}
========================================================
If I added await to client then it works fine. It takes a while for update?
async filter({ value }) {
const list: any = await this.getList() //
const [myData]: any = await this.updateData(this.value) //get the RETURN from server with correct data
await new Promise(resolve => setTimeout(resolve, 5000)) //added for wait for 5 sec
const list: any = await this.getList() // then data is correct (aafer 5 sec)
}
What wrong in this code above ?
Thanks in advance
I found a solution:
It was cache issue. Just need to install this package and all fine !
https://www.npmjs.com/package/nocache
The official documentation of the Node.js Driver version 3.6 contains the following example for the .find() method:
const { MongoClient } = require("mongodb");
// Replace the uri string with your MongoDB deployment's connection string.
const uri = "mongodb+srv://<user>:<password>#<cluster-url>?w=majority";
const client = new MongoClient(uri);
async function run() {
try {
await client.connect();
const database = client.db("sample_mflix");
const collection = database.collection("movies");
// query for movies that have a runtime less than 15 minutes
const query = { runtime: { $lt: 15 } };
const options = {
// sort returned documents in ascending order by title (A->Z)
sort: { title: 1 },
// Include only the `title` and `imdb` fields in each returned document
projection: { _id: 0, title: 1, imdb: 1 },
};
const cursor = collection.find(query, options);
// print a message if no documents were found
if ((await cursor.count()) === 0) {
console.log("No documents found!");
}
await cursor.forEach(console.dir);
} finally {
await client.close();
}
}
To me this somewhat implies that I would have to create a new connection for each DB request I make.
Is this correct? If not, then what is the best practise to keep the connection alive for various routes?
You can use mongoose to set a connection with your database.
mongoose.connect('mongodb://localhost:27017/myapp', {useNewUrlParser: true});
then you need to define your models which you will use to communicate with your DB in your routes.
const MyModel = mongoose.model('Test', new Schema({ name: String }));
MyModel.findOne(function(error, result) { /* ... */ });
https://mongoosejs.com/docs/connections.html
It's 2022 and I stumbled upon your post because I've been running into the same issue. All the tutorials and guides I've found so far have setups that require reconnecting in order to do anything with the Database.
I found one solution from someone on github, that creates a class to create, save and check if a client connection exist. So, it only recreates a client connection if it doesn't already exist.
const MongoClient = require('mongodb').MongoClient
class MDB {
static async getClient() {
if (this.client) {
return this.client
}
this.client = await MongoClient.connect(this.url);
return this.client
}
}
MDB.url='<your_connection_url>'
app.get('/yourroute', async (req, res) => {
try {
const client = await MDB.getClient()
const db = client.db('your_db')
const collection = db.collection('your_collection');
const results = await collection.find({}).toArray();
res.json(results)
} catch (error) {
console.log('error:', error);
}
})
I'm querying a local emulated Cosmos DB instance using the JS #azure/cosmos package. I'm using version 3.1.1 (according to the package-lock.json) and I cannot call the toArray() function on the items for a container.
let databaseID = "database";
let collectionID = "collection";
const endpoint = "https://localhost:8081";
const key = "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==";
const client = new CosmosClient({
endpoint,
key,
agent: new https.Agent({
rejectUnauthorized: false
})
});
let database = await client.databases.createIfNotExists({ id: databaseID });
let container = await database.containers.createIfNotExists({id: collectionID});
let items = container.items;
let readItems = items.readAll(); // crash
I get this crash:
UnhandledPromiseRejectionWarning: TypeError: container.items.readAll(...).toArray is not a function
The samples say I should be able to do this but I can't see the toArray() function in the official documentation. Interestingly toArray()is mentioned in the documentdb documentation. Maybe this function hasn't be re-implemented, or am I doing something wrong?
toArray() listed in the official github source code clearly,so i think it is supported.
const { result: results } = await container.items.query(querySpec).toArray();
if (results.length == 0) {
throw "No items found matching";
} else if (results.length > 1) {
throw "More than 1 item found matching";
}
I would suggest you putting the readAll() in the async function and use with await.(Follow this thread:Cosmos DB Query Works in Data Explorer But Not Node.js)
My sample code:
const cosmos = require('#azure/cosmos');
const CosmosClient = cosmos.CosmosClient;
const endpoint = "https://***.documents.azure.com:443/"; // Add your endpoint
const key = "***"; // Add the masterkey of the endpoint
const client = new CosmosClient({
endpoint,
key
});
const databaseId = "db";
const containerId = "coll";
async function run() {
const { container, database } = await init();
const querySpec = {
query: "SELECT r.id,r._ts FROM root r"
};
const queryOptions = {
maxItemCount : -1
}
const { result: results } = await container.items.query(querySpec).toArray();
if (results.length == 0) {
throw "No items found matching";
} else if (results.length > 1) {
throw "More than 1 item found matching";
}
}
async function init() {
const { database } = await client.databases.createIfNotExists({ id: databaseId });
const { container } = await database.containers.createIfNotExists({ id: containerId });
return { database, container };
}
run().catch(err => {
console.error(err);
});
Update Answer:
I have to say sorry that i'm mislead by the MS official document.If i navigate to Query Documents github source code by the link in the MS document:
I could found such sample code :
However, that's the master branch! Not the latest 3.1.1 version! If i switch the version at the same above page,it shows 404:https://github.com/Azure/azure-cosmos-js/blob/v3.1.1/samples/ItemManagement/app.js
More evidence, some comments:
So i believe that the #azure/cosmos V3 has been updated many things(such as toArray() method has been dropped),meanwhile the official link has not been updated.
Now,as you mentioned in your question,you could use fetchAll() to get the array of results:
const { resources: items } = await container.items.query(querySpec).fetchAll();
console.log(items)
Output:
If any more concern,please let me know.
I'm writing a simple nodejs CLI tool while learning promises (to avoid callback hell), and every tutorial/stackoverflow example I've found only shows how to make a single call. My use case is as follows:
1. Connect to the Database (this I can do)
2. Perform a SQL select statement (also got this part)
3. Do some stuff with the results
4. Repeat steps 2 & 3 a few more times
I'm collecting the MSSQL user name and password (with hard-coded server 'localhost' and database name 'testdb') so when the app executes I can't just jump into the MSSQL connection.
I can get this via callbacks, but right now I have about 50 queries so you can imagine the ugliness. The full code below does get me the first query, and I strongly suspect I'm not passing the "pool" object to the next "then", but when I try
.then((pool,result) => {
//next command
})
It still doesn't recognize pool
Here is the code (index.js):
const mssql = require('mssql');
const qry1 = "select fieldA from tblA";
const qry2 = "select fieldB from tblB";
const qry3 = "select fieldC from tblC";
var dbuser = '';
var dbpass = '';
var config = {}
function init() {
log('Beginning Audit');
collectDbInfo(); //The reason I don't just included it all here
}
function collectDbInfo() {
//code is irrelevant to the problem, this is where I'm collecting the database credentials
}
function start() {
config = {
user: dbuser,
password: dbpass,
server: 'localhost',
database: 'testdb'
}
mssql.connect(config)
.then(pool => {
//FIRST query
return pool.request().query(qry1)
})
.then(result => {
processQryA(result);
//SECOND query
return pool.request().query(qry2)
})
.then(result => {
processQryB(result);
//THIRD query
return pool.request().query(qry3)
})
.then(result => {
processQryC(result);
})
mssql.on('error',err => {
log('SQL Error:' err)
mssql.close();
process.exit(0);
}
}
processQryA(data) {
console.log(data.recordset[0].fieldA)
}
processQryB(data) {
console.log(data.rcordset[0].fieldB)
}
processQryC(data) {
console.log(data.recordset[0].fieldC)
}
init();
I fully appreciate I may be approaching this all wrong, so any advice or especially examples would be greatly appreciated.
If the queries are absolutely sequential in nature, you can achieve that with async/await:
async function start(){
config = {
user: dbuser,
password: dbpass,
server: 'localhost',
database: 'testdb'
}
try {
pool = await mssql.connect(config);
const res1 = await pool.request().query(qry1);
processQryA(res1);
const res2 = await pool.request().query(qry2);
processQryB(res2);
const res3 = await pool.request().query(qry3);
processQryC(res3);
const res4 = await pool.request().query(qry4);
processQryD(res4);
/*..And so on with rest of sequential queries*/
/*Any of them resulting in error will be caught in (catch)*/
} catch (error) {
console.error("Error in start()::", error);
}
}
Also: I would probably have my pool getting method separately from query executions to handle errors/validations nicely.
I would like to setup my prepared statements with the mssql module. I created a query file for all user related requests.
const db = require('../databaseManager.js');
module.exports = {
getUserByName: async username => db(async pool => await pool.request()
.input('username', dataTypes.VarChar, username)
.query(`SELECT
*
FROM
person
WHERE
username = #username;`))
};
This approach allows me to require this query file and access the database by executing the query that is needed
const userQueries = require('../database/queries/users.js');
const userQueryResult = await userQueries.getUserByName(username); // call this somewhere in an async function
My database manager handles the database connection and executes the query
const sql = require('mssql');
const config = require('../config/database.js');
const pool = new sql.ConnectionPool(config).connect();
module.exports = async request => {
try {
const result = await request(pool);
return {
result: result.recordSet,
err: null
};
} catch (err) {
return {
result: null,
err
}
}
};
When I run the code I get the following error
UnhandledPromiseRejectionWarning: TypeError: pool.request is not a
function
Does someone know what is wrong with the code?
I think this happens because the pool is not initialized yet... but I used async/await to handle this...
Here is how I made your code work (I did some drastic simplifications):
const sql = require("mssql");
const { TYPES } = require("mssql");
const CONN = "";
(async () => {
const pool = new sql.ConnectionPool(CONN);
const poolConnect = pool.connect();
const getUserByName = async username => {
await poolConnect;
try {
const result = await pool.request()
.input("username", TYPES.VarChar, username)
.query(`SELECT
*
FROM
person
WHERE
username = #username;`);
return {
result: result.recordset,
err: null
};
} catch (err) {
return {
result: null,
err
};
}
};
console.log(await getUserByName("Timur"));
})();
In short, first read this.
You probably smiled when saw that the PR was created just 2 months before your questions and still not reflected in here.
Basically, instead of:
const pool = new sql.ConnectionPool(config).connect();
you do this:
const pool = new sql.ConnectionPool(config);
const poolConnection = pool.connect();
//later, when you need the connection you make the Promise resolve
await poolConnection;