I've mostly worked with PHP/MySQL but I've now been handed a Node.js/MongoDB project on Github.
Having gone through a Mongo tutorial, I feel I understand the concept to a reasonable extent now, but I am still unsure how to do the most basic thing - view the Mongo database associated with the project.
In the config file, I found the following:
module.exports = {
database: {
url: 'localhost:27017/app_name'
},
But seeing how I'm on a remote machine, how do I reach the database? Do I need to ask the previous dev for the DB so I can set it up locally?
Searching the code for the word mongo it only appears in packages.json so that's not a lot of help.
localhost:27017
means the DB is in the local machine in which your are developing your app.
In your case, you have MONGO DB installed in your local machine and run the project.
Otherwise if you have a centralized DB then you have to configure that IP here as follows:
Also configure your mongodb.config of your remote DB to accept the connection from your local machine by changing the "BIND IP"
module.exports = {
database: {
url: 'yourRemoteIP:27017/app_name'
},
Use a GUI tool, i would recommend MongoChef. All you need to do is when you start the server, just open this GUI and connect to DB. GAME ON!! You will be able to see your collections and you are ready to go. You can also run mongo query direct on the shell. It is a helpful tool for playing with your local DB.
PS I am considering you want to see your local DB.
You can access your DB through terminal as well, all upto you.
Related
How i can backup data my database psql on Heroku to my local computer and i want to move to another sever. can heroku backup a database my server to my local computer?
If you app needs some sql database, you should have access to the connection parameters like the url, user and password. There is no chance for you app to connect to a sql database without these parameters
Reviewing the teledrive docs, I found the variable which contains that values:
DATABASE_URL
format: postgresql://[user]:[password]#[host]:[port][/dbname][?paramspec]
Is similar to java and other languages: https://devcenter.heroku.com/articles/connecting-to-relational-databases-on-heroku-with-java#using-the-jdbc_database_url-in-a-spring-boot-app
You can get these values usually in the environment variables configuration in heroku
Something like this:
Your duty is to find these values.
import & export using some IDE
If you have the database connection parameters and the database has public access, you could use any database ide . I recommend you the dbeaver because is free and powerful. You could use it to connect to any database in the galaxy.
Here are the steps to export and restore a database.
https://community.pyramidanalytics.com/t/h7hk07w/how-to-backup-and-restore-a-postgresql-database-via-dbeaver
Basically you should:
connect to your heroku postgress,
export the data
connect to another postgres
import the data
import & export using the shell
This is more fast than an IDE. Check this to understand it: https://simplebackups.com/blog/postgresql-pgdump-and-pgrestore-guide-examples/
We always normally work in Azure where I write around 200 stored procedures a year in their SQL Server database.
We had to create a SQL Server database in AWS-RDS and still call it in our Node APIs like usual. I was able to quickly and easily set up the AWS DB in SQL Server Management Studio so I do know the credentials.
I created several tables and several stored procedures with no problems and tested to make sure they worked there. When I called them like I normally do in Node, I found I was getting an error
Could not find stored procedure
I went through forums all over but most of the data pertains to MySQL instead of SQL Server, and after trying everything I saw in the forums have not been able to complete what should be a very simple process. I would imagine there is some simple thing I missed, but after 2 days it is time for some fresh ideas.
I am setting up the credentials like this:
var awsConnection = {
host : process.env.RDS_HOSTNAME,
user : process.env.RDS_USERNAME,
password : process.env.RDS_PASSWORD,
port : process.env.RDS_PORT
};
I am using the endpoint provided by AWS for the host, the username and password I use to login to SQL Server Management Tool (which works). The port number is the one specified by AWS (1433 - the default for SQL Server).
I call it in my api like this:
await sql.connect(connectionAWS).then(pool => {
// Stored procedure
console.log("awsConnection INSIDE: " + JSON.stringify(awsConnection));
return pool.request()
.input('repId', sql.VARCHAR(40), repObj.RepID)
.execute('UserExistsBD');
}).then(async result => { ...
I added the console.log to see if we were getting past the login and it appears that we do. I also used Telnet to make sure the endpoint/port combo work and they do. I also checked AWS to make sure the Subnets, Route tables, and gateways were good and to make sure my IP Address was white listed. Any ideas would be very much appreciated!
I did my homework before posting this question
So the case is that I want to create a utility in my nodejs application that will move specific collections from my main database to an archive database and vice versa. I am using mongo db atlas for my application. I have been doing my research and I found two possible ways one is to create a mongodump and store and other is to create a backup file myself using my node application and upload it to archive db. Using the later approach will cause to loose my collection indexes.
I am planning to use mongodump for the purpose but can't find a resource that shows how to achieve that. Any help would be appreciated. Also if any one has any experience with similar situation I am open to suggestions as well.
I recently created a mongodump & mongorestore wrapper for nodejs: node-mongotools
What does it mean?
you have to install mongo binary on your host by following official mongo documentation(example) and then, you could use node-mongotools to call them from nodeJS.
Here is an example but tool doc contains more details:
var mt = new MongoTools();
const dumpResult = await mt.mongodump({ uri, path })
.catch(console.log);
I have access to two separate databases that I'd like to keep in sync, a new one and an existing one, which will be in separate physical locations. The new one is going to be used to service an external API, so to cut down on request time, I think it makes sense to only query the local database for API requests.
My initial approach was to use mongoose.createConnection and limit the local collection to minor metadata and directly access the remote collection, but that's what I'm now looking to avoid.
Another approach might be to use mongoose.createConnection to periodically query the remote db and update the local one, but it could be costly if I want to do make frequent updates.
There are ways to cut down the cost - for example, there is a lastUpdated property in the relevant collection on the existing database, which could be used to limit the remote query to recently updated records such as:
RemoteCollection.find({
lastUpdated: {$gte: Date.now() - lookbackPeriod}
})
But I'm wondering if there's any native functionality of mongoose/mongoDB that can be used more efficiently make the updates. I also thought about mongodump and mongorestore to keep a full local copy of the records I needed, but that also seems costly.
Any help is appreciated.
After a bit of reading and thanks to Jake's comment, it looks like it's working. I need to do some more setup, but the code below should work and is based on this section from the docs:
https://mongoosejs.com/docs/models.html#change-streams
The first step would be start mongod with the --replSet flag:
mongod --replSet "rs0" --bind_ip localhost,<hostname(s)|ip address(es)>
Then close and restart mongo and run rs.initiate() on the database. You can then check the status of the replica set with rs.status(). If that command works and returns a result, the replica set functionality should be there.
Then within Node, you can do something like this:
// The docs reference creating a new model but you can just import an existing one
const RemotePerson = require('./models/RemotePerson');
const LocalPerson = require('./models/LocalPerson');
RemotePerson.watch().on('change', data => {
if (data.operationType === "insert") {
LocalPerson.create(data.fullDocument);
} else if (data.operationType === "update") {
LocalPerson.findByIdAndUpdate(data.documentKey, {
$set: data.updateDescription.updatedFields
});
}
});
I have written some tests for my node.js application and the tests are running locally using a Postgresql test database.
When I run my test script, npm run test, the environment is set to test and when this happens, the database connection string is set for the test database and my queries in the application are now done on the test database. Like so:
let connectionString;
if (process.env.NODE_ENV === 'test') {
connectionString =`postgresql://${process.env.DB_USER}:${process.env.DB_PASS}#${process.env.DB_HOST}:${process.env.DB_PORT}/${process.env.TEST_DB_NAME}`;
} else {
connectionString =`postgresql://${process.env.DB_USER}:${process.env.DB_PASS}#${process.env.DB_HOST}:${process.env.DB_PORT}/${process.env.DB_NAME}`;
}
This way my tests are being run on the test database.
On travis however, I know I am going to need to further configure its own database. On the travisCI docs, I read about how I could set up a PostgreSQL database here, but this doesn't help me because how do I get the full database URL as above? On travisCI, what am I to use as my database hostname or port and how do I set this value inside my code?
How do I set the database connection string and access it in my code?
Thanks for any ideas.
Found this while searching of how to do the exact same thing, and this didn't answer my question, so in case anyone else comes across it, I think I have solved it.
zerosand1s kind of answered it, but there seems to be nowhere online which actually says what the hostname or port should be (maybe I'm being dumb and this is obvious but :shrug:)
As per the psql docs:
Defaults to the value of the PGPORT environment variable or, if not set, to the port specified at compile time, usually 5432.
So I guessed port would be 5432 (you could probably just use PGPORT as your variable).
Also elsewhere on travis, it says local host will bind to 127.0.0.1 for other databases (eg. mongo) so I took a guess on that too.
Travis docs tell us the user as postgres and password is blank.
I was using the whole string to connect, where as you split it up, so as such I set my entire connection string as (you can extract each component):
postgres://postgres#127.0.0.1:5432/testing_db
I did this on the travis dashboard settings.
Amazingly all of this worked :tada:
Hope this helps.
As per docs, you can create a database using before_scriptlike so
before_script:
- psql -c "CREATE DATABASE testing_db;" -U postgres
then you can add your database credentials (along with database name and port etc) to travis environment variables like so
travis encrypt DB_USER=TEST_DB_USER --add env.matrix
You can find more on travis environment variables here. You can also add environment variables on travis dashboard under your repository settings.