In Sequelize, when we create a model using the following command,
sequelize model:generate --name Company --attributes name:string, desc:text
A migration file is also getting created. And we can make the models sync with the DB by adding the following piece of code.
models.sequelize.sync().then(() => {
console.log("DB Synced");
}).catch((error) => {
console.log(error);
});
Therefore, when there's a change in the column names or something, they get synced with the DB.
So, do we really need to run migration ? At any point in the development or production ?
Please correct me if I am mistaken.
I strongly recommend to use explicit migrations and to point tables for storing applied migrations and seeds in a sequelize config in case you have different versions of a database in different production environments (even if you are just planning different production deployments).
If you run a newer version of your app with some model changes on an older production DB then by executing sync method you change this production DB accidentally. Moreover you cannot reverse these changes easily because you don't know anything about differences between models in your current app and tables in the production DB.
Related
I have created a REST-API application using Django Rest Framework.
The API just converts the data from an existing Read-only Postgres DB to a REST- API to be consumed by the front-end.
Now I need to write some unit tests to test the application. Since there are lots of tables and views involved, I cannot create mock data and test.
Is there a way to run the tests in Django using the existing DB (Postgres) without creating any mock data ?
Note: I have read a lot of SO posts related to my problem but none of them worked as they were for old versions of Django.
You can't use the Django test command for looking at production data. It always creates an empty database that is populated from fixtures in the TestCase.
This is not a good practice and should not be done. But if you want to achieve what you stated, you can try this -
DATABASES = {
'default': {
...
'TEST': {
'NAME': 'your prod db'
}
}
in your settings.py file.
(This may interfere with your production database)
I will suggest you use hypothesis and factories for data production to writing tests as it will help you in the long run and will be foolproof.
What you are suggesting will not be called a test though. So it will be helpful for you to reconsider writing tests for your application.
You could do something like this in your settings.py
import sys
if 'test' in sys.argv:
DATABASES['default'] = {
# your default database credentials
}
I have access to two separate databases that I'd like to keep in sync, a new one and an existing one, which will be in separate physical locations. The new one is going to be used to service an external API, so to cut down on request time, I think it makes sense to only query the local database for API requests.
My initial approach was to use mongoose.createConnection and limit the local collection to minor metadata and directly access the remote collection, but that's what I'm now looking to avoid.
Another approach might be to use mongoose.createConnection to periodically query the remote db and update the local one, but it could be costly if I want to do make frequent updates.
There are ways to cut down the cost - for example, there is a lastUpdated property in the relevant collection on the existing database, which could be used to limit the remote query to recently updated records such as:
RemoteCollection.find({
lastUpdated: {$gte: Date.now() - lookbackPeriod}
})
But I'm wondering if there's any native functionality of mongoose/mongoDB that can be used more efficiently make the updates. I also thought about mongodump and mongorestore to keep a full local copy of the records I needed, but that also seems costly.
Any help is appreciated.
After a bit of reading and thanks to Jake's comment, it looks like it's working. I need to do some more setup, but the code below should work and is based on this section from the docs:
https://mongoosejs.com/docs/models.html#change-streams
The first step would be start mongod with the --replSet flag:
mongod --replSet "rs0" --bind_ip localhost,<hostname(s)|ip address(es)>
Then close and restart mongo and run rs.initiate() on the database. You can then check the status of the replica set with rs.status(). If that command works and returns a result, the replica set functionality should be there.
Then within Node, you can do something like this:
// The docs reference creating a new model but you can just import an existing one
const RemotePerson = require('./models/RemotePerson');
const LocalPerson = require('./models/LocalPerson');
RemotePerson.watch().on('change', data => {
if (data.operationType === "insert") {
LocalPerson.create(data.fullDocument);
} else if (data.operationType === "update") {
LocalPerson.findByIdAndUpdate(data.documentKey, {
$set: data.updateDescription.updatedFields
});
}
});
I am using sequelize ORM in nodejs. I have two apps. One app is only for database related task like creating tables and migrations (app1). Other app is rest api (app2). I am using sequelize in app2.I dont want to create table using app2 and i want to throw error here if table doesnt exist or schema is not the same. Is this possible using sequelize?
If you dont want to create the table in your app2, then just dont put the sequelize.sync() code. (if you're using migration and the sequelize-cli, then dont do sequelize db:update before launching your app2)
Concerning the error, if a table is not created, you will have errors when trying to use the schema !
I don't think there is a "clean" way of checking if the schema is inline with yours.
So, I want to write an application which use some data migration. Where should I store a current state of db?
For example: I have a production server and my development machine. I wrote an application and 3 migrations for it. When I deploy the application server also runs 3 migrations.
Now I'm going to write the 4th migration. How does server recognize that it need to run only 4th migration and it already run previous 3 migrations?
Ref https://github.com/sequelize/cli
sequelize init:migrations
will generate migration folder where you need to write migrations and
There are three types of storage that you can use: sequelize, json, and none.
sequelize : stores migrations and seeds in a table on the sequelize
database
json : stores migrations and seeds on a json file
none :
does not store any migration/seed
I installed loopback-component-passport so I can use it with strongloop's Loopback framework. I followed the docs,
but after this command: npm install loopback-component-passport, the following files/models are not created:
UserIdentity model
UserCredential model
ApplicationCredential model
These files are used by Loopback for third-party authentication. Do I have to add them manually? Or am I doing something wrong?
Yes, you have to add them manually. You can refer this example, loopback-example-passport on github.
The reason is the relationships of loopback-component-passport's core models i.e UserIdentity model, UserCredential and ApplicationCredential with Loopback's core model AccessToken and User doesn't come preconfigured which is required for authentication.
Also, you might notice that the developer extended the models but didn't added any model.js. This is so because there was no need to add any runtime logic. For starters, you might want to extend the core models as per the example and create relationship and acls only. Then you can proceed with custom logic.
I think loopback built in models stay tucked away in node_modules, so assuming you mean that these tables are missing from your datastore, you can do an autoupdate in order to get loopback to build any missing tables.
I add the following code to a file in the server/boot directory when I need to during development.
module.exports = function(app, done) {
app.datasources.mysql_db.autoupdate(function(err) {
if (err) throw err;
console.log("autoupdate");
done();
})
}
Where mysql_db is the name of your datastore.
This will update the datasource when the server boot but shouldn't drop existing data (saying that I have seen instances where foreign keys are lost, so care & db backup is still needed).