I have NodeJS app that uses MongoDB as database. I'm using native mongo driver (not mongoess).
The application allow users to work on projects and share them and the logic that decide which projects a user is allowed to see is built as mongo criteria selector.
In order to test that I've found TingoDB which looks like a great candidate for mocking the MongoDB to be able to run the real model code and check that it is working.
My question is what is the best way to load the initial data? keep it in separate file? Keep it as another model?
Thank you,
Ido.
TingoDB actually stores it's data in flat-files, so if you want, you could just keep a copy of the database in a directory and load that.
However, if you're just testing with a small amount of data, you'd probably be better off keeping the test-data as in your testing scripts, and inserting it through your application as part of the test. That way, you can easily compare the data in the application to the data you loaded in your assertions.
Finally, if you're running MongoDB in production, then you should probably use MongoDB in your tests. While they do have nearly identical APIs, they have very different performance, which should be something you're keeping track of in testing. Unless there's a need to use TingoDB during testing, I'd try to make it as similar to the production environment as possible.
Related
Here is my situation. I have an extensive REST based API that connects to a MongoDB database using Mongoose. The API is written as a standard "MEAN" stack application.
Currently, when a developer queries the API they're always connecting to the live production database. What I want to do is have an exact duplicate database as a "staging" database, where new data will be added first, vetted over a period of time, and then move to the live database. Then I want developers to be able to query either one simply by modifying their query.
I started looking into this with the Mongoose documentation, and it appears as though the models are tied to the DB connection, and if I want to have multiple connections I also have to have multiple models, one for each connection. This would be a nightmare of WET code and not the path I want to take.
What I want to do is not touch any of my code at all and simply have a switch that changes to the proper database for a given query. So my question is, how can I achieve this? Is it possible? The documentation seems to imply it is not.
Rather than trying to maintain connections two environments in the same code base have you considered setting up stage version of your application? Which database it connects to could be set through an environment variable or some other configuration option.
The developers would still then only have to make a change to query one or the other and you could migrate data from the stage database to production/live database once you have finished your vetting process.
I have an api written in node with a mongodb back end.
I'm using supertest to automate testing of an api. Of course this results in a lot of changes to database and I like to get some input on options for managing this. The goal is for each test to have no permanent impact on the database. The database should look exactly the same after the test is finished as it did before the test ran.
In my case, I don't want the database to be dropped or fully emptied out between tests. I need some real data maintained in the database at all times. I just want the changes by the tests themselves to be reverted.
With a relational database, I would put a transaction around each unit tests and roll it back after the test was done (pass or fail). As far as I know, this is not an option with mongo.
Some options I have considered:
Fake databases
I've heard of in-memory databases like fongo (which is a Java thing) and tingodb. I haven't used these but the issue with this type of solution is always that it requires a good parity with the actual product to maintain itself as a viable option. As soon as I used a mongo feature that the fake doesn't support I'll have a problem unit testing.
Manual cleanup
There is always the option of just having a routine that finds all the data added by the test (marked in some way) and removes it. You'd have to be careful about updates and deletes here. Also there is likely a lot of upkeep making sure the cleanup routine accurately cleans things up.
Database copying
If it were fast enough, maybe having a baseline test database and making a copy of it before each test could work. It'd have to be pretty fast though.
So how do people generally handle this?
I think this is a brand new way in testing without transaction.
imho - using mongo >=3.2, we can setup inMemory storage engine, which is perfect for this kind of scenario.
Start mongo with inMemory
restore database
create a working copy for test
perform a test on working copy
drop working copy
if more tests GOTO 3
I've got a Express web api using sequelize that i want to do end to end testing with. I want to be able to do end to end testing with an in memory database so i can run it on whatever machine pleases me.
I use mysql database for development and production, however i was thinking about using an in-memory sqlite database for testing but i'm not sure what the best way is to get test data into it.
There are several modules around like squelize-fixtures but none of them seem to be able to just fill the database with data without the need to write code around it to manipulate and insert it.
Anyone here doing integration tests with sequelize and sqlite that has figured out a good way of doing it without all the boilerplate code?
If you test this way it won't actually be end-to-end testing since you're using a different database.
However, what is wrong with changing your Sequelize dialect to be sqlite and then using sequelize-fixtures to ingest a file of your data? If you feel this is too onerous you could slog your way through it one time and then just save the sqlite db file for future use. You would then instantiate your Sequelize object with
storage: 'path/to/database.sqlite'
No matter what you do or what storage you choose you're going to have to do some work to seed your database.
I'm currently building a rest api and I'm struggling to find the best way to unit test each route.
A route handler performs various things and one of them is to execute a query to mongodb. I can unit test the route handler by using stubs, but if I'm testing the query I cannot stub the query itself, I need to have an in-memory mongodb that I could reset and insert new data for each test.
How do you test queries? I'm thinking that the only real way to ensure that the query does what I need is to use a real mongodb database installed in the testing machine (typically in the same machine used for developing).
yes, just as for relation databases, you need to have real base. if mongo offers in-memory auto-created version then it's easy. if not, then each developer has to have running mongo before he runs integration tests. for CI you can have one single dedicated mongo but than you have to prevent concurrent access (schema creation, multiple transactions etc). you should also implement automatic creation of schema if needed and empty database before each test. in relational db rollback is usually enough. when it's not enough then trimming all tables helps. although we had to implement it manually as we couldn't find any existing tools
So here's my deal.
I'm using node on the express framework. The website i'm working on grabs scraped data and stores it for each user on the website. That data can then be displayed on the users page whenever they want to access it, so the data will be scraped, put in a database or storage, whatever i decide the best way to do it is, and then pulled back out for the user.
I'm trying to figure out what the best database setup would be. There will potentially be large amounts of data per user, especially over long periods of time. I've read some stuff about using redis to cache some data like the user login info and that basic stuff, and then using mongodb for the big data. But I don't know, i'm new to database stuff so I am open to some new teachings and some ideas from the masters.
What would you guys suggest I do? I want it to be fast and be able to handle multiple queries at the same time, but really, I have no idea what i'm talking about, so please help me.
What would you guys suggest I do?
This really depends on the nature of your data, how you model your domain and how you want to persist it. I would first try to figure out the basic model and based on that choose the most suitable database system. Don't jump at quick conclusions around caching with redis when you don't even know if you will need it in the first place.
Suggestion might also depend on how much time you want to spend with database layer of your application. Some database systems provide more functionality than others depending on their concepts. If you are a beginner choose a single mainstream solution that is well documented with established community like MongoDB or MySQL that will cover all your needs from the beginning so that you won't end up managing multitude of systems.