I am working on Test Case and I would like to get clarity about which are the directories & files that come under,
Unit Testing.
Integration Testing.
Functional Testing.
My application architecture is as follows,
controllers (which controls the respective action by calling service file from services directory).
models (model for data tables).
routes (for routing).
services (a layer that communicates with model).
db (holds migration for the database).
index.js (which runs the server).
Could anyone help me out by saying which directories & files needs which type of testing (Unit, Integration, Functional)?
Thank you.
Unit tests: As the name suggests these tests cover separate units in your code, so each function/file should have it's own separate unit test(s). Also you don't use real network and database for these tests, they are stubbed/mocked.
Integration tests: These test working of different modules in integration with each other. Looking at your architecture, index.js and services seem to be the main modules and controllers, models and routes seem to be helper modules. So you would be writing tests for index.js and services against a real database and network with configurations same as your application in real life.
Funtional tests: These tests mimick end user experience and they are also called End to End test. These tend to be complex to write and not robust as the application changes over time. This entails writing something like selenium tests where all the browser clicks are automated and it is ensured everyhting displays as expected. I would suggest to write not to many of these and cover only main scenarios.
Related
I have a question regarding the different tests for a node and specifically an express application.
I am new to node/express coming from a PHP background, so have a few questions.
I know about unit testing, using things like PHPUnit, so I have read about about Jest. My specific questions regarding jest and unit testing in an application like express. Is should I be breaking my code apart more? It currently is quite together my routes are basically where all my business logic is found. Which means it's difficult to unit test?
Then with something else like end to end testing, I am looking at testcafe. For this I am really unsure how to get past my authentication and furthermore how to test on my local machine, before pushing code to production.
Full disclosure, I have a CI setup to my main branch, so I am looking to implement these tests to stop me merging breaking code to my master branch and breaking the production site.
Personally I always prefer mocha.js for testing any node app. It specifies how many test cases are passing, generates a report for those which are not passing. Also it specifies the time required to execute a code segment.
I'm also relatively new to node. I use the same stack as you (Express + MongoDB), applying MVC pattern. In Java I used to write a lot of unit tests with Spock, but right now I focus mostly on integration testing.
In my opinion routes should not contain any logic. Try to move it to separate layer - services. This way you can focus on testing logic provided by them, instead of trying to test code hidden in your routes.
For testing I use mocha.js, chai and chai-http.
My approach is to set up test database and form my tests as a sequence of requests. There is no problem with testing authentication that way - just need to correctly set up db with some user data. If you want to cut off the dependencies like database, use sinon for stubbing and mocking.
The obvious downside of this approach is testing time, but you can split tests into unit and integration suites. Run your unit tests locally and integration tests in your CI pipelines.
I'm not sure if it's the best approach, but I'm positive about the effects. Learning new technology means refactoring a lot. I have changed the structure of my project multiple times, moving logic, extracting methods and classes etc. Integration tests assure me that I haven't broken the business logic, despite having changed what's in the black box. This kind of breaking changes would be way harder to maintain with unit tests.
I've built a web app that aggregates trading and blockchain data from several API's and displays them in a React frontend(node backend)
What is the best way to implement tests to check for data integrity or when there are issues?
I am extremely new to testing and would appreciate any guidance/direction. Have gone through several testing frameworks and libraries, and am kind of dumbfounded.
You don't really test apps for 'integrity' of data as you name it.
Especially when data comes from external (not your DB for example) sources.
If you own data, you can test DB integrity, but as you say that is not the case here.
What you do though is - write unit tests (functional, recursive, end2end tests too, but what you want to do will mostly be achieved by using unit tests).
Within tests, you basically provide all kinds of data to your app and check if results are what you expect them to be (both for working and breaking scenarios).
This way, you can be sure it works as you designed it.
If at one point somewhere in future, a bug is exposed or you find it yourself. Define precisely why the bug occurs and add test for it.
When after you fix code responsible for bug, all of your tests pass, you know you are good again.
As for libraries:
"Jest" https://jestjs.io/ is go-to library for many - it's for unit tests mostly.
Jasmine and Mocha are also popular choices.
For end to end testing check Testcafe - I recommend it.
https://github.com/DevExpress/testcafe
You should also test your API with Mocha, Chai, Supertest or Chakram.
This way, all layers of your app are covered and bugs can be spotted quicker.
Is there a way to unit test a code which is using cloud datastore api and written for flexible environment? testbed seems to be tied up with standard environment and it looks like using emulator will require launching/closing emulator process which usually is a flaky way for unit tests.
We end up with end to end testing (launch you tests with real db in dev environment, for example) As we having tenant based application, each test run just creates new tenant and all operations performed in scope of this tenant, so, there should no any inconsistency here. In the other hand, such solution is pretty slow.
The solution above, is just the easiest one, I believe here.
Another option would be to split you code on db dependent parts and business logic part. In this case you will test only business logic part, and mock db dependency. But, as we've investigated such solution, we found that we have a lot of code that have one line of db write operation and 1-3 lines of business logic code. So, splitting such code on different levels would be meaningless for testing and maintenance.
I guess, the last option is more generic relatively previous one, is to mock db. For each module that uses db, before test it you should inject mocked database index, that defines some responses. But in this case it is easy to fall in realization testing, instead of behavioral testing, so again that will mean, that such testing becomes quite ineffective.
I guess, this question is more generic about testing approaches, and not about actually datastore itself.
I am trying to figure out how (or even if) I can replace the API calls made by my app when running a Webdriver test to return stubbed output. The app uses a lot of components that are completely dependent on a time frame or 3rd party info that will not be consistent or reliable for testing. Currently I have no way to test these elements without using 'run this test if...' as an approach which is far from ideal.
My tests are written in C#.
I have found a Javascript library called xhr-mock which seems to kind of do what I want but I can't use that with my current testing solution.
The correct answer to this question may be 'that's not possible' which would be annoying but, after a whole day reading irrelevant articles on Google I fear that may be the outcome.
WebDriver tests are End to End, Black Box, User Interface tests.
If your page depends on an external gateway,
you will have a service and models to wrap that gateway for use throughout your system,
and you will likely already be referencing your models in your tests.
Given the gateway is time dependent, you should use the service consumed by your api layer in your tests as-well, and simply check that the information returned by the gateway at any time is displayed on the as page as you would expect it to be. You'll have unit tests to check the responses model correctly.
As you fear, the obligatory 'this may not be possible': Given the level of change your are subject to from your gateway, you may need to reduce your accuracy or introduce some form of refresh in your tests, as the two calls will arrive slightly apart.
You'll likely have a mock or stub api in order to develop the design, given the unpredictable gateway. It would then be up to you if you used the real or fake gateway for tests in any given environment. These tests shouldn't be run on production, so I would use a fake gateway for a ci-test environment and the real gateway for a manual-test environment, where BBT failures don't impact your release pipeline.
How do you test pages with single sign-on (SSO) login during integration tests (for instance by using caybara or cucumber)? For a normal login, you would write a method which visits the login page, fills out the form, and submits it. This is a bit difficult if the login form comes from an external SSO server like Shibboleth or OpenAM/OpenSSO. How is it possible to write integration tests for pages protected by SSO?
A similar problem is integration testing with a separate search server (Solr or Sphinx). You would probably solve it by using some form of mocks or stubs. Can someone give a good example how to mock or stub a SSO for cucumber or capybara? If this is too difficult, then a comparable example for a search server would be helpful, too.
Integration testing of a SSO application is a
special case of a more general problem: testing
distributed applications. This is a difficult
problem and there does not seem to be a magic
bullet for it. There are various ways to combine
a set of different servers or services and test them
as a whole. The two extremes are
a) Test an instance of the whole system. You don't
need any mocks or stubs then, but you need
a complete, full-fledged setup of the entire stack. This includes
a running instance of every server involved.
For each test, setup the entire application stack,
and test the whole stack, i.e. test the
entire distributed system as a whole with all
the components involved, which is difficult
in general. This whole thing works only if each
components and all connections are working well.
b) Write an integration test for each component,
treat it as a black box, and cover the
missing connections by mocks and stubs.
In practice, this approach is more common for
unit testing, one writes tests for each
MVC layer: model, view, and controller
(view and controller often together).
In both cases, you have not considered
broken connections. In principle one
has to check for each external server/service
the following possibilities
is down
is up and behaves well
is up and and replies wrong
is up, but you send it wrong data
Basically, testing of distributed apps is difficult.
This is one reason why distributed applications are hard to develop.
The more parts and servers a distributed application has, the more difficult it is to setup many full-fledged environments like production, staging, test and development.
The larger the system, the more difficult the
integration testing becomes. In practice,
one uses the first approach and creates a small
but complete version of the whole application.
A typical simple setup would be App Server + DB Server + Search Server.
On your development machine, you would have
two different versions of a complete system:
One DB Server with multiple databases (development and test)
One Search Server with multiple indexes (development and test)
The common Ruby plugins for search servers (Thinking Sphinx for Sphinx
or Sunspot for Solr) support cucumber and integration
tests. They "turn on" the search server for certain portions of
your tests. For the code that does not use the search server,
they "stub" the server or mock out the connection to avoid unneeded
indexing.
For RSpec tests, it is possible
to stub out the authentication methods,
for example for a controller test by
before :each do
#current_user = Factory(:user)
controller.stub!(:current_user).and_return(#current_user)
controller.stub!(:logged_in?).and_return(:true)
end
It also works for helper and view tests, but
not for RSpec request or integration tests.
For cucumber tests, it is possible to stub
out the search server by replacing the connection
to the search server with a stub (for Sunspot and
Solr this can be done by replacing the Sunspot.session,
which encapsulates a connection to Solr).
This all sounds well, unfortunately it is a bit hard to
transfer this solution for a SSO Server. A typical minimal
setup would be App Server + DB Server + SSO Server.
A complete integration test would mean we have to setup one SSO Server with
multiple user data stores (development and test).
Setting up a SSO Server is already difficult enough,
setting up a SSO Server with multiple user data
stores is probably not a very good idea.
One possible solution to the problem is maybe somewhere in the
direction of Fakeweb. FakeWeb is a Ruby library written by
Blaine Cook for faking web requests. It lets you decouple
your test environment from live services. Faking the response
of a SSO server is unfortunately a bit hard.
Another possible solution I ended up using is to use a fake login, i.e.
add a fake login method that can be called within the integration
test. This fake login is a dynamic method only added during the
test (through a form of monkey patching). This is a bit messy, but
it seems to work.