I was looking for easily adoptable functional test (e2e) automation tools using javascript. I did some exploration and picked some (Mocha, Jasmine, nightwatch, and intern). It is mentioned commonly that Mocha, and Jasmine are used for unit test automation, and nightwatch/intern can be used for functional test automation (e2e).
I really don't understand the difference between these tools or what makes them to be categorized under functional or unit test tools? Can't Mocha or Jasmine be used for functional test automation?
Added to that, http://theintern.io/ compares the market leading javascript tools feature wise. "includes functional testing" feature is applicable only for intern and nightwatch. I need clarification that why the same cannot be done in Mocha/Jasmine. Need your expertise answers.
thanks
mani
Mocha and Jasmine are purely unit testing frameworks. They don’t come with any code that could be used for performing functional testing. You could cobble together something yourself, using Mocha or Jasmine as the basis for writing tests and then adding some additional WebDriver client library like Leadfoot, but you’d still need to figure out how to start and stop browsers, how to also run unit tests against your code in each of the browsers, how to run tests in parallel so it isn’t horribly slow (Mocha and Jasmine are both designed to only ever run one test at once), how to hook up to a cloud hosting provider using their software tunnels (if you want), and write and maintain all of that glue code yourself. Oh, and you lose all the other features that come with Intern, like integrated code coverage analysis, source map support, etc.
Intern provides all of this out of the box, and was actually designed with this sort of testing in mind, which means it will always work better than any solution you try to create yourself using some other library that wasn’t designed for the task at hand. This question is a lot like asking, “can’t I just write my own testing framework from scratch using plain JavaScript?”. Sure, you can do everything in plain JS. It doesn’t mean it’s a good idea, though.
Related
We're using in our product Jest for Unit and Integration tests.
At the moment, we're searching for a solution to measure the duration of scenarios (Unit, API and E2E level) between two CI/CD builds, to see if code changes lead to performance decrease/increase.
There are solutions outside like JMeter and Gattling, but it feels not a right fit. On the one side, you have to write tests again that we already have written in Jest. And on the other side, these tools are more focused on Load, Scalability, Breakpoint, Stress testing etc. which feels a bit over dimensioned for our use case. (We're using completely serverless architecture, and we only want to know if code changes have an impact on performance)
So I was thinking if it's not maybe simply possible to utilize the Jest tests that we have already written, to measure also in some way the performance and compare it between CI/CD builds.
Do you know if there is some library or tool that could help me with that? Or do you have perhaps a complete different opinion, how to approach that?
I have a question regarding the different tests for a node and specifically an express application.
I am new to node/express coming from a PHP background, so have a few questions.
I know about unit testing, using things like PHPUnit, so I have read about about Jest. My specific questions regarding jest and unit testing in an application like express. Is should I be breaking my code apart more? It currently is quite together my routes are basically where all my business logic is found. Which means it's difficult to unit test?
Then with something else like end to end testing, I am looking at testcafe. For this I am really unsure how to get past my authentication and furthermore how to test on my local machine, before pushing code to production.
Full disclosure, I have a CI setup to my main branch, so I am looking to implement these tests to stop me merging breaking code to my master branch and breaking the production site.
Personally I always prefer mocha.js for testing any node app. It specifies how many test cases are passing, generates a report for those which are not passing. Also it specifies the time required to execute a code segment.
I'm also relatively new to node. I use the same stack as you (Express + MongoDB), applying MVC pattern. In Java I used to write a lot of unit tests with Spock, but right now I focus mostly on integration testing.
In my opinion routes should not contain any logic. Try to move it to separate layer - services. This way you can focus on testing logic provided by them, instead of trying to test code hidden in your routes.
For testing I use mocha.js, chai and chai-http.
My approach is to set up test database and form my tests as a sequence of requests. There is no problem with testing authentication that way - just need to correctly set up db with some user data. If you want to cut off the dependencies like database, use sinon for stubbing and mocking.
The obvious downside of this approach is testing time, but you can split tests into unit and integration suites. Run your unit tests locally and integration tests in your CI pipelines.
I'm not sure if it's the best approach, but I'm positive about the effects. Learning new technology means refactoring a lot. I have changed the structure of my project multiple times, moving logic, extracting methods and classes etc. Integration tests assure me that I haven't broken the business logic, despite having changed what's in the black box. This kind of breaking changes would be way harder to maintain with unit tests.
I've built a web app that aggregates trading and blockchain data from several API's and displays them in a React frontend(node backend)
What is the best way to implement tests to check for data integrity or when there are issues?
I am extremely new to testing and would appreciate any guidance/direction. Have gone through several testing frameworks and libraries, and am kind of dumbfounded.
You don't really test apps for 'integrity' of data as you name it.
Especially when data comes from external (not your DB for example) sources.
If you own data, you can test DB integrity, but as you say that is not the case here.
What you do though is - write unit tests (functional, recursive, end2end tests too, but what you want to do will mostly be achieved by using unit tests).
Within tests, you basically provide all kinds of data to your app and check if results are what you expect them to be (both for working and breaking scenarios).
This way, you can be sure it works as you designed it.
If at one point somewhere in future, a bug is exposed or you find it yourself. Define precisely why the bug occurs and add test for it.
When after you fix code responsible for bug, all of your tests pass, you know you are good again.
As for libraries:
"Jest" https://jestjs.io/ is go-to library for many - it's for unit tests mostly.
Jasmine and Mocha are also popular choices.
For end to end testing check Testcafe - I recommend it.
https://github.com/DevExpress/testcafe
You should also test your API with Mocha, Chai, Supertest or Chakram.
This way, all layers of your app are covered and bugs can be spotted quicker.
I'm working on strategy for my company which provides testing/development services. I implement both web and mobile apps test automation using Selenium/Appium, Junit, Cucumber.
In my company test cases are written in traditional form:
1) Go to X
2) Perform action Y
3) Go to W
4) Perform action Z
Expected result: The application does ... .
But in Cucumber I use behavioral language which more or less describes similar action. I have also read this article: http://markoh.co.uk/posts/three-reasons-to-use-cucumber-for-test-automation and I wonder if we should write all our test cases in Cucumber language. For test automation, it will be just copy&paste to have a feature written. I assume this is web or mobile app with GUI.
Is this a good idea?
Have you hot any experience with such test
cases documentation in long term?
Can manual testers have difficulties in using test cases written is such manner instead of traditional language?
Any input appreciated!
The main advantage in the Cucumber test cases is their reliability. You will not be able to change the test scenario without the code update. Also the Cucumber allows to figure out your common procedures that may be useful even in the manual tests. The test cases are self documented therefore we usually don't have any difficulties in the scenarios reading by any technical personnel. I succesfully used that approach in my previous job and I going to entry it also now. Also I would suggest to use the Cucumber background feature that allows to define the test prerequisites.
I just started getting into BizTalk at work and would love to keep using everything I've learned about DDD, TDD, etc. Is this even possible or am I always going to have to use the Visio like editors when creating things like pipelines and orchestrations?
You can certainly apply a lot of the concepts of TDD and DDD to BizTalk development.
You can design and develop around the concept of domain objects (although in BizTalk and integration development I often find interface objects or contract first design to be a more useful way of thinking - what messages get passed around at my interfaces). And you can also follow the 'Build the simplest possible thing that will work' and 'only build things that make tests pass' philosophies of TDD.
However, your question sounds like you are asking more about the code-centric sides of these design and development approaches.
Am I right that you would like to be able to follow the test driven development approach of first writing a unti test that exercises a requirement and fails, then writing a method that fulfils the requirement and causes the test to pass - all within a traditional programing language like C#?
For that, unfortunately, the answer is no. The majority of BizTalk artifacts (pipelines, maps, orchestrations...) can only really be built using the Visual Studio BizTalk plugins. There are ways of viewing the underlying c# code, but one would never want to try and directly develop this code.
There are two tools BizUnit and BizUnit Extensions that give some ability to control the execution of BizTalk applications and test them but this really only gets you to the point of performing more controled and more test driven integration tests.
The shapes that you drag onto the Orchestration design surface will largely just do their thing as one opaque unit of execution. And Orchestrations, pipelines, maps etc... all these things are largely intended to be executed (and tested) within an entire BizTalk solution.
Good design practices (taking pointers from approaches like TDD) will lead to breaking BizTalk solutions into smaller, more modular and testable chunks, and are there are ways of testing things like pipelines in isolation.
But the detailed specifics of TDD and DDD in code sadly don't translate.
For some related discussion that may be useful see this question:
Mocking WebService consumed by a Biztalk Request-Response port
If you often make use of pipelines and custom pipeline components in BizTalk, you might find my own PipelineTesting library useful. It allows you to use NUnit (or whatever other testing framework you prefer) to create automated tests for complete pipelines, specific pipeline components or even schemas (such as flat file schemas).
It's pretty useful if you use this kind of functionality, if I may say so myself (I make heavy use of it on my own projects).
You can find an introduction to the library here, and the full code on github. There's also some more detailed documentation on its wiki.
I agree with the comments by CKarras. Many people have cited that as their reason for not liking the BizUnit framework. But take a look at BizUnit 3.0. It has an object model that allows you to write the entire test step in C#/VB instead of XML. BizUnitExtensions is being upgraded to the new object model as well.
The advantages of the XML based system is that it is easier to generate test steps and there is no need to recompile when you update the steps. In my own Extensions library, I found the XmlPokeStep (inspired by NAnt) to be very useful. My team could update test step xml on the fly. For example, lets say we had to call a webservice that created a customer record and then checked a database for that same record. Now if the webservice returned the ID (dynamically generated), we could update the test step for the next step on the fly (not in the same xml file of course) and then use that to check the database.
From a coding perspective, the intellisense should be addressed now in BizUnit 3.0. The lack of an XSD did make things difficult in the past. I'm hoping to get an XSD out that will aid in the intellisense. There were some snippets as well for an old version of BizUnit but those havent been updated, maybe if theres time I'll give that a go.
But coming back to the TDD issue, if you take some of the intent behind TDD - the specification or behavior driven element, then you can apply it to some extent to Biztalk development as well because BizTalk is based heavily on contract driven development. So you can specify your interfaces first and create stub orchestrations etc to handle them and then build out the core. You could write the BizUnit tests at that time. I wish there were some tools that could automate this process but right now there arent.
Using frameworks such as the ESB guidance can also help give you a base platform to work off so you can implement the major use cases through your system iteratively.
Just a few thoughts. Hope this helps. I think its worth blogging about more extensively.
This is a good topic to discuss.Do ping me if you have any questions or we can always discuss more over here.
Rgds
Benjy
You could use BizUnit to create and reuse generic test cases both in code and excel(for functional scenarios)
http://www.codeplex.com/bizunit
BizTalk Server 2009 is expected to have more IDE integrated testability.
Cheers
Hemil.
BizUnit is really a pain to use because all the tests are written in XML instead of a programming language.
In our projects, we have "ported" parts of BizUnit to a plain old C# test framework. This allows us to use BizUnit's library of steps directly in C# NUnit/MSTest code. This makes tests that are easier to write (using VS Intellisense), more flexible, and most important, easier to debug in case of a test failure. The main drawback of this approach is that we have forked from the main BizUnit source.
Another interesting option I would consider for future projects is BooUnit, which is a Boo wrapper on top of BizUnit. It has advantages similar to our BizUnit "port", but also has the advantage of still using BizUnit instead of forking from it.