I am running client unit tests in intern using phantomjs in one grunt task and a selenium server in another grunt task locally without any issues. However, when I bring sauce labs into mix during CI and attempt to dynamically load a module during a test I receive a load modules error as seen in the "Failing CI job" link below. All other modules load as expected, which doesn't make sense because they share the same requirejs loader instance with the same baseUrl. If I remove tests that dynamically load modules then there are not any errors.
Failing CI job
Failing test
Dynamic load in source
Magically passed, https://travis-ci.org/lazojs/lazo/builds/49259156. More words to meet minimum answer length requirement.
Related
We are using TeamCity to run cypress.io for our NodeJs application and some of the tests are failing due to timeouts. These timeouts seem based on latency to the database (AWS RDS) and vary from build-to-build.
What we would like to do is to try setting test coverage to a 95% success rate and see if this allows the build to continue.
There is an option in TeamCity to have build steps to run regardless if the previous steps failed, but we would like our tests to not run in this fashion.
Any advice would be appreciated. Thanks!
We ended up modifying the tests so that they would behave as expected in the new environment. We also decided to run the tests as they were built to run with a local Postgres database.
The significant issue we were dealing with was our Cypress tests were extremely fragile when moving to an RDS database. The tests were configured for a local dev environment with a local Postgres database and moving to RDS in the CI environment broke them.
My recommendation is for anyone setting up automated tests to make sure tests run in your CI environment as they do in their development, not to configure/edit your tests to pass in CI.
In other words, if your tests break in your CI environment, then they need to be fixed in the dev environment.
In our setup we are building and deploying our UI5 app as an embedded static resource within our Spring boot maven-based application. During the CI build with the SAP Cloud SDK pipeline, the frontent tests are however not being executed.
Looking at the pipeline code, it seems to me that those stages are only executed for HTML5 modules and not for Java modules. However, the npm modules should be available as they are collected during initialization stage as far as I can see.
So the question for me is if there is a way to execute the frontend tests also in this scenario or if not, whether this intentionally not being done due to other constraints I am not aware of.
For projects using MTA/Cloud Application Programming Model this is correct. Currently, we expect only html5 modules to contain frontends and the corresponding tests. Reason for that is that MTA brings that structure by default and there were no other request for this yet. However, as it also looks like a valid setup we will discuss whether we implement that in one of the future releases. You are also invited to create pull requests.
If you are using a plain maven project generated with the SAP Cloud SDK, you can have this setup of having the frontend embedded into the webapp folder. In this case, you only need to configure the npm script ci-frontend-unit-test in your package.json in the root of the project.
I have a test suite and because it contains some expensive tests, I disable some of them for our CI. However once a day, I'd like to run the whole test suite.
The issue is that running against the same set of test files, it causes snapshot failures because when running the whole test suite it is missing some. If I generate them, then the CI fails because it complains about snapshots being removed (i.e. the one from the whole test suite that are not being checked on the CI.)
What would be the proper way to handle this with jest?
Thanks!
I'll apologize now because I am very new to Intern and know just enough to know that I don't know anywhere near enough. I am using the latest version of Intern. I see lots of details about how to exclude files from the coverage reports that Intern generates, but nothing on what it includes in coverage by default, and now to get other things included. Intern already instruments and provides coverage reports on the test files that I run, but that doesn't do me any good. I need to I have several custom Dojo modules that need to be instrumented for coverage, but I can't seem to find how to make that happen. I am only running functional tests at this time.
The website under test is being served by local IIS, but the test files are in a completely different folder. Be default, it appears that Intern is instrumenting the test files and showing me nice reports about how much of my tests were covered in the run. Seeing this, my thought was that I needed to move all of the Intern install and configuration to the local IIS folder, which I did. Intern is still only providing coverage reports for the test files and not the Dojo modules.
Folder structure in IIS
wwwroot
|
--js
|
--Chai
--ckeditor
--myScripts
--dojo
--node_modules
Gruntfile.js
internConfig.js
package.json
I need the files in the myScripts folder instrumented for code coverage. Here is what I am excluding:
excludeInstrumentation: /^(?:Chai|node_modules|ckeditor|dojo)\//
It appears that nothing in those folders is being is instrumented, so at least I have that right. I don't have anything defined under loaderOptions at this time, and I'm not entirely sure that that is where the stuff in the myScripts folder should be listed when it comes to functional testing. So, the question is how do I get the stuff in that folder instrumented for code coverage?
In order to be instrumented, code need to be requested from the HTTP server that Intern creates when you run intern-runner. If you are loading code directly from IIS, it will never be instrumented and no code coverage analysis can be performed. If you need to use IIS instead of the built in server, you will also need to configure IIS to reverse proxy requests for these files to Intern, as described in the testing non-CORS APIs documentation.
I have created a few coded ui tests and linked them to the test case, and they now appear as automated and you can see the dll they link to in the test case details.
Now that I want to run the tests, MTM refuses to even start the test unless a build is defined.
However: I want to run the tests against a statically installed application in the lab environment. This is an application that I manually install, and I get this application already compiled, so no need to play around building it.
So how can I take the build server out of the loop? I don't need the application built or deployed, I'm already doing that.
All I want is the tests to run on the lab environment specified against an application that is already preinstalled.
It's asking you to define the build of the test solution, assuming that it's different from your application under test. The test assembly will be deployed to the test environment after you specify it in MTM. This article may help you with the specifics.
It is asking you to create a build for your Coded UI test solution. It requires that the tests be built so that it has something to execute when you run the tests. Assuming that your tests were recorded using your statically deployed application then they will test that same application.