I am currently working on generic selenium tests to add to all my pipelines.
Because all my projects are similar (webshops) but have different solutions, I created a independent project to execute unit tests for all these webshops. I am wondering if it is possible to execute these tests in the independent project the Azure DevOps pipeline.
What are my possibilities?
Thank you,
Thomas
Related
Is it possible to pull / clone bitbucket repository within Azure Logic Apps?
I am curious if it is possible to set up some backend tests within Azure Logic Apps. So to pull repo with tests first and then execute them within CLI. I see that there is bitbucket connector in Logic Apps but there is no option to pull the repo. Or should I check some custom connector to run commands from hand like "git clone" etc. - if yes which one?
Azure Logic Apps is a cloud platform where you can create and run automated workflows with little to no code. By using the visual designer and selecting from prebuilt operations, you can quickly build a workflow that integrates and manages your apps, data, services, and systems.
From: What is Azure Logic Apps?
The key concepts here are "little to no code" and "prebuilt operations". Building your code with a CLI and running its tests is not something Logic Apps is targeting. It would also make executing Logic Apps a LOT more complex on the Azure end, since it would mean installing any and all frameworks, tools, etc. that are needed for building the code/running the tests.
If you look at Bitbucket actions that are supported, you can kind of make out that they're all API calls.
What you're looking for is available for free with GitHub workflows
A workflow is a configurable automated process that will run one or more jobs. Workflows are defined by a YAML file checked in to your repository and will run when triggered by an event in your repository, or they can be triggered manually, or at a defined schedule.
or Azure Pipelines.
Azure Pipelines automatically builds and tests code projects. It supports all major languages and project types and combines continuous integration, continuous delivery, and continuous testing to build, test, and deliver your code to any destination.
Potentially interesting read: Build Bitbucket Cloud repositories.
I want to update the test results of my CodeceptJS scripts in Azure DevOps. I have each test script created for corresponding test case in DevOps. Now I wan the test results to be updated directly in Azure whenever I run the script.
Can someone please suggest me how to link CodeceptJS with Azure DevOps? Is there any API or Plugin available?
I am afraid there is not a plugin available to associate js test scripts to test cases in DevOps.
Currently only Tests that use MSTest v2, NUnit, and xUnit frameworks can be associated with a test case workitem when using Visual Studio 15.9 Preview 2 or later. See document What types of tests are supported.
However, you can use rest api to update the test case results in azure devops.
First, create test plans to include the test cases work item
Second, call Point list rest api to list all the test points associated to your test cases
Then call point update rest api to update the test result.
Currently Azure devops maynot work with javascript tests.
Has anyone found work around for running Jest/Mocha/ codeceptjs/ webdriver io tests as part of Test plans?
Here is a blog introducing Integrate test automation framework with Azure Test Plan. In order to achieve this we will make use of the Azure DevOps APIs.
I was able to successfully integrate this set up with Java-Cucumber
tests and Robot-Python tests. Likewise this can be extended to any JS
framework like WebDriverIO or Cypress.
My website is already deployed and can be accessed by a server I've already provisioned.
I created an automated selenium webdriver test in Visual Studio to test my website. Now I want this to run on a nightly basis automatically, would it be possible to use Azure DevOps for this? I've been checking tutorials on running Automated Tests in Azure DevOps and it always includes deploying the website in Azure DevOps too, which is not applicable for my site.
Now I want this to run on a nightly basis automatically, would it be
possible to use Azure DevOps for this? I've been checking tutorials on
running Automated Tests in Azure DevOps and it always includes
deploying the website in Azure DevOps too, which is not applicable for
my site.
Yes, it's possible. If your source code is in Azure Devops repo, you can add Visual Studio build task to generate the test assembly. And then use Visual Studio Test task to run the tests.
Or if you've got the test assembly in one path, just specify the search folder and Test files:
You don't need deploy-related tasks since your website is deployed. Just make sure your website is running, and then install self-hosted agent in the server you've already provisioned. Run the pipeline with that self-hosted agent from Default agent pool, then you can run tests easily for your website.
We are currently testing this where I'm just running the test.
This is what my Azure pipeline looks like:
Basically we have a console app running the test (With NUnit) on the server.
We deploy the app that contains all the tests. We have variables for the URL to use in the pipeline with the login user etc that will populate our runsettings file.
We then create a Virtual VS Session then run the .dll containing the selenium tests.
Export the .trx report and mail it out.
Hope this helps a bit
I have built an automation framework using Java, Selenium Webdriver, Maven, TestNG. Currently, I am using Jenkins for pipeline and CI.
Now new requirement assigned to me is using Azure DevOps as CI tool and execute all test from there instead of Jenkins.
After some research, I am getting the following :
Upload code to Github or other azure supported repo. and create a pipeline.
Write your Java code using Visual studio code and then it will be far easy to execute from Azure DevOps.
Is there any better way to do this?
You need to follow steps below. Main effort is of tools integration if those are not present in azure devops portal :
I am not sure which code repository you are using however if you are not using one which is supported by Azure devops, then you need integrate it with azure devops portal.
Create agentpool in azure devops with same configuration as your jenkins agent.
Create build pipeline in azure devops. It will ask your repository name . Give the same.
4.While Creating pipeline it will ask you whether to create azure pipline yaml or not . Say "Yes" and it will create sample yaml file in code repository.
Open Yaml file.
give your agentpool name where they have mentioned it.
under section " steps "
mention all steps which you want to do run test cases. you would mentioned same
thing is jenkins pipeline under stages --> steps like shell ''' '''
save yaml and run it. you are done
NOTE : Main thing is configuration of agentpool. you need to make it should have all software tools ( except jenkin agent jar :) ).