I have created an Azure build pipeline using the 'classic editor' (i.e. not yaml). The build consists of two agent jobs:
Job 1 - Build code and deploy to test enviornment using a single agent.
Job 2 - Run tests against the test enviornment in parallel (using max 3 agents at a time).
My problem with this setup is if a build is triggered, and the tests are mid-run, if a second build is triggered, the code that is deployed to the test enviornment will be overwritten by the subsequent build, causing the test run in Job 2 of the first build to fail.
Is it possible to tell the build pipeline to only trigger builds sequentially?
I have figured out how to use Azure DevOps API to check if the latest build has completed, however Im not sure how I can use it in the pipeline. Is it possible to do something like:
1 - Invote REST API to check status of latest build.
2 - Success Criteria met (i.e. the build has completed)? If yes continue build, if not wait a minute and check again.
You have the option to control this in the Build options. Should work based on what you described.
Edit:
After looking again to your question, i noticed that you are running your tests after you deploy your app to test environment, so it means you run your tests during release, so you need to control the flow on your release and not on your buid. In order to do it, you should control on the maximum number of parallel deployments:
Related
I want to add some automation tests (it can be selenium c#/ java tests) to my existing azure pipeline. My pipeline is working correctly i.e. when I push changes to my master branch (GitHub), it triggers the build and it gets deployed to the live. But I want to integrate some automation test which will decide whether to deploy the build or not example:
After committing to the master branch:
run the automation test
if tests passed > deploy
if the test fails > stop deployment
Any suggestions or reference material would be helpful.
Thanks,
Depends on what kind of tests you are going to run - if they are unit tests, it is easy but selenium tests you would need to deploy it to a webserver before running the tests.
When edit the pipeline, and click "Show assistance" if is't not already done, you can see the tasks, including test dashboard etc.
For example if you add a VStests, it will add something like this to the yaml file.
- task: DotNetCoreCLI#2
inputs:
command: test
projects: '**/*Tests/*.csproj'
arguments: '-- no-build --configuration $(buildConfiguration)'
Above just executes 'dotnet test' command which will look for tests in the project and run them.
It is executing a command line, so you can run any tests as far as you can provide a command line using batch/powershell commands.
The Microsoft tutorial at https://learn.microsoft.com/en-us/learn/modules/run-quality-tests-build-pipeline/ explains how to use the test wigits etc.
I would suggest to start with unit tests which is simpler before trying selenium.
Krish
You can specify the conditions under which each stage, job, or step runs. By default, a job or stage runs if it does not depend on any other job or stage, or if all of the jobs or stages that it depends on have completed and succeeded. By default, a step runs if nothing in its job has failed yet and the step immediately preceding it has finished.
In another word, if test step/job/stage passed, the next step/job/stage can precede.
So I have a project set up as a polyrepo, meaning all services and libraries are in different repos.
Now, I've updated the service projects to reference the library project directly when using Debug as the build config in visual studio, and the package on NuGet when building in Release, on Azure DevOps.
The problem I am trying to solve is that my service builds sometimes fail when I've updated the library because the Library build hasn't completed and update NuGet yet.
What are my options here? Is there a way to ensure that the Library build completes first before the service builds start, or something similar?
Since you are using the Release feature in Azure Devops, I suggest that you could split the whole process into two stages (Library Build Stage and Service Build Stage).
For example:
These two stages are connected by After Stage relationship.
Then when the Library Stage completes , the Service Build stage will start.
If the two stages need to specify different files in the build, you could select the target files in the Visual Studio Build task -> Solution.
By the way, if Service Build stage needs to use the build results of Library Build stage, I suggest that you can use the same Self-hosted agents to run the release.
You can define it in Tasks -> Agent Job -> Demands.
Updates:
Since the Service Build and Library Build are two pipelines, you could try the following steps to order the Build running Step.
Step1: Set the CI trigger(continuous integration) in Library Build. This could make sure the commits could trigger the Library Build.
Step2: Disable the CI trigger in Service Build. Then Set the Pipeline trigger in Service Build. You could set the Library Build trigger the Service Build.
In this case, you could make sure that the Service Build will run after the Library Build.
Currently I am trying to using Azure Test Plans to manage all test cases, but I just found how to build / generate manual test cases. I also make some research about Pipeline and I just write some "hello world" test scripts and using Pipeline to automatically test the code. My question is: How could I combine TestPlans with Pipeline to make the test in pipeline recordable? Thank you so much!
this is how the pipeline looks like
this is how the test code be checked
My question is: How could I combine TestPlans with Pipeline to make
the test in pipeline recordable?
I'm a bit confused about what the recordable mean in your question. To combine TestPlan with Pipeline, you should configure the Test Plan Settings. For example, set a build pipeline(build and generates the test binaries) and a release pipeline with test-related task in your Test Plan Settings.
The normal logic is:
A test plan containing your automated tests, which you have associated with automated test methods using Visual Studio 2017, or Visual Studio 2015 or earlier.
A Team Build pipeline that generates builds containing the test binaries.
The app to test. You can deploy the app as part of the build and release workflow and also use it for on-demand testing.
For more details please check Run automated tests from test plans.
Is any possibility to test created Azure Pipelines? From UI or your yaml definition of pipelines?
Mean that I have some yamls pipelines or pipelines defined from UI and I want to ensure by some tests(Unit Tests e.x.) that all have defined variables, build, test, and package parts or something else in each pipeline.
And verify pipelines configurations after some changes of them or after adding some new repos/branches if it's required.
Thanks...
Is any possibility to test created Azure Pipelines? From UI or your
yaml definition of pipelines?
If you want a out of box feature to achieve this, sorry to say, No, there hasn't.
BUT, the work around is using API to check them.
Client API.
You could write a simple script to get Builds definition with Client API.
In this simple script, you first get the whole definition:
List<BuildDefinitionReference> buildDefinitions = new List<BuildDefinitionReference>();
Then you could apply your customized check/test into this definition with scripts. In one word, write some test classes/methods. After the script complete, you can import it into VSTS, and then use task to run those tests part. Only this test succeed, then your builds could be executed.
So, at this time, it need you add 2 agent jobs into your pipeline, the first one is used to run your script test(names test agent job). And the second agent job is the one you want to check. In the second agent job, set its condition as:
At this time, only the test succeed, this current job can be ran.
Or, if you don't want the builds you want to check would be broken because of the test, please consider about using Build completion trigger. Set a separate pipeline to run the test. In the pipeline you want to check, set it can be run only when the test pipeline finished.
Rest API
You could use rest api with powershell which very similar with the above description. Use api to get builds definition, and then write some check powershell script.
I more recommend you to put the test at a separate pipeline. Then the API could only get the part you want check, not including the test part.
I have a pipeline build for a .NET application called Master Site and then I have three other build definitions using different repos. Every time I have to do a build of the Master site and then the subsequent build. I need to know if we can do multiple builds by triggering a single build request. (all the builds are built off of different repos)
I need to know if we can do multiple builds by triggering a single
build request.
For this issue,the answer is yes. You can do this by adding multiple Trigger Build Tasks to the agent job in a build pipeline .
This tasks allows to trigger a new Build (add it to the queue) as part of a Build Definition.