Am working on Secure DevOps Kit for Azure(AzSK) using VSTS CI&CD. For working AzSK in VSTS there were two tasks named "AzSK ARM Templete Checker" and "Secure DevOps Kit(AzSK) CICD Extensions for Azure" which are available from Market place. But, here am unable to access "Secure DevOps Kit(AzSK) CICD Extensions for Azure" task after adding both to my organisation. Is there any extension/install additional tasks to add them for accessing it?
Please suggest me to "How to add it to my CI-Build Definition"
Based on the screenshot, you have installed the extension.
Secure DevOps Kit(AzSK) CICD Extensions for Azure is just the name of the extension. And the real task is AzSK ARM Template Checker.
So, you just need to add the AzSK ARM Template Checker task under test hub.
UPDATE:
Well, please note that another AzSK_SVT task (AzSK Security Verification Tests) is available in 'Release' pipeline tasks only.
So to use it you need to create a release pipeline,... then add the AzSK_SVT task from Test category. Please see Security Verification Tests (SVTs) for details.
First of all I am sorry to ask you a question not relative to your question.
(as it seems that your question is already anwered :) )
I am using the same task in my relaese pipeline and I am trying to add custom checks because the default checks this task is doing is not good enough for me.
This tutorial is really helpfull for trying to set things up, only I seem to just get to add custom baselinecontrols for a local module and not getting it to add it in the azure devops pipeline (more info about my question .
As you are also working on the same task I was hoping that you maybe could me help me figure it out.
Related
I've been setting up a pipeline for our project with Azure Pipeline using yaml. Currently adding review apps with Azure Kubernetes Service and was wondering how we should clean up after the PR is merged.
We are building docker images, pushing them to our registry, deploying them to a new dev space. Then it's all just left there.
After merge we're never going to bed those images again, not like we're going to deploy from a unapproved PR and there's dozens of them every day. We also would like to delete the review app and take down those extra deployments after we're done with them
I can't find anything in the documentation for this. Am I missing something?
I’ve been wondering the same thing, how does one clean-up all the “ephemeral” resources? Seeing the documentation says “ephemeral” I had hoped this functionality was built in.
Even though we trigger on master, I don’t believe there is enough information available in the pipeline to know which PR caused the commit to master, thus we cannot be guaranteed to tear down the correct review app.
I've been working on this today and think I've got it working.
I made a new pipeline that uses the kubectl task to delete the review app namespace, it also uses the azure cli task to run az acr purge to delete the images created for the PR, the tags of which are prefixed with the pr number so it can identify them.
I set this pipeline to not run with CI and not download the source.
Then I made an Azure Function that uses calls the Run Pipeline API to run it, passing the PR number as a variable to the pipeline.
Finally I used Service Hooks to invoke the Azure Function on PR Updated.
The only thing outstanding is the Environment Resource that is created automatically on the DevOps site does not get deleted. I cannot find an API for this so we might have to live with that.
By using browser dev tools and click the delete button in ADO I managed to find out how to delete environment resources using the Azure DevOps Rest API.
You can send:
DELETE <azure-dev-ops-base-url>/<organization>/_apis/distributedtask/environments/{environment-id}/providers/kubernetes/{resource-id}?api-version=6.0-preview.1
To find environment-id and resource-id use:
GET: <azure-dev-ops-base-url>/<organization>/_apis/distributedtask/environments?api-version=6.0-preview.1
GET: <azure-dev-ops-base-url>/<organization>/_apis/distributedtask/environments/{environment-id}?expands=resourceReferences&api-version=6.0-preview.1
I'm using Azure DevOps pipelines and for 'builds' I can define 'Hosted Ubuntu' agent (in my pipelines.yml file). But for 'releases' there's no way to edit yaml just the UI. The agent pool drop down list shows 'Default (no agents)'. I assume my problem is probably related to project permissions but haven't figured out what controls the visibility. On another account I see all the hosted agent options available.
For the possible security permissions issue
Check that you are part of the group(s) assigned as User to the security of the app pool in question.
If this security operates like other Allow, Deny, NotSet permissions, then your occupation in a group that has less than User permission will revoke that permission, but I haven't seen any documentation about this.
For the use of YAML in release pipelines
Yaml usage in the formal release pipeline hub is not supported. If you want to use yaml for your CD release, you're going to need to enable the new multi-stage pipelines experience.
This allows you to create stages in your .yml file that relate to deployment functions in addition to the classic build functions.
I believe this removes your release tracking from the release hub and integrates it directly into the build hub.
This is still a new concept for Azure DevOps, as is the documentation and writing about it. I found this article to be helpful.
Because this new feature seems to be centered around AKS (or other Kubernetes services) and Docker, I'm not sure if this will help you by allowing you to specify the host in yaml, but you mentioned "release" and "yaml" in the same sentence, so I thought I would address that.
I have built an automation framework using Java, Selenium Webdriver, Maven, TestNG. Currently, I am using Jenkins for pipeline and CI.
Now new requirement assigned to me is using Azure DevOps as CI tool and execute all test from there instead of Jenkins.
After some research, I am getting the following :
Upload code to Github or other azure supported repo. and create a pipeline.
Write your Java code using Visual studio code and then it will be far easy to execute from Azure DevOps.
Is there any better way to do this?
You need to follow steps below. Main effort is of tools integration if those are not present in azure devops portal :
I am not sure which code repository you are using however if you are not using one which is supported by Azure devops, then you need integrate it with azure devops portal.
Create agentpool in azure devops with same configuration as your jenkins agent.
Create build pipeline in azure devops. It will ask your repository name . Give the same.
4.While Creating pipeline it will ask you whether to create azure pipline yaml or not . Say "Yes" and it will create sample yaml file in code repository.
Open Yaml file.
give your agentpool name where they have mentioned it.
under section " steps "
mention all steps which you want to do run test cases. you would mentioned same
thing is jenkins pipeline under stages --> steps like shell ''' '''
save yaml and run it. you are done
NOTE : Main thing is configuration of agentpool. you need to make it should have all software tools ( except jenkin agent jar :) ).
Currently, I am working Azure ARM templates, I developed the ARM templates for a web app and SQL database and deployed those are into azure successful through the VSTS by configuring the CI and CD.
But I want to test the Azure ARM templates and validate them whether these are written correctly or not. For that, I Used the ARM VSTS task but it just validates only.
Before posting a question here, I followed this link but I didn’t understand the answer.
So, can anyone give suggestions on how to test the ARM templates?
There is no way of testing ARM templates really. The only real way to test ARM templates is to deploy it and validate that proper resources are being created. You can script it, but that's a lot of effort.
Test-AzureRmResourceGroupDeployment is mostly useless as it only validates the syntax which is helpful, but doesnt really guarantee that the template will do what you think it will do.
I cannot really imagine how hard it is to create a proper test for an ARM template, I don't see a lot of value from investing time in creating tests for ARM Templates. I usually use helper scripts that just deploy the template into the dev environment and I quickly validate it. Most of the changes to the ARM templates (even pretty complex ones) do not really require validation.
We have too many ways to test an ARM Template. You may consider two tests in your CI/CD Pipeline.
Pre deployment Tests :
This can be done using the command :
Test-AzureRmResourceGroupDeployment
To get more control on what you are going to deploy, you can Use Pester + Powershell, write a test scenario and add it as a step in your CI Pipeline.
A new functionnality called "what-if" will be available (according to Build 2019 ARM Template Session)
Post Deployment Tests :
For post deployment tests, you may write a JSON defining the expected resources (By their names) then write a script to compare your JSON Definition with the resources that you've deployed.
I am developing CD release pipeline using TFS 2015 update 2 on-prem instance. I'm relying on ARM template to setup Azure website and Azure Sql server. I'm using FTP method for deploying website bits from internal Build server to Azure website. For this website deployment I'm reading the credentials from PublsihingProfile of the newly created website.
Is this a right way or can you suggest a better way? Any comments are appreciated.
P.S. Customer wants to use FTP method and not WebDeploy.
If you really have to use FTP, and the thing you're not happy with is the process/password secret management, you could try this:
https://marketplace.visualstudio.com/items?itemName=januskamphansen.ftpupload-task
Its a VSTS extension task for release, which works with the vnext build/release system in VSTS or TFS2015 server. This task lets you put the parameters in against each environment you setup, and mark the passwords as secrets so they wont come out in logs or the UI.
The step basically wraps up the process of doing the ftp bit for you - you may want to do other steps as part of the release.