I´m trying to setup a pipeline in Azure DevOps which initiates a Azure Resource Group. The configurations for which are saved in a .tf file in the DevOps repository.
The pipeline was created with the classic editor. Following tasks were added to the job agent in the same order: terraform init (Terraform CLI) and Build Project (.NET Core).
Terraform is already installed (file path was added to environment variables).
I´m really new to this and am trying to do my first steps. So any help would be appreciated. Also, you can tell me if any important information is missing.
This is the job agent´s log for the Terraform init task:
2023-01-19T15:15:19.3880770Z ##[section]Starting: terraform init
2023-01-19T15:15:19.3895740Z ==============================================================================
2023-01-19T15:15:19.3896080Z Task : Terraform CLI
2023-01-19T15:15:19.3896240Z Description : Execute terraform cli commands
2023-01-19T15:15:19.3896440Z Version : 0.7.8
2023-01-19T15:15:19.3896590Z Author : Charles Zipp
2023-01-19T15:15:19.3896770Z Help :
2023-01-19T15:15:19.3896890Z ==============================================================================
2023-01-19T15:15:21.3070520Z ##[error]Error: Unable to locate executable file: 'terraform'. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.
2023-01-19T15:15:21.3186320Z ##[error]Error: Unable to locate executable file: 'terraform'. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.
2023-01-19T15:15:21.5201550Z ##[section]Finishing: terraform init
I have tried running this in various Backend Type settings. Also have tried to change the Agent specifications multiple times.
Furthermore have I tried runnning this after putting the terraform.exe in the repository root.
My expection was that the pipeline creates a new resource group, but the task won´t even be executed.
It looks like you're using the Azure Pipelines Terraform Tasks extension for Azure DevOps which also includes an installer task for Terraform called "TerraformInstaller". Try adding that to your pipeline before "terraform init" to ensure that Terraform is installed (by default it will install the latest version unless you give it a specific one) and is correct for the agent's OS.
As to why the existing terraform binary isn't being found my guess is that you've been trying to use the windows version of terraform (terraform.exe) instead of the linux version which is just terraform and that the pipeline is running on a linux agent. I'm guessing this because the output of the task says that it can't find the file named terraform without the .exe extension.
Problem solved. Two solutions:
Add Terraform install task into the pipeline before adding other Terraform tasks. You can find useful links for this in the approved answer above.
Run pipeline with Ubuntu (any version I guess) agent (Pipeline -> Agent Specification -> ubuntu-latest).
I prefer the second solution because the ubuntu agent finished the tasks significantly faster than the Microsoft agent (and not because of the added install task which only took 7 seconds).
For comparison:
Ubuntu agent: 1 min 16 sec (first execution; the executions afterwards ~30 sec total)
Windows agent: 4 min 31 sec
Related
I am running a test job on Jenkins, using Jenkinsfile pipeline. The job targets the node ubuntu-node.
After the job is done, when I select the "Workspace" link, I get 2 entries, for example:
Workspaces for validation_my_proj #105
/home/jenkins/workspace/validation_my_proj#script on master
/home/jenkins/workspace/validation_my_proj on ubuntu-node
Could someone explain why do I have 2 workspaces? What does the first line validation_my_proj#script on master mean?
I am having some problems with linking executable produced by the build with a shared library (using Meson build system), and I am wandering does this Workspace setup have anything to do with it, because locally all works OK, only on Jenkins not.
I understand that when I run terraform CLI within a directory, it takes all the terraform artifacts from the current directory.
Is it possible to have terraform apply and terraform plan to look for scripts in a directory other than current PWD such that I don't have to keep changing current directory?
At the time I'm writing this, Terraform v0.14.0 is currently in release candidate and its final release is expected in the next few weeks.
That new version will introduce a new feature to allow requesting that Terraform switch directory itself before running any of its subcommands. For example:
terraform -chdir=subdirectory init
terraform -chdir=subdirectory apply
This is essentially the same as launching Terraform with the current working directory already set to the given subdirectory, but with two minor differences:
Terraform will set the path.cwd value to be the directory where Terraform was run from, rather than the subdirectory. As before, path.root is the directory containing the root module, so both directories are available.
If your CLI configuration includes paths that Terraform resolves during its startup then they will be resolved relative to the original working directory, because the directory switch applies only to the behavior of the given subcommand, not to actions Terraform takes prior to running the subcommand.
Neither of these differences are usually significant for common Terraform use, so in most cases you can think of the -chdir= option as having the same effect as switching into the target directlry using cd first.
I've a working build pipeline in Azure Devops that essentially installs Python3.6, sets up a virtual environment (.env) and then executes all unit tests. It then uses as its final step, a copy operation to move all files, including the virtual environment to a drop folder.
My problem arises from creating a release pipe. I am running a bash script for the release pipeline that essentially installs the azure functions command tools, and then I activate the python virtual environment before I call the func azure publish instruction.
The error I get states that settings are encrypted and that I need to call func setting add to add settings, however, when run locally, the script executes without any error whatsoever.
Does anyone have a working release pipeline in Azure Devops for a python-based Azure Function that they'd be able to share with me, so I can perhaps see what I am doing wrong?
Here is the relevant bit of script that executes:
#!/usr/bin/env bash
FUNCTION_APP_NAME="secret"
FUNCTION_APP_FOLDER="evenMoreSecret"
# Install Azure Functions Core Tools
echo "--> Install Azure Functions Core Tools"
wget -q https://packages.microsoft.com/config/ubuntu/16.04/packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get update
sudo apt-get install azure-functions-core-tools -y
echo ">>>>>>>> Initialize Python Virtual Environment"
source .env/bin/activate
echo "--> Publish the function app to Azure Functions"
cd $FUNCTION_APP_FOLDER
func azure functionapp publish $FUNCTION_APP_NAME --build-native-deps
The script is executed using an Azure CLI, using a security principal which is tied to the azure account that it is targeting.
Usually with Azure DevOps you create several build steps that result in some build artifacts - these are defined in the azure-pipelines.yml file. You then do a release step to release the artifacts that you have created - this is created within the UI. This can involve deploying to a test server and then to production or however you want to configure it. What you are describing is doing the build and release step all in the one yaml file as the func publish is essentially doing a release and it seems to all be in the one script.
In the next release of the az cli there is a new command called az functionapp devops-build that will set up the DevOps pipeline with the seperate build and release steps. However, in the mean time, we have created a series of beta yaml files that we hope you can just drag and drop to do the build and release steps just within the build part (as you are doing).
The beta yaml files are here:
https://github.com/Azure/azure-functions-devops-build/wiki/Yaml-Samples
I must disclaim that they are not fully tested, nor are they supported yet.
I will answer myself as I've solved the problem.
To #Oliver Dolk: We do NOT want to publish as part of a build pipeline. The only thing I'm interested in is to set up a virtual environment and then run the unit tests.
The RELEASE stage is where we want to deploy the scripts copied over from the build step. These artifacts are then the basis for releasing into dev, test and prodution environment.
I was missing a very important step in my script; To create a local.settings.json file which contains encrypted settings for the functionapp.
In order to solve the problem, I only had to call the following:
func azure functionapp fetch-app-settings $FUNCTION_APP_NAME
This calls the azure functionApp, and retrieves it's settings into an encrypted local.settings.json which is then used during publishing.
For a complete script reference of both the build YAML script and the bash script that does the deployment, I've put both in an anonimized github repo:
https://github.com/digitaldias/Python-Examples
I am quite new to terraform and golang, I am trying to implement a custom provider, for a POC, to check if we can leverage terraform for our own use.
I was able to write and build the golang provider according to this video and some GitHub examples.
I created a go workspace an set the $GOPATH to default, $HOME/go.
Terraform packages is installed at $GOPATH/src/github/hashicorp.
Terraform binary is installed at $HOME/dev and specified at the $PATH.
according to the video, I created the provider package at /terraform/builtin/providers/mycustomprovider
And "go build"ed the package to $GOPATH/bin
Once I tried to 'terraform plan' I get the following:
provider.incapsula: no suitable version installed
version requirements: "(any version)"
versions installed: none
I added the custom provider binary to terraform.d/plugins and tried to run 'terraform plan' again.
now I am getting the following error:
Error: Error asking for user input: 1 error(s) occurred:
* provider.incapsula: fork/exec ~/.terraform.d/plugins/darwin_amd64/terraform-provider-incapsula: permission denied
I tried to chmod to 666 and chown the binary file, but with no luck, I am still getting the same error.
I tried to look for this kind of issue but couldn't find any reference.
I would appreciate any assistance.
Thanks!
The provider binary needs execute permissions, so try using 755 instead of 666. Also if the binary isn't somewhere in your $PATH, you generally need to run `terraform init -plugin-dir=.terraform/plugins/darwin_amd64" so terraform picks up the provider and updates the md5 lock file.
So try chmod 755 <wherever the provider is> and if it's still not working, use terraform init with the -plugin-dir argument pointing to the plugin directory (your provider should already be in there).
I'm following this guide to create a web app with a custom deploy.cmd file. The article suggests that I can get a copy of the current deploy.cmd file (which I'll then modify) using the following command:
azure site deploymentscript --python
Unfortunately, when I install the Azure CLI using the MSI linked in the article, there is no azure binary on my path. I do have az -- is this a newer version of the same CLI? -- but I can't find an equivalent deployment script generation command for that executable.
I found a deploy.cmd file using Kudu (under D:\home\site\deployments\tools) but am not sure if that's the appropriate file to use. Can anyone suggest the right Azure CLI command for deployment script generation, or confirm that the deploy.cmd file I found is the right one to modify? Thanks in advance!
Based on my knowledge, there is not an equivalent to azure site deploymentscript in azure cli(2.0). So, you could not do deploy custom script with Azure CLI 2.0.
You had better know the difference between Azure cli 2.0(az) with Azure cli 1.0(azure).
Azure CLI 2.0: Our next-generation CLI written in Python, for use with
the Resource Manager deployment model.
Azure CLI 1.0: Our CLI written in Node.js, for use with both the
classic and Resource Managerdeployment models.
For your scenario, if you could install Azure ClI 1.0, you could refer to this link to install Azure CLI 1.0.
Instead of using the command line to generate a starter deployment script, there is an alternative approach that is often easier:
Deploy your repo without any deployment scripts.
Go to the site's Kudu Console.
From the Tools menu, choose 'Download deployment script'. You'll get a zip with a .deployment and deploy.cmd files.
Commit both files at the root of your repo
Tweak them as needed
More information please refer to this link.
You can use kuduscript to generate the deployment script.
npm install -g kuduscript
kuduscript --python
Here is the list of options
Options:
-h, --help output usage information
-V, --version output the version number
-r, --repositoryRoot [dir path] The root path for the repository (default: .)
--aspWAP <projectFilePath> Create a deployment script for .NET web application, specify the project file path
--aspNetCore <projectFilePath> Create a deployment script for ASP.NET Core web application, specify the project file path
--aspWebSite Create a deployment script for basic website
--go Create a deployment script for Go website
--node Create a deployment script for node.js website
--ruby Create a deployment script for ruby website
--php Create a deployment script for php website
--python Create a deployment script for python website
--functionApp [projectFilePath] Create a deployment script for function App, specify the project file path if using msbuild
--basic Create a deployment script for any other website
--dotNetConsole <projectFilePath> Create a deployment script for .NET console application, specify the project file path
-s, --solutionFile <file path> The solution file path (sln)
-p, --sitePath <directory path> The path to the site being deployed (default: same as repositoryRoot)
-t, --scriptType <batch|bash|posh> The script output type (default: batch)
-o, --outputPath <output path> The path to output generated script (default: same as repository root)
-y, --suppressPrompt Suppresses prompting to confirm you want to overwrite an existing destination file.
--no-dot-deployment Do not generate the .deployment file.
--no-solution Do not require a solution file path (only for --aspWAP otherwise ignored).