We're in the middle of working on a small proof of concept project which will deploy infrastructure to Azure using Terraform. Our Terraform source is held in GitHub and we've using Terraform cloud as the backend to store our state, secrets etc.
Within Terraform cloud we've created two workspaces, one for the staging environment and one for the production environment.
So far we've used the guide on the Terraform docs to develop a GitHub action which triggers on a push to the main branch and deploys our infrastructure to the staging environment. This all works great and we can see our state held in Terraform cloud.
The next hurdle is to promote our changes into the production environment.
Unfortunately we've hit a brick wall trying to figure out how to dynamically change the Terraform cloud workspace within the GitHub action so it's operating on production and not staging. I've spent most of the day looking into this with little joy.
For reference the Terraform backend is currently configured as follows:
terraform {
backend "remote" {
organization = "terraform-organisation-name"
workspaces {
name = "staging-workspace-name"
}
}
}
The action itself does an init and then and apply.
Obviously with the workspace name hardcoded this will only work on staging. Ultimately the questions comes down to how to parameterise or dynamically change the Terraform cloud workspace from the command line?
I feel I'm missing something fundamental and any help or suggestions would be greatly appreciated.
Related
I have a set of cloud run services created/maintained via terraform cloud.
When I create a new version, a github actions workflow pushes a new image to gcr.io.
Now in a normal scenario, I'd call:
gcloud run deploy auth-service --image gcr.io/riu-production/auth-service:latest
And a new version would be up. If I do this and the resource is managed by terraform, on the next run, terraform apply will fail saying it can't create that cloud run service due to a service with that name already existing. So it drifts apart in state and terraform no longer recognizes it.
A simple solution is to connect the pipeline to terraform cloud and run terraform apply -auto-approve for deployment purposes. That should work.
The problem with that is I really realy don't want to apply terraform commands in a pipeline, for now.
And the biggest one is I really would like to keep terraform out of the deployment process altogether.
Is there any way to force cloud run to take that new image for a service without messing up the terraform infrastructure?
Cloud run configs:
resource "google_cloud_run_service" "auth-service" {
name = "auth-service"
location = var.gcp_region
project = var.gcp_project
template {
spec {
service_account_name = module.cloudrun-sa.email
containers {
image = "gcr.io/${var.gcp_project}/auth-service:latest"
}
}
}
traffic {
percent = 100
latest_revision = true
}
}
In theory yes it should be possible ...
But I would recommend against that, you should be doing terraform apply on every deployment to guarantee the infrastructure is as expected.
Here are some things you can try:
Keep track of when it changes and use the import on that resource:
https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_service#import
Look into lifecycle ignore, you can ignore the attribute that triggers the change:
https://www.terraform.io/language/meta-arguments/lifecycle#ignore_changes
I have a 'development' - 'staging' and 'production' workspace within the terraform cloud organisation.
I'm attempting to interact with them as per documentation here.
Particularly this:
If you associate the directory with multiple workspaces (using
workspace tags), you can use the terraform workspace commands to
select which remote workspace to use.
Locally, I also have the exact same three terraform workspaces created.
Screenshots:
local workspaces:
remote workspaces and tags:
100% same organisation I was able to interact with the workspaces if I hardcode the workspace value instead of using tags.
My terraform backend cloud definitions:
terraform {
cloud {
organization = "<<myorgname>>"
workspaces {
tags = ["development", "staging", "production"]
}
}
}
When I run a simple terraform init I get greeted by:
No workspaces found.
There are no workspaces with the configured tags (development, production, staging)
in your Terraform Cloud organization. To finish initializing, Terraform needs at
least one workspace available.
Terraform can create a properly tagged workspace for you now. Please enter a
name to create a new Terraform Cloud workspace.
I've been going through the documentation here that goes into CLI-driven runs with this context but I can't figure out the right way to do this.
What I want is:
run a terraform plan or terraform apply whilst in the locally-selected development workspace
and then:
the cloud terraform to perform a run on the remote development workspace.
If I just go ahead and write 'development' as a name, it will then apply all 3 tags in the static definition to the remote 'development' workspace, thus defeating the whole purpose of using tags instead of a name.
What's the right way of doing this?
That is true, however there is also this part of documentation [1]:
tags - (Optional) A set of Terraform Cloud workspace tags. You will be able to use this working directory with any workspaces that have all of the specified tags, and can use the terraform workspace commands to switch between them or create new workspaces. New workspaces will automatically have the specified tags. This option conflicts with name.
EDIT: As mentioned in the comments, in order for local workspaces to be usable in Terraform Cloud as well (i.e., to be able to apply the code in Terraform Cloud), there has to be a "common" or a "main" tag across all workspaces created in the Terraform Cloud.
[1] https://www.terraform.io/cli/cloud/settings#arguments
Learning Terraform, and in one of the tutorials for terraform with azure a requirement was to log in with the az client. Now my understanding is that this was to create a Service Princlple.
I was trying this with Github actions and my assumption was that the properties obtained for the Service Principle. When I tried running terraform plan everything worked out fine.
However, when I tried to do terraform apply it failed until I explicitly did an az login step in the github workflow job.
What am I missing here? Does terraform plan only compare the new configuration file against the state file, not the actual account? Or does it verify the state against the resource-group/subscription in Azure?
I was a little confused with the documentation on terraform plan
I am using Terraform scripts to create azure services, I am having some doubts regarding Terraform,
1) If I have one environment let say dev in azure having some azure resources how can I copy all the resources to new environment lest say prod using terraform script.
2)what are the impact of re-run the terraform file with additional azure resources, what it will do.
3)What if I want to create an app service with the same name from Terraform script that already present in the azure will it update the resource or do nothing after terraform execution completed.
Please feel free to answer the question, it will be great help.
To answer your questions:
You could create a new workspace with terraform workspace new and copy all configuration files (.tf) to the new environment, then run terraform init, plan, apply.
The terraform will compare the content in your current state file with your configuration file, then update the new attributes or creating new resources other than re-creating the existing resources.
You could run terraform import to import existing infrastructure into Terraform. For referencing existing resources in the portal, you can use data sources.
I am working in a project that will be deployed at my client's Microsoft Azure. Thus I am currently testing terraform to assist me when the time comes.
create a azure function with terraform that will trigger on blob storage input data
My question is about how to add the azure functions's javascript/c# code into the terraform script so it will be automatically deployed ?
I checked the terraform docs, but it wasn't of much help:
https://www.terraform.io/docs/providers/azurerm/r/function_app.html
Any ideas?
Terraform doesn't handle pushing code to Azure resources, that's usually done in a following step in the pipeline (e.g. 1- execute terraform 2- deploy code).
However, the Azure Function App does have the ability to connect directly to your repo, and the Terraform azurerm_function_app module exposes the source_control property.
Terraform's azurerm_function_app documentation
So with Terraform you can configure the function app to pull the code directly from the repo when a change is detected.
Microsoft's Azure Function Continuous Deployment documentation