As far as I can tell, terraform doesn't have any support for Sagemaker Ground Truth. However AWS CLI does support it.
I don't want to create a whole new provider as a plugin, especially as this falls under aws.
How do I create my own resource within the existing aws provider?
You have a couple options here (and in general, when something isn't supported by the Terraform AWS provider).
If the resource in question is supported by CloudFormation, you can use the aws_cloudformation_stack Terraform resource to create a custom CloudFormation stack that creates and tracks the state of the resource. Here's the CloudFormation documentation for SageMaker; see if you can find the resource you want in there anywhere.
If it's only supported by the CLI (not by CloudFormation), you can use the CLI in your Terraform configuration. This is the module I like to use for doing CLI work in Terraform. The downside is that you must have the AWS CLI installed on whatever machine you're doing the terraform apply on.
Related
Learning Terraform, and in one of the tutorials for terraform with azure a requirement was to log in with the az client. Now my understanding is that this was to create a Service Princlple.
I was trying this with Github actions and my assumption was that the properties obtained for the Service Principle. When I tried running terraform plan everything worked out fine.
However, when I tried to do terraform apply it failed until I explicitly did an az login step in the github workflow job.
What am I missing here? Does terraform plan only compare the new configuration file against the state file, not the actual account? Or does it verify the state against the resource-group/subscription in Azure?
I was a little confused with the documentation on terraform plan
Am using a combination of these tools
Terraform - To deploy the Application specific AWS resources I need
(For instance a secret)
Skaffold - To help with the inner
development loop, surrounding the deployment of K8s resources to
local and remote cluster
Kustomize - To help with templating of
different configurations for different environment
My github action steps are as follows
Terraform to create the AWS resources. At this point it creates a AWS
secrets arn.
Skaffold to deploy the k8s manifests. Skaffold in-turn delegates K8s manifest generation to Kustomize. Within the Kustomize overlay files i need to be able to access the Secrets arn that was created earlier, this arn needs to be injected into the container that is being deployed. How do I achieve this?
Rephrasing the question: How do I pass resources that were created by terraform to be consumed by something like Kustomize (Which is used by skaffold)
(p.s, I really like the choice of my tools thus far as each one excels at one thing. I realize that terraform can possibly do all of it, but that is a choice that I dont want to make unless there are no easier options)
Here is what I have learnt:
I don't think there are any industry standards in terms of how to share this data between the tools across different steps within github actions. That being said here are some of the options
Have the Terraform store the secrets arn in a parameter store. Retrieve the arn from the parameter store in later steps. This means that the steps have to share a static key
Have Terraform update the kustomize files directly (or use kustomize_overlays as datasource)
There could be other similar approaches, but none of these tools have a native way of passing/sharing data
I worked with Terraform for AWS before successfully. Now I am trying to work with Azure and facing a few challenges. I have successfully authenticated to my azure account using Azure CLI. When I run the basic terraform provider arm .tf and do a terraform init it just works. But when I put in any additional code like container creation or blob creation .tfs, the init is not working and is giving me the below message :
No available provider "azure" plugins are compatible with this Terraform version.
Error: no available version is compatible with this version of Terraform
Terraform version :
bash-3.2$ terraform -v
Terraform v0.12.19
+ provider.azurerm v1.38.0
I used version 1.38.0 and tried many others but it still continues to give me error.
They are the two providers for the different Azure models.
Azure Service Management Provider model is the classic model in Azure and is not recommended to use now. It provides the resources with format azure_xxx.
Azure Resource Manager Provider model is the Resource Manager model which calls ARM and is recommended to use and supported well. It provides the resources with format azurerm_xxx.
You can also learn more about the ASM and ARM model in document Azure Resource Manager vs. classic deployment: Understand deployment models and the state of your resources.
Terraform will try to deploy all resources defined on Terraform configuration files. There are a lot of resources in my application, like lmabda, api gateway, ECS etc. I wonder whether I can specify deploying only one resource. For example, I want to deploy one lambda only and don't want to apply other resources. How can I make it in Terraform?
terraform apply -target=aws_lambda_function.test_function
More information on the usage of -target can be found in the terraform apply documentation.
I heavily use ECS Attributes on our containerized infrastructure. I couldn't find terraform docs to achieve this. Do I need to execute aws cli commands manually to apply those attributes after creating the infrastructure?
I'd recommend having the ECS agent set the ECS attributes if you need these.
You can do this by adding ECS_INSTANCE_ATTRIBUTES to the /etc/ecs/ecs.config file or passing them as an environment variable directly to the ECS agent on startup.
If you have a "base" ECS AMI (either one you rolled your own or the Amazon Linux AMI) then you probably just want to use user data to dynamically set this from Terraform.
You can use "aws_ecs_service" resource and add attributes. For example:
placement_constraints {
type = "memberOf"
expression = "attribute:ecs.availability-zone in [us-west-2a, us-west-2b]"}