Why Pulumi is not creating a resource group with the name I gave?
Here is my small script
const azure = require("#pulumi/azure")
const resourceGroupName = new azure.core.ResourceGroup("test-pulumi", {
location: "francecentral",
});
The name of the resource group is: test-pulumi83d54581
The name parameter that you give is your component's name, not the full name of the resource to be created. By default, Pulumi appends a suffix to all names to avoid name collision e.g. across multiple stacks.
To specify the name explicitly, pass it as options parameter:
const resourceGroupName = new azure.core.ResourceGroup("test-pulumi", {
name: "test-pulumi",
location: "francecentral",
});
That's a bit verbose, and we might get an option to disable name suffixes - see this issue for progress.
Related
Is there a way in Terraform to check if a resource in Google Cloud exists prior to trying to create it?
I want to check if the following resources below exist in my CircleCI CI/CD pipeline during a job. I have access to terminal commands, bash, and gcloud commands. If the resources do exist, I want to use them. If they do not exist, I want to create them. I am doing this logic in CircleCI's config.yml as steps where I have access to terminal commands and bash. My goal is to create my necessary infrastructure (resources) in GCP when they are needed, otherwise use them if they are created, without getting Terraform errors in my CI/CD builds.
If I try to create a resource that already exists, Terraform apply will result in an error saying something like, "you already own this resource," and now my CI/CD job fails.
Below is pseudo code describing the resources I am trying to get.
resource "google_artifact_registry_repository" "main" {
# this is the repo for hosting my Docker images
# it does not have a data source afaik because it is beta
}
For my google_artifact_registry_repository resource. One approach I have is to do a Terraform apply using a data source block and see if a value is returned. The problem with this is that the google_artifact_registry_repository does not have a data source block. Therefore, I must create this resource once using a resource block and every CI/CD build thereafter can rely on it being there. Is there a work-around to read that it exists?
resource "google_storage_bucket" "bucket" {
# bucket containing the folder below
}
resource "google_storage_bucket_object" "content_folder" {
# folder containing Terraform default.tfstate for my Cloud Run Service
}
For my google_storage_bucket and google_storage_bucket_object resources. If I do a Terraform apply using a data source block to see if these exist, one issue I run into is when the resources are not found, Terraform takes forever to return that status. It would be great if I could determine if a resource exists within like 10-15 seconds or something, and if not assume these resources do not exist.
data "google_storage_bucket" "bucket" {
# bucket containing the folder below
}
output bucket {
value = data.google_storage_bucket.bucket
}
When the resource exists, I can use Terraform output bucket to get that value. If it does not exist, Terraform takes too long to return a response. Any ideas on this?
Thanks to the advice of Marcin, I have a working example of how to solve my problem of checking if a resource exists in GCP using Terraform's external data sources. This is one way that works. I am sure there are other approaches.
I have a CircleCI config.yml where I have a job that uses run commands and bash. From bash, I will init/apply a Terraform script that checks if my resource exists, like so below.
data "external" "get_bucket" {
program = ["bash","gcp.sh"]
query = {
bucket_name = var.bucket_name
}
}
output "bucket" {
value = data.external.get_bucket.result.name
}
Then in my gcp.sh, I use gsutil to get my bucket if it exists.
#!/bin/bash
eval "$(jq -r '#sh "BUCKET_NAME=\(.bucket_name)"')"
bucket=$(gsutil ls gs://$BUCKET_NAME)
if [[ ${#bucket} -gt 0 ]]; then
jq -n --arg name "" '{name:"'$BUCKET_NAME'"}'
else
jq -n --arg name "" '{name:""}'
fi
Then in my CircleCI config.yml, I put it all together.
terraform init
terraform apply -auto-approve -var bucket_name=my-bucket
bucket=$(terraform output bucket)
At this point I check if the bucket name is returned and determine how to proceed based on that.
TF does not have any build in tools for checking if there are pre-existing resources, as this is not what TF is meant to do. However, you can create your own custom data source.
Using the custom data source you can program any logic you want, including checking for pre-existing resources and return that information to TF for future use.
There is a way to check if a resource already exists before creating the resource. But you should be aware of whether it exists. Using this approach, you need to know if the resource exists. If the resource does not exist, it'll give you an error.
I will demonstrate it by create/reading data from an Azure Resource Group. First, create a boolean variable azurerm_create_resource_group. You can set the value to true if you need to create the resource; otherwise, if you just want to read data from an existing resource, you can set it to false.
variable "azurerm_create_resource_group" {
type = bool
}
Next up, get data about the resource using the ternary operator supplying it to count, next do the same for creating the resource:
data "azurerm_resource_group" "rg" {
count = var.azurerm_create_resource_group == false ? 1 : 0
name = var.azurerm_resource_group
}
resource "azurerm_resource_group" "rg" {
count = var.azurerm_create_resource_group ? 1 : 0
name = var.azurerm_resource_group
location = var.azurerm_location
}
The code will create or read data from the resource group based on the value of the var.azurerm_resource_group. Next, combine the data from both the data and resource sections into a locals.
locals {
resource_group_name = element(coalescelist(data.azurerm_resource_group.rg.*.name, azurerm_resource_group.rg.*.name, [""]), 0)
location = element(coalescelist(data.azurerm_resource_group.rg.*.location, azurerm_resource_group.rg.*.location, [""]), 0)
}
Another way of doing it might be using terraformer to import the infra code.
I hope this helps.
This work for me:
Create data
data "gitlab_user" "user" {
for_each = local.users
username = each.value.user_name
}
Create resource
resource "gitlab_user" "user" {
for_each = local.users
name = each.key
username = data.gitlab_user.user[each.key].username != null ? data.gitlab_user.user[each.key].username : split("#", each.value.user_email)[0]
email = each.value.user_email
reset_password = data.gitlab_user.user[each.key].username != null ? false : true
}
P.S.
Variable
variable "users_info" {
type = list(
object(
{
name = string
user_name = string
user_email = string
access_level = string
expires_at = string
group_name = string
}
)
)
description = "List of users and their access to team's groups for newcommers"
}
Locals
locals {
users = { for user in var.users_info : user.name => user }
}
I'm using Pulumi 1.16 with dotnet/C# and the AzureNative stack. I try to create an EventGridTopic. To access the created resource's properties later I pull some output values.
Example code:
var topic = new Topic("eventgrid-topic-status", new TopicArgs
{
TopicName = "egt-status-dev",
ResourceGroupName = "rg-testapp-dev",
Location = "westeurope"
});
var endPointOutput = topic.Endpoint;
var endPointAccessKey = ""; // missing output property
The resource is being created. I found no way to get the access key properties:
PrimaryAccessKey
SecondaryAccessKey
In the former (elder) Azure stack the properties exist. But in Azure Native stack not. Is that on purpose, just work in progress, has been forgotten or is there some other way to retrieve these properties on this object?
This is output on Azure (old stack):
This is Azure Native, clearly the keys are missing:
I doubt that this happens accidentally and would like to understand what to do.
Azure API (and therefore Azure Native resources) return no sensitive information in their outputs automatically to minimize security risks. You have to make an explicit call to retrieve those.
In this case, you likely need to invoke the function listTopicSharedAccessKeys.
You will want to call the function from within an Apply to make sure that it's triggered only after the topic is created (e.g., not during preview):
var keys = topic.Name.Apply(topicName => ListTopicSharedAccessKeys.InvokeAsync(
new ListTopicSharedAccessKeysArgs
{
ResourceGroupName = "rg-testapp-dev",
TopicName = topicName
}));
If you don't want to hardcode the resource group name:
let keys = pulumi.all([rg.name, topic.name]).apply(arr =>
azn.eventgrid.listTopicSharedAccessKeys(
{
resourceGroupName: arr[0],
topicName: arr[1]
}
)
);
keys.apply(x => pulumi.log.info(x.key1 ?? ""));
I am trying to fetch the Azure Resource Group Deployments by using filter where name starting with "Deploy" but can't find any documentation on the $filter.
I tried to do something like below:
try
{
var credentials = new ClientSecretCredential(tenantId, clientId, clientSecret);
var resourceClient = new ResourcesManagementClient(subscriptionId, credentials);
var deployments = resourceClient.Deployments;
AsyncPageable<DeploymentExtended> rgDeployments = deployments.ListByResourceGroupAsync("myRG", "name eq 'Deploy-20210412184314'");
await foreach (DeploymentExtended deploymentProperties in rgDeployments)
{
Trace.WriteLine(deploymentProperties.Name);
}
}
catch(Exception e)
{
Trace.WriteLine(e.Message);
}
But it gives error saying -
{"error":{"code":"InvalidProvisioningStateFilter","message":"Invalid $filter 'name eq 'Deploy-20210412184314'' specified in the query string."}}
So can we only use filter like provisioningState eq '{state}' not for name?
I am using Azure Resources Management client library for .NET.
Please refer this documentation.
I am afraid you could not use the filter with name eq 'Deploy-20210412184314', in this case, if you already knew the name of the deployment and want to get it at the resource group scope, no need to use ListByResourceGroupAsync, just use this method DeploymentsOperations.GetAsync(String, String, CancellationToken), pass the resourceGroupName and deploymentName, you can simply get it.
I've been trying to assign multiple group aliases, meaning, multiple AD groups in our company, into one identity group. So far we've had an identity group for each alias, and we realized that doesn't make sense, as they all carry the same policies.
We are using Terraform in order to maintain and provision our infrastructure.
This is my expected form:
resource "vault_identity_group" "saas-mfi" {
metadata = {
productname = "mfi"
}
name = "saas-mfi"
policies = [
"eaas-key",
"secret-store-mfi"
]
type = "external"
}
resource "vault_identity_group_alias" "alias_1" {
canonical_id = vault_identity_group.saas-mfi.id
mount_accessor = var.org_local_mount_accessor
name = "alias_1"
}
resource "vault_identity_group_alias" "alias_2" {
canonical_id = vault_identity_group.saas-mfi.id
mount_accessor = var.org_local_mount_accessor
name = "alias_2"
}
resource "vault_identity_group_alias" "alias_3" {
canonical_id = vault_identity_group.saas-mfi.id
mount_accessor = var.org_local_mount_accessor
name = "alias_3"
}
When I try to apply this configuration, I get the following error:
Error: Provider produced inconsistent result after apply
Of course, the issue does not stand with the provider. But it seems like one identity group can't have more than one alias to itself. Which is weird, as in the UI, there is a tab for identity groups called "Aliases", in plural.
If anybody has any information regarding this matter, I would really appreciate that.
I was trying to do the same thing but just came across the following paragraph in the documentation for identity:
External group serves as a mapping to a group that is outside of the identity store. External groups can have one (and only one) alias. This alias should map to a notion of group that is outside of the identity store.
From the section on External vs Internal Groups.
I'm running Terraform v11.13 with the AWS provider. Is it possible to attach multiple data template files to a single resource?
An example of this is where you have a single aws_iam_policy resouce, but for it to create multiple IAM polices from different data template files.
It works when it is just a single data template file with a count index. It also works when the file is static, as in not a template file.
Here is the code example
variable "policy_list"{
type = "list"
default = ["s3,"emr","lambda"]
}
resource "aws_iam_policy" "many_policies" {
count = "${length(var.policy_list)}"
name = "Policy_${var.policy_list[count.index]}_${var.environment}"
policy = "${file("${path.module}/files/policies/${var.environment}/${var.policy_list[count.index]}.json")}"
}
resource "aws_iam_role_policy_attachment" "many_policies_attachment" {
count = "${length(var.policy_list)}"
role = "${aws_iam_role.iam_roles.*.name[index(var.role_list, "MyRole"))]}"
policy_arn = "${aws_iam_policy.many_policies.*.arn[count.index]}"
}
But what fails is
resource "aws_iam_policy" "many_policies" {
count = "${length(var.policy_list)}"
name = "Policy_${var.policy_list[count.index]}_${var.environment}"
policy = "${data.template_file.${var.policy_list[count.index]}_policy_file.*.rendered[count.index]}"
}
With an error message similar to
parse error expected "}" but found invalid sequence "$"
Any ideas on how this can be achieved?
Based on the errors messages and the suggestion by Matt Schuchard, it's fair to conclude that the data.template_file option does not support interpolation in v11.13