Pulumi: Missing PrimaryAccessKey for EventGrid Topic - azure

I'm using Pulumi 1.16 with dotnet/C# and the AzureNative stack. I try to create an EventGridTopic. To access the created resource's properties later I pull some output values.
Example code:
var topic = new Topic("eventgrid-topic-status", new TopicArgs
{
TopicName = "egt-status-dev",
ResourceGroupName = "rg-testapp-dev",
Location = "westeurope"
});
var endPointOutput = topic.Endpoint;
var endPointAccessKey = ""; // missing output property
The resource is being created. I found no way to get the access key properties:
PrimaryAccessKey
SecondaryAccessKey
In the former (elder) Azure stack the properties exist. But in Azure Native stack not. Is that on purpose, just work in progress, has been forgotten or is there some other way to retrieve these properties on this object?
This is output on Azure (old stack):
This is Azure Native, clearly the keys are missing:
I doubt that this happens accidentally and would like to understand what to do.

Azure API (and therefore Azure Native resources) return no sensitive information in their outputs automatically to minimize security risks. You have to make an explicit call to retrieve those.
In this case, you likely need to invoke the function listTopicSharedAccessKeys.
You will want to call the function from within an Apply to make sure that it's triggered only after the topic is created (e.g., not during preview):
var keys = topic.Name.Apply(topicName => ListTopicSharedAccessKeys.InvokeAsync(
new ListTopicSharedAccessKeysArgs
{
ResourceGroupName = "rg-testapp-dev",
TopicName = topicName
}));

If you don't want to hardcode the resource group name:
let keys = pulumi.all([rg.name, topic.name]).apply(arr =>
azn.eventgrid.listTopicSharedAccessKeys(
{
resourceGroupName: arr[0],
topicName: arr[1]
}
)
);
keys.apply(x => pulumi.log.info(x.key1 ?? ""));

Related

Configuring encryption protection in Azure SQL server using ARM/Bicep

I'm trying to build both ARM and Bicep templates for enabling BYOK/CMK/TDE on Azure SQL server (and databases).
The challenge I'm having is that templates expect KeyVault Key Version to be passed in as an input. I'd really like to avoid that, as version could eventually change and it's not a value I'd like to maintain as an input parameter.
what I've tried so far is to create these 2 resources for SQL:
Microsoft.Sql/servers/keys#2022-05-01-preview
Microsoft.Sql/servers/encryptionProtector#2022-05-01-preview
encryptionProtector seems pretty straighforward, which just uses servers/keys resource. And that's where I'm stuck.
It requires KV key version for 'name' field, which I expected to be able to get from Microsoft.KeyVault/vaults/keys existing resource. However it only has this property:
keyVaultKey.properties.keyUriWithVersion
My next option was to parse the value, like:
var sqlServerKeyName = '${keyVaultName}_${keyVaultKeyName}_${last(split(keyVaultKey.properties.keyUriWithVersion, '/'))}'
but this results in warning:
his expression is being used in an assignment to the "name" property of the "Microsoft.Sql/servers/keys" type, which requires a value that can be calculated at the start of the deployment. You are referencing a variable which cannot be calculated at the start ("keyVaultKeyName" -> "keyVaultKey"). Properties of keyVaultKey which can be calculated at the start include "apiVersion", "id", "name", "type"
So my question is: is it possible to get KV Key Version from Bicep/ARM template and if yes - how? Or is it generally not recommended to do that (especially in the context of transparent data encryption)?
lastly, if there are no ARM/Bicep based solutions, I guess next best solution could be to try to retrieve latest version via powershell and then pass it as input. any suggestions/examples on this approach maybe?
note: KeyVault and Keys are created in separate deployment so I cannot use KV deployment output for this
The error is just about the name of the resource: the value has to be calculated when the deployment starts which is not possible in your case because the name is generated from another resource.
You would need to invoke it through another module:
// sqlserver-keyvault-encryption.bicep
param sqlServerName string
param keyVaultName string
param keyName string
param keyVersion string
param keyUri string
resource sqlServer 'Microsoft.Sql/servers#2022-05-01-preview' existing = {
name: sqlServerName
}
// Create sql server key from key vault
resource sqlServerKey 'Microsoft.Sql/servers/keys#2022-05-01-preview' = {
name: '${keyVaultName}_${keyName}_${keyVersion}'
parent: sqlServer
properties: {
serverKeyType: 'AzureKeyVault'
uri: keyUri
}
}
// Create the encryption protector
resource propector 'Microsoft.Sql/servers/encryptionProtector#2022-05-01-preview' = {
name: 'current'
parent: sqlServer
properties: {
serverKeyType: 'AzureKeyVault'
serverKeyName: sqlServerKey.name
}
}
Then you can invoke it from a parent module:
param sqlServerName string
param keyVaultName string
param keyName string
resource keyVault 'Microsoft.KeyVault/vaults#2022-07-01' existing = {
name: keyVaultName
}
resource keyVaultKey 'Microsoft.KeyVault/vaults/keys#2022-07-01' existing = {
name: keyName
parent: keyVault
}
module encryption 'sqlserver-keyvault-encryption.bicep' = {
name: 'sqlserver-keyvault-encryption'
params: {
sqlServerName: sqlServerName
keyVaultName: keyVault.name
keyName: keyVaultKey.name
keyVersion: last(split(keyVaultKey.properties.keyUriWithVersion, '/'))
keyUri: keyVaultKey.properties.keyUriWithVersion
}
}

[Azure Deployments]: Can we filter Azure Resource Group Deployments by name using c#?

I am trying to fetch the Azure Resource Group Deployments by using filter where name starting with "Deploy" but can't find any documentation on the $filter.
I tried to do something like below:
try
{
var credentials = new ClientSecretCredential(tenantId, clientId, clientSecret);
var resourceClient = new ResourcesManagementClient(subscriptionId, credentials);
var deployments = resourceClient.Deployments;
AsyncPageable<DeploymentExtended> rgDeployments = deployments.ListByResourceGroupAsync("myRG", "name eq 'Deploy-20210412184314'");
await foreach (DeploymentExtended deploymentProperties in rgDeployments)
{
Trace.WriteLine(deploymentProperties.Name);
}
}
catch(Exception e)
{
Trace.WriteLine(e.Message);
}
But it gives error saying -
{"error":{"code":"InvalidProvisioningStateFilter","message":"Invalid $filter 'name eq 'Deploy-20210412184314'' specified in the query string."}}
So can we only use filter like provisioningState eq '{state}' not for name?
I am using Azure Resources Management client library for .NET.
Please refer this documentation.
I am afraid you could not use the filter with name eq 'Deploy-20210412184314', in this case, if you already knew the name of the deployment and want to get it at the resource group scope, no need to use ListByResourceGroupAsync, just use this method DeploymentsOperations.GetAsync(String, String, CancellationToken), pass the resourceGroupName and deploymentName, you can simply get it.

Pulumi is creating an azure resource group with a different name

Why Pulumi is not creating a resource group with the name I gave?
Here is my small script
const azure = require("#pulumi/azure")
const resourceGroupName = new azure.core.ResourceGroup("test-pulumi", {
location: "francecentral",
});
The name of the resource group is: test-pulumi83d54581
The name parameter that you give is your component's name, not the full name of the resource to be created. By default, Pulumi appends a suffix to all names to avoid name collision e.g. across multiple stacks.
To specify the name explicitly, pass it as options parameter:
const resourceGroupName = new azure.core.ResourceGroup("test-pulumi", {
name: "test-pulumi",
location: "francecentral",
});
That's a bit verbose, and we might get an option to disable name suffixes - see this issue for progress.

is there a way to easily log all context information from within an azure apim policy?

Is there a way to easily log all context information from within an Azure APIM policy? The context variable has many available properties:
https://msdn.microsoft.com/en-us/library/azure/dn910913.aspx#ContextVariables
However, based on some fiddling around it seems like each of these properties needs to be specified explicitly for logging. Do you know if there's a way to simply log all available context information like context.ToString()? Simply including context or context.ToString() within the policy does not appear to allow the policy to compile.
You can try to serialize the object into string. checkout http://www.newtonsoft.com/json
Product product = new Product();
product.Name = "Apple";
product.Expiry = new DateTime(2008, 12, 28);
product.Sizes = new string[] { "Small" };
string json = JsonConvert.SerializeObject(product);
// {
// "Name": "Apple",
// "Expiry": "2008-12-28T00:00:00",
// "Sizes": [
// "Small"
// ]
// }

NServiceBus Event Subscriptions Not Working With Azure Service Bus

I'm attempting to modify the Azure-based Video Store sample app so that the front-end Ecommerce site can scale out.
Specifically, I want all instances of the web site to be notified of events like OrderPlaced so that no matter which web server the client web app happens to be connected to via SignalR, it will correctly receive the notification and update the UI.
Below is my current configuration in the Global.asax:
Feature.Disable<TimeoutManager>();
Configure.ScaleOut(s => s.UseUniqueBrokerQueuePerMachine());
startableBus = Configure.With()
.DefaultBuilder()
.TraceLogger()
.UseTransport<AzureServiceBus>()
.PurgeOnStartup(true)
.UnicastBus()
.RunHandlersUnderIncomingPrincipal(false)
.RijndaelEncryptionService()
.CreateBus();
Configure.Instance.ForInstallationOn<Windows>().Install();
bus = startableBus.Start();
And I've also configured the Azure Service Bus queues using:
class AzureServiceBusConfiguration : IProvideConfiguration<NServiceBus.Config.AzureServiceBusQueueConfig>
{
public AzureServiceBusQueueConfig GetConfiguration()
{
return new AzureServiceBusQueueConfig()
{
QueuePerInstance = true
};
}
}
I've set the web role to scale to two instances, and as expected, two queues (ecommerce and ecommerce-1) are created. I do not, however, see additional topic subscriptions being created under the videostore.sales.events topic. Instead, I see:
I would think that you would see VideoStore.ECommerce-1.OrderCancelled and VideoStore.ECommerce-1.OrderPlaced subscriptions under the Videostore.Sales.Events topic. Or is that not how subscriptions are stored when using Azure Service Bus?
What am I missing here? I get the event on one of the ecommerce instances, but never on both. Even if this isn't the correct way to scale out SignalR, my use case extends to stuff like cache invalidation.
I also find it strange that two error and audit queues are being created. Why would that happen?
UPDATE
Yves is correct. The AzureServiceBusSubscriptionNamingConvention was not applying the correct individualized name. I was able to fix this by implementing the following EndpointConfig:
namespace VideoStore.ECommerce
{
public class EndpointConfig : IConfigureThisEndpoint, IWantCustomInitialization
{
public void Init()
{
AzureServiceBusSubscriptionNamingConvention.Apply = BuildSubscriptionName;
AzureServiceBusSubscriptionNamingConvention.ApplyFullNameConvention = BuildSubscriptionName;
}
private static string BuildSubscriptionName(Type eventType)
{
var subscriptionName = eventType != null ? Configure.EndpointName + "." + eventType.Name : Configure.EndpointName;
if (subscriptionName.Length >= 50)
subscriptionName = new DeterministicGuidBuilder().Build(subscriptionName).ToString();
if (!SettingsHolder.GetOrDefault<bool>("ScaleOut.UseSingleBrokerQueue"))
subscriptionName = Individualize(subscriptionName);
return subscriptionName;
}
public static string Individualize(string queueName)
{
var parser = new ConnectionStringParser();
var individualQueueName = queueName;
if (SafeRoleEnvironment.IsAvailable)
{
var index = parser.ParseIndexFrom(SafeRoleEnvironment.CurrentRoleInstanceId);
var currentQueue = parser.ParseQueueNameFrom(queueName);
if (!currentQueue.EndsWith("-" + index.ToString(CultureInfo.InvariantCulture))) //individualize can be applied multiple times
{
individualQueueName = currentQueue
+ (index > 0 ? "-" : "")
+ (index > 0 ? index.ToString(CultureInfo.InvariantCulture) : "");
}
if (queueName.Contains("#"))
individualQueueName += "#" + parser.ParseNamespaceFrom(queueName);
}
return individualQueueName;
}
}
}
I could not, however, get NServiceBus to recognize my EndpointConfig class. Instead, I had to call it manually before starting the bus. From my Global.asax.cs:
new EndpointConfig().Init();
bus = startableBus.Start();
Once I did this, the subscription names appeared as expected:
Not sure why it's ignoring my IConfigureThisEndpoint, but this works.
This sounds like a bug, can you raise a github issue on this at https://github.com/Particular/NServiceBus.Azure
That said, I think it's better to use signalr's scaleout feature instead of using QueuePerInstance as signalr needs to replicate other information like (connection/group mappings) internally as well when running in scaleout mode.
Update:
I think I see the issue, the subscriptions should be individualised as well, which isn't the case in current naming conventions
https://github.com/Particular/NServiceBus.Azure/blob/master/src/NServiceBus.Azure.Transports.WindowsAzureServiceBus/NamingConventions/AzureServiceBusSubscriptionNamingConvention.cs
while it is in the queuenamingconventions
https://github.com/Particular/NServiceBus.Azure/blob/master/src/NServiceBus.Azure.Transports.WindowsAzureServiceBus/NamingConventions/AzureServiceBusQueueNamingConvention.cs#L27
As these conventions are public you can override them to work around the problem by changing the func in IWantCustomInitialization until I can get a fix in, just copy the current method and add the individualizer logic. The queue individualizer is internal though, so you'll have to copy that class from
https://github.com/Particular/NServiceBus.Azure/blob/master/src/NServiceBus.Azure.Transports.WindowsAzureServiceBus/Config/QueueIndividualizer.cs

Resources