I want to use Consul for Service Discovery and as Distributed Configuration for my microservices. Without any security my system is working fine but I don't want every user to be able to change values in the key-value store. So I tried to introduce security with Consul's ACL but now my services aren't able to register themselves anymore.
I set the default policy in my HCL configuration to "deny" and tried to set a write policy for every service so they can register themselves at Consul and a read policy for the key-value-store, so the values can only be read via the UI. But my services receives a 403 during registration process.
agent.hcl
service_prefix "" {
policy = "write"
}
key_prefix "" {
policy = "read"
}
consul-policy.hcl
service_prefix "" {
policy = "write"
}
key_prefix "" {
policy = "read"
}
I started the server with:
consul agent -config-file agent.hcl -dev
And added the policy (after getting and setting an ACL token):
consul acl policy create -name consul-server-one -rules #consul-policy.hcl
How can I define a read-only policy for the key-value store in the UI and a write policy for services?
Is it possible to start Consul with the ACL configuration (instead of delivering it after the startup process)?
Which ACL resources are the right ones for my problem? Are service_prefix and key_prefix correct choices?
What I did miss was the fact that you need to generate tokens for a policy.
consul acl token create -description "Token description" -policy-name "consul-server-one" -token "(your global token)"
And I wanted the usage of a default token which can be set via:
consul acl set-agent-token -token "(your global token)" default (your generated token)
Related
I have a Bicep template where I create an App Service in which I need to link a SSL certificate that exists in Key Vault (both in same resource group).
The Key Vault has Azure RBAC enabled.
I use the following to Bicep template to link the SSL certificate from Key Vault to the App Service:
resource certEncryption 'Microsoft.Web/certificates#2018-02-01' = {
name: '${resourcePrefix}-cert-encryption'
location: location
properties: {
keyVaultId: resourceId('myResourceGroup', 'Microsoft.KeyVault/vaults', keyVaultName)
keyVaultSecretName: '${resourcePrefix}-cert-encryption'
serverFarmId: hostingPlan.id
password: 'SecretPassword'
}
dependsOn: [
webApi
]
}
But it fails with the following message:
The service does not have access to '/subscriptions/3449f-xxxx/resourcegroups/rgabptrialt/providers/microsoft.keyvault/vaults/my-test-vault' Key Vault. Please make sure that you have granted necessary permissions to the service to perform the request operation.
This isn't really telling a lot...
What permission do I need to grant exactly? And to what? And where and how do I even grant these permissions?
Do I have to create a Managed Identity and link that in my App Service? And what Permissions/Roles do I need exactly? Or do I need to do something else to make this work?
I couldn't really find any good info on how to do this.
Give App Service enough permissions to link SSL certificate from Key Vault:
I've imported a self-signed certificate (.pfx) in keyvault -> secrets to authenticate.
To resolve,
The service does not have access to '/subscriptions/3449f-xxxx/resourcegroups/rgabptrialt/providers/microsoft.keyvault/vaults/my-test-vault' Key Vault. Please make sure that you have granted necessary permissions to the service to perform the request operation.
I tried in my environment by referring few steps from this article detailed by #Anuraj and modified accordingly to achieve the expected results.
Step-1: Setting up a Key Vault access policy:
Create a key vault from the portal and Goto Access Policies.
Now creating an access policy to link the ssl certificate by configuring a template as per your requirement.
I've selected key,secret & certificate management to enable the permissions as shown:
Search for the "Name/objectID/AppID" for the respective service principal as keyvault has RBAC enabled.
Note: Register an app under AzureAD -> App registrations if needed.
Review all the permissions and create an access policy:
Step-2: Under App Service -> Web Application -> Certificates, I've added the keyvault certificate ( self-signed certificate (.pfx) in keyvault -> secrets).
1.
2.
3.
Bind the SSL certificate by adding TLS/SSL settings and importing the key vault certificate when it has been added to key vault (.pfx).
Note: Make sure the certificate is in .pfx format to avoid any conflicts.
And I ran below script and deployment got succeeded without any permission blockers.
resource certEncryption 'Microsoft.Web/certificates#2018-02-01' = {
name: 'xcc-cert-encryption'
location: 'EastUS'
properties: {
keyVaultId: '/subscriptions/<subscriptionID>/resourceGroups/xxxxRG/providers/Microsoft.KeyVault/vaults/xxxxxkeyvaults'
keyVaultSecretName: 'jahnss'
password: 'xxxxx' //Certificate protected password
extensionResourceId: '/subscriptions/<subscriptionID>/resourceGroups/xxxxRG/providers/Microsoft.Web/serverfarms/xxxxxappserviceplan'
}
}
Output:
I think yes first you need to give permission in key vault
Ensure that the service principal has all the permissions. The only thing that worked for me though is adding the service principal 24681998-555f-4570-a559-2fced2d7e841 which shows up as Microsoft.Azure.WebSites. You can add this through the portal, by adding an access policy for Microsoft.Azure.WebSites or through arm with the GUID.
I added the following principal to the Key Vault access policies: Microsoft Azure App Service (object id: your object id). Permission to get secrets is enough.
By performing similar the steps mentioned in the below answer
How to access key vault from azure app service
Please let me know if you have any doubts or question. Even if you are facing any issues.
In the terraform documentation for google_pubsub_subscription, it mentions having a oidc_token property under push_configuration allows push-subscriptions to utilise service account, but has no mention on how to use service account for pull subscriptions.
How can explicitly set the service account to be used during the creation of pull pubsub subscriptions?
I tried adding oidc_token block, but it did not worked as it does not expect that block directly.
Scenario:
I have a service account that has access to pubsub topics (and necessary permissions to attach subscriptions to it) in Project_A
I want to create subscriptions to those topics in Project_B in terraform.
I need to explicitly use this service account in terraform, so i can create subscriptions at Project_B to topics of Project_A
google_cloudfunctions_function resource for example, has field called service_account_email for setting service account. But there is no for google_pubsub_subscription resource, for pull subscriptions in the documentation.
Actually the service account (service_account_email)is to be specified inside the oidc_token section (you had to go further in the doc :)
below a working example of a push sub with service account and audience (optional)
variable "project" {
type = string
default = "<YOUR_PROJECT_ID>"
}
resource "google_pubsub_topic" "example" {
project = var.project
name = "example-topic"
}
resource "google_pubsub_subscription" "example" {
project = var.project
name = "example-push-subscription-with-auth"
topic = google_pubsub_topic.example.name
push_config {
push_endpoint = "https://example.com/push"
oidc_token {
service_account_email = "${var.project}#appspot.gserviceaccount.com"
audience = "https://example.com/push"
}
}
}
I'm adding another answer, because the question has changes a lot after the different comments.
Here is the solution with the hypothesis that a topic topic-sof is already created in a different project than the subscription one.
I created a service account (SA) on the subscription project, I called it stackoverflow-playground-sa and gave it only the Pub/Sub Editor role as you can see in the screenshot below.
I gave the SA the the Pub/Sub Subscriber role on the topic topic-sof s show in the screenshot below.
If you don't do this step: you will get this error at terraform apply
Error: Error creating Subscription: googleapi: Error 403: User not authorized to perform this action
Of course I did the 2 first steps of roles assignments with a user with enough permissions on both projects
I created a json key file for my SA and downloaded under /path/to/my/key/stackoverflow-playground-sa.json
I authenticated as my SA using
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/my/key/stackoverflow-playground-sa.json
So it can be used by terraform to create the subscription.
Here is the terraform configuration to create the subscription
variable "subscription_project" {
type = string
default = "<YOUR_SUBSCRIPTION_PROJECT>"
}
variable "topic_project" {
type = string
default = "<YOUR_TOPIC_PROJECT>"
}
resource "google_pubsub_subscription" "pull-subscription-of-topic-in-another-project" {
project = var.subscription_project
name = "pull-subscription-of-topic-in-another-project"
topic = "projects/${var.topic_project}/topics/topic-sof"
}
Run (apply) my terraform and the subscription pull-subscription-of-topic-in-another-project is created and attached to the topic topic-sof in the other project.
I published a message to topic-sof using the web ui.
Still authenticated as stackoverflow-playground-sa thanks to step 4, I pull the message (using gcloud in my treminal) et voilà message received:
To summarise : there is no service account to specify in your terraform configuration for this requirement. The service account and its key is set outside terraform (step 1 to 4) so that terraform process can be authenticated and authorised to create the resources configured (the subscription in our case).
Besides using a service account key as in step 4 is a bad practice, security wise. An alternative is to use
gcloud auth application-default login
That will let you set the default credentials of the user of your choice. Given that user has the roles I set for the SA.
In IBM Cloud, I have an IAM Access Group for security admins. What policy do I need to grant to have their members READ access to user-specific authorizations, i.e., access policies granted to a user, not an Access Group?
The account owner can see those authorizations by, e.g., the List Policies API. The security admin, when calling that API, either receives an empty list or only a partial list. The Access Group for security admins already has Administrator privilege for IAM Identity Service and IAM Access Group Service.
If the policies are based on a resource group you might need Viewer access to the resource group. In terraform it would be something like this:
resource "ibm_iam_access_group_policy" "shared_policy" {
access_group_id = ibm_iam_access_group.shared.id
roles = ["Viewer"]
resources {
resource_type = "resource-group"
resource = ibm_resource_group.shared.id
}
}
New resource groups could be added in the future...
To see access policies, the security administrators and hence their related Access Group need *Viewer* privilege on all resources and services that are directly "authorized" to users or service IDs. It is not enough to have Viewer or even Administrator role on IAM Access Groups Service, Viewer on all Account Management as well as on all IAM-enabled services is required.
The following would give Viewer on Account Management services when using Terraform:
resource "ibm_iam_access_group_policy" "cloud-security-admins-account_viewer" {
access_group_id = ibm_iam_access_group.cloud-security-admins.id
account_management = true
roles = [ "Viewer" ]
}
And the next Terraform snippet could be used to give Viewer on all IAM-enabled services:
resource "ibm_iam_access_group_policy" "cloud-security-admins-viewall-resources" {
access_group_id = ibm_iam_access_group.cloud-security-admins.id
roles = [ "Viewer" ]
resources {
resource_type = "resource-group"
}
}
Which Rbac action would allow Azure App Service Network --> Access Restrictions ? We dont want every user to have auth to set/unset ip-rules using 'Networking --- Access Restrictions' for app-services.
I have tried to change the network access and I can get the action information from the brower.
So the action should between this:
You can use a custom role,
$role = Get-AzRoleDefinition -Name "Virtual Machine Contributor"
$role.Id =$null
$role.Name = "testcustombowman"
$role.Description = "111111111111111111111111111111111111111111111"
$role.Actions.RemoveRange(0,$role.Actions.Count)
$role.Actions.add("Microsoft.Web/sites/config/Read")
$role.AssignableScopes.Clear()
$role.AssignableScopes.Add("/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx")
New-AzRoleDefinition -Role $role
Have a look of this Offcial doc, and this.
But I still recommend you to use a simple way, directly give the relevant people only Read permissions.
It is highly recommended that you use the simple method, add role assignment -> Select Reader -> Select User
As you can see, the relevant users are unable to operate the settings you said in my case.
When I try to link a variable group to key vault in VSTS, every time I select my endpoint, it tells me:
"Specified Azure endpoint needs to have "Get, List" secret management permissions on the selected key vault. Click "Authorize" to enable VSTS to set these permissions or manage secret permissions in Azure portal."
It makes no sense. That specific endpoint has Get and List for secrets, keys, certificates. The endpoint is using the correct Service Principal Client ID- I know because I pulled its corresponding app registration up in Azure by searching with that ID. And the app registration is specifically listed in the key vault's Access policies, with the correct permissions.
Trying to click "Authorize" in VSTS just give me
"Resource not found for the segment 'DirectoryDataService.getServicePrincipalsByAppIds'. For troubleshooting refer to https://go.microsoft.com/fwlink/?linkid=835898"
and the endpoint is broken until I re-verify it.
I'm kind of at my wit's end here- everything is set exactly as https://learn.microsoft.com/en-us/vsts/build-release/concepts/library/variable-groups?view=vsts says it should be.
Edit: Turns out the Azure tried to add the app reg as a person instead of an app when I listed it in the ARM template by object ID (app ID most definitely does not work there). So now I just need to figure out how to add it as an application in the ARM template...
Edit Edit: Soooo... the Object ID the Azure Portal shows for an App Registration? That's not the Object ID the ARM template wants. It wants the... I'm not sure what you'd call it. In Powershell, you use it with -Object ID, but when you list the properties, it's under "Id". Whatever. To get it, you run
Get-AzureRmADServicePrincipal -SearchString "[your-app-reg-name]"
And it shows up under Id. That's what you want to use as an object ID in your ARM template.
According to the error Resource not found for the segment 'DirectoryDataService.getServicePrincipalsByAppIds, the issue seems more related to Azure side.
To access azure-keyvault you need four things :
- clientId = "<client id of your application registed on Azure AD>";
- domain = "<your talnet id>";
- secret = "<client key of your application registed on Azure AD>";
- subscription = "<your subscription id>";
Then these will combine ApplicationTokenCredentials, finally Authorize to KeyVaultClient . Suggest you take a look at this question: Azure keyvault client 1.0.0 initiate client
Besides also double check/confirm the specific endpoint has Get and List for secrets, keys, certificates.