I have an azure function hosted on an (S1) App Service Plan. The Azure Function is integrated to a VNet subnet. This subnet has Microsoft.Storage and Microsoft.Web service endpoints enabled, and also it's delegated to Microsoft.Web/serverFarms
On the other hand, the storage account is configured to accept request only from the same subnet the azure function is part of.
Unfortunately, that doesn't work. When I try to communicate with the storage account from the Azure function, I get the below error
2020-02-18T02:03:03.505 [Error] Faliure Occured
Azure.RequestFailedException : This request is not authorized to perform this operation.
RequestId:0b034a99-701e-002c-09ff-e5bd0a000000
Time:2020-02-18T02:03:03.1177265Z
Status: 403 (This request is not authorized to perform this operation.)
ErrorCode: AuthorizationFailure
Headers:
Server: Microsoft-HTTPAPI/2.0
x-ms-request-id: 0b034a99-701e-002c-09ff-e5bd0a000000
x-ms-client-request-id: 0bbe8185-4657-47f3-8566-5bcbd16c4274
x-ms-error-code: AuthorizationFailure
Date: Tue, 18 Feb 2020 02:03:02 GMT
Content-Length: 246
Content-Type: application/xml
at Azure.Storage.Blobs.BlobRestClient.Container.GetPropertiesAsync_CreateResponse(ClientDiagnostics clientDiagnostics,Response response)
at async Azure.Storage.Blobs.BlobRestClient.Container.GetPropertiesAsync(ClientDiagnostics clientDiagnostics,HttpPipeline pipeline,Uri resourceUri,String version,Nullable`1 timeout,String leaseId,String requestId,Boolean async,String operationName,CancellationToken cancellationToken)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Azure.Storage.Blobs.BlobContainerClient.GetPropertiesInternal(BlobRequestConditions conditions,Boolean async,CancellationToken cancellationToken)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Azure.Storage.TaskExtensions.EnsureCompleted[T](Task`1 task)
at Azure.Storage.Blobs.BlobContainerClient.GetProperties(BlobRequestConditions conditions,CancellationToken cancellationToken)
at SharedLib.Utils.TestStorageAccountAccess() at D:\poc-code\NetworkSecurityPoc\SharedLib\Utils.cs : 13
at async MessengerFunction.Trigger.Run(HttpRequest req,ILogger log) at D:\poc-code\NetworkSecurityPoc\MessengerFunction\Trigger.cs : 25
But when I disable the vnet restriction on the storage account, everything works.
What could I be doing wrong?
Thank you.
The below documentation might help why this is happening:
From MS documentation:
When you create a function app, you must create or link to a general-purpose Azure Storage account that supports Blob, Queue, and Table storage. You can't currently use any virtual network restrictions on this account. If you configure a virtual network service endpoint on the storage account you're using for your function app, that configuration will break your app.
Reference: enter link description here
I would say it's networking problem as per here Function networking So set WEBSITE_VNET_ROUTE_ALL to 1 then it should work..
Related
I am trying to setup the hashicorp Vault server in a Azure VM to connect to a storage blob. I tried uploading files from the VM and listings the blob using the CLI from the VM and was succesful. However, when I try to run my vault server I am getting the following
`
Error initializing storage of type azure: failed to get properties for container "CONTAINER NAME": -> github.com/hashicorp/vault/vendor/github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /gopath/src/github.com/hashicorp/vault/vendor/github.com/Azure/azure-storage-blob-go/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=AuthorizationPermissionMismatch) =====
Description=This request is not authorized to perform this operation using this permission.
RequestId: sdfsdfsdf-601e-00df-87897-f34329000000
Time:2022-11-08T19:57:49.5256170Z, Details:
Code: AuthorizationPermissionMismatch
GET https://MANAGEDIDENTITY.blob.core.windows.net/CONTAINERNAME?restype=container&timeout=5
Authorization: REDACTED
User-Agent: [Azure-Storage/0.11 (go1.15.11; linux)]
X-Ms-Client-Request-Id: [345345345-ee29-428c-7d92-bhjgjhuyssd]
X-Ms-Version: [2019-12-12]
--------------------------------------------------------------------------------
RESPONSE Status: 403 This request is not authorized to perform this operation using this permission.
Content-Length: [279]
Content-Type: [application/xml]
Date: [Tue, 08 Nov 2022 19:57:48 GMT]
Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
X-Ms-Client-Request-Id: [345345345-ee29-428c-7d92-bhjgjhuyssd]
X-Ms-Error-Code: [AuthorizationPermissionMismatch]
X-Ms-Request-Id: [345345345-ee29-428c-7d92-bhjgjhuyssd]
X-Ms-Version: [2019-12-12]
`
Here is my Vault config file
`
{
"listener": [{
"tcp": {
"address" : "127.0.0.1:8200",
"tls_disable" : 1
}
}],
"disable_mlock": "true",
"api_addr": "http://127.0.0.1:8200",
"storage": {
"azure": {
"accountName" : "AccountName",
"accountKey" : "",
"container" : "ContainerName",
"max_parallel": 512
}
},
"ui":true
}
I tried listing the blob from the VM and I was able to.
> --account-name "accountName" \
> --container-name containerName \
> --output table \
> --auth-mode login
Name Blob Type Blob Tier Length Content Type Last Modified Snapshot
---------- ----------- ----------- -------- -------------- ------------------------- ----------
helloworld BlockBlob Hot 13 text/plain 2022-11-08T21:14:44+00:00```
enter code here
403 This request is not authorized to perform this operation using this permission.
The above 403 errors occurs you may not given proper permission to your storage account and also you may not assign roles in storage account.
For service principal authentication purpose you need to assign roles in your storage account.
Storage Blob Data Contributor
Storage Blob Data Reader
Also check the firewall settings whether, In networking if you are access in public enable the select all network or if you enabled selected networks add the virtual networks.
If you add firewall , add your client iP address and also enable "Allow trusted Microsoft services to access this storage account" allows you to access storage account.
Reference:
Azure permission : not authorized to perform this operation - Stack Overflow
I have created a custom template for Dataflow Batch Jobs. Now I need to run every 5 minutes using cloud scheduler.
The template is stored in cloud storage. But I'm getting 401 error, whenever I pass the URI of template in my POST request from scheduler
URI - "https://dataflow.googleapis.com/v1b3/projects/project-id/locations/region/templates:launch?gcsPath=gs://my_bucket/templates/mytemplate
I'm not sure what the body should contain
Error:
jsonPayload: {
targetType: "HTTP"
#type: "type.googleapis.com/google.cloud.scheduler.logging.AttemptFinished"
status: "UNAUTHENTICATED"
url: "https://storage.cloud.google.com/my_bucket/templates/mydemotemp"
jobName: "projects/project-id/locations/us-central1/jobs/finaljob"
}
httpRequest: {
status: 401
Can anyone please help as how to go about it. Since I'm new to GCP
It could happen when the service account that you used for Cloud scheduler don't have permission to access to bucket.
You can use your service account for cloud scheduler authentication.
Set service account like below, Or refer here to get detailed guide.
Before setting your cloud scheduler, grant your service account a role like "Storage Object view" or higher level.
Refer here to check GCS roles that required for bucket access.
I can't find an easy way to communicate to Microsoft AzureDevops team.
I'm working on some tools that are working with Azure 2019 and there's also Azure 2020 RC right now. Microsoft deprecated SOAP API for 2019. And 2020 also, for REST Client.
Thou in 2019 REST client is not fully provided. GraphClient: https://learn.microsoft.com/en-us/rest/api/azure/devops/graph/?view=azure-devops-rest-6.0 is missing there. That means, no way to get UserGraph and Users, but through SOAP API the old way.
I though that maybe with Azure 2020 it's gonna be changed, but actually no.
I'm using Microsoft.VisualStudio.Services.Client in newest preview and latest stable version. That are: 16.173.0-preview and 16.153.0. Calling it manually making REST requests won't help either.
My test code:
var uriToServer = new Uri("http://localhost:81/DefaultCollection");
var vssBasicCredential = new VssCredentials(new WindowsCredential(new NetworkCredential("test", "test")));
using var vssConnection = new VssConnection(uriToServer, vssBasicCredential);
using var graphHttpClient = vssConnection.GetClient<GraphHttpClient>();
var listUsersAsync = await graphHttpClient.ListUsersAsync();
And we're getting:
handled Exception: System.AggregateException: One or more errors occurred. (API resource location 005e26ec-6b77-4e4f-a986-b3827bf241f5 is not registered on http://localhost:81/DefaultCollection.) --->
Microsoft.VisualStudio.Services.WebApi.VssResourceNotFoundException: API resource location 005e26ec-6b77-4e4f-a986-b3827bf241f5 is not registered on http://localhost:81/DefaultCollection.
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.CreateRequestMessageAsync(HttpMethod method, IEnumerable`1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content,
IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken, String mediaType)
at Microsoft.VisualStudio.Services.Graph.Client.GraphHttpClient.ListUsersAsync(IEnumerable`1 subjectTypes, String continuationToken, Object userState, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at ConsoleApp1.Program.Main(String[] args) in C:\Users\user\RiderProjects\ConsoleApp1\ConsoleApp1\Program.cs:line 17
The same problem as with 2019. Of course other clients are working perfectly.
The same code works perfectly for Azure DevOps Cloud - just have to change VssCredentials to VssBasicCredentials. Thou Basic I know can work for server if SSL is active.
Yes, currently graph api is not available for Azure DevOps Server 2019 and Server 2020.
You could add your request for this feature on our UserVoice site, which is our main forum for product suggestions. After suggest raised, you can vote and add your comments for this feedback. The product team would provide the updates if they view it.
As an alternative, you can try to use azure cli. With the Azure DevOps extension for Azure Command Line Interface (CLI), you can manage many Azure DevOps Services from the command line.
The Azure DevOps Command Line Interface (CLI) is available for Azure
DevOps Server 2020 and Azure DevOps Services.
About getting started with Azure DevOps CLI ,please refer to this document.
We use Azure DevOps for building and deploying our applications and services in Azure. Right now we are also switching our on premise deployments to Azure DevOps as well.
Unfortunately we have one issue with our servers which are on the one hand protected by a firewall (outgoing internet traffic is mostly blocked) and on the other hand are part of Azure DevOps deployment groups. As soon as we create a new release and start the deployment to a stage the artifact download process fails since the server is not permitted to download the artifacts from the blob storage where the artifacts are stored.
Warning, https://elgvsblobprodsu6weus10.blob.core.windows.net/dbcXXXXXXX/XXXX
Try 1/5, retryable exception caught. Retrying in 00:00:01. Details:
2019-10-21T19:59:02.7071549Z No LastRequestResponse on exception HttpRequestException: An error occurred while sending the request. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: Error 12029 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'A connection with the server could not be established'.
2019-10-21T19:59:02.7071940Z at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
2019-10-21T19:59:02.7072055Z at System.Net.Http.WinHttpHandler.StartRequest(WinHttpRequestState state)
2019-10-21T19:59:02.7072112Z --- End of inner exception stack trace ---
2019-10-21T19:59:02.7074791Z at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
2019-10-21T19:59:02.7074976Z at Microsoft.VisualStudio.Services.Common.TaskCancellationExtensions.EnforceCancellation[TResult](Task`1 task, CancellationToken cancellationToken, Func`1 makeMessage, String file, String member, Int32 line)
2019-10-21T19:59:02.7075133Z at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.<>c__DisplayClass56_0.<<GetRedirectResponseAsync>b__0>d.MoveNext()
2019-10-21T19:59:02.7075213Z --- End of stack trace from previous location where exception was thrown ---
2019-10-21T19:59:02.7075312Z at Microsoft.VisualStudio.Services.Content.Common.AsyncHttpRetryHelper`1.InvokeAsync(CancellationToken cancellationToken)
So is there any list of endpoints which are used for storing Azure Pipeline artifacts that can be added to our firewall policy, do we have to exclude *.blob.core.windows.net in general or is there any different approach?
Unfortunately the Microsoft docs do only provide a list of Urls which are required for establishing a connection with Azure DevOps in general, but not with the artifacts store.
Is there any list of endpoints which are used for storing Azure
Pipeline artifacts that can be added to our firewall policy.
Of course yes. We store these data in the corresponding region's Azure Data center.
Download the Json file in this link, the file list the IP Address range of different regions. But, here is one important thing you need pay attention to. For security, we update these IP address range regularly. (Also, publish it to users regularly)
If you choose this approach to achieve you want, you need to download the JSON file regularly, then make necessary updates in your access rules:
[!IMPORTANT] IP address ranges for Azure services can change, and
updates are published weekly. Download the JSON file regularly, and
make necessary updates in your access rules. If your scenario involves
configuring network security group rules in an Azure virtual network
to access Azure Container Registry, use the AzureContainerRegistry
service tag instead.
See this doc: Allow access by IP address range.
But, if you think this method is inconvenient for you and your team. Another unique method is the one you are using now: allow access to all Azure blob storage accounts using the wildcard *.blob.core.windows.net.
I am trying to hit Azure blob service from POST man. I have the Azure account name, sharedKey which we are using from java code to hit Azure media services.
I had read through the Azure documentation for the reference but it's not working when I hit from POST man though I passed all the required credentials like 'Azure accountName, shared key', please find the below details which I am using from Postman.
URL - https://xxxx.blob.core.windows.net/yyy/myblob
(Note:xxx- accountName, yyy- container)
header:
x-ms-date: value
x-ms-blob-type: BlockBlob
x-ms-version: 2017-11-09
Authorization: SharedKey xxx:hascodeKey (which we are using for media service which is working one)
I want to know, is the shared key same for Azure media service and Azure blob service?
Can somebody help me on this?