i am calling Azure Rest API to list all blobs in a directory (inside container) throuh informatica cloud using web service transformation.But i am getting error while running the mapping InvalidUriThe requested URI does not represent any resource on the server.
API: https://<account_name>.blob.core.windows.net/training?restype=container&comp=list&prefix=training/Type/Class
Could not reproduce your issue, the REST API - List Blobs works fine on my side.
You could refer to the sample below, make sure you are using it the same as mine.
Note: When you using prefix, you have already used the container name training in the url, don't use it again in the parameter, it should be prefix=Type/Class.
Request URL:
GET https://accountname.blob.core.windows.net/training?restype=container&comp=list&prefix=Type/Class
Request header:
x-ms-version = 2019-12-12
Test in the postman:
My storage structure:
Related
I am using Azure Container apps with Azure Blob Store as a state store. It is a simple Hello World (weather service) app using dotnet 6. App starts up fine, on Post I am trying to save the generated weather information to Azure Blob Store as JSON. I have configured Dapr components in Azure Container Apps for StateStore using Azure blob storage. I am using storage key (secondary key) as explained in this Microsoft documentation
Upon doing a Swagger and looking at log I get the following error.
Dapr.DaprException: State operation failed: the Dapr endpoint indicated a failure. See InnerException for details.
2022-07-17T01:10:35.716245402Z ---> Grpc.Core.RpcException: Status(StatusCode="Internal", Detail="failed saving state in state store statestore: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /home/vsts/work/1/go/pkg/mod/github.com/!azure/azure-storage-blob-go#v0.10.0/azblob/zc_storage_error.go:42
2022-07-17T01:10:35.716524109Z ===== RESPONSE ERROR (ServiceCode=AuthenticationFailed) =====
2022-07-17T01:10:35.716795515Z Description=Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
2022-07-17T01:10:35.716812515Z RequestId:863bcef4-401e-0069-5f7a-99724b000000
2022-07-17T01:10:35.716820115Z Time:2022-07-17T01:10:35.7137648Z, Details:
2022-07-17T01:10:35.716825516Z AuthenticationErrorDetail: Issuer validation failed. Issuer did not match.
2022-07-17T01:10:35.716831516Z Code: AuthenticationFailed
Error is Authentication Failed. I am unsure what I am missing since I am not making any additional config in storage account such as VNET service end point etc. Account is enabled for Key access. Any help is appreciated.
Below is the code that I am using
using var client = new DaprClientBuilder().Build();
var forecast = new WeatherForecast()
{
Date = DateTime.Now.AddDays(1),
TemperatureC = Random.Shared.Next(-20, 55),
Summary = Summaries[Random.Shared.Next(Summaries.Length)]
};
await client.SaveStateAsync<WeatherForecast>(stateStoreName,key,forecast);
Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
i think i found the answer. The issue was with metadata since the key set in metadata of the component.yaml as mentioned in the Microsoft documentation is not working. I changed it to use secretref and referred in metadata directly in the portal. Not sure why the error was showing Authentication error but it is finally working.
I'm able to update/create the function key using the API as per document.
https://learn.microsoft.com/en-us/rest/api/appservice/web-apps/create-or-update-function-secret
My main aim is to update the function key every hour so I'm creating a http trigger (with the above api inside it) and scheduling the trigger.
For testing purpose I stored the url in one parameter.
URL:
'https://management.azure.com/subscriptions/xyz1/resourceGroups/xyz2/providers/Microsoft.Web/sites/func_appname/functions/func_name/keys/poc_testing1?api-version=2021-02-01{"Properties":{"Name": "poc_testing1","Value": "asdsda"}}'
Note: Value here is updating via random gen lib of python
Generated a bearer token using the service principal (which I'm already using to connect my stg acc) storing it in auth_token
header_auth= {'Authorization' : 'Bearer ' + auth_token }
Now running the below command in python
import requests
requests.post(url, headers=header_auth)
I'm getting 403 forbidden error
I'm thinking that it is not because of the bearer token, Did google the error and it is with the IP address. Can someone help me out here
I was referring the (https://learn.microsoft.com/en-us/troubleshoot/azure/general/request-throttling-http-403) doc but I'm not using any APIM service
Till now I referred the doc from MSFT.
https://learn.microsoft.com/en-us/rest/api/appservice/web-apps/create-or-update-function-secret
I was able to create new function key.
I'm trying to do the same using python for which I performed the above steps.
Currently ran the above issue steps in my local Visual studio and tried az cli as well but same 403 error.
Why do you want to update the function key every hour?
If you aim to increased security use AzureAD Auth/OAuth2 rather than the function key.
Regarding the 403 error, please ensure you have assigned proper permissions to the service principal which allow the service principal to modify the azure function.
I'm trying to use a service principal to access and list the contents of an Azure storage container using PowerShell. I've been able to get a token for the service principal and use that to access the storage account but when I run this command
$iwrParams = #{
'Uri' = 'https://myaccount.blob.core.windows.net/mycontainer?restype=container&comp=list'
'UseBasicParsing' = $true
'ContentType' = 'application/xml'
'Headers' = #{
'Authorization' = "Bearer $($token.access_token)"
'x-ms-version' = '2017-11-09'
}
}
Invoke-WebRequest #iwrParams
I get a return from this but the return xml content never shows the URL for each returned blob. If you look at the return at https://learn.microsoft.com/en-us/rest/api/storageservices/enumerating-blob-resources#list-blobs-and-snapshots it shows that for each blob it should return a Name, Url and various other properties. When I run the code above I get everything but the Url.
Now the really interesting thing is if I change the container access to anonymous public and run this code, I get the Url returned as expected.
$iwrParams = #{
'Uri' = 'https://myaccount.blob.core.windows.net/mycontainer?comp=list'
'UseBasicParsing' = $true
'ContentType' = 'application/xml'
}
Invoke-WebRequest #iwrParams
The issue honestly seems to be having to include restype=container when accessing with any sort of authentication.
My question is does anyone know a way to get the URL for each blob returned when not using anonymous access to list container contents?
I'd like to do this without resorting to the Az modules.
The reason you're not able to see the URL returned in the response is because of the storage REST API version used by your code (2017-11-09). Essentially the Blob URL property was removed from the response starting with REST API version 2013-08-15. From this link:
In version 2013-08-15 and newer, the EnumerationResults element
contains a ServiceEndpoint attribute specifying the blob endpoint, and
a ContainerName field specifying the name of the container. In
previous versions these two attributes were combined together in the
ContainerName field. Also in version 2013-08-15 and newer, the Url
element under Blob has been removed.
Regarding your comment about why you can see the URL property if you list blobs anonymously, this is happening because if no REST API version is specified in the request, Storage Service uses the oldest REST API version to process the request if default version has not been set. From this link:
If an anonymous request to a general-purpose storage account does not
specify the x-ms-version header, and the default version for the
service has not been set using Set Blob Service Properties, then the
service uses the earliest possible version to process the request.
Considering you would want to use Azure AD based authorization, the earliest version you will be able to use is 2017-11-09 thus it will not be possible to get the Blob URL returned in the response body.
One option would be to manually construct the Blob URL by using Blob Container URL and Blob name. Other option would be to use Shared Key Authorization instead of Azure AD authorization and specifying a version earlier than 2013-08-15 for x-ms-version request header in your requests. You will need to manually compute Authorization header value in this case using instructions provided here.
I am trying to export an Azure package using the GetPackage method of Service Management API.
I have tried both calling the REST API directly using a WebClient, and by using the Windows Azure Service Management Library package (I have posted the code I used as an answer to that question).
However, no matter the method I tried and how I constructed and/or encoded the container URI, I am always getting the following error:
400 Bad Request
Parameter value '...' specified for parameter 'ContainerUriString' is invalid.
The parameter in question is of the following form:
https://something.blob.core.windows.net/somecontainer
I verified that the storage account exists and is accessible (tried both public and private containers), even tried calling HttpUtility.UrlEncode() on the container URI (even though the SDK does it automatically).
Any ideas how to get this resolved?
Please ensure that the storage account where you want the files to be copied belong to the same subscription as that of Cloud Service.
I'm trying to make some calls to the new Azure Scheduler API. However, all my requests come back with this error:
<Error xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<Code>AuthenticationFailed</Code>
<Message>The server failed to authenticate the request. Verify that the certificate is valid and is associated with this subscription.</Message>
</Error>
I'm pretty sure that I have everything setup correct because I can make calls using the same code and certificate to the Azure Service Management API.
The code I'm using to attach the certificate to the web request is from the MSDN Sample. The Scheduler API calls that I've tried to make are the Check Name Availability, Create Cloud Service, and Create Job Collection.
I've also verified that my subscription is Active for the preview of the Scheduler.
Here is an example of a request I've tried:
Create Cloud Service
Request A cloud service is created by submitting an HTTP PUT operation
to the CloudServices OData collection of the Service Management API
Tenant.Replace with your subscription ID and
with your cloud service ID.
So for this I create a web request pointing to:
https://management.core.windows.net/[MySubId]/cloudServices/[MyNewServiceName]
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(requestUri);
// Define the requred headers to specify the API version and operation type.
request.Headers.Add("x-ms-version", "2012-03-01");
request.Method = "PUT";
request.ContentType = "application/xml";
Next I add the request body as specified in the documentation:
<CloudService xmlns:i='http://www.w3.org/2001/XMLSchema-instance' xmlns='http://schemas.microsoft.com/windowsazure'>
<Label>[MyServiceName]</Label>
<Description>testing</Description>
<GeoRegion>uswest</GeoRegion>
</CloudService>
And finally I add the certificate that I use with my subscription to the account.
// Attach the certificate to the request.
request.ClientCertificates.Add(certificate);
I try to get the response and instead I get the error shown above.
BTW - I've also tried different regions thinking maybe it was a region issue since the scheduler isn't supported in all regions, but I still get the same response.
You need to register the scheduler in your application first by calling (PUT):
<subscription id>/services?service=scheduler.JobCollections&action=register
If you want to do this in .NET you can use the new Management libraries:
var schedulerServiceClient = new SchedulerManagementClient(credentials);
var result = schedulerServiceClient.RegisterResourceProvider();
Console.WriteLine(result.RequestId);
Console.WriteLine(result.StatusCode);
Console.ReadLine();
More detail: http://fabriccontroller.net/blog/posts/a-complete-overview-to-get-started-with-the-windows-azure-scheduler/