I'm trying to create a web activity in ADF to run a query in log analytics and have this result as output in ADF (Data Factory).
Is it possible ?
I tried to generate a ADD token and passed it to web Activity:
and the url that I'm using is: https://api.loganalytics.io/v1/workspaces/[Workspace ID]/query
but I got an error:
I already gave access to app like reader and data access to Log Analytics.
Can anyone please help me in achieving this?
Thank you!
I think you have to get the bearer token and then pass that bearer token when you make the actual call to log analytics API , its going to be two step workflow . I use to test the thing in curl or postman and once it succeed there I try to implement in ADF .
https://learn.microsoft.com/en-us/azure/azure-monitor/logs/api/access-api#client-credentials-flow
Related
I am trying to pull azure cognitive search current and quota .
Can anyone help me on:
" how can I get that information to csv using some service principle."
I found one link to pull this kind of data :
https://learn.microsoft.com/en-us/rest/api/searchservice/get-service-statistics
But I am not sure how to use this api to get above screenshot information.
The link you posted is the correct way to do it, it returns all the information you need.
It looks like this endpoint does not support OAuth2 & RBAC. So instead of using service principal, you need to provide the admin api-key in the request header.
You can check here how to access the api-key. If you'll be doing that from a powershell script, you can authenticate with your service principal and fetch the key using Get-AzSearchAdminKeyPair, then use this key to make a http request to get the statistics and finally convert them to CSV format.
I'm trying to query two different application insight instances (instance A and B) trough the REST API. I'm using postman to send a GET http request to the API and followed the answer in this post, which was aiming towards the same goal, join request data from different applications insights: https://stackoverflow.com/a/52248597/17161618
Just as it says in the above mentioned post I'm acessing the instance A and passed the authentication through the request header in the format keyA:appIdA,KeyB:appIdB.
I'm sending the following query:" union app("AppIdA").traces,app("AppIdB").traces " and receive the following error: enter image description here
Does anyone know how can I get access to read logs in resource B ?
I don't think you can run cross-component queries using API keys. You should use AAD (Azure Active Directory) auth. Give AAD app access to both resources. Then you should be able to run such queries.
I am trying to create a list of all the workspaces and the reports contained in each one of them for a documentation project.
I found online that we can use this to get the workspaces, I wanna use it with a "web" activity:
https://api.powerbi.com/v1.0/myorg/groups
And then I want to use the IDs we get in the output with a foreach and another web activity inside of it and use this to get the reports in each workspace, then copy it somewhere (datalake or DB) :
https://api.powerbi.com/v1.0/myorg/groups/{groupId}/reports
But I don't know how to configure the activity and the authentication.
If there is a better way like connecting directly to Power BI I'm all ears, I tried to do a get data from web source but I don't have any "key" for API and organization authentication doesn't work.
When I run the code here: https://learn.microsoft.com/en-us/rest/api/power-bi/groups/getgroups it works perfectly.
Thanks in advance
'But I don't know how to configure the activity and the authentication.' For this question, I'm not sure if you don't know how to use access token to call an API.
Register an azure ad application with the api permission of 'https://analysis.windows.net/powerbi/api/Workspace.ReadWrite.All'
so that you can generate an access token by this application with
this scope. Add api permission.
Use ropc flow to generate an access token. This flow contains user information so that your api could know who you are to return correct groups.
Add 'Authorization' in request head with value of 'Bearer accessToken' to call the api.
ropc flow 1.0:
I am trying to pull data from Hubspot to ADF using the Hubspot connector in ADF.
I have also registered Hubpost with the Azure AD.
Followed the steps as per the doc, still unable to create the linked service.
https://learn.microsoft.com/en-us/azure/data-factory/connector-hubspot#linked-service-properties
Error message
I have used Postman app to generate the Access & refresh tokens.
Any help is appreciated. Thanks.
In the Hubspot developer account, Select the scope. Then use the Postman app, to generate the Access and refresh tokens by entering the required parameters. Use the POST method to get the tokens.
Use the fresh tokens generated in the ADF linked service to establish connections.
Once the connection is established, Hubspot will list the tables which are available and can be copied using ADF.
Create a new pipeline in ADF, select the Lookup activity, then the copy activity to copy the desired table into the destination.
I am trying to dynamically add/update linked service REST based on certain trigger/events to consume a RESP API to be authenticated using cookie which provides telemetry data. This telemetry data will be stored in Data Lake Gen2 and then will use Data Bricks to move to secondary data storage/SQL Server.
Have someone tried this? I am not able to find the cookie based Auth option while adding the linked service REST.
Also how to create data pipes dynamically or to have the parameters of the rest api to be dynamic ?
Currently, unfortunately this is not possible using Azure data factory native components/activities. For now at least, you cannot get access to the response cookies from a web request in data factory. Someone has put a feature request for this or something that might help, see here
It might be possible to do this via an Azure function to get/save the cookie and then send it as part of a following request. I had a similar problem but resorted to using Azure functions for all of it, but I guess you could just do the authentication part with a function! ;-)
EDIT: update
Actually, after I wrote this I went back to check if this was still the case and looks like things have changed. There now appears (never seen this before) in the web response output, a property called "ADFWebActivityResponseHeaders" and as you can see there is property for the "Set-Cookie"
See example below:-