How to purge records from Application Insights bucket? - azure

I used to following instructions from https://learn.microsoft.com/en-us/rest/api/application-insights/components/purge, but now it produces the following error:
Purge operation on a workspace-based Application Insights resource is not supported. Instead, please run purge operation directly on the workspace, scoped to specific resource id. See more at https://aka.ms/purgeai"
OK, so https://aka.ms/purgeai are the new instructions, but it does not explain how to "scope the operation to the specific resource id", because the records are still in an AI bucket, not the workspace.
So, how do I purge from AI in the new world?
EDIT 1
I should have been more clear. I do not want to purge all the records. For example, consider the following purge request body, which was enough in the past:
{
"table": "customEvents",
"filters": [
{
"column": "customMeasurements",
"key": "seq",
"operator": "==",
"value": "''"
}
]
}
It could be used to purge all the records with customMeasurements.seq being empty. Now how do I compose it with the _ResourceId field?
When giving the following request body:
{
"table": "customEvents",
"filters": [
{
"column": "customMeasurements",
"key": "seq",
"operator": "==",
"value": "''"
},
{
"column": "_ResourceId",
"operator": "==",
"value": "/subscriptions/.../resourcegroups/.../providers/microsoft.insights/components/..."
}
]
}
The request fails with:
{
"error": {
"message": "The request had some invalid properties",
"code": "BadArgumentError",
"correlationId": "...",
"innererror": {
"code": "QueryValidationError",
"message": "'customEvents' is not a valid table"
}
}
}
EDIT 2
There is still a problem. Indeed, consider the following query:
AppEvents
| where Measurements.seq == '' and _ResourceId == '/subscriptions/d...8/resourcegroups/app508-re-mark-test/providers/microsoft.insights/components/app508-re-mark-test-ai'
| summarize count()
This query works and return some count.
Now I want to use the same condition to purge records in the same workspace:
{
"table": "AppEvents",
"filters": [
{
"column": "Measurements",
"key": "seq",
"operator": "==",
"value": "''"
},
{
"column": "_ResourceId",
"operator": "==",
"value": "/subscriptions/d...8/resourcegroups/app508-re-mark-test/providers/microsoft.insights/components/app508-re-mark-test-ai"
}
]
}
But I get back the following error:
{
"error": {
"message": "The request had some invalid properties",
"code": "BadArgumentError",
"correlationId": "c08bb441-a316-4989-a527-202a101e515d",
"innererror": {
"code": "QueryValidationError",
"message": "Unsupported column '_ResourceId'"
}
}
}
Note, that omitting the _ResourceId works, but I do want to be able to use it.
What am I missing?

Thank you for your question, and feedback.
What you will need to do, is to filter by the specific resource id, in addition to any other required filter.
Ex:
`{
"column": "_ResourceId",
"operator": "==",
"value": “/subscriptions/12341234-1234-1234-1234-123412341234/resourceGroups/SomeResourceGroup/providers/microsoft.insights/components/AppInsightResource”
}`
Please note,
The table names have changed between the Classic Applicaiton insights, and the workspace-based Application insights. For example, in your case the table name should be AppEvents instead of customEvents.
You can review this link for other table names changes.
We will also modify the articles to reflect this information.

Related

How to Purges data in an Log Analytics workspace with "between" operator by API?

I want to purges data for my custom log with TimeGenerated range.
Sample Request
POST https://management.azure.com/subscriptions/00000000-0000-0000-0000-00000000000/resourceGroups/OIAutoRest5123/providers/Microsoft.OperationalInsights/workspaces/aztest5048/purge?api-version=2020-08-01
Request Body
{
"table": "MY_CUSTOM_TABLE",
"filters": [
{
"column": "TimeGenerated",
"operator": "between",
"value": ["2022-01-02T04:00:00","2022-01-07T03:59:59"]
}
]
}
Response Code: 400
Response Body:
{
"error": {
"message": "The request had some invalid properties",
"code": "BadArgumentError",
"correlationId": "00000000-0000-0000-0000-00000000000",
"innererror": {
"code": "QueryValidationError",
"message": "The user with object Id '00000000-0000-0000-0000-00000000000' does not have the role 'Data Purger' required to perform purge operation on this resource"
}
}
}
"message": "The user with object Id
'00000000-0000-0000-0000-00000000000' does not have the role 'Data
Purger' required to perform purge operation on this resource"
Looking at the Error Message that you have shared above, it is moreover related to access issue. You need to have DataPurger RBAC role on that resource inorder to perform data purge over a particular table in log analytic workspace.
Here is the reference documentation for more information about DataPurger RBAC role.
We have tested this in our local environment , it is working fine.
In our local environment , we have tried to purge the heartbeat table using the above shared Purge REST API of Azure Log Analytics workspace.
if you are using the between operator in the body of your restAPI , the value should be array of items.
Here is the sample request & body of the Rest API :
https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{workspaceName}/purge?api-version=2020-08-01
Body:
{
"table": "Heartbeat",
"filters": [
{
"column": "TimeGenerated",
"operator": "between",
"value": ["2022-01-18T05:40:00","2022-01-18T05:50:59"]
}
]
}
Here is the sample Output for reference:

Azure DevOps API - Create new branch from master without adding changes

I'm attempting to use Azure DevOps Services Rest API to create a new branch from master but I've been unsuccessful.
Docs: https://learn.microsoft.com/en-us/rest/api/azure/devops/git/refs/update-refs?view=azure-devops-rest-5.1#examples
Endpoint: https://dev.azure.com/{{organization}}/{{project}}/_apis/git/repositories/{{repositoryId}}/refs?api-version={{api-version}}
Body:
[
{
"name": "refs/heads/new-test-branch-from-api-call",
"oldObjectId": "{{masterObjectId}}",
"newObjectId": "{{newObjectId}}"
}
]
Results:
{
"$id": "1",
"innerException": null,
"message": "TF401035: The object '****************************************' does not exist.",
"typeName": "Microsoft.TeamFoundation.Git.Server.GitObjectDoesNotExistException, Microsoft.TeamFoundation.Git.Server",
"typeKey": "GitObjectDoesNotExistException",
"errorCode": 0,
"eventId": 3000
}
A comment on this post states that this is the route to take. Also states that the repositoryId should be used and the newObjectId which results in:
{
"$id": "1",
"innerException": null,
"message": "An object ID must be 40 characters long and only have hex digits. Passed in object ID: ********-****-****-****-************.",
"typeName": "System.ArgumentException, mscorlib",
"typeKey": "ArgumentException",
"errorCode": 0,
"eventId": 0
}
newObjectId is the object id of the existing branch. Really not sure why this is named new when it is actually older. Very odd.
[
{
"name": "refs/heads/new-test-branch-from-api-call",
"newObjectId": "{{BranchObjectIdGoesHere}}",
"oldObjectId": "0000000000000000000000000000000000000000"
}
]

Microsoft Graph Api upload file to SharePoint

I'm using microsoft graph api to interview with sharepoint.
Upload file to sharepoint.
https://graph.microsoft.com/v1.0/sites/abc78c05-a77b-45bf-a1a1-51f09548b497/drive/root:/test1212123.txt:/content
Then we can got the response.
{
"#odata.context": "https://graph.microsoft.com/v1.0/$metadata#sites('abc78c05-a77b-45bf-a1a1-51f09548b497')/drive/root/$entity",
"#microsoft.graph.downloadUrl": "https://yeeofficesg.sharepoint.com/sites/GdTest/_layouts/15/download.aspx?UniqueId=b9d25e13-c915-432f-b9fb-f2d36a188d9f&Translate=false&tempauth=eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAveWVlb2ZmaWNlc2cuc2hhcmVwb2ludC5jb21AMzgzMDNhNTQtMjUwMS00MDcwLTlkYjItYzNmNTY2OTc2NGUxIiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTU4NDY4MjQ5OSIsImV4cCI6IjE1ODQ2ODYwOTkiLCJlbmRwb2ludHVybCI6InltcjVvWHhDU0FIaFhhV0tYVnZuVDVjK05ETnZsejhzcC9YeFp3MStQaHc9IiwiZW5kcG9pbnR1cmxMZW5ndGgiOiIxMzUiLCJpc2xvb3BiYWNrIjoiVHJ1ZSIsImNpZCI6IlpUUmhPVFk1WkdFdE5EQXlOQzAwWlRnMExUazFZelF0WkRkalpqRmpOR1UxTm1ZMCIsInZlciI6Imhhc2hlZHByb29mdG9rZW4iLCJzaXRlaWQiOiJZV0pqTnpoak1EVXRZVGMzWWkwME5XSm1MV0V4WVRFdE5URm1NRGsxTkRoaU5EazMiLCJhcHBfZGlzcGxheW5hbWUiOiJIdHRwUmVxdWVzdCBUZXN0IiwibmFtZWlkIjoiNTk3ZDQ4YmMtMDVmMy00MTU4LThhY2MtYWU1Y2M3YTljNmFkQDM4MzAzYTU0LTI1MDEtNDA3MC05ZGIyLWMzZjU2Njk3NjRlMSIsInJvbGVzIjoiYWxsc2l0ZXMud3JpdGUgYWxsZmlsZXMud3JpdGUiLCJ0dCI6IjEiLCJ1c2VQZXJzaXN0ZW50Q29va2llIjpudWxsfQ.aTVxeDdWNkowcWFDK0xYOHUvZGo3K0VVSEd1dU02MFVheEFJbnBWWUJHTT0&ApiVersion=2.0",
"createdDateTime": "2020-03-20T05:34:59Z",
"eTag": "\"{B9D25E13-C915-432F-B9FB-F2D36A188D9F},1\"",
"id": "016REKDTITL3JLSFOJF5B3T67S2NVBRDM7",
"lastModifiedDateTime": "2020-03-20T05:34:59Z",
"name": "test1212123.txt",
"webUrl": "https://yeeofficesg.sharepoint.com/sites/GdTest/Shared%20Documents/test1212123.txt",
"cTag": "\"c:{B9D25E13-C915-432F-B9FB-F2D36A188D9F},1\"",
"size": 12,
"createdBy": {
"application": {
"id": "597d48bc-05f3-4158-8acc-ae5cc7a9c6ad",
"displayName": "HttpRequest Test"
}
},
"lastModifiedBy": {
"application": {
"id": "597d48bc-05f3-4158-8acc-ae5cc7a9c6ad",
"displayName": "HttpRequest Test"
}
},
"parentReference": {
"driveId": "b!BYzHq3unv0WhoVHwlUi0l_EO2rYM2NNCptmOTvJ-EqeM9aeJ-zj_TZktSrctfA1S",
"driveType": "documentLibrary",
"id": "016REKDTN6Y2GOVW7725BZO354PWSELRRZ",
"path": "/drive/root:"
},
"file": {
"mimeType": "text/plain",
"hashes": {
"quickXorHash": "RBBCDGQwAxrUIARAFAEJSgAAAAA="
}
},
"fileSystemInfo": {
"createdDateTime": "2020-03-20T05:34:59Z",
"lastModifiedDateTime": "2020-03-20T05:34:59Z"
}
}
Then I want to update the customized column of this list.
https://graph.microsoft.com/v1.0/sites/abc78c05-a77b-45bf-a1a1-51f09548b497/lists/89a7f58c-38fb-4dff-992d-4ab72d7c0d52/items/80/fields
step3, I needs the item id (this example is : 80)
but when I upload the file, I can't got the item id directly from the response.
use this api:https://graph.microsoft.com/v1.0/sites/abc78c05-a77b-45bf-a1a1-51f09548b497/lists/89a7f58c-38fb-4dff-992d-4ab72d7c0d52/items/
I can got the items list which include the item id is needed.
Finally, my question is ,when I upload file to sharepoint, how can I got the item id which is needed by update item.
I ended up extracting the Item GUID from the response, i.e.
"#microsoft.graph.downloadUrl": "https://yeeofficesg.sharepoint.com/sites/GdTest/_layouts/15/download.aspx?UniqueId=b9d25e13-c915-432f-b9fb-f2d36a188d9f&Translate=false&tempauth=....
or
"eTag": ""{B9D25E13-C915-432F-B9FB-F2D36A188D9F},1""
or
"cTag": ""c:{B9D25E13-C915-432F-B9FB-F2D36A188D9F},1""
And then use that in the PATCH call where the item ID is required, i.e. https://graph.microsoft.com/v1.0/sites/abc78c05-a77b-45bf-a1a1-51f09548b497/lists/89a7f58c-38fb-4dff-992d-4ab72d7c0d52/items/**B9D25E13-C915-432F-B9FB-F2D36A188D9F**/fields
Might be a more elegant way to solve the problem, however this worked for me

How to get the Site Id of SharePoint using Graph SharePoint Explorer

How to get the Microsoft SharePoint SiteID using Microsoft Graph API Explorer.
Initially i tried with below API i able to get the Site ID
https://graph.microsoft.com/v1.0/sites/tenantName.sharepoint.com:/sites/TestSite:/drives?select=name,id
Sharepoint URL:
https://tenantName.sharepoint.com/sites/TestSite
I output i got is:
{
"#odata.context": "https://graph.microsoft.com/v1.0/$metadata#drives",
"value": [
{
"id": "b!l17-JY9YT67Qp-2TBvsUupBLMUF2SrJHp5VylCDZThT7HpCdF-7uQ6NTp6t-MbR5",
"name": "Documents"
}
]
}
But, when i try with Communication Site
Whose SharePoint URL is:
https://tenantName.sharepoint.com/SitePages/DevHome.aspx
Graph Explorer API
https://graph.microsoft.com/v1.0/sites/tenantName.sharepoint.com:/SitePages/DevHome:/drives?select=name,id
I am getting below error:
{
"error": {
"code": "itemNotFound",
"message": "The provided path does not exist, or does not represent a site",
"innerError": {
"request-id": "8329dfca-c63b-4af5-80b8-75f26be9e2e8",
"date": "2019-10-31T13:18:33"
}
}
}
A sitePage is a fundamentally different resource than a site.
A site is a container that owns any number of sub-sites, apps, lists, document libraries, etc.
A sitePage is just another resource owned by a site.
The sitePage resource is currently only available in the Microsoft Graph Beta version.
So the query for /SitePages/DevHome.aspx would be:
/beta/sites/root/pages/{pageId}
If you don't yet know the correct id for the page, you can filter the SitePage collection based on the page's name:
/beta/sites/root/pages?$filter=name eq 'DevHome.aspx'
This will return a collection with a single entity (the DevHome.aspx page):
{
"#odata.context": "https://graph.microsoft.com/beta/$metadata#sites('root')/pages",
"value": [
{
"eTag": "",
"id": "{id}",
"lastModifiedDateTime": "2014-07-10T05:47:29Z",
"name": "DevHome.aspx",
"webUrl": "SitePages/DevHome.aspx",
"createdBy": {
"user": {
"displayName": "System Account"
}
},
"lastModifiedBy": {
"user": {
"displayName": "System Account"
}
},
"parentReference": {
"siteId": "{id}"
},
"contentType": {
"id": "0x0101080062C83F3CFED6744A882F729480DE6C17",
"name": "Wiki Page"
},
"webParts": [],
"publishingState": {
"level": "published",
"versionId": "1.0"
}
}
]
}
I should also point out that you're misinterpreting the result of your first query. When you request /v1.0/sites/{tenant}:/{path}:/drives?select=name,id, you are not getting the IDs for each Site, you're getting the IDs for each Drive within that Site. You can find the objects contained within a site in the Relationships section of the Site Resource documentation
Your Graph API call is incorrect.
Try this one :
https://graph.microsoft.com/v1.0/sites/tenantName.sharepoint.com?select=name,id

Azure automation job OData $filter

Background
I have an application which get list of runbooks jobs using REST API
I would like to apply $filter on properties parameters(see JSON in the end). Parameters is of type IDictionary<string,string>
Issue
$filter works fine for most of the properties but fails for IDictionary<string,string>
This is what I am trying https://management.azure.com/subscriptions/XXX/resourceGroups/XXX/providers/Microsoft.Automation/automationAccounts/XXX/jobs?$filter=properties/parameters/any(keyValue: keyValue/owner eq 'Adam#test.com')&api-version=2015-10-31
Request fails with error {
"code": "BadRequest",
"message": "Could not find a property named 'owner' on type 'System.Collections.Generic.KeyValuePair_2OfString_String'."
}
Question
Is it possible to filter what I am trying?
If yes, then what I am doing wrong?
JSON response on which I want to apply filter
"value": [
{
"id": "/subscriptions/XXX/resourceGroups/XXX/providers/Microsoft.Automation/automationAccounts/XXX/jobs/XXX",
"properties": {
"jobId": "XXX",
"runbook": {
"name": "HelloWorldRunbook"
},
"schedule": null,
"provisioningState": "Succeeded",
"status": "Completed",
"creationTime": "2018-06-17T05:44:12.197+00:00",
"startTime": "2018-06-17T05:44:21.227+00:00",
"lastModifiedTime": "2018-06-17T05:44:43.43+00:00",
"endTime": "2018-06-17T05:44:43.43+00:00",
"jobScheduleId": "7fc134ac-d8bd-464e-b041-6e6b50f83f0c",
"runOn": null,
"parameters": {
"Owner": "Adam#test.com",
"mailBox": "test_mailbox#test.com"
}
}
............removed for brevity
When I try to run List Job By Automation Account Rest API, there is no properties parameters returned in the response body.
If yes, then what I am doing wrong?
...
"parameters": {
"Owner": "Adam#test.com",
"mailBox": "test_mailbox#test.com"
}
...
But according to you mentioned responsed json format, the parameters properties is an object not a dictionary, so you could use $filter=properties/parameters/Owner eq 'Adam#test.com' to do that if you make sure that parameters is existed.
Get https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Automation/automationAccounts/{automationAccountName}/jobs?api-version=2015-10-31&$filter=properties/parameters/Owner eq 'Adam#test.com'

Resources