How can I find which Log analytics table is supported by which resource/solution? I need to work on log tables and need to find out which I need to support depending on solution or resources.
I am not finding any consolidated information on same. If I try to check a table the I can see supported solution or resources for that particular table.
You can find the Supported Resources and solutions in the Log Analytics workspace by following the below steps:-
I agree with Jul_DW AFAIK you can find the consolidated list of all the Log analytics tables in this MS document as there’s no other document with the list of logs and check the same for the category type in the left Filter by title:-
Azure Monitor table reference index by resource type | Microsoft Learn
You can also make use of Rest API to call the Log analytics workspace and List all the tables in the LA workspace with its resources like below:-
GET https://management.azure.com/subscriptions/{subscriptionId}/resourcegroups/{resourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{workspaceName}/tables?api-version=2021-12-01-preview
Output:-
List Data sources by filtering specific table:-
GET https://management.azure.com/subscriptions/00000000-0000-0000-0000-00000000000/resourcegroups/OIAutoRest5123/providers/Microsoft.OperationalInsights/workspaces/AzTest9724/dataSources?$filter=kind='WindowsEvent'&api-version=2020-08-01
References:-
Tables - List By Workspace - REST API (Azure Log Analytics) | Microsoft Learn
Data Sources - List By Workspace - REST API (Azure Log Analytics) | Microsoft Learn
Related
I have configured a diagnostic setting in app insight to transfer telemetry data to storage account. I do not want to transfer/migrate user_authenticationId column from pageViews data. How can I prevent it from transferring to storage account using Diagnostic settings.
• Using ‘Diagnostic setting’, it is not possible to exclude columns from being exported through ‘pageViews’ category in application insights. Rather, you can exclude the column ‘user_authenticationID’ with an application insights log filter query by executing the same on the ‘pageViews’ table and then save the query as a function to be executed at a time of your choosing or export the output of that query for a particular timestamp to an excel file or a storage account of your choosing.
Please find the below application insights log query for excluding the column as stated above: -
Also, find the below documentation link for more detailed information on exporting the query results in a storage account and the requirements for the same: -
https://learn.microsoft.com/en-us/azure/azure-monitor/app/export-telemetry
Thus, in this way, you can achieve the desired.
Moving from AWS Glue to Azure Purview and i am confused about something
Its is possible to query Azure purview data catalog/assets in the same way we can query from AWS Glue data catalog using AWS Athena?
Unfortunately, you cannot query data from Azure Purview.
The Purview search experience is powered by a managed search index. After a data source is registered with Purview, its metadata is indexed by the search service to allow easy discovery. The index provides search relevance capabilities and completes search requests by querying millions of metadata assets. Search helps you to discover, understand, and use the data to get the most value out of it.
The search experience in Purview is a three stage process:
The search box shows the history containing recently used keywords
and assets.
When you begin typing the keystrokes, the search suggests
the matching keywords and assets.
The search result page is shown with assets matching the keyword entered.
For more details, refer to Understand search features in Azure Purview.
I have a scenario where I would like to query Azure Data warehouse tables within the Log Analytics workspace and using those records I need to create a result set and prepare a chart.
I do see some objects in log analytics workspace like a database, table but not sure what is the purpose and are these objects specific to a resource or generic and how to use them I couldn't get documentation for these objects can somebody guide me on this.
Unfortunately, you cannot use Azure Log Analytics to query Azure SQL Data Warehouse.
Use Azure Data Studio to connect and query data in Azure SQL data warehouse.
Recommended tools for querying data in Azure SQL Data Warehouse.
Azure Log Analytics is used to write, execute, and manage Azure Monitor log queries in the Azure portal. You can use Log Analytics queries to search for terms, identify trends, analyze patterns, and provide many other insights from your data.
For more information about log queries, see Overview of log queries in Azure Monitor.
For a detailed tutorial on writing log queries, see Get started with log queries in Azure Monitor.
I have a working query for my app data to be analyzed.
currently it analyzes the last two weeks data with an ago(14d).
Now i want to use a value containing the release date of the apps current version. Since i havent found a way to add a new database table to the already existing database containing the log data in azure analytics, i created a new database in azure and entered my data there.
Now i just don't know, if i can get access to that database at all from within the web query interface of Azure log analytics, or if i have to use some other tool for that?.
i hope that somebody can help me on this.
As always with azure there is a lot of stuff to read about it, but nothing concrete for my issue (or at least i haven't found it yet).
And yes, i know how to insert the data into the query with a let, but since I want to use the same data in different queries, an external location which can be accessed from all the queries would be the solution I prefer.
Thx in advance.
Maverick
You cannot access a db directly. You are better of using a csv/json file in blob storage. In the following example I uploaded a txt file with csv data like this:
2a6c024f-9093-434c-b3b1-000821a15b1a,"Customer 1"
28a658a8-5466-45ea-862c-003b20507dd4,"Customer 2"
c46fb949-d807-4eea-8de4-005dd4beb39a,"Customer 3"
e05b67ee-ff83-4805-b004-0064449f196c,"Customer 4"
Then I can reference this data from log analytics / application insights in a query like this using the externaldata operator:
let customers = externaldata(id:string, companyName:string) [
h#"https://xxx.blob.core.windows.net/myblob.txt?sv=2019-10-10&st=2020-09-29T11%3A39%3A22Z&se=2050-09-30T11%3A39%3A00Z&sr=b&sp=r&sig=xxx"
] with(format="csv");
requests
| extend CompanyId = tostring(customDimensions.CustomerId)
| join kind=leftouter
(
customers
)
on $left.CompanyId == $right.id
The url https://xxx.blob.core.windows.net/myblob.txt?sv=2019-10-10&st=2020-09-29T11%3A39%3A22Z&se=2050-09-30T11%3A39%3A00Z&sr=b&sp=r&sig=xxx is created by creating a url including a SAS token by using the Microsoft Azure Storage Explorer, selecting a blob and then right click -> Get Shared Access Signature. In the popup create a SAS and then copy the uri.
i know Log Analytics uses Azure Data Explorer in the back-end and Azure Data Explorer has a feature to use External Tables within the queries but I am not sure if Log Analytics support External Tables.
External Tables in Azure Data Explorer
https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/schema-entities/externaltables#:~:text=An%20external%20table%20is%20a,and%20managed%20outside%20the%20cluster.
While creating custom log search alerts in log analytics workspace, I want to store some data and query it in alert query. Basically, it is a mapping like ABC -> DEF, GHI -> JKL. These mappings can be changed manually.
I am looking a solution like creating a table or function in workspace, or reading data from blob in the query. I do not want to create the table or function in the alert query, just read from it. If there are other solutions, please suggest them too.
Have you tried inserting custom data in Log Analytics via the REST API? This will solve your problem--and it's what we do using Runbooks. Works great.
Log Analytics Data Collector API
I realize this is an old thread, but for anyone else looking to do this, see:
Implementing Lookups in Azure Sentinel
Azure Sentinel provides four methods to reference, import, and use lookup information. The methods are:
The built-in Watchlists feature, which enables uploading CSV files as lookup tables.
The externaldata KQL function, which enables referencing an Azure Storage file as a lookup table.
Custom tables, imported using a custom connector.
A KQL function utilizing the datatable operator, which can be updated interactively or using PowerShell.