I'm trying to figure out when a Linked Service was added to a Data Factory instance.
We capture all logs and metrics in Log Analytics but so far nothing seems to show when a Linked Service was created.
The closest I've got is querying the Azure Activity table for this ADF querying for Linked Services/Write and the specific Linked service as shown below:
| where OperationNameValue == "MICROSOFT.DATAFACTORY/FACTORIES/LINKEDSERVICES/WRITE
| where Properties contains "Name of my link service"
This is not telling me the info I need, which is to know when this linked service was created. Doing a distinct search on the OperationNameValue does not yield anything else related to Linked Services. Perhaps the data I'm looking for is somewhere else?
Thanks for any help you may provide.
To test this, I have created a ADF, linked service with blob storage and exported the activity logs to one of the log analytics workspaces.
Using the below KQL query I am able to pull the linked service creation time and followed by the caller who has created it.
AzureActivity
| where OperationNameValue contains "MICROSOFT.DATAFACTORY/FACTORIES/LINKEDSERVICES/WRITE" and ActivityStatusValue contains "Success"
| extend linkedservicename=tostring(Properties_d["resource"])
| where linkedservicename contains "<pass specific linked servicename>"
| project EventSubmissionTimestamp,_ResourceId,Caller
Here is the sample output for reference:
Alternatively, you can use the below PowerShell cmdlet as well to pull creation time of linked service.
Get-AzActivityLog -StartTime (get-date).AddDays(-90) -EndTime (get-date)| Where-Object {$_.Authorization.Action -like "MICROSOFT.DATAFACTORY/FACTORIES/LINKEDSERVICES/WRITE" -and $_.Status -like "Succeeded" } | Select EventTimestamp,SubmissionTimestamp,Caller,ResourceId| ConvertTo-Json
Here is the sample output of reference:
Related
In the ADX web UI at https://dataexplorer.azure.com/dashboards, I can see a list of Kusto based dashboards I've created or been granted access to.
Under what resource type in https://portal.azure.com/ are these assets scoped? I've used the UI to export the report definition JSON. However, I've been unable to find either the Dashboard name or guid in Azure Resource Graph explorer searching in the name & id fields. Notably these same queries (with different arguments) return the ADX clusters which are the data sources for the dashboards in question.
The az portal dashboard commands do not appear to include Kusto dashboards as far as I can tell.
A partial inventory appears to be available in the .show queries system view.
.show queries
| where ClientActivityId startswith "RTD;"
| extend ClientActivityIdBag = split(ClientActivityId,';')
| extend DashboardGuid = tostring(ClientActivityIdBag[1])
| summarize
count(),
min(StartedOn),
max(StartedOn),
take_any(ClientActivityId)
by DashboardGuid
This suggests to me that there is a ReportServer DB analouge from which I'd be able to correlate the dashboard guid to broader dashboard metadata, but I've been unable to proceed further so far.
I want to monitor shared Kusto dashboard assets for modifications. Other than logging into the web UI and using my eyeballs, how can I do this?
I am trying to bring in Azure Synapse logs into Loganalytics to create dashboards on usage level.
I have already setup in diagnostic settings to pass on the logs to my loganalytics workspace.
But while trying to execute queries from below documentation, I am getting error saying -
Query -
//Chart the most active resource classes
AzureDiagnostics | where Category contains "ExecRequests" | where
Status_s == "Completed" | summarize totalQueries = dcount(RequestId_s)
by ResourceClass_s | render barchart
Error:
'where' operator: Failed to resolve column or scalar expression named 'Status_s'...
Documentation link for queries : https://learn.microsoft.com/en-us/azure/synapse-analytics/sql-data-warehouse/sql-data-warehouse-monitor-workload-portal
Please let me know if there is something I am missing. I am directly logging to loganalytics workspace and running these queries inside a workbook...
Also i didnt find any proper documentation/blogs/links for connecting synapse to loganalytics, please let me know if anyone has that..
The documentation linked in your post appears to be out of date even though the last update date is recent.
See this link:
Azure services that use resource-specific mode store data in a table
specific to that service and do not use the AzureDiagnostics
table.
The link also lists a number of resource-specific tables for Synapse. "SynapseSqlPoolExecRequests" and "SynapseSqlPoolSqlRequests" are a few examples that might provide the info you're seeking.
I wanted to create a report that should contain all the Azure Private Endpoints and to which specific resource type it is attached to. Is there a command-line way to figure this out? so that I can add them in my scripting to make the report?
If by "attached to", you're referring to the service connection, you can try:
az network private-endpoint list |
ConvertFrom-Json |
Select -Expand privateLinkServiceConnections
in which the privateLinkServiceId would contain the resource ID, which might contain the type you're looking for.
I need to use the command Get-AzureRMResource and return resources created after a particular date . Is it possible to filter the resources w.r.t creation date. Can someone please help?
The Get-AzureRMResource could not get the creation date of Azure RM Resources. It seems there is no other way to get the creation date except the Activity log.
But still it can return only the items created on past 90 days.
For this issue, you could try to Archive the Azure Activity Log, this option is useful if you would like to retain your Activity Log longer than 90 days (with full control over the retention policy) for audit, static analysis, or backup.
Update:
If you want to get resources created after a particular date, try the command below, it returns the resources created after 11/20/2018 1:57:19 AM.
Get-AzureRmResourceGroupDeployment -ResourceGroupName "<ResourceGroupName>" | Where-Object {$_.Timestamp -gt '11/20/2018 1:57:19 AM'}
This information is available via ARM, but you have to call the API directly rather than the PS Get-AzureRMResource (or Get-AzResource) cmdlets.
See Deleting all resources in an Azure Resource Group with age more than x days.
Essentially, you need to add the $expand=createdTime to your query parameters, ie.:
GET
https://management.azure.com/subscriptions/1237f4d2-3dce-4b96-ad95-677f764e7123/resourcegroups?api-version=2019-08-01&%24expand=createdTime
Like #kwill suggested, this site can also help run the command interactively via your browser and return these results for you:
https://learn.microsoft.com/en-us/rest/api/resources/resources/list#code-try-0
Steps below:
Click on the try it now button
Enter your subscription ID
For a key value name use: $expand
For the key value value use: createdTime
Then run the query and it should produce a JSON file for you
Example
The old silverlight portal used to show the start datetime of each role instance so I could see when it last cycled.
As far as I can see this information is not surfaced in the HTML5 portal.
Is there a way to programmatically access this information via powershell or the REST API?
I can't find a way to see each role instance's uptime, but it looks like this will give you the time it's been since a service was last modified (i.e. with Swap VIPs, Uploading a Package, etc). This and a good old fashioned ping (to make sure it's still alive) should help you know how long your services have been "stable".
[DateTime]::Now - (Get-AzureService |
Where-Object { $_.Label -eq 'MyService' } |
Select-Object -ExpandProperty DateModified)