I wanna know if there is a way to find out the date of the first representation of a resource in DBpedia (for this resource as an example: http://dbpedia.org/page/Mario_Balotelli)?
Related
we are evaluating if Azure Data Catalog will meet our needs and one of our questions is if we can search for Datasets based on column tags. For example, if some of my datasets have the column tag "xyz" for one of their columns, can I search for all datasets with a column tag "xyz" and get all the datasets that have a column (any column) with the tag "xyz"?
Also, can this be done programmatically through the REST APIs? Thanks!
Last I checked you can tag individual columns and you can search on those tags. And REST based search API supports an explicit tag search syntax.
The search at AzureDataCatalog.com will include column tags, column names, and object tags etc. If you implement the high tier options, you can roll up tags into business glosseries. For example the business might use "external supplier" but common alternative's used around the business might be 'panel vender' or 'agency' but really they all mean the same thing.
The full search syntax reference can be found here : https://learn.microsoft.com/en-us/rest/api/datacatalog/data-catalog-search-syntax-reference
I want to write a generic DataFactory in V2 with below scenario.
Source ---> Extracted (Salesforce or some other way), which don't have
extraction timestamp. ---> I want to write it to Blob with extraction
Time Stamp.
I want it to be generic, so I don't want to give column mapping anywhere.
Is there any way to use expression or system variable in Custom activity to append a column in output dataset? I like to have a very simple solution to make implementation realistic.
To do that you should change the query to add the column you need, with the query property in the copy activity of the pipeline. https://learn.microsoft.com/en-us/azure/data-factory/connector-salesforce#copy-activity-properties
I dont know much about Salesforce, but in SQL Server you can do the following:
SELECT *, CURRENT_TIMESTAMP as AddedTimeStamp from [schema].[table]
This will give you every field on your table and will add a column named AddedTimeStamp with the CURRENT_TIMESTAMP value in every row of the result.
Hope this helped!
My aim is to find venues in specific city.
For this purpose i am using /v2/venues/suggestcompletion endpoint. So for example next request https://api.foursquare.com/v2/venues/suggestcompletion/?limit=10&query=McDonald&near=Napa,%20CA,%20United%20States&client_id=xxx&client_secret=yyy&v=20161118&locale=en returns venues not only in Napa, but also in Fairfield, Vallejo etc. Result for this query.
Ok, as far as I know what city I need I can filter result by city on my side. But in this approach I received next problem. Next query https://api.foursquare.com/v2/venues/suggestcompletion/?limit=10&query=Diviera%20Drive&near=New%20York,%20NY,%20United%20States&client_id=xxx&client_secret=yyy&v=20161118&locale=en returns venue in city labeled Brooklyn. Result for this Obviously New York string and Brooklyn does not match. Maybe somehow I can retrieve main city (New York) next to borough (Brooklyn) so I can filter it on my side properly
So my question is: how can I receives venues only in specified city.
The near parameter you're using is not a filter - it's a coarse geocoder that's used to generate a latitude/longitude to search near.
The search documentation does not show a way to limit results to a city name. If you only want to show results in a certain city you'd have to do that filtering yourself. Since the city name may be missing or incorrect (sometimes city boundaries are fuzzy) a more precise way could be filter based on the lat/lng coordinate.
I'm looking to understand the data returned by WADMetricsPT1HP10DV2S20160415 table inside Azure's storage account.
It has the PartitionKey, RowKey and EntityProperties. PartitionKey, I understand. It translates to the resource Id of my resource inside the storage account. However, I partially understand the RowKey. An example RowKey is:
:005CNetworkInterface:005CPacketsReceived__2519410355999999999
I understand the first part, which is a metric name. But what I don't understand in the number/digits that follow. I am assuming it to be the timestamp, but can't say for sure.
I was attempting to use the RowKey filter, but due to this added wrinkle it's almost impossible to generate the RowKey and use it as a filter. Does anyone know how to generate the numbers/digits in order to create a RowKey filter?
If you're curious to know about what 2519410355999999999 number is, it is essentially reverse ticks derived by subtracting the event date/time ticks from the max date/time ticks value. To derive date/time value from this number, here's what you can do:
long reverseTicks = 2519410355999999999;
long maxTicks = DateTime.MaxValue.Ticks;
var dateTime = new DateTime(maxTicks - reverseTicks, DateTimeKind.Utc);
Say I want to fetch WADLogsTable logs for last 24h of a given cloud service, perhaps narrow down to a given role, and even further down to a given instance. What does the query look like? Is there any high level api for that?
As such there is no API specific for querying diagnostics data. Because diagnostics data is stored in Azure Tables, you can simply query the table to fetch the desired data.
One thing you would notice is that PartitionKey in these tables represent time (rounded to nearest minute) represented in ticks. If you're interested in fetching last "x" hours from WADLogsTable table, here's what you would need to do:
Convert from and to date/time (UTC) into ticks.
Prepend 0 in front of them.
Use these in query on PartitionKey. Assuming you're using REST API, the query would look something like:
(PartitionKey ge 'from date/time ticks with prepended zero' and PartitionKey le 'to
date/time ticks with prepended zero')
To further filter down by role or role instance, you would add appropriate filter criteria. For example:
(PartitionKey ge 'from date/time ticks with prepended zero' and PartitionKey le 'to
date/time ticks with prepended zero') and (Role eq 'Role Name')
I wrote a blog post long time back which you may find useful: http://gauravmantri.com/2012/02/17/effective-way-of-fetching-diagnostics-data-from-windows-azure-diagnostics-table-hint-use-partitionkey/ (Caution: The code showing the use of storage client library is based on the older version of the library, so you may want to use that only for understanding purpose).