I'm trying to send a Timestamp field which is ISO 8601 with offset (
"2023-02-01T11:11:12.2220000+03:00" )
Azure doesn't really work with offsets, I first encountered that when sending data to event hub.
I was hoping to resolve this by splitting timestamp field into 2 fields:
timestamp: 2023-02-01T11:11:12.2220000
offset: +03:00
and combining them is SA query.
This seemed to have worked in Query editor, where test output is shown as a correct timestamp+offset
however when data is sent to output (in this case SQL, field type datetimeoffset), value looks like this:
2023-02-01T08:11:12.2220000+00:00
I suspect this is because timestamp field type in SA is datetime (seen in query explorer test results window)
even if I cast to nvarchar field type is still datetime.
is there a way to force SA to use specific types for fields (in this case, treat field as a string and not datetime)?
or, in general, how pass value like "2023-02-01T11:11:12.2220000+03:00" through SA without altering it? bonus points if it can be done in Event Hub as well
Related
I am currently trying to connect 2 different devices to the IoT Hub, and I need to separate the data from each device. In order to do so, I tried configuring my stream analytics query like this:
SELECT
deviceId, temperature, humidity, CAST(iothub.EnqueuedTime AS datetime) AS event_date
INTO
NodeMCUOutput
FROM
iothubevents
WHERE
deviceId = "NodeMCU1"
However, for some reason, the output is not shown if the WHERE statement is in the code (the outputs are shown without it, but the data is not filtered). I need the WHERE statement in order to sort the data the way I want it. Am I missing something? Are there any solutions to this? Thanks a lot. Cheers!
The device ID and other properties that are not in the message itself are included as metadata on the message. You can read that metadata using the GetMetadataPropertyValue() function. This should work for you:
SELECT
GetMetadataPropertyValue(iothubevents, 'IoTHub.ConnectionDeviceId') as deviceId,
temperature,
humidity,
CAST(GetMetadataPropertyValue(iothubevents, 'IoTHub.EnqueuedTime') AS datetime) AS event_date
INTO
NodeMCUOutput
FROM
iothubevents
WHERE
GetMetadataPropertyValue(iothubevents, 'IoTHub.ConnectionDeviceId') = 'NodeMCU1'
I noticed you use a double quote in the WHERE clause.
You need a simple quote to get a match on strings. In this case it will be
WHERE deviceId = 'NodeMCU1'
If the deviceId is the one from IoT Hub metadata, Matthijs answer will help you to retrieve it.
Continuing my research over AZURE FHIR (Sql Version), I found that the lastUpdated search parameter somehow is corelated to the ResourceSurrogateId field (in the Resource table).
Moreover, I believe both are the same but in different formats. However, I did not find any source explaining how the lastupdated (a date-time value) is transformed into the ResourceSurrogateId (numeric Value).
Can somebody explain me how can I get a ResourceSurrogateId based on its original date-time value?
Test:
I run the followig FHIR REST API: https://XXXXXXXXXXX/DeviceComponent?_lastUpdated=gt2019-07-01 and the actual query on the database was:
FROM dbo.Resource r
WHERE ResourceTypeId = #p1
AND ResourceSurrogateId >= #p2
AND IsHistory = 0
AND IsDeleted = 0
ORDER BY r.ResourceSurrogateId ASC
OPTION(RECOMPILE)',N'#p0 int,#p1 smallint,#p2 bigint',#p0=11,#p1=32,#p2=5095809792000000000```
The resource surrogate ID encodes the datetime of insert to millisecond precision along with a "uniquifier" that comes from a cycling sequence on the database. The code that converts from a surrogate ID to datetime and back is here.
You will not be able to get the surrogate ID from the datetime, but you will be able to get lower and upper bounds on it.
I'm creating a new Logic App that reads a table where DateCreated < ADDDAYS(-60,GETDATE()) and updates an Archived bit to 1.
However, I can't for the life of me figure out how to implement that filter as part of the ODATA query.
Here's what I'm trying so far:
DateCreated lt addDays(utcNow(),-60)
However, I get "An unknown function with name 'utcnow' was found. This may also be a function import or a key lookup on a navigation property, which is not allowed.\r\n inner exception: An unknown function with name 'utcnow' was found. This may also be a function import or a key lookup on a navigation property, which is not allowed."
How can I filter on a dynamic date in the filer?
However, I can't for the life of me figure out how to implement that filter as part of the ODATA query.
I suppose you mean on the ODATA Query on the SQL Connector?
Can you try the following:
DateCreated lt #{addDays(utcNow(),-60)}
Based on the previous answer, you should try the same command :
DateCreated lt #{addDays(utcNow(),-60)}
But also, you must ensure that your data type, on the SQL side, must be a datetimeoffset.
Three solutions to do this :
Change the type of your field in your table,
Create a view and cast the field DateCreated to DATETIMEOFFSET
CREATE VIEW [dbo].[myview] AS
SELECT MyFields, ..., CAST(DateCreated AS DATETIMEOFFSET) AS DateCreated
FROM MyTable
Create a store procedure with a DATETIMEOFFSET parameter, and convert the parameter to a DATETIME
If you can not change your SQL code, this piece of code is the solution :
year(DateCreated) lt year(#{addDays(utcNow(),-60)}) or (
year(DateCreated) eq year(#{addDays(utcNow(),-60)}) and (
month(DateCreated) lt month(#{addDays(utcNow(),-60)} or (
month(DateCreated) eq month(#{addDays(utcNow(),-60)}
... <same thing for other date parts>
)
)
)
You have to compare each part of your date :
This is an interesting issue that sometimes shows up when dates,
times, datetimes, and specific time zone come into play. Comparing a
DateTimeZone to a date is problematic, because it might be less in
arithmetic terms but only if the time zone matches... without that
critical piece of information, these data types cannot be compared.
One alternative is to use the standard OData functions to retrieve
parts of the data type. For example:
$filter = year(release_date) lt year(dtz)
Of course, you must be carful to ensure that you are implementing the
correct logic with respect to timezone- but you are probably aware of
that.
OData Reference :
http://www.odata.org/documentation/odata-version-2-0/uri-conventions/
So, a quick overview of what I'm doing:
We're currently storing events to Azure Table storage from a Node.js cloud service using the "azure-storage" npm module. We're storing our own timestamps for these events in storage (as opposed to using the Azure defined one).
Now, we have coded a generic storage handler script that for the moment just stores all values as strings. To save refactoring this script, I was hoping there would be a way to tweak the query instead.
So, my question is, is it possible to query by datetime where the stored value is not actually a datetime field and instead a string?
My original query included the following:
.where( "_timestamp ge datetime'?'", timestamp );
In the above code I need to somehow have the query treat _timestamp as a datetime instead of a string...
Would something like the following work, or what's the best way to do it?
.where( "datetime _timestamp ge datetime'?'", timestamp );
AFAIK, if the attribute type is String in an Azure Table, you can't convert that to DateTime. Thus you won't be able to use .where( "_timestamp ge datetime'?'", timestamp );
If you're storing your _timestamp in yyyy-MM-ddTHH:mm:ssZ format, then you could simply do a string based query like
.where( "_timestamp ge '?'", timestamp );
and that should work just fine other than the fact that this query is going to do a full table scan and will not be an optimized query. However if you're storing in some other format, you may get different results.
I am using Node for fetching data from MySQL. In database, i got record like : 2013-08-13 15:44:53 . But when Node fetches from Database , it assigns value like 2013-08-19T07:54:33.000Z.
I just need to get time format as in MySQL table. Btw ( My column format is DateTime in MySQL)
In Node :
connection.query(post, function(error, results, fields) {
userSocket.emit('history :', {
'dataMode': 'history',
msg: results,
});
});
When retrieving it from the database you most likely get a Date object which is exactly what you should work with (strings are only good to display dates, but working on a string representation of a date is nothing you want to do).
If you need a certain string representation, create it based on the data stored in the Date object - or even better, get some library that adds a proper strftime-like method to its prototype.
The best choice for such a library is moment.js which allows you to do this to get the string format you want:
moment('2013-08-19T07:54:33.000Z').format('YYYY-MM-DD hh:mm:ss')
// output (in my case on a system using UTC+2 as its local timezone):
// "2013-08-19 09:54:33"
However, when sending it through a socket (which requires a string representation) it's a good idea to use the default one since you can pass it to the new Date(..) constructor on the client side and get a proper Date object again.