The manual specifies that uniqueness is based on foreign_id and time fields.
I have an activity where a user joins a specific event. The event has its own start date which I want to display in the timelines, therefore I am submitting it as extra data (as well as name and location etc, but I omitted those here to prevent clutter).
I submitted the activity with the following activity array to a user feed. Notice I am in Amsterdam, which is (at the moment) in a timezone of GMT+0200:
Array
(
[actor] => User:3
[verb] => join
[object] => Event:2
[event_start] => 2016-09-26T19:00:00+0200
[to] => Array
(
[0] => notification:2
)
[foreign_id] => join:2
[time] => 2016-09-16T13:29:13+0200
)
When I retrieve that users' user feed, this comes back as:
{
"duration":"17ms",
"next":"",
"results":[
{
"actor":"User:3",
"event_start":"2016-09-26T19:00:00+0200",
"foreign_id":"join:2",
"id":"cb0f4a80-7c00-11e6-8080-80010479bedb",
"object":"Event:2",
"origin":null,
"target":null,
"time":"2016-09-16T11:29:13.000000",
"to":[
"notification:2"
],
"verb":"join"
}
]
}
Notice the difference in the date fields. Both were submitted as ISO8601 fields, but only the event_start field is returned as such. The time field is returned in a different format but without any timezone indicator. It's apparently converted to UTC time but it's lacking the trailing Z that should indicate that. Therefor I cannot reliable use it as a time indicator.
I could work around this by adding another time field to the activity array (which works fine and returns a correct ISO8601 date), but that feels quite redundant.
Is this a bug, am I doing something wrong or should we simply not rely on the time field for an indication of when the activity took place?
API v1.0 stores the time field in UTC timezone but sadly it does not properly return the trailing Z. This will be fixed in later API versions, but not on this one.
You can safely expect the time field to be always returned as UTC when you query the APIs.
Related
I'm trying to send a Timestamp field which is ISO 8601 with offset (
"2023-02-01T11:11:12.2220000+03:00" )
Azure doesn't really work with offsets, I first encountered that when sending data to event hub.
I was hoping to resolve this by splitting timestamp field into 2 fields:
timestamp: 2023-02-01T11:11:12.2220000
offset: +03:00
and combining them is SA query.
This seemed to have worked in Query editor, where test output is shown as a correct timestamp+offset
however when data is sent to output (in this case SQL, field type datetimeoffset), value looks like this:
2023-02-01T08:11:12.2220000+00:00
I suspect this is because timestamp field type in SA is datetime (seen in query explorer test results window)
even if I cast to nvarchar field type is still datetime.
is there a way to force SA to use specific types for fields (in this case, treat field as a string and not datetime)?
or, in general, how pass value like "2023-02-01T11:11:12.2220000+03:00" through SA without altering it? bonus points if it can be done in Event Hub as well
I am using agenda.js in my Node project, backed with a MongoDB database, to handle batch processes we need to run. This is working well. I do have a question about timezones, however. When I use the every() operation, it seems like it accepts the job name, and the schedule. So I have been seeding jobs to the database like so:
for (let job of dbJobs) {
await agenda.every(schedule, job.name);
}
Note that for the above, schedule is in cron format -- 00 05 * * 1-5.
This works. However, from what I can tell, every() doesn't accept an argument for repeatTimezone. So what does it do to calculate the timezone in those cases?
To clarify, when I look at the document in the database after the job has been added using every(), the repeatTimezone property exists, but its value is set to null.
Other agenda operations, like repeatEvery(), do accept an argument for timezone, like so:
job.repeatEvery('0 6 * * *', {
timezone: 'America/New_York'
});
Since I'm using every(), I have been managing this by first seeding the database using every(), and then running a Mongo updateMany() to add the timzeone explicitly to all jobs:
async function addTimezoneToJobs() {
try {
const db = await client.db(dbName);
await db.collection('batch_processes').updateMany({}, {
$set: {
repeatTimezone: 'America/New_York'
}
});
} catch (error) {
console.log(error);
}
}
But strangely enough, agenda seems to calculate the same time even when I don't explicitly add the repeatTimezone property value to the jobs as when I do.
What's happening here that I'm not understanding? How is the runtime calculated with every(), and is there a way to pass in timezone?
FYI: I am not in the same timezone as that which needs to be set in the db.
Your Question seems to be 2 part, I'm not exactly sure I'll be able to explain it very well but let me try
So, your first question
However, from what I can tell, every() doesn't accept an argument for Timezone
Well Technically you can add Timezone option to every() as well because what this method does is it calls job.repeatEvery internally and as you already know you can add timezone to that. To Support my answer, I found 2 evidence
From Documentation as every accepts 4 parameters
every(interval, name, [data], [options])
options is an optional argument that will be passed to job.repeatEvery. In order to use this argument, data must also be specified.
So you can technically pass timezone if you pass data as well
From SourceCode, here you can see they use job.repeatEvery(interval, options) internally.
Now, To your Second Question
what does it do to calculate the timezone in those cases?
Well They have a very unique yet required module named ComputeNextRunAt().
So I went through their Source Code and figured Out that this is to Compute when will be the next run for your job based on startingTime and Interval.
Your Code works because you have once (Initially) mentioned in your job to follow America/New_York timezone, so every next job interval is calculated based on that, that's the reason you don't need to specify it again.
So, If initially you haven't had specified the timezone attribute, you would have gotten your local Timezone but now you did so, it calculates the next interval based on that.
The first answer of this question suggests that mongoose would adapt the date according to server timezone when retrieving data.
However, I don't have this comportement.
I set the (node) server timezone with :
process.env.TZ='Europe/Paris'
For exemple if I create a simple model like :
const mongoose = require("mongoose");
const testSchema = new mongoose.Schema({
myDate: { type: Date, required: true },
}, { timestamps: true });
exports.Comment = mongoose.default.model('TestSchema', testSchema);
But if I create a date with 2020-01-01 20:20:20, when doing TestSchema.find() the date will be: 2020-01-01T19:20:20.000Z so there are two things that I don't understand :
Europe/Paris is actually UTC +2, so I would expect the date to be either 2020-01-01T18:20:20.000Z in UTC or 2020-01-01T20:20:20.000Z with the server timezone
How to have mongoose automatically set the date to the correct timezone?
I know that myDate is a Date object, so I can convert it manually but I'd rather not have to do it myself for simple reasons like forgetting to convert one of the dates in the application or not having to do it every time a Date field is added
An easy solution that I can think of would be to register a global plugin for mongoose which would use schema.set('toJSON', ... and schema.set('toObject', ...) with the transform method so I can loop through schema fields and if the field is a Date, update it to my timezone.
But I see two problems with this approch :
It doesn't sound very good performance-wise if I am querying a lot of documents each with a lot of fields
As you can see here I am currently not able to register global plugins...
What would be the best method to get the date in the server timezone format? I would rather still store them in UTC but set the hour according to the server timezone.
EDIT :
I just saw that while console.log(myDate) outputs 2018-01-01T19:20:20.000Z console.log(myDate.toString() outputs Mon Jan 01 2018 20:20:20 GMT+0100 (Central European Standard Time) so it seems likes this could be used, even tho I'd rather still have a Date object and converting it to string just before sending it to the client (would need some formatting tho since this format is not very user friendly). But then again, how would I do this globally and not for every date
A few things:
Europe/Paris at 2020-01-01T20:20:20 is UTC+1. It doesn't switch to UTC+2 until Summer Time kicks in on March 29th. Reference here. Thus the conversion to 2020-01-01T19:20:20Z is correct.
The output of console.log when passed a Date object is implementation specific. Some implementations will emit the output of .toString() (which is in local time in RFC 2822 format), and some will emit the output of .toISOString() (which is in UTC in ISO 8601 extended format). That is why you see the difference.
In general, it is not good to send a local time without also sending a time zone offset. ISO 8601 format is ideal, but you should send either 2020-01-01T19:20:20Z, or 2020-01-01T20:20:20+01:00. Don't just send the date and time without an offset to the client. Otherwise, if your client could be in a different time zone then they would interpret the value incorrectly.
Keep in mind that Date objects are not time zone aware. They contain only a Unix timestamp internally, and they convert only to the system's local time zone for the functions that work in local time. They cannot work in any other time zone.
Relying on the system local time zone is bad for portability. One doesn't always have the ability to change it, and it doesn't do well when you have to work in multiple time zones. It would be better to not rely on setting a local time zone from Node's TZ variable. Instead, consider writing your code to be independent of any local time zone setting.
A time zone aware date library can help with most of your concerns. I can recommend Luxon, js-Joda, Moment + Moment-Timezone, or date-fns + date-fns-timezone.
"how would I do this globally" is something I'm not following in your question. Try the approach I described, and if you still have issues then open a new question. Try to be specific and ask a single question. You're likely to get better results that way. Please read How do I ask a good question? and How to create a Minimal, Complete, and Verifiable example. Thanks.
To solve the issue:
npm i mongoose-timezone
In your schema file:
import timeZone from "mongoose-timezone";
const testSchema = new mongoose.Schema({
myDate: { type: Date, required: true },
}, { timestamps: true });
// mongoose will save the dates based on user's timezone
testSchema.plugin(timeZone)
mongoose-timezone basically adds the current timezone offset to the
date before store and removes the offset when retrieving data. This
way dates are kept proportional in the database and in the app.
I'm using NodeJS and DynamoDB. I'm never used DynamoDB before, and primary a C# developer (where this would simply just be a .Where(x => x...) call, not sure why Amazon made it any more complicated then that). I'm trying to simply just query the table based on if an id starts with certain characters. For example, we have the year as the first 2 characters of the Id field. So something like this: 180192, so the year is 2018. The 20 part is irrelevant, just wanted to give a human readable example. So the Id starts with either 18 or 17 and I simply want to query the db for all rows that Id starts with 18 (for example, could be 17 or whatever). I did look at the documentation and I'm not sure I fully understand it, here's what I have so far that is just returning all results and not the expected results.
let params = {
TableName: db.table,
ProjectionExpression: "id,CompetitorName,code",
KeyConditionExpression: "begins_with(id, :year)",
ExpressionAttributeValues: {
':year': '18'
}
return db.docClient.scan(params).promise();
So as you can see, I'm thinking that this would be a begins_with call, where I look for 18 against the Id. But again, this is returning all results (as if I didn't have KeyConditionExpression at all).
Would love to know where I'm wrong here. Thanks!
UPDATE
So I guess begin_with won't work since it only works on strings and my id is not a string. As per commenters suggestion, I can use BETWEEN, which even that is not working either. I either get back all the results or Query key condition not supported error (if I use .scan, I get back all results, if I use .query I get the error)
Here is the code I'm trying.
let params = {
TableName: db.table,
ProjectionExpression: "id,CompetitorName,code",
KeyConditionExpression: "id BETWEEN :start and :end",
ExpressionAttributeValues: {
':start': 18000,
':end': 189999
}
};
return db.docClient.query(params).promise();
It seems as if there's no actual solution for what I was originally trying to do unfortunately. Which is a huge downfall of DynamoDB. There really needs to be some way to do 'where' using the values of columns, like you can in virtually any other language. However, I have to admit, part of the problem was the way that id was structured. You shouldn't have to rely on the id to get info out of it. Anyways, I did find another column DateofFirstCapture which using with contains (all the dates are not the same format, it's a mess) and using a year 2018 or 2017 seems to be working.
if you want to fetch data by id, add it as the partition key. If you want to get data by part of the string, you can use "begins with" on sort key.
begins_with (a, substr)— true if the value of attribute a begins with a particular substring.
source: https://docs.amazonaws.cn/en_us/amazondynamodb/latest/developerguide/Query.html
begins_with and between can only be used on sort keys.
For query you must always supply partition key.
So if you change your design to have unique partition key (or unique combo of partition/sort keys) and strings like 180192 as sort key you will be able to query begins_with(sortkey, ...).
I am using Node for fetching data from MySQL. In database, i got record like : 2013-08-13 15:44:53 . But when Node fetches from Database , it assigns value like 2013-08-19T07:54:33.000Z.
I just need to get time format as in MySQL table. Btw ( My column format is DateTime in MySQL)
In Node :
connection.query(post, function(error, results, fields) {
userSocket.emit('history :', {
'dataMode': 'history',
msg: results,
});
});
When retrieving it from the database you most likely get a Date object which is exactly what you should work with (strings are only good to display dates, but working on a string representation of a date is nothing you want to do).
If you need a certain string representation, create it based on the data stored in the Date object - or even better, get some library that adds a proper strftime-like method to its prototype.
The best choice for such a library is moment.js which allows you to do this to get the string format you want:
moment('2013-08-19T07:54:33.000Z').format('YYYY-MM-DD hh:mm:ss')
// output (in my case on a system using UTC+2 as its local timezone):
// "2013-08-19 09:54:33"
However, when sending it through a socket (which requires a string representation) it's a good idea to use the default one since you can pass it to the new Date(..) constructor on the client side and get a proper Date object again.