I am pretty new to Solr and requires some help. Currently I am pushing all my logs to be indexed by Solr using Flume-ng and Syslog-ng. I would also like to index the date, but I keep on getting exception due to different time format.
Syslog-ng generates ISO date/time according to this format: 2013-12-24T10:36:24.0000+8:00 or in general YYYY-MM-DDTHH:MM:SS:ffffTZD but Solr is only accepting in Zulu time YYYY-MM-DDTHH:MM:SSZ.
Is there any way for me to create a custom Solr date parser so I can correctly parse the date/time. Google gave me no result of how to approach this.
Or is there any way syslog-ng can generates the date/time in format accepted by Solr?
Currently, as a workaround I had to set the servers to UTC and generate the date/time manually using the template. However, I would prefer not to set all servers to UTC since it can be in different timezone and I would like to keep it that way.
Related
In firebase cloud function I am using typescript and I am not able to get a solution to convert firestore time stamp human-readable formate currently it is showing as '063720731421.495000000'
How to solve this any idea?
A Firestore Timestamp just contains two values (seconds and nanosecond) representing an offset from unix epoch time. This is similar to the Date object that some platforms use to represent a point in time.
The Firestore APIs aren't going to do anything to help you format the date. There are lots of libraries out there for various languages and platforms that will format dates for you. It should be trivial to convert Firestore's Timestamp into something you can pass to one of these libraries. In fact, on many platforms, Timestamp has a method to convert it to a native Date object.
In the JavaScript environment, you can get a Date object by simply calling toDate() on the Timestamp. After that, a library such as moment.js can help you format it any way you want.
I'm developing a tennis matchmaking app where you can post a match and decide at what time and day the match is. I cannot figure out the best way to store time and date in mongodb. I want to be able to display time, use it for search queries and display in text form like "the match is in 1 hour" any suggestion? I was thinking to use moment but I still haven't figured out how.
You would store the date in the default ISODate format supported by mongoDB. It will have the date and the time on it. Once you have the records then you can query against them with $gt and $lt operators etc.
On the UI side you can use momentjs (or date-fns etc) to consume those dates and get the 1 hour from now etc. moment actually has build in humanize methods for that where date-fns has various distanceInWords etc methods.
If I understand correctly, if I'm attempting to aggregate activities in an aggregated feed group based on their calendar day, I necessarily do this relative to the UTC date. This unfortunately can yield confusing results if it's done somewhere like North America. Is it possible to aggregate on a date relative to a different time zone? Is there another way to achieve a similar result?
Currently it's not possible to provide an offset to the strftime function inside of an aggregation rule.
Without knowing all the specifics I think you may be able to achieve the desired result by adding a separate custom field such as local_date (e.g. with a string value of '2018-05-15'). This would be pre-computed and included with the Activity when it's added to a Stream feed, and referred to in the aggregation rule like {{ local_date }}.
The cavieat / limitation is that you'll need to decide whether to use the 'local date' from the perspective of a user who creates an activity (which may be different to the user reading a feed containing the activity), or a system-wide date that's applied across the application regardless of where your user's are located.
I'm using IBM's Cloudant (CouchDB) data store. I'm planning on storing dates as integers in the format YYYYMMDD instead of JavaScript Dates. Is there any CouchDB functionality that I'd be missing out on by not storing them as JavaScript Dates? Any other reason I shouldn't do this?
I've read this SO Q&A: What's the best way to store datetimes (timestamps) in CouchDB? and from that there appears to be no objections to storing dates in any format. It doesn't answer what built-in functionality might be lost.
You wouldn't be losing any functionality as you would make the date useful by processing it in a Map function as either a Secondary Index/View, Search Index or part of Cloudant Query.
The only downside is that by formatting them as such, you make it more difficult on yourself to use the JavaScript Date functions to modify the date to needs within a Map function.
Storing it as a String is an option. Might be easier to handle this way than as an Integer.
I just want a confirmation from the experts.
I still don't feel confident in what I believe to be the right way of storing and treating dates in such environment.
I'm developing a little app, just for italian users.
Basically, they can create a list of entries, each having a creationDate (I am just interested in the date part, time is not useful in my scenario).
So, the user enters in the "date" form field a date in this format: 22/06/2014 represents the 22th day of June of year 2014. Then, date is parsed like that:
entryData.dateEntry = moment( $(form).find('input[name=dateEntry]').val(), 'DD-MM-YYYY' ).toDate();
Finally, my entry model is added to a backbone.js collection and stored server-side by Node.js + Express in MongoDB.
Querying Mongo for entries, I see:
2014-06-21 22:00:00 +0000
which corresponds to "dateEntry" : Date( 1403388000000 ).
Googling around, I discovered that MongoDB doesn't have the concept of timezone. All dates are stored in UTC and the date object I created before had GMT+2.
But I'm really scared... how will I get back my local timezone's date the easy way?
Next, I'll display entry data in an underscore template, this way:
<%= moment(dateEntry).format('DD/MM/YYYY') %>
And... voilà! I get my local 'italian' date back: 22/06/2014.
Now, my question: is that the right way to go?
The process is: parse dates in local timezone => store in utc => retrieve dates in local timezone. Is it a common practice?
I also thought: can't I simply avoid using timezones and storing my local (italian) time as it was utc time (2014-06-22 00:00:00)? Is that so bad?
Yes, it's a common practise to store all timestamps in UTC, and then convert it to specific timezones in the outer layers. A lot of frameworks automatically do that for you, including Rails. Let's say if going forward you start catering to other timezones as well, you will not face any problems because your DB has UTC entries. You will be saved the overhead of timezone conversions.
In case you want to save only dates, you can do that as well, but I don't see any harm in the way you do it currently - save everything in UTC. I am not sure about node.js but there would be some setting where you can specify the timezone (of Italy), and all your conversions will happen automatically. You may find this thread useful:
How can I set the default timezone in node.js?