I'm having trouble finding a package that allows me to query the timezone information for any timezone at any point in time. Or at least the next 10 years or so. Historical values are not of interest in my use case.
Ideally, I would like to have an API like:
var tzInfo = require(tz-info);
var utcOffset = tzInfo.getOffset('Europe/Stockholm', new Date()); //query the current offset from UTC
var allTimezones = tzInfo.getAllTimezoneNames();
I have searched but came up short. Most libraries like moment-timezone seem very focused on expressing one time in different timezones. And that is different from my use case.
In linux you can do:
#get all timezone names
cd /usr/share/zoneinfo ; find
#get all offset changes
zdump -v <timezone>
I guess that is an option and parse the output from zdump to build my own database. But I rather not. Platform independence is very nice to have. But not a deal breaker.
Moment-timezone absolutely can fit this requirement.
var moment = require('moment-timezone');
// query the current offset from UTC
var utcOffset = moment.tz('Europe/Stockholm').utcOffset();
// query the offset from UTC for a specific point in time
// (input can be string, date, moment, timestamp, etc. see the docs)
var utcOffset = moment.tz(input, 'Europe/Stockholm').utcOffset();
This works because moment-timezone is an extension to moment.js. All of the moment.js functions are available, including the utcOffset function.
And yes, you can also get the names of the time zones:
var allTimezones = moment.tz.names();
However, I'll highlight one flaw:
... Or at least the next 10 years or so ...
There is no system on the planet that can accurately give you a prediction of time zones 10 years into the future. Governments change their minds constantly, and systems have to be updated accordingly. You should plan for this.
The information in the TZDB (as used by both Linux, moment-timezone, and many others) is only as accurate as the current information given by the governments of the world. A prediction 10 years into the future is most certainly going to need to be re-evaluated multiple times.
Have you looked at tz node module here? And its source here.
It exports an object with all possible time zone 'offsets' as keys, and array of applicable timezones as value. You could simply do var tz = requrie('tz');, iterate over all keys in tz and then over array associated with the key, plucking out name from each object.
The resulting list should give you list of all possible timezones.
I am doing this
datediff = (date1 - DateOfDeposit).TotalDays;
But this gives the no. of days and I want no.of months.
The simplest option is probably to use my Noda Time library, which was designed for exactly this sort of thing (as well as making it cleaner to work with dates and times in general):
LocalDate start = new LocalDate(2013, 1, 5);
LocalDate end = new LocalDate(2014, 6, 1);
Period period = Period.Between(start, end, PeriodUnits.Months);
Console.WriteLine(period.Months); // 16
There's nothing built into .NET to make this particularly easy. You could subtract years and months as per Ani's answer, but then also take the day of month into account to avoid the issue I describe in comments. I'd suggest writing a good set of unit tests first though - bearing in mind leap years and the like.
Assuming you want the number of whole months, I think this should work (not sure what corner-cases this doesn't handle, but can't think of any off the top of my head):
12 * (date1.Year - DateOfDeposit.Year) + (date1.Month - DateOfDeposit.Month)
You could always add any fractional component with DateTime.Day, but it's not clear how that should precisely be defined (how many months are between Jan 29 and Feb 27?).
If you're doing a lot of date-time handling, Jon Skeet's Noda Time library (that he mentions in his answer) is a good choice.
I like the Noda Time handling of formatting and parsing values using the various *Pattern types in the NodaTime.Text namespace. However, user input is often not as regular as a single format. For example, our app uses the time format "h:mm tt" but we would like to be able to parse user input in any of the following formats:
h:mm tt
h:mmtt (no space)
h:mm t
h:mmt (no space)
h tt
hh:mm
and so on...
Is there a way to use Noda Time to parse input that may be in any of a number of formats?
(Sorry for taking so long to respond to this.)
Annoyingly, it looks like we haven't exposed this.
It's present in Noda Time in the CompositePattern class. Unfortunately that's currently internal. I've raised issue 147 to fix this at some point (probably in the 1.1 time frame).
For the moment, it's probably easiest to just use the same code from CompositePattern - you needn't create your own IPattern<T> implementation for this, as it's only really the Parse method which is useful. (Indeed, you could even write it as an extension method on IEnumerable<IPattern<T>>, although I'm not sure offhand whether we've given enough visibility to create the same kind of failure result.)
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
Basically, I have a reasonably large list (a year's worth of data) of times that a single discrete event occurred (for my current project, a list of times that someone printed something). Based on this list, I would like to construct a statistical model of some sort that will predict the most likely time for the next event (the next print job) given all of the previous event times.
I've already read this, but the responses don't exactly help out with what I have in mind for my project. I did some additional research and found that a Hidden Markov Model would likely allow me to do so accurately, but I can't find a link on how to generate a Hidden Markov Model using just a list of times. I also found that using a Kalman filter on the list may be useful but basically, I'd like to get some more information about it from someone who's actually used them and knows their limitations and requirements before just trying something and hoping it works.
Thanks a bunch!
EDIT: So by Amit's suggestion in the comments, I also posted this to the Statistics StackExchange, CrossValidated. If you do know what I should do, please post either here or there
I'll admit it, I'm not a statistics kind of guy. But I've run into these kind of problems before. Really what we're talking about here is that you have some observed, discrete events and you want to figure out how likely it is you'll see them occur at any given point in time. The issue you've got is that you want to take discrete data and make continuous data out of it.
The term that comes to mind is density estimation. Specifically kernel density estimation. You can get some of the effects of kernel density estimation by simple binning (e.g. count the number events in a time interval such as every quarter hour or hour.) Kernel density estimation just has some nicer statistical properties than simple binning. (The produced data is often 'smoother'.)
That only takes care of one of your problems, though. The next problem is still the far more interesting one -- how do you take a time line of data (in this case, only printer data) and produced a prediction from it? First thing's first -- the way you've set up the problem may not be what you're looking for. While the miracle idea of having a limited source of data and predicting the next step of that source sounds attractive, it's far more practical to integrate more data sources to create an actual prediction. (e.g. maybe the printers get hit hard just after there's a lot of phone activity -- something that can be very hard to predict in some companies) The Netflix Challenge is a rather potent example of this point.
Of course, the problem with more data sources is that there's extra legwork to set up the systems that collect the data then.
Honestly, I'd consider this a domain-specific problem and take two approaches: Find time-independent patterns, and find time-dependent patterns.
An example time-dependent pattern would be that every week day at 4:30 Suzy prints out her end of the day report. This happens at specific times every day of the week. This kind of thing is easy to detect with fixed intervals. (Every day, every week day, every weekend day, every Tuesday, every 1st of the month, etc...) This is extremely simple to detect with predetermined intervals -- just create a curve of the estimated probability density function that's one week long and go back in time and average the curves (possibly a weighted average via a windowing function for better predictions).
If you want to get more sophisticated, find a way to automate the detection of such intervals. (Likely the data wouldn't be so overwhelming that you could just brute force this.)
An example time-independent pattern is that every time Mike in accounting prints out an invoice list sheet, he goes over to Johnathan who prints out a rather large batch of complete invoice reports a few hours later. This kind of thing is harder to detect because it's more free form. I recommend looking at various intervals of time (e.g. 30 seconds, 40 seconds, 50 seconds, 1 minute, 1.2 minutes, 1.5 minutes, 1.7 minutes, 2 minutes, 3 minutes, .... 1 hour, 2 hours, 3 hours, ....) and subsampling them via in a nice way (e.g. Lanczos resampling) to create a vector. Then use a vector-quantization style algorithm to categorize the "interesting" patterns. You'll need to think carefully about how you'll deal with certainty of the categories, though -- if your a resulting category has very little data in it, it probably isn't reliable. (Some vector quantization algorithms are better at this than others.)
Then, to create a prediction as to the likelihood of printing something in the future, look up the most recent activity intervals (30 seconds, 40 seconds, 50 seconds, 1 minute, and all the other intervals) via vector quantization and weight the outcomes based on their certainty to create a weighted average of predictions.
You'll want to find a good way to measure certainty of the time-dependent and time-independent outputs to create a final estimate.
This sort of thing is typical of predictive data compression schemes. I recommend you take a look at PAQ since it's got a lot of the concepts I've gone over here and can provide some very interesting insight. The source code is even available along with excellent documentation on the algorithms used.
You may want to take an entirely different approach from vector quantization and discretize the data and use something more like a PPM scheme. It can be very much simpler to implement and still effective.
I don't know what the time frame or scope of this project is, but this sort of thing can always be taken to the N-th degree. If it's got a deadline, I'd like to emphasize that you worry about getting something working first, and then make it work well. Something not optimal is better than nothing.
This kind of project is cool. This kind of project can get you a job if you wrap it up right. I'd recommend you do take your time, do it right, and post it up as function, open source, useful software. I highly recommend open source since you'll want to make a community that can contribute data source providers in more environments that you have access to, will to support, or time to support.
Best of luck!
I really don't see how a Markov model would be useful here. Markov models are typically employed when the event you're predicting is dependent on previous events. The canonical example, of course, is text, where a good Markov model can do a surprisingly good job of guessing what the next character or word will be.
But is there a pattern to when a user might print the next thing? That is, do you see a regular pattern of time between jobs? If so, then a Markov model will work. If not, then the Markov model will be a random guess.
In how to model it, think of the different time periods between jobs as letters in an alphabet. In fact, you could assign each time period a letter, something like:
A - 1 to 2 minutes
B - 2 to 5 minutes
C - 5 to 10 minutes
etc.
Then, go through the data and assign a letter to each time period between print jobs. When you're done, you have a text representation of your data, and that you can run through any of the Markov examples that do text prediction.
If you have an actual model that you think might be relevant for the problem domain, you should apply it. For example, it is likely that there are patterns related to day of week, time of day, and possibly date (holidays would presumably show lower usage).
Most raw statistical modelling techniques based on examining (say) time between adjacent events would have difficulty capturing these underlying influences.
I would build a statistical model for each of those known events (day of week, etc), and use that to predict future occurrences.
I think the predictive neural network would be a good approach for this task.
http://en.wikipedia.org/wiki/Predictive_analytics#Neural_networks
This method is also used for predicting f.x. weather forecasting, stock marked, sun spots.
There's a tutorial here if you want to know more about how it works.
http://www.obitko.com/tutorials/neural-network-prediction/
Think of a markov chain like a graph with vertex connect to each other with a weight or distance. Moving around this graph would eat up the sum of the weights or distance you travel. Here is an example with text generation: http://phpir.com/text-generation.
A Kalman filter is used to track a state vector, generally with continuous (or at least discretized continuous) dynamics. This is sort of the polar opposite of sporadic, discrete events, so unless you have an underlying model that includes this kind of state vector (and is either linear or almost linear), you probably don't want a Kalman filter.
It sounds like you don't have an underlying model, and are fishing around for one: you've got a nail, and are going through the toolbox trying out files, screwdrivers, and tape measures 8^)
My best advice: first, use what you know about the problem to build the model; then figure out how to solve the problem, based on the model.
A client is asking how long does it take to audit the security of his Drupal module that is 29k lines long. Does anyone know at least what ballpark I should give him? His main concerns are file encryption and user permission.
Nope, not a damn clue :-)
However, whatever value you choose, may I suggest one thing?
Monitor your progress! Tell your client that your initial estimate is (for example) twenty-nine working days but that it depends on a great many factors outside your control.
Tell them you plan to mitigate risks of budget overrun by providing a daily snapshot of progress:
current number of lines audited in total [a].
days spent [b].
current "run rate" (number of lines per day, average) [c = a/b].
number of lines yet to be audited [d = 29,000 - a].
estimated days to completion [e = d / c].
Allow them to pull the plug at any time if the run rate is well below what you estimated.
This basic project management/reporting should give them the confidence that you know what you're doing, and will minimise their exposure considerably, to the point where they'll feel a lot more comfortable about taking you on.
Just on that last bullet point above, you may want to consider giving them a range (say +/-5% of the estimate), but don't get too clever about working out best and worst case based on your best and worst days to date. The power of averaging is that it gives you a "best" guess without having to fiddle too much with figures.
Typical estimates I've seen are that you can expect a developer to review 100-150 lines of code per hour. This is a very rough estimate, and it will vary greatly depending upon the nature of the code and the thoroughness of the review. Also, if you can review code for 8 hours a day, 5 days a week, straight, you're inhuman and amazing; for the rest of us, we need a change of activity to clear the brain.