I'm struggling with a vehicle availability problem in Excel. I need to track overlapping date-time ranges in a way that will show me how often more than two incidents are going on at the same time.
To put it differently, if I have only two units available to handle transports and I am given a list of start and end times of historical transports, how many times would I miss a transport on that list (how often are more then two transports going on at the same time)?
For example, if I start with 2 units and a call for service comes in, now I have 1 unit available. If another call comes in before the previous call ends, I now have 0 units available. If I get another call for service before either unit returns to service, I will miss a call.
How can I evaluate a list of date-time intervals in Excel to determine which calls I would have missed?
I've tried using =SUMPRODUCT((A2<=$B$2:$B$3584)*(B2>=$A$2:$A$3584)) but this isn't quite right and I can't quite figure out why.
Please try (where I have assumed as a start point 1/1/15 has both units available):
=IF(INDEX(B:B,MATCH(F2,A:A))>F2,1,0)+IF(INDEX(D:D,MATCH(F2,C:C))>F2,1,0)
Related
I need to simulate "real traffic" on Web farm, by other words I need to generate high peaks but as well periods which less or even no HTTP requests (hits) at all. Reason for that is to test some atomized mechanisms for adding and reducing CPU and memory for Web servers itself (that is another story). That is why I need "totally random" sceneries when I have loads but as well period with zero or less traffic (so I can add or reduce compute power).
This is situation that I get now, as you can see I always have some avg load its always around some number of hits, even if I change 10 to 100 threads. Values (results) will always have some average value. There are no periods with less or more traffic which would be separated be +10 mints or so, only by few seconds.
Current situation
I would like to get "higher" variations by HITS/REQUESTS with some time breaks between it.
Situation that I want: i.stack.imgur.com/I4LhU.png
I tried several timers but no success and I do not want to use "Ultimate Thread Group" and similar components because I want test to be totaly randome and not predefined with time breaks and pause periods (thread delays). I would like test which will be totally randomized by it self - which could for example generate from 1 to 100 users per XY time.
This is my current Jmeter setup: i.stack.imgur.com/I4LhU.png
I do not know if I am missing some parameter in current setup or there is totally another way to do this.
Thanks a lot!
If this is something you really want (I strongly believe that the test needs to be repeatable, not random), I would suggest using Constant Throughput Timer for this. Despite the word "Constant" in its name you can use a Function or a Variable there, for instance __Random() and you will get different controllable "spikes" each iteration.
Moreover, you put a __P() function and amend its value via Beanshell Server while the test is running
I am writing an app and need to do something functionally similar to what url shortening websites do. I will be generating 6 character (case insensitive alphanumeric) random strings which would identify their longer versions of the link. This leads to 2176782336 possibilities ((10+26)^6). While assigning these strings, there are two approaches I can think about.
Approach 1: the system generates a random string at the runtime and checks for it uniqueness in the system, if it is not unique it tries again. and finally reaches a unique string somehow. But it might create issues if the user is "unlucky" maybe.
Approach 2: I generate a pool of some possible values and assign them as soon as they are needed, this however would make sure the user is always allocated a unique string almost instantly, while this could at the same time also mean, I would have to do plenty of computation in crons beforehand and will increase over the period of time.
While I already have the code to generate such values, a help on approach might be insightful as I am looking forward to a highly accelerated app experience. I could not find any comparative study on this.
Cheers!
What I do in similar situations is to keep N values queued up so that I can instantly assign them, and then when the queue's size falls below a certain threshold (say, .2 * N) I have a background task add another N items to the queue. It probably makes sense to start this background task as soon as your program starts (as opposed to generating the first N values offline and then loading them at startup), operating on the assumption that there will be some delay between startup and requests for values from the queue.
I have some metrics (like counts of logged in users, or SQL queries, or whatever), and I want to gather some time-dependent stats on a regular basis.
For example I want to know how many users were registered in some particular year, month, week, day or even hour.
I thought maybe Redis can be a good fit. But I can't imagine a good strategy for storing such stats. The only idea I have is to store independent counters for days, weeks, etc, and bump them all at once.
How do you do it? I need a good trick. Or maybe Redis isn't any good for my task.
If all you need is a count for each period, the multiple counter approach you suggest is a good one. Incrementing 5 counters in a single pipelined transaction is O(1), while set operations are O(log n + m) with potentially large values of n/m.
The set solution Frank suggested does have its place - I use something similar where I need to know which actions happened rather than just how many. Obviously storing details of each action takes more memory than the counters, but with the amount of RAM typically available these days you can store millions of records before that becomes a problem.
I would just use a sorted sets where the score is the timestamp in seconds since the epoch (unix time). Say you have a sorted set of logins and you want to see how many logins occured in the year 2010, just convert 20101231 23:59:59 and 20100101 00:00:00 to seconds and use those are the max and min arguments to zcount.
The obviously difficulty here is handling the time conversion yourself, but its actually very easy because it the standard Unix format. You can use the date command with %S (on linux at least) or use the system calls time(), localtime() and mktime(), as well as any of the myriad ways available within specific languages that are built on top of these system calls.
I am sure there is some equivalent paradigm in Windows, but that I don't have any experience there.
I'm writing an app using sms as communication.
I have chosen to subscribe to an sms-gateway, which provides me with an API for doing so.
The API has functions for sending as well as pulling new messages. It does however not have any kind of push functionality.
In order to do my queries most efficient, I'm seeking data on how long time people wait before they answer a text message - as a probability function.
Extra info:
The application is interactive (as can be), so I suppose the times will be pretty similar to real life human-human communication.
I don't believe differences in personal style will play a big impact on the right times and frequencies to query, so average data should be fine.
Update
I'm impressed and honered by the many great answers recieved. I have concluded that my best shot will be a few adaptable heuristics, including exponential (or maybe polynomial) backoff.
All along I will be gathering statistics for later analysis. Maybe something will show up. I think I will cheat start on the algorithm for generating poll-frquenzies from a probability distribution. That'll be fun.
Thanks again many times.
In the absence of any real data, the best solution may be to write the code so that the application adjusts the wait time based on current history of response times.
Basic Idea as follows:
Step 1: Set initial frequency of pulling once every x seconds.
Step 2: Pull messages at the above frequency for y duration.
Step 3: If you discover that messages are always waiting for you to pull decrease x otherwise increase x.
Several design considerations:
Adjust forever or stop after sometime
You can repeat steps 2 and 3 forever in which case the application dynamically adjusts itself according to sms patterns. Alternatively, you can stop after some time to reduce application overhead.
Adjustment criteria: Per customer or across all customers
You can chose to do the adjustment in step 3 on a per customer basis or across all customers.
I believe GMAIL's smtp service works along the same lines.
well I would suggest finding some statistics on daily SMS/Text Messaging usage by geographical location and age groups and come up with an daily average, it wont be an exact measurement for all though.
Good question.
Consider that people might have multiple tasks and that answering a text message might be one of those tasks. If each of those tasks takes an amount of time that is exponentially distributed, the time to get around to answering the text message is the sum of those task completion times. The sum of n iid random variables has a Gamma distribution.
The number of tasks ahead of the text return also has a dicrete distribution - let's say it's Poisson. I don't have the time to derive the resulting distribution, but simulating it using #Risk, I get either a Weibull or Gamma distribution.
SMS is a store-and-forward messaging service, so you have to add in the delay that can be added by the various SMSCs (Short Message Service Centers) along the way. If you are connecting to one of the big aggregation houses (Sybase, TNS, mBlox etc) commercial bulk SMS providers (Clickatel, etc) then you need to allow for the message to transverse their network as well as the carriers network. If you are using a smaller shop then most likely they are using a GSM Modem (or modems) and there is a throughput limit on the message the can receive and process (as well as push out)
All that said, if you are using a direct connection or one of the big guys MO (mobile originated) messages coming to you as a CP (content provider) should take less than 5 seconds. Add to that the time it takes the Mobile Subscribers to reply.
I would say that anecdotal evidence form services I've worked on before, where the Mobile Subscriber needs to provide a simple reply it's usually within 10 seconds or not at all.
If you are polling for specific replies I would poll at 5 and 10 seconds then apply an exponential back off.
All of this is from a North American point-of-view. Europe will be fairly close, but places like Africa, Asia will be a bit slower as the networks are a bit slower. (unless you are connected directly to the operator and even then some of them are slow).
I'm looking for a design pattern that would fit my application design.
My application processes large amounts of data and produces some graphs.
Data processing (fetching from files, CPU intensive calculations) and graph operations (drawing, updating) are done in seperate threads.
Graph can be scrolled - in this case new data portions need to be processed.
Because there can be several series on a graph, multiple threads can be spawned (two threads per serie, one for dataset update and one for graph update).
I don't want to create multiple progress bars. Instead, I'd like to have single progress bar that inform about global progress. At the moment I can think of MVC and Observer/Observable, but it's a little bit blurry :) Maybe somebody could point me in a right direction, thanks.
I once spent the best part of a week trying to make a smooth, non-hiccupy progress bar over a very complex algorithm.
The algorithm had 6 different steps. Each step had timing characteristics that were seriously dependent on A) the underlying data being processed, not just the "amount" of data but also the "type" of data and B) 2 of the steps scaled extremely well with increasing number of cpus, 2 steps ran in 2 threads and 2 steps were effectively single-threaded.
The mix of data effectively had a much larger impact on execution time of each step than number of cores.
The solution that finally cracked it was really quite simple. I made 6 functions that analyzed the data set and tried to predict the actual run-time of each analysis step. The heuristic in each function analyzed both the data sets under analysis and the number of cpus. Based on run-time data from my own 4 core machine, each function basically returned the number of milliseconds it was expected to take, on my machine.
f1(..) + f2(..) + f3(..) + f4(..) + f5(..) + f6(..) = total runtime in milliseconds
Now given this information, you can effectively know what percentage of the total execution time each step is supposed to take. Now if you say step1 is supposed to take 40% of the execution time, you basically need to find out how to emit 40 1% events from that algorithm. Say the for-loop is processing 100,000 items, you could probably do:
for (int i = 0; i < numItems; i++){
if (i % (numItems / percentageOfTotalForThisStep) == 0) emitProgressEvent();
.. do the actual processing ..
}
This algorithm gave us a silky smooth progress bar that performed flawlessly. Your implementation technology can have different forms of scaling and features available in the progress bar, but the basic way of thinking about the problem is the same.
And yes, it did not really matter that the heuristic reference numbers were worked out on my machine - the only real problem is if you want to change the numbers when running on a different machine. But you still know the ratio (which is the only really important thing here), so you can see how your local hardware runs differently from the one I had.
Now the average SO reader may wonder why on earth someone would spend a week making a smooth progress bar. The feature was requested by the head salesman, and I believe he used it in sales meetings to get contracts. Money talks ;)
In situations with threads or asynchronous processes/tasks like this, I find it helpful to have an abstract type or object in the main thread that represents (and ideally encapsulates) each process. So, for each worker thread, there will presumably be an object (let's call it Operation) in the main thread to manage that worker, and obviously there will be some kind of list-like data structure to hold these Operations.
Where applicable, each Operation provides the start/stop methods for its worker, and in some cases - such as yours - numeric properties representing the progress and expected total time or work of that particular Operation's task. The units don't necessarily need to be time-based, if you know you'll be performing 6,230 calculations, you can just think of these properties as calculation counts. Furthermore, each task will need to have some way of updating its owning Operation of its current progress in whatever mechanism is appropriate (callbacks, closures, event dispatching, or whatever mechanism your programming language/threading framework provides).
So while your actual work is being performed off in separate threads, a corresponding Operation object in the "main" thread is continually being updated/notified of its worker's progress. The progress bar can update itself accordingly, mapping the total of the Operations' "expected" times to its total, and the total of the Operations' "progress" times to its current progress, in whatever way makes sense for your progress bar framework.
Obviously there's a ton of other considerations/work that needs be done in actually implementing this, but I hope this gives you the gist of it.
Multiple progress bars aren't such a bad idea, mind you. Or maybe a complex progress bar that shows several threads running (like download manager programs sometimes have). As long as the UI is intuitive, your users will appreciate the extra data.
When I try to answer such design questions I first try to look at similar or analogous problems in other application, and how they're solved. So I would suggest you do some research by considering other applications that display complex progress (like the download manager example) and try to adapt an existing solution to your application.
Sorry I can't offer more specific design, this is just general advice. :)
Stick with Observer/Observable for this kind of thing. Some object observes the various series processing threads and reports status by updating the summary bar.