I am writing a LoadTestShape Class. I want to be able to set number of transaction per minute. If I try to set it by specifying user_count it doesn't work because the RPS will vary cpu to cpu .
If say I want to set 1000 transaction per minute, how can we achieve that in locust?
The only way to "achieve" this in Locust is implementing Pacing, the logic is:
You know how much time one iteration takes for one user
If user manages to finish faster - you need to introduce some "think time" so your thread "sleeps" till the desired time
If iteration is slower - no sleep is required.
This answer shows how you can create a custom load shape
Alternatively you can consider switching to a load testing tool which provides such functionality, i.e. Apache JMeter with Constant Throughput Timer
for this timing that's enough to add a LoadTestShape class to locustfile.py like the below code which I use for one of my tests you can change the number of users or other parameters as you want(I write a full docstring that describes each parameter in that):
class StagesShape(LoadTestShape):
"""
A simply load test shape class that has different user and spawn_rate at
different stages.
Keyword arguments:
stages -- A list of dicts, each representing a stage with the following keys:
duration -- When this many seconds pass the test is advanced to the next stage
users -- Total user count
spawn_rate -- Number of users to start/stop per second
stop -- A boolean that can stop that test at a specific stage
stop_at_end -- Can be set to stop once all stages have run.
"""
stages = [
{"duration": 60, "users": 3, "spawn_rate": 0.05},
{"duration": 60, "users": 6, "spawn_rate": 0.05},
{"duration": 60, "users": 9, "spawn_rate": 0.05},
{"duration": 60, "users": 12, "spawn_rate": 0.05},
{"duration": 60, "users": 15, "spawn_rate": 0.05},
{"duration": 60, "users": 18, "spawn_rate": 0.05},
{"duration": 60, "users": 21, "spawn_rate": 0.05},
{"duration": 60, "users": 24, "spawn_rate": 0.05},
{"duration": 60, "users": 27, "spawn_rate": 0.05},
{"duration": 60, "users": 30, "spawn_rate": 0.05},
]
def tick(self):
run_time = self.get_run_time()
for stage in self.stages:
if run_time < stage["duration"]:
tick_data = (stage["users"], stage["spawn_rate"])
return tick_data
return None
I ended up using constant_throughput to control the number of requests per second.
For different time of the day I'd use a different throughput value.
Simply set the wait_time = constant_throughput(0.1). Based on RPS you want you can either set the value low for less number of requests and more for more RPS.
Related
Dear Groovy specialists,
I stumbled upon a phenomenon in Groovy which I would describe as follows:
Given a list of maps that share some common keys, it is possible to access the Map values directly via the List.
Example:
ArrayList people = [
["height": 172, "age": 42],
["height": 180, "age": 66],
["height": 180, "age": null],
["height": 180],
["age": 10]
]
println "people.height: " + people.height
println "people.age: " + people.age
Output:
people.height: [172, 180, 180, 180, null]
people.age: [42, 66, null, null, 10]
Does this syntax (e.g. people.height / people.age) have a name?
Thanks in advance!
PS: This answer would be another example of the mentioned syntax
Does this syntax (e.g. people.height / people.age) have a name?
That is Groovy's syntax for property access.
(Note that a property is a different thing than a field and Groovy has a bunch of special dynamic stuff that happens during property access.)
I tried looking through the documentation for a datetime formating that would allow me to show the time elapsed since a post was created e.g "created 15s ago" but I couldn't find one. I figured I had to create a custom filter but I don't have a clue on how to go about it. I am also new to the Django framework unfortunately.
Django has a |timesince template tag [Django-doc], this formats the time since an event in a "human" way. You thus can format this in the template with:
{{ post.created|timesince }} ago
For example:
>>> print(Template('{{ foo|timesince }} ago').render(Context({'foo': datetime(2021, 2, 20, 21, 15)})))
0 minutes ago
>>> print(Template('{{ foo|timesince }} ago').render(Context({'foo': datetime(2021, 2, 20, 21, 14)})))
1 minute ago
>>> print(Template('{{ foo|timesince }} ago').render(Context({'foo': datetime(2021, 2, 20, 21, 10)})))
5 minutes ago
>>> print(Template('{{ foo|timesince }} ago').render(Context({'foo': datetime(2021, 2, 20, 20, 10)})))
1 hour, 5 minutes ago
Since Django runs as backend, this will however not update the content if the page does not refresh. For more dynamic ways, you should look for a JavaScript library where you thus pass the datetime in a standardized way, and let JavaScript fill in the amount of time that has passed.
Let me explain, I'm working in a bank and I'm trying to make a short python script that calculates the percentage of different shareholders.
In my example EnterpriseA is owned by different Shareholders directly and indirectly I stored it as it follows :
EnterpriseA = {'Shareholder0': {'Shareholder1': 25, 'Shareholder2': 31, 'Shareholder3': 17, 'Shareholder4': 27},
'Shareholder3': {'Shareholder1': 34, 'Shareholder4': 66}}
I want to calculate how much each shareholders have of EntrepriseA, but I can't figure how to check if a shareholder appears multiple times in all my dictionaries.
What I'm thinking is checking if Shareholder1 appears multiple times if so calculate how many percentage he owns of EnterpriseA like this :
percentage = EnterpriseA['Shareholder0']['Shareholder1'] + (EnterpriseA['Shareholder0']['Shareholder3']*EnterperiseA['Shareholder3']['Shareholder1']/100)
I've made a quick drawing for better understanding
If the maximum depth is only ever singly nested then you can just write a little helper function.
Edit:
From what you've explained, 'Shareholder0' is basically a list of direct enterprise shares.
I've modified the helper function and included a constant reflecting that.
ENTERPRISE_SHARES = 'Shareholder0'
EnterpriseA = {
'Shareholder0': {
'Shareholder1': 25,
'Shareholder2': 31,
'Shareholder3': 17,
'Shareholder4': 27
},
'Shareholder3': {
'Shareholder1': 34,
'Shareholder4': 66
}
}
def calc_percent(enterprise, name):
parent_percents = enterprise[ENTERPRISE_SHARES]
total_percent = parent_percents.get(name, 0)
for shareholder, shares in enterprise.items():
if shareholder != ENTERPRISE_SHARES and shareholder != name:
total_percent += parent_percents[shareholder] / 100 * shares.get(name, 0)
return total_percent
print(calc_percent(EnterpriseA, 'Shareholder1'))
print(calc_percent(EnterpriseA, 'Shareholder2'))
print(calc_percent(EnterpriseA, 'Shareholder4'))
I am using fio for storage benchmarking and fio2gnuplot for plotting graphs, every time I run a test and look into logfiles of iops, second coloumn is always 1 which is iops value and due to this graphs are just a straight line perpendicular to Y axis .Which makes no sense. I tried various iodepths,ioengines but no use.am I using any parameters(options)wrong?
following is my jobfile.
[global]
enter code here
rw=randwrite
size=128m
thread=1
iodepth=2
ioengine=libaio
per_job_logs=0
directory=/home/fio
[job_512]
write_bw_log=logfiles_libaio/fio-test_512
write_iops_log=logfiles_libaio/fio-test_512
write_lat_log=logfiles_libaio/fio-test_512
bs=512b
and this is the logfile
1, 1, 0, 512
2, 1, 1, 512
18, 1, 1, 512
19, 1, 0, 512
31, 1, 1, 512
53, 1, 1, 512
55, 1, 1, 512
56, 1, 0, 512
59, 1, 1, 512
63, 1, 1, 512
According to fio manual(man fio), under "FIO FILE FORMATS", it says:
Fio supports a variety of log file formats, for logging latencies, bandwidth, and IOPS. The logs
share a common format, which looks like this:
time (msec), value, data direction, offset
Time for the log entry is always in milliseconds. The value logged depends on the type of log, it
will be one of the following:
Latency log
Value is in latency in usecs
Bandwidth log
Value is in KB/sec
IOPS log
Value is in IOPS
Data direction is one of the following:
0 IO is a READ
1 IO is a WRITE
2 IO is a TRIM
However, I think the 'offset' should be 'IO size'.
So, in your bandwidth case, it's:
timestamp(ms), bandwidth(KB/sec), R/W, size
I have a database with documents that are roughly of the form:
{"created_at": some_datetime, "deleted_at": another_datetime, "foo": "bar"}
It is trivial to get a count of non-deleted documents in the DB, assuming that we don't need to handle "deleted_at" in the future. It's also trivial to create a view that reduces to something like the following (using UTC):
[
{"key": ["created", 2012, 7, 30], "value": 39},
{"key": ["deleted", 2012, 7, 31], "value": 12}
{"key": ["created", 2012, 8, 2], "value": 6}
]
...which means that 39 documents were marked as created on 2012-07-30, 12 were marked as deleted on 2012-07-31, and so on. What I want is an efficient mechanism for getting the snapshot of how many documents "existed" on 2012-08-01 (0+39-12 == 27). Ideally, I'd like to be able to query a view or a DB (e.g. something that's been precomputed and saved to disk) with the date as the key or index, and get the count as the value or document. e.g.:
[
{"key": [2012, 7, 30], "value": 39},
{"key": [2012, 7, 31], "value": 27},
{"key": [2012, 8, 1], "value": 27},
{"key": [2012, 8, 2], "value": 33}
]
This can be computed easily enough by iterating through all of the rows in the view, keeping a running counter and summing up each day as I go, but that approach slows down as the data set grows larger, unless I'm smart about caching or storing the results. Is there a smarter way to tackle this?
Just for the sake of comparison (I'm hoping someone has a better solution), here's (more or less) how I'm currently solving it (in untested ruby pseudocode):
require 'date'
def date_snapshots(rows)
current_date = nil
current_count = 0
rows.inject({}) {|hash, reduced_row|
type, *ymd = reduced_row["key"]
this_date = Date.new(*ymd)
if current_date
# deal with the days where nothing changed
(current_date.succ ... this_date).each do |date|
key = date.strftime("%Y-%m-%d")
hash[key] = current_count
end
end
# update the counter and deal with the current day
current_date = this_date
current_count += reduced_row["value"] if type == "created_at"
current_count -= reduced_row["value"] if type == "deleted_at"
key = current_date.strftime("%Y-%m-%d")
hash[key] = current_count
hash
}
end
Which can then be used like so:
rows = couch_server.db(foo).design(bar).view(baz).reduce.group_level(3).rows
date_snapshots(rows)["2012-08-01"]
Obvious small improvement would be to add a caching layer, although it isn't quite as trivial to make that caching layer play nicely incremental updates (e.g. the changes feed).
I found an approach that seems much better than my original one, assuming that you only care about a single date:
def size_at(date=Time.now.to_date)
ymd = [date.year, date.month, date.day]
added = view.reduce.
startkey(["created_at"]).
endkey( ["created_at", *ymd, {}]).rows.first || {}
deleted = view.reduce.
startkey(["deleted_at"]).
endkey( ["deleted_at", *ymd, {}]).rows.first || {}
added.fetch("value", 0) - deleted.fetch("value", 0)
end
Basically, let CouchDB do the reduction for you. I didn't originally realize that you could mix and match reduce with startkey/endkey.
Unfortunately, this approach requires two hits to the DB (although those could be parallelized or pipelined). And it doesn't work as well when you want to get a lot of these sizes at once (e.g. view the whole history, rather than just look at one date).