I have the following scenario to be load tested for a service and it does not seem to work as expected. My scenario is as follows.
Test with rampUsers(100) over 15 minutes duration
Hold the users for about 10 minutes holdFor(10 minutes)
Then again rampUsers(200) over 15 minutes duration
Hold the users for about 10 minutes holdFor(10 minutes)
Then again rampUsers(200) over 15 minutes duration
I am trying to use throttle option for this but it does not seem to work as expected
here is my code snippets combinations that I have tried so far
//NUM_USERS = 300
//DURATION = 15 minutes
//CONSTANT_DURATION = 5 minutes
// Tried with different combinations of NUM_USERS and DURATION but not helpful
scn.inject(
rampUsers(NUM_USERS*1) during DURATION,
constantUsersPerSec(1) during CONSTANT_DURATION,
rampUsers(NUM_USERS*2) during DURATION,
constantUsersPerSec(2) during CONSTANT_DURATION,
rampUsers(NUM_USERS*3) during DURATION,
constantUsersPerSec(3) during CONSTANT_DURATION
)
scn.inject(
rampUsers(NUM_USERS) during DURATION
).throttle(
reachRps(NUM_USERS/4) in (CONSTANT_DURATION),
holdFor(CONSTANT_DURATION),
jumpToRps(NUM_USERS/3),
holdFor(CONSTANT_DURATION),
jumpToRps(NUM_USERS/2),
holdFor(CONSTANT_DURATION)
)
scn.inject(
rampUsers(NUM_USERS) during DURATION
).throttle(
holdFor(CONSTANT_DURATION),
reachRps(NUM_USERS+NUM_USERS) in (DURATION+DURATION),
holdFor(CONSTANT_DURATION)
)
Can any one help on this which one works in this case. I would like to have a graph like this
To target injection rate as you stated you want in the comments, you need something like this
scn.inject(
rampUsersPerSec(0) to (300) during DURATION,
constantUsersPerSec(300) during CONSTANT_DURATION,
rampUsersPerSec(300) to (600) during DURATION,
constantUsersPerSec(600) during CONSTANT_DURATION,
...
)
Related
For LivyOperator we set the following parameters:
polling_interval=60
retries_num_timeout=100
We set it up according to this documentation: https://airflow.apache.org/docs/apache-airflow-providers-apache-livy/stable/_api/airflow/providers/apache/livy/operators/livy/index.html
But, in this configuration after 100 * 60 seconds = 6000 seconds = 1 hour 40 minutes Livy-session is interrupted, operator becomes failed, loading is interrupted. Is there any way to resove such inconsistency on Airflow/Livy side?
I have a list of image urls from which I use MS Azure faceAPI to extract some features from the photos. The problem is that whenever I iterate more than 20 urls, it seems not to work on any url after the 20th one. There is no error shown. However, when I manually changed the range to iterate the next 20 urls, it worked.
Side note, on free version, MS Azure Face allows only 20 requests/minute; however, even when I let time sleep up to 60s per 10 requests, the problem still persists.
FYI, I have 360,000 urls in total, and sofar I have made only about 1000 requests.
Can anyone help tell me why this happens and how to solve this? Thank you so much!
# My codes
i = 0
for post in list_post[800:1000]:
i += 1
try:
image_url = post['photo_url']
headers = {'Ocp-Apim-Subscription-Key': KEY}
params = {
'returnFaceId': 'true',
'returnFaceLandmarks': 'false',
'returnFaceAttributes': 'age,gender,headPose,smile,facialHair,glasses,emotion,hair,makeup,occlusion,accessories,blur,exposure,noise',
}
response = requests.post(face_api_url, params=params, headers=headers, json={"url": image_url})
post['face_feature'] = response.json()[0]
except (KeyError, IndexError):
continue
if i % 10 == 0:
time.sleep(60)
The free version has a max of 30 000 request per month, your 356 000 faces will therefore take a year to run.
The standard version costs USD 1 per 1000 requests, giving a total cost of USD 360. This option supports 10 transactions per second.
https://azure.microsoft.com/en-au/pricing/details/cognitive-services/face-api/
Using the import to models option, I am importing an XLS file with about 15,000 rows.
With the microtime_float function, the script times and echos out how long it takes. At 29.6 secs, this happens, showing it took less than 30 seconds. At that time, I can see the database has all 15k+ records as expected, no issues there.
Problem is, the browser is kept busy and at 1 min 22 secs, 1 min 55 secs and 2 min 26 secs it prompts me to either wait or kill the process. I keep clicking wait and finally it ends at 2 mins 49 secs.
This is a terrible user experience, how can I cut off this extra wait time?
It's a very basic setup: the route calls importcontroller#import with http get and the code is as follows:
public function import()
{
ini_set('memory_limit', '1024M');
$start = $this->microtime_float();
Excel::import(new myImport, 'myfile.xls' , null, \Maatwebsite\Excel\Excel::XLS);
$end = $this->microtime_float();
$t = $end - $start;
return "Time: $t";
}
The class uses certain concerns as follows:
class myImport implements ToModel, WithBatchInserts, WithChunkReading, WithStartRow
I need a way to execute my python script every 15 mins starting from 9:30:42 till 15:00:42 every day from Mon-Fri.
I have explored APScheduler with cron syntax but can't figure out how to code the above condition. I tried below but doesn't work (execute is my function name)
sched.add_cron_job(execute, day_of_week='mon-fri', hour='9:30:42-15:00:42', minute='*/15')
Any pointer is deeply appreciated.
As far as I can tell you won't be able to do what you want with a single job.
This is the closest I could get with one:30-59/15 9-14 * * 1-5 which equates to Every 15 minutes, minutes 30 through 59 past the hour, between 09:00 AM and 02:59 PM, Monday through Friday.
Although it isn't exactly what you wanted I hope this helps as a base.
I wrote custom code to solve my problem. Posting here in case it helps someone. Any optimisations suggestions are welcome.
The first infinite loop starts the job when the start time is hit. The 2nd infinite loop wakes up every x minutes to check if next run time has approached. If yes, it executes else goes back to sleep. If the end time for execution has reached, then it breaks out
def execute_schedule_custom():
start_time_of_day = datetime.combine(date.today(), time(9, 30, 42))
next_run_time = start_time_of_day
end_time_of_day = datetime.combine(date.today(), time(15, 0, 42))
interval = 15
sleep_secs = 60 * 5 #sleep for 5 mins
while True:
if datetime.now() >= start_time_of_day:
execute()
next_run_time = start_time_of_day + timedelta(minutes=interval)
break
while True:
if datetime.now() >= end_time_of_day:
break
elif datetime.now() >= next_run_time:
execute()
next_run_time = next_run_time + timedelta(minutes=interval)
t.sleep(sleep_secs)
I am having issues using rampUser() method in my gatling script. The request is getting stuck after the following entry which had passed half way through.
Version : 3.3
================================================================================
2019-12-18 09:51:44 45s elapsed
---- Requests ------------------------------------------------------------------
> Global (OK=2 KO=0 )
> graphql / request_0 (OK=1 KO=0 )
> rest / request_0 (OK=1 KO=0 )
---- xxxSimulation ---------------------------------------------------
[##################################### ] 50%
waiting: 1 / active: 0 / done: 1
================================================================================
I am seeing the following in the log which gets repeated for ever and the log size increases
09:35:46.495 [GatlingSystem-akka.actor.default-dispatcher-2] DEBUG io.gatling.core.controller.inject.open.OpenWorkload - Injecting 0 users in scenario xxSimulation, continue=true
09:35:47.494 [GatlingSystem-akka.actor.default-dispatcher-6] DEBUG io.gatling.core.controller.inject.open.OpenWorkload - Injecting 0 users in scenario xxSimulation, continue=true
The above issue is happening only with rampUser and not happening with
atOnceUsers()
rampUsersPerSec()
rampConcurrentUsers()
constantConcurrentUsers()
constantUsersPerSec()
incrementUsersPerSec()
Is there a way to mimic rampUser() in some other way or is there a solution for this.
My code is very minimal
setUp(
scenarioBuilder.inject(
rampUsers(2).during(1 minutes)
)
).protocols(protocolBuilder)
I am stuck with this for some time and my earlier post with more information can be found here
Can any of the gatling experts help me on this?
Thanks for looking into it.
It seems you have slightly incorrect syntax for a rampUsers. You should try remove a . before during.
I have in my own script this code and it works fine:
setUp(userScenario.inject(
// atOnceUsers(4),
rampUsers(24) during (1 seconds))
).protocols(httpProtocol)
Also, in Gatling documentation example is also without a dot Open model:
scn.inject(
nothingFor(4 seconds), // 1
atOnceUsers(10), // 2
rampUsers(10) during (5 seconds), // HERE
constantUsersPerSec(20) during (15 seconds), // 4
constantUsersPerSec(20) during (15 seconds) randomized, // 5
rampUsersPerSec(10) to 20 during (10 minutes), // 6
rampUsersPerSec(10) to 20 during (10 minutes) randomized, // 7
heavisideUsers(1000) during (20 seconds) // 8
).protocols(httpProtocol)
)
My guess is that syntax can't be parsed, so instead 0 is substituted. (Here is example of rounding. Not applicable, but as reference: gatling-user-injection-constantuserspersec)
Also, you mentioned that others method work, could you paste working code as well?