pysolr is taking time first time after the solr restart. Scenario is like this.
1)Restart Solr server
2)Execute the query directly in solr is taking 4 sec
3)when we are trying to execute the same query using pysolr, it is taking 300 seconds or more for the first time. After refreshing it is taking only less than 2 sec.
When I check the pysolr code the time is taking in the below code.
resp = requests_method(url, data=bytes_body, headers=headers, files=files,
timeout=self.timeout, auth=self.auth)
4)Can anybody help me in getting the out faster first time also. I think this abnormal because executing the query directly in Solr is taking very few seconds.
Related
I am using Databricks to create an algorithm for big data. I am wondering why the last 1% of my running process takes a lot of time?
I am writing the result in S3, the result for 111991 data (out of 116367) is done in 5 minutes and just for the last 5000 takes more than a hour!!!!!
can I fix this issue?
in the following picture it takes hour 119 become 120, but it came to 199 in a few minutes
Please check you are writing file in one shot or writing in one chunk.
If you are writing in one shot some time it switching log will take time. Also check if you are printing logs then it may take time.
I am trying to execute a simple select query with some where condition from existing table which nearly about 3,00,000 records.
When I am executing it first time it is completing it in around 200 sec while second time it is taking around 20 seconds.
I need to find any requests where 5 of them in a row take 30 seconds or more.
Is there a way to do this using a kusto query?
I guess it could be rephrased as 'if there is a request that takes 30 seconds or more to get a response, check if the next 4 requests also take 30 seconds'.
Check this thread -
Check for the request where response time is higher
Hope it helps.
I have a website that is live. I have a cron job that executes every 24 hours. the cron job fetches and analyzes the data from a database table.
The problem is that the website gets very slow during the time when cron job is running. And gets back to normal after that. It gives me error Too many connections during this time.
I set the maximum allowed connections to 500 in mysql. The number of active connections that I checked in mysql were less than limit during that time.
I am unable to find any relevant help or even a clue to think in a particular direction.
Update:
I noticed one thing. the number of mysql connection continuously increases in this time. Although still less than the maximum limit.
nice command can change priority of a process. You want to lower the priority of the background process so it will try not to execute be executing while the website is being busy. E.g.
0 3 * * * nice -n 20 myjob arg arg
to execute myjob arg arg with lowered priority every day at 3am.
EDIT: Although, if the job is spending most of its time in database queries, this will not affect it much. MySQL has LOW_PRIORITY flag for INSERT and UPDATE statements that will do kind of the same thing for those queries.
I created a CF with caching='ROWS_ONLY' and loaded it with some data. I then configured/enabled row caching by editing the cassandra.yaml file. With tracing on, I ran a select query and it displayed row cache miss (expected) and the query took 10 seconds. I ran the query again and this time it showed row cache hit (again expected) but the time take to run the query was again 10 seconds. Is this normal? I was expecting that the second time the query ran, it should have taken less than 10 sec. Am I missing something here? Are there any other parameters to check/configure to achieve better read performance with row cache enabled?
Thanks in advance ..