How to improve performance of Watir element.present? method? - watir

I experience IMO high delays with the Ruby script using Watir. Here's the problem description: I am testing AJAX based application and I wanted to avoid using of sleep to make sure page gets loaded:
class Page
attr_accessor :expected_elements
def loaded?
# code to make sure AJAX page is loaded
end
end
So instead of this:
def loaded?
# static delay sufficient to get page loaded
sleep(MAX_PAGE_LOAD_TIME)
true
end
I wanted to have something like this:
def loaded?
Watir::Wait.until(MAX_PAGE_LOAD_TIME) do
expected_elements.all? do |element|
element.present?
end
end
end
The problem is the evaluation of the block takes too long. The more elements I check for presence the higher this delay gets. I experienced roughly this delay for one iteration:
Firefox -> 130ms
IE -> 615ms
Chrome -> 115 ms
So to check if N elements are present I get N times corresponding delay... Well the consequence is that eventhough MAX_PAGE_LOAD_TIME expires the Watir::Wait::TimeoutError is not thrown because block evaluation has not been finished yet... So I ended up in the situation where the check for elements presence introduces higher delay than the static delay which is sufficient enough to get page loaded.. I tried to improve performance by locating elements by xpath, but the performance gain was not significant..
What am I doing wrong? Is there a way to speed-up execution time for present? method?? Do these delays correspond with your experience - or are they high?
I checked if the problem could be in the browser-server communication, but here the delays are very low.. I got 100ms time difference for the server response including backend DB request. Of course it takes some time to render page based on this response, but for sure it does not take seconds.
My configuration:
- Win7 OS,
- Firefox 17.0.1
- IE 8.0.7601.17514 with IEDriverServer_x64_2.26.2
- Chrome 23.0.1271.97 m with chromedriver_win_23.0.1240.0,
- Ruby 1.9.3p125,
- watir-webdriver (0.6.1),
- selenium-webdriver (2.27.2)
Thank you for your help!
Based on the discussion I post a sample of benchmarking code:
Benchmark.bm do |x|
x.report("text") do
b.span(:text => "Search").present?
end
end
Benchmark.bm do |x|
x.report("xpath") do
b.span(:xpath => "/html/body/div/div/div[2]/div[2]/div/div/div/div/div/div/div/div[2]/div/div/div[2]/div/div/div/div/div/div[2]/div/div/div/div/div/div/div/div[2]/div/div/div[2]/div/div/div/div/div/div[2]/div/div/div/div/div/span/span").present?
end
end
user system total real
text 0.000000 0.000000 0.000000 ( 0.140405)
xpath 0.000000 0.000000 0.000000 ( 0.120005)
Additional benchmarking results:
container_xpath = "/html/body/div/div/div[2]/div[2]/div/div/div/div/div/div/div/div[2]/div/div/div[2]/div/div/div/div/div/div[2]/div/div/div/div/div/div/div/div/div/div/div/div/div/div/div[2]/div/div/div/div[2]/div/div/div"
Benchmark.bm do |x|
x.report("cntnr") do
#c = b.div(:xpath => container_xpath)
#c.present?
end
end
Benchmark.bm do |x|
x.report("lb1") do
#c.div(:xpath => "div[1]/div/div").present?
##c.div(:text => "Company:").present?
end
end
Benchmark.bm do |x|
x.report("lb2") do
#c.div(:xpath => "div[3]/div/div").present?
##c.div(:text => "Contact person:").present?
end
end
Benchmark.bm do |x|
x.report("lb3") do
#c.div(:xpath => "div[5]/div/div").present?
##c.div(:text => "Address:").present?
end
end
And the results were:
Results for container reference and relative xpath:
user system total real
cntnr 0.000000 0.000000 0.000000 ( 0.156007)
lb1 0.000000 0.000000 0.000000 ( 0.374417)
lb2 0.000000 0.000000 0.000000 ( 0.358816)
lb3 0.000000 0.000000 0.000000 ( 0.358816)
Results for container reference and div's text:
user system total real
cntnr 0.000000 0.000000 0.000000 ( 0.140402)
lb1 0.000000 0.000000 0.000000 ( 0.358807)
lb2 0.000000 0.000000 0.000000 ( 0.358807)
lb3 0.000000 0.000000 0.000000 ( 0.374407)
When absolute xpaths were used:
container_xpath = "/html/body/div/div/div[2]/div[2]/div/div/div/div/div/div/div/div[2]/div/div/div[2]/div/div/div/div/div/div[2]/div/div/div/div/div/div/div/div/div/div/div/div/div/div/div[2]/div/div/div/div[2]/div/div/div"
Benchmark.bm do |x|
x.report("cntnr") do
#c = b.div(:xpath => container_xpath)
#c.present?
end
end
lb1_xpath = container_xpath + "/div[1]/div/div"
Benchmark.bm do |x|
x.report("lb1_x") do
b.div(:xpath => lb1_xpath).present?
end
end
lb2_xpath = container_xpath + "/div[3]/div/div"
Benchmark.bm do |x|
x.report("lb2_x") do
b.div(:xpath => lb2_xpath).present?
end
end
lb3_xpath = container_xpath + "/div[5]/div/div"
Benchmark.bm do |x|
x.report("lb3_x") do
b.div(:xpath => lb3_xpath).present?
end
end
Results were:
user system total real
cntnr 0.000000 0.000000 0.000000 ( 0.140404)
lb1_x 0.000000 0.000000 0.000000 ( 0.124804)
lb2_x 0.000000 0.000000 0.000000 ( 0.156005)
lb3_x 0.000000 0.000000 0.000000 ( 0.140405)

Okay, this answer assumes your site is using jquery. If it's not, you'll have to figure out the library in use and modify the method accordingly...
Write a method that waits for the Ajax calls to finish...
def wait_for_ajax(timeout = 10)
timeout.times do
return true if browser.execute_script('return jQuery.active').to_i == 0
sleep(1)
end
raise Watir::Wait::TimeoutError, "Timeout of #{timeout} seconds exceeded on waiting for Ajax."
end
Call that method when you first load the page you're testing. Then iterate through your expected elements to see if they're present (if you have an array of Watir elements to check, make sure you use .each with it, not .all? as you have in your code there).

Related

Append dataframes while ignoring column names

I have a loop which produces a dataframe at every iteration.
DimensionAll=[]
for i in range(0,10):
###code here###
DimensionAll.append(MatrixDimension)
where MatrixDimension is as follows:
Cameroun Rwanda Niger Zambia Mali Angola Ethiopia
ECON 0.092811 0.088966 0.077843 0.101176 0.080969 0.045516 0.084101
FOOD 0.052086 0.035915 0.037474 0.025168 0.039382 0.015079 0.083499
ENV 0.018479 0.043677 0.004737 0.003744 0.009258 0.034044 0.010285
HEA 0.000061 0.029189 0.012335 0.001238 0.019010 0.007995 0.017359
PERS 0.056941 0.005222 0.048715 0.030879 0.070985 0.064726 0.023330
COM 0.000000 0.000000 0.000000 0.000000 0.009809 0.005251 0.099614
POL 0.025177 0.090846 0.005273 0.029481 0.001929 0.065365 0.034342
How can I ignore the column names when appending? is there a different way to append or concatenate the dataframes after each iteration (while keeping the column names at top without repetition)?
This piece of code gave me the answer i was looking for :
DimensionAll=[]
for i in range(0,10):
###code here###
DimensionAll.append(MatrixDimension)
DimensionAll= pd.concat(DimensionAll)

how to find exponential weighted moving average using dataframe.ewma?

Previously I used the following to calculate the ewma
dataset['26ema'] = pd.ewma(dataset['price'], span=26)
But, in the latest version of pandas pd.ewma has been removed. How to calculate using the new method dataframe.ewma?
dataset['26ema'] = dataset['price'].ewma(span=26)
This is giving an error 'AttributeError: 'Series' object has no attribute 'ewma'
Use Series.ewm:
dataset['price'].ewm(span=26)
See GH11603 for the relevant PR and mapping of the old API to new ones.
Minimal Code Example
s = pd.Series(range(5))
s.ewm(span=3).mean()
0 0.000000
1 0.666667
2 1.428571
3 2.266667
4 3.161290
dtype: float64

CMU Sphinx and time indexed words

I have run the example of Pocketsphinx Python and now I am facing the issue that I want to run a 60sec wav file for speech recognition in English and want as output
- the English translation AND
- at which second each word was mentioned.
Now, I do not know where to start to dome some research to get the required output. Could anyone please point me in the right direction??
ok, the open source tools like Kaldi automatically offers this:
https://americanarchivepb.wordpress.com/2017/12/04/dockerized-kaldi-speech-to-text-tool/
You need recognition with forced alignment. Here is an example for pocketsphinx:
pocketsphinx_continuous
-infile with.wav
-jsgf with-word.jsgf
-dict words.dict
-backtrace yes
-fsgusefiller no
-bestpath no
2>&1 > with-word.txt
Output:
==> with-word.txt <==
INFO: fsg_search.c(869): fsg 0.05 CPU 0.051 xRT
INFO: fsg_search.c(871): fsg 0.09 wall 0.084 xRT
INFO: pocketsphinx.c(1171): sil with sil (-2607)
word start end pprob ascr lscr lback
sil 3 77 1.000 -1602 0 1
with 78 102 1.000 -845 0 1
sil 103 107 1.000 -160 0 1
INFO: fsg_search.c(265): TOTAL fsg 0.05 CPU 0.051 xRT
INFO: fsg_search.c(268): TOTAL fsg 0.09 wall 0.085 xRT
sil with sil
For CMU Sphinx 4 you need the SpeechAligner class from Sphinx API. Here you'll find an implementation of simple aligner tool.
./align.sh sample.wav sample.txt 2>/dev/null
Output:
"it's","IH T S","false","0.0","170","200"
"a","AH","false","-5540774.0","200","390"
"crowd","K R AW D","false","-1.13934288E8","850","1300"
"in","IH N","false","-1.95127088E8","1300","1470"
"two","T UW","false","-2.23176048E8","1470","1700"
"distinct","D IH S T IH NG K T","false","-2.6345264E8","1700","2230"
"ways","W EY Z","false","-3.58427808E8","2230","2730"
"the","DH AH","false","-4.72551168E8","2920","3100"
"fruit","F R UW T","false","-5.24233504E8","3220","3530"
"of","AH V","false","-5.79971456E8","3530","3640"
"a","AH","false","-5.99515456E8","3640","3760"
"figg","F IH G","false","-6.2017152E8","3760","4060"
"tree","T R IY","false","-6.72126656E8","4060","4490"
"is","IH Z","false","-7.4763744E8","4490","4570"
"apple","AE P AH L","false","-7.73581184E8","4630","5040"
"shaped","SH EY P T","false","-8.44424704E8","5040","5340"

kdb/q: Query multiple handles with hopen

I would like to be able to query several handles at once, where the tables have the same format like:
handles: 8000,8001,8003
tables: foo
Want to do something like:
x:hopen `8000`8001`8003
x select from foo col1,col2
So i get rows from each foo table on each handle.
Is there a way to achieve this?
Thank you
Use 'each' to hopen each handle
q)h:hopen each 8000 8001 8002
q)h
476 480 484i
Use apply each-left to send the same query to each server
q)r:h#\:"select col1,col2 from foo"
q)r
+`col1`col2!(1 2;2 3)
+`col1`col2!(1 2;2 3)
+`col1`col2!(1 2;2 3)
Then you'll have to raze the result:
q)show res:raze r
col1 col2
---------
1 2
2 3
1 2
2 3
1 2
2 3
If you are not planning to reuse the handles, you can do
q)raze`::8000`::8001`::8003#\:"select from foo col1,col2"
Same as other answers, but more complicated using set (neg h) rather than get (h)
The cookbook/load-balancing helps in this example, too.
q)h:hopen each 8000 8001 8002
q)h
476 480 484i
q)r:(0#0i)!() /dictionary of handles and results
Set the callback for the response from the servers
q).z.ps:{#[`r;.z.w;:;x]}
Send a "set" query to each handle
q)(neg h)#\:({(neg .z.w)value"select col1,col2 from foo"};`)
Wait until all messages have a response
q)h .\:()
Finally, putting the result together
q)raze r h
The only advantage is concurrency.
As #AlexanderBelopolsky pointed out, don't forget
q)hclose each h

A-frame not loading materials

I understand this seems to be a common problem for beginners in A-Frame but I'm struggling to get a material onto a model in this. I can successfully get the model in but it always comes out grey with the console telling me:
"Material component properties are ignored when a .MTL is provided".
Grey...
I've made this in Blender with a simple glossy material and exported it as an .obj with relative paths (that's what is recommended yes?)
I'm not familiar with .mtl or .obj files but is there something else I have to do to them to get them working in A-Frame? Or is it an issue with the code or CORS?
html
<!DOCTYPE html>
<html>
<head>
<script src="https://aframe.io/releases/0.5.0/aframe.min.js"></script>
</head>
<body>
<a-scene stats>
<a-assets>
<!--Not working-->
<a-asset-item src="meh.obj" id="meh"> </a-asset-item>
<a-asset-item src="meh.mtl" id="mat"> </a-asset-item>
</a-assets>
<a-obj-model src="#meh" mtl="#mat" position ="0 0 5"> </a-obj-model>
</a-scene>
</body>
</html>
obj
# Blender v2.78 (sub 0) OBJ File: ''
# www.blender.org
mtllib meh.mtl
o Cube
v 1.083111 0.312460 -1.083111
v 1.083111 0.312460 1.083111
v -1.083111 0.312460 1.083110
v -1.083110 0.312460 -1.083111
v 1.083111 3.724101 -1.083110
vn 0.0000 -1.0000 0.0000
vn 1.0000 -0.0000 0.0000
vn -0.0000 0.5360 0.8442
vn -0.8442 0.5360 -0.0000
vn 0.0000 0.0000 -1.0000
usemtl RedGloss
s off
f 1//1 2//1 3//1 4//1
f 1//2 5//2 2//2
f 2//3 5//3 3//3
f 3//4 5//4 4//4
f 5//5 1//5 4//5
MTL:
# Blender MTL File: 'None'
# Material Count: 1
newmtl RedGloss
Ns 96.078431
Ka 1.000000 1.000000 1.000000
Kd 0.640000 0.640000 0.640000
Ks 0.500000 0.500000 0.500000
Ke 0.000000 0.000000 0.000000
Ni 1.000000
d 1.000000
illum 2
Appreciate the help :)
The three.js obj/mtl loaders don't work well, and MTL files generated by Maya ( maybe even blender ) are unreadable for the loaders, which are implemented in the a-frame material system. Multiple problems reported here, or here.
The easiest way, as I suggested in one of the links, is uploading the model to clara.io, and downloading an three JSON model, as it seem to work very good. Maybe You can try with Don McCurdy's loaders(though they seem to be more for the geometry, not for the material), but it seems, 3D formats are quite complicated. Since You do it exactly as suggested in the official docs, I wouldn't say there's something wrong with Your code.

Resources