I have a loop which produces a dataframe at every iteration.
DimensionAll=[]
for i in range(0,10):
###code here###
DimensionAll.append(MatrixDimension)
where MatrixDimension is as follows:
Cameroun Rwanda Niger Zambia Mali Angola Ethiopia
ECON 0.092811 0.088966 0.077843 0.101176 0.080969 0.045516 0.084101
FOOD 0.052086 0.035915 0.037474 0.025168 0.039382 0.015079 0.083499
ENV 0.018479 0.043677 0.004737 0.003744 0.009258 0.034044 0.010285
HEA 0.000061 0.029189 0.012335 0.001238 0.019010 0.007995 0.017359
PERS 0.056941 0.005222 0.048715 0.030879 0.070985 0.064726 0.023330
COM 0.000000 0.000000 0.000000 0.000000 0.009809 0.005251 0.099614
POL 0.025177 0.090846 0.005273 0.029481 0.001929 0.065365 0.034342
How can I ignore the column names when appending? is there a different way to append or concatenate the dataframes after each iteration (while keeping the column names at top without repetition)?
This piece of code gave me the answer i was looking for :
DimensionAll=[]
for i in range(0,10):
###code here###
DimensionAll.append(MatrixDimension)
DimensionAll= pd.concat(DimensionAll)
This is the .xyz file that I want to visualise using xyz2graph:
18
Atoms. File created from networkx graph by get_decomposition_calc_SRO1.py_edited_by_hand
Ga 0.0 0.0 0.0
In 1.59 0.917986928012 2.583
In 0.0 0.0 5.166
Ga 1.59 0.917986928012 7.749
Ga 0.0 0.0 10.332
Ga 1.59 0.917986928012 12.915
Ga 1.59 2.75396078403 0.0
In 3.18 3.67194771205 2.583
Ga 1.59 2.75396078403 5.166
In 3.18 3.67194771205 7.749
Ga 1.59 2.75396078403 10.332
Ga 3.18 3.67194771205 12.915
Ga 3.18 5.50792156807 0.0
In 4.77 6.42590849608 2.583
In 3.18 5.50792156807 5.166
Ga 4.77 6.42590849608 7.749
This is a still image of the output: notice how all of the nodes (labelled In and Ga are the SAME color).
The code being used to generate this image is copied directly from this website:
Exact code snippet shown here:
from xyz2graph import MolGraph, to_networkx_graph, to_plotly_figure
from plotly.offline import offline
# Create the MolGraph object
mg = MolGraph()
# Read the data from the .xyz file
mg.read_xyz('path/molecule.xyz')
# Create the Plotly figure object
fig = to_plotly_figure(mg)
# Plot the figure
offline.plot(fig)
How can I change the color of each atom in the chemical structure?
SOLUTION
Open the package and edit the cpk_colors dictionary in helpers.py to include colors for the atomic species
Why?
The problem was that there was no default color associated with either Indium and Gallium atoms in helpers.py
If we examine the source code there are two key files: xyz2graph.py and helpers.py
xyz2graph.py contains the covalent radii of the two atoms in question, but helpers.py has no associated colors in thecpk_colors dict.
As you correctly pointed out, elements that don't have an associated color are by default pink. You can import the cpk_colors dictionary and overwrite available colors or set colors for "pink" elements. For example:
from xyz2graph.helpers import cpk_colors
cpk_colors['Ga'] = 'green'
Result:
I have a lot of columns of numbers (for example, AAA, BBB, CCC, DDD and EEE) in Excel file.
I need to import these columns into Python and find correlation coefficient between every 2 columns.
Only show columns which have correlation coefficient from +0.5 to +1 and -0.5 to -1.
import pandas as pd
data = pd.read_excel('SO.xlsx')
df = pd.DataFrame(data)
df.corr()
Here is a really simple solution to this issue; I don't have your data so I've done it with sample data I found. Here we go:
import pandas as pd
data = pd.read_excel('https://global.oup.com/us/companion.websites/fdscontent/uscompanion/us/static/companion.websites/9780199734177/Example_1_rawdata.xls')
df = pd.DataFrame(data)
df.corr()
The output looks like this:
Hugs Comps PerAd SocAc ProAd ComSt PhyHlp Encour Tutor
Hugs 1.000000 0.666100 0.149995 0.616721 0.541132 0.653129 0.473344 0.549393 0.565627
Comps 0.666100 1.000000 0.247194 0.575720 0.509667 0.642069 0.424696 0.543826 0.487571
PerAd 0.149995 0.247194 1.000000 0.222337 0.081263 0.163510 0.090505 0.181000 0.120080
SocAc 0.616721 0.575720 0.222337 1.000000 0.409031 0.559579 0.338293 0.447923 0.348733
ProAd 0.541132 0.509667 0.081263 0.409031 1.000000 0.666905 0.733851 0.464976 0.754339
ComSt 0.653129 0.642069 0.163510 0.559579 0.666905 1.000000 0.595900 0.540038 0.671789
PhyHlp 0.473344 0.424696 0.090505 0.338293 0.733851 0.595900 1.000000 0.432037 0.717585
Encour 0.549393 0.543826 0.181000 0.447923 0.464976 0.540038 0.432037 1.000000 0.412042
Tutor 0.565627 0.487571 0.120080 0.348733 0.754339 0.671789 0.717585 0.412042 1.000000
If you add the following it will replace all the the values with a Pearson correlation below 0.5 with nulls:
df[df > 0.5]
I have run the example of Pocketsphinx Python and now I am facing the issue that I want to run a 60sec wav file for speech recognition in English and want as output
- the English translation AND
- at which second each word was mentioned.
Now, I do not know where to start to dome some research to get the required output. Could anyone please point me in the right direction??
ok, the open source tools like Kaldi automatically offers this:
https://americanarchivepb.wordpress.com/2017/12/04/dockerized-kaldi-speech-to-text-tool/
You need recognition with forced alignment. Here is an example for pocketsphinx:
pocketsphinx_continuous
-infile with.wav
-jsgf with-word.jsgf
-dict words.dict
-backtrace yes
-fsgusefiller no
-bestpath no
2>&1 > with-word.txt
Output:
==> with-word.txt <==
INFO: fsg_search.c(869): fsg 0.05 CPU 0.051 xRT
INFO: fsg_search.c(871): fsg 0.09 wall 0.084 xRT
INFO: pocketsphinx.c(1171): sil with sil (-2607)
word start end pprob ascr lscr lback
sil 3 77 1.000 -1602 0 1
with 78 102 1.000 -845 0 1
sil 103 107 1.000 -160 0 1
INFO: fsg_search.c(265): TOTAL fsg 0.05 CPU 0.051 xRT
INFO: fsg_search.c(268): TOTAL fsg 0.09 wall 0.085 xRT
sil with sil
For CMU Sphinx 4 you need the SpeechAligner class from Sphinx API. Here you'll find an implementation of simple aligner tool.
./align.sh sample.wav sample.txt 2>/dev/null
Output:
"it's","IH T S","false","0.0","170","200"
"a","AH","false","-5540774.0","200","390"
"crowd","K R AW D","false","-1.13934288E8","850","1300"
"in","IH N","false","-1.95127088E8","1300","1470"
"two","T UW","false","-2.23176048E8","1470","1700"
"distinct","D IH S T IH NG K T","false","-2.6345264E8","1700","2230"
"ways","W EY Z","false","-3.58427808E8","2230","2730"
"the","DH AH","false","-4.72551168E8","2920","3100"
"fruit","F R UW T","false","-5.24233504E8","3220","3530"
"of","AH V","false","-5.79971456E8","3530","3640"
"a","AH","false","-5.99515456E8","3640","3760"
"figg","F IH G","false","-6.2017152E8","3760","4060"
"tree","T R IY","false","-6.72126656E8","4060","4490"
"is","IH Z","false","-7.4763744E8","4490","4570"
"apple","AE P AH L","false","-7.73581184E8","4630","5040"
"shaped","SH EY P T","false","-8.44424704E8","5040","5340"
I experience IMO high delays with the Ruby script using Watir. Here's the problem description: I am testing AJAX based application and I wanted to avoid using of sleep to make sure page gets loaded:
class Page
attr_accessor :expected_elements
def loaded?
# code to make sure AJAX page is loaded
end
end
So instead of this:
def loaded?
# static delay sufficient to get page loaded
sleep(MAX_PAGE_LOAD_TIME)
true
end
I wanted to have something like this:
def loaded?
Watir::Wait.until(MAX_PAGE_LOAD_TIME) do
expected_elements.all? do |element|
element.present?
end
end
end
The problem is the evaluation of the block takes too long. The more elements I check for presence the higher this delay gets. I experienced roughly this delay for one iteration:
Firefox -> 130ms
IE -> 615ms
Chrome -> 115 ms
So to check if N elements are present I get N times corresponding delay... Well the consequence is that eventhough MAX_PAGE_LOAD_TIME expires the Watir::Wait::TimeoutError is not thrown because block evaluation has not been finished yet... So I ended up in the situation where the check for elements presence introduces higher delay than the static delay which is sufficient enough to get page loaded.. I tried to improve performance by locating elements by xpath, but the performance gain was not significant..
What am I doing wrong? Is there a way to speed-up execution time for present? method?? Do these delays correspond with your experience - or are they high?
I checked if the problem could be in the browser-server communication, but here the delays are very low.. I got 100ms time difference for the server response including backend DB request. Of course it takes some time to render page based on this response, but for sure it does not take seconds.
My configuration:
- Win7 OS,
- Firefox 17.0.1
- IE 8.0.7601.17514 with IEDriverServer_x64_2.26.2
- Chrome 23.0.1271.97 m with chromedriver_win_23.0.1240.0,
- Ruby 1.9.3p125,
- watir-webdriver (0.6.1),
- selenium-webdriver (2.27.2)
Thank you for your help!
Based on the discussion I post a sample of benchmarking code:
Benchmark.bm do |x|
x.report("text") do
b.span(:text => "Search").present?
end
end
Benchmark.bm do |x|
x.report("xpath") do
b.span(:xpath => "/html/body/div/div/div[2]/div[2]/div/div/div/div/div/div/div/div[2]/div/div/div[2]/div/div/div/div/div/div[2]/div/div/div/div/div/div/div/div[2]/div/div/div[2]/div/div/div/div/div/div[2]/div/div/div/div/div/span/span").present?
end
end
user system total real
text 0.000000 0.000000 0.000000 ( 0.140405)
xpath 0.000000 0.000000 0.000000 ( 0.120005)
Additional benchmarking results:
container_xpath = "/html/body/div/div/div[2]/div[2]/div/div/div/div/div/div/div/div[2]/div/div/div[2]/div/div/div/div/div/div[2]/div/div/div/div/div/div/div/div/div/div/div/div/div/div/div[2]/div/div/div/div[2]/div/div/div"
Benchmark.bm do |x|
x.report("cntnr") do
#c = b.div(:xpath => container_xpath)
#c.present?
end
end
Benchmark.bm do |x|
x.report("lb1") do
#c.div(:xpath => "div[1]/div/div").present?
##c.div(:text => "Company:").present?
end
end
Benchmark.bm do |x|
x.report("lb2") do
#c.div(:xpath => "div[3]/div/div").present?
##c.div(:text => "Contact person:").present?
end
end
Benchmark.bm do |x|
x.report("lb3") do
#c.div(:xpath => "div[5]/div/div").present?
##c.div(:text => "Address:").present?
end
end
And the results were:
Results for container reference and relative xpath:
user system total real
cntnr 0.000000 0.000000 0.000000 ( 0.156007)
lb1 0.000000 0.000000 0.000000 ( 0.374417)
lb2 0.000000 0.000000 0.000000 ( 0.358816)
lb3 0.000000 0.000000 0.000000 ( 0.358816)
Results for container reference and div's text:
user system total real
cntnr 0.000000 0.000000 0.000000 ( 0.140402)
lb1 0.000000 0.000000 0.000000 ( 0.358807)
lb2 0.000000 0.000000 0.000000 ( 0.358807)
lb3 0.000000 0.000000 0.000000 ( 0.374407)
When absolute xpaths were used:
container_xpath = "/html/body/div/div/div[2]/div[2]/div/div/div/div/div/div/div/div[2]/div/div/div[2]/div/div/div/div/div/div[2]/div/div/div/div/div/div/div/div/div/div/div/div/div/div/div[2]/div/div/div/div[2]/div/div/div"
Benchmark.bm do |x|
x.report("cntnr") do
#c = b.div(:xpath => container_xpath)
#c.present?
end
end
lb1_xpath = container_xpath + "/div[1]/div/div"
Benchmark.bm do |x|
x.report("lb1_x") do
b.div(:xpath => lb1_xpath).present?
end
end
lb2_xpath = container_xpath + "/div[3]/div/div"
Benchmark.bm do |x|
x.report("lb2_x") do
b.div(:xpath => lb2_xpath).present?
end
end
lb3_xpath = container_xpath + "/div[5]/div/div"
Benchmark.bm do |x|
x.report("lb3_x") do
b.div(:xpath => lb3_xpath).present?
end
end
Results were:
user system total real
cntnr 0.000000 0.000000 0.000000 ( 0.140404)
lb1_x 0.000000 0.000000 0.000000 ( 0.124804)
lb2_x 0.000000 0.000000 0.000000 ( 0.156005)
lb3_x 0.000000 0.000000 0.000000 ( 0.140405)
Okay, this answer assumes your site is using jquery. If it's not, you'll have to figure out the library in use and modify the method accordingly...
Write a method that waits for the Ajax calls to finish...
def wait_for_ajax(timeout = 10)
timeout.times do
return true if browser.execute_script('return jQuery.active').to_i == 0
sleep(1)
end
raise Watir::Wait::TimeoutError, "Timeout of #{timeout} seconds exceeded on waiting for Ajax."
end
Call that method when you first load the page you're testing. Then iterate through your expected elements to see if they're present (if you have an array of Watir elements to check, make sure you use .each with it, not .all? as you have in your code there).