I have a real matrix into a prova3hm.txt file, where each row is separated by a blank line. For example:
0.907 0.796 0.796 0.794 0.794 0.793 0.793 0.793 0.793 0.793 0.792 0.792 0.792 0.792 0.792 0.792 0.792 0.792 0.792 0.792 0.791 0.791
0.791 0.791 0.791 0.791 0.791 0.791 0.791 0.791 0.791 0.79 0.79 0.79 0.789 0.789 0.789 0.789 0.789 0.789 0.789 0.789 0.789 0.789
0.789 0.788 0.788 0.788 0.788 0.788 0.788 0.788 0.788 0.787 0.787 0.787 0.787 0.787 0.787 0.787 0.784 0.658 0.658 0.658 0.657 0.657
0.657 0.656 0.656 0.656 0.656 0.656 0.656 0.656 0.656 0.655 0.655 0.655 0.655 0.655 0.655 0.654 0.654 0.654 0.653 0.653 0.653 0.653
0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.652 0.652 0.652 0.652 0.652 0.652 0.652
0.652 0.652 0.652 0.652 0.652 0.652 0.652 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651
0.651 0.651 0.651 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.649 0.649
0.649 0.649 0.649 0.649 0.648 0.648 0.648 0.646 0.242 0.241 0.24 0.239 0.239 0.239 0.239 0.239 0.239 0.238 0.238 0.238 0.238 0.238
0.238 0.238 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.236 0.236 0.236 0.236 0.236 0.236 0.236
0.236 0.236 0.236 0.236 0.236 0.236 0.236 0.236 0.236 0.236 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235
0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235
0.234 0.234 0.234 0.234 0.233 0.233 0.233 0.233 0.233 0.233 0.233 0.233 0.233 0.233 0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.232
0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.231 0.231 0.231 0.231 0.23 0.23 0.23 0.23 0.23 0.23
0.229 0.229 0.165 0.164 0.163 0.162 0.162 0.162 0.162 0.162 0.16 0.16 0.16 0.16 0.16 0.159 0.159 0.159 0.159 0.159 0.159 0.159
0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158
0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157
I need to get a heat map out of this file, with Gnuplot.
I have been trying to use the pm3d package and the splot command, this way:
gnuplot> set pm3d map
gnuplot> splot 'prova3hm.txt' matrix
At this point, I was supposing to get my heat map, that is something like this:
(source: pnas.org)
But, instead, I got an empty image:
Why is my output image empty?
What do I do wrong?
The following script works fine with gnuplot version 4.6.3:
reset
set terminal pngcairo
set output 'test.png'
set autoscale fix
plot 'file.txt' matrix with image
The result is:
To complete is, here the test data:
0.907 0.796 0.796 0.794 0.794 0.793 0.793 0.793 0.793 0.793 0.792 0.792 0.792 0.792 0.792 0.792 0.792 0.792 0.792 0.792 0.791 0.791
0.791 0.791 0.791 0.791 0.791 0.791 0.791 0.791 0.791 0.79 0.79 0.79 0.789 0.789 0.789 0.789 0.789 0.789 0.789 0.789 0.789 0.789
0.789 0.788 0.788 0.788 0.788 0.788 0.788 0.788 0.788 0.787 0.787 0.787 0.787 0.787 0.787 0.787 0.784 0.658 0.658 0.658 0.657 0.657
0.657 0.656 0.656 0.656 0.656 0.656 0.656 0.656 0.656 0.655 0.655 0.655 0.655 0.655 0.655 0.654 0.654 0.654 0.653 0.653 0.653 0.653
0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.653 0.652 0.652 0.652 0.652 0.652 0.652 0.652
0.652 0.652 0.652 0.652 0.652 0.652 0.652 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651 0.651
0.651 0.651 0.651 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.649 0.649
0.649 0.649 0.649 0.649 0.648 0.648 0.648 0.646 0.242 0.241 0.24 0.239 0.239 0.239 0.239 0.239 0.239 0.238 0.238 0.238 0.238 0.238
0.238 0.238 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.237 0.236 0.236 0.236 0.236 0.236 0.236 0.236
0.236 0.236 0.236 0.236 0.236 0.236 0.236 0.236 0.236 0.236 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235
0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235 0.235
0.234 0.234 0.234 0.234 0.233 0.233 0.233 0.233 0.233 0.233 0.233 0.233 0.233 0.233 0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.232
0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.232 0.231 0.231 0.231 0.231 0.23 0.23 0.23 0.23 0.23 0.23
0.229 0.229 0.165 0.164 0.163 0.162 0.162 0.162 0.162 0.162 0.16 0.16 0.16 0.16 0.16 0.159 0.159 0.159 0.159 0.159 0.159 0.159
0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158 0.158
0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157 0.157
Related
I am running cProfile on my python script in order to see where i can improve performance and using snakeviz to visualize. The results are pretty vague however; how do I interpret them? Here are the first few lines of it:
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
242059 0.626 0.000 0.914 0.000 pulp.py:585(__init__)
1 0.413 0.413 0.413 0.413 {built-in method _winapi.WaitForSingleObject}
343978/302 0.293 0.000 0.557 0.002 pulp.py:750(addInPlace)
4159617 0.288 0.000 0.288 0.000 pulp.py:165(__hash__)
112 0.282 0.003 0.282 0.003 {method 'read' of '_ssl._SSLSocket' objects}
1913398 0.172 0.000 0.245 0.000 {built-in method builtins.isinstance}
1 0.171 0.171 0.185 0.185 Betfair_Run_Sheet.pyx:243(betfairFinalArray)
377866 0.168 0.000 0.293 0.000 pulp.py:637(addterm)
2255 0.161 0.000 0.161 0.000 mps_lp.py:249(<listcomp>)
1 0.148 0.148 0.570 0.570 mps_lp.py:174(writeMPS)
117214 0.139 0.000 0.444 0.000 pulp.py:820(__mul__)
2 0.136 0.068 0.196 0.098 pulp.py:1465(variablesDict)
5 0.135 0.027 0.135 0.027 {method 'do_handshake' of '_ssl._SSLSocket' objects}
427 0.111 0.000 0.129 0.000 <frozen importlib._bootstrap_external>:914(get_data)
71 0.108 0.002 0.108 0.002 {built-in method _imp.create_dynamic}
2093 0.102 0.000 0.102 0.000 {built-in method nt.stat}
I am using Pulp so aware that takes the lion's share of the time, but the specifics of the setup is not clear from the above, e.g. for the first line of output it seems to be alluding to a line 585 in my script but that is not where I have called or set up the PULP part in it at all.
Same with the <listcomp> 9th one down, there is no list comprehension on that line of my script.
Other things like {method 'do_handshake' of '_ssl._SSLSocket' objects} I don't have a clue what they mean.
The data are organized where the first column are year and next are monthly average, I need plot x-->month and y--> monthly average and the data base is organized first column= year next columns are monthly mean (12 values) corresponding to year, like that:
1871 -0.107 0.004 -0.503 -0.650 -0.379 0.025 0.317 0.027 -0.732 -0.685 0.037 0.566
1872 0.376 -0.241 -0.904 -1.019 0.367 0.282 -0.061 0.597 0.779 0.818 1.070 1.203
1873 0.831 0.762 0.379 -0.028 0.014 0.349 0.189 0.428 -0.170 0.643 0.859 0.317
1874 0.063 0.125 -0.068 -0.124 0.365 0.535 0.693 1.298 0.554 0.566 0.889 0.185
1875 -0.369 -0.764 -1.238 0.111 0.683 0.696 0.505 1.008 1.210 0.945 -0.307 -0.184
Similar to this graph:
I have a data file:
0.4 -0.97
0.41 -0.96
0.42 -0.95
0.43 -0.93
0.44 -0.92
0.45 -0.91
0.46 -0.90
0.47 -0.88
0.48 -0.87
0.49 -0.86
0.5 -0.84
0.51 -0.83
0.52 -0.82
0.53 -0.81
0.54 -0.80
0.55 -0.78
0.56 -0.77
0.57 -0.76
0.58 -0.74
0.59 -0.73
0.6 -0.72
0.61 -0.71
0.62 -0.70
0.63 -0.69
0.64 -0.67
0.65 -0.66
0.66 -0.65
0.67 -0.64
0.68 -0.62
0.69 -0.61
0.7 -0.60
0.71 -0.59
0.72 -0.58
0.73 -0.56
0.74 -0.55
0.75 -0.54
0.76 -0.53
0.77 -0.52
0.78 -0.51
0.79 -0.50
0.8 -0.49
0.81 -0.47
0.82 -0.47
0.83 -0.46
0.84 -0.44
0.85 -0.43
0.86 -0.42
0.87 -0.41
0.88 -0.40
0.89 -0.39
0.9 -0.38
0.91 -0.49
0.92 -0.48
0.93 -0.47
0.94 -0.46
0.95 -0.44
0.96 -0.43
0.97 -0.42
0.98 -0.41
0.99 -0.40
1.0 -0.39
1.01 -0.38
1.02 -0.37
1.03 -0.36
1.04 -0.35
1.05 -0.34
1.06 -0.33
1.07 -0.32
1.08 -0.31
1.09 -0.30
1.1 -0.30
1.11 -0.29
1.12 -0.28
1.13 -0.27
1.14 -0.26
1.15 -0.25
1.16 -0.24
1.17 -0.24
1.18 -0.23
1.19 -0.22
1.2 -0.21
1.21 -0.20
1.22 -0.20
1.23 -0.19
1.24 -0.18
1.25 -0.17
1.26 -0.17
1.27 -0.16
1.28 -0.15
1.29 -0.14
1.3 -0.13
1.31 -0.12
1.32 -0.11
1.33 -0.11
1.34 -0.10
1.35 -0.09
1.36 -0.08
1.37 -0.08
1.38 -0.07
1.39 -0.06
1.4 -0.05
1.41 -0.04
1.42 -0.03
1.43 -0.03
1.44 -0.02
1.45 -0.01
1.46 -0.01
1.47 -0.00
1.48 0.00
1.49 0.01
1.5 0.02
1.51 0.03
1.52 0.04
1.53 0.04
1.54 0.05
1.55 0.06
1.56 0.06
1.57 0.07
1.58 0.08
1.59 0.08
1.6 0.09
1.61 0.09
1.62 0.10
1.63 0.10
1.64 0.10
1.65 0.11
1.66 0.11
1.67 0.12
1.68 0.12
1.69 0.13
1.7 0.14
1.71 0.14
1.72 0.14
1.73 0.15
1.74 0.15
1.75 0.16
1.76 0.16
1.77 0.17
1.78 0.17
1.79 0.18
1.8 0.19
1.81 0.20
1.82 0.20
1.83 0.21
1.84 0.21
1.85 0.22
1.86 0.22
1.87 0.23
1.88 0.24
1.89 0.24
1.9 0.25
1.91 0.25
1.92 0.26
1.93 0.26
1.94 0.26
1.95 0.27
1.96 0.28
1.97 0.28
1.98 0.28
1.99 0.29
2.0 0.29
2.01 0.29
2.02 0.29
2.03 0.30
2.04 0.30
2.05 0.30
2.06 0.31
2.07 0.32
2.08 0.32
2.09 0.33
2.1 0.33
2.11 0.33
2.12 0.34
2.13 0.34
2.14 0.34
2.15 0.35
2.16 0.35
2.17 0.36
2.18 0.36
2.19 0.36
2.2 0.37
2.21 0.37
2.22 0.37
2.23 0.38
2.24 0.38
2.25 0.38
2.26 0.38
2.27 0.39
2.28 0.39
2.29 0.39
2.3 0.40
2.31 0.40
2.32 0.40
2.33 0.40
2.34 0.41
2.35 0.41
2.36 0.42
2.37 0.42
2.38 0.43
2.39 0.43
2.4 0.43
2.41 0.43
2.42 0.44
2.43 0.44
2.44 0.44
2.45 0.44
2.46 0.45
2.47 0.45
2.48 0.45
2.49 0.45
2.5 0.46
2.51 0.46
2.52 0.46
2.53 0.47
2.54 0.47
2.55 0.47
2.56 0.48
2.57 0.48
2.58 0.49
2.59 0.36
2.6 0.36
2.61 0.36
2.62 0.36
2.63 0.37
2.64 0.37
2.65 0.37
2.66 0.37
2.67 0.38
2.68 0.38
2.69 0.38
2.7 0.38
2.71 0.38
2.72 0.38
2.73 0.38
2.74 0.38
2.75 0.38
2.76 0.38
2.77 0.38
2.78 0.38
2.79 0.39
2.8 0.39
2.81 0.39
2.82 0.39
2.83 0.39
2.84 0.39
2.85 0.28
2.86 0.28
2.87 0.28
2.88 0.28
2.89 0.28
2.9 0.28
2.91 0.28
2.92 0.28
2.93 0.29
2.94 0.29
2.95 0.29
2.96 0.29
2.97 0.29
2.98 0.29
2.99 0.29
3.0 0.19
3.01 0.19
3.02 0.19
3.03 0.19
3.04 0.19
3.05 0.19
3.06 0.19
3.07 0.19
3.08 0.20
3.09 0.20
3.1 0.20
3.11 0.20
3.12 0.20
3.13 0.20
3.14 0.20
3.15 0.20
3.16 0.20
3.17 0.20
3.18 0.21
3.19 0.21
3.2 0.21
3.21 0.21
3.22 0.21
3.23 0.21
3.24 0.21
3.25 0.21
3.26 0.21
3.27 0.21
3.28 0.21
3.29 0.21
3.3 0.21
3.31 0.21
3.32 0.21
3.33 0.21
3.34 0.21
3.35 0.21
3.36 0.21
3.37 0.22
3.38 0.22
3.39 0.22
3.4 0.22
3.41 0.22
3.42 0.22
3.43 0.22
3.44 0.22
3.45 0.22
3.46 0.22
3.47 0.22
3.48 0.22
3.49 0.22
3.5 0.22
3.51 0.23
3.52 0.23
3.53 0.23
3.54 0.23
3.55 0.23
3.56 0.13
3.57 0.13
3.58 0.13
3.59 0.13
3.6 0.13
3.61 0.13
3.62 0.13
3.63 0.13
3.64 0.13
3.65 0.13
3.66 0.13
3.67 0.13
3.68 0.13
3.69 0.13
3.7 0.13
3.71 0.13
3.72 0.14
3.73 0.14
3.74 0.14
3.75 0.14
3.76 0.05
3.77 0.05
3.78 0.05
3.79 0.05
3.8 0.05
3.81 -0.04
3.82 -0.04
3.83 -0.04
3.84 -0.04
3.85 -0.04
3.86 -0.04
3.87 -0.04
3.88 -0.04
3.89 -0.04
3.9 -0.04
3.91 -0.04
3.92 -0.04
3.93 -0.04
3.94 -0.04
3.95 -0.12
3.96 -0.12
3.97 -0.12
3.98 -0.12
3.99 -0.12
4.0 -0.12
4.01 -0.12
4.02 -0.12
4.03 -0.12
4.04 -0.12
4.05 -0.19
4.06 -0.19
4.07 -0.19
4.08 -0.19
4.09 -0.19
4.1 -0.19
4.11 -0.19
4.12 -0.41
4.13 -0.41
4.14 -0.41
4.15 -0.47
4.16 -0.47
4.17 -0.47
4.18 -0.47
4.19 -0.47
4.2 -0.47
4.21 -0.47
4.22 -0.54
4.23 -0.54
4.24 -0.60
4.25 -0.65
4.26 -0.65
4.27 -0.65
4.28 -0.65
4.29 -0.65
4.3 -0.65
4.31 -0.65
4.32 -0.65
4.33 -0.65
4.34 -0.65
4.35 -0.65
4.36 -0.65
4.37 -0.65
4.38 -0.71
4.39 -0.71
4.4 -0.71
4.41 -0.71
4.42 -0.71
4.43 -0.71
4.44 -0.71
4.45 -0.71
4.46 -0.71
4.47 -0.71
4.48 -0.71
4.49 -0.71
4.5 -0.71
4.51 -0.71
4.52 -0.71
4.53 -0.76
4.54 -0.76
4.55 -0.82
4.56 -0.82
4.57 -0.87
4.58 -0.87
4.59 -0.87
4.6 -0.87
4.61 -0.92
4.62 -0.97
4.63 -1.06
4.64 -1.06
4.65 -1.06
4.66 -1.06
4.67 -1.06
4.68 -1.06
4.69 -1.06
4.7 -1.06
4.71 -1.06
4.72 -1.06
4.73 -1.06
4.74 -1.11
4.75 -1.11
4.76 -1.11
4.77 -1.11
4.78 -1.11
4.79 -1.11
4.8 -1.11
4.81 -1.11
4.82 -1.11
4.83 -1.11
4.84 -1.11
4.85 -1.15
4.86 -1.15
4.87 -1.15
4.88 -1.15
I wish to create a "well" smoother curve, so i use
plot "for_gnuplot" lw 3 w l sm b title ""
I get the following image:
This is very nice, but i wish to mark the maximum in some way. I know that with sm b the maximum is not the real maximum of the plot, but i dont know how to mark this new maximum value.
Thanks
You can write the (x,y) data of the smoothed plot to a temporary file, do some statistics on this file, and plot the results:
# Generate the data for the smooth plot
set samples 1000
set table "temp.dat"
plot "for_gnuplot" lw 3 w l sm b title "1"
unset table
# Get maximum values and indices of maximum values:
# A_max_y, A_index_max_y, B_max_y, B_index_max_y
stats "for_gnuplot" prefix "A"
stats "temp.dat" using 1:2 prefix "B"
# Calculate positions from indices.
# We need the x-value (first column) at B_index_max_y. We know that the first
# column of "temp.dat" consists of equidistant x-values. So we just fit a
# linear function to map from index to position. (Could be done analytically.)
pos_from_index(x) = a*x + b
fit pos_from_index(x) "for_gnuplot" using 0:1 via a, b
A_xvalue_max_y = pos_from_index(A_index_max_y)
fit pos_from_index(x) "temp.dat" using 0:1 via a, b
B_xvalue_max_y = pos_from_index(B_index_max_y)
# Make some arrows to indicate maximal values
set arrow 1 from A_xvalue_max_y, graph 0.99 to A_xvalue_max_y, A_max_y fill lw 2
set arrow 2 from B_xvalue_max_y, graph 0.8 to B_xvalue_max_y, B_max_y fill lw 2
set label 1 at A_xvalue_max_y, graph 0.99 "max raw" offset 0.2, -0.3
set label 2 at B_xvalue_max_y, graph 0.8 "max smooth" center offset 0, -0.4
# Finally plot the graphs
set terminal png
set output "graph.png"
plot "for_gnuplot" lw 2 w l title "raw" ,\
"for_gnuplot" lw 2 w l sm b title "smooth"
This produces the following output:
PS: I would be interested if there is a more direct way to access a value from a file at a specific index.
Here is a link: http://www.phyast.pitt.edu/~zov1/gnuplot/html/statistics.html
Scroll to "Determining the position of the minimum and maximum".
I have 1106 columns and 6689 rows to be filled in a sparse matrix that is filled by reading a csv file using DictReader from the csv library.
There are some bad columns so I want to remove those (currently trying to identify which ones are the bad columns).
Anyway here's the code:
def read_annotations(filehandle):
with open(filehandle, 'r', encoding = 'utf-8') as csvfile:
reader = csv.DictReader(csvfile)
workers = defaultdict(list)
ids = []
for row in reader:
# CF annotators given a choice to choose link on their request, We treat it as neither (so do expert raters
if row['which_form_of_hate_speech_is_this'].lower() == 'link':
row['which_form_of_hate_speech_is_this'] = 'neither'
# Create a dictionary containing each annotation by a worker,
# save their trust score, their annotation, the id of the tweet and the tweet in a tuple
workers[row['_worker_id']].append((row['_trust'], row['which_form_of_hate_speech_is_this'], row['id'], row['tweet']))
return workers, set(ids)
def annotation_writer(workers, ids):
""" Writes a annotations to file where each column is a worker and each row is a tweet ID
:param workers: Dictionary containing worker ID as key, trust, annotation, tweet id and tweet as tuple
"""
# Create empty dataframe with tweet IDS as indexes and workers as columns
df = pd.DataFrame(index = ids, columns= list(workers.keys()))
# Go through each worker
bad_annotators = [683, 691, 694, 691]
for i, w_idx in enumerate(workers):
# Go through each list item
if i <= 693:
if i in bad_annotators:
print(w_idx)
continue
for item in workers[w_idx]:
_, label, idx, tweet = item
# Set the label they have chosen for the item at index: TweetID and row worker_id
df.set_value(idx, w_idx, label)
return df
The profiling output ordered by cumulative time:
tottime percall cumtime percall filename:lineno(function)
466/1 0.020 0.000 361.865 361.865 {built-in method builtins.exec}
1 0.000 0.000 361.865 361.865 annotator.py:1(<module>)
1 93.955 93.955 360.789 360.789 annotator.py:24(annotation_writer)
20923 0.143 0.000 266.759 0.013 frame.py:1840(set_value)
6594 0.088 0.000 265.032 0.040 indexing.py:126(__setitem__)
6594 0.478 0.000 263.543 0.040 indexing.py:224(_setitem_with_indexer)
6594 0.077 0.000 254.250 0.039 frame.py:2743(reindex_axis)
6594 0.071 0.000 254.173 0.039 generic.py:2307(reindex_axis)
6594 0.144 0.000 250.682 0.038 generic.py:2320(_reindex_with_indexers)
6594 0.095 0.000 250.327 0.038 internals.py:3560(reindex_indexer)
6594 0.029 0.000 249.124 0.038 internals.py:3595(<listcomp>)
6594 0.155 0.000 249.096 0.038 internals.py:975(take_nd)
6594 0.239 0.000 248.633 0.038 algorithms.py:840(take_nd)
34098 140.266 0.004 140.266 0.004 {built-in method numpy.core.multiarray.empty}
6594 107.927 0.016 107.927 0.016 {pandas.algos.take_2d_axis1_object_object}
40942 0.136 0.000 4.503 0.000 base.py:1915(get_loc)
54130 4.213 0.000 4.242 0.000 {method 'get_loc' of 'pandas.index.IndexEngine' objects}
6594 0.052 0.000 3.299 0.001 base.py:2295(reindex)
6594 0.079 0.000 3.099 0.000 base.py:2028(get_indexer)
6594 2.871 0.000 2.923 0.000 {method 'get_indexer' of 'pandas.index.IndexEngine' objects}
6594 0.075 0.000 2.321 0.000 base.py:3011(insert)
13189 0.112 0.000 1.707 0.000 internals.py:2578(__init__)
19787/13191 0.368 0.000 1.620 0.000 base.py:124(__new__)
6594 0.027 0.000 1.519 0.000 internals.py:2916(setitem)
6594 0.108 0.000 1.492 0.000 internals.py:2811(apply)
20923 0.084 0.000 1.434 0.000 generic.py:1345(_get_item_cache)
6594 0.034 0.000 1.359 0.000 indexing.py:101(_get_setitem_indexer)
6594 0.048 0.000 1.281 0.000 indexing.py:163(_convert_tuple)
13188 0.105 0.000 1.229 0.000 indexing.py:1102(_convert_to_indexer)
13189 0.456 0.000 1.155 0.000 internals.py:2674(_rebuild_blknos_and_blklocs)
6594 0.043 0.000 1.008 0.000 base.py:522(_coerce_scalar_to_index)
13192 0.869 0.000 0.895 0.000 {pandas.lib.infer_dtype}
6594 0.031 0.000 0.761 0.000 base.py:354(_shallow_copy_with_infer)
6594 0.130 0.000 0.720 0.000 internals.py:628(setitem)
7283 0.082 0.000 0.694 0.000 internals.py:3283(get)
20473 0.167 0.000 0.669 0.000 internals.py:2482(make_block)
13188 0.020 0.000 0.639 0.000 base.py:950(is_integer)
6595 0.010 0.000 0.619 0.000 base.py:1172(inferred_type)
571/2 0.004 0.000 0.610 0.305 <frozen importlib._bootstrap>:966(_find_and_load)
571/2 0.003 0.000 0.610 0.305 <frozen importlib._bootstrap>:939(_find_and_load_unlocked)
452/2 0.003 0.000 0.610 0.305 <frozen importlib._bootstrap>:659(_load_unlocked)
387/2 0.002 0.000 0.610 0.305 <frozen importlib._bootstrap_external>:656(exec_module)
599/2 0.001 0.000 0.609 0.305 <frozen importlib._bootstrap>:214(_call_with_frames_removed)
2 0.000 0.000 0.609 0.304 __init__.py:5(<module>)
989631/989629 0.370 0.000 0.603 0.000 {built-in method builtins.isinstance}
7283 0.047 0.000 0.593 0.000 frame.py:2331(_box_item_values)
7283 0.031 0.000 0.480 0.000 frame.py:2338(_box_col_values)
1 0.068 0.068 0.465 0.465 annotator.py:7(read_annotations)
13877 0.069 0.000 0.463 0.000 internals.py:183(make_block_same_class)
7283 0.038 0.000 0.449 0.000 series.py:236(from_array)
432/41 0.001 0.000 0.440 0.011 {built-in method builtins.__import__}
7283 0.133 0.000 0.438 0.000 internals.py:3312(iget)
31512 0.141 0.000 0.389 0.000 csv.py:106(__next__)
20472 0.176 0.000 0.381 0.000 internals.py:1657(__init__)
7284 0.054 0.000 0.369 0.000 series.py:120(__init__)
70099/69243 0.051 0.000 0.367 0.000 <frozen importlib._bootstrap>:996(_handle_fromlist)
6594 0.350 0.000 0.350 0.000 {built-in method numpy.core.multiarray.concatenate}
19787 0.116 0.000 0.329 0.000 common.py:1380(_asarray_tuplesafe)
6594 0.032 0.000 0.310 0.000 internals.py:171(make_block)
533564/425149 0.221 0.000 0.290 0.000 {built-in method builtins.len}
32973 0.122 0.000 0.282 0.000 internals.py:2619(shape)
232315 0.230 0.000 0.270 0.000 {built-in method builtins.getattr}
41781 0.055 0.000 0.267 0.000 common.py:1710(is_datetimetz)
54821 0.182 0.000 0.260 0.000 generic.py:2674(__setattr__)
40673 0.061 0.000 0.251 0.000 numeric.py:414(asarray)
58149 0.225 0.000 0.238 0.000 {built-in method numpy.core.multiarray.array}
126058 0.096 0.000 0.231 0.000 generic.py:7(_check)
103351 0.124 0.000 0.230 0.000 dtypes.py:74(is_dtype)
61570 0.068 0.000 0.227 0.000 common.py:1736(is_categorical_dtype)
278263/278240 0.222 0.000 0.223 0.000 {built-in method builtins.hasattr}
31532 0.214 0.000 0.222 0.000 {built-in method builtins.next}
13188 0.043 0.000 0.220 0.000 indexing.py:183(_convert_scalar_indexer)
15213 0.030 0.000 0.208 0.000 {method 'any' of 'numpy.ndarray' objects}
52752 0.059 0.000 0.206 0.000 generic.py:333(_get_axis)
6595 0.054 0.000 0.201 0.000 internals.py:2799(_verify_integrity)
27 0.001 0.000 0.195 0.007 __init__.py:1(<module>)
20473 0.097 0.000 0.194 0.000 internals.py:77(__init__)
6595 0.060 0.000 0.194 0.000 frame.py:210(__init__)
19782 0.042 0.000 0.179 0.000 generic.py:2713(_protect_consolidate)
15213 0.015 0.000 0.178 0.000 _methods.py:37(_any)
6594 0.013 0.000 0.171 0.000 internals.py:555(_try_coerce_and_cast_result)
13188 0.046 0.000 0.168 0.000 generic.py:1407(_maybe_update_cacher)
1 0.000 0.000 0.167 0.167 api.py:5(<module>)
13189 0.069 0.000 0.164 0.000 internals.py:3004(_consolidate_check)
15213 0.163 0.000 0.163 0.000 {method 'reduce' of 'numpy.ufunc' objects}
98919 0.034 0.000 0.159 0.000 internals.py:2621(<genexpr>)
7284 0.032 0.000 0.156 0.000 series.py:270(_set_axis)
6594 0.029 0.000 0.156 0.000 internals.py:510(_try_cast_result)
1 0.000 0.000 0.155 0.155 groupby.py:1(<module>)
13188 0.023 0.000 0.150 0.000 generic.py:2723(_consolidate_inplace)
1 0.000 0.000 0.145 0.145 frame.py:10(<module>)
6594 0.047 0.000 0.141 0.000 common.py:527(_maybe_promote)
15402 0.024 0.000 0.138 0.000 common.py:1731(is_categorical)
13188 0.046 0.000 0.136 0.000 base.py:972(_convert_scalar_indexer)
13191 0.073 0.000 0.131 0.000 base.py:309(_simple_new)
59346 0.094 0.000 0.128 0.000 generic.py:320(_get_axis_name)
1 0.000 0.000 0.126 0.126 __init__.py:106(<module>)
2 0.000 0.000 0.125 0.063 __init__.py:9(<module>)
1514/1502 0.050 0.000 0.125 0.000 {built-in method builtins.__build_class__}
2 0.000 0.000 0.123 0.062 __init__.py:15(<module>)
54132 0.091 0.000 0.117 0.000 {pandas.lib.values_from_object}
6595 0.029 0.000 0.115 0.000 base.py:1507(equals)
41781 0.037 0.000 0.108 0.000 common.py:1575(is_datetime64tz_dtype)
6595 0.033 0.000 0.106 0.000 base.py:1180(is_all_dates)
52760 0.057 0.000 0.104 0.000 base.py:440(values)
1 0.000 0.000 0.104 0.104 series.py:3(<module>)
1 0.000 0.000 0.102 0.102 add_newdocs.py:10(<module>)
387 0.004 0.000 0.102 0.000 <frozen importlib._bootstrap_external>:726(get_code)
1 0.000 0.000 0.100 0.100 config_init.py:11(<module>)
107722 0.069 0.000 0.097 0.000 base.py:409(__len__)
13188 0.038 0.000 0.095 0.000 generic.py:2726(f)
6594 0.025 0.000 0.093 0.000 indexing.py:1860(maybe_convert_ix)
1 0.000 0.000 0.093 0.093 plotting.py:3(<module>)
1 0.000 0.000 0.090 0.090 converter.py:1(<module>)
1 0.000 0.000 0.089 0.089 format.py:2(<module>)
13189 0.022 0.000 0.089 0.000 internals.py:3005(<listcomp>)
27482 0.087 0.000 0.087 0.000 {method 'fill' of 'numpy.ndarray' objects}
1 0.000 0.000 0.087 0.087 type_check.py:3(<module>)
6596 0.085 0.000 0.085 0.000 {built-in method pandas.lib.list_to_object_array}
435/433 0.001 0.000 0.084 0.000 <frozen importlib._bootstrap>:570(module_from_spec)
20923 0.082 0.000 0.082 0.000 {method 'set_value' of 'pandas.index.IndexEngine' objects}
1 0.002 0.002 0.075 0.075 frame.py:307(_init_dict)
13190 0.075 0.000 0.075 0.000 {built-in method numpy.core.multiarray.arange}
6594 0.024 0.000 0.075 0.000 internals.py:1665(is_bool)
6594 0.057 0.000 0.074 0.000 algorithms.py:807(_get_take_nd_function)
13188 0.054 0.000 0.073 0.000 base.py:506(_get_attributes_dict)
764 0.002 0.000 0.072 0.000 re.py:278(_compile)
192 0.000 0.000 0.071 0.000 re.py:222(compile)
43/42 0.000 0.000 0.071 0.002 <frozen importlib._bootstrap_external>:900(create_module)
43/42 0.055 0.001 0.070 0.002 {built-in method _imp.create_dynamic}
167 0.001 0.000 0.070 0.000 sre_compile.py:531(compile)
387 0.001 0.000 0.070 0.000 <frozen importlib._bootstrap_external>:471(_compile_bytecode)
1 0.000 0.000 0.068 0.068 __init__.py:101(<module>)
387 0.067 0.000 0.067 0.000 {built-in method marshal.loads}
13188 0.028 0.000 0.067 0.000 common.py:1532(is_dtype_equal)
13189 0.060 0.000 0.066 0.000 internals.py:276(ftype)
540/539 0.006 0.000 0.066 0.000 <frozen importlib._bootstrap>:879(_find_spec)
20472 0.065 0.000 0.065 0.000 generic.py:2658(__getattr__)
1 0.000 0.000 0.064 0.064 frame.py:5224(_arrays_to_mgr)
6594 0.011 0.000 0.064 0.000 generic.py:2753(_is_mixed_type)
13189 0.038 0.000 0.063 0.000 internals.py:2579(<listcomp>)
13879 0.062 0.000 0.062 0.000 generic.py:94(__init__)
13191 0.019 0.000 0.057 0.000 common.py:1696(is_bool_dtype)
7283 0.009 0.000 0.057 0.000 common.py:73(isnull)
20923 0.017 0.000 0.057 0.000 series.py:366(_values)
2 0.000 0.000 0.056 0.028 common.py:1(<module>)
535 0.001 0.000 0.055 0.000 <frozen importlib._bootstrap_external>:1130(find_spec)
7284 0.036 0.000 0.055 0.000 internals.py:3778(__init__)
535 0.004 0.000 0.055 0.000 <frozen importlib._bootstrap_external>:1098(_get_spec)
53448 0.044 0.000 0.054 0.000 base.py:3381(_ensure_index)
20473 0.046 0.000 0.054 0.000 internals.py:191(mgr_locs)
59355 0.053 0.000 0.053 0.000 {method 'view' of 'numpy.ndarray' objects}
6594 0.036 0.000 0.052 0.000 common.py:733(_possibly_downcast_to_dtype)
990 0.011 0.000 0.052 0.000 <frozen importlib._bootstrap_external>:1212(find_spec)
6594 0.013 0.000 0.051 0.000 generic.py:1476(_check_is_chained_assignment_possible)
2 0.001 0.000 0.051 0.026 pyparsing.py:58(<module>)
1 0.000 0.000 0.051 0.051 rcsetup.py:15(<module>)
7283 0.022 0.000 0.049 0.000 generic.py:1359(_set_as_cached)
1 0.000 0.000 0.048 0.048 fontconfig_pattern.py:7(<module>)
26380 0.036 0.000 0.048 0.000 common.py:1511(_get_dtype_type)
7283 0.033 0.000 0.048 0.000 common.py:94(_isnull_new)
6594 0.029 0.000 0.048 0.000 base.py:2201(_possibly_promote)
7284 0.020 0.000 0.045 0.000 series.py:302(name)
1 0.002 0.002 0.045 0.045 frame.py:5521(_homogenize)
95791 0.044 0.000 0.044 0.000 {method 'get' of 'dict' objects}
19783 0.017 0.000 0.043 0.000 base.py:870(_values)
13188 0.014 0.000 0.043 0.000 indexing.py:1890(is_list_like_indexer)
1105 0.003 0.000 0.043 0.000 series.py:2787(_sanitize_array)
6594 0.042 0.000 0.042 0.000 {pandas.lib.is_bool_array}
169/167 0.001 0.000 0.042 0.000 sre_parse.py:819(parse)
26377 0.027 0.000 0.041 0.000 internals.py:3276(_consolidate_inplace)
6595 0.041 0.000 0.041 0.000 {pandas.lib.is_datetime_array}
19782 0.027 0.000 0.041 0.000 indexing.py:128(<genexpr>)
20923 0.026 0.000 0.040 0.000 internals.py:3911(internal_values)
194369/194289 0.038 0.000 0.040 0.000 {built-in method builtins.issubclass}
557/169 0.003 0.000 0.040 0.000 sre_parse.py:429(_parse_sub)
751/186 0.014 0.000 0.039 0.000 sre_parse.py:491(_parse)
26376 0.031 0.000 0.039 0.000 common.py:1491(_get_dtype)
6594 0.013 0.000 0.038 0.000 generic.py:1402(_is_view)
6 0.000 0.000 0.038 0.006 api.py:3(<module>)
1 0.000 0.000 0.038 0.038 __init__.py:27(<module>)
2 0.000 0.000 0.037 0.019 __init__.py:7(<module>)
1105 0.002 0.000 0.037 0.000 series.py:2804(_try_cast)
6595 0.019 0.000 0.037 0.000 common.py:272(array_equivalent)
6594 0.010 0.000 0.036 0.000 generic.py:2755(<lambda>)
13207 0.025 0.000 0.036 0.000 __init__.py:168(iteritems)
2 0.000 0.000 0.036 0.018 __init__.py:26(<module>)
6596 0.024 0.000 0.036 0.000 base.py:1143(_engine)
6594 0.014 0.000 0.035 0.000 generic.py:337(_get_block_manager_axis)
7284 0.022 0.000 0.033 0.000 series.py:306(name)
34350 0.032 0.000 0.032 0.000 {pandas.lib.isscalar}
6877 0.006 0.000 0.032 0.000 {built-in method builtins.all}
14294 0.022 0.000 0.031 0.000 common.py:1763(is_list_like)
13188 0.021 0.000 0.030 0.000 indexing.py:547(<genexpr>)
53445 0.030 0.000 0.030 0.000 internals.py:160(mgr_locs)
6596 0.010 0.000 0.030 0.000 base.py:1146(<lambda>)
2 0.000 0.000 0.029 0.014 __init__.py:2912(_call_aside)
1 0.000 0.000 0.029 0.029 __init__.py:2927(_initialize_master_working_set)
7283 0.026 0.000 0.028 0.000 base.py:1247(__getitem__)
1 0.000 0.000 0.028 0.028 generic.py:2(<module>)
6608 0.028 0.000 0.028 0.000 {built-in method builtins.sorted}
13188 0.019 0.000 0.028 0.000 generic.py:1441(_clear_item_cache)
1 0.000 0.000 0.027 0.027 requirements.py:4(<module>)
13190 0.009 0.000 0.026 0.000 base.py:445(get_values)
167 0.000 0.000 0.026 0.000 sre_compile.py:516(_code)
6594 0.012 0.000 0.026 0.000 internals.py:3009(is_mixed_type)
21161 0.026 0.000 0.026 0.000 internals.py:2695(_get_items)
6595 0.011 0.000 0.025 0.000 {built-in method builtins.sum}
6594 0.015 0.000 0.025 0.000 internals.py:3027(is_view)
13188 0.015 0.000 0.025 0.000 internals.py:3260(consolidate)
2 0.000 0.000 0.024 0.012 __init__.py:45(<module>)
39565 0.024 0.000 0.024 0.000 internals.py:2996(is_consolidated)
523 0.001 0.000 0.024 0.000 decorators.py:181(__call__)
7283 0.024 0.000 0.024 0.000 internals.py:301(iget)
545 0.004 0.000 0.023 0.000 textwrap.py:415(dedent)
6594 0.021 0.000 0.023 0.000 common.py:446(_infer_dtype_from_scalar)
2 0.000 0.000 0.023 0.011 index.py:2(<module>)
6594 0.020 0.000 0.022 0.000 generic.py:124(_init_mgr)
63023 0.022 0.000 0.022 0.000 csv.py:92(fieldnames)
6594 0.017 0.000 0.021 0.000 generic.py:307(_get_axis_number)
23 0.000 0.000 0.021 0.001 pyparsing.py:798(_trim_arity)
22 0.000 0.000 0.021 0.001 pyparsing.py:806(extract_stack)
22 0.000 0.000 0.021 0.001 traceback.py:192(extract_stack)
22 0.004 0.000 0.021 0.001 traceback.py:303(extract)
30 0.001 0.000 0.021 0.001 __init__.py:349(namedtuple)
1 0.000 0.000 0.021 0.021 internals.py:1(<module>)
6594 0.012 0.000 0.020 0.000 common.py:1550(is_integer_dtype)
1 0.000 0.000 0.020 0.020 parser.py:5(<module>)
13188 0.016 0.000 0.020 0.000 base.py:508(<listcomp>)
1320/167 0.006 0.000 0.019 0.000 sre_compile.py:64(_compile)
19 0.000 0.000 0.019 0.001 pyparsing.py:958(setParseAction)
1 0.000 0.000 0.019 0.019 feedparser.py:20(<module>)
1 0.000 0.000 0.019 0.019 internals.py:3996(create_block_manager_from_arrays)
19784 0.015 0.000 0.019 0.000 internals.py:2623(ndim)
1 0.000 0.000 0.019 0.019 dates.py:111(<module>)
1 0.003 0.003 0.019 0.019 internals.py:4007(form_blocks)
13188 0.019 0.000 0.019 0.000 internals.py:650(<lambda>)
13198 0.019 0.000 0.019 0.000 dtypes.py:122(construct_from_string)
7284 0.018 0.000 0.018 0.000 series.py:292(_set_subtyp)
13188 0.012 0.000 0.018 0.000 missing.py:559(clean_reindex_fill_method)
1105 0.005 0.000 0.018 0.000 common.py:1011(_possibly_cast_to_datetime)
88922 0.018 0.000 0.018 0.000 {method 'append' of 'list' objects}
3288 0.018 0.000 0.018 0.000 {built-in method posix.stat}
6594 0.005 0.000 0.017 0.000 base.py:3425(_ensure_has_len)
6594 0.012 0.000 0.017 0.000 internals.py:4301(_extend_blocks)
17/16 0.000 0.000 0.017 0.001 <frozen importlib._bootstrap>:630(_load_backward_compatible)
5 0.000 0.000 0.017 0.003 __init__.py:34(load_module)
1 0.000 0.000 0.017 0.017 message.py:5(<module>)
2634 0.002 0.000 0.016 0.000 <frozen importlib._bootstrap_external>:68(_path_stat)
14 0.000 0.000 0.016 0.001 __init__.py:663(add_entry)
I have thickness measurements taken on a flat plate. Is there any excel like conditional formatting option in gnuplot? I would like to have four different plots with the values,
Higher than a given number, say, in this case, 0.5
Lower than 0.5
Between a given range, say, 0.5 and 0.51
Only the values 0.5?
How to modify the codes below?
set pm3d map
splot 't.dat' matrix
Here is my data file
0.509 0.510 0.515 0.529 0.521 0.516 0.515
0.511 0.506 0.512 0.528 0.524 0.517 0.512
0.510 0.506 0.506 0.530 0.524 0.522 0.505
0.511 0.509 0.513 0.516 0.511 0.520 0.510
0.524 0.516 0.512 0.511 0.507 0.518 0.492
0.525 0.521 0.515 0.517 0.518 0.522 0.500
0.530 0.521 0.513 0.512 0.511 0.519 0.503
0.562 0.516 0.510 0.516 0.522 0.518 0.508
0.520 0.518 0.512 0.517 0.518 0.518 0.510
0.510 0.509 0.503 0.507 0.523 0.519 0.522
0.506 0.500 0.424 0.507 0.523 0.527 0.519
0.509 0.430 0.500 0.513 0.519 0.528 0.524
0.506 0.503 0.503 0.506 0.513 0.528 0.533
0.506 0.517 0.519 0.524 0.524 0.526 0.528
0.525 0.517 0.499 0.520 0.521 0.524 0.518
0.519 0.518 0.516 0.519 0.521 0.520 0.519
0.521 0.502 0.515 0.518 0.518 0.523 0.522
0.515 0.519 0.519 0.534 0.524 0.525 0.516
0.517 0.510 0.522 0.532 0.533 0.530 0.525
0.520 0.457 0.526 0.530 0.530 0.531 0.524
0.530 0.520 0.531 0.529 0.527 0.526 0.524
Thanks!
You can do conditional plots like this, but the conditional filtering will leave "holes" in your graph.
Without formatting:
set pm3d map
splot "./data" matrix
Plotting only values larger than 0.5:
set pm3d map
splot "./data" matrix u 1:2:($3 > 0.5 ? $3 : 1/0)
You see you have points missing. If you filter even more, you'll have more points missing, to the extreme that you might not have anything to plot (with < 0.5) because there is no way to interpolate. What you can do is substitute points outside of range by a fixed value. For instance, if the value is smaller than 0.5 substitute by 0.5, if it's larger than 0.53 substitute by 0.53:
set pm3d map
splot "./data" matrix u 1:2:($3 < 0.5 ? 0.5 : $3 > 0.53 ? 0.53 : $3)
For nicer display with these small matrices, you can think about interpolation:
set pm3d map interpolate 32,32
splot "./data" matrix u 1:2:($3 < 0.5 ? 0.5 : $3 > 0.53 ? 0.53 : $3)