Let us consider following stock prices taken from yahoo.finance.com:
Date Open High Low Close Volume Adj Close
3/4/2013 23.15 23.84 23.03 23.67 30908300 23.3
2/25/2013 23.5 23.53 22.81 23.19 40710800 22.83
2/19/2013 23.42 23.75 23.12 23.39 38743400 23.03
2/11/2013 22.49 23.55 22.35 23.29 46448500 22.74
2/4/2013 22.41 22.62 22.27 22.5 34498100 21.97
1/28/2013 22.44 22.64 22.18 22.62 39634900 22.09
1/22/2013 22.18 22.31 21.75 22.29 47826300 21.77
1/14/2013 21.18 22.19 21.01 22.04 54826000 21.52
1/7/2013 21.16 21.24 20.68 21.13 35304100 20.63
12/31/2012 20.29 21.54 20.26 21.2 45796500 20.7
12/24/2012 20.79 20.96 20.42 20.44 28597100 19.96
12/17/2012 21.69 21.95 20.56 20.88 70719700 20.39
12/10/2012 21.43 21.95 21.36 21.62 39455500 20.92
12/3/2012 21.18 21.48 20.71 21.46 35913000 20.77
11/26/2012 20.88 21.36 20.5 21.13 36203100 20.45
11/19/2012 20.41 21.04 20.37 21.04 35401500 20.36
11/12/2012 21.04 21.14 19.87 20.15 45095400 19.5
11/5/2012 21.2 21.78 20.7 21 37812800 20.32
11/2/2012 21.53 21.68 21.26 21.31 47475200 20.62
And i want to do correlation matrix between for example Volume and Low Close variable. I used correlation function from data analysis toolbox from excel, but I got only one side matrix, like this:
Close Volume
Close 1
Volume -0.117267345 1
It does not show me correlation coefficients up side of main diagonal, why? Maybe it is symmetric and because of this?
The correlation matrix is necessarily symmetric, so the above the diagonal element in your case is -0.117257345. If you check the documentation on the correl function, and look at the defining equation, you can see that it is symmetric with respect to exchanging X<->Y
Related
I'm using the Tabulate package to print data in table format. The output is sent to a webpage. While using the default font everything is working fine. However upon changing font family (Outfit from Google fonts or cursive e.g), they stop being aligned. Are there any possible solutions?
Output with default font:
Strength: 16 Dmg: 50 Armor: 3.8 ShadowRes: 3.5%
Agility: 34 Spell: 183 FireRes: 5.1% NatureRes: 6.1%
Intellect: 61 Critical: 3.4% FrostRes: 6.3% ArcaneRes: 3.8%
Output with Google font (looks like this can't really show it because SO font is the default):
Strength: 25 Dmg: 45 Armor: 3.1 ShadowRes: 3.2%
Agility: 20 Spell: 132 FireRes: 3.3% NatureRes: 3.6%
Intellect: 44 Critical: 2.0% FrostRes: 3.6% ArcaneRes: 3.8%
Thanks in advance!
you need a monospace font in order to keep the good size of space
I'm having trouble getting an SVG properly rendered into aframe's scene. I went the htmltexture-component route, which has a dependency on the draw-component.
The problem is in getting the SVG's background to render as transparent. It simply is not working. Instead I'm given a default white background drawn on the canvas. My SVG was created in Illustrator and is a valid HTML object with no background (as a color or a raster image).
<div id="bears">
<svg xmlns="http://www.w3.org/2000/svg" width="518" height="500" xmlns:xlink="http://www.w3.org/1999/xlink">
<path d="M168.958 480.023c-12.18-1.57-24.62-6.847-34.293-14.544-9.103-7.25-11.733-15.34-7.8-23.99 4.594-10.11 22.386-19.1 27.564-13.92 1.71 1.71 2.03 5.14.68 7.27-.63.99-1.03.66-1.51-1.25-1.37-5.43-13.5-2.54-16.44 3.92-2.72 5.95-.57 14.06 4.3 16.28 5.39 2.45 15.09 1.04 23.89-3.49 3.63-1.87 13.35-9.11 19.14-14.25 12.62-11.23 19.38-16.07 28.49-20.42 5.93-2.84 7.37-3.11 16.5-3.11 8.3 0 10.88.39 15.18 2.32 2.84 1.27 5.95 3.17 6.9 4.22 2.05 2.26 3.74 2.42 5.34.5 2.24-2.71 11.83-6.6 17.78-7.23 6.83-.72 17.37.833 22.4 3.3 3.87 1.892 14 8.59 17.72 11.72 1.38 1.155 6.14 5.31 10.59 9.225 14.51 12.8 25.37 18.42 35.56 18.42 4.85 0 9.52-2.153 11.11-5.117 1.75-3.28 1.46-10.15-.56-13.24-1.83-2.793-7.96-6.428-10.258-6.08-.682.102-2.15.243-3.25.312-1.55.097-2 .845-2.03 3.375-.03 3.073-.11 3.144-1.5 1.31-3.32-4.39-.46-9.56 5.28-9.56 7.277 0 18.478 7.596 22.327 15.14 2.34 4.59 2.48 10.893.34 15.403-3.748 7.9-16.18 16.29-31.223 21.08-13.8 4.39-45.63 4.12-65.69-.56-11.47-2.67-21.36-7.31-27.05-12.67-3.21-3.02-6.45-3.65-6.45-1.26 0 .51-2.65 2.63-5.88 4.72-10.35 6.7-24.14 10.68-42.05 12.17-12.99 1.08-26.83 1.08-35.11.02zM374.5 360.566c-7.77-1.094-14.833-3.225-33.893-10.225-2.14-.78-9.34-4.27-16-7.75-14.11-7.37-19.35-9.6-29.607-12.57-12.336-3.57-13.295-3.8-18.5-4.48-8.943-1.16-8.8-.78-6.03-16.03.2-1.1.4-3.03.447-4.3.08-2.21.308-2.27 6.333-1.66 6.334.65 20.588 3.89 24.25 5.51 1.1.49 4.505 1.97 7.567 3.29 3.06 1.32 7.894 4.04 10.74 6.04 2.845 2 5.515 3.64 5.933 3.64.418 0 1.99 1.02 3.49 2.25 4.238 3.49 7.614 5.83 16.27 11.27 7.074 4.45 9.82 5.9 18 9.5 4.746 2.1 11.452 3.81 17.5 4.48 3.575.4 7.85.88 9.5 1.08 3.93.47 14.805-2.08 18.27-4.28 7.812-4.96 12.43-11.71 14.7-21.49 2.514-10.84-.142-21.28-6.64-26.082-5.234-3.87-16.747-1.88-24.316 4.2-1.093.88-.412-.367 1.512-2.76 8.517-10.61 20.494-11.775 30.528-2.965 7.336 6.44 10.342 17.7 7.88 29.525-1.44 6.92-6.636 18.58-9.84 22.07-10.297 11.237-25.51 14.96-48.094 11.78zm-262-2.216c-6.116-.74-13.637-3.196-17-5.55-5.537-3.875-9.688-8.793-13.294-15.754-9.527-18.388-7.015-35.84 6.445-44.778 8.89-5.902 16.55-4.74 25.97 3.944 2.41 2.224 4.38 4.548 4.37 5.165-.01.818-.21.822-.75.014-1.13-1.7-10.33-6.31-13.97-7-15.47-2.9-24.2 15.75-15.88 33.94 8 17.5 25.7 23.2 50.15 16.16 11.14-3.2 19.74-7.56 34.28-17.36 6.22-4.19 12.07-8.07 13-8.61.93-.55 3.04-1.93 4.69-3.08 8.9-6.17 22.62-11.34 35.5-13.39 4.13-.65 8.51-1.35 9.74-1.54 1.23-.19 2.52.11 2.87.68.35.56 1.16 5.43 1.81 10.82l1.17 9.8-5.03.6c-5.65.666-19.46 3.85-27.04 6.237-2.75.864-13.55 5.76-24 10.884-22.3 10.93-26.66 12.68-40 16.04-9.48 2.39-25.33 3.744-33 2.82zM174 237.527c-23.41-4.077-43.362-18.19-56.588-40.027-2.58-4.258-7.44-15.748-8.04-19-1.02-5.543-2.358-14.3-2.364-15.487-.004-.817 1.522.758 3.39 3.5 8.792 12.897 18.184 18.825 30.354 19.16 12.067.332 14.908-1.168 29.013-15.317C181.473 158.612 195.32 148 198.94 148c.912 0 2.224-.563 2.912-1.25.898-.896 5.704-1.244 16.95-1.23 15.634.02 15.722.033 21.698 3.156 9.06 4.735 14.04 8.73 19.248 15.436 4.637 5.97 4.75 6.04 6.237 3.94 2.74-3.865 12.845-14.052 13.94-14.052.592 0 1.078-.338 1.08-.75.012-1.222 6.068-4.334 12.393-6.366 8.55-2.747 24.897-2.706 33.558.084 3.55 1.144 6.7 2.407 7 2.806.3.4 2.138 1.534 4.086 2.522 5.085 2.577 11.184 7.778 21.458 18.298 11.824 12.107 15.94 14.606 25.015 15.19 9.73.623 17.925-2.324 26.59-9.563 1.888-1.57 7.062-8.87 9.733-13.72 1.95-3.54 1.386 5.22-.938 14.56-1.925 7.74-6.9 19.74-9.38 22.63-.56.66-2.145 2.88-3.52 4.95-3.75 5.63-14.84 16.58-20.5 20.25-12.477 8.08-26.436 12.77-40.5 13.6-8.476.5-23.706-.84-27.5-2.43-1.1-.46-4.025-1.38-6.5-2.04-2.475-.66-6.876-2.29-9.78-3.61-2.903-1.32-5.483-2.4-5.732-2.4-1.02 0-13.642-8.13-19.31-12.44-3.345-2.54-7.576-6.37-9.402-8.5l-3.32-3.88-3.18 3.79C257.95 210.93 243.87 222 242.16 222c-.545 0-1.14.395-1.325.878-.183.483-3.933 2.68-8.333 4.882-19.038 9.526-39.892 13.008-58.5 9.767zm-1.594-159.52c-8.636-.944-14.453-3.438-20.44-8.763-11.826-10.52-14.99-29.243-7.322-43.33 3.49-6.416 8.05-10.408 16.31-14.276 5.78-2.71 7.6-3.07 15.546-3.075 7.427-.006 9.75.388 13.29 2.255 5.43 2.866 8.203 7.32 8.203 13.182 0 7.234-4.917 13.106-12.993 15.52-2.437.727-2.455.695-.72-1.25.98-1.097 2.212-3.5 2.74-5.34.846-2.948.645-3.67-1.696-6.113-8.034-8.386-20.356-4.395-23.882 7.736-2.124 7.304-1.174 11.553 3.535 15.807 4.824 4.358 9.81 5.97 16.88 5.453 8.677-.634 14.04-3.842 18.92-11.315 9.25-14.173 24.976-20.235 45.804-17.66 4.91.608 10.27 1.42 11.92 1.806 3.86.903 7.14.91 11.5.026 24.76-5.018 44.31-1.46 53.48 9.733 1.94 2.367 3.53 4.804 3.53 5.417 0 2.032 7.18 8.325 11.71 10.267 9 3.856 21.6 1.406 26.5-5.15 8.07-10.805-3.26-29.497-15.79-26.04-7.94 2.195-10.78 8.818-6.24 14.59l2.17 2.754-2.42-.72c-3.86-1.15-8.41-4.57-10.88-8.17-1.85-2.71-2.19-4.27-1.81-8.28.98-10.11 7.12-14.39 20.78-14.46 15.78-.08 27.03 6.4 33.66 19.41 2.46 4.83 2.8 6.61 2.8 14.5 0 11.84-3.12 19.13-11.48 26.87-7.8 7.22-14 9.01-30.97 8.92-25.37-.13-75.41-11.67-79.54-18.34-.44-.72-1.4-.42-2.81.88-6.42 5.89-50.5 16.3-74.15 17.51-4.95.26-12.19.11-16.09-.31z"/>
</svg>
</div>
The SVG is held within <a-assets></a-assets> along with other assets.
The entity's code looks like this:
<a-entity geometry="primitive: plane" position="10 4 -4" scale="4 4 0" draw="width: 512; height: 512;" htmltexture="asset: #bears">
</a-entity>
The rendered plane holding the SVG texture ends up having a white background. I tried changing the material transparency property, but that didn't work. I also tried changing the draw background property, seeing as how aframe-htmltexture-component has a dependency on aframe-draw-component which creates a canvas element to draw to, but again no luck.
Anyone know a fix for this, or have a better way at rendering HTML (SVG) as an aframe entity?
Try material.opacity:
<a-entity
geometry="primitive: plane"
position="10 4 -4"
scale="4 4 0"
draw="width: 512; height: 512;"
htmltexture="asset: #bears"
material="opacity: 0.0; transparent: true">
</a-entity>
Edit: Transparent must be set to true too.
Did you try transparent="true" ?
I had similar issues with importing PNGs with transparency, and then discovered setting transparent="true" fixed it.
<a-plane position="0 0.01 0" rotation="-90 0 0" height="7" width="7"
src ="#metatron-gates-platform"
transparent ="true">
</a-plane>
I ran into this as well. Try changing the order of the svg entities. Because it is a 3d environment it renders farthest from the camera first, so place those entities first in your code.
Running into the above error message when fitting the model for X and Y. Both are taken from training data and truth respectively. Verified that data does not contain NaN or Inf.
Tried to subset the data into 20*3 matrix and eyeballed the data where nothing seemed out of place. How can I fix it.
Here is the data subset I am working on:
1 2 3
12235 0.0369 -0.1415 -0.4381
11008 0.4285 0.2449 0.7858
15983 0.5557 0.0466 -0.2477
15881 0.8825 1.3252 -0.2296
14037 1.6551 0.5298 0.1924
4860 0.7082 -0.3576 0.5771
13475 0.0103 0.1030 1.4402
7226 0.5135 1.2396 0.9988
2862 0.5454 -0.1530 1.5451
1401 0.7960 0.9605 0.8021
3988 0.2682 0.9393 -0.1930
16346 -0.2303 0.5633 0.5991
15293 0.9816 0.6522 0.1207
895 0.6816 0.6819 0.5101
14781 0.2243 0.0350 -0.6212
14791 0.1902 0.2113 0.4330
4869 0.5471 1.4235 0.4891
1770 0.5270 0.4097 0.3691
15483 1.0364 0.8619 0.6298
17033 0.9304 -0.3223 0.9128
1 2 3
9909 0.0884 0.3513 0.7508
4307 0.3094 0.8885 1.2935
14128 -0.5162 1.0465 -1.1435
15694 0.6993 0.3426 0.9185
3709 -0.6405 -0.3263 0.2199
16190 0.7642 0.4764 0.3143
15877 0.6836 0.2586 0.8664
3319 -0.3437 -0.1538 0.5070
8135 0.1876 0.9128 -0.1812
13035 0.7733 1.7522 0.4158
12168 -0.0617 -0.0897 0.3686
10469 1.1860 0.3772 0.4178
6211 0.8808 1.0333 0.5994
9491 0.5110 0.6489 0.6749
8310 0.5609 0.1232 0.7549
171 1.3448 -0.7569 -0.1178
2068 0.4097 -0.1648 0.1831
4393 -0.2469 -0.4033 0.2077
2134 0.9408 0.2473 0.2176
12191 0.1368 1.5374 0.7149
I was passing dataframe, using .values in fit() fixes the issue and also my Y_Predict is giving few NaNs. It worked after I fixed these things. Thanks!!
I have files with columns like this. This sample input below is partial.
Please check below for main file link. Each file will have only two rows.
Gene 0.4% 0.7% 1.1% 1.4% 1.8% 2.2% 2.5% 2.9% 3.3% 3.6% 4.0% 4.3% 4.7% 5.1% 5.4% 5.8% 6.2% 6.5% 6.9% 7.2% 7.6% 8.0% 8.3% 8.7% 9.1% 9.4% 9.8% 10.1% 10.5% 10.9% 11.2% 11.6% 12.0% 12.3% 12.7% 13.0% 13.4% 13.8% 14.1% 14.5% 14.9% 15.2% 15.6% 15.9% 16.3% 16.7% 17.0% 17.4% 17.8% 18.1% 18.5% 18.8% 19.2% 19.6% 19.9% 20.3% 20.7% 21.0% 21.4% 21.7% 22.1% 22.5% 22.8% 23.2% 23.6% 23.9% 24.3% 24.6% 25.0% 25.4% 25.7% 26.1% 26.4% 26.8% 27.2% 27.5% 27.9% 28.3% 28.6% 29.0% 29.3% 29.7% 30.1% 30.4% 30.8% 31.2% 31.5% 31.9% 32.2% 32.6% 33.0% 33.3% 33.7% 34.1% 34.4% 34.8% 35.1% 35.5% 35.9% 36.2% 36.6% 37.0% 37.3% 37.7% 38.0% 38.4% 38.8% 39.1% 39.5% 39.9% 40.2% 40.6% 40.9% 41.3% 41.7% 42.0% 42.4% 42.8% 43.1% 43.5% 43.8% 44.2% 44.6% 44.9% 45.3% 45.7% 46.0% 46.4% 46.7% 47.1% 47.5% 47.8% 48.2% 48.6% 48.9% 49.3% 49.6% 50.0% 50.4% 50.7% 51.1% 51.4% 51.8% 52.2% 52.5% 52.9% 53.3% 53.6% 54.0% 54.3% 54.7% 55.1% 55.4% 55.8% 56.2% 56.5% 56.9% 57.2% 57.6% 58.0% 58.3% 58.7% 59.1% 59.4% 59.8% 60.1% 60.5% 60.9% 61.2% 61.6% 62.0% 62.3% 62.7% 63.0% 63.4% 63.8% 64.1% 64.5% 64.9% 65.2% 65.6% 65.9% 66.3% 66.7% 67.0% 67.4% 67.8% 68.1% 68.5% 68.8% 69.2% 69.6% 69.9% 70.3% 70.7% 71.0% 71.4% 71.7% 72.1% 72.5% 72.8% 73.2% 73.6% 73.9% 74.3% 74.6% 75.0% 75.4% 75.7% 76.1% 76.4% 76.8% 77.2% 77.5% 77.9% 78.3% 78.6% 79.0% 79.3% 79.7% 80.1% 80.4% 80.8% 81.2% 81.5% 81.9% 82.2% 82.6% 83.0% 83.3% 83.7% 84.1% 84.4% 84.8% 85.1% 85.5% 85.9% 86.2% 86.6% 87.0% 87.3% 87.7% 88.0% 88.4% 88.8% 89.1% 89.5% 89.9% 90.2% 90.6% 90.9% 91.3% 91.7% 92.0% 92.4% 92.8% 93.1% 93.5% 93.8% 94.2% 94.6% 94.9% 95.3% 95.7% 96.0% 96.4% 96.7% 97.1% 97.5% 97.8% 98.2% 98.6% 98.9% 99.3% 99.6% 100.0% 0.4% 0.7% 1.1% 1.4% 1.8% 2.2% 2.5% 2.9% 3.3% 3.6% 4.0% 4.3% 4.7% 5.1% 5.4% 5.8% 6.2% 6.5% 6.9% 7.2% 7.6% 8.0% 8.3% 8.7% 9.1% 9.4% 9.8% 10.1% 10.5% 10.9% 11.2% 11.6% 12.0% 12.3% 12.7% 13.0% 13.4% 13.8% 14.1% 14.5% 14.9% 15.2% 15.6% 15.9% 16.3% 16.7% 17.0% 17.4% 17.8% 18.1% 18.5% 18.8% 19.2% 19.6% 19.9% 20.3% 20.7% 21.0% 21.4% 21.7% 22.1% 22.5% 22.8% 23.2% 23.6% 23.9% 24.3% 24.6% 25.0% 25.4% 25.7% 26.1% 26.4% 26.8% 27.2% 27.5% 27.9% 28.3% 28.6% 29.0% 29.3% 29.7% 30.1% 30.4% 30.8% 31.2% 31.5% 31.9% 32.2% 32.6% 33.0% 33.3% 33.7% 34.1% 34.4% 34.8% 35.1% 35.5% 35.9% 36.2% 36.6% 37.0% 37.3% 37.7% 38.0% 38.4% 38.8% 39.1% 39.5% 39.9% 40.2% 40.6% 40.9% 41.3% 41.7% 42.0% 42.4% 42.8% 43.1% 43.5% 43.8% 44.2% 44.6% 44.9% 45.3% 45.7% 46.0% 46.4% 46.7% 47.1% 47.5% 47.8% 48.2% 48.6% 48.9% 49.3% 49.6% 50.0% 50.4% 50.7% 51.1% 51.4% 51.8% 52.2% 52.5% 52.9% 53.3% 53.6% 54.0% 54.3% 54.7% 55.1% 55.4% 55.8% 56.2% 56.5% 56.9% 57.2% 57.6% 58.0% 58.3% 58.7% 59.1% 59.4% 59.8% 60.1% 60.5% 60.9% 61.2% 61.6% 62.0% 62.3% 62.7% 63.0% 63.4% 63.8% 64.1% 64.5% 64.9% 65.2% 65.6% 65.9% 66.3% 66.7% 67.0% 67.4% 67.8% 68.1% 68.5% 68.8% 69.2% 69.6% 69.9% 70.3% 70.7% 71.0% 71.4% 71.7% 72.1% 72.5% 72.8% 73.2% 73.6% 73.9% 74.3% 74.6% 75.0% 75.4% 75.7% 76.1% 76.4% 76.8% 77.2% 77.5% 77.9% 78.3% 78.6% 79.0% 79.3% 79.7% 80.1% 80.4% 80.8% 81.2% 81.5% 81.9% 82.2% 82.6% 83.0% 83.3% 83.7% 84.1% 84.4% 84.8% 85.1% 85.5% 85.9% 86.2% 86.6% 87.0% 87.3% 87.7% 88.0% 88.4% 88.8% 89.1% 89.5% 89.9% 90.2% 90.6% 90.9% 91.3% 91.7% 92.0% 92.4% 92.8% 93.1% 93.5% 93.8% 94.2% 94.6% 94.9% 95.3% 95.7% 96.0% 96.4% 96.7% 97.1% 97.5% 97.8% 98.2% 98.6% 98.9% 99.3% 99.6% 100.0%
Basically, here is what I need to be done.
a. Start from second column which is 0.4% here.
b. Go until you hit "10" in the header name. If the header name is exactly 10.0%, then include that column too. If not, only include until the column before it. In this example, since we have 10.1% (29th column), we will be including columns starting from 0.4%(second) until 9.8% which is the 28th column. If the 29th column was to be 10.0%, then it would have been included too.
c. Average the values for these respective columns in the second row (data is not presented here - please click this link for total dataset - https://goo.gl/W8jND7). In this example, starting from 0.4%(second column) till 9.8%(28th column).
d. In the output, print first column which is "Gene", and this average value with column header being
Gene Average_10%
e. Then start from 10.1% (29th column) and check until you hit "20" in the header name. Repeat steps b through d. And print output as
Gene Average_10% Average_20%
Repeat this until you have
Gene Average_10% Average_20% Average_30% Average_40% Average_50% Average_60% Average_70% Average_80% Average_90% Average_100%
f. After you hit 100%, it means one dataset is done.
g. If you observe my column header carefully here, there is another 0.4%-100% columns after the first 100%. I will be having 13 of these 0.4%-100%s in the input file at the above link.
i. I have multiple files, the headers can be
1% 2% 3%....100%
1.5% 2.5% 3.5%....100%
It varies from file to file. But the logic of averaging(if you hit "10", "20", etc) is always the same. And the number of samples 13 is also same which means each file will have 100%s for 13 times.
I should say, it's a horrible format for this task. I don't expect anyone to come up with a final solution for you but this is how I would approach this
awk 'NR == 1 {
gsub("%","");
for (f=2; f<=NF; f++) {
for (i=1; i<10; i++)
if ($f<10*i && $(f+1)>=10*i) print f, $f
if ($f==100) print f, $f
}}' file
28 9.8
56 19.9
83 29.7
111 39.9
138 49.6
166 59.8
194 69.9
221 79.7
249 89.9
277 100.0
304 9.8
332 19.9
359 29.7
387 39.9
414 49.6
442 59.8
470 69.9
497 79.7
525 89.9
553 100.0
here printing the column index and the threshold used for verification purposes. Once you have the column boundaries extracted it should be straightforward to sum the respective columns. Note that by your logic 100% should be never included, however it seems wrong so I have special case for it.
I have a list of data (two columns) and I want to plot in gnuplot only the value for which my second column has a local maximum.
To do so I would like to see if the second column of the ith row is bigger than the (i-1) and (i+1)th rows.
It can be done and I was bored enough to do it. I generated the following set of random data:
5191
29375
23222
32118
3185
32355
17173
8734
28850
20811
5956
6950
28560
25770
4630
28272
10035
7209
19428
26187
30784
20326
12865
23288
20924
Plotting the values against their position in the list looks like this:
You can spot the local maxima right away from the graph above. Now I can process the data points storing the two previous values (both x and y coordinates) in temporary variables, when I identify a maximum, I plot the data point:
# Select the columns of your data file that contain x and y data
# (usually 1 and 2 respectively)
xcolumn=0
ycolumn=1
plot "data" u (column(xcolumn)):(column(ycolumn)) w l, \
"data" u (column(0)==0 ? (last2y=column(ycolumn), \
last2x=column(xcolumn), 1/0) : column(0)==1 ? (lasty=column(ycolumn), \
lastx=column(xcolumn), 1/0) : lastx) \
: \
( column(0) < 2 ? 1/0 : (last2y < lasty && \
column(ycolumn) < lasty) ? (value=lasty, last2y=lasty, last2x=lastx, \
lasty=column(ycolumn), lastx=column(xcolumn), value) : (last2y=lasty, \
last2x=lastx, lasty=column(ycolumn), lastx=column(xcolumn), 1/0)) pt 7
A simple gnuplot peak finder
Comparing 3 consecutive data points is a simple way to look for maxima/peaks.
if y(i-1) < y(i) > y(i+1) then you have a maximum/peak at x(i).
However, if you have noisy curves (like experimental spectra typically are) you will get too many "peaks".
The script below basically calculates for each peak a specific number which is a measure for the "independence" of a peak (see Topographic prominence). Note that a high peak intensity does not necessarily mean high prominence. For the script gnuplot>=5.2 is required since it uses datablocks and plot ... with table. With some adaptions, it can maybe run with older versions.
You can give "threshold" to show only a certain number of peaks. Unfortunately, gnuplot does not have an easy internal feature for sorting datablocks, with this one could have easily shown the first N most prominent peaks. Hence, here it is done differently:
Threshold=100 means show all peaks, Threshold=0 means show only the global maximum. Note that the threshold values and number of shown peaks are not linear between 0 and 100 since it depends on the spectral shape.
Comments and improvements are welcome.
Data: SO28173128.dat
# x y
6.02 3.33
9.59 4.03
9.59 4.03
9.59 4.03
9.83 3.66
10.58 3.22
10.62 3.85
11.32 2.33
11.53 3.67
11.88 3.27
13.28 3.97
13.42 4.35
14.56 2.99
14.75 3.50
15.23 3.91
15.59 3.58
16.56 3.85
16.70 4.49
16.97 3.98
17.23 4.49
17.37 3.73
17.90 4.24
17.93 4.91
18.80 4.23
18.83 4.64
20.37 3.59
20.48 3.99
21.70 3.19
22.29 4.36
22.41 3.38
23.34 5.04
23.41 5.38
24.38 4.71
24.58 4.96
25.40 5.13
25.41 4.70
26.20 4.21
26.36 4.84
26.55 4.11
26.79 4.73
27.02 4.13
27.54 4.51
27.55 5.17
28.08 5.42
28.33 4.97
28.61 5.17
29.06 5.10
29.12 4.26
29.61 3.61
30.03 4.84
30.58 5.30
30.68 4.72
31.21 5.82
31.99 5.83
32.58 6.56
33.03 6.23
33.51 6.79
33.73 8.88
34.00 7.85
34.17 9.82
34.32 11.24
34.55 10.78
34.85 12.58
35.14 13.84
35.50 18.07
35.85 18.40
36.55 30.70
36.75 29.27
37.16 28.78
38.15 18.93
38.29 20.52
38.83 14.96
39.63 11.39
40.17 8.67
40.48 9.22
40.88 9.75
40.95 8.77
41.45 10.86
41.58 10.33
41.84 11.12
42.51 12.83
43.33 15.09
43.43 14.20
43.63 14.56
44.22 14.76
44.92 12.58
45.71 9.40
45.98 9.84
47.03 7.01
47.53 7.15
47.95 5.48
48.08 6.21
48.56 4.51
48.64 5.11
49.14 3.69
49.31 4.27
49.86 4.71
50.14 4.13
50.33 5.83
50.37 5.36
51.00 4.60
51.37 4.75
52.00 4.91
52.01 4.34
54.28 3.83
54.46 4.33
54.83 3.19
55.58 5.30
55.86 4.95
55.92 6.48
56.40 5.89
56.59 5.16
57.56 5.96
57.64 5.21
58.31 5.36
58.38 6.08
58.92 5.84
59.00 5.28
59.57 5.96
59.94 5.37
61.31 6.07
61.32 6.78
61.96 7.34
62.37 6.89
63.14 9.45
63.54 9.51
63.80 11.03
63.93 11.56
64.17 11.12
64.84 12.38
65.14 16.28
65.64 18.07
66.22 19.06
66.42 17.70
66.56 18.40
67.10 17.20
67.24 18.00
67.77 16.82
68.04 15.80
68.15 16.88
68.43 14.98
68.86 16.28
69.33 16.55
70.43 29.11
70.93 48.16
71.62 58.09
72.01 66.42
72.43 73.37
72.79 76.74
72.84 81.24
73.07 79.85
73.15 72.04
73.32 74.49
73.58 61.26
73.71 65.36
73.98 51.60
73.98 58.75
74.25 52.93
74.53 39.70
74.90 34.01
75.18 34.80
76.67 23.29
77.05 23.82
77.15 20.94
77.27 27.13
77.52 25.20
77.70 29.76
78.17 29.96
78.36 33.33
78.73 37.05
78.86 34.67
79.03 39.90
79.38 39.70
79.59 40.62
79.88 39.17
80.59 36.52
80.60 33.61
81.26 27.53
81.67 23.69
82.07 23.69
82.90 15.37
83.94 14.04
84.25 14.42
85.01 11.26
85.22 12.18
85.77 11.19
85.89 9.97
86.46 10.72
86.56 11.78
87.06 10.56
87.19 11.53
87.74 9.60
87.89 10.64
88.30 8.74
88.56 9.27
88.98 8.08
89.25 8.70
90.12 9.15
90.14 8.34
90.99 9.59
91.06 8.86
91.72 9.01
91.74 8.48
92.43 7.68
92.48 8.34
93.15 7.68
93.54 8.34
93.72 7.61
94.20 8.00
94.39 8.64
94.82 7.83
95.00 8.40
95.43 6.77
95.66 7.39
95.81 5.44
95.84 6.25
96.53 7.70
96.55 6.70
97.22 9.14
97.47 8.13
97.53 6.75
98.57 7.83
98.85 7.40
98.92 8.45
99.35 8.25
99.40 7.77
100.04 8.33
100.14 7.91
101.00 8.30
101.04 7.80
101.94 7.48
102.02 8.20
102.53 6.56
102.58 7.84
103.23 7.75
103.38 6.82
103.39 8.93
103.75 8.60
104.05 9.26
104.30 8.14
104.55 8.56
104.85 7.15
105.17 10.09
105.21 8.41
105.52 11.78
105.80 10.00
105.92 10.86
106.11 9.54
106.31 9.93
106.86 10.99
107.20 10.88
107.42 12.05
107.62 10.33
107.80 11.16
108.39 9.90
108.79 12.25
109.08 11.85
109.46 13.77
109.73 13.42
109.96 15.16
111.02 20.08
111.30 23.56
111.69 23.56
112.35 30.08
112.61 39.04
112.89 43.53
113.28 43.67
113.81 60.73
113.94 56.63
114.20 63.11
114.46 69.60
115.00 80.44
115.12 86.95
115.27 84.68
115.78 88.78
116.11 89.57
116.40 88.51
116.43 89.90
117.90 80.31
118.18 71.32
118.42 72.51
118.70 67.48
118.83 61.13
119.25 60.34
119.37 54.51
119.65 55.18
120.08 50.28
120.12 48.96
120.84 54.18
120.95 51.87
121.24 56.49
121.50 53.01
121.64 54.44
121.73 51.74
121.84 52.53
122.14 53.72
122.19 52.40
122.44 53.59
122.45 54.91
122.64 51.61
122.76 53.17
123.08 50.79
123.10 50.15
123.34 49.51
123.39 50.81
123.88 50.28
124.25 46.58
124.26 44.06
124.53 43.67
125.18 38.77
125.81 32.16
126.10 32.55
127.32 21.71
127.70 21.18
127.96 18.27
128.22 18.66
128.62 15.71
128.89 16.24
129.30 13.94
129.56 14.47
129.96 12.22
130.21 12.72
130.53 10.83
130.93 10.61
131.15 11.24
131.84 10.06
131.93 11.12
132.71 9.87
133.06 13.17
133.38 12.73
134.17 16.97
134.31 22.37
134.85 30.83
135.38 43.01
135.62 40.09
135.64 50.81
135.75 54.05
136.16 56.10
136.80 50.22
137.21 49.36
138.29 25.41
138.56 26.73
139.98 13.09
140.03 11.92
140.32 14.53
140.55 13.80
140.85 16.25
141.24 15.81
141.45 17.92
141.59 16.51
141.99 20.65
142.25 20.19
142.76 27.79
142.91 24.95
143.72 37.66
143.79 35.37
144.65 29.11
144.91 30.32
145.71 25.74
145.84 24.80
146.54 27.73
146.74 30.57
147.30 31.11
147.42 30.30
148.33 31.23
148.38 30.68
148.93 31.19
149.09 31.74
149.24 28.43
149.49 29.03
149.96 24.55
150.69 23.48
151.83 18.06
151.89 19.39
152.41 18.27
152.96 19.45
153.02 18.53
153.47 19.42
153.71 18.78
153.91 19.44
154.08 17.74
154.54 18.92
154.65 18.20
155.61 18.79
156.78 15.75
157.06 16.08
157.54 12.68
157.84 13.40
158.51 12.73
158.57 11.85
159.45 11.27
159.57 10.33
160.11 8.81
160.24 9.81
160.90 9.27
160.90 8.08
161.17 8.48
161.43 8.10
161.59 8.72
161.83 7.91
162.44 8.48
162.61 7.94
163.41 8.75
163.43 8.19
163.94 8.99
163.96 8.33
164.60 8.23
164.87 8.74
165.81 8.50
166.10 8.08
166.74 8.91
167.02 8.48
167.67 9.68
167.96 9.28
168.90 9.81
168.96 10.34
169.57 9.77
169.76 11.28
170.00 10.75
171.01 13.77
171.12 13.13
171.79 13.71
172.83 19.14
173.19 19.13
173.52 21.06
173.73 20.38
173.87 21.65
174.27 21.90
174.41 25.94
174.94 27.39
175.20 33.48
175.71 35.07
175.84 40.49
176.48 43.27
176.51 46.84
177.34 51.47
177.62 57.30
177.69 55.18
177.87 53.26
178.08 54.46
178.45 49.43
178.76 49.62
179.02 43.53
179.43 42.08
179.96 28.98
180.21 26.40
180.56 26.57
180.57 26.20
181.14 26.43
181.24 25.51
181.68 24.93
181.74 27.13
181.97 28.85
182.05 26.93
182.14 29.78
182.30 27.53
182.59 28.72
182.72 26.93
183.04 27.67
183.21 28.76
183.33 27.92
183.75 29.99
184.06 29.78
184.28 32.07
184.71 32.49
185.40 37.78
185.61 34.01
186.02 33.86
186.20 30.07
186.79 28.72
186.97 25.44
187.34 25.15
188.45 17.65
188.84 16.25
189.08 16.77
189.50 14.08
189.90 13.69
189.92 11.67
190.37 12.04
190.41 12.90
190.90 11.40
190.93 12.58
192.29 9.27
192.33 8.79
193.23 10.21
193.34 9.73
193.78 11.15
194.07 10.20
194.54 10.43
194.93 9.64
195.39 9.82
195.98 10.86
196.08 10.05
196.45 11.78
197.00 8.90
197.03 9.95
197.70 7.15
198.07 10.13
198.10 8.48
198.83 9.67
198.85 10.46
199.27 9.80
199.46 9.46
200.46 11.78
200.73 11.38
201.38 12.37
201.92 9.72
202.06 10.52
203.14 7.67
203.52 7.95
203.93 7.02
204.13 7.55
204.39 6.66
204.63 7.15
206.35 5.53
206.57 6.00
207.21 6.57
207.33 5.76
208.23 7.93
208.52 7.68
208.92 7.15
209.21 7.56
209.41 6.62
209.88 6.89
211.48 4.50
211.70 5.00
212.39 5.37
212.51 4.53
212.84 4.31
212.92 5.17
213.26 4.38
213.44 5.24
213.70 4.57
214.15 4.97
214.24 5.71
215.55 5.20
215.65 6.01
216.28 6.16
216.55 6.87
216.73 5.56
216.95 6.23
217.25 6.03
217.47 5.46
217.93 5.50
218.26 5.83
218.56 5.04
219.31 4.73
219.97 4.72
220.10 5.61
220.79 4.23
220.87 4.94
221.87 4.64
222.07 4.12
222.58 4.47
222.69 5.02
223.21 4.51
223.24 5.22
223.86 4.54
223.90 5.22
224.48 4.29
224.54 5.04
225.22 4.73
225.28 3.86
225.54 4.13
225.76 4.72
225.84 3.93
226.52 4.23
226.60 4.94
226.98 4.02
227.78 3.89
228.29 4.67
228.62 4.29
229.12 4.90
229.46 4.16
229.83 4.48
230.28 3.54
230.50 4.06
230.91 3.03
231.18 3.63
231.79 3.99
231.97 3.45
232.70 4.86
Edit: revised and simplified
Script: (works with gnuplot>=5.2.0)
### a simple gnuplot peak finder
reset session
FILE = "SO28173128.dat"
colX = 1
colY = 2
# extract all peaks
set table $Temp
plot y2=y1=x2=x1=NaN FILE u (x0=x1,x1=x2,x2=column(colX), \
y0=y1,y1=y2,y2=column(colY), \
y0<y1 && y1>=y2 ? sprintf("%g %g",x1,y1) : '') w table
set table $Peaks # remove empty lines and store first values in x0,y0 for later use
plot $Temp u (column(-2)==0 && $0==0?x0=$1:$1):(column(-2)==0 && $0==0?y0=$2:$2) w table
unset table
isNaN(v) = v!=v # check if value is NaN
min(a,b) = isNaN(a) && isNaN(b) ? NaN : isNaN(a) ? b : isNaN(b) ? a : a<b ? a : b # get minimum incl. NaN
# create prominence table
set print $Prominence
do for [n0=0:|$Peaks|-1] {
xn = yn = xp = yp = NaN
stats $Peaks u ($0<n0 && $2>y0 ? (xn=$1,yn=$2) : 0, \
$0>n0 && $2>y0 && yp!=yp ? (xp=$1,yp=$2) : 0, \
$0==n0+1 ? (x1=$1, y1=$2) : 0 ) nooutput
print sprintf("%g %g %g",x0,y0, min(x0-xn,xp-x0))
x0=x1
y0=y1
}
set print
set key noautotitle
set offsets 0,0,1,0
set grid y
# filter Peaks
stats $Prominence u 3 name "P" nooutput # get min, max
Filter(col,t) = (!valid(col) ? 0 : (1-(column(col)-P_min)/(P_max-P_min))*99+1)<=t ? $2 : NaN
Threshold = 100
set label 1 at graph 0.02,0.95 sprintf("Threshold: %g",Threshold)
plot FILE u colX:colY w l lc rgb "blue" ti "Spectrum", \
$Prominence u 1:(Filter(3,Threshold)) w impulses lc rgb "red", \
$Prominence u 1:(Filter(3,Threshold)):1 w labels offset 0,1
### end of script
Results: (for different thresholds)