writing a list of points in a list of files - python-3.x

I have 100 points and I want to devide them to 10 different groups bades on their distance from 10 reference points and write each group in a file.
I write my program as:
from numpy import *
from math import *
from time import *
a=1.0
b=1.0
nx=10 # number of mesh in x
ny=10 # number of mesh in y
dx=a/nx
dy=b/ny
data=loadtxt("cvt_squate.txt",float)
n=data.shape
fids = []
for i in range(n[0]):
ii=str(i)
fids.append(open('file' + ii + '.txt', 'w'))
def calculateDistance(x1,y1,x2,y2):
dist = sqrt((x2 - x1)**2 + (y2 - y1)**2)
return dist
for i in range(nx) :
for j in range(ny) :
distance=10.0
grain=1000
x=(i+0.5)*dx
y=(j+0.5)*dy
for k in range (n[0]):
d = calculateDistance(x,y,data[k,0],data[k,1])
if d<distance:
distance=d
grain=k
print(grain)
kk=str(grain)
outdata = vstack((x,y)).T
savetxt('file' + kk + 'txt', outdata)
But in my results, I have one point in each file instead of group of points.

Without any sample data it's difficult to see how your code is supposed to work. But firstly I'd recommend rewriting you imports to:
import numpy as np
import math
I can't see where you use the time module.
if you define your results as a list outside your For k loop and append all the points which are near the reference points to this list. Something like:
outdata = []
for k in range(n[0]):
d = calculateDistance(x,y,data[k,0],data[k,1])
if d<distance:
outdata.append([data[k,0],data[k,1]])
should get you nearer where you want to be.

Related

How can I interpolate values from two lists (in Python)?

I am relatively new to coding in Python. I have mainly used MatLab in the past and am used to having vectors that can be referenced explicitly rather than appended lists. I have a script where I generate a list of x- and y- (z-, v-, etc) values. Later, I want to interpolate and then print a table of the values at specified points. Here is a MWE. The problem is at line 48:
yq = interp1d(x_list, y_list, xq(nn))#interp1(output1(:,1),output1(:,2),xq(nn))
I'm not sure I have the correct syntax for the last two lines either:
table[nn] = ('%.2f' %xq, '%.2f' %yq)
print(table)
Here is the full script for the MWE:
#This script was written to test how to interpolate after data was created in a loop and stored as a list. Can a list be accessed explicitly like a vector in matlab?
#
from scipy.interpolate import interp1d
from math import * #for ceil
from astropy.table import Table #for Table
import numpy as np
# define the initial conditions
x = 0 # initial x position
y = 0 # initial y position
Rmax = 10 # maxium range
""" initializing variables for plots"""
x_list = [x]
y_list = [y]
""" define functions"""
# not necessary for this MWE
"""create sample data for MWE"""
# x and y data are calculated using functions and appended to their respective lists
h = 1
t = 0
tf = 10
N=ceil(tf/h)
# Example of interpolation without a loop: https://docs.scipy.org/doc/scipy/tutorial/interpolate.html#d-interpolation-interp1d
#x = np.linspace(0, 10, num=11, endpoint=True)
#y = np.cos(-x**2/9.0)
#f = interp1d(x, y)
for i in range(N):
x = h*i
y = cos(-x**2/9.0)
""" appends selected data for ability to plot"""
x_list.append(x)
y_list.append(y)
## Interpolation after x- and y-lists are already created
intervals = 0.5
nfinal = ceil(Rmax/intervals)
NN = nfinal+1 # length of table
dtype = [('Range (units?)', 'f8'), ('Drop? (units)', 'f8')]
table = Table(data=np.zeros(N, dtype=dtype))
for nn in range(NN):#for nn = 1:NN
xq = 0.0 + (nn-1)*intervals #0.0 + (nn-1)*intervals
yq = interp1d(x_list, y_list, xq(nn))#interp1(output1(:,1),output1(:,2),xq(nn))
table[nn] = ('%.2f' %xq, '%.2f' %yq)
print(table)
Your help and patience will be greatly appreciated!
Best regards,
Alex
Your code has some glaring issues that made it really difficult to understand. Let's first take a look at some things I needed to fix:
for i in range(N):
x = h*1
y = cos(-x**2/9.0)
""" appends selected data for ability to plot"""
x_list.append(x)
y_list.append(y)
You are appending a single value without modifying it. What I presume you wanted is down below.
intervals = 0.5
nfinal = ceil(Rmax/intervals)
NN = nfinal+1 # length of table
dtype = [('Range (units?)', 'f8'), ('Drop? (units)', 'f8')]
table = Table(data=np.zeros(N, dtype=dtype))
for nn in range(NN):#for nn = 1:NN
xq = 0.0 + (nn-1)*intervals #0.0 + (nn-1)*intervals
yq = interp1d(x_list, y_list, xq(nn))#interp1(output1(:,1),output1(:,2),xq(nn))
table[nn] = ('%.2f' %xq, '%.2f' %yq)
This is where things get strange. First: use pandas tables, this is the more popular choice. Second: I have no idea what you are trying to loop over. What I presume you wanted was to vary the number of points for the interpolation, which I have done so below. Third: you are trying to interpolate a point, when you probably want to interpolate over a range of points (...interpolation). Lastly, you are using the interp1d function incorrectly. Please take a look at the code below or run it here; let me know what you exactly wanted (specifically: what should xq / xq(nn) be?), because the MRE you provided is quite confusing.
from scipy.interpolate import interp1d
from math import *
import numpy as np
Rmax = 10
h = 1
t = 0
tf = 10
N = ceil(tf/h)
x = np.arange(0,N+1)
y = np.cos(-x**2/9.0)
interval = 0.5
NN = ceil(Rmax/interval) + 1
ip_list = np.arange(1,interval*NN,interval)
xtable = []
ytable = []
for i,nn in enumerate(ip_list):
f = interp1d(x,y)
x_i = np.arange(0,nn+interval,interval)
xtable += [x_i]
ytable += [f(x_i)]
[print(i) for i in xtable]
[print(i) for i in ytable]

Easyviz and scitools.std still works on python3? There is an alternative to making films from a sequence of images?

I am studying the book 'A Primer on Scientific Programming with Python 2nd' by Hans Petter Langtangen. The book uses python2 but I am applying it in python3. The scotools.std library is widely used in the book, but I cannot import or install it in python3. Is there an alternative to scitools.std that works for python3? (This may solve my difficulty in following the book.)
Specifically in this question I'm looking for an alternative to Easyviz, because I can't make a movie with the graph of the Gaussian function by modifying the parameter s as desired in the question.
The python2 code presented in the book is:
from scitools.std import *
import time
def f(x, m, s):
return (1.0/(sqrt(2*pi)*s))*exp(-0.5*((x-m)/s)**2)
m = 0
s_start = 2
s_stop = 0.2
s_values = linspace(s_start,s_stop, 30)
x = linspace(m - 3*s_start, m + 3*s_start, 1000)
# f is max for x=m; smaller s gives larger max value
max_f = f(m, m s_stop)
# Show the movie on the screen
# and make hardcopies of frames simultaneously.
counter = 0
for s in s_values:
y = f(x, m, s)
plot(x, y, axis=[x[0], x[-1], -0.1, max_f],
xlabel='x', ylabel='f', legend='s=%4.2f' % s,
savefig='tmp%04d.png' % counter)
counter += 1
#time.sleep(0.2)
# Make movie file the simplest possible way:
movie('tmp*.png')
My incomplete version in python3 is:
import time
import numpy as np
import matplotlib.pyplot as plt
def f(x, m, s):
return (1.0/(np.sqrt(2*np.pi)*s))*np.exp(-0.5*((x-m)/s)**2)
m = 0
s_start = 2
s_stop = 0.2
s_values = np.linspace(s_start, s_stop, 30)
x = np.linspace(m - 3*s_start, m + 3*s_start, 1000)
# f is max for x=m; smaller s gives larger max value
max_f = f(m, m, s_stop)
# Show the movie on the screen
# and make hardcopies of frames simultaneosly.
counter = 0
for s in s_values:
y = f(x, m, s)
plt.plot(x, y)
plt.xlim(x[0], x[-1])
plt.ylim(-0.1, max_f + 0.1)
plt.xlabel('x')
plt.ylabel('f')
plt.legend('s=%4.2f' % s)
plt.savefig('tmp%04d.png' % counter)
counter += 1
#time.sleep(0.2)
plt.show()
This produces the 30 images correctly, but it does not go ahead and produces the movie.
*Note that I used plt.show () and I have to close 30 windows, if I don't use each generated file it shows the accumulated curves in the same graph.
So I see three ways to solve my problem:
1) Being able to correctly install and import scitools.std (this would be excellent, because the problem goes through the whole book!);
2) Getting an alternative to the scitools.std and Easyviz module;
3) Following the path I adopted in my incomplete version of the code, that is, replacing the command movie ('tmp * .png') presented by the book with something that works well in my code.
Yes, there is a module called scitools3.
You can install by:
pip install scitools3
and reboot your PC.
read more at https://pypi.org/project/scitools3/
/masa

Static Colormap in Scatter Plot during Addition of Datapoints

I would like to add the datapoints to a locked portion of the screen one after the other, save all plots as a *.png and make a gif. Everything is working so far, but I want to lock the colorbar, so that it does not change its range during the addition of the points. I have no idea on how to do tackle this problem...
The input data is (the t_string will be modified to make it work):
x = [2.803480599999999, 5.5502475000000056, 6.984381300000002, 4.115224099999998, 5.746583699999995, 8.971469500000019, 12.028179500000032, 13.451193300000014, 12.457393999999972, 12.027555199999998, 16.077930800000015, 5.021229700000006, 11.206380399999999, 7.903262600000004, 11.98195070000001, 12.21701, 10.35045, 10.231890000000002]
y = [11.961321698938578, 5.218986480632915, 5.211628408660906, 4.847852635777481, 4.936266162218553, 5.233256380128127, 5.441388698929861, 5.461721129728066, 5.722170570613203, 5.2698434785261545, 5.645419662253215, 4.617062894639794, 4.973357261130752, 5.906843248930297, 5.256517482861392, 5.537361432952908, 5.339542403148332, 5.376979880224148]
t_string = ['2019-10-7', '2019-10-13', '2019-11-10', '2019-11-16', '2019-11-17', '2019-11-23', '2019-11-24', '2019-11-27', '2019-12-1', '2019-12-4', '2019-12-8', '2019-12-21', '2019-12-23', '2019-12-25', '2019-12-27', '2020-1-2', '2020-1-5', '2020-1-9']
Below you find the whole code I used. It will create a new directory in your working directory and write all files in there, you will also find the .gif in there. It might be necessary to install some packages. Many thanks in ad
# Import some stuff
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
from datetime import datetime
import pathlib
from pathlib import Path
import moviepy.editor as mpy
# Define Input
x = [2.803480599999999, 5.5502475000000056, 6.984381300000002, 4.115224099999998, 5.746583699999995, 8.971469500000019,
12.028179500000032, 13.451193300000014, 12.457393999999972, 12.027555199999998, 16.077930800000015,
5.021229700000006, 11.206380399999999, 7.903262600000004, 11.98195070000001, 12.21701, 10.35045,
10.231890000000002]
y = [11.961321698938578, 5.218986480632915, 5.211628408660906, 4.847852635777481, 4.936266162218553, 5.233256380128127,
5.441388698929861, 5.461721129728066, 5.722170570613203, 5.2698434785261545, 5.645419662253215, 4.617062894639794,
4.973357261130752, 5.906843248930297, 5.256517482861392, 5.537361432952908, 5.339542403148332, 5.376979880224148]
t_string = ['2019-10-7', '2019-10-13', '2019-11-10', '2019-11-16', '2019-11-17', '2019-11-23', '2019-11-24',
'2019-11-27', '2019-12-1', '2019-12-4', '2019-12-8', '2019-12-21', '2019-12-23', '2019-12-25', '2019-12-27',
'2020-1-2', '2020-1-5', '2020-1-9']
# Define a function to get the datafiles with a certain suffix in a path
def getfile_UI(file_directory, file_suffix):
from glob import glob
path_to_search = file_directory / file_suffix
filenames = glob(str(path_to_search))
return filenames
# Start of script/calculations
t = [mdates.date2num(datetime.strptime(i, "%Y-%m-%d")) for i in t_string]
workingdirectory_projectfolder = Path.cwd().parent
my_dpi = 75
for index, entry in enumerate(t_string):
fig = plt.figure(figsize=(480 / my_dpi, 480 / my_dpi), dpi=my_dpi)
sc = plt.scatter(x[0:index + 1],
y[0:index + 1],
c=t[0:index + 1])
plt.xlim(0, 20)
plt.ylim(4, 7)
plt.title(entry)
plt.xlabel("Distace [km]")
plt.ylabel("Pace [min/km]")
loc = mdates.AutoDateLocator()
fig.colorbar(sc, ticks=loc,
format=mdates.AutoDateFormatter(loc))
filename = 'png_' + str(index) + '.png'
new_dir = workingdirectory_projectfolder / "type 3 gif"
pathlib.Path(new_dir).mkdir(exist_ok=True)
plt.savefig(workingdirectory_projectfolder / "type 3 gif" / filename, dpi=96)
plt.gca()
# Make a GIF from all png files
# http://superfluoussextant.com/making-gifs-with-python.html
fps = 10
gif_name = str(fps) + " fps_""type3_"
workingdirectory_projectfolder = Path.cwd().parent
gif_path = workingdirectory_projectfolder / "type 3 gif" / gif_name
filenamelist_path = workingdirectory_projectfolder / "type 3 gif"
filenamelist_png = getfile_UI(filenamelist_path, "*.png")
list.sort(filenamelist_png, key=lambda x: int(
x.split('_')[1].split('.png')[0])) # Sort the images by #, this may need to be tweaked for your use case
clip = mpy.ImageSequenceClip(filenamelist_png, fps=fps)
clip.write_gif('{}.gif'.format(gif_path), fps=fps)
You could call plt.scatter with vmin=min(t), vmax=max(t). This fixes the limits used for coloring.
Something else you could add in your animation is to only show the tick dates up to the current:
loc = mdates.AutoDateLocator()
cbar = fig.colorbar(sc, ticks=loc, format=mdates.AutoDateFormatter(loc))
ticks = [ti for ti in cbar.get_ticks() if ti <= t[index]]
cbar.set_ticks(ticks)

How to append a loop value to a list?

I don't fathom why the output isn't a list...am I appending wrongly?
from numpy import *
b=0.1;g=0.5;l=632.8;p=2;I1=[I];I=0
for a in arange(-0.2,0.2,0.001):
I+=b**2*(sin(pi/l*b*sin(a)))**2/(pi/l*b*sin(a))**2*(sin(p*pi /l*g*sin(a)))**2/(sin(pi/l*g*sin(a)))**2
I1.append(I)
print (I)
output: 15.999998678557855
Several errors in your code, missing imports etc. See comments inside code for fixes:
from numpy import arange
from math import sin,pi
b = 0.1
g = 0.5
l = 632.8
p = 2
I = 0 # you need to specify I
I1 = [I] # before you can add it
for a in arange(-0.2,0.2,0.001):
I += b**2 * (sin(pi/l*b*sin(a)))**2 / (pi/l*b*sin(a))**2 * (sin(p*pi /l*g*sin(a)))**2 / (sin(pi/l*g*sin(a)))**2
I1.append(I) # by indenting this you move it _inside_ the loop
print (I)
print (I1)
Output:
15.999998678557855
[0, 0.03999999014218294, 0.07999998038139602, 0.1199999707171788, ....] # shortened

How can I make my code call each file in the correct sequence?

I have a folder with 38 files. The names are like this:
AWA_s1_features.mat, AWA_s2_features.mat......AWA_s38_features.mat
Each file is an array with 28 columns but with different # of rows. For example: AWA_s1_features.mat = (139,28), AWA_s2_features.mat = (199, 28) and so on.
As I am doing machine learning I need to join all these files in 1 huge array and label each row. So for the 139 rows of AWA_s1_features.mat there must be 139 1s; for AWA_s2_features.mat there must be 199 2s, and so on until AWA_s38_features.mat which must have a # of 38s.
This is what I mean:
I wrote some code. But I have found that the files are not called in order and therefore the labeling is wrong. For example, AWA_s1_features.mat is not the first file to be called and it has been labeled as 11. AWA_s2_features.mat has been labeled as 21.
So how can I improve my code so that it calls each file in the correct sequence?
Here is the code:
import numpy as np
import scipy.io as sio
import glob
read_files = glob.glob('I:/2D/Features 2D/AWA_s*.mat')
x = np.array([])
y = np.array([])
q = 1
for f in read_files:
l=sio.loadmat(f)['features']
x = np.concatenate((x, l), axis=0) if x.size else l
y_temp = q*np.ones((l.shape[0],1))
y = np.concatenate((y, y_temp), axis=0) if y.size else y_temp
q = q + 1
sio.savemat('AWA_FeaturesAll.mat', {'x':x, 'y':y})
The problem is that the default sorting is alphabetical, meaning that "11" comes before "2". You want numerical sorting and one way would be to use the sorted function with a key parameter, like so:
import numpy as np
import scipy.io as sio
import glob
read_files = glob.glob('I:/2D/Features 2D/AWA_s*.mat')
x = np.array([])
y = np.array([])
q = 1
for f in sorted(read_files, key=lambda f: int(f.split('_')[1][1:])):
l=sio.loadmat(f)['features']
x = np.concatenate((x, l), axis=0) if x.size else l
y_temp = q*np.ones((l.shape[0],1))
y = np.concatenate((y, y_temp), axis=0) if y.size else y_temp
q = q + 1
sio.savemat('AWA_FeaturesAll.mat', {'x':x, 'y':y})

Resources