I have a tuple (h) as follows:
(array([[145, 34, 26, 18, 90, 89],
[ 86, 141, 216, 167, 67, 214],
[ 18, 0, 212, 49, 232, 34],
...,
[147, 99, 73, 110, 108, 9],
[222, 133, 231, 48, 227, 154],
[184, 133, 169, 201, 162, 168]], dtype=uint8), array([[178, 58, 24, 90],
[ 3, 31, 129, 243],
[ 48, 92, 19, 108],
...,
[148, 21, 25, 209],
[189, 114, 46, 218],
[ 15, 43, 92, 61]], dtype=uint8), array([[ 17, 254, 216, ..., 126, 74, 129],
[231, 168, 214, ..., 131, 50, 107],
[ 77, 185, 229, ..., 86, 167, 61],
...,
[105, 240, 95, ..., 230, 158, 27],
[211, 46, 193, ..., 48, 57, 79],
[136, 126, 235, ..., 109, 33, 185]], dtype=uint8))
I converted it into a string s = str(h):
'(array([[ 1, 60, 249, 162, 51, 3],\n [ 57, 76, 193, 244, 17, 238],\n [ 22, 72, 101, 229, 185, 124],\n ...,\n [132, 243, 123, 192, 152, 107],\n [163, 187, 131, 47, 253, 155],\n [ 21, 3, 77, 208, 229, 15]], dtype=uint8), array([[119, 149, 215, 129],\n [146, 71, 121, 79],\n [114, 148, 121, 140],\n ...,\n [175, 121, 81, 71],\n [178, 92, 1, 99],\n [ 80, 122, 189, 209]], dtype=uint8), array([[ 26, 122, 248, ..., 104, 167, 29],\n [ 41, 213, 250, ..., 82, 71, 211],\n [ 20, 122, 4, ..., 152, 99, 121],\n ...,\n [133, 77, 84, ..., 238, 243, 240],\n [208, 183, 187, ..., 182, 51, 116],\n [ 19, 135, 48, ..., 210, 163, 58]], dtype=uint8))'
Now, I want to convert s back to a tuple. I tried using ast.literal_eval(s), but I get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.5/ast.py", line 84, in literal_eval
return _convert(node_or_string)
File "/usr/lib/python3.5/ast.py", line 55, in _convert
return tuple(map(_convert, node.elts))
File "/usr/lib/python3.5/ast.py", line 83, in _convert
raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.Call object at 0x76a6f770>
I could not find this exact solution anywhere. It would be great if someone could help me out.
You can't use str() on numpy arrays (wrapped in tuples or otherwise) and hope to recover the data.
First of all, the ast.literal_eval() function only supports literals and literal displays, not numpy array(...) syntax.
Next, str() on a tuple produces debugging-friendly output; tuples don't implement a __str__ string conversion hook, so their repr() representation is returned instead. Numpy arrays do support str() conversion, but their output is still but a friendly-looking string that omits a lot of detail from the actual values. In your example, those ... ellipsis dots indicate that there is more data in that part of the array, but the strings do not include those values. So you are losing data if you were to try to re-create your arrays from this.
If you need to store these tuples in a file or database column, or need to transmit them over a network connection, you need to serialise the data. Proper serialisation will preserve every detail of the arrays.
For tuples with numpy arrays, you can use pickle.dumps() to produce a bytes object that can be passed back to pickles.loads() to recreate the same value.
You can also convert invidual numpy arrays to a numpy-specific binary format, and load that format again, with the numpy.save() and numpy.load() functions (which operate directly on files, but you can pass in io.BytesIO() objects).
Related
I'm trying to save an array of uint8arrays in an file using node JS, does anyone know how can I do that using fs?
Something like that:
[
Uint8Array(162) [
133, 111, 74, 131, 187, 35, 69, 135, 1, 151, 1, 1,
198, 69, 229, 121, 168, 106, 59, 104, 91, 198, 115, 218,
79, 67, 102, 212, 204, 206, 145, 126, 152, 59, 147, 114,
133, 173, 119, 91, 220, 251, 174, 57, 16, 125, 67, 180,
114, 128, 5, 72, 143, 131, 122, 35, 124, 139, 62, 93,
187, 2, 2, 251, 255, 149, 134, 6, 9, 65, 100, 100,
32, 99, 97, 114, 100, 50, 0, 9, 1, 2, 2, 4,
19, 4, 21, 14, 52, 3, 66, 4, 86, 5, 87, 26,
112, 2, 3, 0,
... 62 more items
],
Uint8Array(170) [
133, 111, 74, 131, 67, 31, 18, 227, 1, 159, 1, 1,
187, 35, 69, 135, 0, 7, 102, 52, 237, 242, 169, 192,
199, 237, 238, 142, 226, 200, 98, 129, 144, 184, 181, 198,
33, 248, 228, 223, 158, 171, 250, 192, 16, 125, 67, 180,
114, 128, 5, 72, 143, 131, 122, 35, 124, 139, 62, 93,
187, 3, 5, 251, 255, 149, 134, 6, 8, 65, 100, 100,
32, 99, 97, 114, 100, 0, 10, 1, 2, 2, 4, 17,
4, 19, 4, 21, 14, 52, 3, 66, 4, 86, 5, 87,
29, 112, 2, 3,
... 70 more items
],
Uint8Array(120) [
133, 111, 74, 131, 71, 17, 199, 138, 1, 110, 1, 67,
31, 18, 227, 131, 113, 252, 222, 71, 172, 34, 205, 40,
96, 11, 236, 242, 153, 43, 182, 4, 136, 135, 36, 6,
79, 52, 223, 123, 188, 42, 7, 16, 125, 67, 180, 114,
128, 5, 72, 143, 131, 122, 35, 124, 139, 62, 93, 187,
4, 8, 251, 255, 149, 134, 6, 11, 68, 101, 108, 101,
116, 101, 32, 99, 97, 114, 100, 0, 10, 1, 2, 2,
2, 17, 2, 19, 2, 52, 1, 66, 2, 86, 2, 112,
2, 113, 2, 115,
... 20 more items
],
Uint8Array(126) [
133, 111, 74, 131, 51, 54, 68, 129, 1, 116, 1, 71,
17, 199, 138, 51, 167, 78, 185, 254, 251, 73, 245, 53,
153, 253, 146, 203, 139, 250, 206, 105, 223, 51, 227, 156,
55, 3, 146, 177, 243, 235, 140, 16, 125, 67, 180, 114,
128, 5, 72, 143, 131, 122, 35, 124, 139, 62, 93, 187,
5, 9, 251, 255, 149, 134, 6, 17, 77, 97, 114, 107,
32, 99, 97, 114, 100, 32, 97, 115, 32, 100, 111, 110,
101, 0, 9, 1, 2, 2, 2, 21, 6, 52, 1, 66,
2, 86, 2, 112,
... 26 more items
]
]
I am simply using the following, for exporting into docx file:
buffer = Uint8Array(86762) [
80, 75, 3, 4, 10, 0, 0, 0, 8, 0, 79, 72,
111, 85, 212, 85, 145, 41, 234, 1, 0, 0, 43, 11,
0, 0, 19, 0, 0, 0, 91, 67, 111, 110, 116, 101,
110, 116, 95, 84, 121, 112, 101, 115, 93, 46, 120, 109,
108, 197, 150, 205, 78, 227, 48, 20, 133, 247, 60, 133,
229, 45, 106, 92, 64, 26, 141, 80, 83, 22, 252, 44,
25, 164, 233, 60, 128, 27, 223, 164, 134, 248, 71, 182,
91, 232, 219, 207, 117, 66, 67, 169, 18, 198, 154, 54,
98, 83, 41, 182,
... 86662 more items
]
fs.writeFileSync('output.docx', buffer)
I suppose restoring it will be just fs.readFileSync(pathToFile)
I'm a Python3 newcomer, and I recently get a strange behavior when printing a set of random integers.
I sometimes get a perfectly sorted set, and sometimes not!
Does somebody know the reason why?
Here is my Python3 code:
import random
n=random.randint(30,90)
print("n=",n)
ens=set()
while len(ens)!=n:
ens.add(random.randint(100,199))
print("len(ens)=",len(ens))
print("ens=",str(ens))
car=input("...?")
Here is one non-sorted resulting text:
n= 84
len(ens)= 84
ens= {128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 147, 148, 150, 151, 152, 154, 156, 157, 158, 160, 161, 162, 163, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 177, 178, 179, 181, 182, 183, 185, 186, 188, 189, 190, 192, 193, 194, 195, 196, 197, 198, 199, 100, 101, 102, 103, 104, 105, 106, 107, 108, 110, 111, 112, 113, 114, 116, 117, 118, 119, 121, 123, 124, 125, 126, 127}
...?
And another sorted resulting text:
n= 86
len(ens)= 86
ens= {100, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 128, 129, 130, 131, 132, 133, 134, 136, 137, 138, 139, 140, 141, 142, 143, 144, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 168, 169, 170, 171, 173, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 186, 188, 189, 191, 192, 195, 196, 197, 198, 199}
...?
I know Python sets are unordered collection of items, but then, why do some of my outputs appear to be perfectly sorted, and some are not?
Just in case it is convincing, I either use Geany IDE, Thonny IDE, or directly Python 3.8 (under Win7 32bit).
Sets are unordered, so when you print your set, python doesn't know what order to print the items, so it prints them in sorted order.
>>> s = {5, 2, 3, 4, 6}
>>> print(s)
{2, 3, 4, 5, 6}
If you want to have the items ordered, use a list.
>>> l = [5, 2, 3, 4, 6]
>>> print(l)
[5, 2, 3, 4, 6]
I have a .mat file and want to convert it into a CV image format such that I can use it for a CNN model.
I am trying to obtain an RGB/ other colored image and not gray.
I tried doing the following(below) but I get a grayscale image, but when I plot the actual mat file using matplotlib it is not grayscale. Also, the .mat file has a px_spacing array apart from the image array. I am not sure how this is helpful.
def mat_to_image(mat_image):
f = loadmat(mat_image,appendmat=True)
image = np.array(f.get('I')).astype(np.float32)
mean = image.mean()
std= image.std()
print(mean, std)
hi = np.max(image)
lo = np.min(image)
image = (((image - lo)/(hi-lo))*255).astype(np.uint8)
im = Image.fromarray(image,mode='RGB')
return im
images=mat_to_image(dir/filename)
cv_img = cv2.cvtColor(np.array(images), cv2.COLOR_GRAY2RGB)
Normally plotting the .mat file fetches a non-grayscale(RGB image)
imgplot= plt.imshow(loadmat(img,appendmat=True).get('I'))
plt.show()
Here is how the mat file looks after print(loadmat('filename'))
{'__header__': b'MATLAB 5.0 MAT-file, Platform: PCWIN64, Created on: Mon Sep 9 11:32:54 2019',
'__version__': '1.0',
'__globals__': [],
'I': array([
[ 81, 75, 74, 75, -11, 14, 49, 37, 29, -24, -183, -349, -581, -740],
[ 51, 33, 67, 36, 1, 42, 30, 49, 47, 42, 14, -85, -465, -727],
[ 23, 31, 36, 20, 54, 70, 44, 56, 56, 79, 62, 19, -204, -595],
[ 7, 12, 36, 47, 59, 68, 74, 56, 59, 100, 74, 34, -3, -353],
[ 23, 19, 51, 87, 86, 79, 91, 76, 96, 95, 52, 51, 74, -141],
[ 18, 51, 54, 97, 93, 94, 98, 83, 119, 71, 36, 69, 50, -16],
[ -10, 5, 53, 92, 69, 87, 103, 114, 118, 77, 51, 68, 30, 0],
[ -24, 11, 74, 80, 49, 68, 106, 129, 107, 63, 57, 70, 39, -1],
[ -45, 43, 83, 69, 43, 64, 98, 108, 90, 35, 27, 55, 31, -13],
[ -9, 32, 83, 78, 66, 106, 89, 85, 58, 43, 31, 39, 28, 7],
[ 45, 35, 76, 45, 51, 84, 55, 66, 49, 41, 39, 28, 13, -7],
[ 85, 67, 61, 45, 69, 53, 23, 32, 31, -12, -34, -182, -376, -425],
[ 136, 93, 71, 54, 30, 39, 17, -21, -29, -43, -101, -514, -792, -816]
], dtype=int16),
'px_spacing': array([[0.78125]])}
Suppose I clustered a data set using sklearn's K-means.
I can see the centroids easily using KMeans.cluster_centers_ but I need to get the clusters as I get centroids.
How can I do that?
You probably look for the attribute labels_.
You need to do the following (see comments in my code):
import numpy as np
from sklearn.cluster import KMeans
from sklearn import datasets
np.random.seed(0)
# Use Iris data
iris = datasets.load_iris()
X = iris.data
y = iris.target
# KMeans with 3 clusters
clf = KMeans(n_clusters=3)
clf.fit(X,y)
#Coordinates of cluster centers with shape [n_clusters, n_features]
clf.cluster_centers_
#Labels of each point
clf.labels_
# !! Get the indices of the points for each corresponding cluster
mydict = {i: np.where(clf.labels_ == i)[0] for i in range(clf.n_clusters)}
# Transform the dictionary into list
dictlist = []
for key, value in mydict.iteritems():
temp = [key,value]
dictlist.append(temp)
RESULTS
{0: array([ 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76,
78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90,
91, 92, 93, 94, 95, 96, 97, 98, 99, 101, 106, 113, 114,
119, 121, 123, 126, 127, 133, 138, 142, 146, 149]),
1: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]),
2: array([ 52, 77, 100, 102, 103, 104, 105, 107, 108, 109, 110, 111, 112,
115, 116, 117, 118, 120, 122, 124, 125, 128, 129, 130, 131, 132,
134, 135, 136, 137, 139, 140, 141, 143, 144, 145, 147, 148])}
[[0, array([ 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76,
78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90,
91, 92, 93, 94, 95, 96, 97, 98, 99, 101, 106, 113, 114,
119, 121, 123, 126, 127, 133, 138, 142, 146, 149])],
[1, array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49])],
[2, array([ 52, 77, 100, 102, 103, 104, 105, 107, 108, 109, 110, 111, 112,
115, 116, 117, 118, 120, 122, 124, 125, 128, 129, 130, 131, 132,
134, 135, 136, 137, 139, 140, 141, 143, 144, 145, 147, 148])]]
It's been very long asked question so I think you already have the answer but let me post as someone can be benefited from it. We can get cluster points by just using its centroid. Scikit-learn has an attribute called cluster_centers_ which returns n_clusters and n_features. The very simple code that you can see it below that to describe the cluster center and please go through all the comments in the code.
import numpy as np
from sklearn.cluster import KMeans
from sklearn import datasets
from sklearn.preprocessing import StandardScaler
# Iris data
iris = datasets.load_iris()
X = iris.data
# Standardization
std_data = StandardScaler().fit_transform(X)
# KMeans clustering with 3 clusters
clf = KMeans(n_clusters = 3)
clf.fit(std_data)
# Coordinates of cluster centers with shape [n_clusters, n_features]
# As we have 3 cluster with 4 features
print("Shape of cluster:", clf.cluster_centers_.shape)
# Scatter plot to see each cluster points visually
plt.scatter(std_data[:,0], std_data[:,1], c = clf.labels_, cmap = "rainbow")
plt.title("K-means Clustering of iris data flower")
plt.show()
# Putting ndarray cluster center into pandas DataFrame
coef_df = pd.DataFrame(clf.cluster_centers_, columns = ["Sepal length", "Sepal width", "Petal length", "Petal width"])
print("\nDataFrame containg each cluster points with feature names:\n", coef_df)
# converting ndarray to a nested list
ndarray2list = clf.cluster_centers_.tolist()
print("\nList of clusterd points:\n")
print(ndarray2list)
OUTPUTS:
This is the output of the above code.
I am trying to make a simple program that displays the palindrome numbers between 2 numbers in Python3 but it doesn't seem to work. I get only 4 output which are 1,2,4 and 8. What about the other numbers like 11, 22, 33, ..., 111, 121,131,..., 191, 222, etc?
Here is my code. I can't figure out why it's not working.
a = 0
b = 500
a += 1
for i in range(a,b):
if(str(a) == str(a)[::-1]):
print(a)
a += a
Maybe
for i in range (0, 500):
if str(i) == str(i)[::-1]:
print(i)
>>> palindromes = [a for a in range(500) if str(a) == str(a)[::-1]]
>>> palindromes
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 22, 33, 44, 55, 66, 77, 88, 99, 101, 111, 121, 131,
141, 151, 161, 171, 181, 191, 202, 212, 222, 232, 242, 252, 262, 272, 282, 292, 303,
313, 323, 333, 343, 353, 363, 373, 383, 393, 404, 414, 424, 434, 444, 454, 464, 474,
484, 494]
That shows you the principle. If you want to do it in a function, you can use yield instead of constructing the whole list in memory (much more efficient):
>>> def palindromes(a, b):
... """Return palindromes in closed interval from a to b"""
... for i in range(a, b):
... if str(i) == str(i)[::-1]:
... yield i
...
>>> list(palindromes(0, 500))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 22, 33, 44, 55, 66, 77, 88, 99, 101, 111, 121, 131, 141, 151, 161, 171, 181, 191, 202, 212, 222, 232, 242, 252, 262, 272, 282, 292, 303, 313, 323, 333, 343, 353, 363, 373, 383, 393, 404, 414, 424, 434, 444, 454, 464, 474, 484, 494]