I am trying to use sklearn nmf on a binary file (.bin) imported via numpy and converted to uint8. I import the file no problem, but it's coming in as a 1D array, and when I try and arrange into a 2D array (which sklearn.NMF requires) it errors. I have imported numpy and sklearn.
Import data:
m1 = np.fromfile('file', dtype='uint8')
Code it errors on (I added the - symbol following advice from the docs, it also errors without the - symbol):
m1.arange(962240400).reshape((31020,-31020))
The error:
AttributeError: 'numpy.ndarray' object has no attribute 'arange'
I have tried looking at the official docs and stack overflow, but nothing seems to be working. If anyone has any ideas as to why my code is wrong that would be great.
Use np.arange(962240400).reshape((31020,-31020)), it is a function of numpy, not a method of the array m1
use arange in place of arrange.there should only one 'r'
Related
I am trying to convert "KerasTensor" into numpy array. I have tried converting KerasTensor to tf.Tensor (with no luck). I have also tried using tensor.numpy(), tensor.eval() and keras.backend.eval(tensor) all of that have not worked. Trying ".numpy()" and ".eval()" I am getting AttributeError: 'KerasTensor' object has no attribute 'numpy' error. How do I convert extracted KerasTensor to numpy array or to EagerTensor so I can use .numpy() method ?
Tensorflow version: 2.8.0
Keras version: 2.8.0
Thanks for help
Edit (Additional info): Model is build using keras functional API. After fit() I am extracting encoded input by: encoded = model.get_layer("encoder_output").output After that I've tried converting the "encoded" KerasTensor like I've described above and it does not work.
There is no value to convert to numpy.
You need an input to have an output.
In keras, the best to do is to build a submodel.
submodel = Model(original_model.inputs, original_model.get_layer("encoder_output").output)
results = submodel.predict(numpy_input)
I'm currently trying to perform a Polar to Cartesian Coordinate Image transformation, to display a raw sonar image into a 'fan-display'.
Initially I have a Numpy Array image of type np.float64, that can be seen below:
After doing some searching, I came across this StackOverflow post Inverse transform an image from Polar to Cartesian in OpenCV with a very similar problem, in which the poster seemed to have solved his/her issue by using the Python Wand library (http://docs.wand-py.org/en/0.5.9/index.html), specifically using their set of Distortion functions.
However, when I tried to use Wand and read the image in, I instead ended up with Wand getting the image below, which seems to be smaller than the original one. However, the weird thing is that img.size still gives the same size number as the original image's shape.
The code for this transformation can be seen below:
print(raw_img.shape)
wand_img = Image.from_array(raw_img.astype(np.uint8), channel_map="I") #=> (369, 256)
display(wand_img)
print("Current image size", wand_img.size) #=> "Current image size (369, 256)"
This is definitely quite problematic as Wand will automatically give the wrong 'fan image'. Is anybody familiar with this kind of problem with the Wand library previously, and if yes, may I ask what is the recommended solution to fix this issue?
If this issue isn't resolved soon I have an alternative backup of using OpenCV's cv::remap function (https://docs.opencv.org/4.1.2/da/d54/group__imgproc__transform.html#ga5bb5a1fea74ea38e1a5445ca803ff121). However the problem with this is that I'm not sure what mapping arrays (i.e. map_x and map_y) to use to perform the Polar->Cartesian transformation, as using a mapping matrix that implements the transformation equations below:
r = polar_distances(raw_img)
x = r * cos(theta)
y = r * sin(theta)
didn't seem to work and instead threw out errors from OpenCV as well.
Any kind of help and insight into this issue is greatly appreciated. Thank you!
- NickS
EDIT I've tried on another image example as well, and it still shows a similar problem. So first, I imported the image into Python using OpenCV, using these lines of code:
import matplotlib.pyplot as plt
from wand.image import Image
from wand.display import display
import cv2
img = cv2.imread("Test_Img.jpg")
img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
plt.figure()
plt.imshow(img_rgb)
plt.show()
which showed the following display as a result:
However, as I continued and tried to open the img_rgb object with Wand, using the code below:
wand_img = Image.from_array(img_rgb)
display(img_rgb)
I'm getting the following result instead.
I tried to open the image using wand.image.Image() on the file directly, which is able to display the image correctly when using display() function, so I believe that there isn't anything wrong with the wand library installation on the system.
Is there a missing step that I required to convert the numpy into Wand Image that I'm missing? If so, what would it be and what is the suggested method to do so?
Please do keep in mind that I'm stressing the conversion of Numpy to Wand Image quite crucial, the raw sonar images are stored as binary data, thus the required use of Numpy to convert them to proper images.
Is there a missing step that I required to convert the numpy into Wand Image that I'm missing?
No, but there is a bug in Wand's Numpy implementation in Wand 0.5.x. The shape of OpenCV's ndarray is (ROWS, COLUMNS, CHANNELS), but Wand's ndarray is (WIDTH, HEIGHT, CHANNELS). I believe this has been fixed for the future 0.6.x releases.
If so, what would it be and what is the suggested method to do so?
Swap the values in img_rgb.shape before passing to Wand.
img_rgb.shape = (img_rgb.shape[1], img_rgb.shape[0], img_rgb.shape[2],)
with Image.from_array(img_rgb) as img:
display(img)
I'm starting learning basic feature extraction with librosa and was trying reading and storing ten kick drums with pathlib, but it doesn't work since I always getting an encoding error, where as there is no error without pathlib.
I tried changing the path, updating every imported library very often, using wav instead of mp3 but had no further idea.
My code:
%matplotlib inline
from pathlib import Path
import numpy, scipy, matplotlib.pyplot as plt, sklearn, urllib, IPython.display as ipd
import librosa, librosa.display
kick_signals = [
librosa.load(p)[0] for p in Path().glob('audio/drum_samples/train/kick_*.mp3')
]
Error messages:
RuntimeError: Error opening 'audio/techno-nine_o_three.mp3': File contains data in an unknown format.
and
AttributeError: 'PosixPath' object has no attribute 'encode'
I would be very thankful, if you would and could help me.
You can convert the PossixPath object to a string, using p.as_posix()
Example:
p = Path(file_path)
p.as_posix()
Try:
kick_signals = [
librosa.load(p.absolute())[0] for p in Path().glob('audio/drum_samples/train/kick_*.mp3')
]
That way you pass a string instead of a PosixPath to librosa.
If that does not fix it, check your mp3 file. Does it play in a regular player? If not, please post the whole error message (stacktrace). Perhaps librosa's dependencies aren't installed properly.
I'm trying to run inference using tf.lite on an mnist keras model that I optimized by doing post-training-quantization according to this
RuntimeError: There is at least 1 reference to internal data
in the interpreter in the form of a numpy array or slice. Be sure to
only hold the function returned from tensor() if you are using raw
data access.
It happens after I resize either the images to be in 4 dimensions, or the interpreter itself as seen in the commented line; since the error before this was something like "expected 4 dimensions but found 3". Here is the code:
import tensorflow as tf
tf.enable_eager_execution()
import numpy as np
from tensorflow.keras.datasets import mnist
import matplotlib.pyplot as plt
%matplotlib inline
mnist_train, mnist_test = tf.keras.datasets.mnist.load_data()
images, labels = tf.cast(mnist_test[0], tf.float32)/255.0, mnist_test[1]
images = np.reshape(images,[images.shape[0],images.shape[1],images.shape[2],1])
mnist_ds = tf.data.Dataset.from_tensor_slices((images, labels)).batch(1, drop_remainder = True)
interpreter = tf.lite.Interpreter(model_path="C:\\Users\\USER\\Documents\\python\\converted_quant_model_cnn_5_100.tflite")
#tf.lite.Interpreter.resize_tensor_input(interpreter, input_index="index" , tensor_size=([1,28,28,1]) )
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
input_index = interpreter.get_input_details()[0]["index"]
output_index = interpreter.get_output_details()[0]["index"]
for img, label in mnist_ds.take(1):
break
#print(img.get_shape)
interpreter.set_tensor(input_index, img)
interpreter.invoke()
predictions = interpreter.get_tensor(output_index)
I was facing the same issue while running inference on a tflite model.
When traced back, I ended up reading the function in which this runtime error occurs.
The functions responsible for this raising this error are:
def _ensure_safe(self)
and
def _safe_to_run(self)
The function "_safe_to_run()" is called from within the function "_ensure_safe()".
_safe_to_run() function either returns True of False. When it return False the above runtime error occurs.
It returns False when there exist numpy array buffers. This means it is not safe to run tflite calls that may destroy (or alter) internally allocated memory.
So for "_ensure_safe()" function to not raise this runtime error we have to make sure that no numpy arrays pointing to internal buffers are active.
Also, for more clarity note that the function "_ensure_safe()" should be called from any function that will call a function on _interpreter that may reallocate memory. Thus when you call the function
interpreter.allocate_tensors()
as you have mentioned in the code above, the first thing that this "interpreter.allocate_tensors()" function does internally is call the "_ensure_safe()" funciton as the "interpreter.allocate_tensors()" involves altering the internal allocated memory (in this case altering means "allocating" as the name suggests). The other example where "_ensure_safe()" is also called is when "invoke()" function is called. And there are many such functions, but you get the idea.
Now that the root cause and working is known, to overcome this runtime error i.e to have no numpy arrays pointing to internal buffers, we have to clear them.
To clear them:
a). Either shutdown you jupyter notebook and restart the kernel, as this will clear all numpy arrays/slices
b). Or simply load the model again i.e run this line again in you jupyter notebook:
interpreter = tf.lite.Interpreter(model_path="C:\\Users\\USER\\Documents\\python\\converted_quant_model_cnn_5_100.tflite")
This hopefully solves your problem, I assure you it did for me.
If both of these options does not, then in the above explanation I have pointed out "why" this error occurs. So if you find out other ways of "having no numpy arrays pointing to internal buffers", do share.
Reference: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/python/interpreter.py
Just to add what solved it for me. I am using scripts, so it is not related to Jupyter Notebooks.
My problem was that I was using predictions = interpreter.tensor(output_index)
instead predictions = interpreter.get_tensor(output_index).
However, the problem appeared as the same error commented in this thread.
I copy the interpreter.tensor object, then it works, hope it helps you!
change
interpreter.set_tensor(input_index, test2)
interpreter.invoke()
output = interpreter.tensor(output_h1)
result_h1 = np.reshape(output(), (224,224))
to
import copy
interpreter.set_tensor(input_index, test2)
interpreter.invoke()
output = interpreter.tensor(output_h1)
result_h1 = np.reshape(copy.copy(output()), (224,224))
I am using scripts, and for me the problem was multiple instances of the same script running at the same time. Killing the instances solved the issue
What solved the issue for me was to avoid calling
interpreter.get_signature_runner()
to get the signature details if the intent is to use set_tensor and invoke for inference.
I installed pyspc and run on Jupyter Notebook successfully when using original samples.
But when I tried introducing a self defined nested list and an error message showed up.
pyspc library: https://github.com/carlosqsilva/pyspc
from pyspc import*
import numpy
abc=[[2,3,4],[4,5.6],[1,4,5],[3,4,4],[4,5,6]]
a=spc(abc)+xbar_rbar()+rules()+rbar()
print(a)
error message for AssertionError
Thank you for advise where went wrong and how to fix it.
Check the data you have accidentally used the . instead of , for value [4,5.6], second element of the list.
Here is the corrected data
abc=[[2,3,4],[4,5,6],[1,4,5],[3,4,4],[4,5,6]]
Hope this will help.