I want to make an application that guesses the name of the selected food image. I used yolov5. Output is correct on test images in colab.
I'm testing with the same image.
https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb
I created the yolov5s-fp16.lite file from here.
https://www.youtube.com/watch?v=tySgZ1rEbW4
I followed this video for it to work on android platform.
my tf lite file on android
try {
Yolov5sFp16 model = Yolov5sFp16.newInstance(context);
// Creates inputs for reference.
TensorBuffer inputFeature0 = TensorBuffer.createFixedSize(new int[]{1, 640, 640, 3}, DataType.FLOAT32);
inputFeature0.loadBuffer(byteBuffer);
// Runs model inference and gets result.
Yolov5sFp16.Outputs outputs = model.process(inputFeature0);
TensorBuffer outputFeature0 = outputs.getOutputFeature0AsTensorBuffer();
// Releases model resources if no longer used.
model.close();
} catch (IOException e) {
// TODO Handle the exception
}
predictBtn
predictBtn.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View view){
try {
Yolov5sFp16 model = Yolov5sFp16.newInstance(MainActivity.this);
// Creates inputs for reference.
TensorBuffer inputFeature0 = TensorBuffer.createFixedSize(new int[]{4, 640, 640, 3}, DataType.FLOAT32);
Log.d("shape", inputFeature0.toString());
bitmap = Bitmap.createScaledBitmap(bitmap, 640 ,640,true );
inputFeature0.loadBuffer(TensorImage.fromBitmap(bitmap).getBuffer());
// Runs model inference and gets result.
Yolov5sFp16.Outputs outputs = model.process(inputFeature0);
TensorBuffer outputFeature0 = outputs.getOutputFeature0AsTensorBuffer();
result.setText(labels[getMax(outputFeature0.getFloatArray())]+" ");
// Releases model resources if no longer used.
model.close();
} catch (IOException e) {
// TODO Handle the exception
}
}
});
}
ERROR
E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.myapplicationdeploy, PID: 6744
java.lang.IllegalArgumentException: The size of byte buffer and the shape do not match.
at org.tensorflow.lite.support.common.SupportPreconditions.checkArgument(SupportPreconditions.java:104)
at org.tensorflow.lite.support.tensorbuffer.TensorBuffer.loadBuffer(TensorBuffer.java:309)
at org.tensorflow.lite.support.tensorbuffer.TensorBuffer.loadBuffer(TensorBuffer.java:328)
at com.example.myapplicationdeploy.MainActivity$3.onClick(MainActivity.java:108)
at android.view.View.performClick(View.java:6597)
at com.google.android.material.button.MaterialButton.performClick(MaterialButton.java:1194)
at android.view.View.performClickInternal(View.java:6574)
at android.view.View.access$3100(View.java:778)
at android.view.View$PerformClick.run(View.java:25885)
at android.os.Handler.handleCallback(Handler.java:873)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:193)
at android.app.ActivityThread.main(ActivityThread.java:6669)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)
When I debug, it gives an error in the buffer part. What should I do? What should the dimensions be?
When using the Alloy API to perform parsing on the following string:
Module mod = CompUtil.parseEverything_fromString("run {} for 5")
I receive the following error:
Fatal error:
Parser Exception
at edu.mit.csail.sdg.alloy4compiler.parser.CompParser.alloy_parseStream(CompParser.java:2260)
at edu.mit.csail.sdg.alloy4compiler.parser.CompUtil.parseRecursively(CompUtil.java:178)
at edu.mit.csail.sdg.alloy4compiler.parser.CompUtil.parseEverything_fromFile(CompUtil.java:280)
at edu.mit.csail.sdg.alloy4compiler.parser.CompUtil.parseEverything_fromString(CompUtil.java:333)
at edu.mit.csail.sdg.alloy4compiler.parser.CompUtil$parseEverything_fromString.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:136)
at MyOM.MyOMPlugin.Modularize(MyOMPlugin.groovy:107)
...
--------------------------------------------------------------------------
class file:
package MyOM
class MyOMPlugin extends Plugin{
void Modularize() throws Err{
//line 107 is the only line in the method and is below
Module mod = CompUtil.parseEverything_fromString("run {} for 5")
}
}
I am performing this task in a groovy class file within a groovy project. The code works fine until I run it in conjunction with the plugin I am using.
EDIT:
The parse function is what causes the parse exception and the code for that method is included below for reference:
#Override public Symbol parse() throws java.lang.Exception {
int act; // current action code
Symbol lhs_sym = null; // the Symbol/stack element returned by a reduce
short handle_size, lhs_sym_num; // information about production being reduced with
boolean logging = "yes".equals(System.getProperty("debug"));
production_tab = production_table();
action_tab = action_table();
reduce_tab = reduce_table();
init_actions();
user_init();
// start
cur_token = scan();
stack.removeAllElements();
stack.push(getSymbolFactory().startSymbol("START", 0, start_state()));
tos = 0;
for (_done_parsing = false; !_done_parsing; ) {
act = get_action(((Symbol)stack.peek()).parse_state, cur_token.sym);
if (act > 0) { // "shift"; thus, we shift to the encoded state by pushing it on the stack
if (logging) System.out.println("shift " + cur_token.sym);
cur_token.parse_state = act-1;
stack.push(cur_token);
tos++;
cur_token = scan();
} else if (act<0) { // "reduce"
if (logging) System.out.println("reduce " + ((-act)-1));
lhs_sym = do_action((-act)-1, this, stack, tos);
lhs_sym_num = production_tab[(-act)-1][0];
handle_size = production_tab[(-act)-1][1];
for (int i = 0; i < handle_size; i++) { stack.pop(); tos--; }
act = get_reduce(((Symbol)stack.peek()).parse_state, lhs_sym_num);
lhs_sym.parse_state = act;
stack.push(lhs_sym);
tos++;
} else { // "error"
if (logging) System.out.println("error");
syntax_error(cur_token);
done_parsing();
}
}
return lhs_sym;
}
I'm trying to use GStreamer Java Binding on Raspberry Pi to play some dynamic audio data which is Opus encoded. I'm almost done. My final issue is I cannot have GStreamer clean up nicely when the data reaching the end. My testing code is as below.
package org.freedesktop.gstreamer.examples;
import java.awt.Dimension;
import java.awt.EventQueue;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.nio.ByteBuffer;
import javax.swing.JFrame;
import org.freedesktop.gstreamer.Bin;
import org.freedesktop.gstreamer.Buffer;
import org.freedesktop.gstreamer.Bus;
import org.freedesktop.gstreamer.Element;
import org.freedesktop.gstreamer.ElementFactory;
import org.freedesktop.gstreamer.FlowReturn;
import org.freedesktop.gstreamer.Gst;
import org.freedesktop.gstreamer.GstObject;
import org.freedesktop.gstreamer.Message;
import org.freedesktop.gstreamer.Pad;
import org.freedesktop.gstreamer.PadLinkReturn;
import org.freedesktop.gstreamer.Pipeline;
import org.freedesktop.gstreamer.State;
import org.freedesktop.gstreamer.elements.AppSrc;
import org.freedesktop.gstreamer.lowlevel.MainLoop;
/**
*
* #author Superrei
*/
public class AudioTest {
//private static final int BUFFER_SIZE = 4096;
private static byte[] soundBytes = null;
private static int pointer = 0;
private static String filename = null;
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
Gst.init("AudioTest", args);
filename = args[0];
EventQueue.invokeLater(new Runnable() {
#Override
public void run() {
File soundFile = new File(filename);
FileInputStream inStream = null;
if (soundFile.exists()){
try {
System.out.println("Read media file.");
long fileSize = soundFile.length();
soundBytes = new byte[(int)fileSize];
inStream = new FileInputStream(soundFile);
int byteCount = inStream.read(soundBytes);
System.out.println("Number of byte read: " + byteCount);
} catch (IOException e) {
e.printStackTrace();
return;
}
}
final MainLoop loop = new MainLoop();
AppSrc source = (AppSrc)ElementFactory.make("appsrc", "app-source");
Element demuxer = ElementFactory.make("oggdemux", "ogg-demuxer");
final Element decoder = ElementFactory.make("opusdec", "opus-decoder");
Element conv = ElementFactory.make("audioconvert", "converter");
Element sink = ElementFactory.make("autoaudiosink", "audio-output");
//source.set("location", "/home/pi/gstJavaTest/transcript.ogg");
Pipeline pipe = new Pipeline();
Bus bus = pipe.getBus();
bus.connect(new Bus.EOS() {
#Override
public void endOfStream(GstObject source) {
loop.quit();
}
});
bus.connect(new Bus.ERROR() {
#Override
public void errorMessage(GstObject source, int code, String message) {
System.out.println("Error detected");
System.out.println("Error source: " + source.getName());
System.out.println("Error code: " + code);
System.out.println("Message: " + message);
loop.quit();
}
});
pipe.addMany(source, demuxer, decoder, conv, sink);
source.link(demuxer);
source.set("emit-signals", true);
source.connect(new AppSrc.NEED_DATA() {
private ByteBuffer bb = ByteBuffer.wrap(soundBytes);
#Override
public void needData(AppSrc elem, int size) {
if (bb.hasRemaining()) {
// TODO Auto-generated method stub
System.out.println("needData: size = " + size);
byte[] tempBuffer;
Buffer buf;
if (bb.remaining() > size) {
tempBuffer = new byte[size];
buf = new Buffer(size);
} else {
tempBuffer = new byte[bb.remaining()];
buf = new Buffer(bb.remaining());
}
//System.out.println("Buffer size: " + buf.map(true).remaining());
//System.arraycopy(soundBytes, pointer, tempBuffer, 0, size);
bb.get(tempBuffer);
System.out.println("Temp Buffer remaining bytes: " + bb.remaining());
buf.map(true).put(ByteBuffer.wrap(tempBuffer));
elem.pushBuffer(buf);
} else {
elem.emit("end-of-stream", elem);
}
//pointer += size;
}
});
source.connect(new AppSrc.ENOUGH_DATA() {
#Override
public void enoughData(AppSrc elem) {
System.out.println("enoughData: " + elem.toString());
}
});
source.connect(new AppSrc.END_OF_STREAM() {
#Override
public FlowReturn endOfStream(AppSrc elem) {
// TODO Auto-generated method stub
return FlowReturn.OK;
}
});
Element.linkMany(decoder, conv, sink);
demuxer.connect(new Element.PAD_ADDED() {
#Override
public void padAdded(Element element, Pad pad) {
System.out.println("Dynamic pad created, linking demuxer/decoder");
System.out.println("Pad name: " + pad.getName());
System.out.println("Pad type: " + pad.getTypeName());
Pad sinkPad = decoder.getStaticPad("sink");
//boolean linked = element.link(decoder);
PadLinkReturn ret = pad.link(sinkPad);
if (ret.equals(PadLinkReturn.OK)){
System.out.println("Pad linked.");
} else {
System.out.println("Pad link failed");
}
//System.out.println("Pad Link Return: " + ret.toString());
}
});
System.out.println("Now Playing...");
pipe.play();
System.out.println("Running...");
loop.run();
System.out.println("Returned, stopping playback");
pipe.stop();
}
});
}
}
I don't use playbin since the audio data will come in byte stream finally.
The test runs well until the end. When I reach the end of file. I get error messages and follow by JVM crush. The error message is like this:
(AudioTest:4643): GLib-GObject-CRITICAL **: g_object_unref: assertion 'G_IS_OBJECT (object)' failed
(AudioTest:4643): GLib-GObject-CRITICAL **: g_object_unref: assertion 'G_IS_OBJECT (object)' failed Returned, stopping playback
(AudioTest:4643): GLib-GObject-CRITICAL **: g_object_ref: assertion 'G_IS_OBJECT (object)' failed
(AudioTest:4643): GLib-GObject-CRITICAL **: g_object_unref: assertion 'G_IS_OBJECT (object)' failed
(AudioTest:4643): GLib-GObject-CRITICAL **: g_object_ref: assertion 'G_IS_OBJECT (object)' failed
(AudioTest:4643): GLib-GObject-CRITICAL **: g_object_unref: assertion 'G_IS_OBJECT (object)' failed
(AudioTest:4643): GLib-GObject-CRITICAL **: g_value_set_object: assertion 'G_IS_OBJECT (v_object)' failed
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x63d8b8ec, pid=4643, tid=1658819680
#
# JRE version: Java(TM) SE Runtime Environment (8.0_65-b17) (build
1.8.0_65-b17)
# Java VM: Java HotSpot(TM) Client VM (25.65-b01 mixed mode linux-arm )
# Problematic frame:
# C [libglib-2.0.so.0+0x928ec] g_rec_mutex_lock+0x8
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/pi/gstJavaTest/hs_err_pid4643.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
# Aborted
I cannot find any example on using AppSrc to play something with an end. Any help will be appreciated.
I fixed the problem. The solution is using AppSrc.endOfStream() method rather than emitting a signal. Also I removed the callback for AppSrc.END_OF_STREAM, since it is not necessary. So my callback for needData is like this:
#Override
public void needData(AppSrc elem, int size) {
if (bb.hasRemaining()) {
System.out.println("needData: size = " + size);
byte[] tempBuffer;
Buffer buf;
int copyLength = (bb.remaining() >= size) ? size : bb.remaining();
tempBuffer = new byte[copyLength];
buf = new Buffer(copyLength);
//System.arraycopy(soundBytes, pointer, tempBuffer, 0, copyLength);
System.out.println("Buffer size: " + buf.map(true).remaining());
bb.get(tempBuffer);
System.out.println("Temp Buffer remaining bytes: " + bb.remaining());
buf.map(true).put(ByteBuffer.wrap(tempBuffer));
elem.pushBuffer(buf);
//pointer += size;
} else {
elem.endOfStream();
}
}
I have another question about a "JDWP Error: 21" logged: Unexpected JDWP Error 21
I am trying to parse some XML I gather from a servlet into a J2ME MIDlet with the code below. So far unsuccessfully, I think due to the JDWP error on my HttpConnection how ever the InputStream is filled with XML and once parsed everything inside the Document is null.
Has anyone got and ideas about the JDWP error, and also does that code look like it should work?
In MIDlet I am using JSR-172 API javax.xml.parsers.*.
if(d==form1 && c==okCommand)
{
// Display Webpage
webPage = new TextBox(txtField.getString(),
"",
100000,
TextField.ANY
);
Display.getDisplay(this).setCurrent(webPage);
try
{
HttpConnection conn = (HttpConnection)Connector.open("http://localhost:8080/Blogging_Home/Interface?Page=Bloggers&Action=VIEW&Type=XML");
conn.setRequestMethod(HttpConnection.GET);
int rc = conn.getResponseCode();
getConnectionInformation(conn, webPage);
webPage.setString(webPage.getString() + "Starting....");
String methodString = getStringFromURL("");
if (rc == HttpConnection.HTTP_OK)
{
InputStream is = null;
is = createInputStream(methodString);
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder;
Document document;
try
{
builder = factory.newDocumentBuilder();
document = builder.parse(is);
} catch (Exception e) {
e.printStackTrace();
}
}
else
{
webPage.setString(webPage.getString() + "ERROR:" + rc);
}
}
catch(Exception ex)
{
// Handle here
webPage.setString("Error" + ex.toString());
}
So I have the following code:
public class Minesweeper extends MIDlet implements CommandListener {
public static String error = "";
public void startApp() throws MIDletStateChangeException {
Display display = Display.getDisplay(this);
canvas = new MCanvas();
canvas.addCommand(exitCommand);
canvas.addCommand(okCommand);
canvas.addCommand(newCommand);
canvas.setCommandListener(this);
try{
display.setCurrent(canvas);
} catch (Exception e) {
error = e.toString();
}
}
}
When I leave display.setCurrent(canvas); outside of the try block, the app fails with a NullPointerException. When I comment out that line, the app works (although obviously no canvas is added). So the error is caused by that line, or something that that line causes.
So I suround that line with try/catch. Although the error is caused by that line, the error still happens when the line is surrounded by try/catch. How can I catch the error? (I've tried this using Throwable as well, and it is still not caught.
MCanvas:
import javax.microedition.lcdui.Canvas;
import javax.microedition.lcdui.Font;
import javax.microedition.lcdui.Graphics;
public class MCanvas extends Canvas {
protected void paint(Graphics g){
//Minesweeper.p("repaint");
Space[] data = Minesweeper.topaint;
for(int x=0; x<data.length; x++){
data[x].print();
int r = data[x].row * 10;
int c = data[x].col * 10;
int v = data[x].value;
String s = "";
//System.out.println("r:"+Integer.toString(r)+" c:"+Integer.toString(c)+" s:"+Integer.toString(v));
g.setColor(250, 0, 0);
//Minesweeper.p("if");
if(data[x].open){
switch(v){
case 0:
g.setColor(50, 50, 50);
break;
case 1:
g.setColor(100, 50, 50);
s = "1";
break;
case 2:
g.setColor(150, 50, 50);
s = "2";
break;
case 3:
g.setColor(200, 50, 50);
s = "3";
break;
case 4:
g.setColor(250, 50, 50);
s = "4";
break;
case 5:
g.setColor(250, 100, 100);
s = "5";
break;
case 6:
g.setColor(250, 125, 125);
s = "6";
break;
case 7:
g.setColor(250, 150, 150);
s = "7";
break;
case 8:
g.setColor(250, 175, 175);
s = "8";
break;
case 9:
g.setColor(250, 200, 200);
break;
default:
g.setColor(250, 100, 100);
}
} else {
g.setColor(0,0,0);
}
g.fillRect(c, r, 10, 10);
g.setColor(250, 250, 250);
Font font = Font.getFont(Font.FACE_SYSTEM, Font.STYLE_PLAIN, Font.SIZE_SMALL);
g.setFont(font);
g.drawString(s, c+5, r+8, Graphics.HCENTER | Graphics.BASELINE);
if(data[x].hover){
g.setColor(250, 250, 250);
g.drawLine(c, r, c, r+9);
g.drawLine(c, r, c+9, r);
g.drawLine(c+9, r, c+9, r+9);
g.drawLine(c, r+9, c+9, r+9);
}
//Minesweeper.p("here?");
}
//Minesweeper.p("here");
//Minesweeper.p(Minesweeper.error);
if(Minesweeper.error != null){
g.drawString(Minesweeper.error, 10, 10, Graphics.HCENTER | Graphics.BASELINE);
}
Minesweeper.p("msg:"+Minesweeper.message);
g.setColor(0, 0, 0);
Font font = Font.getFont(Font.FACE_SYSTEM, Font.STYLE_PLAIN, Font.SIZE_LARGE);
g.setFont(font);
g.drawString(Minesweeper.message, this.getWidth()/2, this.getHeight()-10, Graphics.HCENTER | Graphics.BASELINE);
Font fontsm = Font.getFont(Font.FACE_SYSTEM, Font.STYLE_PLAIN, Font.SIZE_SMALL);
g.setFont(fontsm);
}
protected void keyPressed(int keyCode) {
int gameaction = getGameAction(keyCode);
int c = Minesweeper.selected.col;
int r = Minesweeper.selected.row;
switch (gameaction) {
case UP:
Minesweeper.p("UP");
if(r>0){
Minesweeper.selected.leavehere();
Minesweeper.getSpace(Minesweeper.selected.row - 1, Minesweeper.selected.col).gohere();
}
break;
case DOWN:
Minesweeper.p("DOWN");
if(r<Minesweeper.height-1){
Minesweeper.selected.leavehere();
Minesweeper.getSpace(Minesweeper.selected.row + 1, Minesweeper.selected.col).gohere();
}
break;
case LEFT:
Minesweeper.p("LEFT");
if(c>0){
Minesweeper.selected.leavehere();
Minesweeper.getSpace(Minesweeper.selected.row, Minesweeper.selected.col - 1).gohere();
}
break;
case RIGHT:
Minesweeper.p("RIGHT");
if(c<Minesweeper.length-1){
Minesweeper.selected.leavehere();
Minesweeper.getSpace(Minesweeper.selected.row, Minesweeper.selected.col + 1).gohere();
}
break;
}
repaint();
}
}
As explained in Display.setCurrent API javadocs,
...The setCurrent() method returns immediately, without waiting for the change to take place...
Because of above, exceptions that may occur in calls triggered by setCurrent may (and most likely will) slip through your try-catch.
To be able to catch and report such exceptions, one should study what calls are triggered by setCurrent (in your case, these are explained in API javadocs for Canvas, Event Delivery section), cover these by try-catch blocks where appropriate and design the appropriate way to report exceptions if these occur.
In your case, try-catch could likely surround code in MCanvas.paint (this is where NPE likely occurs) and exceptions could be reported for example by showing appropriate screen with error message (eg Alert) by invoking setCurrent for that screen from catch block.
If I were you I would continue to insert try/catch blocks in the MCanvas class.
It is difficult for anyone to figure out where your NullPointerException occurs without seeing more of the code.
The only question I can come up with, from the code you pasted so far, is:
Does your Minesweeper class contain a static array of object Space called topaint? Did you declare it but maybe forgot to fill it with data?
Space[] topaint = new Space[20]; // Declared, but nothing in it yet.
Trying to access Minesweeper.topaint[0] will give a NullPointerException, unless you also do
topaint[0] = new Space();