I am using the Microsoft azure Speech to text REST API.
According to the docs, Ogg and Wav formats are supported in REST API. But, when I am sending the request for OGG audio file getting 400- Bad request error.
I am using following code to prepare request, And this is working for WAV audio format:
String url= "https://westus.stt.speech.microsoft.com/speech/recognition/conversation/cognitiveservices/v1?language=en-US&format=simple";
private void connect(String extension) throws IOException {
connection = (HttpURLConnection) new URL(url).openConnection();
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setRequestMethod("POST");
if (extension.equalsIgnoreCase(Constants.WAV))
connection.setRequestProperty("Content-type", "audio/wav; codecs=\"audio/pcm\"; samplerate=16000");
else if (extension.equalsIgnoreCase(Constants.OGG))
connection.setRequestProperty("Content-type", "audio/ogg; codecs=\"audio/opus\"");
connection.setRequestProperty("Accept", "application/json;text/xml");
connection.setRequestProperty("Ocp-Apim-Subscription-Key", subscriptionKey);
connection.setRequestProperty("Transfer-Encoding", "chunked");
connection.setRequestProperty("Expect", "100-continue");
connection.setChunkedStreamingMode(0); // 0 == default chunk size
connection.connect();
}
Using this for uploading file:
private void upload(InputStream inputStream) throws IOException {
try (OutputStream output = connection.getOutputStream()) {
byte[] buffer = new byte[1024];
int length;
while ((length = inputStream.read(buffer)) != -1) {
output.write(buffer, 0, length);
}
output.flush();
}}
API is working fine for both WAV and OGG format. The request is also looking fine. The problem may be with the audio file sending in the request.
If you want to use OGG audio file format then the audio file must have the following properties:
OGG (Codec: Opus, Bitrate: 16-bit, Sample rate: 16 kHz, Chanel: mono)
Otherwise, you will get 400 bad request error.
Make sure that codec is Opus because most of the OGG files have Vorbis codec which is not supported by the API.
You can use this website to convert an audio file in the required format.
Related
In my web application I have an image uploading module. I want to check the uploaded file whether it's an image file or any other file. I am using Java in server side.
The image is read as BufferedImage in java and then I am writing it to disk with ImageIO.write()
How shall I check the BufferedImage, whether it's really an image or something else?
Any suggestions or links would be appreciated.
I'm assuming that you're running this in a servlet context. If it's affordable to check the content type based on just the file extension, then use ServletContext#getMimeType() to get the mime type (content type). Just check if it starts with image/.
String fileName = uploadedFile.getFileName();
String mimeType = getServletContext().getMimeType(fileName);
if (mimeType.startsWith("image/")) {
// It's an image.
}
The default mime types are definied in the web.xml of the servletcontainer in question. In for example Tomcat, it's located in /conf/web.xml. You can extend/override it in the /WEB-INF/web.xml of your webapp as follows:
<mime-mapping>
<extension>svg</extension>
<mime-type>image/svg+xml</mime-type>
</mime-mapping>
But this doesn't prevent you from users who are fooling you by changing the file extension. If you'd like to cover this as well, then you can also determine the mime type based on the actual file content. If it's affordable to check for only BMP, GIF, JPG or PNG types (but not TIF, PSD, SVG, etc), then you can just feed it directly to ImageIO#read() and check if it doesn't throw an exception.
try (InputStream input = uploadedFile.getInputStream()) {
try {
ImageIO.read(input).toString();
// It's an image (only BMP, GIF, JPG and PNG are recognized).
} catch (Exception e) {
// It's not an image.
}
}
But if you'd like to cover more image types as well, then consider using a 3rd party library which does all the work by sniffing the file headers. For example JMimeMagic or Apache Tika which support both BMP, GIF, JPG, PNG, TIF and PSD (but not SVG). Apache Batik supports SVG. Below example uses JMimeMagic:
try (InputStream input = uploadedFile.getInputStream()) {
String mimeType = Magic.getMagicMatch(input, false).getMimeType();
if (mimeType.startsWith("image/")) {
// It's an image.
} else {
// It's not an image.
}
}
You could if necessary use combinations and outweigh the one and other.
That said, you don't necessarily need ImageIO#write() to save the uploaded image to disk. Just writing the obtained InputStream directly to a Path or any OutputStream like FileOutputStream the usual Java IO way is more than sufficient (see also Recommended way to save uploaded files in a servlet application):
try (InputStream input = uploadedFile.getInputStream()) {
Files.copy(input, new File(uploadFolder, fileName).toPath());
}
Unless you'd like to gather some image information like its dimensions and/or want to manipulate it (crop/resize/rotate/convert/etc) of course.
I used org.apache.commons.imaging.Imaging in my case. Below is a sample piece of code to check if an image is a jpeg image or not. It throws ImageReadException if uploaded file is not an image.
try {
//image is InputStream
byte[] byteArray = IOUtils.toByteArray(image);
ImageFormat mimeType = Imaging.guessFormat(byteArray);
if (mimeType == ImageFormats.JPEG) {
return;
} else {
// handle image of different format. Ex: PNG
}
} catch (ImageReadException e) {
//not an image
}
This is built into the JDK and simply requires a stream with support for
byte[] data = ;
InputStream is = new BufferedInputStream(new ByteArrayInputStream(data));
String mimeType = URLConnection.guessContentTypeFromStream(is);
//...close stream
Since Java SE 6 https://docs.oracle.com/javase/6/docs/api/java/net/URLConnection.html
Try using multipart file instead of BufferedImage
import org.apache.http.entity.ContentType;
...
public void processImage(MultipartFile file) {
if(!Arrays.asList(ContentType.IMAGE_JPEG.getMimeType(), ContentType.IMAGE_PNG.getMimeType(), ContentType.IMAGE_GIF.getMimeType()).contains(file.getContentType())) {
throw new IllegalStateException("File must be an Image");
}
}
I am sending DICOM images to my API by encoding as base64 from the frontend, which is in Angular CLI. Also, I have Rest API to get those encoded DICOM images and decode them back before had some process with them. But after decoding the DICOM image into the memory stream, metadata of DICOM images are lost. It is appreciatable if I got a better solution. Please find my codes below.
//Angular code
var file = event.dataTransfer ? event.dataTransfer.files[i] :
event.target.files[0];
//var pattern = /.dcm/;
var reader = new FileReader();
reader.onload = this._handleReaderLoaded.bind(this);
reader.readAsDataURL(file);
//Web API Code
[HttpPost("UploadFile/{Id}")]
public async Task<IActionResult> UploadFile(int Id, [FromBody] DICOMFiles
dicomfiles)
{
String base64Encoded = encodedImage;
string output =
encodedImage.Substring(encodedImage.IndexOf(',') + 1);
byte[] data = Convert.FromBase64String(output);
MemoryStream stream = new MemoryStream(data);
client.UploadFile(stream, "Projects/test_images/Test.dcm");
}
At last, I found a solution for this. The problem is not about decode from base64. The actual problem is with the client.UploadFile() method call.
Before using the client.uploadfile(), we need to make sure that the memory stream object is pointing to position "0". This will allow the client.UploadFile() method to create and write all the content of the mentioned file from the start of the byte[] array. we can do this as mentioned below.
stream.Position = 0;
I'm trying to send an audio file to dialogflow API for intent detection. I already have an agent working quite well but only with text. I'm trying to add the the audio feature but with no luck.
I'm using the example (Java) provided in this page:
https://cloud.google.com/dialogflow-enterprise/docs/detect-intent-audio#detect-intent-text-java
This is my code:
public DetectIntentResponse detectIntentAudio(String projectId, byte [] bytes, String sessionId,
String languageCode)
throws Exception {
// Set the session name using the sessionId (UUID) and projectID (my-project-id)
SessionName session = SessionName.of(projectId, sessionId);
System.out.println("Session Path: " + session.toString());
// Note: hard coding audioEncoding and sampleRateHertz for simplicity.
// Audio encoding of the audio content sent in the query request.
AudioEncoding audioEncoding = AudioEncoding.AUDIO_ENCODING_LINEAR_16;
int sampleRateHertz = 16000;
// Instructs the speech recognizer how to process the audio content.
InputAudioConfig inputAudioConfig = InputAudioConfig.newBuilder()
.setAudioEncoding(audioEncoding) // audioEncoding = AudioEncoding.AUDIO_ENCODING_LINEAR_16
.setLanguageCode(languageCode) // languageCode = "en-US"
.setSampleRateHertz(sampleRateHertz) // sampleRateHertz = 16000
.build();
// Build the query with the InputAudioConfig
QueryInput queryInput = QueryInput.newBuilder().setAudioConfig(inputAudioConfig).build();
// Read the bytes from the audio file
byte[] inputAudio = Files.readAllBytes(Paths.get("/home/rmg/Audio/book_a_room.wav"));
byte[] encodedAudio = Base64.encodeBase64(inputAudio);
// Build the DetectIntentRequest
DetectIntentRequest request = DetectIntentRequest.newBuilder()
.setSession("projects/"+projectId+"/agent/sessions/" + sessionId)
.setQueryInput(queryInput)
.setInputAudio(ByteString.copyFrom(encodedAudio))
.build();
// Performs the detect intent request
DetectIntentResponse response = sessionsClient.detectIntent(request);
// Display the query result
QueryResult queryResult = response.getQueryResult();
System.out.println("====================");
System.out.format("Query Text: '%s'\n", queryResult.getQueryText());
System.out.format("Detected Intent: %s (confidence: %f)\n",
queryResult.getIntent().getDisplayName(), queryResult.getIntentDetectionConfidence());
System.out.format("Fulfillment Text: '%s'\n", queryResult.getFulfillmentText());
return response;
}
I have tried with several formats, wav (PCM 16 bits several sample rates) and FLAC, and also converting the bytes to base64 in two different ways as described here (by code or console):
https://dialogflow.com/docs/reference/text-to-speech
I have even tested with the .wav provided in this example creating a new intent in my agent called "book a room" with that training phrase. It works using text and audio from the dialogflow console but only works with text, not audio from my code... and I'm sending the same wav they provide! (code above)
I always receive the same response (QueryResult):
I need a clue or something, I'm totally stuck here. No logs, no errors in the response... but does not work.
Thanks
I wrote to the dialogflow support and the replied my with a working piece of code. It is basically the same posted above, the only difference is the base64 encoding, it is not necessary to do that.
So I removed:
byte[] encodedAudio = Base64.encodeBase64(inputAudio);
(And used inputAudio directly)
Now It is working as expected...
I use javamail to get message, when I get the message i have:
com.sun.mail.util.BASE64DecoderStream,
I know that is part of multipart message, in the source of message I have
Content-Type: image/png; name=index_01.png
Content-Transfer-Encoding: base64
How encode this message??
edit:
I have that code:
else if (mbp.getContent() instanceof BASE64DecoderStream){
InputStream is = null;
ByteArrayOutputStream os = null;
is = mbp.getInputStream();
os = new ByteArrayOutputStream(512);
int c = 0;
while ((c = is.read()) != -1) {
os.write(c);
}
System.out.println(os.toString());
}
And that code return strange string, for example:
Ř˙á?Exif??II*????????????˙ě?Ducky???????˙á)
com.sun.mail.util.BASE64DecoderStream is platform dependent. You cannot rely on that always being the type of class that handles base64 decoding.
Rather the javamail APIs already support decoding for you:
// part is instanceof javax.mail.Part
ByteArrayOutputStream bos = new ByteArrayOutputStream();
part.getDataHandler().writeTo(bos);
String decodedContent = bos.toString()
What are you expecting when you read the contents of an image part? The image is stored in the message in an encoded format, but JavaMail is decoding the data before returning the bytes to you. If you store the bytes in a file, you can display the image with many image viewing/editing applications. If you want to display them with your Java program, you'll need to convert the bytes to an appropriate Java Image object using (e.g.) APIs in the java.awt.image package.
That Sun's base 64 encoder is in the optional package and can be moved or renamed at any time without warning, also may be missing in alternative Java runtimes, also access to these packages may be disabled. It is much better not to rely on it.
I would say, use Base64 from Apache Commons instead, should do the same. Hope you can rebuild and fix the source code.
i need to upload a audio file stored in phone memory using http post. what steps required for it. any helps will be appreciated.
i doing the client side progrmming.
InputStream inputStream = null;
ConnectionFactory connFact = new ConnectionFactory();
ConnectionDescriptor connDesc;
connDesc = connFact.getConnection(url);
if (connDesc != null)
{
HttpConnection httpConn;
httpConn = (HttpConnection)connDesc.getConnection();
httpConn.setRequestMethod(HttpConnection.POST);
URLEncodedPostData encPostData = new URLEncodedPostData("UTF-8", false);
//here i want to append the audio data from phone memmory . how it will be done?
thanks
There is an example of using the HTTP POST protocol in the API documentation for HttpConnection. You will, as PA suggests, have to provide an HTTP server to accept the conection and do something with the data.