Just started exploring Google Cloud Vision APIs. From their guide:
const client = new vision.ImageAnnotatorClient();
const fileName = 'Local image file, e.g. /path/to/image.png';
const [result] = await client.textDetection(fileName);
However, I wanna use base64 representation of binary image data, since they claim that it's possible to use.
I found this reference on SO:
Google Vision API Text Detection with Node.js set Language hint
Instead of imageUri I used "content": stringas mentioned here. But the SO sample uses const [result] = await client.batchAnnotateImages(request);method. I tried using same technique on const [result] = await client.textDetection( method and it gave me an error.
So my question is: Is it possible to use base64 encoded string to represent image in order to perform TEXT_DETECTION ? And if so, how?
Any kind of help is highly appreciated.
You can use the quickstart guide and from there edit the lines after the creation of the client for the following:
// Value of the image in base64
const img_base64 = '/9j/...';
const request = {
image: {
content: Buffer.from(img_base64, 'base64')
}
};
const [result] = await client.textDetection(request);
console.log(result.textAnnotations);
console.log(result.fullTextAnnotation);
You can take a look at the function here, read the description of the request parameter, in particular the following part:
A dictionary-like object representing the image. This should have a
single key (source, content).
If the key is content, the value should be a Buffer.
Which leads to the structure used in the sample code from before. Opposed to when using imageUri or filename, which have to be inside of another object which key is source, as shown in the sample.
content field need to be Buffer.
You use the nodejs client library. The library use the grpc API internally, and grpc API expect bytes type at content field.
However, JSON API expect base64 string.
References
https://cloud.google.com/vision/docs/reference/rpc/google.cloud.vision.v1#image
https://googleapis.dev/nodejs/vision/latest/v1.ImageAnnotatorClient.html#textDetection
Related
In a MERN + Firebase project, I have an image data string that I want to upload and then get the access token of that file.
The image data string is of the following form:
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAOoAAAClCAYAAABSmmH3AAAAAXNSR0IArs4c6QAABFJJREFUeF7t1oENg0AQA8F8/6XSAx8pVWTloQIzZwvOvfd+PAQI/LXAMdS/vo9wBH4ChqoIBAIChho4kogEDFUHCAQEDDVwJBEJGKoOEAgIGGrgSCISMFQdIBAQMNTAkUQkYKg6QCAgYKiBI4lIwFB1gEBAwFADRxKRgKHqAIGAgKEGjiQiAUPVAQIBAUMNHElEAoaqAwQCAoYaOJKIBAxVBwgEBAw1cCQRCRiqDhAICBhq4EgiEjBUHSAQEDDUwJFEJGCoOkAgIGCogSOJSMBQdYBAQMBQA0cSkYCh6gCBgIChBo4kIgFD1QECAQFDDRxJRAKGqgMEAgKGGjiSiAQMVQcIBAQMNXAkEQkYqg4QCAgYauBIIhIwVB0gEBAw1MCRRCRgqDpAICBgqIEjiUjAUHWAQEDAUANHEpGAoeoAgYCAoQaOJCIBQ9UBAgEBQw0cSUQChqoDBAIChho4kogEDFUHCAQEDDVwJBEJGKoOEAgIGGrgSCISMFQdIBAQMNTAkUQkYKg6QCAgYKiBI4lIwFB1gEBAwFADRxKRgKHqAIGAgKEGjiQiAUPVAQIBAUMNHElEAoaqAwQCAoYaOJKIBAxVBwgEBAw1cCQRCRiqDhAICBhq4EgiEjBUHSAQEDDUwJFEJGCoOkAgIGCogSOJSMBQdYBAQMBQA0cSkYCh6gCBgIChBo4kIgFD1QECAQFDDRxJRAKGqgMEAgKGGjiSiAQMVQcIBAQMNXAkEQkYqg4QCAgYauBIIhIwVB0gEBAw1MCRRCRgqDpAICBgqIEjiUjAUHWAQEDAUANHEpGAoeoAgYCAoQaOJCIBQ9UBAgGB8zzPDeQUkcC0wHnf11CnK+DlCwJ+fQtXknFewBd1vgIACgK+qIUryTgvYKjzFQBQEDDUwpVknBcw1PkKACgIGGrhSjLOCxjqfAUAFAQMtXAlGecFDHW+AgAKAoZauJKM8wKGOl8BAAUBQy1cScZ5AUOdrwCAgoChFq4k47yAoc5XAEBBwFALV5JxXsBQ5ysAoCBgqIUryTgvYKjzFQBQEDDUwpVknBcw1PkKACgIGGrhSjLOCxjqfAUAFAQMtXAlGecFDHW+AgAKAoZauJKM8wKGOl8BAAUBQy1cScZ5AUOdrwCAgoChFq4k47yAoc5XAEBBwFALV5JxXsBQ5ysAoCBgqIUryTgvYKjzFQBQEDDUwpVknBcw1PkKACgIGGrhSjLOCxjqfAUAFAQMtXAlGecFDHW+AgAKAoZauJKM8wKGOl8BAAUBQy1cScZ5AUOdrwCAgoChFq4k47yAoc5XAEBBwFALV5JxXsBQ5ysAoCBgqIUryTgvYKjzFQBQEDDUwpVknBcw1PkKACgIGGrhSjLOCxjqfAUAFAQMtXAlGecFDHW+AgAKAoZauJKM8wKGOl8BAAUBQy1cScZ5AUOdrwCAgoChFq4k47yAoc5XAEBBwFALV5JxXsBQ5ysAoCDwBaT9ke70WG4vAAAAAElFTkSuQmCC
This is the reference to where the file should be uploaded:
const imageRef: StorageReference = ref(
storage,
`/issueImages/${firebaseImageId}`
);
So far, I have attempted to use the 'put' function with the imageRef, and when I try using uploadBytes() of firebase, I have to upload it as a Buffer, and even then I cannot seem to find the access token in the metadata.
To upload a data URL to Firebase, you would use storageRef.putString(url, 'DATA_URL') (legacy) or uploadString(storageRef, url, 'DATA_URL') (modern) depending on the SDK you are using.
When you upload a file to a Cloud Storage bucket, it will not be issued an access token until a client calls its version of getDownloadURL(). So to fix your issue, you would call getDownloadURL() immediately after upload.
If Node is running on a client's machine, you would use:
// legacy syntax
import * as firebase from "firebase";
// reference to file
const imageStorageRef = firebase.storage()
.ref(`/issueImages/${firebaseImageId}`);
// perform the upload
await imageStorageRef.putString(dataUrl, 'DATA_URL');
// get the download URL
const imageStorageDownloadURL = await imageStorageRef.getDownloadURL();
// modern syntax
import { getStorage, getDownloadURL, ref, uploadString } from "firebase/storage";
// reference to file
const imageStorageRef = ref(
getStorage(),
`/issueImages/${firebaseImageId}`
);
// perform the upload
await uploadString(imageStorageRef, dataUrl, 'DATA_URL');
// get the download URL
const imageStorageDownloadURL = await getDownloadURL(imageStorageRef);
If Node is running on a private server you control, you should opt to use the Firebase Admin SDK instead as it bypasses the rate limits and restrictions applied to the client SDKs.
As mentioned before, the download URLs aren't created automatically. Unfortunately for us, getDownloadURL is a feature of the client SDKs and the Admin SDK doesn't have it. So we can either let a client call getDownloadURL when it is needed or we can manually create the download URL if we want to insert it into a database.
Nico has an excellent write up on how Firebase Storage URLs work, where they collated information from the Firebase Extensions GitHub and this StackOverflow thread. In summary, to create (or recreate) a download URL once it has been uploaded, you can use the following function:
import { uuid } from "uuidv4";
// Original Credit: Nico (#nicomqh)
// https://www.sentinelstand.com/article/guide-to-firebase-storage-download-urls-tokens
// "file" is an instance of the File class from the Cloud Storage SDK
// executing this function more than once will revoke all previous tokens
function createDownloadURL(file) {
const downloadToken = uuid();
await file.setMetadata({
metadata: {
firebaseStorageDownloadTokens: downloadToken
}
});
return `https://firebasestorage.googleapis.com/v0/b/${file.bucket.name}/o/${encodeURIComponent(file.name)}?alt=media&token=${downloadToken}`;
}
The allows us to change the client-side code above into the following so it can run using the Admin SDK:
// assuming firebase-admin is initialized already
import { getStorage } from "firebase-admin/storage";
// reference to file
const imageStorageFile = getStorage()
.bucket()
.file(`/issueImages/${firebaseImageId}`);
// perform the upload
await imageStorageFile.save(dataUrl);
// get the download URL
const imageStorageDownloadURL = await createDownloadURL(imageStorageFile);
In all of the above examples, a download URL is retrieved and saved to the imageStorageDownloadURL variable. You should store this value as-is in your database. However, if you instead want to store only the access token and reassemble the URL on an as-needed basis, you can extract the token from its ?token= parameter using:
const downloadToken = new URL(imageStorageDownloadURL).searchParams.get('token');
I am sending DICOM images to my API by encoding as base64 from the frontend, which is in Angular CLI. Also, I have Rest API to get those encoded DICOM images and decode them back before had some process with them. But after decoding the DICOM image into the memory stream, metadata of DICOM images are lost. It is appreciatable if I got a better solution. Please find my codes below.
//Angular code
var file = event.dataTransfer ? event.dataTransfer.files[i] :
event.target.files[0];
//var pattern = /.dcm/;
var reader = new FileReader();
reader.onload = this._handleReaderLoaded.bind(this);
reader.readAsDataURL(file);
//Web API Code
[HttpPost("UploadFile/{Id}")]
public async Task<IActionResult> UploadFile(int Id, [FromBody] DICOMFiles
dicomfiles)
{
String base64Encoded = encodedImage;
string output =
encodedImage.Substring(encodedImage.IndexOf(',') + 1);
byte[] data = Convert.FromBase64String(output);
MemoryStream stream = new MemoryStream(data);
client.UploadFile(stream, "Projects/test_images/Test.dcm");
}
At last, I found a solution for this. The problem is not about decode from base64. The actual problem is with the client.UploadFile() method call.
Before using the client.uploadfile(), we need to make sure that the memory stream object is pointing to position "0". This will allow the client.UploadFile() method to create and write all the content of the mentioned file from the start of the byte[] array. we can do this as mentioned below.
stream.Position = 0;
I try atm to classify an image via the API from Microsoft on nodeJS.
The network is alreaday trained and I can "connect" to the my algorithm. I want to send a base64 string as a dataUri but then I get this error-message: "Code: BadRequestImageUrl, message: Invalid image url"
The variable "img" is a base64 string (from a FHIR-Observation-Object) and correct (on a webside the url works with the base64).
I try out to senda image from Wikipedia. But then I have an other error: "NoFoundIteration / Invalid iteration"
const PredictionAPIClient = require("azure-cognitiveservices-customvision-prediction");
const predictionKey = "xxxx";
const endPoint = "https://southcentralus.api.cognitive.microsoft.com"
const projectId = "xxxxx";
const publishedName = "myMLName";
...
var img = 'iVBORw0KGgoAAAANSUhEUgAAAgAAAAJmCAYAAA...'; //base64
...
tempUrl= { url: 'data:image/png;base64,' + img };
...
predictor.classifyImageUrl(projectId, publishedName, tempUrl)
.then((resultJSON) => {
console.log("RESULT ######################")
//console.log(resultJSON);})
.catch((error) => {
console.log("ERROR #####################");
console.log(error);}
);
I should get a JSON form Microsoft Azure with the results.
Have a look to the documentation of the API behind the package you are using: https://southcentralus.dev.cognitive.microsoft.com/docs/services/Custom_Vision_Prediction_3.0/operations/5c82db60bf6a2b11a8247c14
You can see that Classify has 2 methods:
ClassifyImage, which is using an image in application/octet-stream
ClassifyImageUrl, which is using an url for input
Data URL are not supported, you must use classic URL (and the image must be publicly accessible: don't use an URL pointing to an endpoint that need an authentication)
For your iteration error, make sure that you use your iteration name in publishedName, not your project name.
Example here, with value in field "Published as":
I am new to Python.I want to use Google text-to-speech API for that i used below code, but I am unable to access the API due to error. This is the code,
def synthesize_text(text):
"""Synthesizes speech from the input string of text."""
from google.cloud import texttospeech
client = texttospeech.TextToSpeechClient()
input_text = texttospeech.types.SynthesisInput(text=text)
# Note: the voice can also be specified by name.
# Names of voices can be retrieved with client.list_voices().
voice = texttospeech.types.VoiceSelectionParams(
language_code='en-US',
ssml_gender=texttospeech.enums.SsmlVoiceGender.FEMALE)
audio_config = texttospeech.types.AudioConfig(
audio_encoding=texttospeech.enums.AudioEncoding.MP3)
response = client.synthesize_speech(input_text, voice, audio_config)
# The response's audio_content is binary.
with open('output.mp3', 'wb') as out:
out.write(response.audio_content)
print('Audio content written to file "output.mp3"')
This is the error,
google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or
explicitly create credential and re-run the application. For more
information, please see
https://developers.google.com/accounts/docs/application-default-credentials.
I already have credentials JSON file, but I am unable to configure the code to authenticate my request.
Please help!
You could try this code:
from google.oauth2 import service_account
credentials = service_account.Credentials.from_service_account_file('yourkey.json')
client = texttospeech.TextToSpeechClient(credentials=credentials)
There are 2 ways :
1 Way :
if you using Json file then better to set json path into Environment Variable, if you do this then you no need to setup in coding it will automatically get you license from there
GOOGLE_APPLICATION_CREDENTIALS=[path]
2 WAY :
I have Java code i don't know about python so you can get idea from here :
String jsonPath = "file.json";
CredentialsProvider credentialsProvider = FixedCredentialsProvider.create(ServiceAccountCredentials.fromStream(new FileInputStream(jsonPath)));
TextToSpeechSettings settings = TextToSpeechSettings.newBuilder().setCredentialsProvider(credentialsProvider).build();
Instantiates a client
TextToSpeechClient textToSpeechClient = TextToSpeechClient.create(settings)
this seems to be an old discussion but I thought to comment, maybe someone will come across like in my case :))
for nodejs client, I managed to authenticate it this way:
const client = new textToSpeech.TextToSpeechClient({
credentials: {
private_key: "??",
client_email: "???",
}
});
You could authenticate your google credential by different ways.
One is by setting OS environment and another one is authenticate while you initiate a request.
I would suggest oauth2client library for python to authenticate.
In addition to this refer my example on Github (Link).
You need to have a service account, and service account .json key File.
You need to pass the key file name while you creating the client Instance.
const client = new textToSpeech.TextToSpeechClient({
keyFilename: "./auth.json",
});
Download the key file and rename it as auth.json place it root of your project folder.
Make sure Your service account have proper access to call the API.
Here is the full code:
// Imports the Google Cloud client library
const textToSpeech = require("#google-cloud/text-to-speech");
// Import other required libraries
const fs = require("fs");
const util = require("util");
// Creates a client
const client = new textToSpeech.TextToSpeechClient({
keyFilename: "./auth.json",
});
async function quickStart() {
// The text to synthesize
const text = "Hello this is a test";
// Construct the request
const request = {
input: { text: text },
// Select the language and SSML voice gender (optional)
voice: { languageCode: "en-US", ssmlGender: "NEUTRAL" },
// select the type of audio encoding
audioConfig: { audioEncoding: "MP3" },
};
// Performs the text-to-speech request
const [response] = await client.synthesizeSpeech(request);
// Write the binary audio content to a local file
const writeFile = util.promisify(fs.writeFile);
await writeFile("output.mp3", response.audioContent, "binary");
console.log("Audio content written to file: output.mp3");
}
quickStart();
We have a legacy NodeJS API we are looking to move away from. We are extracting the auth component out of it.
We have a function that hashes a password, that looks a bit like:
var sha256 = crypto.createHash('sha256');
sha256.update(password + secret);
return sha256.digest('hex');
Aside from the obvious security implications a function like this has, it's encoded using the binary encoding NodeJS has.
If you pass in a String to update, it will use 'binary' as the encoding format. This results in it actually encoding unicode characters like "Kiełbasa" as "KieBbasa", before SHA256'ing them.
In our Scala code we are now looking to rewrite this legacy function so we can auth old users. But we cannot find a charset to use on these strings that has the same resulting output. Our Scala code looks like:
def encryptPassword(password: String): String = {
Hashing.sha256().hashString(in, Charsets.UTF_8).toString
}
We need the in string to be the same as what Node is using, but we can't figure it out.
Ideas? Node.js... not even once.
Well, turns out this is much easier than it seems. The following code works:
def encryptPassword(password: String): String = {
val in = (password + secret).map(_.toByte.toChar).mkString("")
Hashing.sha256().hashString(in, Charsets.UTF_8).toString
}
Thanks #stefanobaghino