I'm using Swift_Message to construct the message and Swift_Mime_ContentEncoder_Base64ContentEncoder to encode the message,then in Google_Service_Gmail_Message I'm setting the encoded mesage with ->setRaw() method.
I've been sending mail with this for quite some time and it used to work fine. Since yesterday it stopped working and the error message says
"error": {
"code": 400,
"message": "Invalid value at 'message.raw' (TYPE_BYTES), Base64 decoding failed for \"[base64 encoded message with CRLF after every 76th character]\"",
"errors": [
{
"message": "Invalid value at 'message.raw' (TYPE_BYTES), Base64 decoding failed for \"[base64 encoded message with CRLF after every 76th character]\"",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
It works when I remove the CRLF. Any thoughts?
Ref: https://www.rfc-editor.org/rfc/rfc2822#section-2.1.1
My code
$msg = new Swift_Message();
$msg->setCharset('UTF-8')
->addTo(/*recipient*/)
->setSubject(/*sbject*/)
->addPart(/*text content*/, "text/plain")
->addPart(/*html content*/, "text/html");
$base64 = (new Swift_Mime_ContentEncoder_Base64ContentEncoder)->encodeString($msg->toString());
$base64_msg = rtrim(strtr($base64, '+/', '-_'), '=');
$mailer = $this->_getGmailService();// new Google_Service_Gmail(new Google_Client())
$message = new Google_Service_Gmail_Message();
$message->setRaw($base64_msg);
$message->setThreadId($threadId);
$mailer->users_messages->send('me', $message);
I used base64_encode($message->toString()); instead of
(new Swift_Mime_ContentEncoder_Base64ContentEncoder)->encodeString($msg->toString());
The library method you're using does a Base64 encoding and you need a Base64URL encoded string as stated in the documentation
Related
Here is an example of some JSON I am working with:
{"name":"John","attributes":"{\"key\":\"value\"}"}
Here it is again, in a more readable format:
{
"name": "John",
"attributes": "{\"key\":\"value\"}"
}
Notice above that the doublequotes surrounding key and value are escaped. That's necessary, and is valid JSON (checked at jsonlint.com).
I am trying to get the value of "name", but the escaped doublequotes are causing an error.
What am I doing wrong in my node.js code?
var theString = '{"name":"John","attributes":"{\"key\":\"value\"}"}';
var theJSON = JSON.parse(theString);
var theName = theJSON.name;
console.log("name = " + theName);
Below is the output. The error occurs on the 2nd line of my code, where I "JSON.parse()" the string. JSON.parse seems to be removing the backslashes, turning valid JSON into invalid JSON.
undefined:1
{"name":"John","attributes":"{"key":"value"}"}
^
SyntaxError: Unexpected token k in JSON at position 31
Since that part of the data is JSON-within-JSON, you'd parse the JSON, then parse the JSON on the attributes property:
const obj = JSON.parse(json);
obj.attributes = JSON.parse(obj.attributes);
Live Example:
const json = document.getElementById("json").textContent;
const obj = JSON.parse(json);
obj.attributes = JSON.parse(obj.attributes);
console.log(obj);
<pre id="json">{
"name": "John",
"attributes": "{\"key\":\"value\"}"
}</pre>
This should be so simple but I've been stuck on it for more than an hour and it's driving me crazy.
I'm working with an API that's returning data as zipped .json files. I've managed to unzip the files, but now need to parse these files to json objects.
The data is in a buffer, and looks like this:
{ "name": "foo1", "job": "bar1" }
{ "name": "foo2", "job": "bar2" }
{ "name": "foo3", "job": "bar3" }
{ "name": "foo4", "job": "bar4" }
Of course, parsing this with JSON.parse() fails because the data is a .json file, not an array of jsons.
How can I parse this data correctly? fs expects a filepath to read the file, which wouldn't work in my case (as far as I'm aware) because the data is from a buffer, not from a local file.
tl;dr: How do you I parse a .json file that doesn't have a filepath?
First of all, the example you provided (where each line is a string representing a JSON object) is not a JSON file.
It’s a file containing multiple JSON formatted strings, one per line.
Without a surrounding array, it is no wonder you cannot parse it.
I’m also unsure what you mean by the data being kept in a buffer.
Do you mean that you’ve read the contents of the file using the standard fs.readfile() or variant?
If this is the case, you need to convert the Buffer returned from your readfile to a String, as in:
var contents = fs.readfileSync(FILEPATH).toString()
Once you have done so, you could construct an Array using the contents of your file and convert the result to a JSON formatted string using:
fs.readfile(FILEPATH, (err, buf) => {
if (err) {
throw err
}
let objAry = []
buf.toString().split(/\r?\n/).forEach( line => {
objAry.push(JSON.parse(line))
})
let jsonStr = JSON.stringify(objAry)
// do something with the result string
// possibly sending it as a response to an API
// request as ‘Content-Type: application/json’
})
Obviously, you’ll want to add error handling (try/catch) to this.
you can change buffer data to utf-8 charset String then parse it:
JSON.parse(buffer.toString('utf-8'))
I receive the below object in my service but when I am parsing this object I get the error
SyntaxError: Unexpected token : in JSON at position 603069
Code:
var data1 = [];
// Process a chunk of data. This may be called multiple times.
req
.on("data", function(chunk) {
// Append to buffer
data1.push(chunk);
})
.on("end", function() {
var buffer = Buffer.concat(data1);
console.info("Buffer Data Request Body: " + buffer);
buffer = buffer.toString("utf8");
var partsOfStr = buffer.split("&");
//This line gives error
var obj = JSON.parse(
decodeURI(buffer.replace(/&/g, '","').replace(/=/g, '":"'))
);
Object:
{
"type" : "NewThreadVoice",
"incidentId": "398115",
"channel" : "Mobile",
"data": a huge base 64 string
"fileName": "1.aac",
"contentType" : "aac",
"contactId" : "954344"
}
When I reduce the base64 (value of data) to half it works.
A base64 string is not necessary to contain only one "=" character. This character is used for padding (for more information see Why does a base64 encoded string have an = sign at the end )
For example, the codification of home in base64 is aG9tZQ==. Using your code ( .replace(/=/g, '":"') ), this will be transformed into aG9tZQ":"":"
You should use .replace(/=+/g, '":"') for replacing all consecutive = chars.
I am working on a simple nodejs console utility that will upload images for the training of a Custom Vision model. I do this mainly because the customvision web app won't let you tag multiple images at once.
tl;dr: How to post images into the CreateImagesFromFiles API endpoint?
I cannot figure out how to pass images that I want to upload. The documentation just defines a string as a type for one of the properties (content I guess). I tried passing path to local file, url to online file and even base64 encoded image as a string. Nothing passed.
They got a testing console (blue button "Open API testing console" at the linked docs page) but once again... it's vague and won't tell you what kind of data it actually expects.
The code here isn't that relevant, but maybe it helps...
const options = {
host: 'southcentralus.api.cognitive.microsoft.com',
path: `/customvision/v2.0/Training/projects/${projectId}/images/files`,
method: 'POST',
headers: {
'Training-Key': trainingKey,
'Content-Type': 'application/json'
}
};
const data = {
images: [
{
name: 'xxx',
contents: 'iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAEklEQVR42mP8z8AARKiAkQaCAFxlCfyG/gCwAAAAAElFTkSuQmCC',
tagIds: [],
regions: []
}
],
tagIds: []
}
const req = http.request(options, res => {
...
})
req.write(JSON.stringify(data));
req.end();
Response:
BODY: { "statusCode": 404, "message": "Resource not found" }
No more data in response.
I got it working using the "API testing console" feature, so I can help you to identify your issue (but sorry, I'm not expert in node.js so I will guide you with C# code)
Format of content for API
You are right, the documentation is not clear about the content the API is waiting for. I made some search and found a project in a Microsoft's Github repository called Cognitive-CustomVision-Windows, here.
What is saw is that they use a class called ImageFileCreateEntry whose signature is visible here:
public ImageFileCreateEntry(string name = default(string), byte[] contents = default(byte[]), IList<System.Guid> tagIds = default(IList<System.Guid>))
So I guessed it's using a byte[].
You can also see in their sample how they did for this "batch" mode:
// Or uploaded in a single batch
var imageFiles = japaneseCherryImages.Select(img => new ImageFileCreateEntry(Path.GetFileName(img), File.ReadAllBytes(img))).ToList();
trainingApi.CreateImagesFromFiles(project.Id, new ImageFileCreateBatch(imageFiles, new List<Guid>() { japaneseCherryTag.Id }));
Then this byte array is serialized with Newtonsoft.Json: if you look at their documentation (here) it says that byte[] are converted to String (base 64 encoded). That's our target.
Implementation
As you mentioned that you tried with base64 encoded image, I gave it a try to check. I took my StackOverflow profile picture that I downloaded locally. Then using the following, I got the base64 encoded string:
Image img = Image.FromFile(#"\\Mac\Home\Downloads\Picto.jpg");
byte[] arr;
using (MemoryStream ms = new MemoryStream())
{
img.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
arr = ms.ToArray();
}
var content = Convert.ToBase64String(arr);
Later on, I called the API with no tags to ensure that the image is posted and visible:
POST https://southcentralus.api.cognitive.microsoft.com/customvision/v2.2/Training/projects/MY_PROJECT_ID/images/files HTTP/1.1
Host: southcentralus.api.cognitive.microsoft.com
Training-Key: MY_OWN_TRAINING_KEY
Content-Type: application/json
{
"images": [
{
"name": "imageSentByApi",
"contents": "/9j/4AAQSkZJRgA...TOO LONG FOR STACK OVERFLOW...",
"tagIds": [],
"regions": []
}
],
"tagIds": []
}
Response received: 200 OK
{
"isBatchSuccessful": true,
"images": [{
"sourceUrl": "imageSentByApi",
"status": "OK",
"image": {
"id": "GENERATED_ID_OF_IMAGE",
"created": "2018-11-05T22:33:31.6513607",
"width": 328,
"height": 328,
"resizedImageUri": "https://irisscuprodstore.blob.core.windows.net/...",
"thumbnailUri": "https://irisscuprodstore.blob.core.windows.net/...",
"originalImageUri": "https://irisscuprodstore.blob.core.windows.net/..."
}
}]
}
And my image is here in Custom Vision portal!
Debugging your code
In order to debug, you should 1st try to submit your content again with tagIds and regions arrays empty like in my test, then provide the content of the API reply
How can we capture only the email address in an api response for the gmail API. The
fields parameter is set to payload/headers, which returns way more data than we need in the response.
All we need is the value from one name/value pair in the JSON response; for example
The full response as we have it now looks something like this
{
"payload": {
"headers": [
{
"name": "Delivered-To",
"value": "xxxxxxx"
{
"name": "Received",
"value": "xxxxxxxx"
},
{
"name": "Received-SPF",
"value": "----"
},......
{
"name": "To",
"value": "xxxxxxx"
}, ...... E.T.C........E.T.C ......
/*All we want is one name/value pair to be returned e.g. */
{
"payload": {
"headers": [
{
"name": "X-Failed-Recipients",
"value": "............."
}
]
}
A better question might be is there a better way to capture bounced/returned mail than this via the gmail API?
Also, is it possible to request an XML response instead of JSON. How can that be done for the gmail API?
Thanks !!
You can do messages.get(format=METADATA, metadataIncludeHeaders=["To", "From", "Subject"]) for example, now to just request the specific headers you care about . Note this only works with the metadata format (it won't include the body also, if you want all the body you get the full email. Based on the list of headers, shouldn't be too hard to turn that into a map/dict of key => [list, of, values].
As to your second question, yes you can definitely request response in any format you want. That's a standard Google API question though. I can't find a good reference (surely some searching will) but typically you can set an "alt=xml" or "alt=" query parameter to get response in that format. Not sure how this is exposed in any particular client library.
First you need to get your message payload and then you want to use the getall method to return a list of headers and then you can use the getitem to pull any specific header you want from that list.
I converted the String into a String and took it as a JSON Array and iterated through it to take the JSON Object that I required.
private static DirtyMail getHeaderParts(DirtyMail mail, List<MessagePartHeader> headers)
{
try {
//Convert the header into JSON Array for easy processing of data.
JSONArray headerArray = new JSONArray(headers.toString());
for(int n = 0; n < headerArray.length() ; n++) {
JSONObject jsonObject = headerArray.getJSONObject(n);
//Pull date
if(jsonObject.get(Constants.DATE)!=null) {
mail.setDate(jsonObject.getString(Constants.DATE));
}
//Pull Subject
//Pull reply-to address
//Pull delivered-from address
//Pull delivered-to address
Log.d(TAG, "JSON Object : "+ jsonObject);
}
//Log.d(TAG,"header String: "+headers.toString());
} catch (Exception e) {
e.printStackTrace();
}
return mail;
}
I kept these values in the Constant class :
// Data pulled from inside the header
public static final String DATE = "Date";
public static final String SUBJECT = "Subject";
public static final String REPLY_TO = "Reply-To";
public static final String DELIVERED_TO = "Delivered-To";
public static final String FROM = "From";
I don't know if this is the best fix for this, but this works and gives the data as I require.