AWS Lambda fails to return PDF file - node.js

I have created a lambda function using serverless. This function is fired via API Gateway on a GET request and should return a pdf file from a buffer. I'm using html-pdf to create the buffer and trying to return the pdf file with the following command
let response = {
statusCode: 200,
headers: {'Content-type' : 'application/pdf'},
body: buffer.toString('base64'),
isBase64Encoded : true,
};
return callback(null, response);
but the browser is just failing to load the pdf, so I don't know exactly how to return the pdf file directly to the browser. Could'nt find a solution for that.

well, I found the answer.
The settings in my response object are fine, I just had to manually change the settings in API Gateway for this to work in the browser. I have added "*/*" to binary media types under the binary settings in API Gateway console
API GATEWAY
just log into your console
choose your api
click on binary support in the dropdown
edit binary media type and add "*/*"
FRONTEND
opening the api url in new tab (target="_blank"). Probably the browser is handling the encoded base 64 response, In my case with chrome, the browser just opens the pdf in a new tab exactly like I want it to do.

After spending several hours on this I found out that if you set Content handling to Convert to binary (CONVERT_TO_BINARY) the entire response has to be base64, I would otherwise get an error: Unable to base64 decode the body.
Therefore my response now looks like:
callback(null, buffer.toString('base64'));
The Integration response:
The Method response:
And Binary Media Types:

If you have a gigantic PDF, then it will take a long time for Lambda to return it and in Lambda you are billed per 100ms.
I would save it to S3 first then let the Lambda return the S3 url to the client for downloading.

I was having a similar issue where pdf where downloaded as base64 and started happening when changed the serverles.yml file from:
binaryMediaTypes:
- '*/*'
to
binaryMediaTypes:
- 'application/pdf'
- '....other media types'
The issue is because the way AWS implemented this feature. From aws documentation here:
When a request contains multiple media types in its Accept header, API
Gateway honors only the first Accept media type. If you can't control
the order of the Accept media types and the media type of your binary
content isn't the first in the list, add the first Accept media type
in the binaryMediaTypes list of your API. API Gateway handles all
content types in this list as binary.
Basically if the first media type contained in the accept request header is not in your list in binaryMediaTypes then you will get base64 back.
I checked the request in the browser and the first media type in the accept header was text/html so I got it working after changing my settings to:
binaryMediaTypes:
- 'application/pdf'
- '....other media types'
- 'text/html'
Hope this helps anyone with the same issue.

Above solution is only for particular content-type. You can't more content type.
Follow only below two-step to resolve multiple content type issue.
Click on the checkbox of Use Lambda Proxy integration
API gateway --> API --> method --> integration request
Create your response as
let response = {
statusCode: 200,
headers: {
'Content-type': 'application/pdf',//you can change any content type
'content-disposition': 'attachment; filename=test.pdf' // key of success
},
body: buffer.toString('base64'),
isBase64Encoded: true
};
return response;
Note* - It is not secure

Instead of doing all this. It's better to use serverless-apigw-binary plugin in your serverless.yaml file.
Add
plugins:
- serverless-apigw-binary
custom:
apigwBinary:
types:
- "application/pdf"
Hope that will help someone.

Related

Model Derivative Forge API - Field uploadKey not found in signed URL endpoint

I was looking to play around with the Forge API and am trying to translate a rvt file into a dwg. I am just following the steps given in the "Step-by-Step" tutorials and this second step in Task 2
https://forge.autodesk.com/en/docs/model-derivative/v2/tutorials/translate-to-obj/task2-upload_source_file_to_oss/
says to make a post request to this endpoint to get the signed url https://developer.api.autodesk.com/oss/v2/buckets/<YOUR_BUCKET_KEY>/objects/<YOUR_OBJECT_KEY>/signeds3upload?minutesExpiration=<LIFESPAN_OF_URL>.
I make the request and receive a '{'reason': 'Field uploadKey not found'}'. Which in the steps, it shows you get the uploadKey from this endpoint ? So either I'm missing something really big here, or these steps are too smart for a 5 year old.
Here is what I'm passing into my post request:
header = {
'Authorization': 'Bearer ' + access_token,
'Content-Type': 'application/json'
}
body = {
'ossbucketKey': 'bucketName',
'ossSourceFileObjectKey': 'test.rvt',
'access': 'full',
'policyKey': 'transient'
}
Note that the new direct-to-s3 upload consists of multiple steps:
You generate an upload URL using the GET buckets/:bucketKey/objects/:objectKey/signeds3upload
You upload your data to the URL
You complete the upload using the POST buckets/:bucketKey/objects/:objectKey/signeds3upload endpoint you mentioned, incl. the uploadKey you received in step 1
For more details you can refer to this blog post: https://forge.autodesk.com/blog/data-management-oss-object-storage-service-migrating-direct-s3-approach.

Making a PATCH method request with a file to Common Data Service

I'm trying to create a Python script which can use the PATCH method to upload a file into MS's Common data service. I'm successfully making GET, POST, PATCH, and DELETE calls with simple data, but have so far been unable to configure it so that I can upload a file.
I've been using the Requests library for Python, with the requests.patch function in order to try updating the data. I'm attempting to upload a .csv file into the field, the file which i'm uploading has a filesize of 1kb.
If I upload the data directly into the common data service through the in-built data interface, my browser is able to correctly make a PATCH call. I've attempted to copy the call as closely as I can, but have had zero success.
File field in common data service
PATCH call in web browser
What is the correct way to make a PATCH request with a file to Microsoft's Common data service?
Made a mistake with the url in my request - I had missed out which field I was uploading data to
Incorrect URL:
https://90g9j3gf.crm4.dynamics.com/api/data/v9.0/test_entity(34cd854c-1175-4778-bf95-e1ce12dea3b0)
Corrected URL:
https://90g9j3gf.crm4.dynamics.com/api/data/v9.0/test_entity(34cd854c-1175-4778-bf95-e1ce12dea3b0)/test_field
The code I used to make the request:
Import requests
http_headers = {
'Authorization': 'Bearer ' + token['access_token'],
'Content-Type': 'application/octet-stream',
'x-ms-file-name': 'test.csv'
}
filedata = open("project-folder\\test.csv", "rb")
patch_req = requests.patch(
url, # My URL is defined elsewhere
headers=http_headers,
data=filedata
)
This now works correctly for me

Wallet Pass auto update web service using aws api gateway

I am working on a web service to update Apple Wallet passes using AWS Lambda/API gateway/NodeJS. The Apple wallet hit the api to get update pass but each time I am getting following error:
encountered error: Received invalid pass data (The pass cannot be read because it isn’t valid.)
I have tried the same URL in the browser to get the pass. The pass is downloading every time and its opening a valid pass every time. But its not working when Apple wallet hit the URL. I have tried same URL in Postman it gives me base64 instead of binary data.
I have tried to achieve the same functionality with NodeJS and deployed on heroku, its working properly with Wallet(also gives binary in Postman). But I need to use AWS Lambda/API gateway/NodeJS.
I am not sure, if AWS changing something while delivering binary data.
Any help on this is appreciated.
I just experienced this and spent hours trying to diagnose what was happening.
For anyone using AWS API Gateway & Lambda for their PassKit web service endpoints, there's a major "gotcha" (at least as of the date of my response) with how API Gateway's logic determines whether it needs to convert a response from base64 ==> binary.
If you inspect the request headers from Apple Wallet / PassKit, you'll see that the Accept header is */*.
API Gateway apparently iterates through the items in the request Accept header and determines if there is a match with any of the Binary Media Types you've defined under Your API Name > Settings. It will use the first match it finds and then, as you'd hope, convert the base64 string (from Lambda) to binary.
Here's the crazy part -- if you define application/vnd.apple.pkpass as one of your "please convert to binary" media types, requests from Apple Wallet / PassKit will not work. Why? Well, AWS (for whatever reason...) hasn't programmed */* to match any type ... it will literally only match */*.
As a result, the Accept header's */* will not match with application/vnd.apple.pkpass and your base64-encoded .pkpass response (from Lambda) will not be converted to binary, causing PassKit to choke + report errors.
TL;DR -- there is some goofiness with AWS API Gateway. To return PassKit pass data successfully, you need to add */* (not application/vnd.apple.pkpass) under Your API Name > Settings > Binary Media Types.

How to Convert MS SharePoint $metadata XML Response into Standard Odata JSON in NodeJS?

I am working on MS SharePoint 2013 integration and in my use case I want to work on JSON data only.
I am able to convert all the API XML responses like File, Folder, List, ListItem, etc into OData JSON using Node Module - datajs_vanilla, but I am not able to convert https://mysite/_api/$metadata into OData JSON.
I am getting below error while executing datajs for $metadata endpoint:
error:{"message":"no handler for data"}
Below is my code snippet:
OData.read({
requestUri: 'https://example.com/_api/$metadata',
headers: {
'Authorization': 'Bearer token_value',
'Accept':'application/xml;charset=utf-8'
}
}, function (data, response) {
console.log("Operation succeeded."+JSON.stringify(data));
}, function (err) {
console.log("Error occurred " + JSON.stringify(err));
});
I am missing something here to convert into JSON?
or
Could you please suggest me possible solution to convert edmx sharepoint $metadata XML response into OData JSON using node module?
Thanks.
See answer for similar question Get OData $metadata in JSON format .
Metadata document is not defined using OData atom/xml format that's why datajs vanilla is failing to parse it.
Metadata format is CSDL format. Example of metadata doc - http://services.odata.org/V4/TripPinServiceRW/$metadata.
Usually metadata used for discovery and client generation since it has type info and all entity relations. Having service responses being serialized to json and service documentation in json should be sufficient for you to work with service.

File not supported error while uploading profile image file to IBM Connections using REST API

I'm trying to upload a jpeg file for profile image using the Profiles REST API to IBM Connections_v5.0. I however get an error message "The type of the photo file you provided is not supported".
I'm however able to upload the same file directly using the Connections UI interface directly. I'm setting the MIME type correctly as "image/jpeg".
Also tried with GIF and PNG images but get the same error message.
Any pointers would be very helpful.
I'm just using FF restclient addon to fire a REST call. So basically doing a PUT on /profiles/photo.do?key=....
Content-Type is set as "image/jpeg" and the payload consists of the image data in binary (base 64) encoded.
The payload should just be the binary of the image, there is no need to Base64 encode it.
You should refer to Adding a Profile Photo
You need to use a Key (great stackoverflow post here)
If you know the key for a user's profile you can do the following:
key — This is generated by Connections itself during the population
process. It is used to define the users profile within the context of
Profiles and provides Connections with the ability to associate
content with a user when the users LDAP information has been altered.
It provides a separation of identity and helps facilitate user content
synchronization for Connections.
Once you have a key, you make the following request
URL: https://profiles.enterprise.example.com/profiles/photo.do?key=
b559403a-9r32-2c81-c99w-ppq8bb69442
METHOD: PUT
CONTENT-TYPE: image/png
input the binary content on the stream
you should be able to use "image/jpeg", "image/jpg", "image/png" or "image/gif"
If you have an error after the PUT method, you should add the SystemOut.log lines which are relevant.

Resources