I've created a lambda function so that I can use it for validation purposes and then proxy the request to the service layer. Then the service layer response contains a binary blob(PDF), which goes through the lambda function then the API gateway finally would reach the client.
The first problem we ran into was the PDF got transformed or corrupted, just returned blank PDF. And then I found this post which did not make any sense to me at first. Until I saw this aws doc. It turns out it's required to encode the binary data into base64 and then put the indictor 'isBase64Encoded' to true. The gateway eventually converts the response back to the binary blob.
TBH, I am new to aws and I don't really understand why this is the way..what's wrong of passing through the original binary blob, why those conversion steps are necessary?
Here are list of things i had to do
Configured / as a Binary Media Type on gateway. (I tried to use application/pdf, but did not work?)
Make sure the response body from the service layer not transformed into string (I am using request, and by default it gives me string). I send encoding: null along with the request
When i get the Buffer data from the service layer, i use Buffer to convert response body into base64 encoding.
In the lambda output, I set isBase64Encoded to true
Finally, get the unaltered PDF...
I am wondering if someone can confirm i am doing in an expected way? Or maybe if there is a better way?
Also, when we set binary support media type to /, doesn't this mean it accepts all media types? But i only want the PDF to be supported.
This doc (https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-payload-encodings.html) should be able to answer your question. And there are two things you need to note:
You can pass the original binary file (blob) as well as a base64-encoded binary file through API Gateway.
Ref: https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-content-encodings-examples-image-lambda.html
*/* (or /) works in your case, but it means the API Gateway will treat all payload as binary data and this breaks payload with text data, for example JSON payload. So, ideally application/pdf should be used as the "Binary Media Type".
Related
Summary: Logic app inline code (which uses NodeJS) is missing Buffer class.
Detailed: I am trying to trigger a logic app when some content is pushed into SFTP. I want to add some meta-data and save the details in the cosmos DB.
The issue is, The name of the file is received as a base64 encoded string in the inline code and Buffer is not available to parse it.
I even tried to create a set variable step (and decode filename there) but I am unable
to pass this variable to the inline code step. (Not supported)
The final option would be to use cloud functions instead of inline code which I am trying to avoid.
Looking for a workaround for conversion.
Logic App error image
link to ms doc
Doesn't support require() statements
Doesn't work with variables
Inline code can only perform the simplest Javascript operations, we may not be able to use Buffer.
As for passing the base64 encoded string, you can put it in Compose first, and then pass it in the inline code.
I suggest you use the built-in base64 related methods in the Azure logic app first.
If this does not meet your needs, you can create an Azure function and then call it in the Azure logic app.
Is it possible to set a custom header when using lambda non proxy integrations?
At the moment I have enabled binary support and I am returning straight from my handler but I have a requirement to set the file name of the download and was planning to use Content-Disposition: attachment; filename="filename.xlsx" but I am not sure how I can do this if I have lambda proxy integration turned off.
Reading this https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-integration-settings-integration-response.html I am not sure if it only works for json responses?
The example shows the body comment as taking a json object but then says there is a base64 encoding option for binary support, but I am just returning my binary data straight from my function and I had not planned to use lambda proxy at all if possible.
I currently have files downloading but I am using temporary files and I want to name the downloads.
# In my service
with tempfile.NamedTemporaryFile(suffix=".xlsx") as tmp:
pd.DataFrame(report_json).to_excel(tmp.name)
bytes_io = BytesIO(tmp.read())
return bytes_io
# In my handler
return base64.b64encode(bytes_io.getvalue())
Using later versions of the serverless framework a custom header for example Content-Disposition can be set like the following.
Integration: lambda
response:
headers:
Content-Type: "'text/csv'"
Content-Disposition: "'attachment; filename=abc.csv'"
I am not sure yet if it is possible to interpolate values from the context into these values.
Apollo Server 2.0 has the ability to receive file uploads as described in this blog post.
However, all the tutorials and blog posts I found only showed how to upload a file. Nobody demonstrated how to actually retrieve the file back to display it onscreen.
Does anybody know how to properly query the file contents for display onscreen?
Also, there's the possibility that maybe there is no way of querying a file and you have to build a separate rest endpoint to retrieve the contents?
Some thoughts:
I imagine the query to be something like
query {
fetchImage(id: 'someid')
}
with the respective server-side definition
type Query {
fetchImage(id : ID!): Upload //maybe also a custom type, but how do I include the actual file contents?
}
Hint: Upload is a scalar type that apollo-server automatically adds to your type definition. It is used for the upload so I imaging it also being usable for the download/query. Please read the blog post mentioned above for more information.
The response from a GraphQL service is always serialized as a JSON object. Technically, a format other than JSON could be used in serialization but in practice only JSON is used because it meets the serialization requirements in the spec. So, the only way to send a file through GraphQL would be to convert the file into some format that's JSON-compatible. For example, you could convert a Buffer to a byte array and send that as an array of integers. You would also have to send the appropriate mime type. It would be up to the client to convert the byte array back into a usable format on receiving the response.
If you go this route, you'd have to use your own scalar or object type -- the Upload scalar does not support serialization so it will throw if you try to use it as an output type (and it's not really suitable for this sort of thing anyway).
However, while doing this is technically possible, it's also inadvisable. Serializing a larger file could cause you to run out of memory since there's no way to stream data through GraphQL (the entire response has to be in memory before it can be sent). It's much better to serve the file statically (ideally using nginx instead of Node). If your API needs to refer to the file, it can then just return the file's path.
You can do this by installing express with apollo server.
apollo-server-express
Install above package and instantiate Express object with Apollo Server as explained in package docs.
Then set the static folder using express like this
app.use("/uploads", express.static("uploads")); //Server Static files over Http
uploads is my static folder & /uploads will server get request to that path
//Now I can access static files like this
http://localhost:4000/uploads/test.jpg
My server receives a file from a HTTP request and uploads this file to IBM Cloud Object Storage.
Moreover, the server allows to recover this file. Recovery is triggered by a get http request that should return said file.
It works fine for "basic" data format, such as text files. However, I encounter problems with more complex types such as images and the "reformating".
Image is uploaded to the datastore. The element stored is the buffer itself:
req.files[0].buffer
When getting the image back from the datastore, how can I transform it back to a readable format for my computer?
The data look like this and it is, on the server, a string:
If you are using ExpressJS you can do this:
const data = req.files[0].buffer;
res.contentType('image/jpeg'); // don't know what type is
res.send(data);
I am currently writing some micro services with Foxx to be consumed by Ember.js. Ember data plays very nicely with JSON API (http://jsonapi.org) responses. So I tried to serialize the Foxx responses with the json-api-serializer (https://www.npmjs.com/package/json-api-serializer) - but with no luck. I only found the forClient method, but this only allows me to operate on the JSON representation of single objects, not the whole response. So my question: Is it possible to implement JSON API with Foxx/ArangoDB?
You can return arbitrary responses from Foxx routes, so it's entirely possible to generate JSON responses that conform to JSON API.
However there's no built-in way to do this automatically.
I don't see anything in json-api-serializer that shouldn't work in Foxx, so I'm not sure what problems you are encountering. You should be able to simply return the output object with res.json(outputFromSerializer) and set the content type with res.set('content-type', 'application/vnd.api+json').
If everything else fails you can just write your own helper functions to generate the boilerplate and metadata JSON API expects.