I'm attempting to use a pre-signed url for an S3 bucket with a custom endpoint. I seem so close, but I keep getting a Method Not Allowed error. Here's where I'm at.
I have an API Gateway which connects an endpoint to a Lambda function. That function, among other things, generates a pre-signed url. Like so,
var s3 = new AWS.S3({
endpoint: 'custom.domain.com/upload',
s3BucketEndpoint: true,
signatureVersion: 'v4'
});
//...
s3.getSignedUrl('putObject', {
ACL: 'bucket-owner-full-control',
Bucket: process.env.S3_BUCKET_NAME,
ContentType: "image/png",
Key: asset.id + ".png"
};
This code successfully returns a url with what appears to be all the correct query params, correct key name, and the url is pointing to my endpoint. When attempting to upload however, I receive the following error:
MethodNotAllowedThe specified method is not allowed against this resource.PUTSERVICE[request id was here][host id was here]
If I remove my custom endpoint declaration from my S3 config, I receive a standard domain prefixed pre-signed url and the upload works fine.
Other notes on my setup.
I have configured the /upload resource on API Gateway to be an S3 passthrough for the PUT method.
I have enabled CORS where needed. On the bucket and on my API. I have confirmed CORS is good, as the browser passes checks.
I have setup my policies. The lambda function has access to the internet from my VPC, it has full S3 access, and it has a trust relationship with both S3 and API Gateway. This execution role is shared amongst the resources.
I am using the axios package to upload the file via PUT.
I have added a CloudTrail log, but it reports the exact same error as the browser...
Temporarily making my bucket public makes no difference.
I've attempted to add the query strings to the API Gateway Request/Response integrations without success.
I've added the necessary content type headers to the request and to the pre-signed url config.
I Googled the heck out of this thing.
Am I missing something? Is this possible? I plan to disable my custom endpoint and move forward with the default pre-signed url for the time being, but long term, I would like the custom endpoint. Worst case I may pay for some AWS support.
Thanks for the help!
I can't find documentation that states a presigned URL supports proxy (alt/custom domain). IMO the use-case to authenticate and grant requests access to AWS resources from an API gateway ( regardless of if you are proxy'ing S3 ) would be to use an API Gateway authorizer w/lambda to allow the request to assume an IAM role that has access to the AWS resources (in this case PUT OBJECT on an s3 bucket)
https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html
Related
I have node/express APIs running in Lambda function.
The API endpoint is {domain}/api/user/{username} where I pass username in URL itself.
example: to get userA detail, endpoint will be xxx.com/api/user/userA
also sending user_id:xxx in header.
Hitting above endpoint using API gateway/Lambda returns the data without any error.
Problem occurs when I use % in username.
Assume I have username as userA% .
Endpoint would become: xxx.com/api/user/userA%
Now, the problem is when I run this in my local machine with node/express/mysql api with endpoint localhost:2000/api/user/userA%, it returns the data.
But the same API using API gateway/Lambda : xxx.com/api/user/userA% throws
CORS: PreflightMissionAllowOriginHeader.
I have configured some CORS policies as shown in below image but can't seem to figure out what should I configure more to allow this type of requests.
I have a NodeJS application that runs on an EC2 instance that serves API to my customers. EC2 instance have a Instance Role that grants the minimum permissions for the application to access services it needs ( i need sqs, s3 Read and write, and ses ). One particular endpoint in my api is for creating a signed url, in order to be able to access s3 files, and to create the signed url i use an IAM user with only s3 read access to that Bucket.
My issue is that, whenever that endpoint is called the AWS credentials are set using
const awsConfig = {
region,
accessKeyId: ${keyofreadonlyuser},
secretAccessKey: ${secretofreadonlyuser},
};
AWS.config.update(awsConfig);
This way, all subsequent calls to aws sdk will use that credentials resulting in a Access Denied error.
I've tried to set accessKeyId: null, secretAccessKey:null and than call AWS.config.update, but the credentials are not cleared.
What is the best way to handle situations like that ?
I would recommend that instead of updating the default config, you instead use two boto3 sessions objects:
the default, implicitly-created session, that's associated with the assumed IAM role
an explicitly-created session, that's associated with the IAM user credentials
Specifically for the 2nd use case, pass the IAM user credentials to the session constructor.
Is it possible to serve AWS S3 images from a URL on my site's main domain?
My site is an Express app running on a URL like https://example.com. I'm using AWS S3 to host my site's images. But instead of using the raw S3 URL, I'd like to use a URL that's on my main domain like this:
https://example.com/image/imagename.jpg
Most of the articles I've read involving using a CNAME record for a subdomain, or hosting your entire site on S3, neither of which is what I'm trying to do.
You can do it using AWS Cloudfront and AWS Route 53 to create a DNS record in conjunct with AWS S3.
Here i attach a tutorial to do it:
https://www.maketecheasier.com/configure-amazon-s3-as-a-content-delivery-network/
https://reidweb.com/2017/02/06/cloudfront-cdn-tutorial/
https://adefy.com/2016/04/25/setting-up-amazon-s3-cloudfront-route53/
If you have a doub,you can tell me the problem and i can help you because the tutorial is old.
From what I understood is you want a url like:
https://example.com/image/imagename.jpg
and don't want to use S3 to host your website then in this case you can create a bucket policy s3:GetObject Permission with a condition using the AWS:refer key that tells that GetObject request must be from specific website.
Here is the sample bucket policy:
{
"Version":"2012-10-17",
"Id":"http referer policy example",
"Statement":[
{
"Sid":"Allow get requests originating from www.example.com and example.com.",
"Effect":"Allow",
"Principal":"*",
"Action":"s3:GetObject",
"Resource":"arn:aws:s3:::examplebucket/*",
"Condition":{
"StringLike":{"aws:Referer":["http://www.example.com/*","http://example.com/*"]}
}
}
]
}
For more information you can read this document
Hope this helps.
Thanks!
I have created one API endpoint for lambda function, as - https://XXXXXXXXX.execute-api.us-east-1.amazonaws.com/XXXX/XXXXXXXXXXXX/ which is GET method.
While calling that endpoint from postman it is giving me
{
"message": "'XXXXXXXXX3LPDGPBF33Q:XXXXXXXXXXBLh219REWwTsNMyyyfbucW8MuM7' not a valid key=value pair (missing equal-sign) in Authorization header: 'AWS XXXXXXXXX3LPDGPBF33Q:XXXXXXXXXXBLh219REWwTsNMyyyfbucW8MuM7'."
}
This is a screenshot of the Amazon Lambda Upload Site: http://i.stack.imgur.com/mwJ3w.png
I have Access Key Id & Secret Access Key for IAM user. I used it all but no luck. Can anyone suggest tweak about this.
If you're using the latest version of Postman, you can generate the SigV4 signature automatically. The region should correspond to your API region (i.e. "us-east-1") and the service name should be "execute-api"
This is not a solution but it has helped me more than once:
Double-check that you are actually hitting an existing endpoint! Especially if you're working with AWS. AWS will return this error if you don't have the correct handler set up in your Lambda or if your API Gateway is not configured to serve this resource/verb/etc.
I am saving some images/objects in aws s3 bucket from my application. First i am getting signed url from nodejs service api and uploading images or files to singed url using jquery ajax. I can open image or object using the link provided in the properties (https://s3.amazonaws.com/bucketname/objectname).
I want to provide security for each uploaded object. Even by chance if any anonymous user gets the link (https://s3.amazonaws.com/bucketname/objectname) somewhere he should not be able to open it. They (objects) should be accessed and open only cases like when request has some headers key values etc. I tried server side encryption by specifying header key values in request as shown below.
var file = document.getElementById('fileupload').files[0];
$.ajax({
url: signedurl,
type: "PUT",
data: file,
header:{'x-amz-server-side-encryption':'AES256'},
contentType: file.type,
processData: false,
success: function (result) {
var res = result;
},
error: function (error) {
alert(error);
}
Doesn't sever side encryption keep encrypted object on s3 bucket storage? Does it only encrypts while transferring and decrypts before saving on s3 storage?
If it stores encrypted object on s3 storage then how can i open it using the link shown in properties.
Server-Side Encryption (SSE) in Amazon S3 encrypts objects at rest (stored on disk) but decrypts objects when they are retrieved. Therefore, it is a transparent form of encryption.
If you wish to keep objects in Amazon S3 private, but make them available to specific authorized users, I would recommend using Pre-Signed URLs.
This works by having your application generate a URL that provides time-limited access to a specific object in Amazon S3. The objects are otherwise kept private so they are not accessible.
See documentation: Share an Object with Others