Serving AWS S3 images from a URL on my site's main domain - node.js

Is it possible to serve AWS S3 images from a URL on my site's main domain?
My site is an Express app running on a URL like https://example.com. I'm using AWS S3 to host my site's images. But instead of using the raw S3 URL, I'd like to use a URL that's on my main domain like this:
https://example.com/image/imagename.jpg
Most of the articles I've read involving using a CNAME record for a subdomain, or hosting your entire site on S3, neither of which is what I'm trying to do.

You can do it using AWS Cloudfront and AWS Route 53 to create a DNS record in conjunct with AWS S3.
Here i attach a tutorial to do it:
https://www.maketecheasier.com/configure-amazon-s3-as-a-content-delivery-network/
https://reidweb.com/2017/02/06/cloudfront-cdn-tutorial/
https://adefy.com/2016/04/25/setting-up-amazon-s3-cloudfront-route53/
If you have a doub,you can tell me the problem and i can help you because the tutorial is old.

From what I understood is you want a url like:
https://example.com/image/imagename.jpg
and don't want to use S3 to host your website then in this case you can create a bucket policy s3:GetObject Permission with a condition using the AWS:refer key that tells that GetObject request must be from specific website.
Here is the sample bucket policy:
{
"Version":"2012-10-17",
"Id":"http referer policy example",
"Statement":[
{
"Sid":"Allow get requests originating from www.example.com and example.com.",
"Effect":"Allow",
"Principal":"*",
"Action":"s3:GetObject",
"Resource":"arn:aws:s3:::examplebucket/*",
"Condition":{
"StringLike":{"aws:Referer":["http://www.example.com/*","http://example.com/*"]}
}
}
]
}
For more information you can read this document
Hope this helps.
Thanks!

Related

Azure API Management - dynamic urls issue

I have a frontend url say - 'https:\test.helloweb.com\account' which is defined a post request /person and I want to direct any request that comes on that url to a backend url like so
'https:\secure.hiddenapi\vi\api\person'.
The above scenario is easy and works now the place where I'm stuck is if there is a request for
Say -
/person/<id>/membership
/person/<id>/membership/<memid>
/person/<id>/accountdetails/
these requests needs to be directed to
https:\\secure.hiddenapi\vi\api\person\<id>\membership
https:\\secure.hiddenapi\vi\api\person\<id>\membership\<memid>
https:\\secure.hiddenapi\vi\api\person\<id>\accountdetails.
I tried defining the operation as /person/* then if i make a request to /person/<id>/membership then the backend url called is https:\\secure.hiddenapi\vi\api\person\membership without the id.
Seems like a simple issue but cannot make any progress!
Here is how APIM maps URLs. Consider:
An API with:
API URL suffix set to "myapi"
Web service URL set to "https://contoso.com/api"
An operation with:
URL template: "/myoperation"
When APIM receives a request it will split it into two parts:
https://service.azure-api.net/myapi/myoperation
^ ^
API base URL operation path
It will then replace API base URL with "Web service URL" specified for matched API and this will become request to backend:
https://contoso.com/api/myoperation
In your case, given that you end up with "person/person" it is likely you have "person" in both Web service URL and operation URL template. So you have two options.
First option is to configure Web service URL as "https://secure.hiddenapi/vi/api" and let all your operation templates start with "/person".
Another way is to set service URL as "https://secure.hiddenapi/vi/api/person" and do not include "/person" into your operation URL templates. It's perfectly fine if operation URL template starts with a variable like "{id}/membership/{memid}".
It's a good practice to split your whole backend API into different APIs in APIM for control and management. So personally I'd prefer first option, and if later there would be a need to make request to backend with prefix different than "/person" I'd create another API.
Try using rewrite URL policy to route to different backend URL for specific operation as follows:
<inbound>
<base />
<rewrite-uri template="/accountdetails/" />
</inbound>

Best practices for using Azure CDN in scenarios that require authentication to access?

I've never configured a CDN before and I'm looking for guidance for how to approach the configuration for my scenario.
For context, I'm currently using Azure Verizon Premium CDN and an Azure App service (.Net Core). My authentication platform is Azure Active Directory.
I have a static web app that has lots of static content (hundreds of files). Users hit the website at (www.mysite.azurewebsites.net/index.html). Index.html then fetches large amounts of static content (various, images, video, metadata). Initially, the requirement was to just serve the content publicly but now I have requirements around restricting access to some the content based on whether or not a user has a certain role & the user hits a certain path. I also need to be able to keep some content public and highly available to end users.
Before authorization requirements came in, I was able to successfully set up a CDN endpoint (www.mysite-cdn.azureedge.net) and point it to my app service no problem! I can hit the site and see the site content from the cdn endpoint without auth no issue.
Problems came when I added authentication to my site my CDN is redirected during the authentication flow back to the origin endpoint. The origin authentication middleware (Nuget: Microsoft.Identity.Web) automatically assembles the callback uri for AAD as "www.mysite.azurewebsites.net/signin-oidc". So the present flow for a user hitting the CDN endpoint returns them to an error page on the app service endpoint. Although if they manually redirect to "www.mysite.azurewebsites.net" they are logged in, I don't want to redirect the user back to origin, I want to keep them on www.mysite-cdn.azurewebsites.net.
Essentially, I want to be able to enable these flows:
Public End User -> CDN Endpoint + Public path -> (CDN request origin and caches) -> End user sees site at CDN endpoint
Internal End User -> CDN Endpoint + Private path -> (CDN request origin and has access) -> User is permitted into site at CDN endpoint
Internal End User -> CDN Endpoint + Private path -> (CDN request origin and DOESN’T have access) -> User is redirected to public CDN endpoint (or unauthorized page)
This is the auth check for each static file in OnPrepareResponse for static file options. This checks authentication before requesting a static asset in this folder on the server. This works fine without the CDN. It should be noted that I also do role checks and this has been simplified for the sake of the example as it repos with Authentication.
OnPrepareResponse = staticContext =>
{
// require authentication
if (authenticate &&
!staticContext.Context.User.Identity.IsAuthenticated)
{
// Redirect back to index sign in page all unauthorized requests
staticContext.Context.Response.Redirect(unauthenticatedRedirectPath);
}
},
I did find this stack overflow which seemed similar to my problem however I am using a different NuGet package (Microsoft.Identity.Web). I implemented a check to redirect however that did not seem to work and cause an infinite loop when trying to login.
Action<MicrosoftIdentityOptions> _configureMsOptions = (options) =>
{
options.Instance = Configuration["AzureAd:Instance"];
options.Domain = Configuration["AzureAd:Domain"];
options.ClientId = Configuration["AzureAd:ClientId"];
options.TenantId = Configuration["AzureAd:TenantId"];
options.CallbackPath = Configuration["AzureAd:CallbackPath"];
options.Events.OnRedirectToIdentityProvider = async (context) =>
{
// This check doesn’t work because the request host always is mysite.azurewebsites.net in this context
// if (context.Request.Host.Value.Contains("mysite-cdn"))
// {
context.ProtocolMessage.RedirectUri = "https://" + "mysite-cdn-dev.azureedge.net/" + Configuration["AzureAd:CallbackPath"];
//}
};
};
I've started looking into Azure Front door, as that seems to be more applicable to my use case but haven't set it up. It provides some level of caching/POP as well as security. It looks like it's also possible to use with Azure AD with some web server tweaking. It would be good to know from others if Azure Front Door sounds like a more sensible CDN solution approach vs Azure CDN.
I've also looked into Azure CDN Token authentication- but that seems to be something that also requires me to stamp each request with an Auth token. It changes my architecture such that I can no longer just 'point' my cdn at the web app and instead my app would give the browser a token so it could securely request each asset.
All that said, I'm looking for guidance on how best to configure an application using a CDN while also using authentication. Thanks!

ServiceStack Metadata Redirect behind a Azure App Gateway not working

My api is hosted on Azure as an App Service with an Azure App Gateway in front of that.
I have set the webhosturl in my startup and that is working as when I view the metadata page, i see the links pointing to the correct location. And those links work. However when I navigate to the base url for my api, it redirects me to the app service url.
Here is a snippet of my startup...
SetConfig(new HostConfig
{
WebHostUrl = "https://api-dev.hsawaknow.net/link/",
DefaultRedirectPath = "/metadata",
DebugMode = AppSettings.Get(nameof(HostConfig.DebugMode), false)
});
Please see the links below and see the differences.
https://api-dev.hsawaknow.net/link/metadata
vs
https://api-dev.hsawaknow.net/link/
You will get an https error as I am using a self signed cert, until I get this figured out. I have seen other posts that say to make this change and that it works, but not for me.
Please help!
I have this figured out. There were a couple things that I had to do.
First thing I had to do was setup the forwarded headers middleware to recognize and process the correct headers coming from the Azure Application Gateway.
services.Configure<ForwardedHeadersOptions>(options =>
{
options.ForwardedHostHeaderName = "X-ORIGINAL-HOST";
options.ForwardedHeaders = ForwardedHeaders.XForwardedHost | ForwardedHeaders.XForwardedProto | ForwardedHeaders.XForwardedFor;
});
This allowed my site to work with links to the correct pages without setting the WebHostUrl. The only caveat about using the Azure App Gateway is that it uses X-ORIGINAL-HOST instead of the standard X-FORWARDED-HOST.
Next, I had to set the DefaultRedirectPath on the HostConfig dynamically based on settings in appsettings.json. In the case of the Azure App Gateway, my public url was https://api-dev.hsawaknow.net/link/, I had to set the redirect to /link/metadata, instead of just metadata, because of how the host header was getting set in the previous step.
It took a few tries, but this configuration works well, when hosting on Azure App Services fronted with an Azure Application Gateway.
Kudos to the mythz for the quick response, which pointed me in the right direction.
Enable the Request Logger so you can see what requests ServiceStack receives.
Does it work when not specifying a WebHostUrl?

S3 Pre-signed URL with custom endpoint via API Gateway, MethodNotAllowed

I'm attempting to use a pre-signed url for an S3 bucket with a custom endpoint. I seem so close, but I keep getting a Method Not Allowed error. Here's where I'm at.
I have an API Gateway which connects an endpoint to a Lambda function. That function, among other things, generates a pre-signed url. Like so,
var s3 = new AWS.S3({
endpoint: 'custom.domain.com/upload',
s3BucketEndpoint: true,
signatureVersion: 'v4'
});
//...
s3.getSignedUrl('putObject', {
ACL: 'bucket-owner-full-control',
Bucket: process.env.S3_BUCKET_NAME,
ContentType: "image/png",
Key: asset.id + ".png"
};
This code successfully returns a url with what appears to be all the correct query params, correct key name, and the url is pointing to my endpoint. When attempting to upload however, I receive the following error:
MethodNotAllowedThe specified method is not allowed against this resource.PUTSERVICE[request id was here][host id was here]
If I remove my custom endpoint declaration from my S3 config, I receive a standard domain prefixed pre-signed url and the upload works fine.
Other notes on my setup.
I have configured the /upload resource on API Gateway to be an S3 passthrough for the PUT method.
I have enabled CORS where needed. On the bucket and on my API. I have confirmed CORS is good, as the browser passes checks.
I have setup my policies. The lambda function has access to the internet from my VPC, it has full S3 access, and it has a trust relationship with both S3 and API Gateway. This execution role is shared amongst the resources.
I am using the axios package to upload the file via PUT.
I have added a CloudTrail log, but it reports the exact same error as the browser...
Temporarily making my bucket public makes no difference.
I've attempted to add the query strings to the API Gateway Request/Response integrations without success.
I've added the necessary content type headers to the request and to the pre-signed url config.
I Googled the heck out of this thing.
Am I missing something? Is this possible? I plan to disable my custom endpoint and move forward with the default pre-signed url for the time being, but long term, I would like the custom endpoint. Worst case I may pay for some AWS support.
Thanks for the help!
I can't find documentation that states a presigned URL supports proxy (alt/custom domain). IMO the use-case to authenticate and grant requests access to AWS resources from an API gateway ( regardless of if you are proxy'ing S3 ) would be to use an API Gateway authorizer w/lambda to allow the request to assume an IAM role that has access to the AWS resources (in this case PUT OBJECT on an s3 bucket)
https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html

Custom subdomain name for blob CDN endpoint causes error 400

I'm trying to set up a custom domain name for a blob CDN endpoint, following these instructions, but can't seem to access my content using the subdomain static.mydomain.com. I've created the following record with my registrar:
Subdomain: static
Type: CNAME
TTL: 7200
Data: blobconatinername.blob.core.windows.net.
For example, I can access this file (note https):
https://blobcontanername.blob.core.windows.net/somefile.mp3
But trying to access this file
http://static.mydomain.com/somefile.mp3
I get an invalid URI error (an error 400):
This XML file does not appear to have any style information associated with it. The document tree is shown below.
<Error>
<Code>InvalidUri</Code>
<Message>
The requested URI does not represent any resource on the server. RequestId:c5ec4859-0001-0079-0bf8-961dfa000000 Time:2016-04-15T09:22:32.1317877Z
</Message>
<UriPath>
http://static.mydomain.com/somefile.mp3
</UriPath>
</Error>
Resolution?
Can you access the file via the CDN endpoint, yourcdnendpoint.azureedge.net/path/to/file?
Shouldn't static.yourdomain.com point to the CDN endpoint, not your blob storage?
Subdomain: static
Type: CNAME
TTL: 7200
Data: yourcdnendpoint.azureedge.net.
Also, the domain you are using must be verified. The process is described at https://azure.microsoft.com/sv-se/documentation/articles/cdn-map-content-to-custom-domain/
This 400 error occurred for us too - the fix was to assign the Origin Host Header value. We are using the Verizon Premium CDN plan in Azure - MS support advised us that Host Header is required despite it appearing as optional in the Azure Portal UI.
CDN Profile => CDN Endpoint => Origin => Origin Host Header

Resources