Netlify noob here.
I'm currently migrating an old Ruby on Rails app to use Netlify for a static site. There are some legacy static pages that we want to keep on our old code base, and these legacy static pages make POST requests to our server.
It seems like redirects for POST requests aren't possible (see the W3 documentation for 301/302 redirects- If the 301 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.), but I was wondering if this is different when you proxy/rewrite the URL.
Currently, we rewrite a user's request to www.domain.com/legacy_slug via Netlify's splat redirect (similar to what the author of this blog post did). Is it possible for this redirect to work as well when the user sends a POST request to www.domain.com, causing it to go to Netlify? Or would I have to change the client's code to POST to <different_subdomain>.domain.com/legacy_slug and migrate the POST endpoint to the different subdomain?
Proxies (https://www.netlify.com/docs/redirects/#proxying) accept POSTs, redirects (https://www.netlify.com/docs/redirects/#basic-redirects) or rewrites (HTTP 200 that transform from one path to another, both on netlify-hosted sites), no.
Kind of a subtle distinction. So - I'd send the POST to some other path (not some other domain - just /place-we-post-to on your Netlify site, and use a proxy redirect to get to your remote service (/place-we-post-to https://legacybackend.com 200 in _redirects)
Related
I'm building a short link application like bit.ly using the MERN stack. I want the static pages of the app to be hosted on a seperate server from the API. My architecture needs to be:
API:
POST - example.com/link - create short link.
PUT - example.com/link:slug - update short link.
GET - example.com/:slug - fetch/redirect short link to long link.
Static site:
GET - example.com - Homepage.
GET - example.com/:slug-that-doesnt-exist - 404 page.
GET - example.com/blog, example.com/docs, example.com/support etc. - Various static pages
The issue I'm having is getting the server to delineate between the two types of requests to the same domain, often on the same path.
example.com/abc123 is the API while example.com/blog is the static site, even though the express router example.com/:slug is matching both. To complicate it example.com/zxy987 hits the API but doesn't exist so needs to redirect the request to the static server. I also can't figure out how to redirect the request itself without the URL changing to the other server. I considered using NGINX but it doesn't appear it could handle the example.com/:slug-that-doesnt-exist type requests since it wouldn't know if they exist at the time it runs.
What is the 'best' way to approach this?
I have a static site through Cloudfront with an S3 origin & custom domain via Route 53. All works well, except that Google has also indexed the Cloudfront distribution url (d123etc.cloudfront.net) as well as my custom domain, leading to duplicate content issues.
I've tried canonical urls, but the distribution remains indexed. It has been suggested to serve up a different robots.txt depending on what domain is being used, which sounds fine, but there is no .htaccess or web server, leaving it to a Lambda Edge function to try and send the different robots.txt.
The problem is that I can't find how in the function to determine if a request is coming from my custom domain or from the direct distribution url. I've tried white-listing the Origin, but it is not sent through when using an S3 origin. I've also tried white-listing the Referer header, but no referrer is sent through when accessing the robots.txt file as it's a direct request.
For the time-being, I'm adding a meta noindex client-side using js on page load (which I realise is too late), and also redirecting client-side to my actual domain in case someone follows the google indexed cloudfront.net domain.
Does anyone know how to detect in Lambda Edge which domain is being used to make the request? Or some other way of blocking Google from indexing the Cloudfront url, just leaving it to index the custom domain.
So I think the way to do this would be to set up a redirect on your hosted webserver. If you check the 'host' in the request header and check for cloudfront.com, send a 301 response code along with your custom domain name.
S3 has a UI way to do this:
https://medium.com/tensult/how-to-do-site-redirection-using-aws-522a4002c645
It seems you'll need a second bucket behind the same cloudfront url but without the custom domain. Then you can set it to redirect all requests to your custom domain.
The browser or bots would then stop trying cloudfront.com because it doesn't return anything, they would automatically (without the user really noticing) to my domain.xyz and all the links would link to your own domain.
I am looking for a solution to interrupt or pause a 301 ( redirect ) request in Chrome dev tools. My scenario is I have an API integration with a local payment gateway which requires 301 redirect to their server ( HTTP GET and then redirect with 301 to an external URL). As 301 HTTP status code happens on the client-side so this can be compromised as the current system does not protect the integrity of the data, for instance, the amount to pay. I might send 100$ and the user might change it to 20$.
To prove this I need to stop the request manually which is very inconvenient and hard to test. I am looking for a solution that implements in one of my favorite browsers above that allows me to config to interrupt or pause the 301 HTTP status code so I can easily modify the value in the URL before resuming the request.
I found a chrome extension to solve this issue.
Requestly is easy to use and test. This is what being mentioned in the extension page:
Chrome Extension to modify HTTP requests (Setup Redirects, Run Custom
Javascript, Modify Headers
But my scenario is about modifying url query string - not being mentioned in the description but it works perfectly.
For a mobile subdomain (m.domain) I'm simply trying to pull images from the common main domain image folder (www.domain.com/images)
After a bit of tinkering, I've apparently gotten the htaccess rewrite code to work successfully but I found out I was unable to use the P flag on rewrite, and that I don't have mod_proxy enabled due to shared hosting.
simple code in my m.domain.com htaccess file:
RewriteRule ^(images/.*) http://www.domain.com/$1 [L]
What benefit am I not receiving by not using the P flag? What's the difference?
When you proxy a request, the server itself does the external request and passes through the data it receives. If the client would request http://m.example.com/asdf, and the server would proxy to http://www.example.com/asdf, the server would make a proxy-request to that url, then return that data for http://m.example.com/asdf to the client. For the outside world there is duplicate content on http://m.example.com/asdf and http://www.example.com/asdf, and any client requesting both urls will cache things twice. This is not ideal.
With your current approach you make an implicit temporary redirect. When the client requests http://m.example.com/asdf, the server sends back a redirect-response, which tells the client to try again on http://www.example.com/asdf. The client tries again on that url, and receives the data from the server. For the outside world it is clear that the content is in one place, and when caching the data it will only cache it for http://www.example.com/asdf. If you make the redirect permanent, it will skip the redirect in some cases and directly request http://www.example.com/asdf, reducing the load on your server.
Unless you have a good reason to proxy a request (e.g. because the server with the data is only available locally), you should not use a proxy request. If http://m.example.com/ and http://www.example.com are hosted on the same server, and you are unconcerned about hosting duplicate content, then you should use an internal rewrite. If you are concerned about duplicate content, you use an external redirect, using the [R] flag.
Is it possible to manipulate an inbound request at the IIS level, before it even gets assigned to site on the server?
Essentially, I want to rewrite this --
www.somegenericdomain.com?site=someotherdomain
To this --
www.someotherdomain.com
And I need to do this before IIS picks which site the request belongs to, so I need to change the host header prior to this point.
Possible, or crazy? We're running IIS7.
You can rewrite, redirect, or proxy requests.
Rewrite changes the request, but does not change the site to which it is assigned. With a rewrite you can:
return an HTTP error code (503, 404, 401, etc);
manipulate the query string or URL path. one example is to transform a query string param into a URL path element. www.server.com/default.aspx?s=foo becomes www.server.com/foo, or vice versa.
set headers in the request.
Redirect sends back a 301 or 302 response to the browser with an updated address. You can receive a request for www.example.com/foo and respond to the browser with a 302 and an updated address of www.otherdomain.com , etc.
Proxy the request. In this case the web server is said to act as a "transparent proxy". It means the initial IIS server can call out to a second server, grab the response, and then package it up back to the original requester.
These three actions are often done in combination. The tools used to perform these actions are called "URL Rewriters". IIS7 has a built-in option from Microsoft (The IIS URL Rewrite Module), and there are third-party options as well, some free and some commercial, for IIS6, IIS7, and other non-Windows web servers. Apache's mod_proxy is the big one for Linux. All of these tools do basically the same kinds of things.
To answer your specific question, NO, you cannot rewrite a request from one domain to another. For web servers, rewrite is a meaningful term, and a URL Rewrite excludes the possibility of a server change.
It is possible though, to transform a request from one server to another, either via redirect or proxy. One of those may actually be what you want, when you ask about "rewriting" a request.
I guess the whole thing is possible, but not in the way of running before IIS. One part of the server even works as a low-level driver.
But you may use URL rewriting solutions such as mod_rewrite module of Helicon Ape http://www.helicontech.com/ape/doc/mod_rewrite.htm. Having set the software globally for all the sites, you may get what you need as follows:
RewriteEngine on
RewriteCond %{HTTP_HOST} www.somegenericdomain.com [NC]
RewriteProxy (.*) http://www.someotherdomain.com$1