ServiceStack CORS request failing even though OPTIONS preflight checks out - servicestack

I have enabled the CORS feature in ServiceStack, for all verbs, standard headers plus a few custom ones, and all origins. From my Angular application, I am getting the CORS "No 'Access-Control-Allow-Origin' header is present on the requested resource" error when trying to make a PUT call to the server. If I look at my traffic, I see the OPTIONS preflight request to the resource returning with a valid 200 message, and the ACAO header is present and set to *.
// CORS PREFLIGHT REQUEST
OPTIONS /referral HTTP/1.1
Host: api.mydomain.com
Connection: keep-alive
Access-Control-Request-Method: PUT
Origin: http://127.0.0.1:9000
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36
Access-Control-Request-Headers: accept, x-uatoken, x-ualocation, content-type
Accept: */*
Referer: http://127.0.0.1:9000/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
// CORS RESPONSE
HTTP/1.1 200 OK
Cache-Control: private
Vary: Accept
Server: Microsoft-IIS/8.0
X-Powered-By: ServiceStack/4.020 Win32NT/.NET
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Headers: Content-Type, X-UAToken, X-UAUser, X-UALocation, Authorization
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Thu, 05 Jun 2014 17:27:47 GMT
Content-Length: 0
Note: I am using angular-file-upload library to make this request as multipart/form-data (pulling files from Request.Files and deserializing data from Request.FormData on the server). Debugging the second request (the actual PUT) in Chrome has that message about "Provisional headers are shown", so I'm not sure how useful that data is:
Accept:application/json, text/plain, */*
Content-Type:multipart/form-data; boundary=----WebKitFormBoundary3rvpR6k8pz4rghGy
Origin:http://127.0.0.1:9000
Referer:http://127.0.0.1:9000/
User-Agent:Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36
X-UALocation:7
X-UAToken:yoqoByj-T1SBDHCYir92JQ
Request Payload
...etc...
Any ideas?

I see the preflight check is requesting a PUT verb? You might want to check into setting up IIS for PUT/DELETE verbs (it's a whole nother can of worms). Or just try the request as a POST to see if that works, then you know it might be just the verb in IIS.

I've dealt with this issue in the past, and I can't believe I didn't recognize it until now. It's common in any runtime environment you deal with for file uploads like PHP or Tomcat. The problem isn't your CORS setup at all (since the preflight check is passing). Unless its something entirely else my guess is you ran into the ASP.NET and IIS max request limit! By default these are both very low (4MiB). Very simple to solve, just need to add some stuff to your web.config. The IIS settings are only necessary if you on IIS7.5+ I believe. It may also be recommended to change the executionTimeout depending on how long you are going to allow the upload to run (which will depend on speed of client and size of file).
I finally recognized the issue for what it was when I finally ran the ServiceStack project in debug mode and told it to raise all .NET exceptions to me. Attempted to upload a large enough file (I was testing in curl with really small ones! doh!), and then the exception hit me plain and clear. System.Web.HttpException "Maximum request length exceeded."
Here's example of what you need to do
<system.web>
<!-- maxRequestLength size is specified in kiB (int32). this controls max length handled by ASP.NET
set to 2GiB currently
executionTimeout is seconds before request is shutdown.
set to 2 hours currently
-->
<httpRuntime targetFramework="4.5" maxRequestLength="2097152" executionTimeout="7200" />
</system.web>
<system.webServer>
<security>
<requestFiltering>
<!-- size is specified in bytes (uint32). this controls max length of IIS
set to 2GiB currently -->
<requestLimits maxAllowedContentLength="2147483648" />
</requestFiltering>
</security>
</system.webServer>

Related

Granting READER role access to a Subscription in Azure works fine in Postman, but not via Angular. Why?

OK, I might be missing something simple here in Angular, but I could really use some help. I am trying to grant a Service Principal READER role to a Subscription programmatically. If I use PostMan, it works fine. However, when I send the same PUT request via Angular6 I get a 400 error from Azure that says:
The content of your request was not valid, and the original object
could not be deserialized. Exception message: 'Required property
'permissions' not found in JSON. Path 'properties', line 1, position
231.'
The JSON being sent in both cases is:
{
"properties":
{
"roleDefinitionId":"/subscriptions/{some_subscription_guid}/providers/Microsoft.Authorization/roleDefinitions/acdd72a7-3385-48ef-bd42-f606fba81ae7",
"principalId":"{some_service_provider_guid}"
}
}
I've captured traffic from both requests, and they show as application/json payloads on the PUT. So I am at a loss of what is deserializing incorrectly through Azure that is causing this error. I am trying to follow the REST instructions documented here: https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-rest
Any ideas what I am missing?
UPDATE
Adding the RAW REQUEST per request. I have replaced any sensitive data (access token, GUIDs etc) without changing anything else from the Fiddler output.
PUT https://management.azure.com/subscriptions/<VALID_SUBSCRIPTION_WAS_HERE>/providers/Microsoft.Authorization/roleDefinitions/7ec2aca1-e4f2-4152-aee2-68991e8b48ad?api-version=2015-07-01 HTTP/1.1
Host: management.azure.com
Connection: keep-alive
Content-Length: 233
Accept: application/json, text/plain, */*
Origin: http://localhost:4200
Authorization: Bearer <VALID_TOKEN_WAS_HERE>
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Content-Type: application/json
Referer: http://localhost:4200/token/<VALID_DOMAIN_WAS_HERE>.onmicrosoft.com/graph
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
{"properties": { "roleDefinitionId":"/subscriptions/<VALID_SUBSCRIPTION_GUID_HERE>/providers/Microsoft.Authorization/roleDefinitions/acdd72a7-3385-48ef-bd42-f606fba81ae7", "principalId":"<VALID_OBJECTID_HERE>" }}
Alright, I finally figured out what was going on here. It appears that I was posting to the wrong endpoint. I need to be posting to roleAssignment and not roleDefinitions.
So why did it work in PostMan? It seems there is a fallback from a previous version of the API that supported both when using legacy clients, which for some reason PostMan fell under. However, when posting via Angular it was actively rejecting it.
End result... send to "/Microsoft.Authorization/roleassignments/" with an API version later than "api-version=2015-07-01". All will work.

CORS Header Missing on Angular Resource Requests Only

I have a working node/express backend running on localhost. I'm building a project application that needs to fetch data from goodreads api. When I execute the request, I get:
Cross-Origin Request Blocked:
The Same Origin Policy disallows reading the remote resource at
https://www.goodreads.com/book/title.json?author=Arthur+Conan+Doyle&key=[my_key]&title=Hound+of+the+Baskervilles.
(Reason: CORS header 'Access-Control-Allow-Origin' missing).1 <unknown>
Server side, everything is working correctly. I have enabled CORS, and when I check the headers, 'Access-Control-Allow-Origin' is available on everything coming from my server after checking the header in Firefox and Chrome dev tools. When I make a request via $resource, however, 'Allow-Access...' is not present in my header. Here is the code for the resource:
.factory('goodReads', function($resource) {
return $resource('https://www.goodreads.com/book/title.json');
})
.controller('AddBookSelectorController', function($resource, goodReads) {
this.fetch = function() {
var key = '[my_key]';
var data = goodReads.query({author: 'Arthur Conan Doyle', key: key, title: 'Hound of the Baskervilles'});
console.log(data);
};
});
I'm calling fetch via ng-click, and everything executes fine except I get the CORS error. Can anyone point me in the right direction? I am new to angular, and my suspicion is there is a problem with my resource request or something in configuration, but I can't seem to find an answer to fix my problem in the documentation or other stackoverflow questions.
Update 3: It is not a localhost issue. I tried pushing it to my domain and using a simple button which ran an xhr request to the OpenBooks api, and the problem got worse. It is hosted via Openshift, and now the 'Allow-Control-Access-x' headers are gone even for other files on my server. Really beginning to bang my head against the wall here. I am removing the Angular tags, because it has nothing to do with Angular.
UPDATE 2: I got it working after installing 'Allow-Control-Allow-Origin' extension in Chrome. Has my problem been the fact that I'm running this on localhost? Or is there something else going on? The header is still not being set without the extension.
UPDATE: I've been working on this since 8am, and still no luck. I have tried rewriting the request using Angular's $http and also with Javascript's xhr following the example from HTML5 Rocks | Using Cors and I'm still having the same problem with each method. Like I said, the necessary header information is available from files on my server, but it breaks when I make requests to other sites.
I'm starting to think this might not be an Angular problem, but I really have no clue. Just to be safe, here is the code I added to Express to enable CORS, including app.use so you can get an idea for where I called it:
app.use(logger('dev'));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(cookieParser());
app.use(function(req, res, next) {
res.header("Access-Control-Allow-Origin", "*");
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept, Authorization, Content-Length");
res.header("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS");
res.header("Access-Control-Allow-Credentials", "true");
next();
});
app.use(express.static(path.join(__dirname, 'public')));
app.use('/', routes);
Edit: Here are the headers from the API request:
Request Headers
Host: www.goodreads.com
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0
Accept: application/json, text/plain, */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Referer: http://localhost:3000/
Origin: http://localhost:3000
Connection: keep-alive
Response Headers
Cache-Control: max-age=0, private, must-revalidate
Content-Encoding: gzip
Content-Length: 686
Content-Type: application/json; charset=utf-8
Date: Wed, 02 Sep 2015 17:20:35 GMT
Etag: "a2be782f32638d2a435bbeaf4b01274a-gzip"
Server: Server
Set-Cookie: csid=BAhJIhg1MzgtNTk4NjMzNy0wNzQ4MTM5BjoGRVQ%3D--afed14b563e5a6eb7b3fa9005de3010474230702; path=/; expires=Sun, 02 Sep 2035 17:20:33 -0000
locale=en; path=/
_session_id2=fd45336b8ef86010d46c7d73adb5f004; path=/; expires=Wed, 02 Sep 2015 23:20:35 -0000; HttpOnly
Status: 200 OK
Vary: Accept-Encoding,User-Agent
X-Content-Type-Options: nosniff, nosniff
X-Frame-Options: ALLOWALL
X-Request-Id: 1K8EJWG30GWDE4MZ4R5K
X-Runtime: 2.277972
X-XSS-Protection: 1; mode=block
Headers for the .js file from my server:
Request
Host: localhost:3000
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Referer: http://localhost:3000/
Cookie: _ga=GA1.1.1924088292.1439681064; connect.sid=s%3AB4O0Up9WF5iqkfky__I0XCiBD2aMATlq.gbJUC9GseqnJvRTEIbcwxD6cwFQeL7ljNScURCJ5As0
Connection: keep-alive
If-Modified-Since: Wed, 02 Sep 2015 17:08:40 GMT
If-None-Match: W/"886-14f8f0828c1"
Cache-Control: max-age=0
Response:
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Cache-Control: public, max-age=0
Connection: keep-alive
Date: Wed, 02 Sep 2015 17:20:30 GMT
Etag: W/"886-14f8f0828c1"
Last-Modified: Wed, 02 Sep 2015 17:08:40 GMT
X-Powered-By: Express
access-control-allow-headers: Origin, X-Requested-With, Content-Type, Accept
I guess this problem exposed my ignorance, but maybe this will help other newbies to CORs like me. I finally figured out the problem after getting a copy of CORs in Action and working through the first example using the Flickr API.
My problem had nothing to do with the backend, Angular, jQuery's .ajax method, or xhr. All of my requests were properly formatted. The problem was the APIs I attempted to use did not have CORs enabled on their server. O.o As soon as I changed the data type to jsonp, everything went through.
Anyway, for you newbs out there like me, here are some pointers to help you if you run into this problem:
1. Don't assume the API you are using has CORs enabled
I don't know why, but I blindly picked two APIs that don't have CORs enabled, which is what caused all the fuss for me. I have never run into this problem before because the work I have done with APIs have always been from big companies like Flickr that had CORs enabled. If they don't set Access-Control-Allow-Origin on their server, you can request them to enable it and use JSONP in the meantime.
If the API has an option for a callback at the end, that's a good sign you should use JSONP for your request. JSONP works by wrapping your request in a callback and exploiting a feature of the script tag. Scripts can pull other scripts from any domain, so it works as a hack to get the data. Here's a good link that helped me. Exactly What is JSONP? | CameronSpear.com
2. Check The Response Headers
I got tricked by this, but remember that the response header on your request to an external API is the response from their server, not yours. It doesn't matter if CORs is enabled on your server, you are making the request to someone else, and the browser automatically sends your information to them in the request header. Remember, all of this checking is done by the browser for security reasons, so its doing the heavy lifting for you on the request side based on your ajax call. If Access-Control-Whatever doesn't show up in the response header, they don't have CORs enabled. If you are working on the frontend and requesting someone else's data, you can't do anything about it. Use JSONP and your problems will disappear (probably).
This whole fiasco for me started because I was confusing responses coming from my server with responses coming for their server. I correctly enabled CORs on my own server, but I was thinking it wasn't attaching the origin information to the request header which is why it was absent in the response header. In reality, everything was working correctly, but the API server didn't have it enabled.
So a day spent, but many lessons learned. Hopefully my wasted time helps someone else with their CORs problems. Note that my issue was stack agnostic, so regardless of how you are making your request, checking the response header is the first course of action to take if you run into a problem with CORs. After that, I would suggest looking into the request itself for errors.
Check out that book above or this link from the same author for more help, especially when it comes to non-simple requests HTML5 Rocks | Using CORs.

Self-Coded Proxy cannot retrieve image from wikipedia

I'm trying to write a small proxy server in c#. It is working nicely for many webpages I tested (including google.com and microsoft.com). For testing I started my proxy server and configured IE 10 on Windows 8 to use it.
But when I try wikipedia.org it does only load the main page but no pictures. I tried to load a single picture (http://upload.wikimedia.org/wikipedia/commons/6/63/Wikipedia-logo.png). When I use IE without proxy it works, but with the proxy I get a 404 response.
This is the GET Request which IE (my proxy just forwards it) issues:
GET http://upload.wikimedia.org/wikipedia/commons/6/63/Wikipedia-logo.png HTTP/1.1
Accept: text/html, application/xhtml+xml, */*\
Accept-Language: de-CH\
User-Agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)
Accept-Encoding: gzip, deflate
Host: upload.wikimedia.org
DNT: 1
Proxy-Connection: Keep-Alive
IMHO it looks correct. This is the response I get (omited some html tags):
HTTP/1.1 404 Not Found
Content-Type: text/html; charset=UTF-8
X-Varnish: 1427845074 1427806476, 274786836, 3671934588
Via: 1.1 varnish, 1.1 varnish, 1.1 varnish
Content-Length: 262
Accept-Ranges: bytes
Date: Mon, 01 Jul 2013 21:30:54 GMT
Age: 28
Connection: keep-alive
X-Cache: cp1063 hit (1), cp3004 miss (0), cp3003 frontend miss (0)
Access-Control-Allow-Origin: *
...404 Not Found\n The resource could not be found.\nRegexp failed to match URI: "http:/upload.wikimedia.org/wikipedia/commons/6/63/Wikipedia-logo.png"
The strange part is here:
Regexp failed to match URI: "http:/upload.wikimedia.org/wikipedia/commons/6/63/Wikipedia-logo.png"
-> the URL starts with a http:/
In the code I connect to uploads.wikimedia.org like this:
// connect to uploads.wikimedia.org
ServerSocket.Connect(RemoteHost, 80);
byte[] SendBuffer = Request.ToArray();
// send the clients request to the server
ServerSocket.Send(SendBuffer);
I have no idea why it doesn't work. Any help is appreciated. My full code is located on Github: Proxy_C_Sharp
I just found out why.
According to the HTTP/1.1 specification (http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5) in Chapter 5.2.1:
"To allow for transition to absoluteURIs in all requests in future versions of HTTP, all HTTP/1.1 servers MUST accept the absoluteURI form in requests, even though HTTP/1.1 clients will only generate them in requests to proxies."
I tried it out with a small tool. if I make a request like this:
GET /wikipedia/commons/6/63/Wikipedia-logo.png HTTP/1.1
Host: upload.wikimedia.org
It works. So the reason is that Wikipedia is not conform to the standard. It should accept absolute urls. But it works if I visit the site without a proxy because the browser uses absolute URIs only with proxies. If there is no proxy configured it uses a relative one.

Uploading to another domain gives HTTP code 405

I'm trying to upload a file (which can be quite large) from the website of one server to the backend of another server using plupload. Lets say:
domain 1 = http://www.websitedomain.com/uploadform
domain 2 = http://www.backenddomain.com/uploadhandler
Trying to upload i send the following:
OPTIONS /main/uploadnetwork.php HTTP/1.1
Host: backenddomain.com
Connection: keep-alive
Access-Control-Request-Method: POST
Origin: http://www.websitedomain.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.79 Safari/537.4
Access-Control-Request-Headers: origin, content-type
Accept: */*
Referer: http://www.websitedomain.com/uploadform
Accept-Encoding: gzip,deflate,sdch
Accept-Language: nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
DNT: 1
But when I try to start the upload the server returns the following:
HTTP/1.1 405 Method Not Allowed
Allow: GET, HEAD, OPTIONS, TRACE
Content-Type: text/html
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
X-Powered-By-Plesk: PleskWin
Date: Mon, 01 Oct 2012 12:41:57 GMT
Content-Length: 999
After doing some research I found out that a browser does this to check if the server will accept the intended message. It looks like my server doesn't feel like accepting a simple POST call even tho i use post all the time.
The Google Chrome console gives the following error:
XMLHttpRequest cannot load http://www.backenddomain.com/uploadhandler. Origin http://www.websitedomain.com is not allowed by Access-Control-Allow-Origin.
Does anyone know how to stop the browser from checking or how i can tell my server to just accept the POST?
You seem to face a Same origin policy problem
Adding a special header should help on some browsers :
http://en.wikipedia.org/wiki/Cross-origin_resource_sharing
Answers to this question might also be helpfull :
Cross-domain data access in JavaScript
You should also check the cross-domain tag : https://stackoverflow.com/questions/tagged/cross-domain

CORS request isn't allowed despite headers being set

I get the following error trying to do an XHR request. I've setup CORS response headers, but apparently something is wrong. Can anyone spot the error?
XMLHttpRequest cannot load http://domain.dev/path. Origin http://mobile.dev is not allowed by Access-Control-Allow-Origin.
Preflight Request Headers (OPTIONS)
This is a pre-flight request, to verify the request.
Accept:*/*
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Access-Control-Request-Headers:origin, x-pre-process, x-requested-with, x-client, x-client-version, accept, x-session
Access-Control-Request-Method:GET
Cache-Control:max-age=0
Connection:keep-alive
Host:my-app.dev
Origin:http://mobile.dev
Pragma:no-cache
Referer:http://mobile.dev/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.2 (KHTML, like Gecko) Chrome/22.0.1211.0 Safari/537.2
Response Headers
This is the server response to the pre-flight request.
Access-Control-Allow-Headers:origin, x-pre-process, x-requested-with, x-client, x-client-version, accept, x-session
Access-Control-Allow-Methods:GET, POST, PUT, PATCH, DELETE
Access-Control-Allow-Origin:http://mobile.dev
Access-Control-Max-Age:3600
Cache-Control:max-age=0, private, must-revalidate
Connection:close
Content-Type:text/html; charset=utf-8
ETag:"7215ee9c7d9dc229d2921a40e899ec5f"
X-Request-Id:3fca5f24077bcbd1351d552edf311f82
X-Runtime:0.014551
Real Request Header
This is the real request, which is canceled by the browser due to the error quoted above.
Accept:application/json, text/javascript, */*; q=0.01
Cache-Control:no-cache
Origin:http://mobile.dev
Pragma:no-cache
Referer:http://mobile.dev/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.2 (KHTML, like Gecko) Chrome/22.0.1211.0 Safari/537.2
X-Client:mobile
X-Client-Version:1.6
X-Pre-Process:underscore
X-Requested-With:XMLHttpRequest
X-Session:j9y01yw33txmdbcz1ao258uy7bzjlm
It seems that proper handling of the pre-flight OPTIONS request is necessary, but NOT SUFFICIENT for cross-site resource requests to work.
After the OPTIONS request comes back with satisfactory headers, all responses to any subsequent requests to the same URL also have to have the necessary "Access-Control-Allow-Origin" header, otherwise the browser will swallow them, and they won't even show up in the debugger window.
So it will look like the browser cancelled the request because of some problem in the OPTIONS response, but actually, the browser is looking at the response headers from the real request and then rejecting them.
(Answer copied from my own similar question Access-Control-Allow-Origin header not working - What am I doing wrong? in case it's the same thing)

Resources