I am building a project and wish to implement CORS. It is installed on a shared IIS server (shared hosting).
In my apphost.cs I enable and configure CORS according to several of the articles I find on the web.
Plugins.Add(new CorsFeature(
allowedMethods: "GET, POST, PUT, DELETE, OPTIONS",
allowedOrigins: "*",
allowCredentials: true,
allowedHeaders: "content-type, Authorization, Accept"));
I also read that when implementing the newer SS API (IService) I had to provide a hook for the 'Options' call so I added...
this.RequestFilters.Add((httpReq, httpRes, requestDto) =>
{
//Handles Request and closes Responses after emitting global HTTP Headers
if (httpReq.HttpMethod == "OPTIONS")
httpRes.EndRequest(); //add a 'using ServiceStack;'
});
I am able to call my api via REST Console (chrome addin) but when I call with a ajax request I get a 405 error.
Request URL:http://empire.thecodingpit.com/api/engine
Request Method:OPTIONS
Status Code:200 OK
Request Headers view source
Accept:*/*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Access-Control-Request-Headers:accept, origin, content-type
Access-Control-Request-Method:POST
Connection:keep-alive
Host:empire.thecodingpit.com
Origin:http://json.commublogs.com
Referer:http://json.commublogs.com/
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.66 Safari/537.36
Response Headers view source
Cache-Control:private
Content-Length:0
Date:Thu, 26 Sep 2013 17:00:59 GMT
Server:Microsoft-IIS/8.0
X-AspNet-Version:4.0.30319
X-MiniProfiler-Ids: ["f8c3ddb8434149feb689dd44d93bf862","6d0e7e0f8ac1456d98872a82af4d6602","67bdf08a19c641a7b26db0b43fd10888","1819b1ce3e314c0594ef9ecb1ac68fcf","7af8595373e345f3a9f8ade60c8a7817"]
X-Powered-By:ASP.NET
I do not have any [Authentication] on my methods - yet.
I have looked at this question and have implemented it as shown above. This related question is NOT a duplicate. servicestack REST API and CORS
The service call (POST) that I am making is...
public object Post(SelectedTroops request)
{
return new SelectedTroopsResponse { ArrayOfSelectedTroops = request.ArrayOfSelectedTroops };
}
If I am missing anything that may be helpful please let me know. I guess an side note question may be - how do you effectively trace thru and debug something like this? Any helpful hints would be more than appreciated.
You do have to use:
httpRes.EndRequest();
It's important, it's an extension method in the ServiceStack namespace. I recommend using Tools like ReSharper which makes auto-finding and referencing extension methods trivial.
Related
I am using axios and a API (cowin api https://apisetu.gov.in/public/marketplace/api/cowin/cowin-public-v2) which has strong kind of protection against the web requests.
When I was getting error 403 on my dev machine (Windows) then, I solve it by just adding a header 'User-Agent'.
When I have deployed it to heroku I am still getting the same error.
const { data } = await axios.get(url, {
headers: {
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36',
},
})
Using a fake user-agent in your headers can help with this problem, but there are other variables you may want to consider.
For example, if you are making multiple HTTP requests you may want to have multiple fake user-agents to and then randomize the user-agent for every request made. This can help limit the changes of your scraper being detected.
If that still doesn't work you may want to consider optimizing your headers further. Other than sending HTTP requests with a randomized user-agent, you can further imitate a browser's request Headers by adding more Headers than just the "user-agent"- then ensuring that the user-agent that is selected is consistent with the information sent from the rest of the headers.
You can check out here for more information.
On the site it will not only provide information on how to optimize your headers consistently with the user-agent, but also provide more solutions in case the above mentioned still was unsuccessful.
In my situation, it was the case that I had to bypass cloudflare. You can determine if this is your situation as well if you log your error to the terminal and then check if under the "server" key it says "cloudflare". In which case you can use this documentation for further assistance.
OK, I might be missing something simple here in Angular, but I could really use some help. I am trying to grant a Service Principal READER role to a Subscription programmatically. If I use PostMan, it works fine. However, when I send the same PUT request via Angular6 I get a 400 error from Azure that says:
The content of your request was not valid, and the original object
could not be deserialized. Exception message: 'Required property
'permissions' not found in JSON. Path 'properties', line 1, position
231.'
The JSON being sent in both cases is:
{
"properties":
{
"roleDefinitionId":"/subscriptions/{some_subscription_guid}/providers/Microsoft.Authorization/roleDefinitions/acdd72a7-3385-48ef-bd42-f606fba81ae7",
"principalId":"{some_service_provider_guid}"
}
}
I've captured traffic from both requests, and they show as application/json payloads on the PUT. So I am at a loss of what is deserializing incorrectly through Azure that is causing this error. I am trying to follow the REST instructions documented here: https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-rest
Any ideas what I am missing?
UPDATE
Adding the RAW REQUEST per request. I have replaced any sensitive data (access token, GUIDs etc) without changing anything else from the Fiddler output.
PUT https://management.azure.com/subscriptions/<VALID_SUBSCRIPTION_WAS_HERE>/providers/Microsoft.Authorization/roleDefinitions/7ec2aca1-e4f2-4152-aee2-68991e8b48ad?api-version=2015-07-01 HTTP/1.1
Host: management.azure.com
Connection: keep-alive
Content-Length: 233
Accept: application/json, text/plain, */*
Origin: http://localhost:4200
Authorization: Bearer <VALID_TOKEN_WAS_HERE>
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Content-Type: application/json
Referer: http://localhost:4200/token/<VALID_DOMAIN_WAS_HERE>.onmicrosoft.com/graph
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
{"properties": { "roleDefinitionId":"/subscriptions/<VALID_SUBSCRIPTION_GUID_HERE>/providers/Microsoft.Authorization/roleDefinitions/acdd72a7-3385-48ef-bd42-f606fba81ae7", "principalId":"<VALID_OBJECTID_HERE>" }}
Alright, I finally figured out what was going on here. It appears that I was posting to the wrong endpoint. I need to be posting to roleAssignment and not roleDefinitions.
So why did it work in PostMan? It seems there is a fallback from a previous version of the API that supported both when using legacy clients, which for some reason PostMan fell under. However, when posting via Angular it was actively rejecting it.
End result... send to "/Microsoft.Authorization/roleassignments/" with an API version later than "api-version=2015-07-01". All will work.
I have a working node/express backend running on localhost. I'm building a project application that needs to fetch data from goodreads api. When I execute the request, I get:
Cross-Origin Request Blocked:
The Same Origin Policy disallows reading the remote resource at
https://www.goodreads.com/book/title.json?author=Arthur+Conan+Doyle&key=[my_key]&title=Hound+of+the+Baskervilles.
(Reason: CORS header 'Access-Control-Allow-Origin' missing).1 <unknown>
Server side, everything is working correctly. I have enabled CORS, and when I check the headers, 'Access-Control-Allow-Origin' is available on everything coming from my server after checking the header in Firefox and Chrome dev tools. When I make a request via $resource, however, 'Allow-Access...' is not present in my header. Here is the code for the resource:
.factory('goodReads', function($resource) {
return $resource('https://www.goodreads.com/book/title.json');
})
.controller('AddBookSelectorController', function($resource, goodReads) {
this.fetch = function() {
var key = '[my_key]';
var data = goodReads.query({author: 'Arthur Conan Doyle', key: key, title: 'Hound of the Baskervilles'});
console.log(data);
};
});
I'm calling fetch via ng-click, and everything executes fine except I get the CORS error. Can anyone point me in the right direction? I am new to angular, and my suspicion is there is a problem with my resource request or something in configuration, but I can't seem to find an answer to fix my problem in the documentation or other stackoverflow questions.
Update 3: It is not a localhost issue. I tried pushing it to my domain and using a simple button which ran an xhr request to the OpenBooks api, and the problem got worse. It is hosted via Openshift, and now the 'Allow-Control-Access-x' headers are gone even for other files on my server. Really beginning to bang my head against the wall here. I am removing the Angular tags, because it has nothing to do with Angular.
UPDATE 2: I got it working after installing 'Allow-Control-Allow-Origin' extension in Chrome. Has my problem been the fact that I'm running this on localhost? Or is there something else going on? The header is still not being set without the extension.
UPDATE: I've been working on this since 8am, and still no luck. I have tried rewriting the request using Angular's $http and also with Javascript's xhr following the example from HTML5 Rocks | Using Cors and I'm still having the same problem with each method. Like I said, the necessary header information is available from files on my server, but it breaks when I make requests to other sites.
I'm starting to think this might not be an Angular problem, but I really have no clue. Just to be safe, here is the code I added to Express to enable CORS, including app.use so you can get an idea for where I called it:
app.use(logger('dev'));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(cookieParser());
app.use(function(req, res, next) {
res.header("Access-Control-Allow-Origin", "*");
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept, Authorization, Content-Length");
res.header("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS");
res.header("Access-Control-Allow-Credentials", "true");
next();
});
app.use(express.static(path.join(__dirname, 'public')));
app.use('/', routes);
Edit: Here are the headers from the API request:
Request Headers
Host: www.goodreads.com
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0
Accept: application/json, text/plain, */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Referer: http://localhost:3000/
Origin: http://localhost:3000
Connection: keep-alive
Response Headers
Cache-Control: max-age=0, private, must-revalidate
Content-Encoding: gzip
Content-Length: 686
Content-Type: application/json; charset=utf-8
Date: Wed, 02 Sep 2015 17:20:35 GMT
Etag: "a2be782f32638d2a435bbeaf4b01274a-gzip"
Server: Server
Set-Cookie: csid=BAhJIhg1MzgtNTk4NjMzNy0wNzQ4MTM5BjoGRVQ%3D--afed14b563e5a6eb7b3fa9005de3010474230702; path=/; expires=Sun, 02 Sep 2035 17:20:33 -0000
locale=en; path=/
_session_id2=fd45336b8ef86010d46c7d73adb5f004; path=/; expires=Wed, 02 Sep 2015 23:20:35 -0000; HttpOnly
Status: 200 OK
Vary: Accept-Encoding,User-Agent
X-Content-Type-Options: nosniff, nosniff
X-Frame-Options: ALLOWALL
X-Request-Id: 1K8EJWG30GWDE4MZ4R5K
X-Runtime: 2.277972
X-XSS-Protection: 1; mode=block
Headers for the .js file from my server:
Request
Host: localhost:3000
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Referer: http://localhost:3000/
Cookie: _ga=GA1.1.1924088292.1439681064; connect.sid=s%3AB4O0Up9WF5iqkfky__I0XCiBD2aMATlq.gbJUC9GseqnJvRTEIbcwxD6cwFQeL7ljNScURCJ5As0
Connection: keep-alive
If-Modified-Since: Wed, 02 Sep 2015 17:08:40 GMT
If-None-Match: W/"886-14f8f0828c1"
Cache-Control: max-age=0
Response:
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Cache-Control: public, max-age=0
Connection: keep-alive
Date: Wed, 02 Sep 2015 17:20:30 GMT
Etag: W/"886-14f8f0828c1"
Last-Modified: Wed, 02 Sep 2015 17:08:40 GMT
X-Powered-By: Express
access-control-allow-headers: Origin, X-Requested-With, Content-Type, Accept
I guess this problem exposed my ignorance, but maybe this will help other newbies to CORs like me. I finally figured out the problem after getting a copy of CORs in Action and working through the first example using the Flickr API.
My problem had nothing to do with the backend, Angular, jQuery's .ajax method, or xhr. All of my requests were properly formatted. The problem was the APIs I attempted to use did not have CORs enabled on their server. O.o As soon as I changed the data type to jsonp, everything went through.
Anyway, for you newbs out there like me, here are some pointers to help you if you run into this problem:
1. Don't assume the API you are using has CORs enabled
I don't know why, but I blindly picked two APIs that don't have CORs enabled, which is what caused all the fuss for me. I have never run into this problem before because the work I have done with APIs have always been from big companies like Flickr that had CORs enabled. If they don't set Access-Control-Allow-Origin on their server, you can request them to enable it and use JSONP in the meantime.
If the API has an option for a callback at the end, that's a good sign you should use JSONP for your request. JSONP works by wrapping your request in a callback and exploiting a feature of the script tag. Scripts can pull other scripts from any domain, so it works as a hack to get the data. Here's a good link that helped me. Exactly What is JSONP? | CameronSpear.com
2. Check The Response Headers
I got tricked by this, but remember that the response header on your request to an external API is the response from their server, not yours. It doesn't matter if CORs is enabled on your server, you are making the request to someone else, and the browser automatically sends your information to them in the request header. Remember, all of this checking is done by the browser for security reasons, so its doing the heavy lifting for you on the request side based on your ajax call. If Access-Control-Whatever doesn't show up in the response header, they don't have CORs enabled. If you are working on the frontend and requesting someone else's data, you can't do anything about it. Use JSONP and your problems will disappear (probably).
This whole fiasco for me started because I was confusing responses coming from my server with responses coming for their server. I correctly enabled CORs on my own server, but I was thinking it wasn't attaching the origin information to the request header which is why it was absent in the response header. In reality, everything was working correctly, but the API server didn't have it enabled.
So a day spent, but many lessons learned. Hopefully my wasted time helps someone else with their CORs problems. Note that my issue was stack agnostic, so regardless of how you are making your request, checking the response header is the first course of action to take if you run into a problem with CORs. After that, I would suggest looking into the request itself for errors.
Check out that book above or this link from the same author for more help, especially when it comes to non-simple requests HTML5 Rocks | Using CORs.
I have enabled the CORS feature in ServiceStack, for all verbs, standard headers plus a few custom ones, and all origins. From my Angular application, I am getting the CORS "No 'Access-Control-Allow-Origin' header is present on the requested resource" error when trying to make a PUT call to the server. If I look at my traffic, I see the OPTIONS preflight request to the resource returning with a valid 200 message, and the ACAO header is present and set to *.
// CORS PREFLIGHT REQUEST
OPTIONS /referral HTTP/1.1
Host: api.mydomain.com
Connection: keep-alive
Access-Control-Request-Method: PUT
Origin: http://127.0.0.1:9000
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36
Access-Control-Request-Headers: accept, x-uatoken, x-ualocation, content-type
Accept: */*
Referer: http://127.0.0.1:9000/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
// CORS RESPONSE
HTTP/1.1 200 OK
Cache-Control: private
Vary: Accept
Server: Microsoft-IIS/8.0
X-Powered-By: ServiceStack/4.020 Win32NT/.NET
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Headers: Content-Type, X-UAToken, X-UAUser, X-UALocation, Authorization
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Thu, 05 Jun 2014 17:27:47 GMT
Content-Length: 0
Note: I am using angular-file-upload library to make this request as multipart/form-data (pulling files from Request.Files and deserializing data from Request.FormData on the server). Debugging the second request (the actual PUT) in Chrome has that message about "Provisional headers are shown", so I'm not sure how useful that data is:
Accept:application/json, text/plain, */*
Content-Type:multipart/form-data; boundary=----WebKitFormBoundary3rvpR6k8pz4rghGy
Origin:http://127.0.0.1:9000
Referer:http://127.0.0.1:9000/
User-Agent:Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36
X-UALocation:7
X-UAToken:yoqoByj-T1SBDHCYir92JQ
Request Payload
...etc...
Any ideas?
I see the preflight check is requesting a PUT verb? You might want to check into setting up IIS for PUT/DELETE verbs (it's a whole nother can of worms). Or just try the request as a POST to see if that works, then you know it might be just the verb in IIS.
I've dealt with this issue in the past, and I can't believe I didn't recognize it until now. It's common in any runtime environment you deal with for file uploads like PHP or Tomcat. The problem isn't your CORS setup at all (since the preflight check is passing). Unless its something entirely else my guess is you ran into the ASP.NET and IIS max request limit! By default these are both very low (4MiB). Very simple to solve, just need to add some stuff to your web.config. The IIS settings are only necessary if you on IIS7.5+ I believe. It may also be recommended to change the executionTimeout depending on how long you are going to allow the upload to run (which will depend on speed of client and size of file).
I finally recognized the issue for what it was when I finally ran the ServiceStack project in debug mode and told it to raise all .NET exceptions to me. Attempted to upload a large enough file (I was testing in curl with really small ones! doh!), and then the exception hit me plain and clear. System.Web.HttpException "Maximum request length exceeded."
Here's example of what you need to do
<system.web>
<!-- maxRequestLength size is specified in kiB (int32). this controls max length handled by ASP.NET
set to 2GiB currently
executionTimeout is seconds before request is shutdown.
set to 2 hours currently
-->
<httpRuntime targetFramework="4.5" maxRequestLength="2097152" executionTimeout="7200" />
</system.web>
<system.webServer>
<security>
<requestFiltering>
<!-- size is specified in bytes (uint32). this controls max length of IIS
set to 2GiB currently -->
<requestLimits maxAllowedContentLength="2147483648" />
</requestFiltering>
</security>
</system.webServer>
I get the following error trying to do an XHR request. I've setup CORS response headers, but apparently something is wrong. Can anyone spot the error?
XMLHttpRequest cannot load http://domain.dev/path. Origin http://mobile.dev is not allowed by Access-Control-Allow-Origin.
Preflight Request Headers (OPTIONS)
This is a pre-flight request, to verify the request.
Accept:*/*
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Access-Control-Request-Headers:origin, x-pre-process, x-requested-with, x-client, x-client-version, accept, x-session
Access-Control-Request-Method:GET
Cache-Control:max-age=0
Connection:keep-alive
Host:my-app.dev
Origin:http://mobile.dev
Pragma:no-cache
Referer:http://mobile.dev/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.2 (KHTML, like Gecko) Chrome/22.0.1211.0 Safari/537.2
Response Headers
This is the server response to the pre-flight request.
Access-Control-Allow-Headers:origin, x-pre-process, x-requested-with, x-client, x-client-version, accept, x-session
Access-Control-Allow-Methods:GET, POST, PUT, PATCH, DELETE
Access-Control-Allow-Origin:http://mobile.dev
Access-Control-Max-Age:3600
Cache-Control:max-age=0, private, must-revalidate
Connection:close
Content-Type:text/html; charset=utf-8
ETag:"7215ee9c7d9dc229d2921a40e899ec5f"
X-Request-Id:3fca5f24077bcbd1351d552edf311f82
X-Runtime:0.014551
Real Request Header
This is the real request, which is canceled by the browser due to the error quoted above.
Accept:application/json, text/javascript, */*; q=0.01
Cache-Control:no-cache
Origin:http://mobile.dev
Pragma:no-cache
Referer:http://mobile.dev/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.2 (KHTML, like Gecko) Chrome/22.0.1211.0 Safari/537.2
X-Client:mobile
X-Client-Version:1.6
X-Pre-Process:underscore
X-Requested-With:XMLHttpRequest
X-Session:j9y01yw33txmdbcz1ao258uy7bzjlm
It seems that proper handling of the pre-flight OPTIONS request is necessary, but NOT SUFFICIENT for cross-site resource requests to work.
After the OPTIONS request comes back with satisfactory headers, all responses to any subsequent requests to the same URL also have to have the necessary "Access-Control-Allow-Origin" header, otherwise the browser will swallow them, and they won't even show up in the debugger window.
So it will look like the browser cancelled the request because of some problem in the OPTIONS response, but actually, the browser is looking at the response headers from the real request and then rejecting them.
(Answer copied from my own similar question Access-Control-Allow-Origin header not working - What am I doing wrong? in case it's the same thing)