I would like to implement Event Source feed of _changes for some documents in publicly readable database in CouchDB. Unfortunately, I get 405 error and:
Access to fetch at 'https://COUCHDOMAIN:6984/DBNAME/_changes?feed=eventsource&filter=_doc_ids&doc_ids=[%22DOCID%22]&since=now&include_docs=true&heartbeat=44000' from origin 'https://CLIENTDOMAIN' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status.
I do not understand why I get Allow: GET,POST,HEAD in response while in CORS config, there is also OPTIONS... EventSource constructed with withCredentials: true/false, no difference...
I will appreciate any suggestion.
My CORS configuration:
[cors]
origins = https://CLIENTDOMAIN
headers = accept, authorization, content-type, origin, referer
methods = GET, PUT, POST, HEAD, DELETE, OPTIONS
credentials = true
Request Headers:
OPTIONS /DBNAME/_changes?feed=eventsource&filter=_doc_ids&doc_ids=[%22DOCID%22]&since=now&include_docs=true&heartbeat=44000 HTTP/1.1
Host: COUCHDOMAIN:6984
Connection: keep-alive
Accept: */*
Access-Control-Request-Method: GET
Access-Control-Request-Headers: cache-control
Origin: https://CLIENTDOMAIN
User-Agent: Mozilla/5.0 (Linux; Android 10; LM-V405) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.87 Mobile Safari/537.36
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
Sec-Fetch-Dest: empty
Referer: https://CLIENTDOMAIN/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9,pl-PL;q=0.8,pl;q=0.7,en-US;q=0.6
Response Headers:
HTTP/1.1 405 Method Not Allowed
Access-Control-Allow-Origin: https://CLIENTDOMAIN
Access-Control-Expose-Headers: content-type, cache-control, accept-ranges, etag, server, x-couch-request-id, x-couch-update-newrev, x-couchdb-body-time
Allow: GET,POST,HEAD
Cache-Control: must-revalidate
Content-Length: 69
Content-Type: application/json
Date: Fri, 11 Feb 2022 20:04:06 GMT
Server: CouchDB/3.2.1 (Erlang OTP/20)
X-Couch-Request-ID: ab68310597
X-CouchDB-Body-Time: 0
General (from chrome dev tools):
Request URL: https://COUCHDOMAIN:6984/DBNAME/_changes?feed=eventsource&filter=_doc_ids&doc_ids=[%22DOCID%22]&since=now&include_docs=true&heartbeat=44000
Request Method: OPTIONS
Status Code: 405 Method Not Allowed
Remote Address: COUCHIP:6984
Referrer Policy: strict-origin-when-cross-origin
Solution (inspired by: https://stackoverflow.com/a/50914749/2626200):
[cors]
origins = https://CLIENTDOMAIN
headers = accept, authorization, content-type, origin, referer, cache-control
...
Related
I am trying to make a single API call using different approaches in Python3 but they are all incredibly slow if I compare with the same request in Postman and/or cURL.
This is what I am trying:
headers = {
"Authorization": "Bearer " + self.access_token,
"Content-Type": "application/json",
"Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive",
"Accept": "*/*",
"User-Agent": "PostmanRuntime/7.28.2",
"Cache-Control": "no-cache"
}
session = requests.Session()
api_res = session.get(self.api_endpoint, headers=headers, timeout=self.req_timeout,)
When running this call, it gets stuck for a few minutes until I receive a response. If I use Postman for example, I get the result in 1 second or less.
I also tried using http.client, urllib3 but I still see a huge delay in the call.
I also tried debugging the call:
import http.client
http.client.HTTPConnection.debuglevel = 1
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True
Debug response:
[DEBUG] Starting new HTTPS connection (1): ...:443
header: Content-Type: application/json; charset=utf-8
header: Content-Length: 61
header: Connection: keep-alive
header: Date: ...
header: Cache-Control: no-cache
header: Pragma: no-cache
header: Expires: -1
header: WWW-Authenticate: Bearer
header: Access-Control-Allow-Origin: *
header: P3P: CP="IDC DSP COR ADM DEVi TAIi PSA PSD IVAi IVDi CONi HIS OUR IND CNT"
header: Access-Control-Allow-Methods: GET, POST, OPTIONS, HEAD
header: X-XSS-Protection: 1; mode=block
header: Access-Control-Allow-Headers: accept, authorization, content-type, Cache-Control, P3P, GE-IGNORE-CACHE, Signature, fromMobile, ssoToken, fromAdmin, fromGameapp, fromGtv, fromGameapp, ManagerOrgUnitUserName, ManagerOrgUnitId, g-s-x, g-s-x-t, g-s-i-i, User-Agent, Referer, Origin, Access-Control-Allow-Headers
header: X-Content-Type-Options: nosniff
header: X-XSS-Protection: 1; mode=block
header: Referrer-Policy: same-origin
header: Strict-Transport-Security: max-age=31536000; includeSubDomains
header: X-Cache: Error from cloudfront
header: Via: 1.1 ....cloudfront.net (CloudFront)
header: X-Amz-Cf-Pop: ORD52-C1
header: X-Amz-Cf-Id: ...
Any ideas about why is it so slow? What it doesn't happen when I replicate to Postman for example?
Even from Google is taking a lot of time:
requests.get("https://www.google.com")
I also realized that it's working with ipv4 and NOT with ipv6.
Thanks!
I found out that IPv6 was not working but IPv4 was. I had to force calls to IPv4 like this:
import socket
import requests.packages.urllib3.util.connection as urllib3_cn
urllib3_cn.allowed_gai_family = lambda: socket.AF_INET
Im trying to POST request using Python to retreive a specific File. Since the URL is behind a server with authorized access theres no use posting it here
However the form data contains a field called base64 and lengthy which I cant figure out if its a form data value or base64 encoding of post request
Here are browser parameters
General:
Request URL: http://exampleapi.com/api/Document/Export
Request Method: POST
Status Code: 200 OK
Remote Address: XX.XXX.XXX.XX:XX
Referrer Policy: no-referrer-when-downgrade
Response Headers:
Access-Control-Allow-Origin: http://example.com
Cache-Control: no-cache
Content-Disposition: attachment; filename=location-downloads.xlsx
Content-Length: 7148
Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Date: Tue, 23 Jul 2019 21:00:18 GMT
Expires: -1
Pragma: no-cache
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Request Headers :
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 10162
Content-Type: application/x-www-form-urlencoded
Cookie: abcConnection=!UA7tkC3iZCmVNGRUyRpDWARVBWk/lY6SZvgxLlaygsQKk+vuwA1NxvhwE9ph4i+3NZlKeepIfuHhUvyQjl68fhhrT9ueqMx/3mBKUDcT
DNT: 1
Host: exampleapi.com
Origin: http://example.com
Referer: http://example.com/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36
Form Data:
fileName: location-downloads.xlsx
contentType: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
base64: UEsDBAoAAAAAAAh4904AAAAAAAAAAAAAAAAJAAAAZG9jUHJvcHMvUEsDBAoAAAAIAAh490(shortened for simplicity)
Here is what I tried
url='http://example.com'
urllib3.disable_warnings()
headers = {
"Content-Type": "application/x-www-form-urlencoded",
"User-Agent": "Mozilla/5.0",
}
with requests.session() as s:
r=s.get(url,headers={"User-Agent":"Mozilla/5.0"},verify=False)
data=r.content
soup=BeautifulSoup(data,'html.parser')
form_data = {
"fileName":"location-downloads.xlsx",
"contentType":"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
}
r2=s.post('http://exampleapi.com/api/Document/Export',data=json.dumps(form_data,ensure_ascii=True).encode('utf-8'),headers=headers,verify=False)
print(r2.status_code)
Any idea how i should proceed. My status code also shows 500 for the post here
Basically after an auth, I setting a cookie, but apparently after page refresh on the cookie that was set by cloudflare is saved
And the cookie that I transmitted with set-cookie is not used in after set-cookie requests
# Response headers
HTTP/2.0 200 OK
date: Thu, 18 Jul 2019 10:03:25 GMT
content-type: application/json; charset=utf-8
content-length: 29
set-cookie: __cfduid=d578c7a5e4378dc1b1946964a08ebc4ec1563444205; expires=Fri, 17-Jul-20 10:03:25 GMT; path=/; domain=.doc.io; HttpOnly; Secure
set-cookie: __doc=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlIjoiNzQ2NTczNzQ0MDY2NjE3Mzc0NmQ2MTY5NmMyZTZkNzgiLCJwIjoiNTUzMjQ2NzM2NDQ3NTY2YjU4MzEzODMzNTU0NjUwNTU2ZjRiMzkzMTY3NDUzNDY5NDc3MzM3MzgzOTU5MzczMDUxNjk0ZjQxNjQ0OTM5Nzg0YjZiNzU1Njc3Nzk0NDc0NjE3NDMxNTE0NzcwMzE0YjQxNmY1MjU5MzM3YTZhNDU2NDJiNmU0ZTc0NGE3NTMyNTQ1ODc2NjI1YTczNDc1MTQ1Njc0MjVhNGQ0MTNkM2QiLCJkIjoiMzEzNTM2MzMzNDM0MzQzMjMwMzUzNTM2MzMiLCJpYXQiOjE1NjM0NDQyMDV9.go1jDpc2rBe5FjK2sKX4ybW4PhCPFq1xT1WIX-mSI84; Domain=.doc.io; Path=/; Expires=Thu, 18 Jul 2019 16:03:25 GMT; HttpOnly; Secure
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 4f83a02a3dc36455-FRA
X-Firefox-Spdy: h2
reply
.code(200)
.header('Access-Control-Allow-Origin', '*')
.header('Content-Type', 'application/json; charset=utf-8')
.setCookie('__doc', token, {
domain: '.doc.io',
path: '/',
secure: true,
httpOnly: true,
expires: new Date(new Date().setHours(new Date().getHours() + 6))})
.send({ 'success': 'Sign In success' })
All my websites are https
First I do POST request for an auth on /auth, and you could see response in response headers above and after I do GET on (trying to load page) from /page and get cookies, but with reply.log.info(request.cookies) I see only cookies from cloudflare. Surely I tried to refresh and go to address in different table, there just no any cookies, but from cloudflare.
# Request headers
Host: test.doc.io
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
DNT: 1
Connection: keep-alive
Cookie: __cfduid=d578c7a5e4378dc1b1946964a08ebc4ec1563444205
Upgrade-Insecure-Requests: 1
TE: Trailers
After I tested it with an actual page with js code for a XHR request it works fine. However cookie was only displayed, but not actually saved in the storage when I was sending request directly in Firefox's inspector. Lost a day to figure out that created in inspector cookies seems preventing from installing
Have a simple get Customer api that's returning list of customers fine.
Setting up for service to service authentication, if I make this [Authenticated] and try to implement using ApiKeyAuthProvider, the req.GetApiKey returns null and I get an error;
Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request starting HTTP/1.1 POST https://localhost:44347/api/customers application/json 0
Microsoft.AspNetCore.Hosting.Internal.WebHost:2019-07-01 16:50:34,004 [16] INFO Microsoft.AspNetCore.Hosting.Internal.WebHost - Request starting HTTP/1.1 POST https://localhost:44347/api/customers application/json 0
The thread 0x42cc has exited with code 0 (0x0).
The thread 0x302c has exited with code 0 (0x0).
ServiceStack.ServiceStackHost:2019-07-01 17:01:14,601 [16] ERROR ServiceStack.ServiceStackHost - ServiceBase<TRequest>::Service Exception
System.ArgumentOutOfRangeException: Length cannot be less than zero.
Parameter name: length
at System.String.Substring(Int32 startIndex, Int32 length)
at ServiceStack.Host.HttpRequestAuthentication.GetBasicAuth(IRequest httpReq) in C:\BuildAgent\work\3481147c480f4a2f\src\ServiceStack\Host\HttpRequestAuthentication.cs:line 45
at ServiceStack.Host.HttpRequestAuthentication.GetBasicAuthUserAndPassword(IRequest httpReq) in C:\BuildAgent\work\3481147c480f4a2f\src\ServiceStack\Host\HttpRequestAuthentication.cs:line 50
at ServiceStack.Auth.ApiKeyAuthProvider.PreAuthenticate(IRequest req, IResponse res) in C:\BuildAgent\work\3481147c480f4a2f\src\ServiceStack\Auth\ApiKeyAuthProvider.cs:line 232
at ServiceStack.AuthenticateAttribute.PreAuthenticate(IRequest req, IEnumerable`1 authProviders) in C:\BuildAgent\work\3481147c480f4a2f\src\ServiceStack\AuthenticateAttribute.cs:line 96
at ServiceStack.AuthenticateAttribute.ExecuteAsync(IRequest req, IResponse res, Object requestDto) in C:\BuildAgent\work\3481147c480f4a2f\src\ServiceStack\AuthenticateAttribute.cs:line 74
at ServiceStack.Host.ServiceRunner`1.ExecuteAsync(IRequest req, Object instance, TRequest requestDto) in C:\BuildAgent\work\3481147c480f4a2f\src\ServiceStack\Host\ServiceRunner.cs:line 127
Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request finished in 640574.8754ms 400 application/json; charset=utf-8
Microsoft.AspNetCore.Hosting.Internal.WebHost:2019-07-01 17:01:14,607 [16] INFO Microsoft.AspNetCore.Hosting.Internal.WebHost - Request finished in 640574.8754ms 400 application/json; charset=utf-8
Clearly I have missed something obvious...any pointers appreciated.
// Register ORMLite connection
container.Register<IDbConnectionFactory>(dbFactory);
//Tell ServiceStack you want to persist User Auth Info in SQL Server
container.Register<IAuthRepository>(c => new OrmLiteAuthRepository(dbFactory));
// See https://docs.servicestack.net/api-key-authprovider
Plugins.Add(new AuthFeature(() => new AuthUserSession(),
new IAuthProvider[] {
new ApiKeyAuthProvider(AppSettings) {
SessionCacheDuration = TimeSpan.FromMinutes(10),
AllowInHttpParams = true, // Whether to allow API Keys in 'apikey' QueryString or FormData (e.g. `?apikey={APIKEY}`)
RequireSecureConnection = true,
},
}
) {
IncludeRegistrationService = true,
});
GlobalRequestFilters.Add((req, res, dto) =>
{
LastApiKey = req.GetApiKey();
});
Request
POST https://localhost:44347/api/customers HTTP/1.1
Host: localhost:44347
Connection: keep-alive
Content-Length: 2
Accept: application/json
Origin: https://localhost:44347
Authorization: yDOr26HsxyhpuRB3qbG07qfCmDhqutnA-yDOr26HsxyhpuRB3qbG07qfCmDhqutnA-yDOr26HsxyhpuRB3qbG07qfCmDhqutnA
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36
Content-Type: application/json
Referer: https://localhost:44347/swagger-ui/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9,en-US;q=0.8
{}
Response
HTTP/1.1 400 ArgumentOutOfRangeException
Transfer-Encoding: chunked
Content-Type: application/json; charset=utf-8
Vary: Accept,Origin
Server: Microsoft-IIS/10.0
X-Powered-By: ServiceStack/5.50 NetCore/Windows
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, PATCH, OPTIONS
Access-Control-Allow-Headers: Content-Type
X-Startup-Errors: 1
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: Content-Disposition
X-SourceFiles: =?UTF-8?B?QzpcUmVwb3NcTUJXZWJccnZhcGlcUnZXZWJcUnZBcGlcYXBpXGN1c3RvbWVycw==?=
X-Powered-By: ASP.NET
Date: Wed, 03 Jul 2019 08:07:40 GMT
13e
{"responseStatus":{"errorCode":"ArgumentOutOfRangeException","message":"Length cannot be less than zero.\r\nParameter name: length","errors":[{"errorCode":"ArgumentOutOfRangeException","fieldName":"length","message":"Length cannot be less than zero.\r\n"}]},"responseCreatedUtcDateTime":"2019-07-03T08:07:40.7955827Z"}
0
Your client is sending an invalid Authorization Bearer Token, it needs to have the Authroization Bearer Token format:
Authorization: Bearer {Token}
If you're sending an Authenticated API Key or JWT Request via Open API it needs to have the Bearer prefix as per the Open API docs:
OK I had manually created a User and APIKey in the underlying tables and had used a UserAuthId 'SomeAuthId' i.e. letter in them, and the ORM repository code is expecting these to be integers. Its cool that I can see the code in github and debug this myself - thanks for the comment as it got me thinking and looking into my Auth setup.
When trying to redirect the user to the login page, I always get this error:
Response to preflight request doesn't pass access control check: The
value of the 'Access-Control-Allow-Origin' header in the response must
not be the wildcard '*' when the request's credentials mode is
'include'. Origin 'null' is therefore not allowed access. The
credentials mode of requests initiated by the XMLHttpRequest is
controlled by the withCredentials attribute.
On my express application, I already implemented CORS:
var corsOptions = {
origin: 'http://localhost:4200',
credentials: true
}
app.use(cors(corsOptions));
Then I try to redirect the user:
router.use(function(req,res,next){
if((req.session.user == null){
res.redirect('http://localhost:4200' + '/login')
}
else
next();
});
On Angular 4 I'm sending all requests with {withCredentials : true}, because I'm using cookies;
These are the request/response headers:
Response:
Access-Control-Allow-Credentials:true
Access-Control-Allow-Origin:http://localhost:4200
Connection:keep-alive Date:Thu, 10 Aug 2017 12:09:55 GMT
Location:http://localhost:4200/login
set-cookie:semanaintegrada.session=s%3A6nieAv1pfn2V-2x7H4HbnqJFbYsgJmwy.hO1QKm%2Fgm6Kmso8pLCQ5zrZAVNhIYgfr%2BgzOB0oI9UA;
Path=/; Expires=Thu, 10 Aug 2017 13:09:55 GMT
Transfer-Encoding:chunked Vary:Origin X-Powered-By:Express
Request:
Accept:application/json, text/plain, / Accept-Encoding:gzip, deflate,
br Accept-Language:pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4
Cache-Control:no-cache Connection:keep-alive Content-Length:39
content-type:application/json
Cookie:semanaintegrada.session=s%3AwuiVwYs3Ahs4dfLULfpqMBcrnbthY7sZ.OYZk%2FCnZGHAe8v1T8nWpbAdFQVsXjUFAQxnYI27%2FZlE
Host:localhost:3000 Origin:http://localhost:4200 Pragma:no-cache
Referer:http://localhost:4200/ User-Agent:Mozilla/5.0 (Windows NT
10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36
Also, in Chrome there are 4 requests that are created ( I dont know why ).
3 to my express app
and 1 to localhost:4200
The last one have Access-Control-Allow-Origin:* and Origin null
Could this be related?
"credentials flag" refers to XMLHttpRequest.withCredentials of the request being made, not to an Access-Control-Allow-Credentials header.
If the request's withCredentials is true, Access-Control-Allow-Origin: * can't be used, even if there is no Access-Control-Allow-Credentials header.