I'm trying to load the socketstream application with express and cors, which is throwing an error stating 404. All the project directories are available in browser but no contents.
var app = require('express')();
var ss = require('socketstream');
var cors = require('cors');
ss.client.define('main', {
view : 'app.html',
css : [],
code : [],
tmpl : '*'
});
app.use(cors());
app.get('/', function(req, res) {
res.serve('main');
});
ss.start(app.listen(port));
And this is the response in browser
Headers
Remote Address:"RemoteAddress":"Port"
Request URL:http://"RemoteAddress":"Port"/_serveDev/code/app/someFile.js?ts=1449057700740&pathPrefix=app
Request Method:GET
Status Code:404 Not Found
Request headers
Connection:keep-alive
Content-Length:76
Content-Type:text/html; charset=utf-8
Date:Wed, 02 Dec 2015 12:03:20 GMT
X-Content-Type-Options:nosniff
X-Powered-By:Express
Response headers
Accept:*/*
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-US,en;q=0.8
Cache-Control:max-age=0
Connection:keep-alive
Host:"RemoteAddress":"Port"
Referer:http://"RemoteAddress":"Port"/
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36
Update
The app.html is getting loaded in browser but 404 is only for scripts and css that is being requested from app.html.
Thanks.
Try to change the permissions of the files by running
sudo chmod 777 -R *
in the code directory.
Related
I've found a lot of CORS-related questions, but none of them solved my problem. As I'm quite new to web-dev, I'd appreciate any help with getting my project infrastructure up and running.
My application is made of two parts:
Server part
A server made with Haskell, Servant. The most important endpoint I've been trying to connect to is:
type Unprotected =
"login"
:> ReqBody '[JSON] Credentials
:> Verb 'POST 204 '[JSON] (Headers '[Header "Set-Cookie" SetCookie, Header "Set-Cookie" SetCookie] NoContent)
It's based on the project described here: https://github.com/haskell-servant/servant-auth
I've already found a SO question about CORS: CORS header ‘Access-Control-Allow-Origin’ missing in servant, but that didn't help.
During development I launch the server on port 8081.
UI Part
I have also a UI project, that's hosted independently on port 8833. It uses WebPack. I've also found some SO posts regarding CORS, advising to use:
devServer: {
headers: {
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Credentials": "true",
"Access-Control-Allow-Methods": "GET, POST, PUT, DELETE, PATCH, OPTIONS",
"Access-Control-Allow-Headers": "X-Requested-With, content-type, Authorization"
}
That didn't help either.
Problem description:
When I make a "fetch" call in the UI project, trying to communicate with the Servant endpoint I get:
Access to fetch at 'http://localhost:8081/login' from origin 'http://localhost:8833'
has been blocked by CORS policy: Response to preflight request doesn't pass access control
check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an
opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource
with CORS disabled.
The way I use fetch:
let data = {
credentialsUserName: this.state.login,
credentialsPassword: this.state.password
};
let opts : RequestInit = {
method: "POST",
headers: new Headers({ "Content-Type": "application/json" }),
body: JSON.stringify(data)
};
fetch("http://localhost:8081/login", opts).then((value) => {
console.log(value);
// do something later
});
Questions
What shall I do in order to make the fetch work fine with the servant endpoint. Also, is there anything wrong with my application architecture ? Maybe there's a better way to design it.
Last thing: The entire code of my application is quite long, so I decided to put only to most important parts here. Feel free to request more if needed.
Edit 1
After sending the fetch request I can see two entries in "Network" tab in my browser.
The first one with the following headers:
content-type: application/json
Referer: http://localhost:8833/Login
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36
and json data I included in its body. It's status is "(failed) net::ERR_FAILED".
The other's status is 400 and contains more headers:
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Access-Control-Request-Headers: content-type
Access-Control-Request-Method: POST
Connection: keep-alive
Host: localhost:8081
Origin: http://localhost:8833
Referer: http://localhost:8833/Login
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-site
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36
Interestingly, In "General" tab I can see:
Request URL: http://localhost:8081/login
Request Method: OPTIONS
Status Code: 400 Bad Request
Remote Address: 127.0.0.1:8081
Referrer Policy: no-referrer-when-downgrade
Request Method "OPTIONS" looks mysterious. Why is it there ?
As a quick guess I tried adding "OPTIONS" to CORS definition on Haskell side, but no luck this time either:
port :: Int
port = 8081
corsPolicy :: Middleware
corsPolicy = cors (const $ Just policy)
where
policy = simpleCorsResourcePolicy
{ corsMethods = [ "GET", "POST", "PUT", "OPTIONS" ]}
main :: IO ()
main = do
migrateDB
jwtConfig <- getJwtConfig
putStrLn $ "Serving endpoint " ++ (show port)
run port $ corsPolicy $ serveWithContext proxy (context cookieConfig jwtConfig) (appAPI cookieConfig jwtConfig)
EDIT 2
Thanks for the help - I've managed to configure Servant, here it is:
corsPolicy :: Middleware
corsPolicy = cors (const $ Just policy)
where
policy = simpleCorsResourcePolicy
{
corsMethods = [ "GET", "POST", "PUT", "OPTIONS" ],
corsOrigins = Just (["http://localhost:8833"], True),
corsRequestHeaders = [ "authorization", "content-type" ]
}
The remaining part of Main as in my previous listing.
By default Haskell CORS doesn't accept any headers. Not even "Content-Type", which I intend to send.
Have a simple get Customer api that's returning list of customers fine.
Setting up for service to service authentication, if I make this [Authenticated] and try to implement using ApiKeyAuthProvider, the req.GetApiKey returns null and I get an error;
Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request starting HTTP/1.1 POST https://localhost:44347/api/customers application/json 0
Microsoft.AspNetCore.Hosting.Internal.WebHost:2019-07-01 16:50:34,004 [16] INFO Microsoft.AspNetCore.Hosting.Internal.WebHost - Request starting HTTP/1.1 POST https://localhost:44347/api/customers application/json 0
The thread 0x42cc has exited with code 0 (0x0).
The thread 0x302c has exited with code 0 (0x0).
ServiceStack.ServiceStackHost:2019-07-01 17:01:14,601 [16] ERROR ServiceStack.ServiceStackHost - ServiceBase<TRequest>::Service Exception
System.ArgumentOutOfRangeException: Length cannot be less than zero.
Parameter name: length
at System.String.Substring(Int32 startIndex, Int32 length)
at ServiceStack.Host.HttpRequestAuthentication.GetBasicAuth(IRequest httpReq) in C:\BuildAgent\work\3481147c480f4a2f\src\ServiceStack\Host\HttpRequestAuthentication.cs:line 45
at ServiceStack.Host.HttpRequestAuthentication.GetBasicAuthUserAndPassword(IRequest httpReq) in C:\BuildAgent\work\3481147c480f4a2f\src\ServiceStack\Host\HttpRequestAuthentication.cs:line 50
at ServiceStack.Auth.ApiKeyAuthProvider.PreAuthenticate(IRequest req, IResponse res) in C:\BuildAgent\work\3481147c480f4a2f\src\ServiceStack\Auth\ApiKeyAuthProvider.cs:line 232
at ServiceStack.AuthenticateAttribute.PreAuthenticate(IRequest req, IEnumerable`1 authProviders) in C:\BuildAgent\work\3481147c480f4a2f\src\ServiceStack\AuthenticateAttribute.cs:line 96
at ServiceStack.AuthenticateAttribute.ExecuteAsync(IRequest req, IResponse res, Object requestDto) in C:\BuildAgent\work\3481147c480f4a2f\src\ServiceStack\AuthenticateAttribute.cs:line 74
at ServiceStack.Host.ServiceRunner`1.ExecuteAsync(IRequest req, Object instance, TRequest requestDto) in C:\BuildAgent\work\3481147c480f4a2f\src\ServiceStack\Host\ServiceRunner.cs:line 127
Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request finished in 640574.8754ms 400 application/json; charset=utf-8
Microsoft.AspNetCore.Hosting.Internal.WebHost:2019-07-01 17:01:14,607 [16] INFO Microsoft.AspNetCore.Hosting.Internal.WebHost - Request finished in 640574.8754ms 400 application/json; charset=utf-8
Clearly I have missed something obvious...any pointers appreciated.
// Register ORMLite connection
container.Register<IDbConnectionFactory>(dbFactory);
//Tell ServiceStack you want to persist User Auth Info in SQL Server
container.Register<IAuthRepository>(c => new OrmLiteAuthRepository(dbFactory));
// See https://docs.servicestack.net/api-key-authprovider
Plugins.Add(new AuthFeature(() => new AuthUserSession(),
new IAuthProvider[] {
new ApiKeyAuthProvider(AppSettings) {
SessionCacheDuration = TimeSpan.FromMinutes(10),
AllowInHttpParams = true, // Whether to allow API Keys in 'apikey' QueryString or FormData (e.g. `?apikey={APIKEY}`)
RequireSecureConnection = true,
},
}
) {
IncludeRegistrationService = true,
});
GlobalRequestFilters.Add((req, res, dto) =>
{
LastApiKey = req.GetApiKey();
});
Request
POST https://localhost:44347/api/customers HTTP/1.1
Host: localhost:44347
Connection: keep-alive
Content-Length: 2
Accept: application/json
Origin: https://localhost:44347
Authorization: yDOr26HsxyhpuRB3qbG07qfCmDhqutnA-yDOr26HsxyhpuRB3qbG07qfCmDhqutnA-yDOr26HsxyhpuRB3qbG07qfCmDhqutnA
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36
Content-Type: application/json
Referer: https://localhost:44347/swagger-ui/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9,en-US;q=0.8
{}
Response
HTTP/1.1 400 ArgumentOutOfRangeException
Transfer-Encoding: chunked
Content-Type: application/json; charset=utf-8
Vary: Accept,Origin
Server: Microsoft-IIS/10.0
X-Powered-By: ServiceStack/5.50 NetCore/Windows
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, PATCH, OPTIONS
Access-Control-Allow-Headers: Content-Type
X-Startup-Errors: 1
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: Content-Disposition
X-SourceFiles: =?UTF-8?B?QzpcUmVwb3NcTUJXZWJccnZhcGlcUnZXZWJcUnZBcGlcYXBpXGN1c3RvbWVycw==?=
X-Powered-By: ASP.NET
Date: Wed, 03 Jul 2019 08:07:40 GMT
13e
{"responseStatus":{"errorCode":"ArgumentOutOfRangeException","message":"Length cannot be less than zero.\r\nParameter name: length","errors":[{"errorCode":"ArgumentOutOfRangeException","fieldName":"length","message":"Length cannot be less than zero.\r\n"}]},"responseCreatedUtcDateTime":"2019-07-03T08:07:40.7955827Z"}
0
Your client is sending an invalid Authorization Bearer Token, it needs to have the Authroization Bearer Token format:
Authorization: Bearer {Token}
If you're sending an Authenticated API Key or JWT Request via Open API it needs to have the Bearer prefix as per the Open API docs:
OK I had manually created a User and APIKey in the underlying tables and had used a UserAuthId 'SomeAuthId' i.e. letter in them, and the ORM repository code is expecting these to be integers. Its cool that I can see the code in github and debug this myself - thanks for the comment as it got me thinking and looking into my Auth setup.
I need to occasionally access an HTML page to update a database. This page is easily accessible via a web browser, but when I try to access it via a node.js application it doesn't work (the website detect that the request is made by a robot). However,
The robot request contains the same headers (including the
user-agent) that the web browser request.
The robot request doesn't contains referer header or cookie header, but the browser request either.
The IP of the robot is the same that the IP that I use
to browse the website.
In my eyes the robot request and the browser request are strictly identical. Nevertheless they are processed differently.
I'm running out of ideas... Maybe the request contains metadata like "this request was sent by node.js" but it would be really weird.
EDIT, here is a code sample :
// callback (error, responseContent)
function getPage (callback){
let options = {
protocol : 'https:',
hostname : 'xxx.yyy.fr',
port : 443,
path : '/abc/def',
agent : false,
headers : {
'Accept' : 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Encoding' : 'gzip, deflate, br',
'Accept-Language' : 'fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3',
'Cache-Control' : 'no-cache',
'Connection' : 'keep-alive',
'DNT' : '1',
'Host' : 'ooshop.carrefour.fr',
'Pragma' : 'no-cache',
'Upgrade-Insecure-Requests' : '1',
'User-Agent' : 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0'
}
};
https.get (options, function (res){
if (res.statusCode !== 200){
res.resume ();
callback ('Error : res code != 200, res code = ' + res.statusCode);
return;
}
res.setEncoding ('utf-8');
let content = '';
res.on ('data', chunk => content += chunk);
res.on ('end', () => callback (null, content));
}).on ('error', e => callback (e));
}
EDIT : here is a comparison of the requests/responses :
Mozilla Firefox
request headers :
GET /3274080001005/eau-de-source-cristaline HTTP/1.1
Host: ooshop.carrefour.fr
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate, br
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Pragma: no-cache
Cache-Control: no-cache
response headers :
HTTP/2.0 200 OK
date: Wed, 11 Jul 2018 21:25:25 GMT
server: Unknown
content-type: text/html; charset=UTF-8
age: 0
x-varnish-cache: MISS
accept-ranges: bytes
set-cookie: visid_incap_1213048=G8a0mWzmQYi0GKuT2Ht7YeQ9QVsAAAAAQkIPAAAAAADvVZnsZHK18dQQxHakBprg; expires=Thu, 11 Jul 2019 11:17:56 GMT; path=/; Domain=.carrefour.fr
incap_ses_466_1213048=/2NKHS4HXU0T7FpkwpJ3BsV1RlsAAAAAAY3wbUkXacAceu2NkgUrhw==; path=/; Domain=.carrefour.fr
x-iinfo: 7-11020186-11020187 NNNN CT(1 2 0) RT(1531344324722 0) q(0 0 0 0) r(4 4) U12
x-cdn: Incapsula
content-encoding: gzip
X-Firefox-Spdy: h2
response content : expected HTML page
Node.js robot
request headers :
Accept : text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding : gzip, deflate, br
Accept-Language : fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3
Cache-Control : no-cache
Connection : keep-alive
DNT : 1
Host : ooshop.carrefour.fr
Pragma : no-cache
Upgrade-Insecure-Requests : 1
User-Agent : Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0
response headers :
content-type : text/html
connection : close, close
cache-control : no-cache
content-length : 210
x-iinfo : 1-17862634-0 0NNN RT(1531344295049 65) q(0 -1 -1 0) r(0 -1) B10(4,314,0) U19
set-cookie : incap_ses_466_1213048=j34jMBWkPFYT7FpkwpJ3Bqd1RlsAAAAAVBfoZBShAvoun/M8UFxPPA==; path=/; Domain=.carrefour.fr
response content :
<html>
<head>
<META NAME="robots" CONTENT="noindex,nofollow">
<script src="/_Incapsula_Resource?SWJIYLWA=5074a744e2e3d891814e9a2dace20bd4,719d34d31c8e3a6e6fffd425f7e032f3">
</script>
<body>
</body></html>
When trying to redirect the user to the login page, I always get this error:
Response to preflight request doesn't pass access control check: The
value of the 'Access-Control-Allow-Origin' header in the response must
not be the wildcard '*' when the request's credentials mode is
'include'. Origin 'null' is therefore not allowed access. The
credentials mode of requests initiated by the XMLHttpRequest is
controlled by the withCredentials attribute.
On my express application, I already implemented CORS:
var corsOptions = {
origin: 'http://localhost:4200',
credentials: true
}
app.use(cors(corsOptions));
Then I try to redirect the user:
router.use(function(req,res,next){
if((req.session.user == null){
res.redirect('http://localhost:4200' + '/login')
}
else
next();
});
On Angular 4 I'm sending all requests with {withCredentials : true}, because I'm using cookies;
These are the request/response headers:
Response:
Access-Control-Allow-Credentials:true
Access-Control-Allow-Origin:http://localhost:4200
Connection:keep-alive Date:Thu, 10 Aug 2017 12:09:55 GMT
Location:http://localhost:4200/login
set-cookie:semanaintegrada.session=s%3A6nieAv1pfn2V-2x7H4HbnqJFbYsgJmwy.hO1QKm%2Fgm6Kmso8pLCQ5zrZAVNhIYgfr%2BgzOB0oI9UA;
Path=/; Expires=Thu, 10 Aug 2017 13:09:55 GMT
Transfer-Encoding:chunked Vary:Origin X-Powered-By:Express
Request:
Accept:application/json, text/plain, / Accept-Encoding:gzip, deflate,
br Accept-Language:pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4
Cache-Control:no-cache Connection:keep-alive Content-Length:39
content-type:application/json
Cookie:semanaintegrada.session=s%3AwuiVwYs3Ahs4dfLULfpqMBcrnbthY7sZ.OYZk%2FCnZGHAe8v1T8nWpbAdFQVsXjUFAQxnYI27%2FZlE
Host:localhost:3000 Origin:http://localhost:4200 Pragma:no-cache
Referer:http://localhost:4200/ User-Agent:Mozilla/5.0 (Windows NT
10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36
Also, in Chrome there are 4 requests that are created ( I dont know why ).
3 to my express app
and 1 to localhost:4200
The last one have Access-Control-Allow-Origin:* and Origin null
Could this be related?
"credentials flag" refers to XMLHttpRequest.withCredentials of the request being made, not to an Access-Control-Allow-Credentials header.
If the request's withCredentials is true, Access-Control-Allow-Origin: * can't be used, even if there is no Access-Control-Allow-Credentials header.
I need to get data from site. To get this data user have to enter post code first. After exploring the source code i got the following.
Response result (that is what i need after all)
{PostCodePK: 16666, PostCode: "7468", City: "MACQUARIE HEADS", State: "TAS", Country: "AUST",…}
1
:
{PostCodePK: 16667, PostCode: "7468", City: "STRAHAN", State: "TAS", Country: "AUST",…}
Request Data.
Request URL:http://www.lucasmill.com/Resources/ws-common.aspx
Request Method:POST
Status Code:200 OK
Remote Address:111.67.1.113:80
Response Headers
view source
Cache-Control:private
Content-Length:321
Content-Type:application/json; charset=utf-8
Date:Tue, 21 Feb 2017 20:19:26 GMT
Expires:Tue, 21 Feb 2017 20:19:26 GMT
Server:Microsoft-IIS/8.5
Set-Cookie:dnn_IsMobile=False; path=/; HttpOnly
Request Headers
view source
Accept:application/json, text/javascript, */*; q=0.01
Accept-Encoding:gzip, deflate
Accept-Language:en-US,en;q=0.8,ru;q=0.6,uk;q=0.4
Connection:keep-alive
Content-Length:18
Content-Type:application/json; charset=UTF-8
Cookie:.ASPXANONYMOUS=ptTlH_jC0gEkAAAAZTU4MTA5NTItZmNlZS00MzRjLThmYTgtMWZkYWNkOTEwZmY00; dnn_IsMobile=False; language=en-AU; __utmt=1; __utma=97280258.254723646.1487697408.1487697408.1487708346.2; __utmb=97280258.1.10.1487708346; __utmc=97280258; __utmz=97280258.1487697408.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)
DNN-Service:true
DNN-Service-Method:GetTown
DNT:1
Host:www.lucasmill.com
Origin:http://www.lucasmill.com
Referer:http://www.lucasmill.com/Sawmilling-Contractors
User-Agent:Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36
X-Requested-With:XMLHttpRequest
Request Payload
{ 'param':'7468' }
Here is my python code.
import requests
r = requests.post('http://www.lucasmill.com/Resources/ws-common.aspx',data={ 'param':'7468' })
print(r.text)
but all i receive in response is an empty string.
Where am i wrong?
You need to add a few extra headers that particular application requires.
If you look at the request headers dump in the browser, you can see the following:
So, translated to python it will look like:
import json
import requests
headers = {
'DNN-Service': 'true',
'DNN-Service-Method': 'GetTown',
}
r = requests.post('http://www.lucasmill.com/Resources/ws-common.aspx',data=json.dumps({'param':7468}), headers=headers)
print(r.text)
Output:
[none][22:28:30] vlazarenko#alluminium (~/tests)$ python post.py
[{"PostCodePK":16666,"PostCode":"7468","City":"MACQUARIE
HEADS","State":"TAS","Country":"AUST","Latitude":"-42.2149353","Longitude":"145.1951436","CanGeocode":true},{"PostCodePK":16667,"PostCode":"7468","City":"STRAHAN","State":"TAS","Country":"AUST","Latitude":"-42.1534771","Longitude":"145.3281242","CanGeocode":true}]