Let's start simple server:
var http = require('http');
http.createServer(function (req, res) {
console.log('asdasd');
res.end('asdasd');
}).listen(8898)
And make a simple request
curl -v 'localhost:8898/?ab'
* Trying ::1...
* Connected to localhost (::1) port 8898 (#0)
> GET /?ab HTTP/1.1
> Host: localhost:8898
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 13 Oct 2016 20:26:14 GMT
< Connection: keep-alive
< Content-Length: 6
<
* Connection #0 to host localhost left intact
asdasd
Looks like everything is all right.
But if we add a literal space to it...
cornholio-osx:~/>curl -v 'localhost:8898/?a b'
* Trying ::1...
* Connected to localhost (::1) port 8898 (#0)
> GET /?a b HTTP/1.1
> Host: localhost:8898
> User-Agent: curl/7.43.0
> Accept: */*
>
* Empty reply from server
* Connection #0 to host localhost left intact
Nothing is logged and no body is written.
I assume, literal spaces in URLs are violation of HTTP protocol but is this behavior HTTP-complaint?
Related
Our React Development build runs flawless with Express Gateway setup on localhost. After build React for production and when we run serve -s build login page comes as it is the entry point of the app. It gets 200 ok response when we put sign-in credential. But when we looked into it we can see the request to server was not successful cause token it saves to browser application is undefined and we checked the response, It is "You need to enable javascript...". JS is enabled no doubt. I have checked By using
axios.post('http://localhost:8080/api/v1/auth/sign-in', userData)
It works fine but when setup proxy:
axios.post('/auth/sign-in', userData)
react doesn’t run
Here is the part of yml for express gateway setup:
http:
port: 8080
apiEndpoints:
auth-service:
host: "*"
paths: ["/api/v1/auth/*", "/api/v1/auth"]
mail-service:
host: "*"
paths: ["/api/v1/mail/*", "/api/v1/mail"]
serviceEndpoints:
auth-service-endpoint:
url: http://localhost:3003/
mail-service-endpoint:
url: http://localhost:3005/
policies:
- proxy
pipelines:
auth-service-pipeline:
apiEndpoints:
- auth-service
policies:
- proxy:
action:
serviceEndpoint: auth-service-endpoint
changeOrigin: true
stripPath: true
mail-service-pipeline:
apiEndpoints:
- mail-service
policies:
- proxy:
action:
serviceEndpoint: mail-service-endpoint
changeOrigin: true
stripPath: true
I put the setupProxy.js on src directory of React:
const { createProxyMiddleware } = require('http-proxy-middleware');
module.exports = function(app) {
app.use(createProxyMiddleware('/api/v1',
{ target: 'http://localhost:8080',
secure: false,
changeOrigin: true,
// pathRewrite: {
// "^/api": "/api/v1",
// }
}
));
}
Currently everything is on same machine. We are not using docker.
The application runs on Dev environment but shows 200 ok response in production build
Any help will be appreciated.
[Edit]
krypton:admin-dashboard-server hasan$ curl -v http://localhost:3001/find_all_services/1/10
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 3001 (#0)
> GET /find_all_services/1/10 HTTP/1.1
> Host: localhost:3001
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Access-Control-Allow-Origin: *
< X-DNS-Prefetch-Control: off
< X-Frame-Options: SAMEORIGIN
< Strict-Transport-Security: max-age=15552000; includeSubDomains
< X-Download-Options: noopen
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Content-Type: application/json; charset=utf-8
< Content-Length: 1833
< ETag: W/"729-LM91B3vCUrbvesBrp32ykiXXkQo"
< Date: Tue, 12 Jan 2021 14:57:24 GMT
< Connection: keep-alive
<
* Connection #0 to host localhost left intact
[{"id":1,"name":"Laser Hair Remove"},
{"id":2,"name":"Facial Treatments"}
]
krypton:admin-dashboard-server hasan$ curl -v
http://localhost:8080/api/v1/services/find_all_services/1/10
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> GET /api/v1/services/find_all_services/1/10 HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< access-control-allow-origin: *
< x-dns-prefetch-control: off
< x-frame-options: SAMEORIGIN
< strict-transport-security: max-age=15552000; includeSubDomains
< x-download-options: noopen
< x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
< content-type: application/json; charset=utf-8
< content-length: 1833
< etag: W/"729-LM91B3vCUrbvesBrp32ykiXXkQo"
< date: Tue, 12 Jan 2021 15:03:45 GMT
< connection: keep-alive
<
* Connection #0 to host localhost left intact
[{"id":1,"name":"Laser Hair Remove"},
{"id":2,"name":"Facial Treatments"}
]
krypton:admin-dashboard-server hasan$ curl -v -H "Content-Type: application/json" -X POST -d
'{"email":"mh.mithun#gmail.com","password":"safe123"}'
http://localhost:8080/api/v1/auth/sign-in
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> POST /api/v1/auth/sign-in HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 52
>
* upload completely sent off: 52 out of 52 bytes
< HTTP/1.1 200 OK
< access-control-allow-origin: *
< x-dns-prefetch-control: off
< x-frame-options: SAMEORIGIN
< strict-transport-security: max-age=15552000; includeSubDomains
< x-download-options: noopen
< x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
< content-type: application/json; charset=utf-8
< content-length: 270
< etag: W/"10e-S+kd8b4Yfl7un04FVGe3MFLFEaY"
< date: Tue, 12 Jan 2021 15:40:12 GMT
< connection: keep-alive
<
* Connection #0 to host localhost left intact
"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhbGdvcml0aG0iOiJIUzI1N"
krypton:admin-dashboard-server hasan$ curl -v -H "Content-Type: application/json" -X POST -d '{"email":"mh.mithun#gmail.com","password":"safe123"}' http://localhost:3003/sign-in
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 3003 (#0)
> POST /sign-in HTTP/1.1
> Host: localhost:3003
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 52
>
* upload completely sent off: 52 out of 52 bytes
< HTTP/1.1 200 OK
< Access-Control-Allow-Origin: *
< X-DNS-Prefetch-Control: off
< X-Frame-Options: SAMEORIGIN
< Strict-Transport-Security: max-age=15552000; includeSubDomains
< X-Download-Options: noopen
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Content-Type: application/json; charset=utf-8
< Content-Length: 270
< ETag: W/"10e-LW/1l5fXf5BaiF3KJMvG60xRthE"
< Date: Tue, 12 Jan 2021 15:45:33 GMT
< Connection: keep-alive
<
* Connection #0 to host localhost left intact
"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhbGdvcml0aG0i"
I am trying to get a response from portquiz.net when probing port 80. For example, if we do this:
curl portquiz.net:80
we get the response:
Port 80 test successful!
Here is the python code:
import socket
server = "portquiz.net"
port = 80
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((server, port))
response = s.recv(1024)
print(repr(response))
With this code I get no response, the script just seems to hang.
Is this an issue with my code or is it something to do with portquiz's server?
Fingers, that is an HTTP server, so you need to make a GET request to it. As soon as you connect it is waiting for you to send data, that is why it hangs.
You can do this more easily with an HTTP library, however, if you want to use socket here is the code, with example run:
(xcve) ttucker#plato:~/tmp/stackoverflow/portquiz.net$ cat test.py
import socket
server = "portquiz.net"
port = 80
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((server, port))
# Send HTTP GET request to /
s.send('GET / HTTP/1.1\r\nHOST: {}\r\n\r\n'.format(server).encode())
response = s.recv(1024)
print(repr(response))
(xcve) ttucker#plato:~/tmp/stackoverflow/portquiz.net$ python test.py
b'HTTP/1.1 200 OK\r\nDate: Sun, 21 Jul 2019 22:25:32 GMT\r\nServer: Apache/2.4.29 (Ubuntu)\r\nVary: Accept-Encoding\r\nContent-Length: 2747\r\nContent-Type: text/html; charset=UTF-8\r\n\r\n\n<html>\n<head>\n<title>Outgoing Port Tester</title>\n<style type="text/css">\nbody {\n\tfont-family: sans-serif;\n\tfont-size: 0.9em;\n}\n</style>\n\n</head>\n\n<body>\n<h1>Outgoing port tester</h1>\n\nThis server listens on all TCP ports, allowing you to test any outbound TCP port.\n\n<p>\nYou have reached this page on port <b>80</b>.<br/>\n</p>\n\nYour network allows you to use this port.\n(Assuming that your network is not doing advanced traffic filtering.)\n\n<p>\nNetwork service: http<br/>\nYour outgoing IP: 207.135.66.186</p>\n\n<h2>Test a port using a command</h2>\n\n<pre>\n$ telnet portquiz.net 80 \nTrying ...\nConnected to portquiz.net.\nEscape character is \'^]\'.\n</pre>\n<pre>\n$ nc -v portquiz.net 80 \nConnection to portquiz.net 80 port [tcp/daytime] succeeded!\n</pre>\n<pre>\n$ curl portquiz.net:80 \nPort 80 test successful!\nYour IP: 207.135.66.186</pre>\n<pre>\n$ wget -q'
Further thoughts on this, it looks like the sight might be looking for a CURL header when to determine which version - either HTML, or TEXT - that it sends. It may behoove you to specify the same header after the "HOST: " header that CURL does so the response is easier to parse.
For the example of it with a curl header, and how I figured out what to put there:
(xcve) ttucker#plato:~/tmp/stackoverflow/portquiz.net$ curl portquiz.net:80 -vvvv
* Rebuilt URL to: portquiz.net:80/
* Hostname was NOT found in DNS cache
* Trying 52.47.209.216...
* Connected to portquiz.net (52.47.209.216) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: portquiz.net
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sun, 21 Jul 2019 22:34:36 GMT
* Server Apache/2.4.29 (Ubuntu) is not blacklisted
< Server: Apache/2.4.29 (Ubuntu)
< Content-Length: 49
< Content-Type: text/html; charset=UTF-8
<
Port 80 test successful!
Your IP: 207.135.66.186
* Connection #0 to host portquiz.net left intact
(xcve) ttucker#plato:~/tmp/stackoverflow/portquiz.net$ cat test.py
import socket
server = "portquiz.net"
port = 80
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((server, port))
get_request = "GET / HTTP/1.1\r\nHOST: {}\r\n" \
"User-Agent: curl/7.35.0\r\n\r\n".format(server)
s.send(get_request.encode())
response = s.recv(1024)
print(repr(response))
(xcve) ttucker#plato:~/tmp/stackoverflow/portquiz.net$ python test.py
b'HTTP/1.1 200 OK\r\nDate: Sun, 21 Jul 2019 22:34:47 GMT\r\nServer: Apache/2.4.29 (Ubuntu)\r\nContent-Length: 49\r\nContent-Type: text/html; charset=UTF-8\r\n\r\nPort 80 test successful!\nYour IP: 207.135.66.186\n'
sendFile is for sending files and it also figures out some interesting headers from the file (like content length). For a HEAD request I would ideally want the exact same headers but just skip the body.
There doesn't seem to be an option for this in the API. Maybe I can override something in the response object to stop it from sending anything?
Here's what I got:
res.sendFile(file, { headers: hdrs, lastModified: false, etag: false })
Has anyone solved this?
As Robert Klep has already written, the sendFile already has the required behavior of sending the headers and not sending the body if the request method is HEAD.
In addition to that, Express already handles HEAD requests for routes that have GET handlers defined. So you don't even need to define any HEAD handler explicitly.
Example:
let app = require('express')();
let file = __filename;
let hdrs = {'X-Custom-Header': '123'};
app.get('/file', (req, res) => {
res.sendFile(file, { headers: hdrs, lastModified: false, etag: false });
});
app.listen(3322, () => console.log('Listening on 3322'));
This sends its own source code on GET /file as can be demonstrated with:
$ curl -v -X GET localhost:3322/file
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 3322 (#0)
> GET /file HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:3322
> Accept: */*
>
< HTTP/1.1 200 OK
< X-Powered-By: Express
< X-Custom-Header: 123
< Accept-Ranges: bytes
< Cache-Control: public, max-age=0
< Content-Type: application/javascript
< Content-Length: 267
< Date: Tue, 11 Apr 2017 10:45:36 GMT
< Connection: keep-alive
<
[...]
The [...] is the body that was not included here.
Without adding any new handler this will also work:
$ curl -v -X HEAD localhost:3322/file
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 3322 (#0)
> HEAD /file HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:3322
> Accept: */*
>
< HTTP/1.1 200 OK
< X-Powered-By: Express
< X-Custom-Header: 123
< Accept-Ranges: bytes
< Cache-Control: public, max-age=0
< Content-Type: application/javascript
< Content-Length: 267
< Date: Tue, 11 Apr 2017 10:46:29 GMT
< Connection: keep-alive
<
This is the same but with no body.
Express uses send to implement sendFile, which already does exactly what you want.
I have an API server running at localhost:3000 and I am trying to query it using these two expressions:
[wani#lenovo ilparser-docker]$ time (curl "localhost:3000/parse?lang=hin&data=देश" )
{"tokenizer":"<Sentence id=\"1\">\n1\tदेश\tunk\n<\/Sentence>\n"}
real 0m0.023s
user 0m0.009s
sys 0m0.004s
[wani#lenovo ilparser-docker]$ time (curl -XGET localhost:3000/parse -F lang=hin -F data="देश" )
{"tokenizer":"<Sentence id=\"1\">\n1\tदेश\tunk\n<\/Sentence>\n"}
real 0m1.101s
user 0m0.020s
sys 0m0.070s
Why does the second expression take so much more time?
With more verbosity:
[wani#lenovo ilparser-docker]$ time curl -v localhost:3000/parse -F lang=hin -F data="देश"
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 3000 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
> POST /parse HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 244
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=------------------------1eb5e5991b976cb1
>
* Done waiting for 100-continue
< HTTP/1.1 200 OK
< Content-Length: 70
< Server: Mojolicious (Perl)
< Content-Type: application/json;charset=UTF-8
< Date: Mon, 21 Mar 2016 11:06:09 GMT
< Connection: keep-alive
<
* Connection #0 to host localhost left intact
{"tokenizer":"<Sentence id=\"1\">\n1\tदेश\tunk\n<\/Sentence>\n"}
real 0m1.106s
user 0m0.027s
sys 0m0.068s
[wani#lenovo ilparser-docker]$ time curl -v localhost:3000/parse --data lang=hin --data data="देश"
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 3000 (#0)
> POST /parse HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 23
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 23 out of 23 bytes
< HTTP/1.1 200 OK
< Server: Mojolicious (Perl)
< Content-Length: 70
< Connection: keep-alive
< Date: Mon, 21 Mar 2016 11:06:24 GMT
< Content-Type: application/json;charset=UTF-8
<
* Connection #0 to host localhost left intact
{"tokenizer":"<Sentence id=\"1\">\n1\tदेश\tunk\n<\/Sentence>\n"}
real 0m0.031s
user 0m0.011s
sys 0m0.003s
Expect: 100-continue sounded fishy, so I cleared that header:
[wani#lenovo ilparser-docker]$ time curl -v -F lang=hin -F data="देश" "localhost:3000/parse" -H Expect: --trace-time
16:48:04.513691 * Trying 127.0.0.1...
16:48:04.513933 * Connected to localhost (127.0.0.1) port 3000 (#0)
16:48:04.514083 * Initializing NSS with certpath: sql:/etc/pki/nssdb
16:48:04.610095 > POST /parse HTTP/1.1
16:48:04.610095 > Host: localhost:3000
16:48:04.610095 > User-Agent: curl/7.43.0
16:48:04.610095 > Accept: */*
16:48:04.610095 > Content-Length: 244
16:48:04.610095 > Content-Type: multipart/form-data; boundary=------------------------24f30647b16ba82d
16:48:04.610095 >
16:48:04.618107 < HTTP/1.1 200 OK
16:48:04.618194 < Content-Length: 70
16:48:04.618249 < Server: Mojolicious (Perl)
16:48:04.618306 < Content-Type: application/json;charset=UTF-8
16:48:04.618370 < Date: Mon, 21 Mar 2016 11:18:04 GMT
16:48:04.618430 < Connection: keep-alive
16:48:04.618492 <
16:48:04.618590 * Connection #0 to host localhost left intact
{"tokenizer":"<Sentence id=\"1\">\n1\tदेश\tunk\n<\/Sentence>\n"}
real 0m0.117s
user 0m0.023s
sys 0m0.082s
Now the only time taking thing left is: Initializing NSS with certpath: sql:/etc/pki/nssdb. Why does curl do that in this context?
After a little help on IRC from #DanielStenberg, I came to know that the db load is present because curl inits nss in that case since curl needs a good random source for the boundary separator used for -F . Curl could have used getrandom() syscall or read bits out of /dev/urandom since boundary separators don't need to be cryptographically secure in any way, but curl just wants secure random in some other places so curl reuses the random function that it already has.
I am trying to POST a JPEG image from a particular file directory to the server curl. This is what I typed:
curl -v -include --form filedata='/home/pi/Documents/2014-01-18-09:11:25.jpeg' http://hostdomain.me/file/upload
Upon executing this command, the following is returned from the Terminal:
* Couldn't find host rdnvpfwnwk.localtunnel.me in the .netrc file; using defaults
* About to connect() to rdnvpfwnwk.localtunnel.me port 80 (#0)
* Trying 192.34.58.73...
* connected
* Connected to rdnvpfwnwk.localtunnel.me (192.34.58.73) port 80 (#0)
> POST /file/upload HTTP/1.1
> User-Agent: curl/7.26.0
> Host: rdnvpfwnwk.localtunnel.me
> Accept: */*
> Content-Length: 186
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=----------------------------affc91df7bc3
>
* additional stuff not fine transfer.c:1037: 0 0
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 100 Continue
HTTP/1.1 100 Continue
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 500 Internal Server Error
HTTP/1.1 500 Internal Server Error
< Server: nginx
Server: nginx
< Date: Sat, 18 Jan 2014 11:44:39 GMT
Date: Sat, 18 Jan 2014 11:44:39 GMT
< Content-Type: text/plain
Content-Type: text/plain
< Content-Length: 9245
Content-Length: 9245
< Connection: keep-alive
Connection: keep-alive
< X-Powered-By: Sails <sailsjs.org>
X-Powered-By: Sails <sailsjs.org>
* Added cookie sails.sid="s%3AAucWnGhDSzZSGB_tBgSXJoU2.DMQ4FuVVRRGLFGheMgr4CvIFUICCiP9Gqd5GIjRevA8" for domain rdnvpfwnwk.localtunnel.me, path /, expire 0
< Set-Cookie: sails.sid=s%3AAucWnGhDSzZSGB_tBgSXJoU2.DMQ4FuVVRRGLFGheMgr4CvIFUICCiP9Gqd5GIjRevA8; Path=/; HttpOnly
Set-Cookie: sails.sid=s%3AAucWnGhDSzZSGB_tBgSXJoU2.DMQ4FuVVRRGLFGheMgr4CvIFUICCiP9Gqd5GIjRevA8; Path=/; HttpOnly
* HTTP error before end of send, stop sending
<
TypeError: Cannot read property 'name' of undefined
When using the form on the site it sends properly. Here is the form:
<form id="uploadForm"
enctype="multipart/form-data"
action="/file/upload"
method="post">
<input type="file" id="userPhotoInput" name="userPhoto" />
<input type="submit" value="Submit">
</form>
How can this be fixed?
use this :
curl -v -include --form "userPhoto=#/home/pi/Documents/2014-01-18-09:11:25.jpeg" http://hostdomain.me/file/upload
^^^^^^^^^ ^