Why does this work:
exec 3<>/dev/tcp/www.google.com/80
echo -e "GET / HTTP/1.1\n\n">&3
cat <&3
And this fail:
echo -e "GET / HTTP/1.1\n\n" > /dev/tcp/www.google.com/80
cat </dev/tcp/www.google.com/80
Is there a way to do it in one-line w/o using wget, curl, or some other library?
The second snippet fails because it opens two separate TCP sockets. The echo connects to www.google.com and writes the HTTP request; and then the second line opens another connection and tries to read from that socket. The second socket simply blocks because Google is waiting for the HTTP request to be sent.
Not my area of expertise, but I think that the second sample will open a second connection, while the first sample keeps an open handle to the same connection. So any solution which involves opening only one connection should work.
Related
I try to read/fetch this file:
https://blockchain-office.com/file.txt with a bash script over dev/tcp without using curl,wget, etc..
I found this example:
exec 3<>/dev/tcp/www.google.com/80
echo -e "GET / HTTP/1.1\r\nhost: http://www.google.com\r\nConnection: close\r\n\r\n" >&3
cat <&3
I change this to my needs like:
exec 3<>/dev/tcp/www.blockchain-office.com/80
echo -e "GET / HTTP/1.1\r\nhost: http://www.blockchain-office.com\r\nConnection: close\r\n\r\n" >&3
cat <&3
When i try to run i receive:
400 Bad Request
Your browser sent a request that this server could not understand
I think this is because strict ssl/only https connections is on.
So i change it to :
exec 3<>/dev/tcp/www.blockchain-office.com/443
echo -e "GET / HTTP/1.1\r\nhost: https://www.blockchain-office.com\r\nConnection: close\r\n\r\n" >&3
cat <&3
When i try to run i receive:
400 Bad Request
Your browser sent a request that this server could not understand.
Reason: You're speaking plain HTTP to an SSL-enabled server port.
Instead use the HTTPS scheme to access this URL, please.
So i even can't get a normal connection without get the file!
All this post's does not fit, looks like ssl/tls is the problem only http/80 works, if i don't use curl, wget, lynx, openssl, etc...:
how to download a file using just bash and nothing else (no curl, wget, perl, etc.)
Using /dev/tcp instead of wget
How to get a response from any URL?
Read file over HTTP in Shell
I need a solution to get/read/fetch a normal txt file from a domain over https only with /dev/tcp no other tools like curl, and output in my terminal or save in a variable without wget, etc.., is it possible and how, or is it there an other solution over the terminal with the standard terminal utilities?
You can use openssl s_client to perform the equivalent operation but delegate the SSL part:
#!/bin/sh
host='blockchain-office.com'
port=443
path='/file.txt'
crlf="$(printf '\r\n_')"
crlf="${crlf%?}"
{
printf '%s\r\n' \
"GET ${path} HTTP/1.1" \
"host: ${host}" \
'Connection: close' \
''
} |
openssl s_client -quiet -connect "${host}:${port}" 2 >/dev/null | {
# Skip headers by reading up until encountering a blank line
while IFS="${crlf}" read -r line && [ -n "$line" ]; do :; done
# Output the raw body content
cat
}
Instead of cat to output the raw body, you may want to check some headers like Content-Type, Content-Transfer-Encoding and even maybe navigate and handle recursive MIME chunks, then decode the raw content to something.
After all the comments and research, the answer is no, we can't get/fetch files using only the standard tools with the shell like /dev/tcp because we can't handle ssl/tls without handle the complete handshake.
It is only possbile with the http/80.
i dont think bash's /dev/tcp supports ssl/tls
If you use /dev/tcp for a http/https connection you have to manage the complete handshake including ssl/tls, http headers, chunks and more. Or you use curl/wget that manage it for you.
then shell is the wrong tool because it is not capable of performing any of the SSL handshake without using external resources/commands. Now relieve and use what you want and can from what I show you here as the cleanest and most portable POSIX-shell grammar implementation of a minimal HTTP session through SSL. And then maybe it is time to consider alternative options (not using HTTPS, using languages with built-in or standard library SSL support).
We will use curl, wget and openssl on seperate docker containers now.
I think there are still some requirements in the future to see if we keep only one of them or all of them.
We will use the script from #Léa Gris in a docker container too.
I'm using netcat to connect to a server.
the problem is that i want to send somme non printable to the server caracters.
I wanted to achieve this with a command redirection in linux.
lets say this is the command: nc hostname port
so when i checked the file descriptors of the command nc in the folder: cd /proc/$(pidof nc)/fd is saw the there was another fd with number 3 that conserns the socket. 3 -> socket:[1675643]
the problem is that i wanted to redirect the output of let's say echo -ne '\xff\x0f\xab\xde' > ./3 to the socket.
I couldn't do so and the ouput is: bash: ./3: No such device or address
One cannot output something to a socket which is opened only by another process.
In order to first use interactive input/output and afterwards send the echo string, you can do:
(cat; echo -ne '\xff\x0f\xab\xde')|nc hostname port
(press the EOF character Ctrl-D to end your input and start the echo).
I am outputting content of a log via netcat to an application over the network. I don't know if what I'm doing is the most efficient, especially when I notice the netcat session becomes non-responsive. I have to stop netcat and start it again for the application to work again.
The command I run is:
/bin/tail -n1 -f /var/log/custom_output.log | /bin/nc -l -p 5020 --keep-open
This needs to run like this 24/7. Is this the most efficient way of doing it? How can I improve on it so I don't have to restart the process daily?
EDIT
So I realised that when the log is being rotated, netcat is locked onto a file that's not longer being written to. I can deal with this easily enough.
The question still stands. Is this the best way to do something like this?
It's been 6 years, but maybe someone will come in handy.
To account for log rotation, use tail with the -F flag.
nc (aka netcat) variant
LOG_FILE="/var/log/custom_output.log"
PORT=5020
tail -n0 -F "$LOG_FILE" | nc -k -l -p $PORT
Notes:
Flag -k in nc is analog to --keep-open in "the OpenBSD rewrite of netcat";
Multiple clients can connect to nc at the same time, but only the first one will be receive appended log lines;
tail will run immediately, so it will collect appended log lines even if no client is connected. Thus, the first client can receive some buffered data - all log lines that have been appended since tail was run.
socat variant
LOG_FILE="/var/log/custom_output.log"
PORT=5020
socat TCP-LISTEN:$PORT,fork,reuseaddr SYSTEM:"tail -n0 -F \"$LOG_FILE\" </dev/null"
Note: here socat will fork (clone itself) on each client connection and start a separate tail process. Thus:
Each connected client will receive appended log lines at the same time;
Clients will not receive any previously buffered by tail strings.
additional
You can redirect stderr to stdout in the tail process by adding 2>&1 (in both variants). In this case, clients will receive auxiliary message lines, e.g.:
tail: /var/log/custom_output.log: file truncated;
tail: '/var/log/custom_output.log' has become inaccessible: No such file or directory - printed when the log file has been removed or renamed, only if -F is used;
tail: '/var/log/custom_output.log' has appeared; following new file - printed when a new log file is created, only if -F is used.
I have BusyBox v1.23.2 multi-call binary. with simple tftp-client.
I need to check connectivity with a tftp server and accessibility of file there.
For ftp it may look like this:
if wget -q -s $url; then
echo "found"
fi
Is there a reliable solution for tftp?
p.s. I can't try to download the file (it's too big).
Update: I solved the problem by adding hack to source code of BusyBox, which allows to implement a scenario, like that:
"No.","Source","Destination","Info"
"1","192.168.0.8","192.168.0.6","Read Request, File: some_folder/file.txt, Transfer type: octet, blksize\\000=4096\\000, tsize\\000=0\\000"
"2","192.168.0.6","192.168.0.8","Option Acknowledgement, blksize\\000=4096\\000, tsize\\000=10094\\000"
"3","192.168.0.8","192.168.0.6","Error Code, Code: Not defined, Message: Connection checking"
I guess this will work for you.
$ wget --spider http://henning.makholm.net/
Spider mode enabled. Check if remote file exists.
--2011-08-08 19:39:48-- http://henning.makholm.net/
Resolving henning.makholm.net (henning.makholm.net)... 85.81.19.235
Connecting to henning.makholm.net (henning.makholm.net)|85.81.19.235|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9535 (9.3K) [text/html] <-------------------------
Remote file exists and could contain further links,
but recursion is disabled -- not retrieving.
I have a linux machine which is listening for connections on port 4450. Where there is an incomming connection, this is supposed to send continuously over the socket the content of a file. Did you do this before ?
What I've done so far was to send once the content of the file like this:
x=$(filename); echo $x | nc -l 4450
On the client side I have an Android app, which connects to the server and then using a BufferedReader gets the data from the stream and processes it.
Any help would be highly appreciated.
Thanks
Use socat instead of netcat (nc). With socat you can do almost everything that can be done with netcat. But socat has a lot more features and is easier to use.
socat TCP-LISTEN:4450,fork OPEN:/tmp/filename,rdonly
You can also use the output of a command instead of some file contents:
socat TCP-LISTEN:4450,fork EXEC:/bin/date