I'm working with curl in Linux. I'm downloading a part of a file from Media-fire using bad internet connection , the download always stops after few minutes and when i use the parameter -C - instead of continue downloading only the part of a file i mentioned from where the download stopped , it starts downloading the hole file .
This is command i have used :
curl -v -o file.part8 -r3000000001-3200000000 --retry 999 --retry-max-time 0 -C - http://download2331.mediafire.com/58gp2u3yjuzg/6upaohqwd8kdi9n/Olarila+High+Sierra+2020.raw.bz2
It's clear that server doesn't support byte ranges
i tried with :
curl -L -k -C - -O --header "Range: bytes=0-1000000" https://cdimage.ubuntu.com/kubuntu/releases/20.10/release/kubuntu-20.10-desktop-amd64.iso
and i get :
curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.
it seems that the problem is in the server.
Related
I am attempting to do a http request within another http request. Is there a way to do this via command line in linux?
wget http://request another wget http://request
Use $() to substute the output of a command:
wget http://someURL?param="$(wget -O - http://otherURL)"
The -O - option tells wget to write the output to standard output instead of a file.
I have to fetch some data using curl linux utility. There are two cases, one request is successful and second it is not. I want to save output to a file if request is successful and if request is failed due to some reason then error code should be saved only to a log file. I have search a lot on www but could not found exact solution that's why I have posted a new question on curl.
One option is to get the response code with -w, so you could do it something like
code=$(curl -s -o file -w '%{response_code}' http://example.com/)
if test "$code" != "200"; then
echo $code >> response-log
else
echo "wohoo 'file' is fine"
fi
curl -I -s -L <Your URL here> | grep "HTTP/1.1"
curl + grep is your friend, then you can extract the status code later for your need.
I've got a NodeJS script which spins up a ngrok instance, which starts the ngrok binary.
However I'm needing to be able to return back the automatically generated url. I can't find anywhere in the documentation about how to do this.
for example, when you run ngrok http 80 it spins up, generates you a random unique url each time it starts
This question is kinda old, however, I thought to give another more generic option as it doesn't require NodeJS
curl --silent --show-error http://127.0.0.1:4040/api/tunnels | sed -nE 's/.*public_url":"https:..([^"]*).*/\1/p'
This one just inspects the response of calling api/tunnels by applying text processing (sed) to the resulted text and identifies the public URL.
ngrok serves tunnel information at http://localhost:4040/api/tunnels.
curl + jq
curl -Ss http://localhost:4040/api/tunnels | jq -r '.tunnels[0].public_url'
=> https://719c933a.ap.ngrok.io
curl + ruby
curl -Ss http://localhost:4040/api/tunnels | \
ruby -e 'require "json"; puts JSON.parse(STDIN.read).dig("tunnels", 0, "public_url")'
=> https://719c933a.ap.ngrok.io
curl + node
json=$(curl -Ss http://127.0.0.1:4040/api/tunnels);
node -pe "var data = $json; data.tunnels[0].public_url"
=> https://719c933a.ap.ngrok.io
I need to follow a log file on linux machine and stream the updates of log file over http port to a remote machine. I have written a command with the combination of "tail" and "curl".
To test it initially, i used "tail -n", it works well and posts data successfully to remote machine. Below is the command.
$tail -n 200 /path/to/logfile/file1.log | curl --data-binary #- http://remotemachineIP:9000
Now, When i try to run the same command with "tail -f", it's not posting any data over http even though the log file is updated multiple times. Below is the command
$tail -f --follow=name /path/to/logfile/file1.log | curl --data-binary #- http://remotemachineIP:9000
As per my understanding, "tail -f" is not conveying my "curl" command that "input feed is complete over stdin(#-)". Any help on how to rectify this issue?
Thanks in advance
curl will make a single HTTP POST request with the piped data. What you want to do instead is to continuously send the data.
Assuming that by "HTTP port" you actually meant TCP there is a way using netcat:
Remote
nc -l 9000
Local
tailf /path/to/log/file | nc remote_ip 9000
I'm trying to use cURL in a script and get it to not show the progress bar.
I've tried the -s, -silent, -S, and -quiet options, but none of them work.
Here's a typical command I've tried:
curl -s http://google.com > temp.html
I only get the progress bar when pushing it to a file, so curl -s http://google.com doesn't have a progress bar, but curl -s http://google.com > temp.html does.
curl -s http://google.com > temp.html
works for curl version 7.19.5 on Ubuntu 9.10 (no progress bar). But if for some reason that does not work on your platform, you could always redirect stderr to /dev/null:
curl http://google.com 2>/dev/null > temp.html
In curl version 7.22.0 on Ubuntu and 7.24.0 on OSX the solution to not show progress but to show errors is to use both -s (--silent) and -S (--show-error) like so:
curl -sS http://google.com > temp.html
This works for both redirected output > /some/file, piped output | less and outputting directly to the terminal for me.
Update: Since curl 7.67.0 there is a new option --no-progress-meter which does precisely this and nothing else, see clonejo's answer for more details.
I found that with curl 7.18.2 the download progress bar is not hidden with:
curl -s http://google.com > temp.html
but it is with:
curl -ss http://google.com > temp.html
Since curl 7.67.0 (2019-11-06) there is --no-progress-meter, which does exactly this, and nothing else. From the man page:
--no-progress-meter
Option to switch off the progress meter output without muting or
otherwise affecting warning and informational messages like -s,
--silent does.
Note that this is the negated option name documented. You can
thus use --progress-meter to enable the progress meter again.
See also -v, --verbose and -s, --silent. Added in 7.67.0.
It's available in Ubuntu ≥20.04 and Debian ≥11 (Bullseye).
For a bit of history on curl's verbosity options, you can read Daniel Stenberg's blog post.
Not sure why it's doing that. Try -s with the -o option to set the output file instead of >.
this could help..
curl 'http://example.com' > /dev/null
On macOS 10.13.6 (High Sierra), the -sS option works. It is especially useful inside Perl, in a command like curl -sS --get {someURL}, which frankly is a whole lot more simple than any of the LWP or HTTP wrappers, for just getting a website or web page's contents.