i need to reboot my Orbi (RBS50) router and satelites (RBS50) frequently (don't ask). Unfortunaly Netgear doesnt't offer ssh/telnet access anymore. So i need to use the webinterface.
When i'll do this in the browser everything is fine. I'll open the page, do the Basic auth, and push the button. So i did this in the firefox developer console and exported the curl command. Here is exactly how it works in a clean Browser (Firefox in Private-Mode) : full HAR-Export of the Login and the Reboot-Request i
Exportetd as Curl the two requests looks like this (Original from Firefox no edits):
Open the Reboot-Page with Basic Auth:
curl 'https://192.168.0.6/reboot.htm' -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8' -H 'Accept-Language: de,en-US;q=0.7,en;q=0.3' -H 'Accept-Encoding: gzip, deflate, br' -H 'DNT: 1' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Sec-Fetch-Dest: document' -H 'Sec-Fetch-Mode: navigate' -H 'Sec-Fetch-Site: none' -H 'Sec-Fetch-User: ?1'
Press the "Yes" button to perform the Reboot
curl 'https://192.168.0.6/apply.cgi?/reboot_waiting.htm%20timestamp=71222162842' -X POST -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8' -H 'Accept-Language: de,en-US;q=0.7,en;q=0.3' -H 'Accept-Encoding: gzip, deflate, br' -H 'Content-Type: application/x-www-form-urlencoded' -H 'Origin: https://192.168.0.6' -H 'DNT: 1' -H 'Authorization: Basic YWRtaW46UGFzc3dvcnQwMA==' -H 'Connection: keep-alive' -H 'Referer: https://192.168.0.6/reboot.htm' -H 'Cookie: auth_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOiI4OTg3NiIsImlzcyI6Ind3dy5uZXRnZWFyLmNvbSIsInN1YiI6Ik1vemlsbGEvNS4wIChYMTE7IFVidW50dTsgTGludXggeDg2XzY0OyBydjoxMDkuMCkgR2Vja28vMjAxMDAxMDEgRmlyZWZveC8xMDkuMCJ9.2b3420f66ed35dd347c38e280f6b0ebedfd5ee2fdd5b3ab53384965fb241c6ba' -H 'Upgrade-Insecure-Requests: 1' -H 'Sec-Fetch-Dest: document' -H 'Sec-Fetch-Mode: navigate' -H 'Sec-Fetch-Site: same-origin' -H 'Sec-Fetch-User: ?1' --data-raw 'submit_flag=reboot&yes=Ja+'
Here is what I've done from it.
Curl the Reeboot-Page with Basic out works fine: The html is shown, the Cookie is written.
curl --insecure -c cookies.txt -b cookies.txt 'https://192.168.0.6/reboot.htm' -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8' -H 'Accept-Language: de,en-US;q=0.7,en;q=0.3' -H 'Accept-Encoding: gzip, deflate, br' -H 'DNT: 1' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Sec-Fetch-Dest: document' -H 'Sec-Fetch-Mode: navigate' -H 'Sec-Fetch-Site: none' -H 'Sec-Fetch-User: ?1' -u "admin:Passwort00"
Try to send the POST to perform the Reboot:
curl --insecure -c cookies.txt -b cookies.txt 'https://192.168.0.6/apply.cgi?/reboot_waiting.htm%20timestamp=71222162842' -X POST -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8' -H 'Accept-Language: de,en-US;q=0.7,en;q=0.3' -H 'Accept-Encoding: gzip, deflate, br' -H 'Content-Type: application/x-www-form-urlencoded' -H 'Origin: https://192.168.0.6' -H 'DNT: 1' -H 'Authorization: Basic YWRtaW46UGFzc3dvcnQwMA==' -H 'Connection: keep-alive' -H 'Referer: https://192.168.0.6/reboot.htm' -H 'Upgrade-Insecure-Requests: 1' -H 'Sec-Fetch-Dest: document' -H 'Sec-Fetch-Mode: navigate' -H 'Sec-Fetch-Site: same-origin' -H 'Sec-Fetch-User: ?1' --data-raw 'submit_flag=reboot&yes=Ja+'
The Answher:
`400 Bad Requestclasses.num.all.fontFamily = Courier; classes.num.all.fontSize = 10pt;
400 Ungültige Anforderung
Der vom Client angeforderte Vorgang wird vom Server nicht unterstützt.
`
I have absolute no idea why the reboot button works at Firefox, but not as a Curl.
I've tried lots of combination of all headers that are in the original request.
Related
I wrote a script to download chunks (.ts) from a web page that stream some soocer videos and combine all the chunks in a one file. The issue is that the final result is a video "lagging", it seems that some frames are missing.
I encoded the references of the website to avoid get the question removed.
m3u8 url
BASE64(aHR0cHM6Ly9uZXdlZGdlLmV1LWNlbnRyYWwtMS5lZGdlLm15Y2RuLmxpdmUvbGl2ZS92aW50ZXF1YXRyb2hyczIvdmludGVxdWF0cm9ocnMyXzIwMDAvaW5kZXgubTN1OA==)
Referer and Origin headers
BASE64(aHR0cHM6Ly9mdXRlbWF4LmFwcA==)
Script
#!/bin/bash
#./hls-download.sh "$url" "$title" "$(date +\"%Y-%m-%d %H:%M:%S\")"
url_m3u8="$1"
title="$2"
duration=$(date -d "$3 minutes" +%s)
cmd_curl="curl -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:101.0) Gecko/20100101 Firefox/101.0' -H 'Accept: */*' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate, br' -H 'Origin: BASE64(aHR0cHM6Ly9mdXRlbWF4LmFwcA==)' -H 'Connection: keep-alive' -H 'Referer: BASE64(aHR0cHM6Ly9mdXRlbWF4LmFwcA==)' -H 'Sec-Fetch-Dest: empty' -H 'Sec-Fetch-Mode: cors' -H 'Sec-Fetch-Site: cross-site' --retry 5 --fail --compressed "
mkdir -- "$title"
cd "$title"
while [[ $(date -u +%s) -le $duration ]]
do
response=$(eval "$cmd_curl $url_m3u8")
urls=$(echo $response | grep -oP '(https[^ ]*)')
for url in $urls
do
filename=$(basename $url)
if ! [ -f $filename ]; then
touch $filename
eval "$cmd_curl -O $url"
fi
done
done
cat *.ts > ../"$title".ts
cd ../
rm -rf "$title"
Run this script passing m3u8 url, program title and duration in minutes
./hls-download.sh BASE64(aHR0cHM6Ly9uZXdlZGdlLmV1LWNlbnRyYWwtMS5lZGdlLm15Y2RuLmxpdmUvbGl2ZS92aW50ZXF1YXRyb2hyczIvdmludGVxdWF0cm9ocnMyXzIwMDAvaW5kZXgubTN1OA==) program 1
I still looking an answer to my question but I found a better way to solve the issue.
ffmpeg -headers $'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:101.0) Gecko/20100101 Firefox/101.0\r\nAccept: */*\r\nAccept-Language: en-US,en;q=0.5\r\nAccept-Encoding: gzip, deflate, br\r\nOrigin: BASE64(aHR0cHM6Ly9mdXRlbWF4LmFwcA==)\r\nConnection: keep-alive\r\nReferer: BASE64(aHR0cHM6Ly9mdXRlbWF4LmFwcA==)\r\nSec-Fetch-Dest: empty\r\nSec-Fetch-Mode: cors\r\nSec-Fetch-Site: cross-site' -i "BASE64(aHR0cHM6Ly9uZXdlZGdlLmV1LWNlbnRyYWwtMS5lZGdlLm15Y2RuLmxpdmUvbGl2ZS92aW50ZXF1YXRyb2hyczIvdmludGVxdWF0cm9ocnMyXzIwMDAvaW5kZXgubTN1OA==)" -c copy -bsf:a aac_adtstoasc "output.mp4"
I am trying to generate a file using REST API by Header authorization "Bearer Token" and password encryption.
token=$(curl --header "Content-Type: application/json" --header "aesiv: xxxxxxxx" \
--request POST \
--data '{"login":"username","password":"encrypted_pswd"}' \
https://ssssss/api/login)
curl -X GET https://ssssss/api/generate-report -H "Accept: encryptionIntVeD" "Authorization: aesiv {xxxxxxxxxxx}" -H "Accept: application/json" -H "Authorization: Bearer {token}"
File not getting generated, can anyone guide me where I am stuck.
I am referring to the thread: Passing a large text to bash curl url
I have a shell script that looks like:
#!bin/bash
curl --location --request POST 'example.com/abcxyz' \
--header 'Content-Type: application/json' \
--header 'Cookie: my_secret_key' \
--data-raw '{_my_very_long_json_content_here}'
Approach 1: I put everything after the curl, i.e. from --location ... to the end to a test.txt file then
tempfile = test.txt
curl "#$tempfile"
and I get the error curl: (6) Could not resolve host: test.txt
Approach 2: I put everything in a test.sh file:
#!bin/bash
curl --location --request POST 'example.com/abcxyz' --header 'Content-Type: application/json' --header 'Cookie: my_secret_key' --data-raw '#-' <<< 'my_very_long_json'
and I have an error {"error":"invalid character '#' looking for beginning of value","code":3}(base)
What should I do to pass a very long JSON content to curl POST?
Note that, if the JSON content is not too long, I can put everything in a SH file then run
bash test.sh
and it is okay.
--data-raw specifically disables # interpretation. Use --data #- or --data-binary #- to read from stdin.
curl --location --request POST 'example.com/abcxyz' \
--header 'Content-Type: application/json' \
--header 'Cookie: my_secret_key' \
--data #- <<< '{_my_very_long_json_content_here}'
You might find a heredoc more ergonomic than a long string:
curl --location --request POST 'example.com/abcxyz' \
--header 'Content-Type: application/json' \
--header 'Cookie: my_secret_key' \
--data #- <<'DATA'
{
'json': 'here'
}
DATA
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I'm trying to parse content of div element with class "error-icon" and grep shows unrecognized option with multiple attempts.Here is my code.Please help
#!/bin/sh
verifyCard=$1
if [ -z "${verifyCard}" ]; then echo "No argument supplied"; exit 1; fi
response=$(curl 'https://www.isic.org/verify/' -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:80.0) Gecko/20100101 Firefox/80.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Content-Type: application/x-www-form-urlencoded' -H 'Origin: https://www.isic.org' -H 'Connection: keep-alive' -H 'Referer: https://www.isic.org/verify/' -H 'Cookie: PHPSESSID=46plnh6b31e2pusv1do6thfbm7; AWSELB=AF73034F18B38D7DCED6DEDC728D31BA3F3A73F96747FEE7FA7C4F7A74BC9954E5928CBDDD5C053FFB2A37CE37136C4BA008B15163192B34CFA04D35BEC4ED0D0D2913A2FB; AWSELBCORS=AF73034F18B38D7DCED6DEDC728D31BA3F3A73F96747FEE7FA7C4F7A74BC9954E5928CBDDD5C053FFB2A37CE37136C4BA008B15163192B34CFA04D35BEC4ED0D0D2913A2FB; _ga=GA1.2.650910486.1600495658; _gid=GA1.2.731428038.1600495658; _gat=1' -H 'Upgrade-Insecure-Requests: 1' --data-raw 'verify_card_number=${$verifyCard}')
output=$(grep -o '<div class="error-icon">[^<]*' "$response" | grep -o '[^>]*$')
echo "$output"
The line
response=$(curl ...)
puts the output of the curl command in a variable named response.
In your grep command you try to pass the expansion of the variable as an argument.
output=$(grep -o '<div class="error-icon">[^<]*' "$response" | ...)
grep tries to interpret the value as command line arguments which may result in various errors depending on the actual output. In my test I got a message grep: <some html code>: File name too long because it tries to interpret this as a file name argument.
You should save the data in a file and pass this to grep. Adapt the name and location of the temporary file as necessary. Example:
#!/bin/sh
verifyCard=$1
if [ -z "${verifyCard}" ]; then echo "No argument supplied"; exit 1; fi
curl -o response-file 'https://www.isic.org/verify/' -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:80.0) Gecko/20100101 Firefox/80.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Content-Type: application/x-www-form-urlencoded' -H 'Origin: https://www.isic.org' -H 'Connection: keep-alive' -H 'Referer: https://www.isic.org/verify/' -H 'Cookie: PHPSESSID=46plnh6b31e2pusv1do6thfbm7; AWSELB=AF73034F18B38D7DCED6DEDC728D31BA3F3A73F96747FEE7FA7C4F7A74BC9954E5928CBDDD5C053FFB2A37CE37136C4BA008B15163192B34CFA04D35BEC4ED0D0D2913A2FB; AWSELBCORS=AF73034F18B38D7DCED6DEDC728D31BA3F3A73F96747FEE7FA7C4F7A74BC9954E5928CBDDD5C053FFB2A37CE37136C4BA008B15163192B34CFA04D35BEC4ED0D0D2913A2FB; _ga=GA1.2.650910486.1600495658; _gid=GA1.2.731428038.1600495658; _gat=1' -H 'Upgrade-Insecure-Requests: 1' --data-raw 'verify_card_number=${$verifyCard}'
output=$(grep -o '<div class="error-icon">[^<]*' response-file | grep -o '[^>]*$')
rm response-file
echo "$output"
If you use bash or zsh instead of sh, there are ways to substitute some variable value as an input file, see e.g. the answers in using a Bash variable in place of a file as input for an executable.
I'm trying to replace 1.2.3.4 with the contents of variable $wanip in the following script.
wanip="4.3.2.1"
echo $wanip
content=$(curl --insecure -H "X-DNSimple-Token: foo:bar" -H "Accept: application/json" -H "Content-Type: application/json" -X PUT -d "{\"record\": {\"name\": \"foo\",\"content\": \"1.2.3.4\"}}" https://acme.com/records/123)
echo $content
If I literally replace 1.2.3.4 with $wanip*, when I run the script I'm getting a message saying: "message":"Problems parsing JSON".
Try adding a layer of abstraction:
#!/bin/bash
wnip="4.3.2.1"
echo $wanip
command="curl --insecure -H 'X-DNSimple-Token: foo:bar' -H 'Accept: application/json' -H 'Content-Type: application/json' -X PUT -d '{\"record\": {\"name\": \"foo\",\"content\": \"${wnip}\"}}' https://acme.com/records/123"
echo $command
content=$($command)
echo $content
After some hacking, I got this to work. Strange.
wanip=\"4.3.2.1\"
echo $wanip
content=$(curl --insecure -H "X-DNSimple-Token: foo:bar" -H "Accept: application/json" -H "Content-Type: application/json" -X PUT -d "{\"record\": {\"name\": \"foo\",\"content\": $wanip }}" https://acme.com/records/123)
echo $content