I am attempting to run the following script on a virtual instance I have created on the google cloud:
#!/bin/bash
set -eu
DS=$(date "+%Y-%m-%d" -d "7 days ago")
DE=$(date "+%Y-%m-%d" -d "1 day ago")
account=123
## above specifies last weeks delivery
rm -f cookiejar
curl /dev/null -s -S -L -f -c cookiejar 'https://url.io/auth/authenticate' -d name=usr -d passwd='pwd'
curl -o /dev/null -s -S -L -f -b cookiejar -c cookiejar 'https://adloox.io/auth/adminaccounts' -d account=$account
curl -s -S -L -f -o "report1.xlsx" -J -b cookiejar -c cookiejar "https://url.io/adquality/ajax-adblocking?categoryFw=&platform_id[]=7&id1=All&id2=&id3=All&id4=All&id5=&id11=&date=2019-12-09&date_start=$DS&date_end=$DE&website=&keywords=&zfTablePage=1&zfTableColumn=&zfTableOrder=desc&zfTableQuickSearch=&zfTableItemPerPage=100&zfTableDataTablesMaxRows=2628&zfTableItemPerPage=10000&zfTableExport=xlsx"
curl -s -S -L -f -o "report2.xlsx" -J -b cookiejar -c cookiejar "https://url.io/report/ajax-by-tag2?platform_id[]=7&id1=All&id2=&id3=All&id4=All&id5=&id11=&date=2019-12-09&date_start=$DS&date_end=$DE&website=&zfTablePage=1&zfTableColumn=&zfTableOrder=desc&zfTableQuickSearch=&zfTableItemPerPage=100&zfTableDataTablesMaxRows=10000&zfDetails=true&by_viewability=imps_sivt&device_id[]=all&tag_type_id[]=all&support_id[]=all&by_website=1&zfTableItemPerPage=10000&zfTableExport=xlsx"
Set Up
I have curl installed on my local desktop (running windows 10 with `Ubuntu 18.04.3 LTS)
I have curl installed on my remote (google cloud virtual instance set up with Ubuntu 18.04.3 LTS`)
Both have version curl 7.58.0
Issue
When run on my local desktop there is no issue and the files download. When run on my remote I am able to log in however I receive the following error for the next curl line:
+ curl -o /dev/null -s -S -L -f -b cookiejar -c cookiejar https://url.io/auth/adminaccounts -d account=123
curl: (22) The requested URL returned error: 500 Internal Server Error
Can someone confirm what else I should be looking at here? I would have thought if my linux and curl versions are the same, there wouldn't be an issue. Sorry if there are some other straight forward checks to do. This is my first time setting up a server.
If your virtual instance does not have a public IP address, this might help you out.
#!/bin/bash
set -eu
DS=$(date "+%Y-%m-%d" -d "7 days ago")
DE=$(date "+%Y-%m-%d" -d "1 day ago")
account=123
## above specifies last weeks delivery
rm -f cookiejar
curl /dev/null -s -S -L -f -c cookiejar 'https://url.io/auth/authenticate' -d name=usr -d passwd='pwd'
curl -o /dev/null -s -S -f -b cookiejar -c cookiejar 'https://adloox.io/auth/adminaccounts' -d account=$account
curl -s -S -L -f -o "report1.xlsx" -J -b cookiejar -c cookiejar "https://url.io/adquality/ajax-adblocking?categoryFw=&platform_id[]=7&id1=All&id2=&id3=All&id4=All&id5=&id11=&date=2019-12-09&date_start=$DS&date_end=$DE&website=&keywords=&zfTablePage=1&zfTableColumn=&zfTableOrder=desc&zfTableQuickSearch=&zfTableItemPerPage=100&zfTableDataTablesMaxRows=2628&zfTableItemPerPage=10000&zfTableExport=xlsx"
curl -s -S -L -f -o "report2.xlsx" -J -b cookiejar -c cookiejar "https://url.io/report/ajax-by-tag2?platform_id[]=7&id1=All&id2=&id3=All&id4=All&id5=&id11=&date=2019-12-09&date_start=$DS&date_end=$DE&website=&zfTablePage=1&zfTableColumn=&zfTableOrder=desc&zfTableQuickSearch=&zfTableItemPerPage=100&zfTableDataTablesMaxRows=10000&zfDetails=true&by_viewability=imps_sivt&device_id[]=all&tag_type_id[]=all&support_id[]=all&by_website=1&zfTableItemPerPage=10000&zfTableExport=xlsx"
Related
When I install gitea with docker like that https://docs.gitea.io/en-us/install-with-docker/ always I have to visit http://localhost:3000/install and click "Install gitea" to really finish installation.
After that I have to visit http://localhost:3000/user/sign_up and register a user.
Is there any way to make it by command line instead of manually?
What I looking for is a script that run a clean gitea installation in docker ready to start without have to fill http://localhost:3000/install and http://localhost:3000/user/sign_up
you can make a curl request like this:
curl -H "Content-type: application/x-www-form-urlencoded" -d "db_type=postgres" -d "db_host={host}" -d "db_user={user}" -d "db_passwd={pass}" -d "db_name=gitea" -d "ssl_mode=disable" -d "db_schema=" -d "charset=utf8" -d "db_path=%2Fdata%2Fgitea%2Fgitea.db" -d "app_name={title}" -d "repo_root_path=%2Fdata%2Fgit%2Frepositories" -d "lfs_root_path=%2Fdata%2Fgit%2Flfs" -d "run_user=git" -d "domain={domain}" -d "ssh_port=22" -d "http_port=3000" -d "app_url={appURL}" -d "log_root_path=%2Fdata%2Fgitea%2Flog" -d "smtp_addr=" -d "smtp_port=" -d "smtp_from=" -d "smtp_user=" -d "smtp_passwd=" -d "enable_federated_avatar=on" -d "enable_open_id_sign_in=on" -d "enable_open_id_sign_up=on" -d "enable_captcha=on" -d "require_sign_in_view=on" -d "default_allow_create_organization=on" -d "default_enable_timetracking=on" -d "no_reply_address=noreply.localhost" -d "password_algorithm=pbkdf2" -d "admin_name={username}" -d "admin_passwd={password}" -d "admin_confirm_passwd={password}" -d "admin_email={email}" -X POST http://localhost:3000/
I wrote a script to capture ngrep log daily. But it's not creating file automatically when the date has been changed.
#!/bin/bash
month=$( date +%B )
mkdir -p /home/log/$month
NOW=$(date '+%Y%m%d')
LOGFILE1="/home/log/$month/5110_$NOW.txt"
LOGFILE2="/home/log/$month/5150_$NOW.txt"
LOGFILE3="/home/log/$month/5160_$NOW.txt"
while true
do
ngrep -t -q -d any -W byline port 5110 >> $LOGFILE1 &
ngrep -t -q -d any -W byline port 5150 >> $LOGFILE2 &
ngrep -t -q -d any -W byline port 5160 >> $LOGFILE3
exec bash
sleep 2
done
Please help.
Thanks
You would have to move the variable definitions into the while loop for that to happen.
#!/bin/bash
while true
do
month=$( date +%B )
mkdir -p /home/log/$month
NOW=$(date '+%Y%m%d')
LOGFILE1="/home/log/$month/5110_$NOW.txt"
LOGFILE2="/home/log/$month/5150_$NOW.txt"
LOGFILE3="/home/log/$month/5160_$NOW.txt"
ngrep -t -q -d any -W byline port 5110 >> $LOGFILE1 &
ngrep -t -q -d any -W byline port 5150 >> $LOGFILE2 &
ngrep -t -q -d any -W byline port 5160 >> $LOGFILE3
exec bash
sleep 2
done
I use this curl command to download a setup.exe :
curl -k -u login:Password -O "https://url/to/my/setup.exe"
I want to get the time total :
curl -s -w 'total : %{time_total}\n' -k -u login:Password -O "https://url/to/my/setup.exe"
total : 41.165108
I want to redirect the output ( total : 41.165108 ) in an external file. I try :
curl -o test.txt -s -w 'total : %{time_total}\n' -k -u login:Password -O "https://url/to/my/setup.exe"
curl -s -w 'total : %{time_total}\n' -k -u login:Password -O "https://url/to/my/setup.exe" > test.txt
But it doesn't work...
Someone to show me ?
Thanks !
Just combine the two things you've tried:
curl -s -w 'total : %{time_total}\n' -k -u login:Password \
-O "https://url/to/my/setup.exe" -o setup.exe > total.txt
# ~~~~~~~~~~~~ ~~~~~~~~~~~
setup.exe will contain the downloaded file
total.txt will contain the total
The -o doesn't influence where the output of -w goes.
The -o parameter is in lowercase, and the file to output to, is to be entered after the parameter, an example is below,
curl -K myconfig.txt -o output.txt
I am using two linux machines to simulate some firewall tests... I execute the tests by running nc through ssh on a remote machine... if I spawn the ssh like this, it works...
ssh -i id_dsa -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no \
-p 2224 root#a2-idf-lab nc -s 10.26.216.82 10.195.18.132 \
21 < /var/log/messages
However, if I try to control how much of /var/log/messages with head -c 20 /var/log/messages, the command hangs but I don't understand why...
ssh -i id_dsa -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no \
-p 2224 root#a2-idf-lab nc -s 10.26.216.82 10.195.18.132 \
21 < head -c 20 /var/log/messages
I also tried this with no better success...
ssh -i id_dsa -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no \
-p 2224 root#a2-idf-lab nc -s 10.26.216.82 10.195.18.132 \
21 < (head -c 20 /var/log/messages)
Question: Why does the second command hang, and how can I accomplish what I need?
FYI, these experiments were really in preparation for sending cat /dev/urandom | base64 | head -c 20 - into nc... bonus points if you can give me cli that would work with nc through an ssh session...
< is shell redirection, it redirects the input stream to read from a file, not to execute a command. try:
head -c 20 /var/log/messages | ssh -i id_dsa -o UserKnownHostsFile=/dev/null \
-o StrictHostKeyChecking=no \
-p 2224 root#a2-idf-lab nc -s 10.26.216.82 10.195.18.132 21
this pipes /var/log/messages from the local machine into nc on the remote machine.
if you want to use the /var/log/messages file on the remote machine, use quotes around the command:
ssh -i id_dsa -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no \
-p 2224 root#a2-idf-lab "head -c 20 /var/log/messages |\
nc -s 10.26.216.82 10.195.18.132 21"
Try to use
head -n 20
My guess is the problem is the lack of carriage return at the end.
Ho to insert cookies value in curl? from firebug request headers I can see in the following
Cookie: PHPSESSID=gg792c2ktu6sch6n8q0udd94o0; was=1; uncheck2=1; uncheck3=1; uncheck4=1; uncheck5=0; hd=1; uncheck1=1"
I have tried the following:
curl http://site.com/ -s -L -b cookie.c -c cookie.c -d "was=1; uncheck2=1; uncheck3=1; uncheck4=1; uncheck5=0; hd=1; uncheck1=1" > comic
and the only thing i see in cookie.c is
PHPSESSID=gg792c2ktu6sch6n8q0udd94o0; was=1;
To pass keys/values to cURL cookie, you need the -b switch, not -d.
For the forms -d, the data will be separated by & and not by ; in your curl command.
So :
curl http://site.com/ \
-s \
-L \
-b cookie.c \
-c cookie.c \
-b "was=1; uncheck2=1; uncheck3=1; uncheck4=1; uncheck5=0; hd=1; uncheck1=1"
> comic
If you need to know the names of the forms to be POSTed, you can run the following command :
mech-dump --forms http://site.com/
It comes with libwww-mechanize-perl package with debian or derivated.