I'm finding a method to post a multiline comment through environment variable with curl and GitHub api.
In GitHub actions, I used npm run unit-test-ci-coverage >> $GITHUB_STEP_SUMMARY to redirect the result to $GITHUB_STEP_SUMMARY. The result would be like these
> my-repo#1.0.0 unit-test-ci-coverage
> node --inspect ./node_modules/.bin/jest --coverage --config ./jest.config.js
---------------|---------|----------|---------|---------|---------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
---------------|---------|----------|---------|---------|---------------------
All files | 80 | 80 | 80 | 80 |
Then I used environment variable $RES to store the part that I want to add to the comment.
echo "RES<<EOF" >> $GITHUB_ENV
sed -n 5,7p $GITHUB_STEP_SUMMARY >> $GITHUB_ENV
echo "EOF" >> $GITHUB_ENV
$RES could get the part (File, % Stmts...) successfully. I wanted to create the comment containing the step summary url and the result, so $SUMMARY_URL is the url, and $RES is the result.
However, I used the following commands
curl \
-X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
https://api.github.com/repos/$REPO/issues/$PR_NUMBER/comments \
-d '{"body":"## Result\n '$SUMMARY_URL' \n '$RES'"}'
It showed
curl: option ---------------|---------|----------|---------|---------|---------------------: is unknown
It seemed that curl didn't get the correct values through $RES. How to pass it to curl?
Related
bash get-token.sh | grep x-subject-token | awk '{print $2}' >> token.txt
token=`cat token.txt`
echo $token this is true
data=`curl -s -H "X-Auth-Token: ${token}" -X GET "https://url"`
echo $token = correct token but $data is invalid.
When I write the token in curl and run it manually I get the correct data.
How can I use this token in my script?
I've found this bash script to check status of URLs from text file and print the destination URL when having redirections :
#!/bin/bash
while read url
do
dt=$(date '+%H:%M:%S');
urlstatus=$(curl -kH 'Cache-Control: no-cache' -o /dev/null --silent --head --write-out '%{http_code} %{redirect_url}' "$url" )
echo "$url $urlstatus $dt" >> urlstatus.txt
done < $1
I'm not that good in bash : I'd like to add - for each url - the value of its Robots meta tag (if is exists)
Actually I'd really suggest a DOM parser (e.g. Nokogiri, hxselect, etc.),
but you can do this for instance (Handles lines starting with <meta and "extracts" the value of the robots' attribute content):
curl -s "$url" | sed -n '/\<meta/s/\<meta[[:space:]][[:space:]]*name="*robots"*[[:space:]][[:space:]]*content="*\([^"]*\)"*\>/\1/p'
This will print the value of the attribute or the empty string if not available.
Do you need a pure Bash solution? Or do you have sed?
You can add a line to extract the meta header for robots from the source code of the page and modify the line with echo to show its value:
#!/bin/bash
while read url
do
dt=$(date '+%H:%M:%S');
urlstatus=$(curl -kH 'Cache-Control: no-cache' -o /dev/null --silent --head --write-out '%{http_code} %{redirect_url}' "$url" )
metarobotsheader=$(curl -kH 'Cache-Control: no-cache' --silent "$url" | grep -P -i "<meta.+robots" )
echo "$url $urlstatus $dt $metarobotsheader" >> urlstatus.txt
done < $1
This example records the original line with the meta header for robots.
If you want to put a mark "-" when the page has no meta header for robots, you can change the metarobotsheader line, and put this one:
metarobotsheader=$(curl -kH 'Cache-Control: no-cache' --silent "$url" | grep -P -i "<meta.+robots" || echo "-")
If you want to extract the exact value of the attribute, you can change that line:
metarobotsheader="$(curl -kH 'Cache-Control: no-cache' --silent "$url" | grep -P -i "<meta.+robots" | perl -e '$line = <STDIN>; if ( $line =~ m#content=[\x27"]?(\w+)[\x27"]?#i) { print "$1"; } else {print "no_meta_robots";}')"
When the URL doesn't contain any meta header for robots, it will show no_meta_robots.
I've created a cURL bash script in which I want to save the response body into a file called output.log, but when I open the file output.log it looks like this:
Here is my bash script:
#!/bin/bash
SECRET_KEY='helloWorld'
FILE_NAME='sma.txt'
function save_log()
{
printf '%s\n' \
"Header Code : $1" \
"Executed at : $(date)" \
"Response Body : $2" \
'==========================================================\n' > output.log
}
while IFS= read -r line;
do
HTTP_RESPONSE=$(curl -I -L -s -w "HTTPSTATUS:%{http_code}\\n" -H "X-Gitlab-Event: Push Hook" -H 'X-Gitlab-Token: '$SECRET_KEY --insecure $line 2>&1)
HTTP_STATUS=$(echo $HTTP_RESPONSE | tr -d '\n' | sed -e 's/.*HTTPSTATUS://')
save_log $HTTP_STATUS $HTTP_RESPONSE
done < $FILE_NAME
Can anyone help me get my desired output in my output.log?
From the Curl documentation:
-I, --head Show document info only
Removing the -I or replace it with -i should solve your problem
I'm new to writing scripts in bash and I am having an issue with performing a while read loop when trying to access data from the star wars api. My problem is that I am trying to get the name of all the characters and the name of all their associated starships (can be more than one). My below code will get the name of the character, then get the url to be passed to the spacecraft and retrieve the name of the spacecraft.The issue is when I'm trying to put this all together the output seems to overwrite what person belongs to what spacecraft rather than assigning each person to that spacecraft. I would expect to see taking Luke Skywalker as the example:
Luke Skywalker X-Wing
Luke Skywalker Imperial Shuttle
But the output I am getting is blank for Luke Skywalker as the X-Wing seems to attach to another character as the code progresses through all people associated with it. Any guidance would be greatly appreciated.
Bash Shell Script:
#!/usr/bin/env bash
url='https://swapi.co/api'
n=1
while true
do
response=$( curl -sL -H 'Accept: application/json' ${url}/people
page=${n} )
if [[ ${response} =~ .*detail.*Not.* ]]; then
break
fi
n=$((n + 1))
name=$( echo ${response} | jq -r '.results[] | "\(.name)"' | tr -d '"')
echo ${response} | jq -r '.results[] | { starships: .starships }' |
egrep /starships/ |\
sed -e 's/^[ \t]*//' | tr -d '"' | tr -d ',' |\
while read shipurl
do
curl -s ${shipurl} | jq '.name' | tr -d '"'
done |\
while read shipname
do
echo "${name}" " ${shipname}"
done
done
Thanks!
oh boy! This should be done using python... but here, I gave it a try. Not a hardcore Star Wars fan, so don't know if this turns out ok, but this will give you some more idea...
Note, I have disabled while true for my test case. I am sure some of those jq related stuff can be cleaned up more. I use pilots for each of these starships and match according to that because each character name has more than one matched starship.
$ cat crazystarwars.sh
#!/usr/bin/env bash
url='https://swapi.co/api'
n=1
#while true
#do
response=$(curl -sL -H 'Accept: application/json' "${url}/people?page=${n}")
if [[ ${response} =~ .*"detail".*"Not".* ]]; then
break
fi
readarray -t name < <( echo "${response}" | jq -r '.results[] | "\(.name)"')
readarray -t shipurls < <(echo ${response} | jq -r '.results[] | { starships: .starships }' | egrep /starships/ | sed -e 's/^[ \t]*//' | tr -d '"' | tr -d ',')
for p in "${shipurls[#]}"
do
shipname=$(curl -s ${p} | jq '.name' | tr -d '"')
readarray -t _pilots < <(curl -s ${p} | jq -r '.pilots[]' | xargs -I {} sh -c 'curl -s {}' | jq '.name')
for _name in "${name[#]}"
do
if [[ "${_pilots[#]}" =~ "${_name}" ]];
echo "${_name} ${shipname}"
break
fi
done
done
# ((n++))
#done
Output is something like (not entirely sure if its the right one):
$ ./crazystarwars.sh
Luke Skywalker X-wing
Luke Skywalker Imperial shuttle
Darth Vader TIE Advanced x1
Luke Skywalker X-wing
Obi-Wan Kenobi Jedi starfighter
Obi-Wan Kenobi Trade Federation cruiser
Obi-Wan Kenobi Naboo star skiff
Obi-Wan Kenobi Jedi Interceptor
Obi-Wan Kenobi Belbullab-22 starfighter
I have this command that fetches 2 files, performs some formatting and outputs the result.
curl https://www.cloudflare.com/ips-v{4,6} |
sed 's/^/set_real_ip_from /;s/$/\;/' \ > /home/user/output.txt
It works just fine, the question is how can I add a line break between file 1 and file 2 (preserving the changes to formatting)?
Like this :
{
curl -s https://www.cloudflare.com/ips-v4
printf '\n\n'
curl -s https://www.cloudflare.com/ips-v6
} | sed -r 's/^(.+)$/set_real_ip_from \1;/g' > /home/user/output.txt