Specific output from a two dimensional array - linux

I am running an API call to get my node status in the cluster. the output is in the below format. i am writing a shell script to post the status of each node that is in not ready state. However i am unable to design up logic for the same.
node1 ready
node2 disconnected
node3 ready
Below is my script. Please advice on the changes.
#!/bin/bash
auth_token=$(curl -sk -d '{"username":"","password":""}' https://url/auth/login | jq -r .auth_token)
echo $auth_token
status=$(curl -X GET "https://<url>/nodes" -H "accept: application/json" -H "Authorization: Bearer $auth_token" | jq -r '.[] | .Description.Hostname + " " + .Status.State')
STATUS=($status)
alenght=${#STATUS[#]}
for (( i=0; i<${alenght}; i++));
do
# echo ${org[i]}
if [ ${org[i]} != "ready" ]
then
$dd_status = 3
$hostname = <hostname with status not ready>
curl -X POST https://api.datadoghq.com/api/v1/check_run?api_key=${DD_CLIENT_API_KEY} \
-H "Content-Type: application/json" \
-d #- << EOF
{
"check": "check_name",
"host_name": "$host_name",
"status": "$dd_status",
"tags": [
"environment:test"
]
}
EOF
else
$dd_status = 0
$hostname = <hostname with status ready >
curl -X POST https://api.datadoghq.com/api/v1/check_run?api_key=${DD_CLIENT_API_KEY} \
-H "Content-Type: application/json" \
-d #- << EOF
{
"check": "check_name",
"host_name": "$host_name",
"status": "$dd_status",
"tags": [
"environment:test"
]
}
EOF
fi
done

Try this logic
while read -r name status; do
case $status in
ready) good_code;;
*) fail_code;;
esac
done < <(code_to_generate_log)

Related

GitLab CI How to POST JSON data to a url within a CI job using here-document?

Hoping someone can help. I am experiencing difficulty making a curl POST request with JSON data from a gitlab CI job.
The curl request works fine in a local terminal session, (N.B I did not use double quotes in terminal session). If I do not escape double quotes in the gitlab CI yaml I get the error curl: (3) [globbing] nested brace in column 112
If I escape the double quotes in the GitLab CI job, as shown below I get the error:
curl: (3) [globbing] unmatched brace in column 1
In all cases I get the error /bin/bash: line 134: warning: here-document at line 134 delimited by end-of-file (wanted `EOF')
Is it possible to POST JSON data using here-documents from a GitLab CI job?
.gitlab-ci.yml job extract
release:
image: node:12-stretch-slim
stage: release
before_script:
- apt-get update && apt-get install -y curl git jq
script:
- version=$(git describe --tags | cut -d'-' -f 1 | sed 's/^v*//')
- echo "generating release for version ${version}"
- npm pack
# - >
# url=$(curl
# --header "Private-Token: $API_TOKEN"
# -F "file=#${CI_PROJECT_DIR}/objectdetection-plugin-${version}.tgz" "https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/uploads"
# |
# jq '.url')
- url="http://www.example.com"
- echo "Retrieved url from file upload as ${url}"
- echo "The full url would be ${CI_PROJECT_URL}/${url}"
- >
curl -X POST https://requestbin.io/1f84by61
--header 'Content-Type: application/json; charset=utf-8'
--data-binary #- << EOF
{
\"name\": \"Release v${version}\",
\"tag_name\": \"v${version}\",
\"ref\": \"v${version}\",
\"description\": \"Test\",
\"assets\": {
\"links\": [
{
\"name\": \"objectdetection-plugin-source\",
\"url\": \"CI_PROJECT_URL/${url}\",
\"filepath\": \"${url}\",
\"link_type\": \"other\"
}
]
}
}
EOF
when: manual
only:
- /^release-.*$/
Solved it after reading this
Using |- preserves newlines within the command and does not append a newline at the end of the command string. Used this principle to save the JSON data to a variable and then referenced the variable in the subsequent curl command.
Below I have included the script:
release:
image: node:12-stretch-slim
stage: release
before_script:
- apt-get update && apt-get install -y curl git jq
script:
- git fetch --prune --unshallow
- version=$(git describe --tags | cut -d'-' -f 1 | sed 's/^v*//')
- npm pack
- >
url=$(curl --silent --show-error
--request POST
--header "Private-Token: $API_TOKEN"
-F "file=#${CI_PROJECT_DIR}/objectdetection-plugin-${version}.tgz" "https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/uploads"
|
jq --raw-output --monochrome-output '.url')
- |-
PAYLOAD=$(cat << JSON
{
"name": "Release v$version",
"tag_name": "v$version",
"ref": "v$version",
"description": "$(sed -zE 's/\r\n|\n/\\n/g' < CHANGELOG.md)",
"assets": {
"links": [
{
"name": "objectdetection-plugin-source",
"url": "$CI_PROJECT_URL$url",
"filepath": "$url",
"link_type": "other"
}
]
}
}
JSON
)
- echo "$PAYLOAD"
- >
http_response=$(curl --silent --show-error --write-out "%{http_code}" -o response.txt
--request POST "https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/releases"
--header 'Content-Type: application/json'
--header 'Accept: application/json'
--header "Private-Token: ${API_TOKEN}"
--data-binary "${PAYLOAD}")
- |-
if [ $http_response != "201" ]; then
exit 1
else
echo "Server returned:"
cat response.txt
fi
when: manual
allow_failure: false
only:
- /^release-.*$/
I was trying to send a message to my Google Chat, WebHook URL, and the above solution didn't work.
So I used this instead
'curl --silent --show-error --location --request POST ${GCHAT_DEVOPS_WEBHOOK} --header ''Content-Type: application/json; charset=UTF-8'' --data-raw ''{"text":"Hey <users/all>: A new build has been deployed to *''${APP_ENVIRONMENT}''* via ''${CI_JOB_NAME}'' - *''${APP_BRANCH}''*"}'''

parse error: Invalid numeric literal at line 1, column 8

I am trying to get a token to some site by using curl. It looks like request is done correctly because I have to wait a bit for response however something is during deserialization because I always got error: parse error: Invalid numeric literal at line 1, column 8
This is how script looks like:
TOKEN=$(curl --request POST \
--url 'https://${DOMAIN_NAME}/getmy/token' \
--header 'content-type: application/json' \
--data '{"grant_type":"password", "username":"${USER_EMAIL}",
"password":"${USER_PASSWORD}",
"audience":"https://localhost:8443/my-composite-service", "scope":"openid
email test:read test:write", "client_id": "${CLIENT_ID}",
"client_secret": "${CLIENT_SECRET}"}' -s | jq -r .access_token)
Is it because of jq?
What is more I am sure that env variables are there, even with hard coded values the same error will be thrown.
Thank you in advance
Some hints:
Do not put everything in one line, make it readable instead.
Structure your code with functions.
Do error handling.
Use Bash's debugging functionality.
Do not build JSON with string concatenation, use JQ instead, because only JQ quotes JSON data correctly. A password may contain quoting characters.
An example:
set -eu
set -x
USER_EMAIL="user#domain.org"
USER_PASSWORD="password"
CLIENT_ID="id"
CLIENT_SECRET="secret"
DOMAIN_NAME="domain.org"
data()
{
local template='
{
"grant_type": "password",
"username": $username,
"password": $password,
"audience": "https://localhost:8443/my-composite-service",
"scope": "openid email test:read test:write",
"client_id": $client_id,
"client_secret": $client_secret
}'
if jq <<<null -c \
--arg username "${USER_EMAIL}" \
--arg password "${USER_PASSWORD}" \
--arg client_id "${CLIENT_ID}" \
--arg client_secret "${CLIENT_SECRET}" \
"$template"
then
return
else
printf "ERROR: Can not format request data." >&2
exit 1
fi
}
post()
{
if curl --request POST \
--url 'https://${DOMAIN_NAME}/getmy/token' \
--header 'content-type: application/json' \
--data "$1" \
-s
then
return
else
printf "ERROR: Can not send post request." >&2
exit 1
fi
}
token()
{
if jq -r .access_token
then
return
else
printf "ERROR: Can not parse JSON response." >&2
exit 1
fi
}
TOKEN="$(post "$(data)" | token)"

How to get a list of all forks of a GitHub repo on Linux?

I would like to have a simple, non-interactive way to get a list of forks of a GitHub repo.
For me personally, it has to run on at least Linux.
Using GraphQL (GiHub API v4) from a bash script, using cURL:
#!/bin/bash
# Returns a list of all forks of a github repo.
# See the output of "$0 -h" for more details.
set -e
# initial default values
access_token=""
repo_owner=$USER
repo_name=$(basename $(pwd))
res_file=res.json
function print_help() {
echo "Returns a list of all forks of a github repo."
echo
echo "Usage:"
echo " `basename $0` [OPTIONS]"
echo "Options:"
echo " -h, --help Show this help message"
echo " -o, --owner <string> Name of the GitHub user or organization of the original repo"
echo " -r, --repo <string> Name of the GitHub original repo"
echo " -t, --token <string> GitHub personal access token, required for authentication"
echo " To get one, see: https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line"
}
# read command-line args
POSITIONAL=()
while [[ $# -gt 0 ]]
do
arg="$1"
shift
case "${arg}" in
-h|--help)
shift # past argument
print_help
exit 0
;;
-o|--owner)
repo_owner="$1"
shift # past argument
;;
-r|--repo)
repo_name="$1"
shift # past argument
;;
-t|--token)
access_token="$1"
shift # past argument
;;
*) # non-/unknown option
POSITIONAL+=("${arg}") # save it in an array for later
shift # past argument
;;
esac
done
set -- "${POSITIONAL[#]}" # restore positional parameters
if [ -z "$access_token" ]
then
>&2 echo "WARNING: Access token not specified, though it is required!"
print_help
exit 1
fi
curl \
'https://api.github.com/graphql' \
-H 'Accept-Encoding: gzip, deflate, br' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-H 'Connection: keep-alive' \
-H 'Origin: altair://-' \
-H "Authorization: Bearer $access_token" \
--data-binary '{"query":"query { repository(owner: \"'$repo_owner'\", name: \"'$repo_name'\") { forks(first:100) { edges { node { nameWithOwner } } } } }","variables":{}}' \
--compressed \
> "$res_file"
cat "$res_file" \
| sed -e 's/nameWithOwner/\nXXX/g' \
| grep XXX \
| awk -e 'BEGIN { FS="\""; } { print $3; }'
You need to create an access token.
Sample invocation of the above script:
git-hub-list-forks -o hoijui -r JavaOSC -t 1234567890abcdef1234567890abcdef
NOTE: GitHub limits the maximum amount of forks to fetch to 100 at a time
if you are interested in a js based solution you try GitHub rest API:
/repos/{owner}/{repo}/forks
fetch("https://api.github.com/repos/ONLYOFFICE/CommunityServer/forks?sort=stargazers")
.then(response => response.json())
.then(res=> {
let result = res.map(e=> {
return {
url: e.html_url,
watchers: e.watchers,
starts: e.stargazers_count,
updated_at:e.updated_at
};
});
console.log(result);
// or console.table(result);
});

"local: not in a function"

We want to send information to the channel in Mattermost, but I get this error in the script.
#!/bin/bash
#start
matterSend() {
# Lowercase variable names; declare them local
local endpoint=https://mattermost.ltd/hooks/hash..
local username=$USER
# Pro tip: don't use a variable for the payload if it's effectively static
payload=$(cat <<-__EOF
payload={
"username" : "$username",
"channel" : "Genel_Log",
"text" : "#### ---\\n| Yedekeleme | Drive Gönderim | İşlem *** |\\n|:-----------|:-----------:|-----------------------------------------------:|\\n| ${2} | ${3} | ${1} :white_check_mark: |\\n"
}
__EOF
)
echo "CURL: curl -i -X POST -d $payload $endpoint"
curl -i -X POST -d "$payload" "$endpoint"
}
STRING="Starting.."
matterSend
Result:
fileName.sh: 5: local: not in a function
What is the reason?

POST request containing CSR fails in Bash

I've written a bash script that sends a POST request to a server. The request contains a certificate signing request and the server signs it and returns a certificate.
When I copy and paste the CSR text in the POST's body, then the POST request is successful. But when I read the CSR from a variable, then the POST request fails. I've attached a snippet of the program below.
PROGRAM - Bash
openssl req -new -newkey rsa:2048 -nodes -out cert.csr -keyout priv.key -subj "/C=MyCountry/ST=MyState/L=MyCity/O=MyCompany/OU=MyDept/CN=MyComp"
if [ $? == 0 ]; then
csr=$(<cert.csr)
fi
response=$(curl -o - -s -w "%{http_code}\n" -X POST \
https://xxx.xxx.com/URI-END-POINT \
-H "authorization: $token" \
-H "content-type: application/json" \
-d '{
"digicert": {
"csr": "'$csr'",
"profileName": "pn123",
"signatureHash": "sh123",
"userPrincipalName": "pn123",
"validationScopeId": "vsi123"
},
"IccId": "sim123",
"MacAddress": "mac123"
}')
if [ $?==0 ]; then
status=$(echo $response | tail -c 4)
if [ "$status" == "$http_success" ]; then
echo -e "Request for certificate SUCCESS"
else
echo -e "Request for certificate FAILED with return code $status"
fi
else
echo -e "Request for certificate FAILED"
fi
OUTPUT - Bash
curl: option -----END: is unknown
curl: try 'curl --help' or 'curl --manual' for more information
In the above script, if I replace the line "csr": "'$csr'", with "csr": "----BEGIN CERTIFICATE REQUEST---- XXXXXXX ----END CERTIFICATE REQUEST----", then this will work fine.
Can you help me debug this?
Thanks!
Maybe the string in $csr is being evaluated, like if put in double quotes and the resulting string is something different than expected.
For start, try to see if $csr is same as "$csr".
To post the contents of a file, use jq to generate the JSON blob for you: this will take care of any necessary quoting automatically. The output of jq is pipe directly to curl by using the #- argument for the -d option. (A #-prefixed string indicates the name of a file curl should read from; - is the alias for standard input.)
response=$(jq -n --arg csr "$(<csr)" '{
digicert: {
csr: $csr,
profileName: "pn123",
signatureHash : "sh123",
userPrincipalName: "pn123",
validationScopeId: "vsi123"
},
IccId: "sim123",
MacAddress: "mac123"
}' |
curl -o - -s -w "%{http_code}\n" -X POST \
https://xxx.xxx.com/URI-END-POINT \
-H "authorization: $token" \
-H "content-type: application/json" \
-d #-
)

Resources