Below is my bash script:
#!/bin/bash
message=$1
hostname=$2
appname=$3
severity=$4
data='{"action":"EventsRouter","method":"add_event","data":[{"summary":"'$message'","device":"'$hostname'","message":"message","component":"'$appname'","severity":"'$severity'","evclasskey":"nxlog","evclass":"nxlog","monitor":"localhost"}],"type":"rpc","tid":2}'
echo "Total number of args : $#"
echo "message = $message"
echo "hostname = $hostname"
echo "appname = $appname"
echo "data = $data"
curl -u "Eve:Welcome666" -k "https://myurlcom/zport/dmd/evconsole_router" -d $data -H "Content-Type:application/json"
and my error is:
Total number of args : 4
message = mes
hostname = hos
appname = sev
data = {"action":"EventsRouter","method":"add_event","data":[{"summary":"mes","device":"hos","message":"message","component":"sev","severity":"ap","evclasskey":"nxlog","evclass":"nxlog","monitor":"localhost"}],"type":"rpc","tid":2}
{"uuid": "9b66e70d-44fc-463d-a347-786fe5502c13", "action": "EventsRouter", "result": {"msg": "Failed to create event: NoRouteException: Reply code: 312, Reply text: NO_ROUTE, Exchange:zenoss.zenevents.raw, Routing key: zenoss.zeneventnxlog", "type":"exception", "success": false}, "tid": 2, "type": "rpc", "method": "add_event"}
However, when i curl a data directly, it is working.
curl -u Eve:Welcome666 -k https://myurl.com/zport/dmd/evconsole_router -d '{"action":"EventsRouter", "method":"add_event","data":[{"summary":"test","device":"test","message":"msg","component":"testhost","severity":"5", "evclasskey":"nxlog", "evclass":"/nxlog/perf","monitor":"localhost"}],"type":"rpc","tid":2}' -H "Content-Type:application/json"
{"uuid": "1654584c-5f86-489e-a5e7-35d45e462066", "action": "EventsRouter", "result": {"msg": "Created event", "success": true},"tid": 2, "type": "rpc", "method": "add_event"}
I think you are missing quotes around $data in the script, the correct form is:
curl -u "Eve:Welcome666" -k "https://myurlcom/zport/dmd/evconsole_router" -d "'$data'" -H "Content-Type:application/json"
Related
I need to run a curl command several times, and check the result every time. I used a for loop to check the output from curl, and I want to exit the loop when the correct status is reached.
The very first time the curl command is run, it will show me this output :
{
"name": "organizations/org_name/operations/long_running_operation_ID",
"metadata": {
"#type": "type.googleapis.com/google.cloud.apigee.v1.OperationMetadata",
"operationType": "INSERT",
"targetResourceName": "organizations/org_name",
"state": "IN_PROGRESS"
}
}
It takes time to complete, that's why it says IN_PROGRESS. It took around 30 to 60 sec, and if we run the same curl command after 30 to 60 sec then it will show the output below :
{
"error": {
"code": 409,
"message": "org graceful-path-310004 already associated with another project",
"status": "ALREADY_EXISTS",
"details": [
{
"#type": "type.googleapis.com/google.rpc.RequestInfo",
"requestId": "11430211642328568827"
}
]
}
}
Here is the script I have so far :
test=$(curl -H "Authorization: Bearer $TOKEN" -X POST -H "content-type:application/json" \
-d '{
"name":"'"$ORG_NAME"'",
"displayName":"'"$ORG_DISPLAY_NAME"'",
"description":"'"$ORGANIZATION_DESCRIPTION"'",
"runtimeType":"'"$RUNTIMETYPE"'",
"analyticsRegion":"'"$ANALYTICS_REGION"'"
}' \
"https://apigee.googleapis.com/v1/organizations?parent=projects/$PROJECT_ID" | jq '.error.status')
echo $test
while [ $test = "ALREADY EXISTS" || $test != "IN_PROGRESS" ]
do
echo $test
echo "Organization is creating"
test=$(curl -H "Authorization: Bearer $TOKEN" -X POST -H "content-type:application/json" \
-d '{
"name":"'"$ORG_NAME"'",
"displayName":"'"$ORG_DISPLAY_NAME"'",
"description":"'"$ORGANIZATION_DESCRIPTION"'",
"runtimeType":"'"$RUNTIMETYPE"'",
"analyticsRegion":"'"$ANALYTICS_REGION"'"
}' \
"https://apigee.googleapis.com/v1/organizations?parent=projects/$PROJECT_ID" | jq '.error.status')
done
echo "exit from loop"
To reduce the duplicated code, use functions.
Also, best practice to use jq to generate json -- it will safely handle any embedded quotes in the data:
json() {
jq --arg name "$ORG_NAME" \
--arg displayName "$ORG_DISPLAY_NAME" \
--arg description "$ORGANIZATION_DESCRIPTION" \
--arg runtimeType "$RUNTIMETYPE" \
--arg analyticsRegion "$ANALYTICS_REGION" \
--null-input --compact-output \
'$ARGS.named'
}
errorStatus() {
curl -H "Authorization: Bearer $TOKEN" \
-X POST \
-H "content-type:application/json" \
-d "$(json)" \
"${URL}?parent=projects/$PROJECT_ID" \
| jq -r '.error.status'
}
while true; do
status=$(errorStatus)
[ "$status" = "ALREADY EXISTS" ] && break
done
echo "exit from loop"
The while true; do some code; condition && break; done bit is the bash version of a do-while loop
I am writing a bash script to be run on a linux appliance so I do not have access to install jq, I need to be able to use native Linux tools
curl --location --request POST "https://api-mp.meraki.com/api/v1/networks/12345/switch/stacks/12345/routing/interfaces" \
--header "X-Cisco-Meraki-API-Key: 123key" \
--header "Content-Type: application/json" \
--data '{
"name": "L3 int",
"subnet": "192.168.249.0/24",
"interfaceIp": "192.168.249.2",
"multicastRouting": "disabled",
"vlanId": 140,
"ospfSettings": {
"area": "0",
"cost": 1,
"isPassiveEnabled": true
}
}' | sed -n 's|.*"interfaceID":"\([^"]*\)".*|\1|p' > $newVar
the response from the server returns this:
{
"interfaceId": "12345",
"name": "PA L3 DR Int",
"subnet": "192.168.249.0/24",
"interfaceIp": "192.168.249.2",
"multicastRouting": "disabled",
"vlanId": 140,
"ospfSettings": {
"area": "0",
"cost": 1,
"isPassiveEnabled": true
}
}
I then need to be able to access the interfaceID new variable in a query string paramter
You just got the case wrong (ID vs Id in interfaceId) and forgot about the space after :, but you were close:
$ sed -n 's|.*"interfaceId": *"\([^"]*\)".*|\1|p' file
12345
Using cat file in place of your curl command to show how to save that to a variable in case you don't know:
$ var=$(cat file | sed -n 's|.*"interfaceId": *"\([^"]*\)".*|\1|p')
$ echo "$var"
12345
Could you please try following. Tested successfully in link
https://ideone.com/EsdCQp
your_command |
awk '
/"interfaceId": "/{
gsub(/[^0-9]+|"\,/,"")
print
}
'
I am trying to curl via bash script but unable to pass the value of var1 in curl request and getting error upon execution ...
#!/bin/bash
var1="some test message"
curl 'https://link' \
-H 'Content-Type: application/json' \
-d '
{"msgtype": "text",
"text": {
"content": $var1
}
}'
Your variable $var1 doesn't get expanded by the shell because it is inside single quote '.
You need to use double quote to let bash do the parameter expansion, and escape your json data:
#!/bin/bash
var1="some test message"
curl 'https://link' \
-H 'Content-Type: application/json' \
-d "
{
\"msgtype\": \"text\",
\"text\": {
\"content\": \"$var1\"
}
}"
Or you can use inline document (without the escaping hell, but the command becomes awkward):
#!/bin/bash
var1="some test message"
curl 'https://link' \
-H 'Content-Type: application/json' \
-d "$(cat <<EOT
{
"msgtype": "text",
"text": {
"content": "$var1"
}
}
EOT
)"
In general, don't use parameter expansion to define JSON dynamically like this. There is no guarantee that your template, combined with the contents of the variable, produces well-formed JSON. (This is for the same reasons you don't use string interpolation to create dynamic SQL queries.) Use a tool like jq instead.
curl 'https://link' ... \
-d "$(jq --argjson x "$var" \
-n '{msgtype: "text", text: {content: $x}}')"
Use below script.
https://aaa.com' -H 'Content-Type: application/json' -d '{"msgtype": "text", "text": { "content": '"'$var1'"' }}'
I have multiple elastic indices. The name of the indices are of a specific format. Following are an example of my indices:
abc_2ab99742-94d2-43f8-a582-ce10a0f031dc;
abc_e8241182-1a40-410b-a95d-c883472444f4;
Now I need to reindex all the data in these indices. For this I have written a shell script, which is doing the following thing.
1. Loop for all indices
1.1 create a temporary index like abc_2ab99742-94d2-43f8-a582-ce10a0f031dc_tmp.
1.2 reindix all the data from the original index to temp.
1.3 delete and re-create the original index.
1.4 reindex the data from temp to original index.
1.5 delete the temporary index.
Following is the shell script , I have written for the same.
#!/bin/bash
ES_HOST="localhost"
ES_PORT="9200"
TMP="_tmp"
indices=$(curl -s "http://${ES_HOST}:${ES_PORT}/_cat/indices/abc_*?h=index" | egrep 'abc_[0-9a-f]{8}-([0-9a-f]{4}-){3}[0-9a-f]{8}*')
# do for all abc elastic indices
for index in $indices
do
echo "Reindex process starting for index: $index"
tmp_index=$index${TMP}
output=$(curl -X PUT "http://${ES_HOST}:${ES_PORT}/$tmp_index" -H 'Content-Type: application/json' -d'
{
"settings" : {
"index" : {
"number_of_shards" : 16,
"number_of_replicas" : 1
}
}
}')
echo "Temporary index: $tmp_index created with output: $output"
echo "Starting reindexing elastic data from original index:$index to temporary index:$tmp_index"
output=$(curl -X POST "http://${ES_HOST}:${ES_PORT}/_reindex" -H 'Content-Type: application/json' -d'
{
"source": {
"index": '"$index"'
},
"dest": {
"index": '"$tmp_index"'
}
}
')
echo "Reindexing completed from original index:$index to temporary index:$tmp_index with output: $output"
echo "Deleting $index"
output=$(curl -X DELETE "http://${ES_HOST}:${ES_PORT}/$index")
echo "$index deleted with status: $output"
echo "Creating index: $index"
output=$(curl -X PUT "http://${ES_HOST}:${ES_PORT}/$index" -H 'Content-Type: application/json' -d'
{
"settings" : {
"index" : {
"number_of_shards" : 16,
"number_of_replicas" : 1
}
}
}')
echo "Index: $index creation status: $output"
echo "Starting reindexing elastic data from temporary index:$tmp_index to original index:$index"
output=$(curl -X POST "http://${ES_HOST}:${ES_PORT}/_reindex" -H 'Content-Type: application/json' -d'
{
"source": {
"index": '"$tmp_index"'
},
"dest": {
"index": '"$index"'
}
}
')
echo "Reindexing completed from temporary index:$tmp_index to original index:$index with output: $output"
echo "Deleting $tmp_index"
output=$(curl -X DELETE "http://${ES_HOST}:${ES_PORT}/$tmp_index")
echo "$tmp_index deleted with status: $output"
done
But I am getting exception in the reindex command. Following is the exception
Reindexing completed from original index:abc_58b888be-a90f-e3be-838d-88877aee572c to temporary index:abc_58b888be-a90f-e3be-838d-88877aee572c_tmp with output: {"error":{"root_cause":[{"type":"parsing_exception","reason":"[reindex] failed to parse field [source]","line":4,"col":9}],"type":"parsing_exception","reason":"[reindex] failed to parse field [source]","line":4,"col":9,"caused_by":{"type":"json_parse_exception","reason":"Unrecognized token 'abc_58b888be': was expecting ('true', 'false' or 'null')\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput#39913ba; line: 4, column: 31]"}},"status":400}
Can anybody help me, as I am not very good in shell scripting.
The problem is with your shell script, Check the following part;
output=$(curl -X POST "http://${ES_HOST}:${ES_PORT}/_reindex" -H 'Content-Type: application/json' -d'
{
"source": {
"index": '"$index"'
},
"dest": {
"index": '"$tmp_index"'
}
}
')
Here you are suppose to post a json, but the json is not valid, Change as following, then your script will work:
output=$(curl -XPOST "http://${ES_HOST}:${ES_PORT}/_reindex" -H 'Content-Type: application/json' -d'
{
"source": {
"index": "'"$index"'"
},
"dest": {
"index": "'"$tmp_index"'"
}
}
')
"index": "'"$index"'"
This is creating a valid key value pair , as per json format
Here's the same modified script to reindex elasticsearch indices.
#!/bin/bash
#ES_HOST="localhost"
#ES_PORT="9200"
TMP="_v2"
echo "**** Script Execution Started ****"
echo " "
echo "**** Fetching List of Indices Elasticsearch Reindexing ****"
curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc'
echo " "
sleep 5
indices_list=$(curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc' | awk '{print $3}' | sed -n '1!p')
#indices=$(curl -s "http://${ES_HOST}:${ES_PORT}/_cat/indices/abc_*?h=index" | egrep 'abc_[0-9a-f]{8}-([0-9a-f]{4}-){3}[0-9a-f]{8}*')
# do for all abc elastic indices
count=0
echo "$indices_list"
for index in $indices_list
do
echo " "
echo "**** Index No: $count ****"
echo "**** Present Index we are iterating: ${index} ****"
count=`expr $count + 1`
echo " "
echo " "
echo "Reindex process starting for index: $index"
tmp_index=$index${TMP}
output=$(curl -s -X PUT "http://localhost:9200/$tmp_index" -H 'Content-Type: application/json' -d'
{
"settings" : {
"index" : {
"number_of_shards" : 5,
"number_of_replicas" : 1
}
}
}')
echo " "
echo "Temporary index: $tmp_index created with output: $output"
echo "Starting reindexing elastic data from original index: $index to temporary index: $tmp_index"
output=$(curl -s -X POST "http://localhost:9200/_reindex" -H 'Content-Type: application/json' -d'
{
"source": {
"index": "'$index'"
},
"dest": {
"index": "'$tmp_index'"
}
}
')
echo " "
echo "Reindexing completed from original index: $index to temporary index: $tmp_index with output: $output"
echo " "
echo "Deleting $index"
output=$(curl -s -X DELETE "http://localhost:9200/$index")
echo "$index deleted with status: $output"
echo " "
echo "Creating index: $index"
output=$(curl -s -X PUT "http://localhost:9200/$index" -H 'Content-Type: application/json' -d'
{
"settings" : {
"index" : {
"number_of_shards" : 5,
"number_of_replicas" : 1
}
}
}')
echo " "
echo "Index: $index creation status: $output"
echo " "
echo "Starting reindexing elastic data from temporary index: $tmp_index to original index: $index"
output=$(curl -s -X POST "http://localhost:9200/_reindex" -H 'Content-Type: application/json' -d'
{
"source": {
"index": "'$tmp_index'"
},
"dest": {
"index": "'$index'"
}
}
')
echo " "
echo "Reindexing completed from temporary index:$tmp_index to original index:$index with output: $output"
echo " "
echo "Deleting $tmp_index"
output=$(curl -s -X DELETE "http://localhost:9200/$tmp_index")
echo "$tmp_index deleted with status: $output"
echo " "
done
echo " "
sleep 5
echo "**** Fetching List of Indices After Elasticsearch Reindexing ****"
curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc'
echo " "
sleep 5
echo " "
echo "**** Original indices list ****"
echo "$indices_list"
echo "**** No. of indices in original list: $count ****"
echo " "
count1=0
MIndices_list=$(curl -s -X GET 'http://localhost:9200/_cat/indices/%2A?v=&s=index:desc' | awk '{print $3}' | sed -n '1!p')
echo " "
echo "**** Modified indices list ****"
echo "$MIndices_list"
for j in $MIndices_list
do
count1=`expr $count1 + 1`
done
echo " "
echo "**** No. of indices in modified list: $count1 ****"
echo "**** Script Execution Ended ****"
I found this.
And I wrote this variant:
#!/bin/bash
while read line ; do
headers="$headers -H '$line'"
done < public/headers.txt
echo $headers
curl -X PUT \
$headers \
-d #'public/example.json' \
echo.httpkit.com
In headers.txt I have:
X-PAYPAL-SECURITY-USERID:123
X-PAYPAL-SECURITY-PASSWORD:123
But when I run ./public/curl.sh I am not getting the headers I am sending.
I isolated the issue with an env var:
$ x='-H some:asd'
$ curl $x echo.httpkit.com
=> header was NOT present
$ curl -H 'some:asd' echo.httpkit.com
=> header was present
$ curl -H some:asd echo.httpkit.com
=> header was present
How can I correctly insert a variable in the header section?
Let's ask shellcheck:
In yourscript line 3:
headers="$headers -H '$line'"
^-- SC2089: Quotes/backslashes will be treated literally.
Use an array.
Ok, then let's do that:
#!/bin/bash
while read line ; do
headers=("${headers[#]}" -H "$line")
done < public/headers.txt
echo "${headers[#]}"
curl -X PUT \
"${headers[#]}" \
-d #'public/example.json' \
echo.httpkit.com
Result:
{
"method": "PUT",
"uri": "/",
"path": {
"name": "/",
"query": "",
"params": {}
},
"headers": {
"host": "echo.httpkit.com",
"user-agent": "curl/7.35.0",
"accept": "*/*",
"x-paypal-security-userid": "123", // <----- Yay!!
"x-paypal-security-password": "123",
"content-length": "32",
"content-type": "application/x-www-form-urlencoded"
},
"body": "\"This is text from example.json\"",
"ip": "127.0.0.1",
"powered-by": "http://httpkit.com",
"docs": "http://httpkit.com/echo"
}
If you don't want to put the HTTP headers on a command line (perhaps for security reasons), you can still have curl read them directly from a file.
curl -H #headerfile.txt https://www.google.com/ # requires curl >=7.55.0
If your curl is older than 7.55.0:
Use the option -K/--config <config file>, and put several -H/--header <header> lines in the text file.
For more details, please see my answer in the original article:
https://stackoverflow.com/a/48762561/5201675