I was following this documentation : https://docs.gitlab.com/ee/api/projects.html#upload-a-file
I want to save this response to a variable :
{
"alt": "dk",
"url": "/uploads/66dbcd21ec5d24ed6ea225176098d52b/dk.png",
"full_path": "/namespace1/project1/uploads/66dbcd21ec5d24ed6ea225176098d52b/dk.png",
"markdown": ""
}
The above response is from this curl :
curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
--form "file=#dk.png" "https://gitlab.example.com/api/v4/projects/5/uploads"
I tried :
$json=$(curl --request POST --header "PRIVATE-TOKEN:<my token>" --header "Content-Type:multipart/form-data" --form "file=#./ticket.json" "https://gitlab.com/api/v4/projects/<project id>/uploads")
json=curl --request POST --header "PRIVATE-TOKEN:<my token>" --header "Content-Type:multipart/form-data" --form "file=#./ticket.json" "https://gitlab.com/api/v4/projects/<project-id>/uploads"
Both method doesn't return the json object that i want to save.
I managed to get it work by storing the result to a file like this :
curl --request POST --header "PRIVATE-TOKEN:${TOKEN}" --header
"Content-Type:multipart/form-data" --form "file=#./ticket.json"
"${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/uploads" >> path.json
Related
I'd like to add sleep to this request so as not to stress the server with too many requests at a go. I've tried adding sleep but I don't get the expected behaviour. The help is appreciated.
xargs -I{} curl --location --request POST 'https://g.com' \
--header 'Authorization: Bearer cc' \
--header 'Content-Type: application/json' \
--data-raw '{
"c_ids": [
"{}"
]
}' '; sleep 5m' < ~/recent.txt
Escaping arbitrary strings into valid JSON is a job for jq.
If you don't have a particular reason to define the curl args outside your loop:
while IFS= read -r json; do
curl \
--location --request POST 'https://g.com' \
--header 'Authorization: Bearer cc' \
--header 'Content-Type: application/json' \
--data-raw "$json"
sleep 5m
done < <(jq -Rc '{"c_ids": [ . ]}' recent.txt)
...or if you do:
curl_args=(
--location --request POST 'https://g.com' \
--header 'Authorization: Bearer cc' \
--header 'Content-Type: application/json' \
)
while IFS= read -r json; do
curl "${curl_args[#]}" --data-raw "$json"
sleep 5m
done < <(jq -Rc '{"c_ids": [ . ]}' recent.txt)
Putting the sleep in the xargs is a bit wonky. I advise the following approach which is more likely to supply the desired result.
#!/bin/sh
siteCommon="--location --request POST 'https://g.com' \
--header 'Authorization: Bearer cc' \
--header 'Content-Type: application/json' "
while read -r line
do
eval curl ${siteCommon} --data-raw \'{ \"c_ids\": [ \"${line}\" ] }\'
sleep 5m
done < ~/recent.txt
I am integrating Java application with Skype for Business in Azure using UCWA and here is the list of actions that I performed. When it seemed that everything is working and covered, I am stuck at unexpected place. Probably the solution is something trivial, like adding an addition permission, but I cannot find it. Also, I am sure that this post will help someone who is stuck in earlier stage.
Register application in Azure portal:
Register as Native application
Add required permissions to all Skype for Business Online Permissions
Grant permissions to all users
Get the app id (will use it later as client id)
HTTP Get, btw: (tenant) should be replaced with actual tenant name
Request:
curl -X GET \
http://lyncdiscover.(tenant).onmicrosoft.com/ \
-H 'cache-control: no-cache' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
-H 'postman-token: b45b8fee-852f-4678-3631-3a06727d99fc' \
-F Capture=undefined
Response:
`{
"_links": {
"self": {
"href": "https://webdir0a.online.lync.com/Autodiscover/AutodiscoverService.svc/root?originalDomain=(tenant).onmicrosoft.com"
},
"xframe": {
"href": "https://webdir3a.online.lync.com/Autodiscover/AutodiscoverService.svc/root/xframe"
},
"redirect": {
"href": "https://webdir3a.online.lync.com/Autodiscover/AutodiscoverService.svc/root?originalDomain=(tenant).onmicrosoft.com"
}
}
}`
HTTP Get redirect url
curl -X GET \
'https://webdir3a.online.lync.com/Autodiscover/AutodiscoverService.svc/root?originalDomain=(tenant).onmicrosoft.com' \
-H 'cache-control: no-cache' \
-H 'postman-token: 273cad2b-a9a9-9882-8634-b52f9a9976b5'
{
"_links": {
"self": {
"href": "https://webdir3a.online.lync.com/Autodiscover/AutodiscoverService.svc/root?originalDomain=(tenant).onmicrosoft.com"
},
"user": {
"href": "https://webdir3a.online.lync.com/Autodiscover/AutodiscoverService.svc/root/oauth/user?originalDomain=(tenant).onmicrosoft.com"
},
"xframe": {
"href": "https://webdir3a.online.lync.com/Autodiscover/XFrame/XFrame.html"
}
}
}
Get user url
curl -X GET \
'https://webdir3a.online.lync.com/Autodiscover/AutodiscoverService.svc/root/oauth/user?originalDomain=(tenant).onmicrosoft.com' \
-H 'cache-control: no-cache' \
-H 'postman-token: af9ab0bd-dc6f-b2f3-e7d9-23941aac5537'
response: 401 Unauthorized
Read response http header and extract
`authorization_uri="https://login.windows.net/common/oauth2/authorize"`
Post to authorization url:
client+id = app id from azure portal app registration
resource=00000004-0000-0ff1-ce00-000000000000 (SfB resource id)
curl -X POST \
https://login.windows.net/common/oauth2/token \
-H 'cache-control: no-cache' \
-H 'content-type: application/x-www-form-urlencoded;charset=UTF-8' \
-H 'postman-token: 39902b3f-00c3-e7a8-75d0-6b94f10e07ed' \
-d 'resource=00000004-0000-0ff1-ce00-000000000000&client_id=XXXX-XXXX-XXXX&grant_type=password&username=actualUserName#tenant.com&password=actual_password&scope=openid'
response:
`{
"token_type": "Bearer",
"scope": "Contacts.ReadWrite Conversations.Initiate Conversations.Receive Meetings.ReadWrite User.ReadWrite",
"expires_in": "3599",
"ext_expires_in": "0",
"expires_on": "1518196708",
"not_before": "1518192808",
"resource": "00000004-0000-0ff1-ce00-000000000000",
"access_token": "eyJ0...",
"refresh_token": "AQABA...",
"id_token": "eyJ0e..."
}`
Yes, I am getting actual token and everything seems to be fine, but it is not. When I use this token to get user url again the response now is 403 Forbidden and I am stuck.
`curl -X GET \
'https://webdir3a.online.lync.com/Autodiscover/AutodiscoverService.svc/root/oauth/user?originalDomain=(tenant).onmicrosoft.com' \
-H 'authorization: Bearer eyJ0eXA...' \
-H 'cache-control: no-cache' \
-H 'postman-token: ff0a80bd-5025-5b28-3f1c-cf9205890812'`
response: 403 Forbidden
` <body>
<div id="header">
<h1>Server Error</h1>
</div>
<div id="content">
<div class="content-container">
<fieldset>
<h2>403 - Forbidden: Access is denied.</h2>
<h3>You do not have permission to view this directory or page using the credentials that you supplied.</h3>
</fieldset>
</div>
</div>
</body>`
The mistake is in step #4, parameter resource. Correct request uses user server url as resource parameter:
curl -X POST \
https://login.windows.net/common/oauth2/token \
-H 'cache-control: no-cache' \
-H 'content-type: application/x-www-form-urlencoded;charset=UTF-8' \
-H 'postman-token: 39902b3f-00c3-e7a8-75d0-6b94f10e07ed' \
-d 'resource=https://webdir3a.online.lync.com&client_id=XXXX-XXXX-XXXX&grant_type=password&username=actualUserName#tenant.com&password=actual_password&scope=openid
and then use received token to get the application url from user url.
Once application url is retrieved, a new token request must be posted to get token for application server, in my case that was:
curl -X POST \
https://login.windows.net/common/oauth2/token \
-H 'cache-control: no-cache' \
-H 'content-type: application/x-www-form-urlencoded;charset=UTF-8' \
-H 'postman-token: 39902b3f-00c3-e7a8-75d0-6b94f10e07ed' \
-d 'resource=https://webpoolsn23a14.infra.lync.com&client_id=XXXX-XXXX-XXXX&grant_type=password&username=actualUserName#tenant.com&password=actual_password&scope=openid
This token can finally be used to create an application and other messaging services.
I want to do cURL GET-request. The following URL should be used:
https://iant.toulouse.inra.fr/bacteria/annotation/cgi/rhime.cgi' -H 'Host: iant.toulouse.inra.fr' -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:56.0) Gecko/20100101 Firefox/56.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: de,en-US;q=0.7,en;q=0.3' --compressed -H 'Referer: https://iant.toulouse.inra.fr/bacteria/annotation/cgi/rhime.cgi?__wb_cookie=&__wb_cookie_name=auth.rhime&__wb_cookie_path=/bacteria/annotation/cgi&__wb_session=WB84Qfsf&__wb_main_menu=Genome&__wb_function=$parent' -H 'Content-Type: application/x-www-form-urlencoded' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' --data '__wb_function=PortalExtractSeq&mode=run&species=rhime&fastafile=%2Fwww%2Fbacteria%2Fannotation%2F%2Fsite%2Fprj%2Frhime%2F%2Fdb%2F$ab.genomic&begin=$start&end=$end&strand=$strand
At the end of the URL, I have some words, which I want to design as variables, so depending on the input, the URL is different and I then request another resource.
The end of the URL. $ab, $start, $end and $strand are the variables, all of them are Strings.
...2Frhime%2F%2Fdb%2F$ab.genomic&begin=$start&end=$end&strand=$strand
I came across "urlencode" and I though of storing my URL as one big String in a variable and pass it to URL encode, but I am not sure, how to do it.
I tried this/I am searching for something like this:
#!bin/bash
[...]
cURL="https://iant.toulouse.inra.fr/bacteria/annotation/cgi/rhime.cgi' -H 'Host: iant.toulouse.inra.fr' -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:56.0) Gecko/20100101 Firefox/56.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: de,en-US;q=0.7,en;q=0.3' --compressed -H 'Referer: https://iant.toulouse.inra.fr/bacteria/annotation/cgi/rhime.cgi?__wb_cookie=&__wb_cookie_name=auth.rhime&__wb_cookie_path=/bacteria/annotation/cgi&__wb_session=WB84Qfsf&__wb_main_menu=Genome&__wb_function=$parent' -H 'Content-Type: application/x-www-form-urlencoded' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' --data '__wb_function=PortalExtractSeq&mode=run&species=rhime&fastafile=%2Fwww%2Fbacteria%2Fannotation%2F%2Fsite%2Fprj%2Frhime%2F%2Fdb%2F$ab.genomic&begin=$start&end=$end&strand=$strand"
# storing HTTP response code in variable response. Only if the
# reponse code is OK (200), we move on
response=$(curl -X HEAD -I --header 'Accept:txt/html' "https://iant.toulouse.inra.fr/bacteria/annotation/cgi/rhime.cgi?__wb_cookie=&__wb_cookie_name=auth.rhime&__wb_cookie_path=/bacteria/annotation/cgi&__wb_session=WB8jqwTM&__wb_main_menu=Genome&__wb_function="$location""|head -n1|awk '{print $2}')
echo "$response"
# getting information via curl request
if [ $response = 200 ] ; then
info=$(curl -G "$ (urlencode "$cURL")")
fi
echo $info
For my response-code checkup, the method of directly passing $location seems to work, but with more variables, I get an error (response code 100, whereas I get 200 with the code-checkup)
Do I have a general error in understanding curl/urlencode? What did I miss?
Thanks for you time and effort in advance :)
UPDATE
#!/bin/sh
# handling command-line input
file=$1
ecf=$2
# iterating through file and pulling out
# information for the GET- and POST-request
while read -r line
do
parent=$(echo $line | awk '{print substr($1,2,3)}')
start=$(echo $line | awk '{print substr($2,2,6)}')
end=$(echo $line | awk '{print substr($3,2,6)}')
strand=$(echo $line | awk '{print substr($4,2,1)}')
locus=$(echo $line | awk '{print substr($6,2,8)}')
# depending on $parent, the right insertion for the URL is generated
if [ $parent = "SMc" ] ; then
location="Genome"
ab="SMc"
elif [ $parent = "SMa" ] ; then
location="PrintPsyma"
ab="pSymA"
else [ $parent = "SMb" ]
location="PrintPsymb"
ab="pSymB"
fi
# building variables for curl content request
options=( --compressed)
headers=(
-H 'Host: iant.toulouse.inra.fr'
-H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:56.0) Gecko/20100101 Firefox/56.0'
-H 'Accept: txt/html,application/xhtml+xml,application/xml;1=0.9,*/*;q=0.8'
-H 'Accept-Language: de,en-US;q=0.7,en;q=0.3'
-H 'Referer: https://iant.toulouse.inra.fr/bacteria/annotation/cgi/rhime.cgi?__wb_cookie=&__wb_cookie_name=auth.rhime&__wb_cookie_path=/bacteria/annotation/cgi&__wb_session=WB84Qfsf&__wb_main_menu=Genome&__wb_function=$parent'
-H 'Content-Type: application/x-www-form-urlencoded'
-H 'Connection: keep-alive'
-H 'Upgrade-Insecure-Requests: 1'
-H 'Pragma: no-cache'
-H 'Cache-Control: no-cache'
)
url='https://iant.toulouse.inra.fr/bacteria/annotation/cgi/rhime.cgi'
ab=$(urlencode "${ab}")
start=$(urlencode "${start}")
end=$(urlencode "${end}")
strand=$(urlencode "${strand}")
data="__wb_function=PortalExtractSeq&mode=run&species=rhime&fastafile=%2Fwww%2Fbacteria%2Fannotation%2F%2Fsite%2Fprj%2Frhime%2F%2Fdb%2F$ab.genomic&begin=$start&end=$end&strand=$strand"
# storing HTTP response code in variable response. Only if the
# reponse code is OK (200), we move on
response=$(curl -X HEAD -I --header 'Accept:txt/html' "https://iant.toulouse.inra.fr/bacteria/annotation/cgi/rhime.cgi?__wb_cookie=&__wb_cookie_name=auth.rhime&__wb_cookie_path=/bacteria/annotation/cgi&__wb_session=WB8jqwTM&__wb_main_menu=Genome&__wb_function="$location""|head -n1|awk '{print $2}')
echo "$response"
# getting information via curl request
if [ $response = 200 ] ; then
info=$(curl -G "${options[#]}" "${headers[#]}" --data "${data}" "${url}")
fi
echo $info
done < $file
You need to separate concepts. That string that you put in cURL variable is not a URL, it is URL + set of headers + parameters + one option for compression. They all are different things.
Define them separately like this:
url='https://iant.toulouse.inra.fr/bacteria/annotation/cgi/rhime.cgi'
headers=(
-H 'Host: iant.toulouse.inra.fr'
-H 'User-Agent: ...'
-H 'Accept: ...'
-H 'Accept-Language: ...'
... other headers from your example ...
)
options=(
--compressed
)
data="__wb_function=PortalExtractSeq&mode=run&species=rhime&fastafile=%2Fwww%2Fbacteria%2Fannotation%2F%2Fsite%2Fprj%2Frhime%2F%2Fdb%2F$ab.genomic&begin=$start&end=$end&strand=$strand"
And then run curl in this fashion:
curl -G "${options[#]}" "${headers[#]}" --data "${data}" "${url}"
This will expand to correct curl command.
About urlencode part: You need encode each of $ab, $start, $end and $strand separately. If you insert them in the string and then encode, then all special characters in that string like & and = will be encoded too, and those already encoded ones like %2F in your example will be encoded twice (will become %252F).
To keep the code tidy, you can encode them beforehand:
ab=$(urlencode "${ab}")
start=$(urlencode "${start}")
end=$(urlencode "${end}")
strand=$(urlencode "${strand}")
data="__wb_function=PortalExtractSeq&mode=run&species=rhime&fastafile=%2Fwww%2Fbacteria%2Fannotation%2F%2Fsite%2Fprj%2Frhime%2F%2Fdb%2F$ab.genomic&begin=$start&end=$end&strand=$strand"
... or do it in a cumbersome way:
data="__wb_function=PortalExtractSeq&mode=run&species=rhime&fastafile=%2Fwww%2Fbacteria%2Fannotation%2F%2Fsite%2Fprj%2Frhime%2F%2Fdb%2F$(urlencode "${ab}").genomic&begin=$(urlencode "${start}")&end=$(urlencode "${end}")&strand=$(urlencode "${strand}")"
I hope this helps.
I wanna make a http request as shown in the below curl command:
curl -X PUT \
https://anypoint.mulesoft.com/cloudhub/api/v2/applications/highfiles \
-H 'authorization: Bearer XXX' \
-H 'cache-control: no-cache' \
-H 'content-length: 0' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
-H 'host: anypoint.mulesoft.com' \
-H 'postman-token: XXX' \
-H 'x-anypnt-env-id: XXX' \
-H 'x-anypnt-org-id: XXX' \
-F 'appInfoJson={
"muleVersion": {
"version": "3.8.5"
},
"properties":{"env":"dev"}
}'
I have tried the below request but all vain
files = {'file': open('C:\Users\\highfiles.zip', 'rb')}
appInfoJson1 = {
"muleVersion": {
"version": "3.8.5"
},
"properties": {"env":"dev1"}
}
print dict(appInfoJson=appInfoJson1)
headers = {"X-ANYPNT-ORG-ID": "XXXX",
"X-ANYPNT-ENV-ID": "XXXX",
"Authorization": "Bearer " + access_token,
}
response = requests.put("https://anypoint.mulesoft.com/cloudhub/api/v2/applications/highfiles",
data=dict(appInfoJson=appInfoJson1) , files=files, headers = headers)
How do I give a form-Param values and file in a python http request.
I was doing it wrong.
The change was only w.r.t handling the dict values, shown below:
response = requests.put("https://anypoint.mulesoft.com/cloudhub/api/v2/applications/highfiles",
data=dict(appInfoJson=appInfoJson1.values()) , files=files, headers = headers)
Tried to enter vertexes:
curl -X 'POST' -d '{"vertexLables": [{"name": "event"},{"name": "category"}]}' -H 'content-type: application/json' -H 'authorization: gds-token yyyy https://ibmgraph-alpha.ng.bluemix.net/zzzz/g/schema
{"code":"NotAuthorized","message":""}