Gitlab API Miroring dost not accept address with %20 - gitlab

I am trying to create mirrors on my GitLab via API and I am able to do that just fine. Problem appears when I try to create a mirror to an address that has URL encoded space "%20" in it.
When I call git lab API with following curl:
curl --request POST --data 'url=https://user:password#remotegit.com/Repo%20Mirror%20Test/_git/Repo%20Mirror%20Test' --data "enabled=true" --header "Authorization: Bearer TOKEN" "https://mygit.com/api/v4/projects/171/remote_mirrors"
I get following response:
{"id":171,"enabled":true,"url":null,"update_status":"none","last_update_at":null,"last_update_started_at":null,"last_successful_update_at":null,"last_error":null,"only_protected_branches":false,"keep_divergent_refs":null}
"url":null does not appear when I use any other address without "%20"
When I do this via UI with the same URL it works just fine. I tried to change " for ' to stop any expansion from happening but no luck. In UI I also looked to Developer tools if there is anything different but string is sent in the same format, unchanged.
GitLab server version: 15.2.2
Any ideas what could cause this problem?

All right the whole URL needed to be encoded, you can do that in "https://www.urlencoder.org" or in my case I needed to use it in Bash script so I installed a package "gridsite-clients" which contains "urlencode" which does basically the same thing. Once the address was encoded all worked as expected. So the final request would look like this:
curl --request POST --data 'url=https%3A%2F%2Fuser%3Apassword%40remotegit.com%2FRepo%2520Mirror%2520Test%2F_git%2FRepo%2520Mirror%2520Test' --data "enabled=true" --header "Authorization: Bearer TOKEN" "https://mygit.com/api/v4/projects/171/remote_mirrors"

Related

why don't my authorization header need Bearer?

I am using curl command to fetch the manifest file of the singularity image as below
curl -H "Authorization :"
and it is working fine for me and i am able to fetch the manifest file andi it is working without appending Bearer
but when I append Bearer string in the Authorization header as below
curl -H "Authorization : Bearer "
curl command is failing to fetch the manifest
but actually it should work when we appended Bearer to Authorization header and it should fail when we dont append Bearer to Authorization header right
why is this happening so and where am i going wrong?

CURL Rest API via Proxy

I want to get data from ServiceNow via a CURL in Putty via a Proxyserver (Plan is to implement it to a PySpark script later on) and then save the data onto the server.
My Command looks like this:
curl -x <proxyadress:port> -U proxyuser:proxypassword -u '<apiuser:apipassword>' -d status="message" "https:/apiadress" -H 'Accept: application/json'
I get the error message:
{"error":{"message":"Invalid content-type. Supported request media types for this service are: [application/json, application/xml, text/xml]","detail":null},"status":"failure"}
A few days ago I was able to have the data printed into the log but didn't manage to replicate the command ... what's wrong?
Thanks for your help

How to upload files to SharepointOnline using curl on Linux

I am trying to automate file uploads to SharePoint Online. Problem is I keep getting 401 unauthorized error when trying to upload the files. I created a script to retrieve the token as suggested in curl request to Microsoft Sharepoint API?, but uploads are still failing, even though I do get a positive response when running curl -i -H "Authorization: Bearer $(./get_access_token.sh)" -H "Accept: application/json;odata=verbose" -s "https://YourTenant.sharepoint.com/_api/web". I have afeeling that I am just malforming my curl command. Any suggestion on command format to be using?

POST request - 406 Security Incident Detected

I'm trying to fetch a schedule from a URL by submitting the form values in a POST request. For this purpose, I created a flow in node-red and it worked fine for few weeks. However, now I've started to receive the following security error.
406 Security Incident Detected406 Security Incident DetectedYour request was blocked. Please try again later.XrxWwwzlK-WmpFeEbGejEwAAAEo - www.islamiskaforbundet.se - xxx.xxx.xxx.xxx
Interesting thing is, when I put the parameters in a REST development tool (such as Postman or Open RESTED) and post the request, I get the data without any issues.
Below is the CURL request generated from Postman:
curl -X POST \
https://www.islamiskaforbundet.se/wp-content/plugins/bonetider/Bonetider_Widget.php \
-H 'cache-control: no-cache' \
-H 'content-type: application/x-www-form-urlencoded' \
-H 'postman-token: eb701a0e-406e-4851-653f-2b7818c7d455' \
-d 'ifis_bonetider_widget_city=G%C3%B6teborg%2C%20SE&ifis_bonetider_widget_date=2020-05-14'
Upon execution, it fails in terminal.
One could say that the server may have learned my IP address and started to block all incoming requests, but then why the same request would work fine in the above mentioned tools' UI but not in CURL or my node-red flow?
Is there anything I'm missing? or how to avoid this security error?
The server at the other end has a number of ways to identify the which client is making the request, but the most likely one is that it is looking at the User Agent string from the client making the request.
e.g. you can make the curl command work by adding the Firefox User Agent header:
curl -X POST https://www.islamiskaforbundet.se/wp-content/plugins/bonetider/Bonetider_Widget.php \
-A "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0" \
-H 'cache-control: no-cache' \
-H 'content-type: application/x-www-form-urlencoded' \
-d 'ifis_bonetider_widget_city=G%C3%B6teborg%2C%20SE&ifis_bonetider_widget_date=2020-05-14'
(I also removed the postman token)
Also 406 is probably the wrong error for the server to be returning, a better match for the text of the error would be something like 429

How to find Windows Azure Image IDs?

How can I find the publicly available Image IDs on Windows Azure?
I found this related question - Azure: List OS Images
But, the answer requires Windows+PowerShell while I need a way to get it on Linux or REST/
Use the URL specified here:
http://msdn.microsoft.com/en-us/library/windowsazure/jj157191.aspx
You'll need to provide a client certificate when sending the request.
If you are using curl on Linux, add the --cert to point to a .pem file (you'll need to upload it to the administrator's management certificate as a .cer file first).
Don't forget to add the x-ms-version header for it to work:
-H "x-ms-version: 2013-03-01"
Here is an example of using curl to get the auto-scale information for a cloud service
curl -H "accept: application/json" -H "x-ms-version: 2013-10-01"
--cert azure-cert.pem $AUTOSCALEURL

Resources