I'm trying to use cURL to POST the contents of a file, as if I'd pasted that contents in to an html textarea. That's to say I don't want to upload the file, I just want a post parameter called foo to be filled with text from a file called bar.txt. bar.txt's contents may include newlines, quotes, and so on.
Is this possible?
Thanks.
Edit: I found out how to do it in the end:
curl --data-urlencode "foo#bar.txt" http://example.com/index.php
This will take the contents of the file bar.txt, url encode it, place the resultant string in a parameter called foo in a POST request of http://example.com/index.php.
I can't speak to whether the solutions others have suggested will work or not, but the one above seems like the best way.
You can by doing something like:
$ curl --data "foo:$(cat foo.txt)" http://localhost/yourfile.php
Note that you'll probably want to encode the file, as cacheguard said. To encode it in base64, just modify the previous command like this:
$ curl --data "foo:$(cat foo.txt | base64)" http://localhost/yourfile.php
You should encode/decode the content of your file (for instance by using the base64 command under Linux).
file foo.txt:
8<----------------------------
Hello World
I am a Secure Web Gateway
8<----------------------------
base64 foo.txt | base64 -d
Related
How to view unprocessed versions of files? I want to use it for curl install script, but didn't find what counterpart is in gitlab.
The following schema is used in gitlab: https://gitlab.com/<group>/<repository>/-/raw/<branch>/<file>
You will probably have to use the gitlab api because your request has to be authorized.
Update
As iugo commented a curl request against this url https://gitlab.com/<group>/<repository>/-/raw/<branch>/<file> redirects.
To get the raw file we need to use the endpoint 'Get raw file from repository' from Repository files API:
curl --header "PRIVATE-TOKEN:<token>" "https://<server>/api/v4/projects/<project-id>/repository/files/<path-to-file>?ref=<branch>"
As output we get the the content (base64 encoded) of the requested file and additional metadata. To get the raw content we need to clip the content part of the returned json and decode it.
To clip the content part we could use jq:
jq -r '.content'
--raw-output / -r With this option, if the filter's result is a string then it will be written directly to standard output rather than being
formatted as a JSON string with quotes.
To decode the encoded content we could use base64
base64 -d > file
-d, --decode Decode data.
In summary, the following should be executed:
curl --header "PRIVATE-TOKEN:<token>" "https://<server>/api/v4/projects/<project-id>/repository/files/<path-to-file>?ref=<branch>" | jq -r '.content' | base64 -d > file
So for example, say you have:
curl -d#- http://localhost:8000
what does each part of -d#- mean? This type of thing is difficult to google and I couldn't find any info about it in the docs or curl --help.
Thanks guys.
The -d#- in this command line is explained as this.
-d is the Data flag defining what is sent in the POST request.
# means send the file after this character as the body of the POST request.
- means stdin.
So -d#- means to read from stdin to send to the URL as the body of the POST request.
I have a json file with the structure seen below:
{
url: "https://mysite.com/myjsonfile",
version_number: 69,
}
This json file is accessed from mysite.com/myrootjsonfile
I want to run a load data script to access mysite.com/myrootjsonfile and load the json content from the url field using curl and save the resulting content to local storage.
This is my attempt so far.
curl -o assets/content.json 'https://mysite.com/myrootjsonfile' | grep -Po '(?<="url": ")[^"]*'
unfortunately, instead of saving the content from mysite.com/myjsonfile its saving the content from above: mysite.com/myrootjsonfile. Can anyone point out what i might be doing wrong? Bear in mind in a completely new to curl. Thanks!
It is saving the content from myrootjsonfile because that is what you are telling curl to do - to save that file to the location assets/content.json, and then greping stdin, which is empty. You need to use two curl commands, one to download the root file (and process it to find the URL of the second), and the second to download the actual content you want. You can use command substitution for this:
my_url=$(curl https://mysite.com/myrootjsonfile | grep -Po '(?<=url: )[^,]*')
curl -o assets/content.json "$my_url"
I also changed the grep regex - this one matches a string of non-comma characters which follow after "url: ".
Assuming you wished to save the file to assets/content.json, note that flags are case sensitive.
Use -o instead of -O to redirect the output to assets/content.json.
I have file that ends by -comps.xml and has the following form:
http://some/url/<sha256sum>-<2 chars>-x86_64-comps.xml
sha256sum is alphanumeric string of 65 length.
For example:
http://some/url/0dae8d32824acd9dbdf7ed72f628152dd00b85e4bd802e6b46e4d7b78c1042a3-c6-x86_64-comps.xml
How I can download this file using curl?
I've found solution using wget:
wget --recursive --level=1 --no-parent --no-directories --accept '*-comps.xml' --directory-prefix=. http://some/url
Assuming that you already know the filename, to download the contents of the file, then simply use
curl -O http://some/url/0dae8d32824acd9dbdf7ed72f628152dd00b85e4bd802e6b46e4d7b78c1042a3-c6-x86_64-comps.xml
If you are looking to somehow predetermine the file name based on an SHA256 of the file's contents, then you will need to either already have access to these contents to be able to determine the SHA256 part of the URL, or to have access to an alternative source for this information.
I'm trying to POST to the HTTP gateway of an SMS provider (Sybase 365) using CURL from a Linux shell script.
I need to pass the following data (note the [ ] and LF characters)
[MSISDN]
List=+12345678
[MESSAGE]
Text=Hello
[END]
If I submit a file using the -F parameter, CURL removes the LF e.g.
curl -F #myfile "http://www.sybase.com/..."
results in this at the server (which is rejected)
[MSISDN]List=+12345678[MESSAGE]Text=Hello[END]
Is there anything I can do to avoid this or do I need an alternative tool?
I'm using a file containing my data for testing but I'd like to avoid that in practice and POST directly from the script.
Try using --data-binary instead of -d(ata-ascii).
From the manual:
--data-binary (HTTP) This posts data in a similar manner as --data-ascii does, although when using this option the entire context of the posted data is kept as-is.
If you want to post a binary file without the strip-newlines feature of the --data-ascii option, this is for you. If this option is used several times, the ones following the first will append data.
ETA: oops, I should read the question more closely. You're using -F, not -d. But --data-binary may be still be worth a shot.
Probably a silly thought, but I don't suppose it actually requires CRLF instead of just LF?
Alternatively, have you tried using the --data-binary option instead of -F?
I've got this working using -d
request=`printf "[MSISDN]\nList=$number\n[MESSAGE]\nText=$message\n[END]\n"`
response=`curl -s -u $username:$password -d "$request" http://www.sybase.com/...`
Curiously, if I use -d #myfile (where myfile contains LF separated text), it doesn't work.
I also tried --data-binary without success.
curl "url" --data-binary #myfile
posts new lines in the data [tested on curl 7.12.1]