I,am building a script to update files on Bitbucket using the rest api.
My problems are:
Running the command using subprocess lib and running the command directly on the command line gives two different results.
If I run the command using the command line, when I inspect my commits on the Bit bucket app I can see a Commit message and a Issue.
If I run the command using the help of the subprocess lib I don't have a commit message and a Issue in the end. The commit message sets itself by default to "edited by bitbucket" and the issue is null.
This is the command:
curl -X PUT -u user:pass -F content=#conanfile_3_f62hu.py -F 'message= test 4' -F branch=develop -F sourceCommitId={} bitbucket_URL".format(latest_commit)
The other problem is that I need to pass a file to the content in order to update it.
If I pass it like above it works. The problem is that I am generating the file content as raw string and creating a temporary file with that content.
And when I pass the file as a variable, it does not get the content of the file.
My code:
content = b'some content'
current_dir = os.getcwd()
temp_file=tempfile.NamedTemporaryFile(suffix=".py",prefix="conanfile", dir=current_dir)
temp_file.name = temp_file.name.split("\\")
temp_file.name = [x for x in temp_file.name if x.startswith("conanfile")][0]
temp_file.name = "#" + temp_file.name
temp_file.write(content)
temp_file.seek(0)
update_file_url = "curl -X PUT -u user:pass -F content={} -F 'message=test 4' -F branch=develop -F sourceCommitId={} bitbucket_url".format(temp_file.name, latest_commit)
subprocess.run(update_file_url)
Basically I'am passing the file like before, just passing the name to the content, but it does not work.
If I print the command everything looks good, so I don't know why the commit message does not get set and the file content as well.
Updated:
I was able to pass the file, My mistake was that I was not passing it like temp_file.name.
But I could not solve the problem of the message.
What I found is that the message will only take the first word. If there is a space and one more word after, it will ignore it.
The space is causing some problem.
I found the solution, if someone found himself with this problem we need to use a \ before the message= .
Example: '-F message=\" Updated with latest dependencies"'
Related
I need to download most recent file based on the file creation time from the remote site using curl. How can I achieve it?
These are files in remote site
user-producer-info-etl-2.0.0-20221213.111513-53-exec.jar
user-producer-info-etl-2.0.0-20221212.111513-53-exec.jar
user-producer-info-etl-2.0.0-20221214.111513-53-exec.jar
user-producer-info-etl-2.0.0-20221215.111513-53-exec.jar
Above user-producer-info-etl-2.0.0-20221215.111513-53-exec.jar is the most recent file that I want to download? How can I achieve it?
Luckily for you, file names contains dates that are alphabetically sortable !
I don't know where you are so I'm guessing you have at least a shell and I propose this bash answer:
First get the last file name
readonly endpoint="https://your-gitlab.local"
# Get the last filename
readonly most_recent_file="$(curl -s "${endpoint}/get_list_uri"|sort|tail -n 1)"
# Download it
curl -LOs curl "${endpoint}/get/${most_recent_file}"
You will obviously need to replace urls accordingly but I'm sure you get the idea
-L : follow HTTP 302 redirects
-O : download file to local dir keeping the name as is
-s : silent don't show network times and stuff
you can also specify another local name with -o <the_most_recent_file>
for more info:
man curl
hth
i run the following function in (git-)bash under windows:
function config_get_container_values() {
local project_name=$1
local container_name=$2
#local container_name="gitea"
echo "###"
buildcmd="jq -r \".containers[]."
echo "$buildcmd"
buildcmd="${buildcmd}${container_name}"
echo "$buildcmd"
buildcmd="${buildcmd}foobar"
echo "$buildcmd"
echo "###"
}
The output of this is the following. Whyever, after using the variable to extend the string, he starts to overwrite $buildcmd. I tried this also with everything in one line as well with the append command (=+). Everytime the same result.
###
jq -r ".containers[].
jq -r ".containers[].gitea
foobar".containers[].gitea
###
The really strange thing is: When i enable the line local container_name="gitea" everything works as expected. The output is:
###
jq -r ".containers[].
jq -r ".containers[].gitea
jq -r ".containers[].giteafoobar
###
When i put this all into a news file, its also works as expected. So i think something goes wrong in the thousands of line before calling this function. Any idea, what could be cause of this behavior?
Regards
Dave
This is not how you should build up the command, DOS line endings aside. Use --arg to pass the name into the filter as a variable. For example,
config_get_container_values() {
local project_name=$1
local container_name=$2
jq -r --arg n "$container_name " '.containers[][$n+"foobar"]'
}
config_get_container foo gitea < some.json
If the function is invoked with
config_get_container_values proj gitea
it produces the "expected" output. If it is invoked with
config_get_container_values proj $'gitea\r'
it produces output that looks like the first output example. $'gitea\r' expands to a string that consists of 'gitea' followed by a Carriage return (CR) character.
One possible cause of the problem is that the container name (gitea) was read from a file that had Windows/DOS line endings (CR-LF). Problems like that are common. See the first question ("Check whether your script or data has DOS style end-of-line characters") in the "Before asking about problematic code" section of the Stack Overflow 'bash' Info page.
My Python 3.7.1 script generates a fasta file called
pRNA.sites.fasta
Within the same script, I call following system command:
cmd = "weblogo -A DNA < pRNA.sites.fasta > OUT.eps"
os.system(cmd)
print(cmd) #for debugging
I am getting the following error message and debugging message on the command line.
Error: Please provide a multiple sequence alignment
weblogo -A DNA < pRNA.sites.fasta > OUT.eps
"OUT.eps" file is generated but it's emtpy. On the other hand, if I run the following 'weblogo' command from the command line, It works just find. I get proper OUT.eps file.
$ weblogo -A DNA<pRNA.sites.fasta>OUT.eps
I am guessing my syntax for os.system call is wrong. Can you tell me what is wrong with it? Thanks.
Never mind. It turned out to be that I was not closing my file, "pRNA.sites.fasta" before I make system call that uses this file.
I have a json file with the structure seen below:
{
url: "https://mysite.com/myjsonfile",
version_number: 69,
}
This json file is accessed from mysite.com/myrootjsonfile
I want to run a load data script to access mysite.com/myrootjsonfile and load the json content from the url field using curl and save the resulting content to local storage.
This is my attempt so far.
curl -o assets/content.json 'https://mysite.com/myrootjsonfile' | grep -Po '(?<="url": ")[^"]*'
unfortunately, instead of saving the content from mysite.com/myjsonfile its saving the content from above: mysite.com/myrootjsonfile. Can anyone point out what i might be doing wrong? Bear in mind in a completely new to curl. Thanks!
It is saving the content from myrootjsonfile because that is what you are telling curl to do - to save that file to the location assets/content.json, and then greping stdin, which is empty. You need to use two curl commands, one to download the root file (and process it to find the URL of the second), and the second to download the actual content you want. You can use command substitution for this:
my_url=$(curl https://mysite.com/myrootjsonfile | grep -Po '(?<=url: )[^,]*')
curl -o assets/content.json "$my_url"
I also changed the grep regex - this one matches a string of non-comma characters which follow after "url: ".
Assuming you wished to save the file to assets/content.json, note that flags are case sensitive.
Use -o instead of -O to redirect the output to assets/content.json.
when I use this to download a file from an ftp server:
wget ftp://blah:blah#ftp.haha.com/"$(date +%Y%m%d -d yesterday)-blah.gz" /myFolder/Documents/"$(date +%Y%m%d -d yesterday)-blah.gz"
It says "20131022-blah.gz saved" (it downloads fine), however I get this:
/myFolder/Documents/20131022-blah.gz: Scheme missing (I believe this error prevents it from saving the file in /myFolder/Documents/).
I have no idea why this is not working.
Save the filename in a variable first:
OUT=$(date +%Y%m%d -d yesterday)-blah.gz
and then use -O switch for output file:
wget ftp://blah:blah#ftp.haha.com/"$OUT" -O /myFolder/Documents/"$OUT"
Without the -O, the output file name looks like a second file/URL to fetch, but it's missing http:// or ftp:// or some other scheme to tell wget how to access it. (Thanks #chepner)
If wget takes time to download a big file then minute will change and your download filename will be different from filename being saved.
In my case I had it working with the npm module http-server.
And discovered that I simply had a leading space before http://.
So this was wrong " http://localhost:8080/archive.zip".
Changed to working solution "http://localhost:8080/archive.zip".
In my case I used in cpanel:
wget https://www.blah.com.br/path/to/cron/whatever