I have a web server that basically accepts input for a knife ec2 create server command.
In my knife command, I pass in
-j '{"branch":"clone"}'
From documentation I thought I would be able to do this in my recipes that are being loaded:
##branch = node['branch']
...
git clone git#github.com/blah -b #{##branch}
However, the git clones are failing because ##branch doesn't return. Or atleast the chef logs say begin output of result:
git clone git#github.com/blah -b
~Git error showing options and appropriate input~
How do I load in my passed JSON attribute? What is the correct way?
Edit:
I manually put in the knife command, and the JSON Attributes seem to load using Ruby hash instead of JSON
JSON Attributes: {"branch"=>"Air"}
Last Edit:
My command:
sudo knife ec2 server create -x ubuntu -i ~/.ssh/key.pem -I ami-0eca6e67 -d ubuntu-12.04 -j '{ "branch" : "clone" }' -Z us-east-1a -s subnet-6827ec00 -f c1.medium -g sg-bc9d86d0 -r 'role[role]'
Answer below.
So it seems if I use a bootstrap template that already exists, -d ubuntu-12.04, knife ec2 does NOT load -j json-attributes.
Get rid of the -d option. Json loads appropriately.
Related
First: I have searched the forum and also went through documentation, but still cannot get it right.
So, I have a docker command I want to run on a remote server, from my bash script. I want to pass an environment variable – on the local machine running the script – to the remote server. Furthermore, I need a response from the remote command.
Here is what I actually am trying to do and what I need: the script is a tiny wrapper around our Traefik/Docker/Elixir/Phoenix app setup to be able to connect easily to the running Elixir application, inside the Erlang observer. With the script, the steps would be:
ssh into the remote machine
docker ps to see all running containers, since in our blue/green deploy the active one changes name
docker exec into the correct container
execute a command inside the docker container to connect to the running Elixir application
The command I am using now is:
CONTAINER=$(ssh -q $USER#$IP 'sudo docker ps --format "{{.Names}}" | grep ""$APP_NAME"" | head -n 1')
The main problem is the part with the grep and the ENV var... It is empty, and does not get replaced. It makes sence, since that var does not exist on the remote machine, it does on my local machine. I tried single quotes, $(), ... Either it just does not work, or the solutions I find online execute the command but then I have no way of getting the container name, which I need for the subsequent command:
ssh -o 'RequestTTY force' $USER#$IP "sudo docker exec -i -t $CONTAINER /bin/bash -c './bin/app remote'"
Thanks for your input!
First, are you sure you need to call sudo docker stop? as stopping the containers did not seem to be part of the workflow you mentioned. [edit: not applicable anymore]
Basically, you use a double-double-quote, grep ""$APP_NAME"", but it seems this variable is not substituted (as the whole command 'sudo docker ps …' is singled-quoted); according to your question, this variable is available locally, but not on the remote machine, so you may try writing:
CONTAINER=$(ssh -q $USER#$IP 'f() { sudo docker ps --format "{{.Names}}" | grep "$1" | head -n 1; }; f "'"$APP_NAME"'"')
You can try this single command :
ssh -t $USER#$IP "docker exec -it \$(docker ps -a -q --filter Name=/$APP_NAME) bash -c './bin/app remote'"
You will need to redirect the command with the local environmental variable (APP_NAME) into the ssh command using <<< and so:
CONTAINER=$(ssh -q $USER#$IP <<< 'sudo docker ps --format "{{.Names}}" | grep "$APP_NAME" | head -n 1 | xargs -I{} sudo docker stop {}')
I'm trying to set up a gitlab CI/CD.
1 of the script I use is :
sshpass -p $PRIVATE_KEY ssh -p $PORT -o StrictHostKeyChecking=no $USER#$SERVER01 "cd /var/www/html/app && export HISTIGNORE='*sudo -S*' && echo "$PRIVATE_KEY" | ( sudo -S -k git fetch && sudo -S -k git pull )"
as you can see I'm trying to update the application in my server.
FYI, I have already set up the variables in the gitlab CI/CD settings page.
But, when the job runs, it always returns this error message :
Conflicting password source
Usage: sshpass [-f|-d|-p|-e] [-hV] command parameters
-f filename Take password to use from file
-d number Use number as file descriptor for getting password
-p password Provide password as argument (security unwise)
-e Password is passed as env-var "SSHPASS"
With no parameters - password will be taken from stdin
-P prompt Which string should sshpass search for to detect a password prompt
-v Be verbose about what you're doing
-h Show help (this screen)
-V Print version information
At most one of -f, -d, -p or -e should be used
I have already googled around, but found no clue.
Any clue would be much appreciated.
I finally found the solution.
So the above command worked for master branch, but it didn't work for develop, knowing this pattern, I checked the variables settings, then found out that I turned on the Protect variable flag, the flag says Export variable to pipelines running on protected branches and tags only.
Since develop is not a protected branch, I was thinking that the variable values were not passed into the pipeline.
So, I unchecked this flag for all variables and finally got it working.
I want to download an image accessible from this link: https://www.python.org/static/apple-touch-icon-144x144-precomposed.png into my local system. Now, I'm aware that the curl command can be used to download remote files through the terminal. So, I entered the following in my terminal in order to download the image into my local system:
curl https://www.python.org/static/apple-touch-icon-144x144-precomposed.png
However, this doesn't seem to work, so obviously there is some other way to download images from the Internet using curl. What is the correct way to download images using this command?
curl without any options will perform a GET request. It will simply return the data from the URI specified. Not retrieve the file itself to your local machine.
When you do,
$ curl https://www.python.org/static/apple-touch-icon-144x144-precomposed.png
You will receive binary data:
|�>�$! <R�HP#T*�Pm�Z��jU֖��ZP+UAUQ#�
��{X\� K���>0c�yF[i�}4�!�V̧�H_�)nO#�;I��vg^_ ��-Hm$$N0.
���%Y[�L�U3�_^9��P�T�0'u8�l�4 ...
In order to save this, you can use:
$ curl https://www.python.org/static/apple-touch-icon-144x144-precomposed.png > image.png
to store that raw image data inside of a file.
An easier way though, is just to use wget.
$ wget https://www.python.org/static/apple-touch-icon-144x144-precomposed.png
$ ls
.
..
apple-touch-icon-144x144-precomposed.png
For those who don't have nor want to install wget, curl -O (capital "o", not a zero) will do the same thing as wget. E.g. my old netbook doesn't have wget, and is a 2.68 MB install that I don't need.
curl -O https://www.python.org/static/apple-touch-icon-144x144-precomposed.png
If you want to keep the original name — use uppercase -O
curl -O https://www.python.org/static/apple-touch-icon-144x144-precomposed.png
If you want to save remote file with a different name — use lowercase -o
curl -o myPic.png https://www.python.org/static/apple-touch-icon-144x144-precomposed.png
Create a new file called files.txt and paste the URLs one per line. Then run the following command.
xargs -n 1 curl -O < files.txt
source: https://www.abeautifulsite.net/downloading-a-list-of-urls-automatically
For ones who got permission denied for saving operation, here is the command that worked for me:
$ curl https://www.python.org/static/apple-touch-icon-144x144-precomposed.png --output py.png
try this
$ curl https://www.python.org/static/apple-touch-icon-144x144-precomposed.png > precomposed.png
I'm learning shell scripting! for the same I've tried downloading the facebook page using curl on ubuntu terminal.
t.sh content
vi#vi-Dell-7537(Desktop) $ cat t.sh
curlCmd="curl \"https://www.facebook.com/vivekkumar27june88\""
echo $curlCmd
($curlCmd) > ~/Desktop/fb.html
Getting error when running the script as
vi#vi-Dell-7537(Desktop) $ ./t.sh
curl "https://www.facebook.com/vivekkumar27june88"
curl: (1) Protocol "https not supported or disabled in libcurl
But if the run the command directly then it is working fine.
vi#vi-Dell-7537(Desktop) $ curl "https://www.facebook.com/vivekkumar27june88"
<!DOCTYPE html>
<html lang="hi" id="facebook" class="no_js">
<head><meta chars.....
I will appreciate if anyone let me know the mistake I am doing in the script.
I've verified that curl library have ssl enabled.
A command embedded within a parenthesis runs as a sub-shell so your environment variables will be missing.
Try eval:
curlCmd="curl 'https://www.facebook.com/vivekkumar27june88' > ~/Desktop/fb.html"
eval $curlCmd
Create your script t.sh as this single line only:
curl -k "https://www.facebook.com/vivekkumar27june88" -o ~/Desktop/fb.html
As per man curl:
-k, --insecure
(SSL) This option explicitly allows curl to perform "insecure" SSL connections transfers.
All SSL connections are attempted to be made secure by using the CA certificate bundle
installed by default. This makes all connections considered "insecure" fail unless -k,
--insecure is used.
-o file
Store output in the given filename.
As #Chepner said, go read BashFAQ #50: I'm trying to put a command in a variable, but the complex cases always fail!. To summarize, how you should do things like this depends on what your goal is.
If you don't need to store the command, don't! Storing commands is difficult to get right, so if you don't need to, just skip that mess and execute it directly:
curl "https://www.facebook.com/vivekkumar27june88" > ~/Desktop/fb.html
If you want to hide the details of the command, or are going to use it a lot and don't want to write it out each time, use a function:
curlCmd() {
curl "https://www.facebook.com/vivekkumar27june88"
}
curlCmd > ~/Desktop/fb.html
If you need to build the command piece-by-piece, use an array instead of a plain string variable:
curlCmd=(curl "https://www.facebook.com/vivekkumar27june88")
for header in "${extraHeaders[#]}"; do
curlCmd+=(-H "$header") # Add header options to the command
done
if [[ "$useSilentMode" = true ]]; then
curlCmd+=(-s)
fi
"${curlCmd[#]}" > ~/Desktop/fb.html # This is the standard idiom to expand an array
If you want to print the command, the best way to do it is usually with set -x:
set -x
curl "https://www.facebook.com/vivekkumar27june88" > ~/Desktop/fb.html
set +x
...but you can also do something similar with the array approach if you need to:
printf "%q " "${curlCmd[#]}" # Print the array, quoting as needed
printf "\n"
"${curlCmd[#]}" > ~/Desktop/fb.html
Install following softwares in ubuntu 14.04
sudo apt-get install php5-curl
sudo apt-get install curl
then run sudo service apache2 restart
check your phpinfo() is enable with curl "cURL support: enabled"
Then check your command in shell script
RESULT=curl -L "http://sitename.com/dashboard/?show=api&action=queue_proc&key=$JOBID" 2>/dev/null
echo $RESULT
You will get response;
Thanks you.
I know this is more like a serverfault question than a stackoverflow question, but since serverfault isn't up yet, here I go:
I'm supposed to move an application from one redhat server to another, and without very good knowledge of the internal workings of the application, how would I move the OpenLDAP database from the one machine to the other, with schemas and all.
What files would I need to copy over? I believe the setup is pretty standard.
The problem with SourceRebels' answer is that slapcat(8) does not guarantee that the data is ordered for ldapadd(1)/ldapmodify(1).
From man slapcat (from OpenLDAP 2.3) :
The LDIF generated by this tool is suitable for use with slapadd(8).
As the entries are in database order, not superior first order, they
cannot be loaded with ldapadd(1) without first being reordered.
(FYI: In OpenLDAP 2.4 that section was rephrased and expanded.)
Plus using a tool that uses the backend files to dump the database and then using a tool that loads the ldif through the ldap protocol is not very consistent.
I'd suggest to use a combination of slapcat(8)/slapadd(8) OR ldapsearch(1)/ldapmodify(1). My preference would go to the latter as it does not need shell access to the ldap server or moving files around.
For example, dump database from a master server under dc=master,dc=com and load it in a backup server
$ ldapsearch -Wx -D "cn=admin_master,dc=master,dc=com" -b "dc=master,dc=com" -H ldap://my.master.host -LLL > ldap_dump-20100525-1.ldif
$ ldapadd -Wx -D "cn=admin_backup,dc=backup,dc=com" -H ldap://my.backup.host -f ldap_dump-20100525-1.ldif
The -W flag above prompts for ldap admin_master password however since we are redirecting output to a file you wont see the prompt - just an empty line. Go ahead and type your ldap admin_master password and enter and it will work. First line of your output file will need to be removed (Enter LDAP Password:) before running ldapadd.
Last hint, ldapadd(1) is a hard link to ldapmodify(1) with the -a (add) flag turned on.
ldapsearch and ldapadd are not necessarily the best tools to clone your LDAP DB. slapcat and slapadd are much better options.
Export your DB with slapcat:
slapcat > ldif
Import the DB with slapadd (make sure the LDAP server is stopped):
slapadd -l ldif
Some appointments:
Save your personalized schemas and objectclasses definitions on your new server. You can look for your included files at slapd.conf to obtain it, for example (this is a part of my slapd.conf):
include /etc/ldap/schema/core.schema
Include your personalized schemas and objectclasses in your new openLDAP installation.
Use slapcat command to export your full LDAP tree to a single/various ldif files.
Use ldapadd to import the ldif files on to your new LDAP installation.
I prefer copy the database through the protocol:
first of all be sure you have the same schemas on both servers.
dump the database with ldapsearch:
ldapsearch -LLL -Wx -D "cn=admin,dc=domain" -b "dc=domain" > domain.ldif
and import it in the new server:
ldapmodify -Wx -D "cn=admin,dc=domain" -a -f domain.ldif
in one line:
ldapsearch -LLL -Wx -D "cn=admin,dc=domain" -b "dc=domain" | ldapmodify -w pass -x -D "cn=admin,dc=domain" -a
By using the bin/ldap* commands you are talking directly with the server while using bin/slap* commands you are dealing with the backend files
(Not enough reputation to write a comment...)
Ldapsearch opens a connection to the LDAP server.
Slapcat instead accesses the database directly, and this means that ACLs, time and size limits, and other byproducts of the LDAP connection are not evaluated, and hence will not alter the data. (Matt Butcher, "Mastering OpenLDAP")
Thanks, Vish. Worked like a charm! I edited the command:
ldapsearch -z max -LLL -Wx -D "cn=Manager,dc=domain,dc=fr" -b "dc=domain,dc=fr" >/tmp/save.ldif
ldapmodify -c -Wx -D "cn=Manager,dc=domain,dc=fr" -a -f /tmp/save.ldif
Just added the -z max to avoid the size limitation and the -c to go on even if the target domain already exists (my case).