Cloud-init File Command line option 'S' [from -fsSL] is not understood in combination with the other options - linux

i want to execute this cloud-init file and terraform file:
Cloud-init:
#cloud-config
runcmd:
- mkdir react
- cd react
- type -p curl >/dev/null || sudo apt install curl -y
curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg \
&& sudo chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg \
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null \
&& sudo apt update \
&& sudo apt install gh -y
- curl -o actions-runner-linux-x64-2.301.1.tar.gz -L https://github.com/actions/runner/releases/download/v2.301.1/actions-runner-linux-x64-2.301.1.tar.gz
- tar xzf ./actions-runner-linux-x64-2.301.1.tar.gz
- yes "" | ./config.sh --url https://github.com/yuuval/react-deploy --token AVYXWHXNRBPIDXJDPUDK6QTD2LIPE
- sudo ./svc.sh install
- sudo ./svc.sh start
- yes "" | sudo apt install nginx
- gh auth login --hostname github.com --with-token <<< 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 --git-protocol https
- gh repo clone yuuval/react-deploy
- cd react-deploy
- gh workflow run node.js.yml
- sleep 70
- cd /etc/nginx/sites-available
- sudo rm default
- echo "server {
listen 80 default_server;
server_name _;
# react app & front-end files
location / {
root /home/ubuntu/react/_work/react-deploy/react-deploy/build;
try_files \$uri /index.html;
}
}" | sudo tee /etc/nginx/sites-available/default
- sudo service nginx restart
- sudo chmod +x /home
- sudo chmod +x /home/ubuntu
- sudo chmod +x /home/ubuntu/react
- sudo chmod +x /home/ubuntu/react/_work
- sudo chmod +x /home/ubuntu/react/_work/react-deploy
- sudo chmod +x /home/ubuntu/react/_work/react-deploy/react-deploy
- sudo chmod +x /home/ubuntu/react/_work/react-deploy/react-deploy/build
The terraform file isn't relevant i think. So when i run this whole thing with terraform init and terraform apply, its going threw but nothing is hapenning. In the /var/log in the file cloud-init-output file i found this error:
dd: unrecognized operand ‘ ’
Try 'dd --help' for more information.
E: Command line option 'S' [from -fsSL] is not understood in combination with the other options.
I guess its from this command, which should install gh cli (found here: https://github.com/cli/cli/blob/trunk/docs/install_linux.md):
type -p curl >/dev/null || sudo apt install curl -y
curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg \
&& sudo chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg \
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null \
&& sudo apt update \
&& sudo apt install gh -y
If i do this whole cloud-init file manually it works. So i don't know what to do else.

You seem to be missing \ and && after install curl -y, since I just tried on two WSL machines (that's all I have with me right now) and it was just fine there.
So my suspicion is that your curl command got dazed inside, since you're not exactly running that smaller command and bigger one separately, but they should be rather sundered, so maybe give it a shot?
On this weird page (came up by exact search) https://ouyen.github.io/github/ I found no install curl -y but the next one, which clearly indicated it being ran separately, so I think your issue is just there.

Related

Linux paste content to default file of Nginx without formation error in cloud-init

i have a problem with my formation in yaml. First of all here is the file:
#cloud-config
runcmd:
- mkdir react
- cd react
- type -p curl >/dev/null || sudo apt install curl -y
curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg \
&& sudo chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg \
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null \
&& sudo apt update \
&& sudo apt install gh -y
- curl -o actions-runner-linux-x64-2.301.1.tar.gz -L https://github.com/actions/runner/releases/download/v2.301.1/actions-runner-linux-x64-2.301.1.tar.gz
- tar xzf ./actions-runner-linux-x64-2.301.1.tar.gz
- yes "" | ./config.sh --url https://github.com/yuuval/react-deploy --token AVYXWHXNRBPIDXJDPUDK6QTD2LIPE
- sudo ./svc.sh install
- sudo ./svc.sh start
- yes "" | sudo apt install nginx
- gh auth login --hostname github.com --with-token <<< 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 --git-protocol https
- gh repo clone yuuval/react-deploy
- cd react-deploy
- gh workflow run node.js.yml
- sleep 70
- cd /etc/nginx/sites-available
- sudo rm default
- echo "server {
listen 80 default_server;
server_name _;
# react app & front-end files
location / {
root /home/ubuntu/react/_work/react-deploy/react-deploy/build;
try_files \$uri /index.html;
}
}" | sudo tee /etc/nginx/sites-available/default
- sudo service nginx restart
- sudo chmod +x /home
- sudo chmod +x /home/ubuntu
- sudo chmod +x /home/ubuntu/react
- sudo chmod +x /home/ubuntu/react/_work
- sudo chmod +x /home/ubuntu/react/_work/react-deploy
- sudo chmod +x /home/ubuntu/react/_work/react-deploy/react-deploy
- sudo chmod +x /home/ubuntu/react/_work/react-deploy/react-deploy/build
The problem is here:
echo "server {
listen 80 default_server;
server_name _;
# react app & front-end files
location / {
root /home/ubuntu/react/_work/react-deploy/react-deploy/build;
try_files \$uri /index.html;
}
}" | sudo tee /etc/nginx/sites-available/default
The lines that should go into the default file must have a certain structure. This is the case with the current cloud-init, but it becomes invalid because it does not start at the same line as the "-". Does anyone have an idea how to get around this?
This is the error of a yaml linter tool:
All mapping items must start at the same column at line 25, column 1
Implicit keys need to be on a single line at line 25, column 3
Implicit map keys need to be followed by map values at line 25, column 3
Unexpected flow-map-end token in YAML stream: "}" at line 32, column 1
Unexpected double-quoted-scalar token in YAML stream: "\" | sudo tee /etc/nginx/sites-available/default\n - sudo service nginx restart\n - sudo chmod +x /home\n - sudo chmod +x /home/ubuntu\n - sudo chmod +x /home/ubuntu/react\n - sudo chmod +x /home/ubuntu/react/_work\n - sudo chmod +x /home/ubuntu/react/_work/react-deploy\n - sudo chmod +x /home/ubuntu/react/_work/react-deploy/react-deploy\n - sudo chmod +x /home/ubuntu/react/_work/react-deploy/react-deploy/build" at line 32, column 2
line 25 is here: listen 80 default_server;

/bin/sh: passwd: command not found

I tried to execute Docker-compose build but getting the below error.
I'm using centos7 and completely new to Linux.
/bin/sh: passwd: command not found.
ERROR: Service 'remote_host' failed to build: The command '/bin/sh -c useradd remote_user && echo "welcome1" | passwd remote_user --stdin && mkdir /home/remote_user/.ssh && chmod 700 /home/remote_user/.ssh' returned a non-zero code: 127.
DockerFile.
FROM centos: latest
RUN yum -y install OpenSSH-server
RUN useradd remote_user && \
echo "welcome1" | passwd remote_user --stdin && \
mkdir /home/remote_user/.ssh && \
chmod 700 /home/remote_user/.ssh`enter code here`
COPY remote-key.pub /home/remote_user/.ssh/authorized_keys
RUN chown remote_user:remote_user -R /home/remote_user chmod 600 /home/remote_user/.ssh/authorized_keys
RUN /usr/sbin/sshd-keygen
CMD /usr/sbin/sshd -D
whoami: mosses987
$PATH: /usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/mosses987/.local/bin:/home/mosses987/bin
add this line its working:
RUN yum install -y passwd
And comment this line:
RUN /usr/sbin/sshd-keygen
This should work,
FROM centos
RUN yum -y install openssh-server
RUN yum install -y passwd
RUN useradd remote_user && \
echo "1234" | passwd remote_user --stdin && \
mkdir /home/remote_user/.ssh && \
chmod 700 /home/remote_user/.ssh
COPY remote-key.pub /home/remote_user/.ssh/authorized_keys
RUN chown remote_user:remote_user -R /home/remote_user/.ssh/ && \
chmod 600 /home/remote_user/.ssh/authorized_keys
#RUN /usr/sbin/sshd-keygen
CMD /usr/sbin/sshd -D
You need to install passwd because the remote host does not have passwd installed. Add below line before the passwd command.
RUN yum install -y passwd
add this line
RUN yum install -y passwd
That should work
FROM centos:7
RUN yum update -y && \
yum -y install openssh-server && \
yum install -y passwd
RUN useradd remote_user && \
echo "1234" | passwd remote_user --stdin && \
mkdir /home/remote_user/.ssh && \
chmod 700 /home/remote_user/.ssh
COPY remote-key.pub /home/remote_user/.ssh/authorized_keys
RUN chown -R remote_user:remote_user /home/remote_user/.ssh && \
chmod -R 600 /home/remote_user/.ssh/authorized_keys
RUN /usr/sbin/sshd-keygen
CMD /usr/sbin/sshd -D

public key is not available: NO_PUBKEY F76221572C52609D

For the below docker file:
FROM microsoft/aspnetcore-build:1.0.1
ENV DOTNET_SKIP_FIRST_TIME_EXPERIENCE 1
# This is FROM openjdk:8-jdk
RUN apt-get update && apt-get install -y --no-install-recommends \
bzip2 \
unzip \
xz-utils \
apt-transport-https \
&& rm -rf /var/lib/apt/lists/*
RUN echo 'deb http://deb.debian.org/debian jessie-backports main' > /etc/apt/sources.list.d/jessie-backports.list
RUN echo 'deb https://apt.dockerproject.org/repo debian-jessie main' > /etc/apt/sources.list.d/docker.list
# Default to UTF-8 file.encoding
ENV LANG C.UTF-8
# add a simple script that can auto-detect the appropriate JAVA_HOME value
# based on whether the JDK or only the JRE is installed
RUN { \
echo '#!/bin/sh'; \
echo 'set -e'; \
echo; \
echo 'dirname "$(dirname "$(readlink -f "$(which javac || which java)")")"'; \
} > /usr/local/bin/docker-java-home \
&& chmod +x /usr/local/bin/docker-java-home
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
ENV JAVA_VERSION 8u111
ENV JAVA_DEBIAN_VERSION 8u111-b14-2~bpo8+1
# see https://bugs.debian.org/775775
# and https://github.com/docker-library/java/issues/19#issuecomment-70546872
ENV CA_CERTIFICATES_JAVA_VERSION 20140324
RUN set -x \
&& apt-get update \
&& apt-get install -y \
openjdk-8-jdk="$JAVA_DEBIAN_VERSION" \
ca-certificates-java="$CA_CERTIFICATES_JAVA_VERSION" \
&& rm -rf /var/lib/apt/lists/* \
&& [ "$JAVA_HOME" = "$(docker-java-home)" ]
# see CA_CERTIFICATES_JAVA_VERSION notes above
RUN /var/lib/dpkg/info/ca-certificates-java.postinst configure
##### END OF THE JDK
##### START Jenkins Slave Node Config settings
# Create Jenkins User
RUN useradd jenkins -m -s /bin/bash
RUN chown -R jenkins /home/jenkins
RUN chgrp -R jenkins /home/jenkins
RUN chown -R jenkins /tmp
RUN chgrp -R jenkins /tmp
# Add the jenkins user to sudoers
RUN echo "jenkins ALL=(ALL) ALL" >> etc/sudoers
# Must install docker to create docker images from docker container. Inception. Head... hurts.
# container must be called with -v /var/run/docker.sock:/var/run/docker.sock
RUN apt-get install -y --no-install-recommends apt-transport-https ca-certificates
RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
RUN apt-get update && apt-get install -y --no-install-recommends \
docker-engine \
&& rm -rf /var/lib/apt/lists/*
# This must run after the docker install
RUN gpasswd -a jenkins docker
USER jenkins
build image is failing for command at line #38
RUN set -x \
&& apt-get update \
&& apt-get install -y \
openjdk-8-jdk="$JAVA_DEBIAN_VERSION" \
ca-certificates-java="$CA_CERTIFICATES_JAVA_VERSION" \
&& rm -rf /var/lib/apt/lists/* \
&& [ "$JAVA_HOME" = "$(docker-java-home)" ]
with error:
W: GPG error: https://apt.dockerproject.org debian-jessie InRelease:
The following signatures couldn't be verified because the public key
is not available: NO_PUBKEY F76221572C52609D
W: There is no public key available for the following key IDs:
AA8E81B4331F7F50
W: Failed to fetch http://deb.debian.org/debian/dists/jessie-backports/main/binary-amd64/Packages
404 Not Found
E: Some index files failed to download. They have been ignored, or old ones used instead.
ERROR: Service 'slavedotnet' failed to build: The command '/bin/sh -c set -x && apt-get update && apt-get install -y openjdk-8-jdk="$JAVA_DEBIAN_VERSION"
ca-certificates-java="$CA_CERTIFICATES_JAVA_VERSION" && rm -rf
/var/lib/apt/lists/* && [ "$JAVA_HOME" = "$(docker-java-home)" ]'
returned a non-zero code: 100
How public key error can get resolved?
There are several issues here:
1) W: GPG error: https://apt.dockerproject.org debian-jessie InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY F76221572C52609D
W: There is no public key available for the following key IDs:
AA8E81B4331F7F50
Solution:
Move the keyserver add actions to the place before RUN echo 'deb http://deb.debian.org/debian jessie-backports main' > /etc/apt/sources.list.d/jessie-backports.list, meanwhile add AA8E81B4331F7F50 also as next:
RUN apt-get install -y --no-install-recommends apt-transport-https ca-certificates
RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys AA8E81B4331F7F50
2) W: Failed to fetch http://deb.debian.org/debian/dists/jessie-backports/main/binary-amd64/Packages 404 Not Found
E: Some index files failed to download. They have been ignored, or old ones used instead.
Solution:
microsoft/aspnetcore-build:1.0.1 base on debian8, and you want to use openjdk8 which was default not in apt repository. So you use deb http://deb.debian.org/debian jessie-backports main.
Unfortunately, if you check http://ftp.debian.org/debian/dists/, you will find jessie-backports had been removed. So you had to switch to archived url like next (Comment the old url, just use the url next):
#RUN echo 'deb http://deb.debian.org/debian jessie-backports main' > /etc/apt/sources.list.d/jessie-backports.list
RUN echo 'deb http://archive.debian.org/debian jessie-backports main' > /etc/apt/sources.list.d/jessie-backports.list
Meanwhile, you had to add next after doing above to resolve release-file-expired-problem:
RUN echo "Acquire::Check-Valid-Until \"false\";" > /etc/apt/apt.conf.d/100disablechecks
3) ENV JAVA_VERSION 8u111
ENV JAVA_DEBIAN_VERSION 8u111-b14-2~bpo8+1
Solution:
Not sure how you get this version, but in fact after change to archive jessie backports, what you could get is something like next:
root#2ecaeffec483:/etc/apt# apt-cache policy openjdk-8-jdk
openjdk-8-jdk:
Installed: (none)
Candidate: 8u171-b11-1~bpo8+1
Version table:
8u171-b11-1~bpo8+1 0
100 http://archive.debian.org/debian/ jessie-backports/main amd64 Packages
So, you had to change to next:
ENV JAVA_VERSION 8u171
ENV JAVA_DEBIAN_VERSION 8u171-b11-1~bpo8+1
This script will automatically add most missing GPG keys:
#!/bin/bash
set -e
for PUBKEY in $(apt-get update 2>&1 | grep NO_PUBKEY | awk '{print $NF}')
do
wget -q "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x${PUBKEY}" -O - | sed -n '/BEGIN/,/END/p' | apt-key add - 2>/dev/null
done
The only prerequisite is to have wget installed. It can also be used with curl.
To fix the below:
W: There is no public key available for the following key IDs:
AA8E81B4331F7F50
Just use the below code:
sudo apt-get install debian-keyring debian-archive-keyring
sudo apt-key update
sudo apt-get update

how do i fix my script problem issuing SED command

i am trying to write my first script and everything is working fine it is to automatically install a new server, the only problem i have is using sed to change the ssl certificate file i have followed all the answers in the forums available here but i still cant get it to overwrite i have used 2 other sed commands and working fine
i am running script on ubuntu 16.04 with apache2 and php7.0 lamp
the script completes but no rewrite of conf
this is my script just in case anything is conflicting
#!/bin/bash
apt-get -y update
apt-get -y upgrade
apt-get -y install apache2
apt-get install -y php7.0 libapache2-mod-php7.0 php7.0-cli php7.0-common php7.0-mbstring php7.0-gd php7.0-intl php7.0-xml php7.0-mysql php7.0-mcrypt php7.0-zip
echo mysql-server-5.1 mysql-server/root_password password PASSWORD | debconf-set-selections
echo mysql-server-5.1 mysql-server/root_password_again password PASSWORD | debconf-set-selections
apt-get install -y mysql-server
/etc/init.d/mysql restart
a2enmod ssl
a2ensite default-ssl.conf
service apache2 restart
APP_PASS="PASSWORD"
ROOT_PASS="PASSWORD"
APP_DB_PASS="PASSWORD"
echo "phpmyadmin phpmyadmin/dbconfig-install boolean true" | debconf-set-selections
echo "phpmyadmin phpmyadmin/app-password-confirm password $APP_PASS" | debconf-set-selections
echo "phpmyadmin phpmyadmin/mysql/admin-pass password $ROOT_PASS" | debconf-set-selections
echo "phpmyadmin phpmyadmin/mysql/app-pass password $APP_DB_PASS" | debconf-set-selections
echo "phpmyadmin phpmyadmin/reconfigure-webserver multiselect apache2" | debconf-set-selections
apt-get install -y phpmyadmin
sed -i 's/Port 22/Port 4747/g' /etc/ssh/sshd_config
sed -i 's/PermitRootLogin yes/PermitRootLogin no/g' /etc/ssh/sshd_config
service sshd restart
apt-get install vsftpd -y
sed -i 's/root/#root/g' /etc/ftpusers
service vsftpd restart
apt-get install software-properties-common -y
add-apt-repository ppa:certbot/certbot -y
apt-get update -y
apt-get install python-certbot-apache -y
service apache2 stop
certbot certonly --standalone --non-interactive --agree-tos -m EMAIL#mymail.com -d domain.com
adduser --quiet --disabled-password --shell /bin/bash --home /home/USERNAME --gecos "User" USERNAME
echo "USERNAME:PASSWORD" | chpasswd
usermod -aG sudo USERNAME
iptables -I INPUT 1 -p udp -m udp --dport 1900 -j DROP
crontab -l > mycron
echo "#daily letsencrypt renew --quiet && systemctl reload apache2" >> mycron
crontab mycron
rm mycron (WORKS BUT GIVES ERROR no crontab for root)
#sed -i "s|SSLCertificateFile=/etc/ssl/certs/ssl-cert-snakeoil.pem|SSLCertificateFile=/letsencrypt/live/domain.com/fullchain.pem|g" /etc/apache2/sites-enabled/default-ssl.conf (NOT WORKING)
#SSL_DEFAULT_CERT_PATH="SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem"
#SSL_CERT_PATH="SSLCertificateFile /letsencrypt/live/domain.com/fullchain.pem"
#sed -i "s|.*\b$SSL_DEFAULT_CERT_PATH\b.*|$SSL_CERT_PATH|" /etc/apache2/sites-enabled/default-ssl.conf (NOT WORKING)
service apache2 restart
these are the two i have tried but no luck
sed -i "s|SSLCertificateFile=/etc/ssl/certs/ssl-cert-snakeoil.pem|SSLCertificateFile=/letsencrypt/live/domain.com/fullchain.pem|g" /etc/apache2/sites-enabled/default-ssl.conf
does not work
SSL_DEFAULT_CERT_PATH="SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem"
SSL_CERT_PATH="SSLCertificateFile /letsencrypt/live/domain.com/fullchain.pem"
sed -i "s|.*\b$SSL_DEFAULT_CERT_PATH\b.*|$SSL_CERT_PATH|" /etc/apache2/sites-enabled/default-ssl.conf
does not work
original file SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
not sure if the spaces make a difference

Docker, running NVM script in a new bash shell

I have the following in my Dockerfile:
run apt-get update; \
apt-get install -y curl && \
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.5/install.sh | bash
Following this line of code, I need to run a command in a new bash shell so that the environment variable set from the NVM script are used.
I have tired the following to install Nodejs and it does not work:
run ["/bin/bash", "-c", "nvm install 8.7.0"]
What can I do?
It's better to use a Dockerhub repo and use it in your Dockerfile.
You can check this repositorie or this link for more repositories, please read description before choosing a repositorie.
So for example, you can add the code line below in your Dockerfile it will pull the nvm image and install it then add your app instructions.
FROM livingdocs/nvm
Or you can read their Dockerfile and use the command they used it to install nvm
ADD ./.nvmrc /app/.nvmrc
RUN bash -c '. /usr/share/nvm/nvm.sh && cd /app && nvm install && nvm alias default'
if it didn't put this one from another repositorie:
RUN sudo apt-get update && \
sudo apt-get install -y build-essential libssl-dev libmysqlclient-dev && \
sudo apt-get clean && \
sudo rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN curl --location https://raw.github.com/creationix/nvm/master/install.sh | sh && \
sudo /bin/bash -c "echo \"[[ -s \$HOME/.nvm/nvm.sh ]] && . \$HOME/.nvm/nvm.sh\" >> /etc/profile.d/npm.sh" && \
echo "[[ -s $HOME/.nvm/nvm.sh ]] && . $HOME/.nvm/nvm.sh" >> $HOME/.bashrc

Resources