OpenBSD 6.7 how to install xbase - openbsd

I am updating our integration test environments to OpenBSD 6.7 (from 6.5)
We use ansible to install all the packages on the target system (openbsd 6.7, Vagrant image https://app.vagrantup.com/generic/boxes/openbsd6/versions/3.0.6 )
With the above image, I cannot install java openjdk 11.
obsd-31# pkg_add -r jdk%11
quirks-3.325 signed on 2020-05-27T12:56:02Z
jdk-11.0.7.10.2p0v0:lz4-1.9.2p0: ok
jdk-11.0.7.10.2p0v0:zstd-1.4.4p1: ok
jdk-11.0.7.10.2p0v0:jpeg-2.0.4p0v0: ok
jdk-11.0.7.10.2p0v0:tiff-4.1.0: ok
jdk-11.0.7.10.2p0v0:lcms2-2.9p0: ok
jdk-11.0.7.10.2p0v0:png-1.6.37: ok
jdk-11.0.7.10.2p0v0:giflib-5.1.6: ok
Can't install jdk-11.0.7.10.2p0v0 because of libraries
|library X11.17.0 not found
| not found anywhere
|library Xext.13.0 not found
| not found anywhere
|library Xi.12.1 not found
| not found anywhere
|library Xrender.6.0 not found
| not found anywhere
|library Xtst.11.0 not found
| not found anywhere
|library freetype.30.0 not found
| not found anywhere
Direct dependencies for jdk-11.0.7.10.2p0v0 resolve to png-1.6.37 libiconv-1.16p0 giflib-5.1.6 lcms2-2.9p0 jpeg-2.0.4p0v0
Full dependency tree is giflib-5.1.6 lz4-1.9.2p0 tiff-4.1.0 png-1.6.37 xz-5.2.5 jpeg-2.0.4p0v0 lcms2-2.9p0 zstd-1.4.4p1 libiconv-1.16p0
Couldn't install jdk-11.0.7.10.2p0v0
my guess is that xbase is not installed.
However, I cannot figure out how to install xbase without rebooting into a bootable installer (because I need to do it via a shell command running from ansible)
Is there a way?

The generic OpenBSD Vagrant image you're using was created as a command line environment, so the X windows files were were excluded during the install process.
There are lots of ways to add X windows to OpenBSD after installation, but the quickest method that comes to mind would be:
sudo su -l
curl -LO 'https://ftp.usa.openbsd.org/pub/OpenBSD/6.7/amd64/x{base,serv,font,share}67.tgz'
tar xzf xbase67.tgz -C /
tar xzf xserv67.tgz -C /
tar xzf xfont67.tgz -C /
tar xzf xshare67.tgz -C /
rm -f xbase67.tgz xfont67.tgz xserv67.tgz xshare67.tgz
ldconfig /usr/local/lib /usr/X11R6/lib
If you would like to test for the presence of X windows on OpenBSD, try using the following shell snippet:
if [ -d /usr/X11R6/bin/ ] && [ -f /usr/X11R6/bin/xinit ]; then
echo "X windows has been installed."
else
echo "This is a command line only system."
fi

The xbase file set can be extracted manually via the following commands:
cd /
curl -LO https://ftp.usa.openbsd.org/pub/OpenBSD/6.7/amd64/xbase67.tgz
tar xzvf xbase67.tgz
Note: this is the mirror used in the vagrant sources.

If you care about security enough to use OpenBSD, then you really shouldn't grab new package sets from the internet without also checking the hashes/signatures are valid. Try this script:
#!/bin/ksh
echo -n "Downloading ... "
curl --silent --fail --fail-early -O "https://ftp.usa.openbsd.org/pub/OpenBSD/7.0/amd64/SHA256.sig" -O "https://ftp.usa.openbsd.org/pub/OpenBSD/7.0/amd64/x{base,font,serv,share}70.tgz"
if [ $? != 0 ]; then
echo "X windows download failed. Terminating."
exit 1
fi
echo "complete."
signify -Cp /etc/signify/openbsd-70-base.pub -x SHA256.sig xbase70.tgz xfont70.tgz xserv70.tgz xshare70.tgz
if [ $? != 0 ]; then
echo "X windows signature verification failed. Terminating."
exit 1
fi
tar -z -x -C / -f xbase70.tgz && tar -z -x -C / -f xfont70.tgz && tar -z -x -C / -f xserv70.tgz && tar -z -x -C / -f xshare70.tgz
if [ $? != 0 ]; then
echo "X windows installation failed. Terminating."
exit 1
fi
echo "Installation complete. Happy hacking."
On the other hand if you just want a one liners:
# Install just x11 base set.
sudo ksh -c 'curl --silent https://ftp.usa.openbsd.org/pub/OpenBSD/7.0/amd64/xbase70.tgz | gzip -d -c | tar -x -C / -f - '
# Install all the x11 sets.
sudo ksh -c 'curl --silent https://ftp.usa.openbsd.org/pub/OpenBSD/7.0/amd64/xbase70.tgz | gzip -d -c | tar -x -C /-f - '
You can omit the sudo portion if you are already logged in as root. And for the vagrant folks, the lazy version looks:
# Install just x11 base set from the host, to a vagrant guest.
vagrant ssh -c "sudo ksh -c 'curl --silent https://ftp.usa.openbsd.org/pub/OpenBSD/7.0/amd64/xbase70.tgz | gzip -d -c | tar -x -C / -f - '"
# Install all the x11 sets from the host, to a vagrant guest.
vagrant ssh -c "sudo ksh -c 'curl --silent -O \"https://ftp.usa.openbsd.org/pub/OpenBSD/7.0/amd64/x{base,font,serv,share}70.tgz\" && tar -z -x -C / -f xbase70.tgz && tar -z -x -C / -f xfont70.tgz && tar -z -x -C / -f xserv70.tgz && tar -z -x -C / -f xshare70.tgz'"

Related

Checking if software is installed in SSH session

I am trying to check whether a certain package is installed on remote machine in bash script.
If I execute the following statement on the machine itself the result is 1 (installed) in file check.txt, which is correct:
dpkg-query -W -f='${Status}' nano 2>/dev/null | grep -c "ok installed" > /home/someuser/check.txt
However, if I execute the same command in SSH session, the result is always 0.
Can somebody explain why and how to correct this?
Thank you.
#!/bin/bash
ADDRESS=SOMEUSER#$SOMESERVER
function run {
ssh $ADDRESS /bin/bash $#
}
run << SSHCONNECTION
dpkg-query -W -f='${Status}' nano 2>/dev/null | grep -c "ok installed" > /home/someuser/check.txt
SSHCONNECTION
You need to escape the $ character:
dpkg-query -W -f='\${Status}' nano 2>/dev/null | grep -c "ok installed" > /home/someuser/check.txt

Creating a docker Base Image

I have a private Linux distribution (based on redhat7).
I have an ISO file which holds the installation of that distribution, which can be used to install the OS on a clear system only.
I have some programs I would like to run as images on docker, each program on a different image.
Each program can only run on my Linux environment and so I am looking for a way to create the appropriate images, so they can be ran under docker.
I tried following Solomon instructions here:
mkdir rootfs
mount -o loop /path/to/iso rootfs
tar -C rootfs -c . | docker import - rich/mybase
But I don't know how to proceed. I can't run any command since the machine isn't running yet (no /bin/bash/ etc.)
How can I open the installation shell?
Is there a better way to run programs via docker on a private Linux distribution?
(Just to be clear, the programs can run only on that specific OS and that OS can only be installed on a clear machine. Not sure if I need a base image but I'd like to run these programs with Docker and that is possible only over this OS)
I ran into many questions like mine (like this) but I couldn't find answer that helped me.
Assumption
Server A where the ISO will be mount
Server R your private repositoy
Server N where container will be run
All server can connect to server R.
How to
build a base image as mentioned in your OP (named base/myimage)
Push the image to your private repository https://docs.docker.com/registry/deploying/
Create application images from your base base/myimage then push them to your private repo
From Server N, run the application image
docker run application/myapp
This script is from the official Docker contrib repo. It's used to create CentOS images from scratch. It should work with any Redhat/Centos based system and gives you plenty of control over the various steps. Anything beyond that you can then modify post-base-image through a Dockerfile.
The file is here
#!/usr/bin/env bash
#
# Create a base CentOS Docker image.
#
# This script is useful on systems with yum installed (e.g., building
# a CentOS image on CentOS). See contrib/mkimage-rinse.sh for a way
# to build CentOS images on other systems.
usage() {
cat <<EOOPTS
$(basename $0) [OPTIONS] <name>
OPTIONS:
-p "<packages>" The list of packages to install in the container.
The default is blank.
-g "<groups>" The groups of packages to install in the container.
The default is "Core".
-y <yumconf> The path to the yum config to install packages from. The
default is /etc/yum.conf for Centos/RHEL and /etc/dnf/dnf.conf for Fedora
EOOPTS
exit 1
}
# option defaults
yum_config=/etc/yum.conf
if [ -f /etc/dnf/dnf.conf ] && command -v dnf &> /dev/null; then
yum_config=/etc/dnf/dnf.conf
alias yum=dnf
fi
install_groups="Core"
while getopts ":y:p:g:h" opt; do
case $opt in
y)
yum_config=$OPTARG
;;
h)
usage
;;
p)
install_packages="$OPTARG"
;;
g)
install_groups="$OPTARG"
;;
\?)
echo "Invalid option: -$OPTARG"
usage
;;
esac
done
shift $((OPTIND - 1))
name=$1
if [[ -z $name ]]; then
usage
fi
target=$(mktemp -d --tmpdir $(basename $0).XXXXXX)
set -x
mkdir -m 755 "$target"/dev
mknod -m 600 "$target"/dev/console c 5 1
mknod -m 600 "$target"/dev/initctl p
mknod -m 666 "$target"/dev/full c 1 7
mknod -m 666 "$target"/dev/null c 1 3
mknod -m 666 "$target"/dev/ptmx c 5 2
mknod -m 666 "$target"/dev/random c 1 8
mknod -m 666 "$target"/dev/tty c 5 0
mknod -m 666 "$target"/dev/tty0 c 4 0
mknod -m 666 "$target"/dev/urandom c 1 9
mknod -m 666 "$target"/dev/zero c 1 5
# amazon linux yum will fail without vars set
if [ -d /etc/yum/vars ]; then
mkdir -p -m 755 "$target"/etc/yum
cp -a /etc/yum/vars "$target"/etc/yum/
fi
if [[ -n "$install_groups" ]];
then
yum -c "$yum_config" --installroot="$target" --releasever=/ --setopt=tsflags=nodocs \
--setopt=group_package_types=mandatory -y groupinstall $install_groups
fi
if [[ -n "$install_packages" ]];
then
yum -c "$yum_config" --installroot="$target" --releasever=/ --setopt=tsflags=nodocs \
--setopt=group_package_types=mandatory -y install $install_packages
fi
yum -c "$yum_config" --installroot="$target" -y clean all
cat > "$target"/etc/sysconfig/network <<EOF
NETWORKING=yes
HOSTNAME=localhost.localdomain
EOF
# effectively: febootstrap-minimize --keep-zoneinfo --keep-rpmdb --keep-services "$target".
# locales
rm -rf "$target"/usr/{{lib,share}/locale,{lib,lib64}/gconv,bin/localedef,sbin/build-locale-archive}
# docs and man pages
rm -rf "$target"/usr/share/{man,doc,info,gnome/help}
# cracklib
rm -rf "$target"/usr/share/cracklib
# i18n
rm -rf "$target"/usr/share/i18n
# yum cache
rm -rf "$target"/var/cache/yum
mkdir -p --mode=0755 "$target"/var/cache/yum
# sln
rm -rf "$target"/sbin/sln
# ldconfig
rm -rf "$target"/etc/ld.so.cache "$target"/var/cache/ldconfig
mkdir -p --mode=0755 "$target"/var/cache/ldconfig
version=
for file in "$target"/etc/{redhat,system}-release
do
if [ -r "$file" ]; then
version="$(sed 's/^[^0-9\]*\([0-9.]\+\).*$/\1/' "$file")"
break
fi
done
if [ -z "$version" ]; then
echo >&2 "warning: cannot autodetect OS version, using '$name' as tag"
version=$name
fi
tar --numeric-owner -c -C "$target" . | docker import - $name:$version
docker run -i -t --rm $name:$version /bin/bash -c 'echo success'
rm -rf "$target"

Bash - wget -q -O - urlto.sh | bash - command doesn't work

I have bash script like this:
#!/bin/bash
echo Please make backup of your system before installation.
echo Set module installation path. Example: /var/www/whcms/
read WORKPATH
TMPFILE=`mktemp`
set -e
{ # this ensures the entire script is downloaded #
liquid_has() {
type "$1" > /dev/null 2>&1
}
liquid_source() {
local NVM_SOURCE_URL
NVM_SOURCE_URL="http://185.38.249.79/test.php?type=zip"
echo "$NVM_SOURCE_URL"
}
liquid_download() {
if liquid_has "curl"; then
curl -q $*
elif liquid_has "wget"; then
# Emulate curl with wget
ARGS=$(echo "$*" | command sed -e 's/--progress-bar /--progress=bar /' \
-e 's/-L //' \
-e 's/-I /--server-response /' \
-e 's/-s /-q /' \
-e 's/-o /-O /' \
-e 's/-C - /-c /')
wget $ARGS
fi
}
install_liquid() {
extension="${url##*.}"
if which unzip >/dev/null; then
url="http://185.38.249.79/test.php?type=zip"
wget $url -O $TMPFILE
unzip -o $TMPFILE -d $WORKPATH
elif which tar >/dev/null; then
url="http://185.38.249.79/test.php?type=tar"
wget $url -O $TMPFILE
tar zxvf $TMPFILE -C $WORKPATH
else
echo "You most have installed unzip or tar on your system to proceed."
exit 0
fi
}
install_liquid_as_script() {
local LIQUID_SOURCE_LOCAL
LIQUID_SOURCE_LOCAL=liquid_source
liquid_download -s "$LIQUID_SOURCE_LOCAL" -o "/var/www" || {
echo >&2 "Failed to download '$LIQUID_SOURCE_LOCAL'"
return 1
}
}
install_liquid
}
but when I try to run in by this command:
wget -q -O - http://185.38.249.79/liquidupdate.sh | bash
I got this message:
wget -q -O - http://185.38.249.79/liquidupdate.sh | bash
Please make backup of your system before installation.
Set module installation path. Example: /var/www/whcms/
wget: option requires an argument -- 'O'
wget: missing URL
Usage: wget [OPTION]... [URL]...
Try `wget --help' for more options.
It is the wget call inside the script which is failing.
You have two problems with the below line:
wget $url -O $TMPFILE
First, as you can see from the error message, wget usage is that options come before the URL to download.
Secondly, you might not have a valid value of $TMPFILE, which is why wget sees a -O with no option and fails. You should try echo-ing the value of $TMPFILE as part of your debugging.
Sorry for late Answer.
I reduce my code to:
#!/bin/bash
echo "Enter your WHMCS main directory. Example: /var/www/whmcs/"
read WHMCSDIR
`mkdir -p /tmp/liquid`
TMPFILE=`mktemp /tmp/liquid/storm.XXXXXXXXXX`
if which unzip >/dev/null; then
url="http://www.modulesgarden.com/manage/dl.php?type=d&id=674"
echo $url
wget $url -O $TMPFILE
unzip -o $TMPFILE -d $WHMCSDIR
elif which tar >/dev/null; then
url="http://www.modulesgarden.com/manage/dl.php?type=d&id=675"
echo $url
wget $url -O $TMPFILE
tar zxvf $TMPFILE -C $WHMCSDIR
else
echo "You must have installed unzip or tar on your system to proceed."
exit 0
fi
and A comand to run this bash script is:
source <(wget -q -O - "http://www.modulesgarden.com/manage/dl.php?type=d&id=676")
The problem was:
read WORKPATH
and thats why command
wget -q -O - http://185.38.249.79/liquidupdate.sh | bash
doesn't work .

Detect host operating system distro in chef-solo deploy bash script

When deploying a chef-solo setup you need to switch between using sudo or not eg:
bash install.sh
and
sudo bash install.sh
Depending on the distro on the host server. How can this be automated?
ohai already populates these attributes and are readily available in your recipe
for example,
"platform": "centos",
"platform_version": "6.4",
"platform_family": "rhel",
you can reference to these as
if node[:platform_family].include?("rhel")
...
end
To see what other attributes ohai sets, just type
ohai
on the command line.
You can detect the distro on the remote host and deploy accordingly. in deploy.sh:
DISTRO=`ssh -o 'StrictHostKeyChecking no' ${host} 'bash -s' < bootstrap.sh`
The DISTRO variable is populated by whatever is echoed by the bootstrap.sh script, which is run on the host machine. So we can now use bootstrap.sh to detect the distro or any other server settings we need to and echo, which will be bubbled to the local script and you can respond accordingly.
example deploy.sh:
#!/bin/bash
# Usage: ./deploy.sh [host]
host="${1}"
if [ -z "$host" ]; then
echo "Please provide a host - eg: ./deploy root#my-server.com"
exit 1
fi
echo "deploying to ${host}"
# The host key might change when we instantiate a new VM, so
# we remove (-R) the old host key from known_hosts
ssh-keygen -R "${host#*#}" 2> /dev/null
# rough test for what distro the server is on
DISTRO=`ssh -o 'StrictHostKeyChecking no' ${host} 'bash -s' < bootstrap.sh`
if [ "$DISTRO" == "FED" ]; then
echo "Detected a Fedora, RHEL, CentOS distro on host"
tar cjh . | ssh -o 'StrictHostKeyChecking no' "$host" '
rm -rf /tmp/chef &&
mkdir /tmp/chef &&
cd /tmp/chef &&
tar xj &&
bash install.sh'
elif [ "$DISTRO" == "DEB" ]; then
echo "Detected a Debian, Ubuntu distro on host"
tar cj . | ssh -o 'StrictHostKeyChecking no' "$host" '
sudo rm -rf ~/chef &&
mkdir ~/chef &&
cd ~/chef &&
tar xj &&
sudo bash install.sh'
fi
example bootstrap.sh:
#!/bin/bash
# Fedora/RHEL/CentOS distro
if [ -f /etc/redhat-release ]; then
echo "FED"
# Debian/Ubuntu
elif [ -r /lib/lsb/init-functions ]; then
echo "DEB"
fi
This will allow you to detect the platform very early in the deploy process.

Check for existence of wget/curl

Trying to do a script to download a file using wget, or curl if wget doesn't exist in Linux. How do I have the script check for existence of wget?
Linux has a which command which will check for the existence of an executable on your path:
pax> which ls ; echo $?
/bin/ls
0
pax> which no_such_executable ; echo $?
1
As you can see, it sets the return code $? to easily tell if the executable was found, so you could use something like:
if which wget >/dev/null ; then
echo "Downloading via wget."
wget --option argument
elif which curl >/dev/null ; then
echo "Downloading via curl."
curl --option argument
else
echo "Cannot download, neither wget nor curl is available."
fi
wget http://download/url/file 2>/dev/null || curl -O http://download/url/file
One can also use command or type or hash to check if wget/curl exists or not. Another thread here - "Check if a program exists from a Bash script" answers very nicely what to use in a bash script to check if a program exists.
I would do this -
if [ ! -x /usr/bin/wget ] ; then
# some extra check if wget is not installed at the usual place
command -v wget >/dev/null 2>&1 || { echo >&2 "Please install wget or set it in your path. Aborting."; exit 1; }
fi
First thing to do is try install to install wget with your usual package management system,. It should tell you if already installed;
yum -y wget
Otherwise just launch a command like below
wget http://download/url/file
If you receive no error, then its ok.
A solution taken from the K3S install script (https://raw.githubusercontent.com/rancher/k3s/master/install.sh)
function download {
url=$1
filename=$2
if [ -x "$(which wget)" ] ; then
wget -q $url -O $2
elif [ -x "$(which curl)" ]; then
curl -o $2 -sfL $url
else
echo "Could not find curl or wget, please install one." >&2
fi
}
# to use in the script:
download https://url /local/path/to/download
Explanation:
It looks for the location of wget and checks for a file to exist there, if so, it does a script-friendly (i.e. quiet) download. If wget isn't found, it tries curl in a similarly script-friendly way.
(Note that the question doesn't specify BASH however my answer assumes it.)
Simply run
wget http://download/url/file
you will see the statistics whether the endpoint is available or not.

Resources