When I try to install LogZilla NEO (docker based) on CoreOS, I get:
docker run logzilla/kickstart -p >/usr/bin/logzilla; chmod 755 /usr/bin/logzilla; logzilla install
-bash: /usr/bin/logzilla: Read-only file system
chmod: cannot access '/usr/bin/logzilla': No such file or directory
-bash: logzilla: command not found
I found this: https://coreos.com/os/docs/latest/sdk-disk-partitions.html
But am not sure if LogZilla has to be put in /usr/bin?
Welcome to Stackoverflow :)
The path to the LogZilla binary doesn't matter. In CoreOS, you can make a directory on the server where it is writeable, like /opt/bin.
So just use the following command:
mkdir -p /opt/bin/; \
docker run logzilla/kickstart -p >/opt/bin/logzilla; \
chmod 755 /opt/bin/logzilla; \
logzilla install
Side note: The default ports used to map syslog and http in the LogZilla docker install are 32514, 32601, and 3280 (done intentionally so as not to try and use already-taken ports). But if you are certain the standard syslog/http ports are not in use on the host, you can set this during install, like so:
logzilla install \
--syslog-port-mapping=tcp/514:514,udp/514:514,tcp/601:601 \
--http-port-mapping=tcp/80:80
Update:
The install process is now much easier. Just use:
curl -fsSL https://logzilla.sh | bash
Related
Title is the question.
I'm hosting many docker containers on a rather large linux ec2 instance. One container in particular needs access to a file that gets transferred to the host before run time. The file in question is copied from a windows file server to the ec2 instance using control-m.
When the container image runs, we give it -v to specify a volume mount with a path on the host to that transferred file.
The file is not found in the container. If I make a new file in the container, the new file appears on the host. When I make a file on the host, it appears in the container. When I make a copy of the transferred file using cp -p the copied file DOES show up in the container, but the original still does not.
I don't understand why this is? My suspicion is something to do with it being on a windows server before control-m copies it to the ec2 instance.
Details:
The file lives in the path /folder_path/project_name/resources/file.txt
Its permissions are -rwxrwxr-x 1 pyadmin pyadmin where pyadmin maps to the containers root user.
It's approximately 38mb in size and when I run file file.txt I get the output ASCII text, with CRLF line terminators.
The repo also has a resources folder with files already in it when it is cloned, but none of their names conflict.
Docker Version: 20.10.13
Dockerfile:
FROM python:3.9.11-buster
SHELL ["/bin/bash", "-c"]
WORKDIR /folder_path/project_name
RUN apt-get auto-clean && apt-get update && apt-get install -y unixodbc unixodbc-dev && apt-get upgrade -y
RUN python -m pip install --upgrade pip poetry
COPY . .
RUN python -m pip install --upgrade pip poetry && \
poetry config virtualenvs.create false && \
poetry install
ENTRYPOINT [ "python" ]
Command to start container:
docker run --pull always --rm \
-v /folder_path/project_name/logs:/folder_path/project_name/logs \
-v /folder_path/project_name/extracts:/folder_path/project_name/extracts \
-v /folder_path/project_name/input:/folder_path/project_name/input \
-v /folder_path/project_name/output:/folder_path/project_name/output \
-v /folder_path/project_name/resources:/folder_path/project_name/resources \
my-registry.com/folder_path/project_name:image_tag
i need deploy to fargate, but nodered rebuild will follow hostname to create flow.json, this make me so hard to load old config to new nodered.
But now, if using docker run -h is work,but in fargate dose not work, how can i do?
Of course, release nodered docker version is solved this problem,but i don’t know how to call cli tools,if base on node-red, how can i install aws-cli2 and call it in nodered dashboard?
FROM nodered/node-red:latest
#USER root
RUN curl https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip
RUN unzip awscliv2.zip
RUN ./aws/install
CMD ["node-red"]
The correct Dockerfile would be:
FROM nodered/node-red:latest
USER root
RUN curl https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip
RUN unzip awscliv2.zip
RUN ./aws/install
RUN rm -rf ./aws
USER node-red
But the problem is that the image is based on Alpine Linux which uses the musl standard libraries instead of glibc. And the AWS tools will not work with this runtime.
The easiest solution will be to use the Debian based build that I mentioned in the first comment. The docker file for that can be found here, follow the instructions there to use the docker-debian.sh which will create an image called
testing:node-red-build which you can then use as the base for the Dockerfile I showed earlier:
FROM testing:node-red-build
USER root
RUN curl https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip
RUN unzip awscliv2.zip
RUN ./aws/install
RUN rm -rf ./aws
USER node-red
First of all, let me state I'm not the most virtuous of Linux users, so bear with me...
Below is a brief run-down of all the steps I took. Ultimately the question/issue is is that it seems impossible for me to get a proper docker-compose installation downloaded.
Followed instructions to install docker https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html
sudo curl -L "https://github.com/docker/compose/releases/download/1.26.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
Tried 4 variations of the above command to try to install docker-compose. As shown in the URLs below.
https://www.codegrepper.com/code-examples/php/how+to+install+docker+compose+in+ec2
https://portal.cloud303.io/forum/aws-1/question/i-want-to-install-docker-compose-on-an-amazon-linux-2-ec2-instance-9
https://acloudxpert.com/how-to-install-docker-compose-on-amazon
https://gist.github.com/npearce/6f3c7826c7499587f00957fee62f8ee9
When typing "docker-compose", "sudo docker-compose" etc. All it will say is
"Line 1: Not: command not found".
It seems to be the issue that the docker-compose file is only 9kb in size. Because this is what I get back every time I use the above mentioned docker-compose install sequences.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 9 100 9 0 0 58 0 --:--:-- --:--:-- --:--:-- 58
This issue is sort of addressed here:
https://github.com/docker/compose/issues/6268
Where it is said that the OS is not supported or that we're running a 32bit instance, but of which seem to be strange because all the above tutorials are specifically for AWS EC2 Linux 2.
fwiw 'uname -m' returns aarch64.
So, does anyone have an idea of how to get a full-sized version of docker-compose instead of the 9kb file?
Thanks!
followed the link to install the docker-compose link
Basically, thee are only two steps as below:
sudo curl -L https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
docker-compose version
Amazon Linux is "just" the OS, but compiled binaries also depend on your processor architecture. For instance, EC2 t3a series are based on x86_64 architecture wheras the new t4g types are of aarch64 (which is the reason I run into as similar issue as you and ended up here). uname -m returns that identifier, in your case apparently aarch64.
Since the docker-compose github repo contains no released binary for this architeture (as opposed to x86_64), the resolved full download URL returns a "Not found" body instead of the expected binary, thus the described error message when you try to execute it.
As you already found out, there a multiple threads which discuss this issue. The only thing that eventually worked for me was to install docker-compose manually via python's packaga manager pip. Since this approach involves a couple of compilation steps, you need to add various additional OS packages and may face other mostly dependency related errors in the process. But the following steps eventually worked for me:
uname -s -m
1> Linux aarch64
cat /etc/system-release
1> Amazon Linux release 2 (Karoo)
sudo yum install -y python37 \
python3-devel.$(uname -m) libpython3.7-dev \
libffi-devel openssl-devel
# need gcc and friends
sudo yum groupinstall -y "Development Tools"
# make sure pip is up2date
sudo python3 -m pip install -U pip
python3 -m pip install docker-compose
docker-compose --version
1> docker-compose version 1.27.4, build unknown
Hope it works for you as well. Good Luck!
You can start from the scratch on Amazon Linux ec2 instance for installing Docker by following the step:
sudo yum update -y
sudo amazon-linux-extras install docker
sudo yum install docker
sudo service docker start
sudo usermod -a -G docker ec2-user
Then logout from the instance and login again to verify the installation of Docker
docker info
To install Docker-compose follow the below steps:
sudo curl -L https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
docker-compose version
I tried everything here mentioned, in the end, it was a problem with the symlink. The official docker page gave me the solution.
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
If it doesn't work after that command, check if you can find docker in your bin folder, e.g. with: ls /usr/local/bin/
If you can see docker-compose there, you almost made it.
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
An easy way to check if docker-compose is there and Linux able to find it, is to use `which docker-compose, also if it is linked correctly, you will get the path to docker-compose as a response.
This worked for me on a AWS EC2 instance with Linux2 as OS.
I faced the same issue. My configuration is T4Large - ARM64 - Ubuntu Server 20.04 LTS and I resolved using the following command
$ sudo apt install docker-compose
This will install the 1.25.0 version of docker compose which is a bit older. At the time of writing this answer the current version is 1.29.1
Another alternative is to use Docker Compose as Docker Container.
Use the following commands to run Docker Compose Container
$ sudo curl -L --fail https://raw.githubusercontent.com/linuxserver/docker-docker-compose/master/run.sh -o /usr/local/bin/docker-compose
$ sudo chmod +x /usr/local/bin/docker-compose
This should work on any Linux distro and on architecture like x86_64, armv7l/armhf, and aarch64/armv8/arm64 if you already have docker installed.
The important point to note is, this runs compose inside a container and does not require installing python on the host system.
Docker hub reference: https://hub.docker.com/r/linuxserver/docker-compose
Installing Docker-Compose in EC2 instance follow below steps
1st command: # sudo curl -L https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
2nd command: # sudo chmod +x /usr/local/bin/docker-compose
3rd command: # ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
4th Command to verify docker compose: # docker-compose version
It will install latest version of docker-compose. 3rd command is necessary.
This will do everything you need.
Cut. paste. done.
cat ec2-al2-docker-compose.sh
#the following script will install docker and docker compose on amazon linux 2
# run this script then
# sudo docker ps; docker-compose up
# note docker-compose up doesnt use sudo
sudo yum update -y
sudo yum install docker git -y
sudo usermod -a -G docker ec2-user
id ec2-user
newgrp docker
sudo yum install python3-pip -y
sudo pip3 install docker-compose
sudo systemctl enable docker.service
sudo systemctl start docker.service
I am trying to run docker-compose using sudo.
I have both docker and docker-compose installed on Ubuntu 16.01.
Due to an error while trying to download compose using curl, I ended up installing it using pip.
Docker version 1.12.0, build 8eab29e
docker-compose version 1.8.0, build 94f7016
Yet, when I try to run docker-compose with sudo I get the following (using sudo with docker is fine)
sudo: docker-compose: command not found
I suppose there are differing definitions of what 'installed' means. I have been using docker-compose on the same computer that claims it is not installed.
$ dpkg -s docker-compose
dpkg-query: package 'docker-compose' is not installed and no information is available
Use dpkg --info (= dpkg-deb --info) to examine archive files,
and dpkg --contents (= dpkg-deb --contents) to list their contents.
$ whereis docker-compose
docker-compose: /home/user/.local/bin/docker-compose
$ pip show --files docker-compose
---
Metadata-Version: 2.0
Name: docker-compose
Version: 1.8.0
Summary: Multi-container orchestration for Docker
Home-page: https://www.docker.com/
Author: Docker, Inc.
Author-email: UNKNOWN
Installer: pip
License: Apache License 2.0
Location: /home/anton/.local/lib/python2.7/site-packages
Requires: six, jsonschema, enum34, cached-property, websocket-client, docker-py, requests, docopt, dockerpty, PyYAML, texttable
Classifiers:
Development Status :: 5 - Production/Stable
Environment :: Console
Intended Audience :: Developers
License :: OSI Approved :: Apache Software License
Programming Language :: Python :: 2
Programming Language :: Python :: 2.7
Programming Language :: Python :: 3
Programming Language :: Python :: 3.4
Files:
../../../bin/docker-compose
compose/GITSHA
compose/__init__.py
compose/__init__.pyc
compose/__main__.py
compose/__main__.pyc
compose/bundle.py
compose/bundle.pyc
compose/cli/__init__.py
compose/cli/__init__.pyc
compose/cli/colors.py
compose/cli/colors.pyc
compose/cli/command.py
compose/cli/command.pyc
compose/cli/docker_client.py
compose/cli/docker_client.pyc
compose/cli/docopt_command.py
compose/cli/docopt_command.pyc
compose/cli/errors.py
compose/cli/errors.pyc
compose/cli/formatter.py
compose/cli/formatter.pyc
compose/cli/log_printer.py
compose/cli/log_printer.pyc
compose/cli/main.py
compose/cli/main.pyc
compose/cli/signals.py
compose/cli/signals.pyc
compose/cli/utils.py
compose/cli/utils.pyc
compose/cli/verbose_proxy.py
compose/cli/verbose_proxy.pyc
compose/config/__init__.py
compose/config/__init__.pyc
compose/config/config.py
compose/config/config.pyc
compose/config/config_schema_v1.json
compose/config/config_schema_v2.0.json
compose/config/environment.py
compose/config/environment.pyc
compose/config/errors.py
compose/config/errors.pyc
compose/config/interpolation.py
compose/config/interpolation.pyc
compose/config/serialize.py
compose/config/serialize.pyc
compose/config/sort_services.py
compose/config/sort_services.pyc
compose/config/types.py
compose/config/types.pyc
compose/config/validation.py
compose/config/validation.pyc
compose/const.py
compose/const.pyc
compose/container.py
compose/container.pyc
compose/errors.py
compose/errors.pyc
compose/network.py
compose/network.pyc
compose/parallel.py
compose/parallel.pyc
compose/progress_stream.py
compose/progress_stream.pyc
compose/project.py
compose/project.pyc
compose/service.py
compose/service.pyc
compose/state.py
compose/state.pyc
compose/utils.py
compose/utils.pyc
compose/volume.py
compose/volume.pyc
docker_compose-1.8.0.dist-info/DESCRIPTION.rst
docker_compose-1.8.0.dist-info/INSTALLER
docker_compose-1.8.0.dist-info/METADATA
docker_compose-1.8.0.dist-info/RECORD
docker_compose-1.8.0.dist-info/WHEEL
docker_compose-1.8.0.dist-info/entry_points.txt
docker_compose-1.8.0.dist-info/metadata.json
docker_compose-1.8.0.dist-info/pbr.json
docker_compose-1.8.0.dist-info/top_level.txt
Entry-points:
[console_scripts]
docker-compose=compose.cli.main:main
I have tried the following - but still get the same error:
$ chmod +x /home/username/.local/bin/docker-compose
$ chmod +x /home/username/.local/lib/python2.7/site-packages
On Ubuntu 16.04
Here's how I fixed this issue: Refer Docker Compose documentation
sudo curl -L https://github.com/docker/compose/releases/download/1.21.0/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
After you do the curl command , it'll put docker-compose into the
/usr/local/bin
which is not on the PATH.
To fix it, create a symbolic link:
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
And now if you do:
docker-compose --version
You'll see that docker-compose is now on the PATH
The output of dpkg -s ... demonstrates that docker-compose is not installed from a package. Without more information from you there are at least two possibilities:
docker-compose simply isn't installed at all, and you need to install it.
The solution here is simple: install docker-compose.
docker-compose is installed in your $HOME directory (or other location not on root's $PATH).
There are several solution in this case. The easiest is probably to replace:
sudo docker-compose ...
With:
sudo `which docker-compose` ...
This will call sudo with the full path to docker-compose.
You could alternatively install docker-compose into a system-wide directory, such as /usr/local/bin.
If you have tried installing via the official docker-compose v1 page, where you need to download the binary using curl:
sudo curl -L https://github.com/docker/compose/releases/download/1.29.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
Then do not forget to add executable flag to the binary:
sudo chmod +x /usr/local/bin/docker-compose
If docker-compose is installed using python-pip
sudo apt-get -y install python-pip
sudo pip install docker-compose
try using pip show --files docker-compose to see where it is installed.
If docker-compose is installed in user path, then try:
sudo "PATH=$PATH" docker-compose
As I see from your updated post, docker-compose is installed in user path /home/user/.local/bin and if this path is not in your local path $PATH, then try:
sudo "PATH=$PATH:/home/user/.local/bin" docker-compose
I have same issue , i solved issue :
Following installs Docker Compose v1:
step-1 : download docker-compose using following command.
1. sudo su
2. sudo curl -L https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
Step-2 : Run command
chmod +x /usr/local/bin/docker-compose
Step-3 : Check docker-compose version
docker-compose --version
I will leave this here as a possible fix, worked for me at least and might help others. Pretty sure this would be a linux only fix.
I decided to not go with the pip install and go with the github version (option one on the installation guide).
Instead of placing the copied docker-compose directory into /usr/local/bin/docker-compose from the curl/github command, I went with /usr/bin/docker-compose which is the location of Docker itself and will force the program to run in root. So it works in root and sudo but now won't work without sudo so the opposite effect which is what you want to run it as a user anyways.
If docker-compose is installed for your user but not installed for root user and if you need to run it only once and forget about it afterwords perform the next actions:
Find out path to docker-compose:
which docker-compose
Run the command specifying full path to docker-compose from the previous command, eg:
sudo /home/your-user/your-path-to-compose/docker-compose up
On Ubuntu just run
sudo apt install docker-compose
This worked on Ubuntu 22*
You might be using a deprecated code. Try writing docker compose instead of docker-compose.
Simple, just reinstall the docker-compose. It will fix the issue.
sudo apt install docker-compose
There are too many pending edits on Stack Overflow, so I'm submitting a new answer that supplements earlier answers.
I propose the answer is syntax: docker compose vs docker-compose depending on how you have it installed, which is the same proposed answer by ibrahim shazly on Sep 27, 2022 and Gaurav Kumar Verma on Sep 29, 2022.
Here is an explanation. I followed https://docs.docker.com/engine/install/ubuntu/#install-docker-engine to install Docker on Ubuntu 22 LTS. Then, I attempted to run docker compose like so:
$ docker-compose up -d myserver
Command 'docker-compose' not found, but can be installed with:
...
As Gaurav Kumar Verma points out, https://docs.docker.com/compose/install/other/ has a disclaimer:
Note that Compose standalone uses the -compose syntax instead of the current standard syntax compose.
For example type docker-compose up when using Compose standalone, instead of docker compose up.
Thus, I ran:
$ docker compose up -d myserver
And it ran successfully.
Or, just add your binary path into the PATH. At the end of the bashrc:
...
export PATH=$PATH:/home/user/.local/bin/
save the file and run:
source .bashrc
and the command will work.
docker-compose is not installed. Please try to install it first from their official documentation
Whenever this type of error occurs please try to verify the installation of the apps by typing which command_name it will search for the executables and show you the location.
Had to delete a large log file then restart docker-compose for the space to clear up.
I used a script shell that cron runs once a day:
rm /var/log/cron/log_prodTEST.log
cd /opt/test/metrics
/usr/local/bin/docker-compose up --build --force-recreate -d -T
Apperntly by using the full path to docker-compose, I could use it inside the corn shell
you might try run the code by using docker compose instead of docker-compose
if it work you might want to check this link her
https://docs.docker.com/compose/install/other/
sudo curl -L "https://github.com/docker/compose/releases/download/v2.12.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo mv /usr/local/bin/docker-compose /usr/bin/docker-compose
sudo chmod +x /usr/bin/docker-compose
Running the following docker command on mac works and on linux, running ubuntu cannot find the aws cli credentials. It returns the following message: Unable to locate credentials
Completed 1 part(s) with ... file(s) remaining
The command which runs an image and mounts a data volume and then copies a file from and s3 bucket, and starts the bash shell in the docker container.
sudo docker run -it --rm -v ~/.aws:/root/.aws username/docker-image sh -c 'aws s3 cp s3://bucketname/filename.tar.gz /home/emailer && cd /home/emailer && tar zxvf filename.tar.gz && /bin/bash'
What am I missing here?
This is my Dockerfile:
FROM ubuntu:latest
#install node and npm
RUN apt-get update && \
apt-get -y install curl && \
curl -sL https://deb.nodesource.com/setup | sudo bash - && \
apt-get -y install python build-essential nodejs
#install and set-up aws-cli
RUN sudo apt-get -y install \
git \
nano \
unzip && \
curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip" && \
unzip awscli-bundle.zip
RUN sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
# Provides cached layer for node_modules
ADD package.json /tmp/package.json
RUN cd /tmp && npm install
RUN mkdir -p /home/emailer && cp -a /tmp/node_modules /home/emailer/
Mounting $HOME/.aws/ into the container should work. Make sure to mount it as read-only.
It is also worth mentioning, if you have several profiles in your ~/.aws/config -- you must also provide the AWS_PROFILE=somethingsomething environment variable. E.g. via docker run -e AWS_PROFILE=xxx ... otherwise you'll get the same error message (unable to locate credentials).
Update: Added example of the mount command
docker run -v ~/.aws:/root/.aws …
You can use environment variable instead of copying ~/.aws/credentials and config file into container for aws-cli
docker run \
-e AWS_ACCESS_KEY_ID=AXXXXXXXXXXXXE \
-e AWS_SECRET_ACCESS_KEY=wXXXXXXXXXXXXY \
-e AWS_DEFAULT_REGION=us-west-2 \
<img>
Ref: AWS CLI Doc
what do you see if you run
ls -l ~/.aws/config
within your docker instance?
the only solution that worked for me in this case is:
volumes:
- ${USERPROFILE}/.aws:/root/.aws:ro
There are a few things that could be wrong. One, as mentioned previously you should check if your ~/.aws/config file is set accordingly. If not, you can follow this link to set it up. Once you have done that you can map the ~/.aws folder using the -v flag on docker run.
If your ~/.aws folder is mapped correctly, make sure to check the permissions on the files under ~/.aws so that they are able to be accessed safely by whatever process is trying to access them. If you are running as the user process, simply running chmod 444 ~/.aws/* should do the trick. This will give full read permissions to the file. Of course, if you want write permissions you can add whatever other modifiers you need. Just make sure the read octal is flipped for your corresponding user and/or group.
The issue I had was that I was running Docker as root. When running as root it was unable to locate my credentials at ~/.aws/credentials, even though they were valid.
Directions for running Docker without root on Ubuntu are here: https://askubuntu.com/a/477554/85384
You just have to pass the credential in order to be the AWS_PROFILE, if you do not pass anything it will use the default, but if you want you can copy the default and add your desired credentials.
In Your credentials
[profile_dev]
aws_access_key_id = xxxxxxxxxxxxxxxxxxxxxxxx
aws_secret_access_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
output = json
region = eu-west-1
In Your docker-compose
version: "3.8"
services:
cenas:
container_name: cenas_app
build: .
ports:
- "8080:8080"
environment:
- AWS_PROFILE=profile_dev
volumes:
- ~/.aws:/app/home/.aws:ro