I'm having a makefile which under proj root dir.
Folder proj is the main folder and there is folders such as ws-led or tools-ext etc under it which contains docker files.
In addition, there is also Makefile which is under the root that needs to run all the commands.
This is the folder structure
proj
- ws-led
— Dockerfile
- tools-ext
— Dockerfile
- Makefile
What I need is to cd to each of the folders under the rot (we have many more) and run:
docker build <folder name> .
Example: ( exactly like running the following command manually )
cd ws-led
docker build -t ws-led .
cd tools-ext
docker build -t tools-ext .
I try with the following (maybe instead of repo param I get run on all the folders in the same level of the Makefile )
Like (CURDIR) …
all: pre docker-build
.PHONY: pre docker-build
repos := ws-led tools-ext
pre:
$(patsubst %,docker-build,$(repos))
docker-build:pre
cd $*; docker build -t $* . >&2 | tee docker-build
while using this im getting an error:
invalid argument "." for "-t, --tag" flag: invalid reference format
Any idea what is wrong here ? or i could do it better?
As I've many repos/folders I want to use make to handle it
There's more than one way to do it.
You could use a bash for loop:
docker-build:
for dir in $(repos); do cd $$dir; docker build -t $$dir . >&2 | tee docker-build; done
Or use a pattern rule (or in this case a static pattern rule):
REPO_BUILDS := $(addsuffix -build, $(repos))
docker-build: $(REPO_BUILDS)
.PHONY: $(REPO_BUILDS)
$(REPO_BUILDS): %-build:
cd $*; docker build -t $* . >&2 | tee docker-build
Related
I have an interesting setup with my project where I have a directory structure like this:
script.sh
scripts/
create-docker-container.sh
...
src/
...
The idea is that you can run any script from the root directory by proxying the commands through the script.sh script which does the following:
#!/bin/bash
FOUND_SCRIPT=`ls scripts | grep ^$1$`;
if [ "$FOUND_SCRIPT" != "$1" ]; then
echo "Could not find script: $1";
echo "Available...";
ls scripts;
exit 1;
fi
TMPFILE=`mktemp tmp.XXXX`
cp ./scripts/$1 "$TMPFILE" && chmod +x "$TMPFILE";
function cleanup {
rm "$TMPFILE";
}
trap cleanup EXIT
trap cleanup SIGINT
shift;
"./$TMPFILE" "$#";
This normally works fine, however my create-docker-container.sh script isn't working anymore, I get the error:
error checking context: file ('/home/circleci/project/tmp.MK8H') not found or excluded by .dockerignore
My dockerignore looks like
tmp.*
I'm not really sure why this script suddenly started failing with the above error. I assume its because the docker script itself is running from tmp.XXX, which at some point is deleted by script.sh, and it then fails for some sort of context switching issue.
I hope someone with more knowledge about docker can help me.
Thanks :)
I have tried modifying the .dockerignore and removing the cleanup step, but neither have worked.
EDIT:
Here is my dockerfile:
WORKDIR /usr/src/app
COPY . /usr/src/app
CMD [ "./script", "run", "${serviceName}"]
And here is the script to deploy the container:
function cleanup {
./script temp-install-restore
}
trap cleanup EXIT
BUNDLE_FILE=$(node -e "console.log(require(\"./services.json\")[\"$1\"].services[0][\"emit-point\"])")/main.js
./script build $1 >/dev/null;
./script temp-install $(cat "$BUNDLE_FILE" | grep -o 'require("[^"]*")')
./script generate-dockerfile $1 | docker build --no-cache -q -f - .
Makefile:
# Defines
BUILD_PATH ?= out
REPO_NAME ?= my_work_dir
REPO_URL ?= git#github.com:your_org/your_repo
REPO_BRANCH := main
### Functions
define clone_repository
echo Cloning repository...
git -C $(1) init --quiet
git -C $(1) remote add origin $(2)
git -C $(1) fetch origin --progress --quiet --depth 1 $(3)
git -C $(1) reset --quiet --hard FETCH_HEAD
endef
define get_sha1
$(2):=$(shell git -C $(1) rev-parse HEAD)
endef
### Targets
do_the_work:
# Prepare
rm -rf $(BUILD_PATH)/$(REPO_NAME)
mkdir -p $(BUILD_PATH)/$(REPO_NAME)
# Clone
$(call clone_repository,$(BUILD_PATH)/$(REPO_NAME),$(REPO_URL),$(REPO_BRANCH))
# SHA1
$(eval $(call get_sha1,$(BUILD_PATH)/$(REPO_NAME),REPO_BRANCH_SHA1))
# Do...
# do_something --sha1 REPO_BRANCH_SHA1
What I tried to do in the do_the_work is:
Step 1: create an empty dir
Step 2: clone repository
Step 3: get sha1 from repository
Step 4: do something with sha1 info
However, when I execute the do_the_work, I get error (please not that $(BUILD_PATH)/$(REPO_NAME) evaluates to out/my_work_dir):
fatal: cannot change to 'out/my_work_dir': No such file or directory
rm -rf out/my_work_dir
mkdir -p out/my_work_dir
echo Cloning repository...
.
.
.
etc
But, when I run the same command again, it executes OK! Is this because the out/my_work_dir is in place already? It also works if the dir is completely empty.
Seems like the Step 3 $(eval $(call get_sha1...) executed before the Step 1 mkdir finished? How do I fix this?
To me it seems that rm -rf gets prolonged if the dir does not exist at all.
The error is in the $(shell ...) invocation in define get_sha1 which runs when you define the function, not when you call it.
This seems like an overcomplication anyway; I would simply get rid of the define and replace the recipe with one where the result is stored in a shell variable.
do_the_work:
...
# SHA1
sha1=$$(git -C $(BUILD_PATH)/$(REPO_NAME) rev-parse HEAD); \
# Do...
# do_something --sha1 "$$sha1"
I have directory structure like this
containers/con1
containers/con2
containers/con3
Now every folder like con1, con2 has Makefile in it with targets like build, run
I run it like make run and make build
But i have to go inside that folder.
Is it possible that i have another Makefile in containers/Makefile
and i can run like
Make con1.run Make con2.run
Yes, you can do that. Something like the following should do what you want.
$ cat containers/Makefile
%.run: %
$(MAKE) -C $#
That being said as you can see the command to do what you want is trivial enough to make such a makefile not really necessary (and a simple shell script is as useful here as a makefile).
$ cat run.sh
[ -d "$1" ] || { echo 'No such directory.' >&2; exit 1; }
#make -C "$1"
# OR
#cd "$1" && make
If you wanted to be able to build all the sub-directory projects at once then a makefile could help you with that but even that is a simple enough shell one-liner.
$ for mkfile in */Makefile; do make -C "$(dirname "$mkfile"); done
$ for mkfile in */Makefile; do (cd "$(dirname "$mkfile") && make); done
As far as I understand you want this:
-C dir, --directory=dir
Change to directory dir before reading the makefiles or doing anything else. If multiple -C options are specified, each is interpreted relative to the previous one: -C / -C etc is equivalent to -C /etc. This is typi‐
cally used with recursive invocations of make.
Add -C option like this: make -C con1/
Recursive makes are evil, but if you want that:
# con1.run is a phony target...
.PHONY: con1.run
con1.run:
$(MAKE) -C con1
I have this makefile tha sthould download and build openssh (along with other things):
ROOT_DIR=$(PWD)
DATA_DIR=$(ROOT_DIR)/data
SOURCES_DIR=$(ROOT_DIR)/sources
RESOURCES_DIR=$(ROOT_DIR)/resources
DRAFTS_DIR=$(ROOT_DIR)/drafts
$(SOURCES_DIR):
mkdir $(SOURCES_DIR)
$(RESOURCES_DIR):
mkdir $(RESOURCES_DIR)
$(DRAFTS_DIR):
mkdir $(DRAFTS_DIR)
openssh-tar-url="ftp://ftp.cc.uoc.gr/mirrors/OpenBSD/OpenSSH/portable/openssh-6.2p2.tar.gz"
TAR_PROJECTS += openssh
openssh:
echo "Building $#"
openssh-clean: openssh-archive-clean
.SECONDEXPANSION :
$(TAR_PROJECTS) : $(SOURCES_DIR) $(SOURCES_DIR)/$$#-archive
$(DRAFTS_DIR)/%.tar.gz: $(DRAFTS_DIR)
echo "Pulling $*."
wget $($*-tar-url) -O $(DRAFTS_DIR)/$*.tar.gz
.SECONDEXPANSION :
$(SOURCES_DIR)/%-archive : | $(DRAFTS_DIR)/$$*.tar.gz
mkdir $#
cd $# && tar xvzf $(DRAFTS_DIR)/$*.tar.gz
%-archive-clean:
rm -rf $(SOURCES_DIR)/$*-archive $(DRAFTS_DIR)/$*.tar.gz
When i run make openssh it runs correctly but at the end it removes the archive it downloaded. This is very strange to me:
$ make openssh --just-print
echo "Pulling openssh."
wget "ftp://ftp.cc.uoc.gr/mirrors/OpenBSD/OpenSSH/portable/openssh-6.2p2.tar.gz" -O /home/fakedrake/Projects/ThinkSilicon/xilinx-zynq-bootstrap/drafts/openssh.tar.gz
mkdir /home/fakedrake/Projects/ThinkSilicon/xilinx-zynq-bootstrap/sources/openssh-archive
cd /home/fakedrake/Projects/ThinkSilicon/xilinx-zynq-bootstrap/sources/openssh-archive && tar xvzf /home/fakedrake/Projects/ThinkSilicon/xilinx-zynq-bootstrap/drafts/openssh.tar.gz
echo "Building openssh"
rm /home/fakedrake/Projects/ThinkSilicon/xilinx-zynq-bootstrap/drafts/openssh.tar.gz
Pretty sure you can list targets (and intermediates) as .PRECIOUS to avoid them being deleted for you. I'm afraid you'll need to RTFM for more details - I'm in visual studio rather than make these days, so my make skills are a bit rusty...
I'm trying to compile Quarter and package it using checkinstall.
If I do the standard ./configure && make && sudo make install, things go fine.
$ wget http://ftp.coin3d.org/coin/src/all/Quarter-1.0.0.tar.gz
$ tar xzf Quarter-1.0.0.tar.gz
$ cd Quarter-1.0.0
$ ./configure
$ make
$ sudo make install
But when I use checkinstall, it fails on a mkdir -p that should work perfectly fine. The way it fails is exactly how it would as if the -p option weren't given. This is the checkinstall command line I'm using:
$ checkinstall -D -y --install=no --pkgname=libquarter --pkgversion=1.0.0 \
--arch=i386 --pkglicense=GPL --maintainer=me#example.com --reset-uids=yes
This is the failure:
....
/bin/bash ../../../cfg/mkinstalldirs /usr/local/include/Quarter/devices
mkdir -p -- /usr/local/include/Quarter/devices
mkdir: cannot create directory `/usr/local/include/Quarter': No such file or directory
make[4]: *** [install-libdevicesincHEADERS] Error 1
....
This is the relevant part of the script:
$ cat cfg/mkinstalldirs
....
case $dirmode in
'')
if mkdir -p -- . 2>/dev/null; then
echo "mkdir -p -- $*"
exec mkdir -p -- "$#"
fi
;;
....
I don't understand why that exec is there -- doesn't that guarantee that the remainder of the script (after the esac) will never execute? (If the if test passes, then the script assumes mkdir -p works correctly, so once it does the real mkdir -p it can quit; otherwise the remainder of the script implements proper mkdir -p behavior.) I also don't understand why it uses "$*" in the echo and "$#" in the next line, but it doesn't seem to matter -- they're both the same thing since this script is being called with just one argument. (Tom explained in comment.)
If I add two lines between echo and exec that does mkdir -p -- "$#" and then echo "Now doing the exec mkdir..." then it works like this -- better, but still bewildering:
/bin/bash ../../../cfg/mkinstalldirs /usr/local/include/Quarter/devices
mkdir -p -- /usr/local/include/Quarter/devices
mkdir: cannot create directory `/usr/local/include/Quarter': No such file or directory
Now doing the exec mkdir...
/usr/bin/install -c -m 644 InputDevice.h /usr/local/include/Quarter/devices/InputDevice.h
.... finishes successfully!
Now, the fact that doing the mkdir line twice made it work tells me it's not a permissions issue (beside, that would generate a different diagnostic from mkdir, and this is being run as sudo, and it's actually working in /var/tmp/... not the real /usr/local/...). I think what's happening is that the first mkdir invocation (the one I added) is actually creating just the Quarter directory and bailing out, and then when the second mkdir runs, it's able to create the devices subdirectory, because the Quarter directory is already there. But why would mkdir work that way???
My workaround is to patch that mkinstalldirs script somehow, but I'm really curious why this is breaking!
This is a Ubuntu 10.10 guest running in VirtualBox on Win7, checkinstall version 1.6.2 installed thru apt-get.
EDIT:
I did some testing to see what works and what fails in this environment...
mkdir -p /foo works correctly
mkdir -p /foo && mkdir -p /foo/bar works correctly
mkdir -p foo/bar works correctly
mkdir /foo/bar failed as expected (correct)
mkdir foo/bar failed as expected (correct)
mkdir -p /foo/bar fails
Weird that -p works for relative pathnames but not for absolute pathnames. Or maybe the correct distinction is that -p works outside of the "chroot" tree (if it's even really using chroot) but not within it.
I also verified that despite the failure, it is able to create the first directory level.
Still a mystery.
Using
checkinstall --fstrans=no
should fix this.
Or
Set "TRANSLATE=0"
in /etc/checkinstallrc and try again.
mkdir -p isn't working like it should because it's a checkinstall version of mkdir, not the "true" mkdir. Must be some bug in checkinstall that makes it work a bit differently.
This patch works around the bug:
./configure
sed -i 's/if mkdir .*-p --.*; then/if false; then ## &/' cfg/mkinstalldirs
....
sed -i -e 's/TRANSLATE=1/TRANSLATE=0/g' /etc/checkinstallrc