AWS: C compiler on EC2 not working when using Elastic Beanstalk container_commands - linux

Question

I've been trying to install some binaries in my EC2 instances using Elastic Beanstalk container_commands and a custom shell script. When the container command fires, I basically wget a library, uncompress it, configure it and make && make install it like so:
wget LIBRARY_URL
mkdir -p libfolder
tar xf LIBRARY.tar.gz -C libfolder --strip-components 1
cd libfolder
./configure
make clean
make install
On the ./configure step I get the following:
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... no
So I get from here that the C compiler doesn't work yet gcc is already installed. The whole container_command fails to do what the shell script says.
Nevertheless, when is SSH into the EC2 instance, if I manually execute the shell script then everything goes smoothly and the "C compiler works..." check returns "yes".
Here is the according .ebextensions config YAML file that I use:
packages:
yum:
gcc: []
make: []
openssl-devel: []
libxml2: []
libxml2-devel: []
container_commands:
01setupaudio:
command: "sh audio.sh"
I am wondering why this is and if the reason this happens is because container_commands run first and then packages get installed, because the fact that SSHing into the instance and running the shell script manually confirms that there is no problem with it.
Any help would be much appreciated. Here is the config.log:
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
It was created by lame configure 3.99.5, which was
generated by GNU Autoconf 2.68. Invocation command line was
$ ./configure CC=/usr/bin/gcc
## --------- ##
## Platform. ##
## --------- ##
hostname = ip-10-0-1-182
uname -m = x86_64
uname -r = 3.10.35-43.137.amzn1.x86_64
uname -s = Linux
uname -v = #1 SMP Wed Apr 2 09:36:59 UTC 2014
/usr/bin/uname -p = unknown
/bin/uname -X = unknown
/bin/arch = x86_64
/usr/bin/arch -k = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo = unknown
/bin/machine = unknown
/usr/bin/oslevel = unknown
/bin/universe = unknown
PATH: /usr/local/bin
PATH: /bin
PATH: /usr/bin
## ----------- ##
## Core tests. ##
## ----------- ##
configure:2555: checking build system type
configure:2569: result: x86_64-unknown-linux-gnu
configure:2589: checking host system type
configure:2602: result: x86_64-unknown-linux-gnu
configure:2639: checking for a BSD-compatible install
configure:2707: result: /usr/bin/install -c
configure:2718: checking whether build environment is sane
configure:2768: result: yes
configure:2909: checking for a thread-safe mkdir -p
configure:2948: result: /bin/mkdir -p
configure:2961: checking for gawk
configure:2977: found /bin/gawk
configure:2988: result: gawk
configure:2999: checking whether make sets $(MAKE)
configure:3021: result: yes
configure:3104: checking whether to enable maintainer-specific portions of Makefiles
configure:3113: result: no
configure:3133: checking for style of include used by make
configure:3161: result: GNU
configure:3237: checking for gcc
configure:3264: result: /usr/bin/gcc
configure:3493: checking for C compiler version
configure:3502: /usr/bin/gcc --version >&5
gcc (GCC) 4.8.2 20131212 (Red Hat 4.8.2-7)
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
configure:3513: $? = 0
configure:3502: /usr/bin/gcc -v >&5
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-amazon-linux/4.8.2/lto-wrapper
Target: x86_64-amazon-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,fortran,ada,lto --enable-plugin --enable-initfini-array --disable-libgcj --without-isl --without-cloog --with-tune=generic --with-arch_32=i686 --build=x86_64-amazon-linux
Thread model: posix
gcc version 4.8.2 20131212 (Red Hat 4.8.2-7) (GCC)
configure:3513: $? = 0
configure:3502: /usr/bin/gcc -V >&5
gcc: error: unrecognized command line option '-V'
gcc: fatal error: no input files
compilation terminated.
configure:3513: $? = 4
configure:3502: /usr/bin/gcc -qversion >&5
gcc: error: unrecognized command line option '-qversion'
gcc: fatal error: no input files
compilation terminated.
configure:3513: $? = 4
configure:3533: checking whether the C compiler works
configure:3555: /usr/bin/gcc conftest.c >&5
collect2: fatal error: cannot find 'ld'
compilation terminated.
configure:3559: $? = 1
configure:3597: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "lame"
| #define PACKAGE_TARNAME "lame"
| #define PACKAGE_VERSION "3.99.5"
| #define PACKAGE_STRING "lame 3.99.5"
| #define PACKAGE_BUGREPORT "lame-dev#lists.sf.net"
| #define PACKAGE_URL ""
| #define PACKAGE "lame"
| #define VERSION "3.99.5"
| /* end confdefs.h. */
|
| int
| main ()
| {
|
| ;
| return 0;
| }
configure:3602: error: in `/tmp/deployment/application/soundpuzzle_installation/lame-folder/lame-tar':
configure:3604: error: C compiler cannot create executables
See `config.log' for more details
## ---------------- ##
## Cache variables. ##
## ---------------- ##
ac_cv_build=x86_64-unknown-linux-gnu
ac_cv_env_CC_set=set
ac_cv_env_CC_value=/usr/bin/gcc
ac_cv_env_CFLAGS_set=
ac_cv_env_CFLAGS_value=
ac_cv_env_CPPFLAGS_set=
ac_cv_env_CPPFLAGS_value=
ac_cv_env_CPP_set=
ac_cv_env_CPP_value=
ac_cv_env_LDFLAGS_set=
ac_cv_env_LDFLAGS_value=
ac_cv_env_LIBS_set=
ac_cv_env_LIBS_value=
ac_cv_env_PKG_CONFIG_LIBDIR_set=
ac_cv_env_PKG_CONFIG_LIBDIR_value=
ac_cv_env_PKG_CONFIG_PATH_set=
ac_cv_env_PKG_CONFIG_PATH_value=
ac_cv_env_PKG_CONFIG_set=
ac_cv_env_PKG_CONFIG_value=
ac_cv_env_SNDFILE_CFLAGS_set=
ac_cv_env_SNDFILE_CFLAGS_value=
ac_cv_env_SNDFILE_LIBS_set=
ac_cv_env_SNDFILE_LIBS_value=
ac_cv_env_build_alias_set=
ac_cv_env_build_alias_value=
ac_cv_env_host_alias_set=
ac_cv_env_host_alias_value=
ac_cv_env_target_alias_set=
ac_cv_env_target_alias_value=
ac_cv_host=x86_64-unknown-linux-gnu
ac_cv_path_install='/usr/bin/install -c'
ac_cv_path_mkdir=/bin/mkdir
ac_cv_prog_AWK=gawk
ac_cv_prog_ac_ct_CC=/usr/bin/gcc
ac_cv_prog_make_make_set=yes
## ----------------- ##
## Output variables. ##
## ----------------- ##
ACLOCAL='${SHELL} /tmp/deployment/application/soundpuzzle_installation/lame-folder/lame-tar/missing --run aclocal-1.11'
ALLOCA=''
AMDEPBACKSLASH='\'
AMDEP_FALSE='#'
AMDEP_TRUE=''
AMTAR='${SHELL} /tmp/deployment/application/soundpuzzle_installation/lame-folder/lame-tar/missing --run tar'
ANSI2KNR=''
AR=''
AUTOCONF='${SHELL} /tmp/deployment/application/soundpuzzle_installation/lame-folder/lame-tar/missing --run autoconf'
AUTOHEADER='${SHELL} /tmp/deployment/application/soundpuzzle_installation/lame-folder/lame-tar/missing --run autoheader'
AUTOMAKE='${SHELL} /tmp/deployment/application/soundpuzzle_installation/lame-folder/lame-tar/missing --run automake-1.11'
AWK='gawk'
CC='/usr/bin/gcc'
CCDEPMODE=''
CFLAGS=''
CONFIG_DEFS=''
CONFIG_MATH_LIB=''
CPP=''
CPPFLAGS=''
CPUCCODE=''
CPUTYPE=''
CYGPATH_W='echo'
DEFS=''
DEPDIR='.deps'
DSYMUTIL=''
DUMPBIN=''
ECHO_C=''
ECHO_N='-n'
ECHO_T=''
EGREP=''
EXEEXT=''
FGREP=''
FRONTEND_CFLAGS=''
FRONTEND_LDADD=''
FRONTEND_LDFLAGS=''
GREP=''
GTK_CFLAGS=''
GTK_CONFIG=''
GTK_LIBS=''
HAVE_NASM_FALSE=''
HAVE_NASM_TRUE=''
INCLUDES=''
INSTALL_DATA='${INSTALL} -m 644'
INSTALL_PROGRAM='${INSTALL}'
INSTALL_SCRIPT='${INSTALL}'
INSTALL_STRIP_PROGRAM='$(install_sh) -c -s'
LD=''
LDADD=''
LDFLAGS=''
LIBICONV=''
LIBOBJS=''
LIBS=''
LIBTOOL=''
LIBTOOL_DEPS=''
LIB_MAJOR_VERSION=''
LIB_MINOR_VERSION=''
LIB_WITH_DECODER_FALSE=''
LIB_WITH_DECODER_TRUE=''
LIPO=''
LN_S=''
LTLIBICONV=''
LTLIBOBJS=''
MAINT='#'
MAINTAINER_MODE_FALSE=''
MAINTAINER_MODE_TRUE='#'
MAKEDEP=''
MAKEINFO='${SHELL} /tmp/deployment/application/soundpuzzle_installation/lame-folder/lame-tar/missing --run makeinfo'
MKDIR_P='/bin/mkdir -p'
NASM=''
NASM_FORMAT=''
NM=''
NMEDIT=''
OBJDUMP=''
OBJEXT=''
OTOOL64=''
OTOOL=''
PACKAGE='lame'
PACKAGE_BUGREPORT='lame-dev#lists.sf.net'
PACKAGE_NAME='lame'
PACKAGE_STRING='lame 3.99.5'
PACKAGE_TARNAME='lame'
PACKAGE_URL=''
PACKAGE_VERSION='3.99.5'
PATH_SEPARATOR=':'
PKG_CONFIG=''
PKG_CONFIG_LIBDIR=''
PKG_CONFIG_PATH=''
RANLIB=''
RM_F=''
SED=''
SET_MAKE=''
SHELL='/bin/sh'
SNDFILE_CFLAGS=''
SNDFILE_LIBS=''
STRIP=''
U=''
VERSION='3.99.5'
WITH_FRONTEND=''
WITH_MP3RTP=''
WITH_MP3X=''
WITH_VECTOR_FALSE=''
WITH_VECTOR_TRUE=''
WITH_XMM_FALSE=''
WITH_XMM_TRUE=''
ac_ct_CC='/usr/bin/gcc'
ac_ct_DUMPBIN=''
am__EXEEXT_FALSE=''
am__EXEEXT_TRUE=''
am__fastdepCC_FALSE=''
am__fastdepCC_TRUE=''
am__include='include'
am__isrc=''
am__leading_dot='.'
am__quote=''
am__tar='${AMTAR} chof - "$$tardir"'
am__untar='${AMTAR} xf -'
bindir='${exec_prefix}/bin'
build='x86_64-unknown-linux-gnu'
build_alias=''
build_cpu='x86_64'
build_os='linux-gnu'
build_vendor='unknown'
datadir='${datarootdir}'
datarootdir='${prefix}/share'
docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
dvidir='${docdir}'
exec_prefix='NONE'
host='x86_64-unknown-linux-gnu'
host_alias=''
host_cpu='x86_64'
host_os='linux-gnu'
host_vendor='unknown'
htmldir='${docdir}'
includedir='${prefix}/include'
infodir='${datarootdir}/info'
install_sh='${SHELL} /tmp/deployment/application/soundpuzzle_installation/lame-folder/lame-tar/install-sh'
libdir='${exec_prefix}/lib'
libexecdir='${exec_prefix}/libexec'
localedir='${datarootdir}/locale'
localstatedir='${prefix}/var'
mandir='${datarootdir}/man'
mkdir_p='/bin/mkdir -p'
oldincludedir='/usr/include'
pdfdir='${docdir}'
prefix='NONE'
program_transform_name='s,x,x,'
psdir='${docdir}'
sbindir='${exec_prefix}/sbin'
sharedstatedir='${prefix}/com'
sysconfdir='${prefix}/etc'
target_alias=''
## ----------- ##
## confdefs.h. ##
## ----------- ##
/* confdefs.h */
#define PACKAGE_NAME "lame"
#define PACKAGE_TARNAME "lame"
#define PACKAGE_VERSION "3.99.5"
#define PACKAGE_STRING "lame 3.99.5"
#define PACKAGE_BUGREPORT "lame-dev#lists.sf.net"
#define PACKAGE_URL ""
#define PACKAGE "lame"
#define VERSION "3.99.5"
configure: exit 77

Answer 1

I was able to fix the issue by calling the shell script via commands: instead of container_commands:

Answer 2

Looks like Elastic Beanstalk container_commands executed without PATH and SHELL environment variables. This is what I got in /var/log/cfn-init-cmd.log
[INFO] Command env
[INFO] -----------------------Command Output-----------------------
[INFO] RDS_HOSTNAME=******
[INFO] RDS_DB_NAME=*****
[INFO] PHP_ALLOW_URL_FOPEN=On
[INFO] PHP_DOCUMENT_ROOT=
[INFO] RDS_PASSWORD=*****
[INFO] RDS_USERNAME=root
[INFO] PHP_MEMORY_LIMIT=256M
[INFO] PWD=/var/app/ondeck
[INFO] PHP_MAX_EXECUTION_TIME=60
[INFO] PHP_DISPLAY_ERRORS=Off
[INFO] SHLVL=1
[INFO] EB_IS_COMMAND_LEADER=true
[INFO] PHP_COMPOSER_OPTIONS=
[INFO] PHP_DATE_TIMEZONE=UTC
[INFO] PHP_ZLIB_OUTPUT_COMPRESSION=Off
[INFO] RDS_PORT=5432
[INFO] _=/bin/env
Easy solution is to use shell wrapper that will reset needed variables to their default values.
scripts/env_exec
#!/bin/bash
export SHELL="${SHELL:-default}"
export PATH="${PATH:-default}"
exec $#
And launch needed scripts through it:
container_commands:
env:
command: "env"
env_exec:
command: "scripts/env_exec env"
compile_stuff:
command: "scripts/env_exec scripts/compile_stuff.sh"
Result:
[INFO] Command env_exec
[INFO] -----------------------Command Output-----------------------
[INFO] RDS_HOSTNAME=******
[INFO] RDS_DB_NAME=*****
[INFO] SHELL=/bin/bash
[INFO] PHP_ALLOW_URL_FOPEN=On
[INFO] RDS_PASSWORD=*****
[INFO] PHP_DOCUMENT_ROOT=
[INFO] RDS_USERNAME=root
[INFO] PHP_MEMORY_LIMIT=256M
[INFO] PATH=/usr/local/bin:/bin:/usr/bin
[INFO] PWD=/var/app/ondeck
[INFO] PHP_MAX_EXECUTION_TIME=60
[INFO] PHP_DISPLAY_ERRORS=Off
[INFO] SHLVL=1
[INFO] PHP_COMPOSER_OPTIONS=
[INFO] EB_IS_COMMAND_LEADER=true
[INFO] PHP_DATE_TIMEZONE=UTC
[INFO] PHP_ZLIB_OUTPUT_COMPRESSION=Off
[INFO] RDS_PORT=5432