I'm trying to build the latest version of glibc (2.22). I've not modified any sources of glibc. On my x86_64 Ubuntu 14.04.1 machine I'm using the following extract of a makefile to build:
HOST ?= x86_64-linux-gnu
TARGET ?= x86_64-linux-gnu
CROSS_OUT = $(shell pwd)/$(TARGET)
CC ?= gcc
CXX ?= g++
LD ?= ld
[...]
CFLAGS ?= "-I$(CROSS_OUT)/include -L$(CROSS_OUT)/lib"
CXXFLAGS ?= "-I$(CROSS_OUT)/include -L$(CROSS_OUT)/lib"
CPPFLAGS ?= "-I$(CROSS_OUT)/include -L$(CROSS_OUT)/lib"
LDFLAGS ?= "-I$(CROSS_OUT)/include -L$(CROSS_OUT)/lib"
[...]
GLIBC_PATH=$(shell pwd)/glibc
GLIBC_BUILD_PATH=$(shell pwd)/glibc-build
glibc: glibc-clean
mkdir -p $(GLIBC_BUILD_PATH)
cd $(GLIBC_BUILD_PATH) && \
CC=$(CC) \
CXX=$(CXX) \
LD=$(LD) \
CFLAGS=$(CFLAGS) \
CXXFLAGS=$(CXXFLAGS) \
CPPFLAGS=$(CPPFLAGS) \
LDFLAGS=$(LDFLAGS) \
$(GLIBC_PATH)/configure \
--host=$(TARGET) \
--build=$(HOST) \
--prefix=$(CROSS_OUT) \
--disable-shared \
--enable-add-ons \
--enable-static-nss && \
make && \
make install
glibc-clean:
rm -r -f $(GLIBC_BUILD_PATH)
Make stops nearly immediatly with the following error:
In file included from <command-line>:0:0:
../include/stdc-predef.h:64:1: fatal error: /home/leon/reaper/glibc-build/libc-modules.h: No such file or directory
#endif
^
compilation terminated.
Unfortunately, the file definitely doesn't exist.
In met the same error when I used --disable-shared option for configure.
When I removed the option, the build passed correctly.
I also use -fno-stack-protector -U_FORTIFY_SOURCE in CFLAGS according to glibc FAQ (my OS is Debian).
Also this topic was helpful to set up environment:
https://lists.debian.org/debian-user/2015/07/msg00120.html
This is my script for build glibc:
#!/bin/bash
# sudo aptitude install linux-headers-$(uname -r)
# sudo aptitude install build-essentials
# sudo aptitude install gawk
export CFLAGS="-fPIC -O2 -fno-stack-protector -U_FORTIFY_SOURCE"
mkdir glibc-build
cd glibc-build
../glibc-2.23/configure --disable-werror --prefix=/home/alexey/projects/work/build-dir/glibc-prefix
make
Try installing these:
$ sudo apt-get install build-essential
$ sudo apt-get install libc6
And link from where you got that source code please.
If you want to build an alien libc6 / an extra libc6, the build of glibc-2.22 is described here http://www.linuxfromscratch.org/lfs/view/stable/chapter05/glibc.html
Ref. http://www.linuxfromscratch.org/lfs/view/stable/
Suggest : --prefix=/opt/glibc222
The header 'libc-modules.h' is a generated header. Appears when configuring is done the right way.
glibc222
Related
I have an alpine running container which contains some binaries in usr/local/bin
When I ls the content of usr/local/bin I got this output :
/usr/local/bin # ls
dwg2SVG dwg2dxf dwgadd dwgbmp dwgfilter dwggrep dwglayers dwgread dwgrewrite dwgwrite dxf2dwg dxfwrite
Which is what I expected.
However if I execute one of these binaries by calling it I got a not found error from the shell :
/usr/local/bin # dwg2dxf
sh: dwgread: not found
/usr/local/bin # ./dwg2dxf
sh: ./dwgread: not found
I tested my $PATH which seems to be correct :
/usr/local/bin # echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
How can I make those binaries callable or "foundable" ? Did I miss something in my Dockerfile build ?
I suppose that there is something with the ldconfig command in alpine that went wrong but I'm not sure.
EDIT
As suggested in one answer here I executed the file command and here is the output :
/usr/local/bin # file dwg2dxf
dwgread: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=7835d4a42651a5fb7bdfa2bd8a76e40096bacb07, with debug_info, not stripped
Thoses binaries are from the LibreDWG Official repository as well as the first part of my Dockerfile. Here is the complete Dockerfile :
# podman/docker build -t libredwg .
############################
# STEP 1 build package from latest tar.xz
############################
FROM python:3.7.7-buster AS extracting
# libxml2-dev is broken so we need to compile it by our own
ARG LIBXML2VER=2.9.9
RUN apt-get update && \
apt-get install -y --no-install-recommends autoconf libtool swig texinfo \
build-essential gcc libxml2 python3-libxml2 libpcre2-dev libpcre2-32-0 curl \
libperl-dev libxml2-dev && \
mkdir libxmlInstall && cd libxmlInstall && \
wget ftp://xmlsoft.org/libxml2/libxml2-$LIBXML2VER.tar.gz && \
tar xf libxml2-$LIBXML2VER.tar.gz && \
cd libxml2-$LIBXML2VER/ && \
./configure && \
make && \
make install && \
cd /libxmlInstall && \
rm -rf gg libxml2-$LIBXML2VER.tar.gz libxml2-$LIBXML2VER
WORKDIR /app
RUN tarxz=`curl --silent 'https://ftp.gnu.org/gnu/libredwg/?C=M;O=D' | grep '.tar.xz<' | \
head -n1|sed -E 's/.*href="([^"]+)".*/\1/'`; \
echo "latest release $tarxz"; \
curl --silent --output "$tarxz" https://ftp.gnu.org/gnu/libredwg/$tarxz && \
mkdir libredwg && \
tar -C libredwg --xz --strip-components 1 -xf "$tarxz" && \
rm "$tarxz" && \
cd libredwg && \
./configure --disable-bindings --enable-release && \
make -j `nproc` && \
mkdir install && \
make install DESTDIR="$PWD/install" && \
make check DOCKER=1 DESTDIR="$PWD/install"
############################
# STEP 2 install into stable-slim
############################
# pull official base image
FROM osgeo/gdal:alpine-normal-latest
# set work directory
WORKDIR /usr/src/app
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# copy requirements file
COPY ./requirements.txt /usr/src/app/requirements.txt
# install dependencies
RUN set -eux \
&& apk add --no-cache --virtual .build-deps build-base \
py3-pip libressl-dev libffi-dev gcc musl-dev python3-dev postgresql-dev\
&& pip3 install --upgrade pip setuptools wheel \
&& pip3 install -r /usr/src/app/requirements.txt \
&& rm -rf /root/.cache/pip
# Install libredwg binaries
COPY --from=extracting /app/libredwg/install/usr/local/bin/* /usr/local/bin/
COPY --from=extracting /app/libredwg/install/usr/local/include/* /usr/local/include/
COPY --from=extracting /app/libredwg/install/usr/local/lib/* /usr/local/lib/
COPY --from=extracting /app/libredwg/install/usr/local/share/* /usr/local/share/
RUN ldconfig /usr/local/bin/
RUN ldconfig /usr/local/include/
RUN ldconfig /usr/local/lib/
RUN ldconfig /usr/local/share/
# copy project
COPY . /usr/src/app/
On Alpine Linux, the not found error is a typical symptom of dynamic link failure. It is indeed a rather confusing error by musl's ldd linker.
Most of the world Linux software is linked against glibc, the GNU libc library (libc provides the standard C library and POSIX API). Most Linux distributions are based on glibc. OTOH, Alpine Linux is based on the musl libc library, which is a minimal implementation and strictly POSIX compliant. Executables built on glibc distributions depend on /lib/x86_64-linux-gnu/libc.so.6, for example, which is not available on Alpine (unless, they are statically linked).
Except for this dependency, it's important to note that while musl attempts to maintain glibc compatibility to some extent, it is far from being fully compatible, and complex software that's built against glibc won't work with musl-libc, so simply symlinking /lib/ld-musl-x86_64.so.1 to the glibc path isn't likely going to work.
Generally, there are several ways for running glibc binaries on Alpine:
Install one the glibc compatibility packages, libc6-compat or gcompat:
# apk add gcompat
apk add libc6-compat
Both packages provide a light weight glibc compatibility layer which may be suitable for running simple glibc applications.
libc6-compat implements glibc compatibility APIs and provides symlinks to glibc shared libraries such as libm.so, libpthread.so and libcrypt.so. The gcompat package is based on Adelie Linux gcompat project and does the same but provides a single library libgcompat.so. Both libraries install loader stubs. Depdending on the application, one of them may work while the other won't, so it's good to try both.
Install proper glibc on Alpine, for providing all glibc methods and functionalities. There are glibc builds available for Alpine, which should be installed in the following procedure (example):
# Source: https://github.com/anapsix/docker-alpine-java
ENV GLIBC_REPO=https://github.com/sgerrand/alpine-pkg-glibc
ENV GLIBC_VERSION=2.30-r0
RUN set -ex && \
apk --update add libstdc++ curl ca-certificates && \
for pkg in glibc-${GLIBC_VERSION} glibc-bin-${GLIBC_VERSION}; \
do curl -sSL ${GLIBC_REPO}/releases/download/${GLIBC_VERSION}/${pkg}.apk -o /tmp/${pkg}.apk; done && \
apk add --allow-untrusted /tmp/*.apk && \
rm -v /tmp/*.apk && \
/usr/glibc-compat/sbin/ldconfig /lib /usr/glibc-compat/lib
Use statically linked executables. Static executables don't carry dynamic dependencies and could run on any Linux.
Alternatively, the software may be built from source on Alpine.
For LibreDWG, let's first verify the issue:
/usr/local/bin # ./dwg2dxf
/bin/sh: ./dwg2dxf: not found
/usr/local/bin
/usr/local/bin # ldd ./dwg2dxf
/lib64/ld-linux-x86-64.so.2 (0x7fd375538000)
libredwg.so.0 => /usr/local/lib/libredwg.so.0 (0x7fd3744db000)
libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fd375538000)
libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fd375538000)
Error relocating /usr/local/lib/libredwg.so.0: __strcat_chk: symbol not found
Error relocating /usr/local/lib/libredwg.so.0: __snprintf_chk: symbol not found
Error relocating /usr/local/lib/libredwg.so.0: __memcpy_chk: symbol not found
Error relocating /usr/local/lib/libredwg.so.0: __stpcpy_chk: symbol not found
Error relocating /usr/local/lib/libredwg.so.0: __strcpy_chk: symbol not found
Error relocating /usr/local/lib/libredwg.so.0: __printf_chk: symbol not found
Error relocating /usr/local/lib/libredwg.so.0: __fprintf_chk: symbol not found
Error relocating /usr/local/lib/libredwg.so.0: __strncat_chk: symbol not found
Error relocating /usr/local/lib/libredwg.so.0: __sprintf_chk: symbol not found
Error relocating ./dwg2dxf: __snprintf_chk: symbol not found
Error relocating ./dwg2dxf: __printf_chk: symbol not found
Error relocating ./dwg2dxf: __fprintf_chk: symbol not found
You can see that dwg2dxf depends on several glibc symbols.
Now, let's follow option 2 for installing glibc:
/usr/src/app # cd /usr/local/bin
/usr/local/bin # ls
dwg2SVG dwg2dxf dwgadd dwgbmp dwgfilter dwggrep dwglayers dwgread dwgrewrite dwgwrite dxf2dwg dxfwrite
/usr/local/bin # ./dwg2dxf
/bin/sh: ./dwg2dxf: not found
/usr/local/bin # export GLIBC_REPO=https://github.com/sgerrand/alpine-pkg-glibc && \
> export GLIBC_VERSION=2.30-r0 && \
> apk --update add libstdc++ curl ca-certificates && \
> for pkg in glibc-${GLIBC_VERSION} glibc-bin-${GLIBC_VERSION}; \
> do curl -sSL ${GLIBC_REPO}/releases/download/${GLIBC_VERSION}/${pkg}.apk -o /tmp/${pkg}.apk; done && \
> apk add --allow-untrusted /tmp/*.apk && \
> rm -v /tmp/*.apk && \
> /usr/glibc-compat/sbin/ldconfig /lib /usr/glibc-compat/lib
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/x86_64/APKINDEX.tar.gz
(1/1) Installing curl (7.74.0-r1)
Executing busybox-1.32.1-r3.trigger
OK: 629 MiB in 126 packages
(1/2) Installing glibc (2.30-r0)
(2/2) Installing glibc-bin (2.30-r0)
Executing glibc-bin-2.30-r0.trigger
/usr/glibc-compat/sbin/ldconfig: /usr/local/lib/libredwg.so.0 is not a symbolic link
/usr/glibc-compat/sbin/ldconfig: /usr/glibc-compat/lib/ld-linux-x86-64.so.2 is not a symbolic link
OK: 640 MiB in 128 packages
removed '/tmp/glibc-2.30-r0.apk'
removed '/tmp/glibc-bin-2.30-r0.apk'
/usr/glibc-compat/sbin/ldconfig: /usr/glibc-compat/lib/ld-linux-x86-64.so.2 is not a symbolic link
/usr/glibc-compat/sbin/ldconfig: /usr/local/lib/libredwg.so.0 is not a symbolic link
Voila:
/usr/local/bin # ./dwg2dxf
Usage: dwg2dxf [-v[N]] [--as rNNNN] [-m|--minimal] [-b|--binary] DWGFILES...
Try apk add gcompat (https://pkgs.alpinelinux.org/package/edge/community/x86/gcompat).
gcompat provides both /lib64/ld-linux-x86-64.so.2 and /lib/ld-linux-x86-64.so.2 (https://pkgs.alpinelinux.org/contents?file=ld-linux-x86-64.so.2).
EDIT :
For a complete solution, please see the #valiano'response.
Here is just a workaround that I've found before reading the #valiano'response
WORKAROUND
I've found a workaround by switching to another base image (Ubuntu based)
Here is the new working Dockerfile :
# podman/docker build -t libredwg .
############################
# STEP 1 build package from latest tar.xz
############################
FROM python:3.7.7-buster AS extracting
# libxml2-dev is broken so we need to compile it by our own
ARG LIBXML2VER=2.9.9
RUN apt-get update && \
apt-get install -y --no-install-recommends autoconf libtool swig texinfo \
build-essential gcc libxml2 python3-libxml2 libpcre2-dev libpcre2-32-0 curl \
libperl-dev libxml2-dev && \
mkdir libxmlInstall && cd libxmlInstall && \
wget ftp://xmlsoft.org/libxml2/libxml2-$LIBXML2VER.tar.gz && \
tar xf libxml2-$LIBXML2VER.tar.gz && \
cd libxml2-$LIBXML2VER/ && \
./configure && \
make && \
make install && \
cd /libxmlInstall && \
rm -rf gg libxml2-$LIBXML2VER.tar.gz libxml2-$LIBXML2VER
WORKDIR /app
RUN tarxz=`curl --silent 'https://ftp.gnu.org/gnu/libredwg/?C=M;O=D' | grep '.tar.xz<' | \
head -n1|sed -E 's/.*href="([^"]+)".*/\1/'`; \
echo "latest release $tarxz"; \
curl --silent --output "$tarxz" https://ftp.gnu.org/gnu/libredwg/$tarxz && \
mkdir libredwg && \
tar -C libredwg --xz --strip-components 1 -xf "$tarxz" && \
rm "$tarxz" && \
cd libredwg && \
./configure --disable-bindings --enable-release && \
make -j `nproc` && \
mkdir install && \
make install DESTDIR="$PWD/install" && \
make check DOCKER=1 DESTDIR="$PWD/install"
############################
# STEP 2 install into stable-slim
############################
# pull official base image
FROM osgeo/gdal:ubuntu-small-latest
# set work directory
WORKDIR /usr/src/app
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# copy requirements file
COPY ./requirements.txt /usr/src/app/requirements.txt
# install dependencies
RUN set -eux \
&& apt-get update && apt-get install -y --no-install-recommends build-essential \
libc6 python3-pip libffi-dev musl-dev gcc python3-dev postgresql-server-dev-all\
&& pip3 install --upgrade pip setuptools wheel \
&& pip3 install -r /usr/src/app/requirements.txt \
&& rm -rf /root/.cache/pip
# Install libredwg binaries
COPY --from=extracting /app/libredwg/install/usr/local/bin/* /usr/local/bin/
COPY --from=extracting /app/libredwg/install/usr/local/include/* /usr/local/include/
COPY --from=extracting /app/libredwg/install/usr/local/lib/* /usr/local/lib/
COPY --from=extracting /app/libredwg/install/usr/local/share/* /usr/local/share/
RUN ldconfig
# copy project
COPY . /usr/src/app/
Basically I've just changed the
FROM osgeo/gdal:alpine-normal-latest
To
FROM osgeo/gdal:ubuntu-small-latest
I've updated the dependencies installation as well (switching from apk add alpine PM to apt-get)
This is not an ideal solution for me since using an alpine based image generate more lightweight container but it's working so I post it as a possible solution.
The #valiano'response is the optimal solution. Please refer to it if you care about lightweigth images.
I solved it like this:
rm /usr/glibc-compat/lib/ld-linux-x86-64.so.2
ln -s /usr/glibc-compat/sbin/ldconfig /usr/glibc-compat/lib/ld-linux-x86-64.so.2
EDIT: explanation:
both /usr/glibc-compat/sbin/ldconfig and /usr/glibc-compat/lib/ld-linux-x86-64.so.2 were normal files in the docker container, so I tried one of them.
First I removed /usr/glibc-compat/sbin/ldconfig and made it a symlink. But I got an error, don't remember which.
Next I tried to remove ld-linux-x86-64.so.2 and make it a symlink. That worked.
It may not be in a binary format you can use on the system in question. Check your architecture and the file format (for example using the file command).
Edit:
Does /lib64/ld-linux-x86-64.so.2 exist? Can you run it?
Further edit:
The general idea here is that a dynamically linked binary can be thought of as a script with an interpreter. See this LWN article for additional details to understand what may be going on here. If your binaries are for the wrong platform, you will need new binaries or you will need to run them on the proper platform.
Another thing you can check is whether the output of file for this binary differs from the output of file for binaries that are working correctly (e.g. /bin/ls).
Now we can use docker compose to replace docker-compose
For Alpine Linux, install by
apk add docker-cli-compose
If you still prefer docker-compose, you can save a file under /usr/local/bin/docker-compose with a+x:
#!/bin/sh
docker compose $#
Then you still can run the old scripts which use docker-compose.
I am trying to compile FFmpeg with SVT-AV1 codec, following instructions from here: https://github.com/OpenVisualCloud/SVT-AV1/tree/master/ffmpeg_plugin
Everything goes well, but when I try to run
./configure --enable-libsvtav1
I am getting
ERROR: SvtAv1Enc not found using pkg-config
If you think configure made a mistake, make sure you are using the latest
version from Git. If the latest version fails, report the problem to the
ffmpeg-user#ffmpeg.org mailing list or IRC #ffmpeg on irc.freenode.net.
Include the log file "ffbuild/config.log" produced by configure as this will help
solve the problem.
The content of the ffbuild/config.log: https://pastebin.com/euPriFAp
There is an exact issue on the github: https://github.com/OpenVisualCloud/SVT-AV1/issues/35, but is closed as solved.
I have tried both on my Mac and in the Docker container with Ubuntu 18.04, but getting the same result.
Could anyone please help, what am I doing wrong?
The problem was in the lack of requred libraries. Please find the complete installation instruction below.
Installing packages required for compiling:
sudo apt-get update
sudo apt-get install \
autoconf \
automake \
build-essential \
cmake \
git-core \
libass-dev \
libfreetype6-dev \
libsdl2-dev \
libtool \
libva-dev \
libvdpau-dev \
libvorbis-dev \
libxcb1-dev \
libxcb-shm0-dev \
libxcb-xfixes0-dev \
pkg-config \
texinfo \
wget \
zlib1g-dev
Installing assemblers used by some libraries:
sudo apt-get install nasm
sudo apt-get install yasm
Build and install SVT-AV1:
git clone --depth=1 https://github.com/OpenVisualCloud/SVT-AV1
cd SVT-AV1
cd Build
cmake .. -G"Unix Makefiles" -DCMAKE_BUILD_TYPE=Release
make -j $(nproc)
sudo make install
Apply SVT-AV1 plugin and enable libsvtav1 to FFmpeg:
cd ~
git clone -b release/4.2 --depth=1 https://github.com/FFmpeg/FFmpeg ffmpeg
cd ffmpeg
export LD_LIBRARY_PATH+=":/usr/local/lib"
export PKG_CONFIG_PATH+=":/usr/local/lib/pkgconfig"
git apply ../SVT-AV1/ffmpeg_plugin/0001-Add-ability-for-ffmpeg-to-run-svt-av1.patch
./configure --enable-libsvtav1
(Note: if you want other codecs to be supported please add the required flags to the ./configure command)
Build FFmpeg:
make
make install
hash -r
source ~/.profile
Now you should have ffmpeg command working and have svt-av1 in encoders list:
ffmpeg -encoders
...
V..... libsvt_av1 SVT-AV1(Scalable Video Technology for AV1) encoder (codec av1)
...
I used next docs a reference:
https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu
https://github.com/OpenVisualCloud/SVT-AV1/tree/master/ffmpeg_plugin
Coming from a cushy Windows background, this building in linux is harder to understand, so apologies if this is an obvious question.
I have a c++ program that uses libcurl. It built fine on windows, but I must now port it to linux (specifically the raspberry pi). It uses the mime functions, so near-enough the latest version must be used. I've installed curl 7.62 by downloading it from https://curl.haxx.se/download.html and installing it as per the instructions at https://curl.haxx.se/docs/install.html to /home/pi/libraries/.
I have a dummy localhost web-server up for testing. When I call curl localhost from the terminal, I get: curl: (48) An unknown option was passed in to libcurl. When building and running my program I get an undefined symbol: curl_mime_init at run-time. After some googling, this appears to be a version mis-match and I confirmed this by checking versions:
Calling curl-config --version gives: libcurl 7.62.0, but calling curl --version gives: curl 7.62.0 (armv7l-unknown-linux-gnueabihf) libcurl/7.52.1 <other stuff>....
I assume that my program and curl are both looking else-ware for libcurl, but how can I specify the manually installed version of libcurl to g++, to my prog at run-time and to curl when calling from the terminal?
My program's makefile:
all: obj/curltests.o obj/Utility.o obj/curl_handler.o bin/curltests
obj/Utility.o: src/Utility.cpp src/Utility.h
g++ -g -c -pthread src/Utility.cpp -o obj/Utility.o
obj/curl_handler.o: src/curl_handler.cpp src/curl_handler.h
g++ -g -c -pthread src/curl_handler.cpp -o obj/curl_handler.o
obj/curltests.o: src/curltests.cpp
g++ -g -c -pthread src/curltests.cpp -o obj/curltests.o
bin/curltests: obj/curltests.o obj/Utility.o obj/curl_handler.o makefile
g++ -Wall -g \
-I src \
-L/home/pi/libraries/boost_1_68_0/stage/lib -lboost_system \
-lboost_filesystem -lboost_program_options -lboost_date_time \
obj/Utility.o obj/curltests.o obj/curl_handler.o \
-L/home/pi/libraries/curl-7.62.0/lib -lcurl \
-o bin/curltests
Not sure if this helps, but running whereis libcurl.so shows libcurl[.la][.so] in /usr/lib/ and libcurl[.a][.la][.so] in /usr/local/lib, but none in /home/pi/libraries/curl-7.62.0/lib as specified in the makefile above.
I am trying to build vim with the following options on my Fedora 23
I want +python +python3 +perl +lua +ruby +gui +conceal +gui for some plugins.
I could not find a version with all those built in (so I do it myself)
The journey started by following Valloric
sudo yum install -y ruby ruby-devel lua lua-devel luajit \
luajit-devel ctags git python python-devel \
python3 python3-devel tcl-devel \
perl perl-devel perl-ExtUtils-ParseXS \
perl-ExtUtils-XSpp perl-ExtUtils-CBuilder \
perl-ExtUtils-Embed
I also have ncurses (raw,devel,static,...)
Then
$ cd $HOME/Sources
$ git clone https://github.com/vim/vim.git
$ cd vim
$ ./configure --with-tlib=ncurses \
--with-features=huge \
--enable-fail-if-missing \
--enable-luainterp=yes \
--enable-mzschemeinterp \
--enable-perlinterp \
--enable-pythoninterp=yes \
--with-python-config-dir=/usr/lib64/python2.7/config \
--enable-python3interp=yes \
--enable-tclinterp=yes \
--enable-rubyinterp=yes \
--enable-cscope \
--enable-multibyte \
--enable-gui=auto \
--prefix=$HOME/Build/vim \
--with-compiledby=statquant | tee configure.log
The following is printed on screen:
/home/statquant/Sources/vim/src/config-PyMake3137:1478: warning: overriding recipe for target 'Modules/_math.o'
/home/statquant/Sources/vim/src/config-PyMake3137:1475: warning: ignoring old recipe for target 'Modules/_math.o'
/home/statquant/Sources/vim/src/config-PyMake3137:1517: warning: overriding recipe for target 'Modules/timemodule.o'
/home/statquant/Sources/vim/src/config-PyMake3137:1482: warning: ignoring old recipe for target 'Modules/timemodule.o'
configure: error: NOT FOUND!
You need to install a terminal library; for example ncurses.
Or specify the name of the library with --with-tlib.
Then I
make | tee make.log
I realized that it actually built vim in $HOME/Sources/vim/src instead of $HOME/Build/vim (maybe I was wrong to expect that)
When I run :version on ./vim -g (vim has been build with GUI support) there is
NO python NO python3 ....
configure.log is there, make.log is there
EDIT1: the following works, I now only miss +perl +ruby
./configure --with-features=huge \
--enable-tclinterp=yes \
--enable-luainterp=yes \
--enable-pythoninterp=yes \
--enable-python3interp=yes \
--with-compiledby=statquant \
--prefix=$HOME/Build/vim \
make install # and yes it installs in $HOME/Build/vim
EDIT2
Here is what happen when I try to add
+perl
I can run
./configure --with-features=huge \
--enable-tclinterp=yes \
--enable-luainterp=yes \
--enable-pythoninterp=yes \
--enable-python3interp=yes \
--enable-perlinterp=yes \
--prefix=$HOME/Build/vim \
--with-compiledby=statquant | tee configure.log
It appears to work (no error) : configure.log
make | tee make.log
[...]
cc1: error: -Wformat-security ignored without -Wformat [-Werror=format-security]
<command-line>:0:0: warning: "_FORTIFY_SOURCE" redefined
<command-line>:0:0: note: this is the location of the previous definition
cc1: some warnings being treated as errors
make[1]: *** [objects/option.o] Error 1
Makefile:2907: recipe for target 'objects/option.o' failed
make[1]: Leaving directory '/home/statquant/Sources/vim/src'
make: *** [first] Error 2
Makefile:26: recipe for target 'first' failed
I get no vim built in /home/statquant/Sources/vim/src : make.log
+ruby
./configure --with-features=huge \
--enable-tclinterp=yes \
--enable-luainterp=yes \
--enable-pythoninterp=yes \
--enable-python3interp=yes \
--enable-rubyinterp=yes \
--prefix=$HOME/Build/vim \
--with-compiledby=statquant | tee configure.log
It does not even run configure
checking --with-tlib argument... empty: automatic terminal library selection
checking for tgetent in -ltinfo... no
checking for tgetent in -lncurses... no
checking for tgetent in -ltermlib... no
checking for tgetent in -ltermcap... no
checking for tgetent in -lcurses... no
no terminal library found
checking for tgetent()... configure: error: NOT FOUND!
You need to install a terminal library; for example ncurses.
Or specify the name of the library with --with-tlib.
Here is the log: configure.log
Ok... so here's the thing. VIM is one of those programs that has a huge amount of patches, Fedora enforces some security checks at GCC level that won't allow you to build it (easily) from source code, my advice: rebuild the source rpm, is not that hard:
Install required dependencies to rebuild VIM
These packages are required for you to build VIM within F23
sudo dnf install gtk2-devel ncurses-devel lua-devel perl-devel perl-ExtUtils-Embed perl-ExtUtils-ParseXS perl-ExtUtils-XSpp perl-ExtUtils-CBuilder python-devel python3-devel tcl-devel ruby-devel
Install racket repository
I see you are passing the --enable-mzschemeinterp flag which requires racket, which is not included on the official repos.
cat >/etc/yum.repos.d/rpm-sphere.repo <<EOF
[rpm-sphere]
name=RPM Sphere
baseurl=http://download.opensuse.org/repositories/home:/zhonghuaren/Fedora_23/
gpgkey=http://download.opensuse.org/repositories/home:/zhonghuaren/Fedora_23/repodata/repomd.xml.key
enabled=1
gpgcheck=1
EOF
Install RPM developer tools:
These are required to install the tools to build/rebuild rpms
sudo dnf install #development-tools
sudo dnf install fedora-packager
sudo dnf install rpmdevtools
Setup your RPM build root
Create a file on your home folder called rpmmacros like so:
vim ~/.rpmmacros
Add the following contents to it:
%_topdir /home/statquant/Src/rpm
Then create the folder /home/statquant/Src/rpm
Create RPM build necessary folders
You need to have these folders so the source code and spec files are stored when you install the source rpm:
mkdir -p ~/Src/rpm
cd ~/Src/rpm
mkdir BUILD RPMS SOURCES SPECS SRPMS
mkdir RPMS/{noarch,x86,x86_64}
Download VIM source RPM:
dnf dnl --source vim
This will download the source rpm to the current folder, say vim-7.4.1718-1.fc23.src.rpm
Once it has been downloaded, install it:
rpm -ivh vim-7.4.1718-1.fc23.src.rpm
Once installed the RPM should have created the file /home/statquant/Src/rpm/SPECS/vim.spec open this file.
Modify configure options
You'll see that the SPEC file has the configure options and the make commands to build the actual thing. This file is huge and it has a lot of patches. You can go ahead and modify the areas where the configure command is, there are 3:
One for vim minimal
One for vim enhanced
One for the GUI version of vim
Make sure you modify accordingly. I did a test build with your options. You can download my SPEC file here, and vim, vim-enhanced and gvim worked just fine.
Rebuild the RPM
Once you modified (or copied from my template) the spec file you want to build the RPM based of that spec, go to /home/statquant/Src/rpm and then:
rpmbuild -ba SPECS/vim.spec
It will take a while, after that, you should see some ouput saying that the packages were built successfully, in my case:
Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/gustavo/Src/rpm/BUILDROOT/vim-7.4.1718-1.fc23.x86_64
Wrote: /home/gustavo/Src/rpm/SRPMS/vim-7.4.1718-1.fc23.src.rpm
Wrote: /home/gustavo/Src/rpm/RPMS/x86_64/vim-common-7.4.1718-1.fc23.x86_64.rpm
Wrote: /home/gustavo/Src/rpm/RPMS/x86_64/vim-minimal-7.4.1718-1.fc23.x86_64.rpm
Wrote: /home/gustavo/Src/rpm/RPMS/x86_64/vim-enhanced-7.4.1718-1.fc23.x86_64.rpm
Wrote: /home/gustavo/Src/rpm/RPMS/x86_64/vim-filesystem-7.4.1718-1.fc23.x86_64.rpm
Wrote: /home/gustavo/Src/rpm/RPMS/x86_64/vim-X11-7.4.1718-1.fc23.x86_64.rpm
Wrote: /home/gustavo/Src/rpm/RPMS/x86_64/vim-debuginfo-7.4.1718-1.fc23.x86_64.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.tNsBuH
+ umask 022
+ cd /home/gustavo/Src/rpm/BUILD
+ cd vim74
+ rm -rf /home/gustavo/Src/rpm/BUILDROOT/vim-7.4.1718-1.fc23.x86_64
+ exit 0
Install the new version
So now you have the packages built following fedora recommendations and patches, you can just go ahead and reinstall them (in case you already have the default fedora vim version which is probably true) like so:
cd /home/statquant/Src/rpm/RPMS/x86_64/
sudo dnf reinstall ./vim-common-7.4.1718-1.fc23.x86_64.rpm
sudo dnf reinstall ./vim-enhanced-7.4.1718-1.fc23.x86_64.rpm
etc...
Please note the dot and slash meaning that dnf is trying to install a local package instead of searching through its database which would download the versions you already have.
Notes
Out of security concern I did not uploaded my custom-built packages but if you don't mind or want to try them out ping me and I can upload them so you save yourself all this hassle... I already had all RPM build tools because I build some for me here.
You may also want to create a COPR repository so that you automate this process and if you need to format your disk, reinstall fedora, install it on a new machine etc. you can just add the repository and use dnf to install your custom package.
If you have any other questions let me know, good luck!
Update there are comments in the spec file beginning with # options for so you can jump to the actual configure options
You have enabled -Werror=format-security. This treats any susceptible usage of format of printf or scanf currently treated as `error.
You can either fix the usage in if_perl.c, I believe this is generated file. or remove the flag -Werror=format-security
The python problem was this...
--with-python-config-dir=/usr/lib64/python2.7/config \
--enable-python3interp=yes \
Version 2 config with a version 3 interpreter.
Your missing the ncurses-devel library, I can see that you dropped it from configure.
sudo yum install ncurses-devel
configure actually worked for me even when I didn't have the perl dev packages installed ie it didn't fail asking me to install them which I found odd. Do you have all the dev packages you need installed.
sudo yum install perl-devel
sudo yum install ruby-devel
If you are missing some dev package for ruby or perl the configure script say something like : "disabling this option". Just did this with ruby. After installing the ruby-dev package the ruby option is enabled in vim. Same with libperl-dev.
There is now a Vim issue filed about this:
https://github.com/vim/vim/issues/1081
I'm using a work-around that I mentioned in a comment to that issue:
https://github.com/vim/vim/issues/1081#issuecomment-269920486
I've copied most of that comment below in case it may be helpful.
I ran into this problem on Fedora 25 (x86_64). Vim's configure script asks ruby about LDFLAGS without a corresponding query regarding CFLAGS:
https://github.com/vim/vim/blob/v8.0.0134/src/auto/configure#L7174
rubyldflags=`$vi_cv_path_ruby -r rbconfig -e "print $ruby_rbconfig::CONFIG['LDFLAGS']"`
Since Fedora 23, all RPMs have been "hardened" by default:
https://fedoraproject.org/wiki/Changes/Harden_All_Packages
When an RPM is built, the %configure macro used in the RPM SPEC file is automatically adjusted to provide CFLAGS, LDFLAGS, and several other environment variables using hardened settings. When using Fedora's ruby package, the above query using rbconfig returns hardened linker flags. Because Vim's configure script doesn't use a corresponding rbconfig to query ruby's CFLAGS, the compiling and linking flags are mismatched, leading to the error:
configure: error: NOT FOUND!
You need to install a terminal library; for example ncurses.
Or specify the name of the library with --with-tlib.
In src/auto/config.log, this mismatch of flags causes this error:
configure:11318: checking for tgetent in -lncurses
configure:11343: gcc -o conftest -g -O2 -L. -Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -fstack-protector -rdynamic -Wl,-export-dynamic -Wl,--enable-new-dtags -Wl,-z,relro -L/usr/local/lib conftest.c -lncurses -lselinux >&5
/usr/bin/ld: /tmp/cckpDslF.o: relocation R_X86_64_PC32 against undefined symbol `tgetent' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
I can successfully build with a work-around, but I don't know enough about autoconf to suggest a proper fix to the configure script. The work-around involves first installing some RPM-related packages:
sudo dnf install -y rpm-build redhat-rpm-config
Next, set these variables at the shell prompt via:
eval $(rpmbuild --eval '%{configure}' | egrep '^\s*[A-Z]+=')
The above uses rpmbuild --eval '%{configure}' to query the definition of the %configure macro which begins with a number of environment variable assignments. This provides the same default compilation environment that is provided in the %configure macro. Now running the below invocation succeeds:
./configure --quiet --with-features=huge --enable-rubyinterp
This is what I did:
# Install lua
curl -R -O http://www.lua.org/ftp/lua-5.2.2.tar.gz
tar zxf lua-5.2.2.tar.gz
cd lua-5.2.2
sudo make linux install
# build vim
sudo apt-get install libncurses5-dev libgnome2-dev libgnomeui-dev \
libgtk2.0-dev libatk1.0-dev libbonoboui2-dev \
libcairo2-dev libx11-dev libxpm-dev libxt-dev python-dev ruby-dev mercurial
sudo apt-get remove vim vim-runtime gvim
sudo apt-get remove vim-tiny vim-common vim-gui-common
cd ~
hg clone https://code.google.com/p/vim/
cd vim
./configure --with-features=huge \
--enable-rubyinterp \
--enable-pythoninterp \
--with-python-config-dir=/usr/lib/python2.7-config \
--enable-perlinterp \
--enable-gui=gtk2 --enable-cscope --prefix=/usr \
--enable-luainterp \
--with-lua-prefix=/usr/local/bin/lua
make VIMRUNTIMEDIR=/usr/share/vim/vim74
sudo make install
But the ./configure step returns:
checking --enable-luainterp argument... yes
checking --with-lua-prefix argument... /usr/local/bin/lua
checking --with-luajit... no
checking for lua... (cached) /usr/local/bin/lua
checking Lua version... (cached) 5.2
checking if lua.h can be found in /usr/local/bin/lua/include... no
checking if lua.h can be found in /usr/local/bin/lua/include/lua5.2... no
I can verify that lua.h can't be found in those locations, but I don't know where it can be found.
Edit
I tried this again, ran into problems, and discovered a package vim-nox that already has vim support.
Original answer
I'm not entirely sure how I did this in the end, but thanks to #wrikken for the tip about headers.
# Install lua from binaries (these are out-of-date but at least they worked).
sudo apt-get install lua50 liblua50-dev liblualib50-dev
# Remove old vims
sudo apt-get remove vim vim-runtime gvim
sudo apt-get remove vim-tiny vim-common vim-gui-common
# Download and build a new vim
sudo apt-get install libncurses5-dev libgnome2-dev libgnomeui-dev \
libgtk2.0-dev libatk1.0-dev libbonoboui2-dev \
libcairo2-dev libx11-dev libxpm-dev libxt-dev python-dev ruby-dev mercurial
cd ~
hg clone https://code.google.com/p/vim/
cd vim
cd ~/vim
./configure --with-features=huge \
--enable-rubyinterp \
--enable-pythoninterp \
--with-python-config-dir=/usr/lib/python2.7-config \
--enable-perlinterp \
--enable-gui=gtk2 --enable-cscope --prefix=/usr \
--enable-luainterp \
--with-lua-prefix=/usr/local
At this point, check the output of ./configure to see that it found lua.h. If not, find out where it is (I'm afraid I can't remember where it was). Symlink to it in /usr/local with e.g. sudo ln -s ../lua.h and rerun ./configure.
Finally:
sudo make VIMRUNTIMEDIR=/usr/share/vim/vim74
sudo make install
If it still won't work, post on a forum somewhere and go for a walk in the outdoors. You'll find it suddenly starts to behave.
What worked for me:
sudo apt-get install liblua5.1-dev
copy all files from /usr/include/lua5.1/ to /usr/include/lua5.1/include/
sudo ln -s /usr/lib/x86_64-linux-gnu/liblua5.1.so /usr/local/lib/liblua.so
Go to the vim source folder
cd src
make distclean
clear
./configure --with-features=huge --enable-cscope --enable-pythoninterp=yes --with-python-config-dir=/usr/lib/python2.7/config-x86_64-linux-gnu --enable-multibyte --enable-fontset --enable-gui=gnome2 --disable-netbeans --enable-luainterp=yes --with-lua-prefix=/usr/include/lua5.1 --enable-largefile --enable-rubyinterp
sudo make
sudo make install
This will also install the GUI version, remove the --enable-gui=gnome2 if you will only use it in the command line.
Most of these I found it in here