Building x264 with YASM: failing the ASM check - linux

My question up front is, "I have new yasm, I think x264 is supposed to be cool with that, why is x264 not cool with that?"
For reasons, I am building a CentOS docker image (based on centos:latest) that contains a from-scratch ffmpeg build, following the guide here. It's a good guide, it's worked for me before, so I was feeling good about it.
Today I'm hitting a choke point on the libx264 build point: specifically, I say
PKG_CONFIG_PATH="/tmp/ffmpeg_build/lib/pkgconfig" \
./configure \
--prefix="/tmp/ffmpeg_build" \
--bindir="/tmp/bin" \
--enable-static
And I get a reply back
Found no assembler
Minimum version is nasm-2.13
If you really want to compile without asm, configure with --disable-asm.
That's unexpected. I have yasm, which I understand to be 1) there to do the things nasm does but better, and 2) to be the daisy-fresh most modern version given that I pulled it from its repo about an hour ago, and built it about fifty-nine minutes ago. For what it's worth, nasm is on the box too since the instructions request it, but it's below their stated version (i.e. it's "NASM version 2.10.07 compiled on Jun 9 2014")
So it seems like yasm is not being found. There's another StackExchange question that mentions that problem, which came out to a pathing issue. So, I added yasm to my path like so:
PATH=/tmp/ffmpeg_sources/yasm:$PATH \
PKG_CONFIG_PATH="/tmp/ffmpeg_build/lib/pkgconfig" \
./configure
...etc
That still gave the Found-no-assembler problem. As a last, confused resort, I told the script explicitly what I wanted to use for the variable $AS, because based on my quick look into configure, that looked like where yasm/nasm was meant to go. The command becomes:
AS=`which yasm`
PKG_CONFIG_PATH="/tmp/ffmpeg_build/lib/pkgconfig" \
./configure
...etc
That at least gave
Found yasm 1.3.0
Minimum version is nasm-2.13
If you really want to compile without asm, configure with --disable-asm.
Looking in the config.log I see the following:
checking whether /tmp/bin/yasm supports vmovdqa32 [eax]{k1}{z}, zmm0... no
Failed commandline was:
--------------------------------------------------
/tmp/bin/yasm conftest.asm -I. -I$(SRCPATH) -DARCH_X86_64=1 -I$(SRCPATH)/common/x86/ -f elf64 -o conftest.o
conftest.asm:1: error: instruction expected after label
conftest.asm:1: warning: ignoring unrecognized character `{'
conftest.asm:1: warning: ignoring unrecognized character `}'
conftest.asm:1: warning: ignoring unrecognized character `{'
conftest.asm:1: warning: ignoring unrecognized character `}'
--------------------------------------------------
Failed program was:
--------------------------------------------------
vmovdqa32 [eax]{k1}{z}, zmm0
--------------------------------------------------
So: what's the deal here? Is my assumption that yasm drop-in replaces nasm bad? Is yasm good for this purpose, but I'm not providing the right information to ./configure? Are my instructions for building ffmpeg for CentOS simply out of date with respect to this prerequisite and I should just try harder to get a modern nasm?

Older versions of x264 used to default to using yasm, new versions default to using nasm.
You can override the assembler by settings AS, however configure checks for instructions that don't appear to be compatible with yasm.
In addition, yasm was last released in 2014 vs nasm that is under active development.

This worked for me for OSX.
You need to set flags like
export ASFLAGS="-f macho64"
export NASM="${OPT}/bin/nasm"
export YASM="${OPT}/bin/yasm"
export AS="${NASM}"

Related

arm-none-eabi-objdump: error while loading shared libraries: libdebuginfod.so.1: cannot open shared object file

If you have an answer for this, or further information, I'd welcome it. I'm following advice from here, to offer some unsolicited help by posting this question then an answer I've already found for it.
I have a bare-metal ARM board for which I'm building a cross-toolchain, from sources for GNU binutils, gcc and gdb, and for SourceWare's Newlib. I got those four working and cross-built a DoNothing.c into an ELF file - but I couldn't disassemble it with this:
$ arm-none-eabi-objdump -S DoNothing.elf
The error was:
$ arm-none-eabi-objdump: error while loading shared libraries: libdebuginfod.so.1: cannot open shared object file: No such file or directory
I'll follow up with a solution.
The error was correct - my system didn't have libdebuginfod.so.1 installed - but I have another cross-binutils, installed from binary for a different target, and its objdump -S works fine on the same host. Why would one build of objdump complain about missing that shared library, when clearly not all builds of objdump need it?
First I tried rebuilding cross binutils, specifying --without-debuginfod as a configure option. No change, which seems odd: surely that should build tools that not only don't use debuginfod but which don't depend on it in any way. (If someone can answer that, or point out what I've misunderstood, it may help people.)
Next I figured debuginfod was inescapable (for my cross-tools built from source at least), so I'd install it to get rid of the error. It's a component of the elfutils package, but installing the latest elfutils available for my Ubuntu 20.04 system didn't bring libdebuginfod.so.1 with it.
I found a later one, for Arch Linux, whose package contents suggested it would - but its package format doesn't match Ubuntu's and installing it was going to involve a lot of work. Instead I opted to build it from the Arch Linux source package. However, running ./configure on that gave a couple of infuriatingly similar errors:
configure: checking libdebuginfod dependencies, --disable-libdebuginfod or --enable-libdebuginfo=dummy to skip
...
configure: error: dependencies not found, use --disable-libdebuginfod to disable or --enable-libdebuginfod=dummy to build a (bootstrap) dummy library.
No combination of those suggestions would allow configure for elfutils-0.182 to run to completion.
The problem of course was my own lack of understanding. The solution came from the Linux From Scratch project: what worked was to issue configure with both of the suggested options, like this:
$ ./configure --prefix=/usr \
--disable-debuginfod \
--enable-libdebuginfod=dummy \
--libdir=/lib
That gave a clean configure; make worked first time, as did make check and then sudo make install which of course installed libdebuginfod.so.1 as required. I then had an arm-none-eabi-objdump which disassembles cross-compiled ELF files without complaining.

Linking with gfortran: _edata: invalid version 21 (max 4)

I'm working with RHEL6 systems, but need to port the code using C++11 (and even C++14) features. This forced me to build gcc-8.2 by hand, installed under a private prefix (/prod/pfe/local). This created a number of executables under /prod/pfe/local/bin: gcc, g++, ld, and gfortran.
I'm now trying to build CBLAS, which uses the above gfortran. Building the library (cblas_LINUX.a) works fine, but creating an executable fails with a cryptic errors cited in the title:
gfortran -o xscblat1 c_sblat1.o c_sblas1.o ../lib/cblas_LINUX.a
/prod/pfe/local/lib/gcc/x86_64-pc-linux-gnu/8/../../../../x86_64-pc-linux-gnu/bin/ld: /prod/pfe/local/lib/gcc/x86_64-pc-linux-gnu/8/../../../../lib64/libgfortran.so: _edata: invalid version 21 (max 4)
/prod/pfe/local/lib/gcc/x86_64-pc-linux-gnu/8/../../../../x86_64-pc-linux-gnu/bin/ld: /prod/pfe/local/lib/gcc/x86_64-pc-linux-gnu/8/../../../../lib64/libgfortran.so: error adding symbols: bad value
Did I configure build gfortran incorrectly? If not, how do I solve this problem -- additional FFLAGS or LDFLAGS of some kind?
Ok, according to the gcc-developers, this is a known bug triggered by the use of the new linker (gold).
Rebuilding the compiler suit with --disable-gold solves the problem.
Update: correction -- somehow, disabling gold is not good enough. Going back to the binutils-2.30 is what I ended up doing...

How do I install tcpslice from source, released for fedora, on a mac 64 bit?

The newest publicly available tcpslice version 1.2a1 (found on its github) has a bug where it expects 8 bytes for a time field but gets 16 (when in 64 bit). This leads to the error:
tcpslice: problems finding end packet of file ./abc1234.bin
I got my info for this error from: https://bugzilla.redhat.com/show_bug.cgi?id=485670
This led me to grab the updated version (1.2a3) from here and try compiling from source: http://pkgs.fedoraproject.org/repo/pkgs/tcpdump/tcpslice-1.2a3.tar.gz/. I think this is or close to the actual patch
HOWEVER, I couldn't ./configure this on my mac because it says (understandably) this:
checking build system type... configure: error: cannot guess build type; you must specify one
Not one to give up, and becuase I don't wnat to switch over to linux to continue deveolping my wrapper script, I tried to compile like this:
./configure --build=i686-pc-linux-gnu
This configured and make'd!!! However, its 32 bit which still gives the same error when I run the newly made tcpdump!!
Is this possible at all, and what other --build type can I use that is for x86_64 bit systems? No other --build types are compiling for me. I have no idea how to find acceptable build types, and am currently referenceing this: https://gcc.gnu.org/gcc-4.2/buildstat.html
EDIT: I tried Warren Young's suggestion here but libtoolize --force did not remake config.guess, so I manually downloaded the "newest" config.guess from the link on this page with curl. However, now ./configure says:
checking build system type... Invalid configuration `x86_64-apple-darwin15.4.0': machine `x86_64-apple' not recognized
configure: error: /bin/sh ./config.sub x86_64-apple-darwin15.4.0 failed
Am I screwed? Or can I sitll try a default x86_64 -build type (which I still cannot find):
checking build system type... Invalid configuration `x86_64': machine `x86_64' not recognized
configure: error: /bin/sh ./config.sub x86_64 failed
Welp, the answer was to manually download the newest config.guess and config.sub files from here: https://www.gnu.org/software/gettext/manual/html_node/config_002eguess.html and to overwrite the ones in the tcpslice directory.
However, the same problem still existed. There must be another reason the time issue is happening on the mac because this version of tcpslice worked fine on ubuntu on the same pcap.
Thanks to Warren Young's post in this question for guiding me in the right (though not successful) direction: Compiling tcpsplice on a 64-bit machine. Guess I'll be using ubuntu to finish my script!

custom asm mnemonics unrecognized in current riscv-gnu-toolchain

For example, the following assembler statement should translate to the opcode 0x0000000b
custom0 0,0,0,0
With an older version of riscv-gnu-toolchain (built on Jun 4th) no -march= option was needed to assemble this. However, with todays git head of riscv-tools I get:
$ /opt/riscv_new/bin/riscv64-unknown-elf-as -m32 test.s
test.s: Assembler messages:
test.s:1: Error: unrecognized opcode `custom0 0,0,0,0'
From reading the source code I'd guess passing the arch Xcustom to the tools should fix that, but:
$ /opt/riscv_new/bin/riscv64-unknown-elf-as -m32 -march=RV32IMXcustom test.s
test.s: Assembler messages:
test.s:1: Error: unrecognized opcode `custom0 0,0,0,0'
(At the moment the PicoRV32 test firmware does not build with the latest riscv-gnu-toolchain because of this.)
Edit: The last version of riscv-tools that work for me is commit 84a47e0b4e from Aug 20. With that version the custom0 opcode is recognized with and without the -march= option.
Sorry this took so long to find -- I don't read stackoverflow. For future reference, if you submit a github issue/PR or email me directly, I'll deal with these sorts of things.
A fix should be here, do you mind verifying it?
https://github.com/riscv/riscv-gnu-toolchain/pull/97
Sorry for introducing the regression!
I also faced this issue, in yesterdays update of the gnu toolchain if you look into the opcodes.custom file in the risv-opcodes folder, all the custom opcodes are commented, you need to uncomment those that you need and recompile the toolchain

gcc 4.x not supporting x87 FPU math?

I've been trying to compile gcc 4.x from the sources using --with-fpmath=387 but I'm getting this error: "Invalid --with-fpmath=387". I looked in the configs and found that it doesn't support this option (even though docs still mention it as a possible option):
case ${with_fpmath} in
avx)
tm_file="${tm_file} i386/avxmath.h"
;;
sse)
tm_file="${tm_file} i386/ssemath.h"
;;
*)
echo "Invalid --with-fpmath=$with_fpmath" 1>&2
exit 1
Basically, I started this whole thing because I need to supply an executable for an old target platform (in fact, it's an old Celeron but without any SSE2 instructions that are apparently used by libstdc++ by DEFAULT). The executable crashes at the first instruction (movq XMM0,...) coming from copying routines in libstdc++ with an "Illegal instruction" message.
Is there any way to resolve this? I need to be on a fairly recent g++ to be able to port my existing code base.
I was wondering if it's possible to supply these headers/sources from an older build to enable support for regular x87 instructions, so that no SSE instructions are referenced?
UPDATE: Please note I'm talking about compiled libstdc++ having SSE2 instructions in the object code, so the question is not about gcc command line arguments. No matter what I'm supplying to gcc when compiling my code, it will link with libstdc++ that already has built-in SSE2 instructions.
The real answer is not to use ANY --with-fpmath switches when compiling GCC. I got confused by the configure script switch statement thinking that it only supports sse or avx, while, in fact, the default value (not mentioned in this switch is "387"). So make sure you don't use --with-fpmath when running configure. I recompiled GCC without it and it now works fine.
Thanks.
The argument to tell gcc to produce code for a specific target is -march=CPU where CPU is the particular cpu you want. For an old celeron, you probably want -march=pentium2 or -march=pentium3
To control the fp codegen separately, newer versions of gcc use -mfpmath= -- in your case, you want -mfpmath=387.
All of these and many others are covered in the gcc documentation
edit
In order to use those flags for building libraries (such as libstdc++) that you'll later link in to programs, you need to configure the build for the library to use the appropriate flags. libstdc++ gets built as part of the g++ build, so you'll need to do a custom build -- you can use configure CXXFLAGS=-mfpmath=387 to set extra flags to use while building things.
Please note the question was about compiled libstdc++ having SSE2 instructions in the object code, so the question was not about gcc command line arguments. No matter what I'm supplying to gcc when compiling my code, it will link with libstdc++ that already has built-in SSE2 instructions.
The real answer is not to use ANY --with-fpmath switches when compiling GCC. I got confused by the configure script switch statement thinking that it only supports sse or avx, while, in fact, the default value (not mentioned in this switch is "387"). So make sure you don't use --with-fpmath when running configure. I recompiled GCC without it and it now works fine.

Resources