i'm trying to emulate a series of ARM firmwares using qemu-system. I'm starting off with a raspberry image to experiment a bit, and I wonder if there's a way to log every action that the (emulated) raspberry makese (eg. system calls). The command i'm using right now is
sudo qemu-system-arm -kernel kernel-qemu -cpu arm1176 -m 256 -M versatilepb -dtb versatile-pb.dtb -no-reboot -serial stdio -append "root=/dev/sda2 rootfstype=ext4 rw" -hda rpitest.qcow2 -net nic -net user -net tap,ifname=vnet0,script=no,downscript=no -trace guest_user_syscall,file=trace
but the file trace remains empty during execution. Do you have any advice? Thanks in advance :)
The 'guest_user_syscall' tracepoint is used only by the user-mode emulator (qemu-arm), not by the system emulator (qemu-system-arm), so it will never fire for your command line.
In general QEMU does not have a tracing system that will record all information about what the guest CPU is doing: QEMU's design favours "run correct code quickly", not "provide a lot of introspection and information about what possibly-buggy code is doing". The facilities that are available, in rough order of user convenience, are:
the gdbstub. You can connect a remote gdb that knows about your target architecture to QEMU's gdbstub and do all the usual things like singlestep, set breakpoints, etc. Often the most 'user-friendly' way to debug guest code, especially during early bootup.
tracepoints (loggable via -trace or also via a suboption of-d). You get whatever the authors of the device model code etc felt was useful, which varies a lot from device to device. Can be very helpful for debugging guest code that's interacting with a specific device, provided that the device model has tracepoints (or you are capable of adding them).
the -d debug logging options. Mostly these are biased towards logging things that are easy to log and things that assist in debugging QEMU itself. You need some awareness of what QEMU does under the hood (and in particular the distinction between "translation time" and "runtime") to interpret them correctly, but they can be useful for figuring out what guest code is doing.
TCG plugins: this is a relatively new feature provided by QEMU where you can write a C plugin which can "instrument" the emulated guest code. In theory this lets you do quite complex things such as count and classify executed instructions (for instance you could count syscall insns); however the API is pretty complicated and there are still some obvious missing pieces (for instance you can't write a plugin that traces all memory accesses).
Related
I've written and compiled a RISC-V Linux application.
I want to dump all the instructions that get executed at run-time (which cannot be achieved by static analysis).
Is it possible to get a dynamic assembly instruction execution historgram from QEMU (or other tools)?
For instruction tracing, I go with -singlestep -d nochain,cpu, combined with some awk. This can become painfully slow and large depending on the code you run.
Regarding the statistics you'd like to obtain, delegate it to R/numpy/pandas/whatever after extracting the program counter.
The presentation or video of user "yvr18" on that topic, might cover some aspects of QEMU tracing at various levels (as well as some interesting heatmap visualization).
QEMU doesn't currently support that sort of trace of all instructions executed.
The closest we have today is that there are various bits of debug logging under the -d switch, and you can combine the tracing of "instructions translated from guest to native" with the "blocks of translated code executed" translation to work out what was executed, but this is pretty awkward.
Alternatively you could try scripting the gdbstub interface to do something like "disassemble instruction at PC; singlestep" which will (slowly!) give you all the instructions executed.
Note: There ongoing work to improve QEMU's ability to introspect guest execution so that you can write a simple 'plugin' with functions that are called back on events like guest instruction execution; with that it would be fairly easy to write a dump of guest instructions executed (or do more interesting processing), but this is still work-in-progress, so not available yet.
It seems you can do something similar with rv8 (https://github.com/rv8-io/rv8), using the command:
rv-jit -l
The "spike" RISC-V emulator allows tracing instructions executed, new values stored into registers, or just simply a histogram of PC values (from which you can extract what instruction was at each PC location).
It's not as fast as qemu, but runs at 100 to 200 MIPS on current x86 hardware (at least without tracing enabled)
i'm running raspbian (2015-05-05-raspbian-wheezy.img) in qemu using compiled kernel (https://github.com/dhruvvyas90/qemu-rpi-kernel) on ubuntu 14.04. my final goal is to launch my python script within the emulation.
i'm following manual from http://www.unixmen.com/emulating-raspbian-using-qemu/, though many others suggest very similar sequence of actions.
things i'm trying and issues i'm experiencing:
first boot is more or less ok. i comment the line in /etc/ld.so.preload as suggested and reboot.
on second boot (after i remove init=/bin/bash) and all subsequent boots i get
ERROR ../libkmod/libkmod.c:554 kmod_search_moddep: could not open moddep file '/lib/modules/3.10.25/modules.dep.bin'
some googling suggested to run "sudo rpi-update". it didn't help, same message during boot.
on second boot (after i remove init=/bin/bash) and all subsequent boots i get
fsck died with exit status 6
looking into "/var/log/fsck/checkfs" as suggested tells that some location is not there, but it doesn't say which one
running "startx" produces error message from 1. it loads the UI eventually, but desktop only has "wastebasket" icon. there is also a thick white stripe on top of the screen blinking, like it keeps trying to load a tab but fails everytime. qemu window stops to respond to further interaction after this.
running "sudo apt-get upgrade" installs some packages, but after reboot i can't even get to UI - just blank screen with mouse cursor.
i'm not very experienced with how linux is configured at low level. i understand that i might be doing something completely stoopid.
so, my questions are:
how do i debug? i couldn't figure out the settings for qemu to write logs. i really don't want to fallback to gdb, as i'm not debugging qemu itself, just want to get notification on it's events.
ctrl key doesn't seem to work inside qemu window.
no copy-paste available. or i can't see how to turn it on.
am i missing something? from all the manuals i have seen it seems like this should go much much smoother. like it should "just work".
Since your post many things changed. The most important things is that now using Andrew Baumann GitHub repo you can build QEMU that boots recent Raspbian. I described my experience woth this code here. Instructions are straight forward. Implementation needs polishing but it best compilation of work so far.
To answer your questions:
QEMU have -s and -S options for GDB. First option setup gdb server hook and second freez CPU, so you can connect debugger. This is not for QEMU debugging this for guest system debugging. Default QEMU logging is to stderr, so if something valuable happen you will see it in terminal. You can raise QEMU verbosity by uncommenting various *DEBUG_ statements in source code. Also check help for -d and -D command line flags of QEMU.
Not sure I can help with this. Only thing that I can say is that my QEMU version 2.5.50 reacts to Ctrl+Alt which exits from GUI after capturing cursor, so it looks like QEMU understand Ctrl key. I assume that QEMU do not capture your special keys combination because your window manager do it before passing to QEMU.
This also not work for me, but I see some work was done in this area. Not sure how to enable and use that feature.
Emulating any hardware is very complex and requires a lot of work. All emulated targets are limited to some most important features. BCM2835/BCM2836 (Raspberry Pi/Raspberry Pi 2) SoC are still not accepted by mainline QEMU, so just work will not apply to those platforms.
I was wondering if there exists a way to run an untrusted C program under a sandbox in Linux. Something that would prevent the program from opening files, or network connections, or forking, exec, etc?
It would be a small program, a homework assignment, that gets uploaded to a server and has unit tests executed on it. So the program would be short lived.
I have used Systrace to sandbox untrusted programs both interactively and in automatic mode. It has a ptrace()-based backend which allows its use on a Linux system without special privileges, as well as a far faster and more poweful backend which requires patching the kernel.
It is also possible to create a sandbox on Unix-like systems using chroot(1), although that is not quite as easy or secure. Linux Containers and FreeBSD jails are a better alternative to chroot. Another alternative on Linux is to use a security framework like SELinux or AppArmor, which is what I would propose for production systems.
We would be able to help you more if you told as what exactly it is that you want to do.
EDIT:
Systrace would work for your case, but I think that something based on the Linux Security Model like AppArmor or SELinux is a more standard, and thus preferred, alternative, depending on your distribution.
EDIT 2:
While chroot(1) is available on most (all?) Unix-like systems, it has quite a few issues:
It can be broken out of. If you are going to actually compile or run untrusted C programs on your system, you are especially vulnerable to this issue. And if your students are anything like mine, someone WILL try to break out of the jail.
You have to create a full independent filesystem hierarchy with everything that is necessary for your task. You do not have to have a compiler in the chroot, but anything that is required to run the compiled programs should be included. While there are utilities that help with this, it's still not trivial.
You have to maintain the chroot. Since it is independent, the chroot files will not be updated along with your distribution. You will have to either recreate the chroot regularly, or include the necessary update tools in it, which would essentially require that it be a full-blown Linux distribution. You will also have to keep system and user data (passwords, input files e.t.c.) synchronized with the host system.
chroot() only protects the filesystem. It does not prevent a malicious program from opening network sockets or a badly-written one from sucking up every available resource.
The resource usage problem is common among all alternatives. Filesystem quotas will prevent programs from filling the disk. Proper ulimit (setrlimit() in C) settings can protect against memory overuse and any fork bombs, as well as put a stop to CPU hogs. nice(1) can lower the priority of those programs so that the computer can be used for any tasks that are deemed more important with no problem.
I wrote an overview of sandboxing techniques in Linux recently. I think your easiest approach would be to use Linux containers (lxc) if you dont mind about forking and so on, which don't really matter in this environment. You can give the process a read only root file system, an isolated loopback network connection, and you can still kill it easily and set memory limits etc.
Seccomp is going to be a bit difficult, as the code cannot even allocate memory.
Selinux is the other option, but I think it might be more work than a container.
Firejail is one of the most comprehensive tools to do that - it support seccomp, filesystem containers, capabilities and more:
https://firejail.wordpress.com/features-3/
You can use Qemu to test assignments quickly. This procedure below takes less than 5 seconds on my 5 year old laptop.
Let's assume the student has to develop a program that takes unsigned ints, each on their own line, until a line with "-1" arrives. The program should then average all the ints and output "Average: %f". Here's how you could test program completely isolated:
First, get root.bin from Jslinux, we'll use that as the userland (it has the tcc C-compiler):
wget https://github.com/levskaya/jslinux-deobfuscated/raw/master/root.bin
We want to put the student's submission in root.bin, so set up the loop device:
sudo losetup /dev/loop0 root.bin
(you could use fuseext2 for this too, but it's not very stable. If it stabilizes, you won't need root for any of this)
Make an empty directory:
mkdir mountpoint
Mount root.bin:
sudo mount /dev/loop0 mountpoint
Enter the mounted filesystem:
cd mountpoint.
Fix rights:
sudo chown -R `whoami` .
mkdir -p etc/init.d
vi etc/init.d:
#!/bin/sh
cd /root
echo READY 2>&1 > /dev/ttyS0
tcc assignment.c 2>&1 > /dev/ttyS0
./a.out 2>&1 > /dev/ttyS0
chmod +x etc/init.d/rcS
Copy the submission to the VM:
cp ~/student_assignment.c root/assignment.c
Exit the VM's root FS:
cd ..
sudo umount mountpoint
Now the image is ready, we just need to run it. It will compile and run the submission after booting.
mkfifo /tmp/guest_output
Open a seperate terminal and start listening for guest output:
dd if=/tmp/guest_output bs=1
In another terminal:
qemu-system-i386 -kernel vmlinuz-3.5.0-27-generic -initrd root.bin -monitor stdio -nographic -serial pipe:/tmp/guestoutput
(I just used the Ubuntu kernel here, but many kernels will work)
When the guest output shows "READY", you can send keys to the VM from the qemu prompt.
For example, to test this assignment, you could do
(qemu) sendkey 1
(qemu) sendkey 4
(qemu) sendkey ret
(qemu) sendkey 1
(qemu) sendkey 0
(qemu) sendkey ret
(qemu) sendkey minus
(qemu) sendkey 1
(qemu) sendkey ret
Now Average = 12.000000 should appear on the guest output pipe. If it doesn't, the student failed.
Quit qemu: quit
A program passing the test is here: https://stackoverflow.com/a/14424295/309483. Just use tcclib.h instead of stdio.h.
Try User-mode Linux. It has about 1% performance overhead for CPU-intensive jobs, but it may be 6 times slower for I/O-intensive jobs.
Running it inside a virtual machine should offer you all the security and restrictions you want.
QEMU would be a good fit for that and all the work (downloading the application, updating the disk image, starting QEMU, running the application inside it, and saving the output for later retrieval) could be scripted for automated tests runs.
When it goes about sanboxing based on ptrace (strace) check-out:
"sydbox" sandbox and "pinktrace" programming library ( it's C99 but there are bindings to python and ruby as far as I know).
Collected links related to topic:
http://www.diigo.com/user/wierzowiecki/sydbox
(sorry that not direct links, but no enough reputation points yet)
seccomp and seccomp-bpf accomplish this with the least effort: https://www.kernel.org/doc/Documentation/prctl/seccomp_filter.txt
ok thanks to all the answers they helped ME a lot. But i would suggest none of them as an solution for the person who asked the original question. All mentioned tools require to much work for the purpose to test students code as a teacher,tutor,prof. The best way in this case would be in my opinion virtualbox. Ok, its emulates an complete x68-system and has nothing to do with the meaning of sandboxing in this way but if i imagine my programming teacher it would be the best for him. So "apt-get install virtualbox" on debian based systems, all others head over to http://virtualbox.org/ , create a vm, add an iso, click install, wait some time and be lucky. It will be much easier to use as to set up user-mode-linux or doing some heavy strace stuff...
And if you have fears about your students hacking you i guess you have an authority problem and a solution for that would be threaten them that you will sue the living daylights out of them if you can prove just one bite of maleware in the work they give you...
Also if there is a class and 1% of it is as good as he could do such things, dont bore them with such simple tasks and give them some big ones where they have to code some more. Integrative learning is best for everyone so dont relay on old deadlocked structures...
And of cause, never use the same computer for important things (like writing attestations and exams), that you are using for things like browsing the web and testing software.
Use an off line computer for important things and an on line computer for all other things.
However to everyone else who isnt a paranoid teacher (dont want to offend anybody, i am just the opinion that you should learn the basics about security and our society before you start being a programmers teacher...)
... where was i ... for everyone else:
happy hacking !!
I have a problem probably with my arm toolchain but maybe there's something other that I do wrong. I have Chinese made dev board qq2440 using Samsung s3c2440 ARM9 uC. I'm using Ubuntu x86 with native gcc(4.3.3) and cross-compile version arm-unknown-linux-uclibc-gcc (crosstool-NG-1.3.2) 4.3.2
I followed tutorials from http://blog.leshak.ru/english/pages/how-to-install-u-boot-linux-2629-rootfsjffs2-busybox-1132-into-nand-qq2440/
and used Leshak's kernel patches for that board. Problem is that his binaries work perfectly and mine don't...
I communicate with my board over RS232 (serial port) and I have serial terminal configured on target Linux. I use Leshak's uboot image. To configure my kernel I use following command line:
qq2440> setenv bootargs 'noinitrd root=/dev/mtdblock2 rootfstype=jffs2 rw console=ttySAC0,115200'
For target I use vanilla Linux sources version 2.6.29, with patches created by Leshak. I don't honestly believe that this will ever be supported officially by Linux as it's not mainstream product.
My kernel image starts booting up, but it probably changes bandwidth (or CPU frequency) to some non standard value (tried all standard ones already). Instead of dots indicating loading kernel into memory I've got only trash instead. Unfortunately it doesn't probably finish the boot process as the network interface nor file system don't come up. So I figured out that it panics somewhere in the middle.
Any ideas what should I do next?
Thanks & regards,
Chris
There are a lot of different things that could be going on here.
It sounds like you are talking about a serial port, and that it appears to be giving garbage once control is passed to the kernel from uboot. Am I understanding that correctly?
Look into specifying the baud rate, parity, etc. for the serial console on the kernel commandline.
Oh, and IIRC, there was some 'early_printk' thing in the ARM Linux tree that might help you debug serial console problems. (But I'll warn you -- it's been a couple years since I dealt with that so my memory is fuzzy.)
Double-check that the memory address layout (the locations of all the various devices) matches what your board has. (I think this is probably not the issue, but wanted to mention it for completeness.)
You say that you have a binary kernel that works correctly; compare the kernel config of that kernel to the config you are using for building your kernel. Investigate every difference, particularly any specific to ARM.
You may want to double-check the endianness of your toolchain vs what your board is expecting. Some of the ARM / XScale processors can be configured to big-endian or little-endian in software, so it might be worth double-checking.
Just enable the debug build of the kernel[while building the uImage] so that you get a more clearer picture of the scenario [Just would make your boot up somewhat slow since all the printk's would be enabled].
Can you check whether you are passing the correct parameters to the UART ie. Serial Port Name, it's baud rate etc This would be provided by the board manufacturer-Samsung
WRT the network instead of DHCP can you just assign a static ip address to your system as it might be possible that the DHCP process is still not ON.
Also a better option would be to use NFS but yeah, it depends on your choice and the purpose of your application. To use NFS, your network should be UP & running and your filesystem should be shared.
As retracile has already pointed out "Endianness" could be a point to look into !!!
You can refer this link which might help you out since it is specific to S3C2440
Hope this helps.
-hjsblogger
I had a similar problem at one point when I omitted --send-cmd from picocom. this is the command I issue to picocom for serial uBoot comms with the mini2440.
picocom -b 115200 /dev/ttyS0 --send-cmd "sx -vv"
I'm working on keyboard input for a very basic kernel that I'm developing and I'm completely stuck. I can't seem to find any information online that can show me the information I need to know.
My kernel is running in protected mode right now, so I can't use the real mode keyboard routines without jumping into real mode and back, which I'm trying to avoid. I want to be able to access my keyboard from protected mode. Does anyone know how to do this? The only thing I have found so far is that it involves talking to the controller directly using in/out ports, but beyond that I'm stumped. This is, of course, is not something that comes up very often. Normally, Assembly tutorials assume you're running an operating system underneath.
I'm very new to the x86 assembly, so I'm just looking for some good resources for working with the standard hardware from protected mode. I'm compiling the Assembly source code with NASM and linking it to the C source code compiled with DJGPP. Any suggestions?
The MIT operating systems class has lots of good references. In particular, check out Adam Chapweske's resources on keyboard and mouse programming.
In short, yes, you will be using the raw in/out ports, which requires either running in kernel mode, or having the I/O permission bits (IOPL) set in the EFLAGS register. See this page for more details on I/O permissions.
You work with standard legacy hardware the same way on real and protected modes. In this case, you want to talk with the 8042 at I/O ports 0x60 to 0x6f, which in turn will talk to the controller within the keyboard at the other end of the wire.
A quick Google search found me an interesting resource at http://heim.ifi.uio.no/~stanisls/helppc/8042.html (for the 8042) and http://heim.ifi.uio.no/~stanisls/helppc/keyboard_commands.html (for the keyboard).
In case you are not used to it, you talk with components at I/O ports via the IN (read) and OUT (write) opcodes, which receive the I/O port number (a 16-bit value) and the value to be read or written (either 8, 16, or 32 bits). Note that the size read or written is important! Writing 16 bits to something which is expecting 8 bits (or vice versa) is a recipe for disaster. Get used to these opcodes, since you will be using them a lot (it is the only way to talk to some peripherals, including several essential ones; other peripherals use memory-mapped I/O (MMIO) or bus-mastering DMA).
The 8042 PS/2 Controller looks like the simplest possibility.
The oszur11 OS tutorial contains a working example under https://sourceforge.net/p/oszur11/code/ci/master/tree/Chapter_06_Shell/04_Makepp/arch/i386/arch/devices/i8042.c
Just:
sudo apt-get install build-essential qemu
sudo ln -s /usr/bin/qemu-system-i386 /usr/bin/qemu
git clone git://git.code.sf.net/p/oszur11/code oszur11
cd oszur11/Chapter_06_Shell/04_Makepp
make qemu
Tested on Ubuntu 14.04 AMD64.
My GitHub mirror (upstream inactive): https://github.com/cirosantilli/oszur11-operating-system-examples
Not reproducing it here because the code it too long, will update if I manage to isolate the keyboard part in a minimal example.