Spin Model Checker does not find a bug when using remote varrefs - model-checking

I am trying to formally verify mutual exclusion on the following Promela Model:
/* Mutex with (bugged) Peterson algorithm */
bool flag[2];
int turn = -1;
active [2] proctype agent() {
bool in_cs;
int other = 1 - _pid;
do
:: true ->
/* trying section */
turn = other;
flag[_pid] = true;
!( flag[other] && (turn == other));
/* critical section */
in_cs = true;
printf("Agent %d in CS\n", _pid);
in_cs = false;
/* exit section */
flag[_pid] = false;
od;
}
ltl mutex { [] ! (agent[0]:in_cs && agent[1]:in_cs) }
To verify it, I use the following chain of commands:
~$ spin -a test.pml ; gcc -o run pan.c -DNOREDUCE ; ./run -a -N p1
which prints the following output:
...
Full statespace search for:
never claim + (mutex)
assertion violations + (if within scope of claim)
acceptance cycles + (fairness disabled)
invalid end states - (disabled by never claim)
State-vector 40 byte, depth reached 53, errors: 0
...
Q: i know for a fact that the Promela Model does not satisfy mutual exclusion, but Spin claims that it does. Why is it?
NOTE: I added option -DNOREDUCE when compiling with gcc because Spin 6.4.8 prints a warning message requesting me to do so if I don't:
~$ spin -a test.pml ; gcc -o run pan.c ; ./run -a -N p1
...
pan.c: In function ‘main’:
pan.c:9478:3: warning: #warning p.o. reduction not compatible with remote varrefs (use -DNOREDUCE) [-Wcpp]
#warning p.o. reduction not compatible with remote varrefs (use -DNOREDUCE)
^~~~~~~
...

Here is the solution:
The problem here is that to disable partial order reduction you not
only have to compile with -DNOREDUCE but also generate the model while
disabling statement merging (merging of subsequent local actions) --
which is also a partial order reduction strategy. So, for the second
example you also get the counter-example if you do:
spin -o3 -a test.pml; gcc -DNOREDUCE -o pan pan.c; ./pan -a
[G. H., private communication]
The output of the suggested solution matches our expectation:
~$ spin -o3 -a test.pml ; gcc -o run pan.c -DNOREDUCE ; ./run -a -N p1
...
pan:1: assertion violated !( !( !((agent[0].in_cs&&agent[1].in_cs)))) (at depth 55)
pan: wrote test.pml.trail
...
Full statespace search for:
never claim + (mutex)
assertion violations + (if within scope of claim)
acceptance cycles + (fairness disabled)
invalid end states - (disabled by never claim)
State-vector 40 byte, depth reached 69, errors: 1
...
Alternatively, one can circumvent the problem by using label remote references instead of variable remote references, e.g.:
/* Mutex with (bugged) Peterson algorithm */
bool flag[2];
int turn = -1;
active [2] proctype agent() {
int other = 1 - _pid;
do
:: true ->
/* trying section */
turn = other;
flag[_pid] = true;
!( flag[other] && (turn == other));
/* critical section */
cs:
printf("Agent %d in CS\n", _pid);
/* exit section */
flag[_pid] = false;
od;
}
ltl mutex { [] ! (agent[0]#cs && agent[1]#cs) }
With this Model, we find a counter-example, as expected:
~$ spin -search test.pml
...
pan:1: assertion violated !( !( !(((agent[0]._p==cs)&&(agent[1]._p==cs))))) (at depth 28)
pan: wrote test.pml.trail
...

Related

Why does system call fails?

I try to compile simple utility (kernel 3.4.67), which
all it does is trying to use system call very simply as following:
int main(void)
{
int rc;
printf("hello 1\n");
rc = system("echo hello again");
printf("system returned %d\n",rc);
rc = system("ls -l");
printf("system returned %d\n",rc);
return 0;
}
Yet, system call fails as you can see in the following log:
root#w812a_kk:/ # /sdcard/test
hello 1
system returned 32512
system returned 32512
I compile it as following:
arm-linux-gnueabihf-gcc -s -static -Wall -Wstrict-prototypes test.c -o test
That's really wierd becuase I used system in past in different linux and never had any issue with it.
I even tried another cross cpompiler but I get the same failure.
Version of kernel & cross compiler:
# busybox uname -a
Linux localhost 3.4.67 #1 SMP PREEMPT Wed Sep 28 18:18:33 CST 2016 armv7l GNU/Linux
arm-linux-gnueabihf-gcc --version
arm-linux-gnueabihf-gcc (crosstool-NG linaro-1.13.1-4.7-2013.03-20130313 - Linaro GCC 2013.03) 4.7.3 20130226 (prerelease)
EDIT:
root#w812a_kk:/ # echo hello again && echo $? && echo $0
hello again
0
tmp-mksh
root#w812a_kk:/ #
But I did find something interesting:
On calling test_expander() withing the main, it works OK. so I suspect that maybe system call try to find a binary which is not founded ?
int test_expander(void)
{
pid_t pid;
char *const parmList[] = {"/system/bin/busybox", "echo", "hello", NULL};
if ((pid = fork()) == -1)
perror("fork error");
else if (pid == 0) {
execv("/system/bin/busybox", parmList);
printf("Return not expected. Must be an execv error.n");
}
return 0;
}
Thank you for any idea.
Ran
The return value of system(), 32512 decimal, is 7F00 in hex.
This value is strangely similar to 0x7F, which is the result of system() if /bin/sh can not be executed. It seems there is some problem with byte ordering (big/little endian). Very strange.
Update: while writing the answer, you edited the question and pulled in something about /system/bin/busybox.
Probably you simply don't have /bin/sh.
I think I understand what happens
From system man page:
The system() library function uses fork(2) to create a child process
that executes the shell command specified in command using execl(3)
as follows:
execl("/bin/sh", "sh", "-c", command, (char *) 0);
But in my filesystem sh is founded only /system/bin , not in /bin
So I better just use execv instead. (I can't do static link becuase it's read-only filesystem)
Thanks,
Ran

systemtap global variable allocation failed

I want to use systemtap for extracting details of my linux production server. my systemtap script is
global bt;
global quit = 0
probe begin {
printf("start profiling...\n")
}
probe timer.profile {
if (pid() == target()) {
if (!quit)
{
bt[backtrace(), ubacktrace()] <<< 1
}
else
{
foreach ([sys, usr] in bt- limit 1000)
{
print_stack(sys)
print_ustack(usr)
printf("\t%d\n", #count(bt[sys, usr]))
}
exit()
}
}
}
probe timer.s(20) {
quit = 1
}
When I start run this script with command
sudo stap --ldd -d $program_name --all-modules \
-D MAXMAPENTRIES=10240 -D MAXACTION=20000 -D MAXTRACE=40 \
-D MAXSTRINGLEN=4096 -D MAXBACKTRACE=40 -x $program_pid \
profile.stp --vp 00001 > profile.out
It fails, and prints following error:
ERROR: error allocating hash
ERROR: global variable 'bt' allocation failed
WARNING: /usr/bin/staprun exited with status: 1
my production server memory info is
total used free shared buffers cached
Mem: 16008 15639 368 0 80 3090
-/+ buffers/cache: 12468 3539
I think it is enough, because in my test server, there is only 2G memory, and the systemtap script runs well for another server
Unfortunately, this is intended behavior, see my discussion here: https://sourceware.org/ml/systemtap/2015-q1/msg00033.html
The problem is that SystemTap allocates associative arrays at once (to prevent allocation failures in future) and on per-cpu basis (to prevent locking), which means that bt will require (2 * MAXSTRINGLEN + sizeof(statistic)) * MAXMAPENTRIES * NR_CPU =~ 2 Gb if NR_CPU == 128.
Reduce MAXSTRINGLEN (which is set to 4k in your case) or size of bt array:
global bt[128];

How to load LUKS passphrase from USB, falling back to keyboard?

I want to set up a headless Linux (Debian Wheezy) PC with whole disk encryption, with the ability to unlock the disk either with a USB drive, or by entering a passphrase by keyboard. My starting point is a fresh install using the basic whole disk encryption option in the Debian Installer, which manages everything besides /boot as a LUKS-encrypted logical volume group and gives me the keyboard option. I will describe my current solution in an answer, in hopes that it will be useful and that others can improve on it.
Here are some of the issues I had:
Setting up a passphrase and putting it on the USB drive.
Loading the USB modules in time.
Waiting for the USB drive to recognized by Linux before trying to read from it.
Identifying the correct USB drive (not some other drive that happens to be inserted).
Writing a "keyscript" to pull a passphrase off the USB drive.
Ensuring that the fall-back to keyboard kicks in in all USB failure cases.
I will accept an answer with significant improvements and upvote answers that offer contributions.
A lot of my solution is derived from the post, Using A USB Key For The LUKS Passphrase.
Create a random passphrase:
dd if=/dev/urandom bs=1 count=256 > passphrase
Insert a USB drive. dmesg output will show the device name; assume /dev/sdd. Figure out its size:
blockdev --getsize64 /dev/sdd
I decided to install the passphrase at the end of the raw device, figuring it might survive any accidental use of the USB drive.
dd if=passphrase of=/dev/sdd bs=1 seek=<size-256>
Add the passphrase to the LUKS volume:
cryptsetup luksAddKey /dev/sda5 passphrase
This does not affect the existing hand-entered passphrase from the installer. The passphrase file can be deleted:
rm passphrase
Find a unique name for the USB stick, so we can identify it when present:
ls -l /dev/disk/by-id | grep -w sdd
You should see one symlink. I will call it /dev/disk/by-id/<ID>.
Edit /etc/crypttab. You should see a line like:
sdc5_crypt UUID=b9570e0f-3bd3-40b0-801f-ee20ac460207 none luks
Modify it to:
sdc5_crypt UUID=b9570e0f-3bd3-40b0-801f-ee20ac460207 /dev/disk/by-id/<ID> luks,keyscript=/bin/passphrase-from-usb
The keyscript referred to above will need to read the passphrase from the USB device. However, it needs to do more than that. To understand how it is used, check /usr/share/initramfs-tools/scripts/local-top/cryptroot, the script that runs at boot time to unlock the root device. Note when a keyscript is set, it is simply run and the output piped to luksOpen with no other checking. There is no way to signal an error (USB drive not present) or fall back to keyboard input. If the passphrase fails, the keyscript is run again in a loop, up to some number of times; however we are not told which iteration we are on. Also, we have no control over when the keyscript is run, so we can't be sure Linux has recognized the USB drive.
I addressed this with some hacks:
Poll on the USB drive and wait 3 seconds for it to appear. This works for me, but I would love to know a better way.
Create a dummy file /passphrase-from-usb-tried on first run to indicate that we have been run at least once.
If we have been run at least once, or the USB drive cannot be found, run the askpass program used by cryptroot for keyboard input.
The final script:
#!/bin/sh
set -e
if ! [ -e /passphrase-from-usb-tried ]; then
touch /passphrase-from-usb-tried
if ! [ -e "$CRYPTTAB_KEY" ]; then
echo "Waiting for USB stick to be recognized..." >&2
sleep 3
fi
if [ -e "$CRYPTTAB_KEY" ]; then
echo "Unlocking the disk $CRYPTTAB_SOURCE ($CRYPTTAB_NAME) from USB key" >&2
dd if="$CRYPTTAB_KEY" bs=1 skip=129498880 count=256 2>/dev/null
exit
else
echo "Can't find $CRYPTTAB_KEY; USB stick not present?" >&2
fi
fi
/lib/cryptsetup/askpass "Unlocking the disk $CRYPTTAB_SOURCE ($CRYPTTAB_NAME)\nEnter passphrase: "
Finally, we need to ensure that this script is available in the initramfs. Create /etc/initramfs-tools/hooks/passphrase-from-usb containing:
#!/bin/sh
PREREQ=""
prereqs() {
echo "$PREREQ"
}
case "$1" in
prereqs)
prereqs
exit 0
;;
esac
. "${CONFDIR}/initramfs.conf"
. /usr/share/initramfs-tools/hook-functions
copy_exec /bin/passphrase-from-usb /bin
The USB drivers were not present in my initramfs. (It appears they are by default in later versions of Debian.) I had to add them by adding to /etc/initramfs-tools/modules:
uhci_hcd
ehci_hcd
usb_storage
When all is done, update the initramfs:
update-initramfs -u
It would be ideal to me if I could simply have a small USB stick containing a passphrase that will
unlock the disk. Not only would that be handy for servers (where you could leave the USB stick in the
server - the goal is to be able to return broken harddisks without having to worry about confidential data), it would also be great for my laptop: Insert the USB stick when booting and remove it after
unlocking the cryptodisk.
I have now written a patch that will search the root dir of all devices for the file 'cryptkey.txt' and try decrypting
with each line as a key. If that fails: Revert to typing in the pass phrase.
It does mean the key cannot contain \n, but that would apply to any typed in key, too. The good part is that you can use the same USB disk to store the key for multiple machines: You do not need a separate USB disk for each. So if you have a USB drive in your physical key ring, you can use the same drive for all the machines you boot when being physically close.
You add the key with:
cryptsetup luksAddKey /dev/sda5
And then put the same key as a line in a file on the USB/MMC disk called 'cryptkey.txt'. The patch is here:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=864647
If the USB drivers, MMC drivers or the filesystems are not present in your initramfs, you need to add them by adding to /etc/initramfs-tools/modules:
uhci_hcd
ehci_hcd
usb_storage
nls_utf8
nls_cp437
vfat
fat
sd_mod
mmc_block
tifm_sd
tifm_core
mmc_core
tifm_7xx1
sdhci
sdhci_pci
When all is done, update the initramfs:
update-initramfs -u
It can be found as patch and file at: https://gitlab.com/ole.tange/tangetools/tree/master/decrypt-root-with-usb
despite the great answer from #Andrew which works in previous versions. The solution actually is outdated and needs lots of tuning for ubuntu 18.04 and 19.10. So I want to share my research on this.
There are several catches about crypttab. The sepcs actually changed a lot from 14.04 to 18.04 and to 19.10. It starts to support more parameters for cryptsetup. For example keyfile-offset, keyfile-size, etc. Some of the options e.g. nobootwait are gone. Some parameters supported in other distro already but is not supported in ubuntu yet (for example very nice parameter keyfile-timeout. This can eliminate the entire keyscript since it will automatically fallback to keyboard input after the keyfile-timeout.)
The major pit-fall for crypttab on ubuntu is that it actually processed by 2 different processes. One is the traditionally initramfs and another is the modern systemd. Systemd is supposed to be more advanced and flexiable in many aspect. However, systemd has poor support for crypptab, there are many options such as keyscript just silently ignored. so I have no idea what is going on, until I spotted this post. Almost all the posts online about crypttab settings is for initramfs not for systemd. So we need to add initramfs to all the entries in crypttab to avoid problems.
I also discovered a nice way to debug our keyscript and crypttab without VM or repeatedly rebooting. It is cryptdisks_start. Before we actually propagate our changes to initramfs, we should always test it with this nice command. Otherwise, you have to end-up locked out from your system and can only recover it through chroot environment.
#andrew posted a nice way to use data hide in the raw area of a file system. However, I found it is very annoying when we want to automatically create partitions and dd the raw data to lots of usbkeys, we have to calculate the offset for all different file systems and different partition sizes. Moreover, if a user accidentally write onto the FS, there is some risk that the key got overritten. A raw partition without any FS on it makes more sense in this case. However raw partition does not have UUID which is not very useful for automatic unlocking. Thus, I would like introduce a way just use normal passphrase files on the usbkey filesystem. The major issue of passdev is it does not seek/stop during reading the file. Thus we cannot use the keyfile-offset and keyfile-size option when we want to fallback to keyboard input. Because cryptsetup will actually try to skip in the input content and if the content is shorter than keyfile-size, it raises an error. This also means if there is large offset, passdev can be very slow since it always read from beginning. However, there is no point to implement offset and keyfile size for a actual file on file system. I believe those are created for raw device.
The crypttab
luks-part UUID="<uuid>" /dev/disk/by-uuid/<keyfile FS uuid>:/<keyfile path relative to usbkey root>:<timeout in sec> luks,keyfile-offset=<seek to the key>,keyfile-size=<>,keyscript=/bin/passphrase-from-usbfs.sh,tries=<number of times to try>,initramfs
the keyscript passphrase-from-usbfs.sh utilized the /lib/cryptsetup/scripts/passdev which will wait the usb device and mount the fs then pipe out the file content. It supports the CRYPTTAB_KEY in format of /device-path/<keyfile FS uuid>:/<keyfile path relative to usbkey root>:<timeout in sec>.
#!/bin/sh
#all message need to echo to stderr, the stdout is used for passphrase
# TODO: we may need to do something about the plymouth
echo "CRYPTTAB_KEY=$CRYPTTAB_KEY" >&2
echo "CRYPTTAB_OPTION_keyfile_offset=$CRYPTTAB_OPTION_keyfile_offset" >&2
#set your offset and file size here if your system does not support those paramters
#CRYPTTAB_OPTION_keyfile_offset=
#CRYPTTAB_OPTION_keyfile_size=
echo "timeout=$CRYPTTAB_OPTION_keyfile_timeout" >&2
CRYPTTAB_OPTION_keyfile_timeout=10 # keyfile-timeout is not supported yet
pass=$(/lib/cryptsetup/scripts/passdev $CRYPTTAB_KEY)
rc=$?
if ! [ $rc -eq 0 ]; then
echo "Can't find $CRYPTTAB_KEY; USB stick not present?" >&2
/lib/cryptsetup/askpass "Unlocking the disk $CRYPTTAB_SOURCE ($CRYPTTAB_NAME) Enter passphrase: "
else
echo "successfully load passphrase." >&2
echo -n $pass
fi
The hook tell update-initramfs to copy our scripts.
#!/bin/sh
PREREQ=""
prereqs() {
echo "$PREREQ"
}
case "$1" in
prereqs)
prereqs
exit 0
;;
esac
. "${CONFDIR}/initramfs.conf"
. /usr/share/initramfs-tools/hook-functions
copy_exec /bin/passphrase-from-usbfs.sh
copy_exec /bin/passphrase-from-usb.sh
#when using passdev we need to hook additionaly FS and binary
copy_exec /lib/cryptsetup/scripts/passdev
manual_add_modules ext4 ext3 ext2 vfat btrfs reiserfs xfs jfs ntfs iso9660 udf
Finally I posted the updated version of passphrase-from-usb.sh which can use the new parameters in crypttab:
To accompany excellent answers above please see C routines you could use to write/generate and read raw block device key. The "readkey.c" extracts key of given size from block device and "writekey.c" can generate or write existing key to raw device. The "readkey.c" once compiled can be used in custom script to extract key of known size from raw block device like so:
readkey </path/to/device> <keysize>
To see usage for "writekey", after compiled run it with no flags.
To compile just use:
gcc readkey.c -o readkey
gcc writekey.c -o writekey
I tested both on Verbatim 16GB USB 2.0 USB flash drive with custom "keyscript=" in crypttab also published below. The idea for "crypto-usb.sh" is from "debian etch" cryptsetup guide.
crypto-usb.sh:
#!/bin/sh
echo ">>> Trying to get the key from agreed space <<<" >&2
modprobe usb-storage >/dev/null 2>&1
sleep 4
OPENED=0
disk="/sys/block/sdb"
boot_dir="/boot"
readkey="/boot/key/readkey"
echo ">>> Trying device: $disk <<<" >&2
F=$disk/dev
if [ 0`cat $disk/removable` -eq 1 -a -f $F ]; then
mkdir -p $boot_dir
mount /dev/sda1 $boot_dir -t ext2 >&2
echo ">>> Attempting key extraction <<<" >&2
if [ -f $readkey ]; then
# prints key array to the caller
$readkey /dev/sdb 4096
OPENED=1
fi
umount $boot_dir >&2
fi
if [ $OPENED -eq 0 ]; then
echo "!!! FAILED to find suitable key !!!" >&2
echo -n ">>> Try to enter your password: " >&2
read -s -r A
echo -n "$A"
else
echo ">>> Success loading key <<<" >&2
fi
When generating the key size of the key has to be provided, generated key is saved to ".tmpckey" file with file permissions 0600 for later use. When writing existing key, size is determined by measuring the existing key size. This looks like complex approach however once compiled with simple "gcc" it can provide easy way of manipulating the raw key content.
readkey.c:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
void main(int argc, char *argv[])
{
int blockSize = 512;
int keySize = 2048;
FILE *device;
if ( argc == 3
&& (sizeof(argv[1]) / sizeof(char)) > 1
&& (sizeof(argv[2]) / sizeof(char)) > 1
&& (atoi(argv[2]) % 512) == 0
) {
device = fopen(argv[1], "r");
if(device == NULL) {
printf("\nI got trouble opening the device %s\n", argv[1]);
exit(EXIT_FAILURE);
}
keySize = atoi(argv[2]);
}
else if ( argc == 2
&& (sizeof(argv[1]) / sizeof(char)) > 1
) {
device = fopen(argv[1], "r");
if(device == NULL) {
printf("\nI got trouble opening the device %s\n", argv[1]);
exit(EXIT_FAILURE);
}
}
else {
printf("\nUsage: \n");
printf("\nKey Size Provided: \n");
printf("\n\t\treadkey </path/to/device> <keysize> \n");
printf("\nDefault key size: %d\n", keySize);
printf("\n\t\treadkey </path/to/device>\n");
exit(1);
}
int count;
char *block;
/* Verify if key is multiple of blocks */
int numBlocks = 0;
if (keySize % 512 != 0) {
printf("\nSory but key size is not multiple of block size, try again. TA.\n");
exit(1);
}
/* Seek till the end to get disk size and position to start */
fseek(device, 0, SEEK_END);
/* Determine where is the end */
long endOfDisk = ftell(device);
/* Make sure we start again */
rewind(device); // Do I need it ???
/* Get the required amount minus block size */
long startFrom = endOfDisk - blockSize - keySize;
/* Allocate space for bloc */
block = calloc(keySize, sizeof(char));
/* Start reading from specified block */
fseek(device, startFrom, SEEK_SET);
fread(block, 1, keySize, device);
/* Do something with the data */
for(count = 0; count < keySize/*sizeof(block)*/; count++){
printf("%c", block[count]);
}
/* Close file */
fclose(device);
/* Make sure freed array is zeroed */
memset(block, 0, keySize);
free(block);
}
writekey.c:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(int argc, char *argv[])
{
int blockSize = 512;
int keySize = 2048;
int count;
unsigned char *block;
/*
Thing to always remember that argv starts from 0 - the name of the program, and argc starts from 1 i.e. 1 is the name of the program.
*/
if ( argc == 3
&& strcmp(argv[1], "genwrite") != 0
&& (sizeof(argv[2]) / sizeof(char)) > 2
) {
char ch;
FILE *keyF;
keyF = fopen(argv[1], "r");
if (keyF == NULL) exit(EXIT_FAILURE);
/* Tell key Size */
fseek(keyF, 0, SEEK_END);
keySize = ftell(keyF);
rewind(keyF);
printf("\nKey Size: %d\n", keySize);
block = calloc(keySize, sizeof(char));
printf("\n-- Start Key --:\n");
for(count = 0; count < keySize/*sizeof(block)*/; count++){
char ch = fgetc(keyF);
block[count] = ch;
/*
Uncomment below to see your key on screen
*/
// printf("%c",ch);
}
printf("\n-- End Key --:\n");
fclose(keyF);
}
else if ( argc == 3
&& strcmp(argv[1], "genwrite") == 0
&& (sizeof(argv[2]) / sizeof(char)) > 2
)
{
printf("\n-- Attempting to create random key(ish --) of size: %d\n", keySize);
block = calloc(keySize, sizeof(char));
int count;
for(count = 0; count < keySize/*sizeof(block)*/; count++){
block[count] = (char) rand();
}
FILE *tmpfile;
tmpfile = fopen(".tmpckey", "w");
if(tmpfile == NULL) exit(EXIT_FAILURE);
fwrite(block, 1, keySize, tmpfile);
fclose(tmpfile);
chmod(".tmpckey", 0600);
}
else if ( argc == 4
&& strcmp(argv[1], "genwrite") == 0
&& (sizeof(argv[2]) / sizeof(char)) > 2
&& ((atoi(argv[3]) % 512) == 0)
)
{
keySize = atoi(argv[3]);
printf("\n-- Attempting to create random key(ish --) of size: %d\n", keySize);
block = calloc(keySize, sizeof(char));
int count;
for(count = 0; count < keySize/*sizeof(block)*/; count++){
block[count] = (char) rand();
}
FILE *tmpfile;
tmpfile = fopen(".tmpckey", "w");
if(tmpfile == NULL) exit(EXIT_FAILURE);
fwrite(block, 1, keySize, tmpfile);
fclose(tmpfile);
chmod(".tmpckey", 0600);
}
else {
printf("\n");
printf("################################################################################\n");
printf("# #\n");
printf("# Usage: #\n");
printf("# #\n");
printf("################################################################################\n");
printf("#> To write existing key to device: #\n");
printf("# #\n");
printf("# writekey </path/to/keyfile> </path/to/removable/sd*> #\n");
printf("# #\n");
printf("#> To generate and write pseudo random key, #\n");
printf("#> key will be saved to temporary file .tmpckey #\n");
printf("# #\n");
printf("# writekey genwrite </path/to/removable/sd*> <keysize in multiples of 512> #\n");
printf("# #\n");
printf("#> When keysize is not provided default size is set to %d. #\n", keySize);
printf("# #\n");
printf("################################################################################\n");
exit(1);
}
/*
Some printf debugging below, uncomment when needed to see what is going on.
*/
/*
printf("\nNumber of Args: %d\n", argc);
printf("\nCurrently block array contains: \n");
for(count = 0; count < keySize; count++){
printf("%c", block[count]);
}
printf("\n-- End block -- \n");
*/
/* Open Device itp... */
FILE *device = fopen(argv[2], "a");
if(device == NULL) exit(EXIT_FAILURE);
printf("\nDevice to write: %s\n", argv[2]);
fseek(device, 0, SEEK_END);
/* Determine where is the end */
long endOfDisk = ftell(device);
printf("\nDevice Size: %ld\n", endOfDisk);
/* Verify if key is multiple of blocks */
int numBlocks = 0;
if (keySize % 512 != 0 || endOfDisk < (blockSize + keySize) ) {
printf("\nSorry but key size is not multiple of block size or device you trying to write to is too small, try again. TA.\n");
fclose(device);
exit(1);
}
/* Make sure we start again */
rewind(device);
/* Get the required amount sunbstracting block size */
long startFrom = endOfDisk - blockSize - keySize;
/* Write some data to the disk */
printf("\nWriting data starting from: %ld\n", startFrom);
fseek(device, startFrom, SEEK_SET);
fwrite(block, 1, keySize, device);
printf("\nBlock Position after data write procedure : %ld\n", ftell(device));
/*
Below is just for convenience, to read what was written,
can aid in debugging hence left commented for later.
*/
/*
printf("\nAmount of Data written : %ld\n", ftell(device) - startFrom);
// Start reading from specified block
printf("\n>>>>>>>> DEBUGGING SECTION <<<<<<<<<\n");
rewind(device); //
fseek(device, startFrom, SEEK_SET);
printf("\nBlock Position before read attempted: %d\n", ftell(device));
printf("\nKey size: %d\n", keySize);
fread(block, 1, keySize, device);
// Do something with the data
printf("\nBlock Position startFrom: %ld\n", startFrom);
printf("\nBlock Position after read: %d\n", ftell(device));
printf("\n-- Buffer Read: --\n");
for(count = 0; count < keySize; count++){
printf("%c", block[count]);
}
printf("\n-- End block -- \n");
printf("\n-- -- \n");
printf("\n-- -- \n");
*/
/* Close file */
fclose(device);
/* Make sure freed array is zeroed */
memset(block, 0, keySize);
free(block);
/* Return success, might change it to be useful return not place holder */
return 0;
}
To verify key written to raw device is the same as the one in file(below will output nothing if keys are identical):
diff -B <(./readkey </path/to/device> 4096) <(cat .tmpckey)
Or for existing key generated using own means:
diff -B <(./readkey </path/to/device> <generated elsewhere key size>) <(cat </path/to/keyfile>)
Thank You
Here is a solution similar to the one by Andrew, but
using CRYPTTAB_TRIED described in the Debian crypttab man page to distinguish tries, and
calling the existing standard keyscript /lib/cryptsetup/scripts/passdev on the first try.
Create your keyfile or keypartition as usual for the passdev script.
Create the following file /usr/local/bin/key-from-usb and make it executable.
#!/bin/sh
set -e
if [ $CRYPTTAB_TRIED -ge 1 ]; then
/lib/cryptsetup/askpass "Second try to unlock $CRYPTTAB_SOURCE ($CRYPTTAB_NAME). Please enter passphrase: "
else
/lib/cryptsetup/scripts/passdev $CRYPTTAB_KEY
fi
In /etc/crypttab use the parameter keyscript=/usr/local/bin/key-from-usb.
Create /etc/initramfs-tools/hooks/key-from-usb with this content:
#!/bin/sh
PREREQ=""
prereqs() {
echo "$PREREQ"
}
case "$1" in
prereqs)
prereqs
exit 0
;;
esac
. "${CONFDIR}/initramfs.conf"
. /usr/share/initramfs-tools/hook-functions
manual_add_modules vfat
copy_exec /usr/lib/cryptsetup/scripts/passdev /usr/lib/cryptsetup/scripts/passdev
copy_exec /usr/local/bin/key-from-usb /usr/local/bin/key-from-usb
The first copy_exec line here is needed because passdev is not copied if it is not mentioned in crypttab. Similarly, manual_add_modules vfat will ensure that a vfat usb disk can still be used.
Hint: Use lsinitramfs /boot/initrd.img-... and diff/compare the results to check that the script and all its dependencies are included.

How do I get an equivalent of /dev/one in Linux

You can use
dd if=/dev/zero of=file count=1024 bs=1024
to zero fill a file.
Instead of that I want to one fill a file. How do I do that?
There is no /dev/one file, so how can I simulate that effect via on bash shell?
tr '\0' '\377' < /dev/zero | dd bs=64K of=/dev/sdx
This should be much faster. Choose your blocksizes (or add counts) like you need at. Writing ones to a SSD-Disk till full with a blocksize of 99M gave me 350M/s write performance.
Try this:
dd if=<(yes $'\01' | tr -d "\n") of=file count=1024 bs=1024
Substitute $'\377' or $'\xFF' if you want all the bits to be ones.
MacOS tr may complain about "Illegal byte sequence". Setting LC_CTYPE=C will prevent that. This version can also be used in Linux:
dd if=<(yes $'\01' | LC_CTYPE=C tr -d "\n") of=file count=1024 bs=1024
Well, you could do this:
dd if=/dev/zero count=1024 bs=1024 |
tr '\000' '\001' > file
pv /dev/zero |tr \\000 \\377 >targetfile
...where \377 is the octal representation of 255 (a byte with all bits set to one). Why tr only works with octal numbers, I don't know -- but be careful not to subconsciously translate this to 3FF.
The syntax for using tr is error prone. I recommend verifying that it is making the desired translation...
cat /dev/zero |tr \\000 \\377 |hexdump -C
Note: pv is a nice utility that replaces cat and adds a progress/rate display.
I created a device driver in my github. Installing it creates a file /dev/one that is writing only bits set to 1.
The c file called one.c (the only interesting part is in device_file_read):
// File Driver to create a devince /dev/one like the /dev/zero
#include <linux/init.h>
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/uaccess.h>
MODULE_LICENSE("GPL");
static int device_file_major_number = 0;
static const char device_name[] = "one";
static ssize_t device_file_read(
struct file *file_ptr,
char __user *user_buffer,
size_t count,
loff_t *position) {
printk( KERN_NOTICE "One: Device file is read at offset = %i, read bytes count = %u\n" , (int)*position , (unsigned int)count );
// Allocate Kernel buffer
char* ptr = (char*) vmalloc(count);
// Fill it with one, byte per byte
// -- Note that byte is the smallest accesible data unit
memset(ptr, 0xFF, count);
char res = copy_to_user(user_buffer, ptr, count);
if (res != 0){ return -EFAULT; }
// Return number of byte read
return count;
}
static struct file_operations simple_driver_fops = {
.owner = THIS_MODULE,
.read = device_file_read,
};
int register_device(void) {
int res = 0;
printk( KERN_NOTICE "One: register_device() is called.\n" );
res = register_chrdev( 0, device_name, &simple_driver_fops );
if( res < 0 ) {
printk( KERN_WARNING "One: can\'t register character device with error code = %i\n", res );
return res;
}
device_file_major_number = res;
printk( KERN_NOTICE "One: registered character device with major number = %i and minor numbers 0...255\n", device_file_major_number );
return 0;
}
void unregister_device(void) {
printk( KERN_NOTICE "One: unregister_device() is called\n" );
if(device_file_major_number != 0) {
unregister_chrdev(device_file_major_number, device_name);
}
}
static int my_init(void) {
register_device();
return 0;
}
static void my_exit(void) {
unregister_device();
return;
}
// Declare register and unregister command
module_init(my_init);
module_exit(my_exit);
The Makefile
TARGET_MODULE:=one
BUILDSYSTEM_DIR:=/lib/modules/$(shell uname -r)/build
PWD:=$(shell pwd)
obj-m := $(TARGET_MODULE).o
# See: https://stackoverflow.com/questions/15910064/how-to-compile-a-linux-kernel-module-using-std-gnu99
ccflags-y := -std=gnu99 -Wno-declaration-after-statement
build:
# run kernel build system to make module
$(MAKE) -C $(BUILDSYSTEM_DIR) M=$(PWD) modules
clean:
# run kernel build system to cleanup in current directory
$(MAKE) -C $(BUILDSYSTEM_DIR) M=$(PWD) clean
rm -f MOK.priv MOK*.der
key:
echo "Creating key"
openssl req -new -x509 -newkey rsa:2048 -days 36500 -keyout MOK.priv -outform DER -out MOK.der -nodes -subj "/CN=TinmarinoUnsafe/"
#
echo "\e[31;1mPlease enter a password you will be asked for on reboot:\e[0m"
mokutil --import MOK.der
echo "\e[31;1mNow you must: 1/ reboot, 2/ Select Unroll MOK, 3/ Enter password you previously gave\e[0m"
sign:
cp one.ko one.ko.bck
/usr/src/linux-headers-$(shell uname -r)/scripts/sign-file sha256 MOK.priv MOK.der one.ko
load:
insmod ./$(TARGET_MODULE).ko
unload:
rmmod ./$(TARGET_MODULE).ko
create:
mknod /dev/one c $(shell cat /proc/devices | grep one$ | cut -d ' ' -f1) 0
delete:
rm /dev/one
test:
[ "$(shell xxd -p -l 10 /dev/one)" = "ffffffffffffffffffff" ] \
&& echo "\e[32mSUCCESS\e[0m" \
|| echo "\e[31mFAILED\e[0m"
The instalation is long (3min) due to the driver signature enforcement. Froget this part if you disabled it in your UEFI.
git clone https://github.com/tinmarino/dev_one.git DevOne && cd DevOne # Download
make build # Compile
make key # Generate key for signing
sudo make sign # Sign driver module to permit MOK enforcement (security)
sudo reboot now # Reboot and enable Mok
A blue screen (MOK manager) will appear
Choose "Enroll MOK"
Choose "Continue"
Choose "Yes" (when asked "Enroll the key")
Enter the password you gave at make sign
Choose "Reboot" (again)
sudo make load # Load
sudo make device # Create /dev/one
make test # Test if all is ok
You can simulate a /dev/one without a special device, with a FIFO + yes:
mkfifo ddfifo
dd if=ddfifo of=<file> iflag=fullblock count=1024 bs=1024 status=progress & yes "" | tr '\n' '\1' > ddfifo
tee may be used to double the throughput:
mkfifo ddfifo
dd if=ddfifo of=<file> iflag=fullblock count=1024 bs=1024 status=progress & yes "" | tr '\n' '\1' | tee ddfifo > ddfifo
If you'd like bytes with all bits set to one, swap '\1' for '\377'.

Why is there a difference using std::thread::hardware_concurrency() and boost::thread::hardware_concurrency()?

The description of the problem itself is pretty simple. I'm testing the differences of std::thread library in C++11 and boost::thread library.
The output of these:
#include <iostream>
#include <thread>
#include <boost/thread.hpp>
int main() {
std::cout << std::thread::hardware_concurrency() << std::endl;
std::cout << boost::thread::hardware_concurrency() << std::endl;
return 0;
}
gives me different results:
0
4
Why is that?
PS: The version of the gcc package is 4.6.2-1.fc16 (x86_64). I'm using
g++ test.cc -Wall -std=c++0x -lboost_thread-mt -lpthread
After reviewing /usr/include/c++/4.6.2/thread
it can be seen that the implementation is actually:
// Returns a value that hints at the number of hardware thread contexts.
static unsigned int
hardware_concurrency()
{ return 0; }
So problem solved. It's just another feature that hasn't been implemented in gcc 4.6.2
The method employed by your compiler installation of boost is supported for your target, whereas your installation of boost compiler does not support this feature for your target.
TFM says:
The number of hardware threads available on the current system (e.g. number of CPUs or cores or hyperthreading units), or 0 if this information is not available.
EDIT: scratch that, reverse it.
EDIT2: This feature is present on the trunk, but absent in 4.6.2:
~/tmp/gcc-4.6.2/libstdc++-v3/src> wc -l thread.cc
104 thread.cc
~/tmp/gcc-4.6.2/libstdc++-v3/src> grep concurrency thread.cc | wc -l
0
~/tmp/gcc-4.6.2/libstdc++-v3> grep -C 2 VERIFY testsuite/30_threads/thread/members/hardware_concurrency.cc
// Current implementation punts on this.
VERIFY( std::thread::hardware_concurrency() == 0 );
return 0;

Resources