I am trying to read a single specific sector from the disk directly. I've currently run out of ideas and any suggestions how to go about it would be great!
Try something like this to do it from the CLI:
# df -h .
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 27G 24G 1.6G 94% /
# dd bs=512 if=/dev/sda2 of=/tmp/sector200 skip=200 count=1
1+0 records in
1+0 records out
From man 4 sd:
FILES
/dev/sd[a-h]: the whole device
/dev/sd[a-h][0-8]: individual block partitions
And if you want to do this from within a program, just use a combination of system calls from man 2 ... like open, lseek,, and read, with the parameters from the dd example.
I'm not sure what the best programmatic approach is, but from the Linux command-line you could use the dd command in combination with the raw device for your disk to directly read from the disk.
You need to sudo this command to get access to the raw disk device (e.g. /dev/rdisk0).
For example, the following will read a single 512-byte block from an offset of 900 blocks from the top of disk0 and output it to stdout.
sudo dd if=/dev/rdisk0 bs=512 skip=900 count=1
See the dd man page to get additional information on the parameters to dd.
In C it is something like the following... It would require root permissions. I think you need to open the file with O_DIRECT if you want to read single sectors. Otherwise you'll get a page. I'm not sure if the aligned buffer is required for a read, but it is for a write.
#include <stdio.h>
#include <errno.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#define SECTOR_SIZE 512
int main(int argc, char *argv[]) {
int offset = 0;
int length = 5;
int rc = -1;
char *sector = aligned_alloc(SECTOR_SIZE, SECTOR_SIZE);
memset(sector, 0, SECTOR_SIZE);
/* replace XXX with the source block device */
int fd=open("/dev/XXX", O_RDWR | O_DIRECT);
lseek(fd, offset, SEEK_SET);
for (int i = 0; i < length; i++) {
rc = read(fd, sector, SECTOR_SIZE);
if (rc < 0)
printf("sector read error at offset = %d + %d\n %s", offset, i, strerror(errno));
printf("Sector: %d\n", i);
for (int j = 0; j < SECTOR_SIZE; j++) {
printf("%x", sector[i]);
if ((j + 1) % 16 == 0)
printf("\n");
}
}
free(sector);
close(fd);
}
The other folks have pretty much covered it. You need to
access to the disk's device file (either be root or, better, change the permissions on it)
use the file IO functions to read sectors = chunks of (usually) 512 bytes from said disk.
Another alternative is to use hdparm
For instance-
hdparm --read-sector 16782858 /dev/sda
Related
I am trying to mmap a 64M file into memory, then read & write into it. But sometimes a certain range within this file will no longer be used, so I called fallocate(fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, 16 << 20, 16 << 20); to release a 16M range inside it.
However, I find that the memory consumption seems not changed(both from free -m and from cat /proc/meminfo).
I understand that fallocate will dig some holes inside the file and return such range back to the filesystem. But I'm not sure whether it will reduce memory consumption for mmaped files.
If it does not reduce the memory consumption, where does such range go? Can another process get the underlying memory previously allocated to it?
The big.file is a normal 64M file instead of a sparse file
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <linux/falloc.h>
#include <stdint.h>
int main(int argc, char *argv[])
{
uint8_t *addr;
int fd = open("home/big.file", O_RDWR | O_CREAT, 0777);
if (fd < 0)
return -1;
addr = (uint8_t *)mmap(NULL, MMAP_SIZE , PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
if (addr < 0)
return -1;
printf("data[0x%x] = %d\n", offset, addr[offset]);
getchar();
fallocate(fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, 16 << 20, 16 << 20);
getchar();
close(fd);
return 0;
}
I am running a test to check disk read statistics. Here is the code for the same:
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
int main(int argc, char* argv)
{
int count=1000, size;
char block[4096]="0";
int fd = open("file1.txt",O_RDONLY | O_SYNC);
//int pid = getpid();
system("pid=$(ps -a | grep 'a.out' | awk '{print $1}'); iotop -bokp $pid > test1c.out &");
system("echo 'Starts reading in 10'");
srand(time(NULL));
system("sleep 1");
while(count--){
int random = (rand()%16)*666;
printf("%d;",random);
lseek(fd, random, SEEK_SET);
size = read(fd,block,4096);
printf("Number of bytes read: %d\n", size);
fsync(fd);
//printf("Read 4kb from the file.\n");
}
system("sleep 1");
system("killall iotop");
}
As you can see, I am opening a large file, getting the PID of my a.out file, and passing it to iotop. After that I am randomly seeking to a 4kb block in the file and reading data.
If you run this code on your system, you'll realize that iotop output shows 0kb reads throughout, which makes no sense. Am I doing something wrong?
Clearing the caches solved the issue. I found the script for clearing caches on this page:
https://www.tecmint.com/clear-ram-memory-cache-buffer-and-swap-space-on-linux/
sync; echo 1 > /proc/sys/vm/drop_caches
sync; echo 2 > /proc/sys/vm/drop_caches
sync; echo 3 > /proc/sys/vm/drop_caches
Does the trick!
I am trying to read data sent from the tty/USB0 and print it out with byte format.
Question:
I expect to print out the data once the reading bytes reach 40 However, the time takes much longer than I expect. The read() system call hangs and I believe the data should already be larger than 40. The data will finally be printed out but it should not take so long. Did I make anything wrong in this programming ?
thanks
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <termios.h>
#include <stdio.h>
#define BAUDRATE B9600
#define MODEMDEVICE "/dev/ttyUSB0"
#define FALSE 0
#define TRUE 1
main()
{
int fd,c, res;
struct termios oldtio,newtio;
unsigned char buf[40];
fd = open(MODEMDEVICE, O_RDWR | O_NOCTTY );
if (fd <0) {perror(MODEMDEVICE); exit(-1); }
tcgetattr(fd,&oldtio);
bzero(&newtio, sizeof(newtio));
newtio.c_cflag = BAUDRATE | CS8 | CLOCAL | CREAD;
newtio.c_iflag = IGNPAR | ICRNL;
newtio.c_oflag = 1;
newtio.c_lflag = ICANON;
tcflush(fd, TCIOFLUSH);
tcsetattr(fd,TCSANOW,&newtio);
int i;
while (1) {
res = read(fd,buf,40);
if(res==40){
printf("res reaches 40 \n");
}
printf("res: %d\n",res);
for(i=0;i<res;++i){
printf("%02x ", buf[i]);
}
return;
}
}
--------------------raw mode code------------------------
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <termios.h>
#include <stdio.h>
#define BAUDRATE B9600
#define MODEMDEVICE "/dev/ttyUSB0"
#define _POSIX_SOURCE 1 /* POSIX compliant source */
#define FALSE 0
#define TRUE 1
volatile int STOP=FALSE;
main()
{
int fd,c, res;
struct termios oldtio,newtio;
unsigned char buf[255];
fd = open(MODEMDEVICE, O_RDWR | O_NOCTTY );
if (fd <0) {perror(MODEMDEVICE); exit(-1); }
tcgetattr(fd,&oldtio); /* save current port settings */
bzero(&newtio, sizeof(newtio));
newtio.c_cflag = BAUDRATE | CRTSCTS | CS8 | CLOCAL | CREAD;
newtio.c_iflag = IGNPAR;
newtio.c_oflag = 0;
/* set input mode (non-canonical, no echo,...) */
newtio.c_lflag = 0;
newtio.c_cc[VTIME] = 0;
newtio.c_cc[VMIN] = 40;
tcflush(fd, TCIFLUSH);
tcsetattr(fd,TCSANOW,&newtio);
int i;
while (STOP==FALSE) {
res = read(fd,buf,255);
for( i=0;i<res;++i){
printf("%02x \n", buf[i]);
}
}
tcsetattr(fd,TCSANOW,&oldtio);
}
It now can print out the data once buffer capacity is full ( which is 40 ).
1 question:
When I modified the printf
printf("%02x ", buf[i]); ( remove "\n" )
It will not print out when the buffer is full until more bytes are received. Why this happens?
Thanks
You need to switch the terminal to raw mode to disable line buffering.
Citing this answer:
The terms raw and cooked only apply to terminal drivers. "Cooked" is
called canonical and "raw" is called non-canonical mode.
The terminal driver is, by default a line-based system: characters are
buffered internally until a carriage return (Enter or Return) before
it is passed to the program - this is called "cooked". This allows
certain characters to be processed (see stty(1)), such as Cntl-D,
Cntl-S, Ctrl-U Backspace); essentially rudimentary line-editing. The
terminal driver "cooks" the characters before serving them up.
The terminal can be placed into "raw" mode where the characters are
not processed by the terminal driver, but are sent straight through
(it can be set that INTR and QUIT characters are still processed).
This allows programs like emacs and vi to use the entire screen more
easily.
You can read more about this in the "Canonical mode" section of the
termios(3) manpage.
See e.g. this or this how to achieve that programmatically (did not check the code, but it should be easy to find it).
Alternatively you could use e.g. strace or ltrace to check what stty -F /dev/ttyUSB0 raw does (or read the manual page where it is described).
EDIT>
Regarding printf without a newline -- fflush(stdout); immediately after it should help (another line-buffering is taking place).
You might consider reading this and maybe this.
On windows there is this program called pcitree that allows you to set and read memory without writing a device driver. Is there a linux alternative to pcitree that will allow me read memory on block 0 of my pcie card?
A simple use case would be that I use driver code to write a 32bit integer on the first memory address in block zero of my pci-e card. I then use pcitree alternative to read the value at the first memory address of block zero and see my integer.
Thank you
I found some code online that does what I want here github.com/billfarrow/pcimem.
As I understand it this link offers code that maps kernel memory to user memory via the system call "mmap"
This was mostly stolen from the readme of the program, and the man pages of mmap.
mmap takes
a start address
a size
memory protection flags
file descriptor that that is linked to bar0 of your pci-card.
and an offset
mmap returns a userspace pointer to the memory defined by the start address and size parameters.
This code shows an example of mmaps usage.
//The file handle can be found by typing "lspci -v "
// and looking for your device.
fd = open("/sys/devices/pci0001\:00/0001\:00\:07.0/resource0", O_RDWR | O_SYNC);
//mmap returns a userspace address
//0, 4096 tells it to pull one page
ptr = mmap(0, 4096, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
printf("PCI BAR0 0x0000 = 0x%4x\n", *((unsigned short *) ptr);
I use the way to get PCI BAR0 register described above but get the segmentation fault back. I use gdb to debug the error from my code as follows and it shows the return value of mmap() is (void *) 0xffffffffffffffff
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <signal.h>
#include <fcntl.h>
#include <ctype.h>
#include <termios.h>
#include <sys/types.h>
#include <sys/mman.h>
#define PRINT_ERROR \
do { \
fprintf(stderr, "Error at line %d, file %s (%d) [%s]\n", \
__LINE__, __FILE__, errno, strerror(errno)); exit(1); \
} while(0)
#define MAP_SIZE 4096UL
#define MAP_MASK (MAP_SIZE - 1)
int main(int argc, char **argv) {
int fd;
void *ptr;
//The file handle can be found by typing lscpi -v
//and looking for your device.
fd = open("/sys/bus/pci/devices/0000\:00\:05.0/resource0", O_RDWR | O_SYNC);
//mmap returns a userspace address
//0, 4096 tells it to pull one page
ptr = mmap(0, 4096, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
printf("PCI BAR0 0x0000 = 0x%4x\n", *((unsigned short *) ptr));
if(munmap(ptr, 4096) == -1) PRINT_ERROR;
close(fd);
return 0;
}
On a system with functioning /dev/mem in the kernel it is possible to read a bar for a device using:
sudo dd if=/dev/mem skip=13701120 count=1 bs=256 | hexdump
Look at the dd man page. In the above example 13701120 * 256 is the start physical address at which 256 bytes will be read.
I'm writing an (un)archiving tool and the way it is designed it first creates a regular file from the archive before it examines the special attributes and may decide that this item is a symlink, in fact.
Note: Before more people misunderstand me for wanting to make a symlink of a file. No, I write the symlink data, i.e. its path, into the file, and then I want to tell the file system that this is a symlink
I've been developing this on OS X, where it's possible to turn a regular file into a symlink by simply setting its Type and Creator codes accordingly.
Now I like to get this code working on Linux as well. So I like to find a similar way there.
I am aware that the normal way to create a symlink is to call the symlink() function, but I wonder if there is also a way to change a regular file into a symlink, just like it's possible in OSX's BSD system, so that I do not have to refactor my working code too much?
There is lstat(), which returns the file type in st_mode's upmost bits. Now I wonder if there's also an analogous setter function for this mode field.
I don't believe there is a way in Linux to do this as you describe. IIRC, the filesystem stores symlink information in the inode table and not in a regular file so there's no direct way of turning a file into a link.
If the symlink's path is stored inside the file, why not read out the path, delete the file, and create a symlink in its place?
Demonstrating what I wrote as a comment to bmarguiles's answer,
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
int main(int argc, char **argv) {
char *buffer = 0, *name = 0;
int i;
for (i = 1; i < argc; i++) {
struct stat st;
int fd = open(argv[i], O_RDONLY);
fstat(fd, &st);
buffer = realloc(buffer, st.st_size + 1);
read(fd, buffer, st.st_size);
close(fd);
buffer[st.st_size] = '\0';
name = realloc(name, strlen(argv[i]) + 2);
sprintf(name, "%s~", argv[i]);
symlink(buffer, name);
rename(name, argv[i]);
}
free(buffer);
free(name);
return 0;
}
$ vi f2s.c
...
$ cc -o f2s f2s.c
$ echo -n / > test
$ ./f2s test
$ ls -l test
lrwxrwxrwx 1 me me 1 Feb 24 23:17 test -> /
$ echo -n / > test2
$ strace ./f2s test2
open("test2", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=1, ...}) = 0
read(3, "/", 1) = 1
close(3) = 0
symlink("/", "test2~") = 0
rename("test2~", "test2") = 0
This is just a demonstration; it really needs more error-handling and maybe a better temporary filename.
No, you can't turn one into the other. You have to unlink to kill the file and then symlink to create a symlink as a replacement.