I am trying to get performance of mounted Sd card to my board and i am using Iozone tool to do that but i am getting starnge results:
command:
# mount /dev/mmcblk2p2 /mnt/SD
# cd /mnt/SD
# iozone -a -s 10M -r 5K -w -e
results:
random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
10240 5 4283 4136 68681 378738 337652 3871 133905 96074 216912 4122 5013 364024 376181
the results are in Kbytes that's mean the speed random read is 300MB/s ??
my card is class 4 normally the write speed is 4 MB/s and the reading speed is not very different to this value ??
iozone -a -s 10M -r 5K -w -e
random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
10240 5 4283 4136 68681 378738 337652 3871 133905 96074 216912 4122 5013 364024 376181
Yes, your results are in kilobyte/s (KB/s; don't use -s silent option and iozone will say it Output is in kBytes/sec), and yes, there was 380 MB/s for "reread" speed (and 200 MB/s for read after reread?). But reread may be not the speed of your block device (SD card/HDD/SSD) if you test set (10 MB) is smaller than your RAM amount (it is).
Most OS (and Linux too) have software cache-in-RAM for filesystems and block devices. When you access some block for first time (since boot), it will be read from the device and stored in Page Cache of OS. Next access (read) of this block will be served directly from RAM, not from the device itself (unless O_DIRECT option was used in I/O operation, -I option of iozone).
So, your test run is incorrect. Read man page of iozone before use: http://linux.die.net/man/1/iozone and try bigger test set (gigabytes) or use -I to bypass page cache.
here is the results when i am using the -I option
random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
10240 1024 2356 2950 19693 20865 20833 2095 20111 1734 14375 2875 3566 386809 389443
write seq : 2,3 Mo/s
read seq: 19,2 Mo/s
write rand: 2 Mo/s
read rand: 20 Mo/s
read blk 20 Mo/s
why the read speed still so high ?
Related
Within a bash script i need to get the total disk size and the currently used size of the complete disk.
I know i can get the total disk size without needed to be root with this command:
cat /sys/block/sda/size
This command will output the count of blocks on device SDA.
Multiply it with 512 and you'll get the amount of bytes on this device.
This is sufficient with the total disk size.
Now for the currently used space. I want to get this value without being root.
I can assume the device name is SDA.
Now there is this command: df
I thought i could use this command but it seems this command only outputs data of the currently mounted partitions.
Is there a way to get a total of space used on disk SDA without needing to be root and not all partitions needs to be mounted?
Let's assume the following example:
/dev/sda1 80GB Linux partition 20GB Used
/dev/sda2 80GB Linux Home partition 20GB Used
/dev/sda3 100GB Windows Parition. 30GB Used
Let's assume my SDA disk is partitioned like above. But while i'm on Linux my Windows partition (sda3) is not mounted.
The output of df will give me a grand total of 40 GB Used so it doesn't take sda3 in account.
Again the question:
Is there a way without root to get me a grand total of 70 GB used space?
I think stat command should help. You can get the get the partitions from /proc/partitions.
Sample stat command output:
$ stat -f /dev/sda1
File: "/dev/sda1"
ID: 0 Namelen: 255 Type: tmpfs
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 237009 Free: 236970 Available: 236970
Inodes: Total: 237009 Free: 236386
You can use df.
df -h --output='used' /home
Used
3.2G
If you combine this with some sed or awk you can have the value you seek
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 9 years ago.
Improve this question
Please do not waste anymore of your time on this question...I ended up deleting the whole VM and creating another. The time it took me to do this is less than the time it would take to fix the issue. I have couple of SSDs in RAID mode.
Thank you for all those who tried to troubleshoot the issue!
I am having this problem with ubnuntu not showing active swap spaces when I run the command cat /proc/swaps. Here is a list of commands I ran. I even added a new swap space (file: /swapfile1) just to make sure that at least one swap space, but still I get nothing.
hebbo#ubuntu-12-lts:~$ sudo fdisk -l
[sudo] password for hebbo:
Disk /dev/sda: 26.8 GB, 26843545600 bytes
255 heads, 63 sectors/track, 3263 cylinders, total 52428800 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000e3a7a
Device Boot Start End Blocks Id System
/dev/sda1 * 46569472 52426751 2928640 82 Linux swap / Solaris
/dev/sda2 2046 46567423 23282689 5 Extended
/dev/sda5 2048 46567423 23282688 83 Linux
Partition table entries are not in disk order
hebbo#ubuntu-12-lts:~$ sudo su
root#ubuntu-12-lts:/home/hebbo# cat /proc/swaps
Filename Type Size Used Priority
root#ubuntu-12-lts:/home/hebbo# dd if=/dev/zero of=/swapfile1 bs=1024 count=524288
524288+0 records in
524288+0 records out
536870912 bytes (537 MB) copied, 1.18755 s, 452 MB/s
root#ubuntu-12-lts:/home/hebbo# mkswap /swapfile1
Setting up swapspace version 1, size = 524284 KiB
no label, UUID=cb846612-5f27-428f-9f83-bbe24b410a78
root#ubuntu-12-lts:/home/hebbo# chown root:root /swapfile1
root#ubuntu-12-lts:/home/hebbo# chmod 0600 /swapfile1
root#ubuntu-12-lts:/home/hebbo# swapon /swapfile1
root#ubuntu-12-lts:/home/hebbo# cat /proc/swaps
Filename Type Size Used Priority
root#ubuntu-12-lts:/home/hebbo#
Any idea how to fix this?
This is ubuntu 12.04 LTS running kernel 3.9.0 in a vmware VM.
Thanks in advance!
To activate /swapfile1 after Linux system reboot, add entry to /etc/fstab file. Open this file using a text editor such as vi:
# vi /etc/fstab
Add the following line:
/swapfile1 swap swap defaults 0 0
Save and close the file. Next time Linux comes up after reboot, it enables the new swap file for you automatically.
Have a look here for more info.
I just tried it and it works on my box.
Linux fileserver 3.8.0-32-generic #47~precise1-Ubuntu SMP Wed Oct 2 16:19:35 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
ortang#fileserver:~$ cat /proc/swaps
Filename Type Size Used Priority
/dev/dm-2 partition 4194300 0 -1
ortang#fileserver:~$ sudo su
root#fileserver:/home/ortang# dd if=/dev/zero of=/swapfile bs=1M count=512
512+0 records in
512+0 records out
536870912 bytes (537 MB) copied, 0.695721 s, 772 MB/s
root#fileserver:/home/ortang# chmod 600 /swapfile
root#fileserver:/home/ortang# mkswap /swapfile
Setting up swapspace version 1, size = 524284 KiB
no label, UUID=63cdcf3d-ba03-42ce-b598-15b6aa3ca67d
root#fileserver:/home/ortang# swapon /swapfile
root#fileserver:/home/ortang# cat /proc/swaps
Filename Type Size Used Priority
/dev/dm-2 partition 4194300 0 -1
/swapfile file 524284 0 -2
One thing i can imagine why it is working on my box, is that i already have a working swap partition, and it seems you don't.
It could also be caused by the kernel you use, 3.9.0 is not the regular 12.04.3 LTS kernel? Have you built the kernel yourself?
Whats the output of
grep CONFIG_SWAP /boot/config-`uname -r`
or
zcat /proc/config.gz | grep CONFIG_SWAP
is swap enabled in your kernel?
I ended up deleting the whole VM and creating another. The time it took me to do this is less than the time it would take to fix the issue. I have couple of SSDs in RAID mode. And I already had all the downloads on the same host machine. All in all ~7 minutes.
Thanks for all those who helped troubleshoot the issue.
I believe /images/backups is using the space in /images ?
/dev/sdb1 820G 645G 135G 83% /images
/dev/loop0 296G 296G 0 100% /images/backups
I've a similar kind of partition in another machine /images which is 500G free, and I would want to take out 350G for /images/backups, how to do it ?
Is it right that, it is a simple loop mount which can give specified amount of space or we should create a NULL file of required size and mount ? If so, what are the mount options should be used to specify the size ?
You'll need to create the destination with a fixed size, but can use a "sparse file" which doesn't actually have any blocks written to it yet (and which thus doesn't actually consume space until you write to it).
For instance:
dd if=/dev/zero of=file.img bs=1 count=0 seek=20G
will create a sparse file preallocated to 20GB. That said, actually writing 20GB of zeros to disk up-front (making the file non-sparse) will be faster on writes and lead to less fragmentation.
This can be attached to a loopback device with the losetup command, have a filesystem created, and be mounted:
losetup /dev/loop1 file.img
mke2fs -j /dev/loop1
mount /dev/loop1 /mnt/somewhere
If you want to know if an existing file is sparse, the following will do the trick (on a system with GNU tools; some of the below is not supported in a pure POSIX environment):
{
read block_count block_size file_size
if (( block_count * block_size < file_size )) ; then
echo "Sparse"
else
echo "Non-Sparse"
fi
} < <(stat --format='%b %B %s'$'\n' /images/backups.img)
I have read this question:
How to create a file with a given size in Linux?
But I havent got answer to my question.
I want to create a file of 372.07 MB,
I tried the following commands in Ubuntu 10.08:
dd if=/dev/zero of=output.dat bs=390143672 count=1
dd: memory exhausted
390143672=372.07*1024*1024
Is there any other methods?
Thanks a lot!
Edit:
How to view a file's size on Linux command line with decimal. I mean, the command line ls -hl just says: '373M' but the file is actually "372.07M".
Sparse file
dd of=output.dat bs=1 seek=390143672 count=0
This has the added benefit of creating the file sparse if the underlying filesystem supports that. This means, no space is wasted if some of the pages (_blocks) ever get written to and the file creation is extremely quick.
Non-sparse (opaque) file:
Edit since people have, rightly pointed out that sparse files have characteristics that could be disadvantageous in some scenarios, here is the sweet point:
You could use fallocate (in Debian present due to util-linux) instead:
fallocate -l 390143672 output.dat
This still has the benefit of not needing to actually write the blocks, so it is pretty much as quick as creating the sparse file, but it is not sparse. Best Of Both Worlds.
Change your parameters:
dd if=/dev/zero of=output.dat bs=1 count=390143672
otherwise dd tries to create a 370MB buffer in memory.
If you want to do it more efficiently, write the 372MB part first with large-ish blocks (say 1M), then write the tail part with 1 byte blocks by using the seek option to go to the end of the file first.
Ex:
dd if=/dev/zero of=./output.dat bs=1M count=1
dd if=/dev/zero of=./output.dat seek=1M bs=1 count=42
truncate - shrink or extend the size of a file to the specified size
The following example truncates putty.log from 298 bytes to 235 bytes.
root#ubuntu:~# ls -l putty.log
-rw-r--r-- 1 root root 298 2013-10-11 03:01 putty.log
root#ubuntu:~# truncate putty.log -s 235
root#ubuntu:~# ls -l putty.log
-rw-r--r-- 1 root root 235 2013-10-14 19:07 putty.log
Swap count and bs. bs bytes will be in memory, so it can't be that big.
On the command line this can be found out using the 'free' utility and 'cat /proc/meminfo'.
What would be the different ways to find out the physical RAM size in Linux programatically from a :
Userspace Application
Kernel Module
What API calls are available ?
#include <unistd.h>
long long physical_mem_bytes = (long long) sysconf (_SC_PHYS_PAGES) * sysconf (_SC_PAGESIZE);
Other than the command line ulimit, I don't know of a way of finding maximum memory for an individual process.
Programmatically, Linux won't tell you the actual physical size. Instead you should read this info from SMBIOS with, e.g.,
sudo dmidecode -t memory | fgrep -ie 'size:'
This will give you results like the following (from a box with 4 RAM banks, only 2 installed):
Maximum Memory Module Size: 16384 MB
Maximum Total Memory Size: 65536 MB
Installed Size: 2048 MB (Single-bank Connection)
Enabled Size: 2048 MB (Single-bank Connection)
Installed Size: Not Installed
Enabled Size: Not Installed
Installed Size: 2048 MB (Single-bank Connection)
Enabled Size: 2048 MB (Single-bank Connection)
Installed Size: Not Installed
Enabled Size: Not Installed
Size: 2048 MB
Size: No Module Installed
Size: 2048 MB
Size: No Module Installed
Add the reported sizes (or Enabled Sizes, but some BIOSes empirically don't report that) to get (in this case) 4096 MB. (Extra points for code that automates the parsing and arithmetic, but you can probably do that in your head nearly as reliably.)
To check your computation, run
fgrep -e 'MemTotal:' /proc/meminfo
The value reported by /proc/meminfo should not be more than the value you compute from dmidecode. In this case, empirically I get
MemTotal: 3988616 kB
cat /proc/meminfo
specifically from memory, I got this result from what Jared said
sudo dmidecode -t memory
there you can read the specs for each individual memory slot, so you will read something like 2048MB, in my case I have 2 of these being 4gb, despite my non PAE kernel only shows about 3.3gb and all other applications wont say the real physical memory, only dmidecode, thx!