golang read directly from blocks - linux

I have a file of roughly 300MB. and I run the following command on Linux:
sudo debugfs -R "stat /home/user1/Documents/test.csv" /dev/sda2
I get the following output:
Inode: 16913580 Type: regular Mode: 0644 Flags: 0x80000
Generation: 3920968942 Version: 0x00000000:00000001
User: 1000 Group: 1000 Project: 0 Size: 301526706
File ACL: 0
Links: 1 Blockcount: 588928
Fragment: Address: 0 Number: 0 Size: 0
ctime: 0x5df9e6c2:7636fdcc -- Wed Dec 18 14:13:46 2019
atime: 0x5e67b696:d6bfb018 -- Tue Mar 10 21:17:34 2020
mtime: 0x5df11990:00000000 -- Wed Dec 11 22:00:08 2019
crtime: 0x5df9e6c0:63257940 -- Wed Dec 18 14:13:44 2019
Size of extra inode fields: 32
Inode checksum: 0x35f4d147
EXTENTS:
(ETB0):67695655, (0-8191):162652160-162660351, (8192-40959):165953536-165986303, (40960-57343):165986304-166002687, (57344-61439):166129664-166133759, (61440-65535):166135808-166139903, (65536-67583):166164480-166166527, (67584-71679):166168576-166172671, (71680-73614):166207623-166209557
I think the EXTENTS: shows the physical blocks of the file.
My question is, is it possible to directly read from these block numbers in golang as byte array?

My question is, is it possible to directly read from these block numbers in golang as byte array?
No.

Related

how to recover files delted using "find /export/reports -mtime +112 -type f -delete"

I wanted to delete .pdf files, i used "find /export/reports -mtime +112 -type f -delete", however i did not applied any filter for .pdf file and end up deleting required files as well.
Can any one please help me with how to recover them?
you can try this:
Use debugfs to view a filesystems log
$ debugfs -w /dev/mapper/wks01-root
At the debugfs prompt
debugfs: lsdel
Sample output
Inode Owner Mode Size Blocks Time deleted
23601299 0 120777 3 1/ 1 Tue Mar 13 16:17:30 2012
7536655 0 120777 3 1/ 1 Tue May 1 06:21:22 2012
2 deleted inodes found.
Run the command in debugfs
debugfs: logdump -i <7536655>
5) Determine files inode
> ... ... .... output truncated
> Fast_link_dest: bin
> Blocks: (0+1): 7235938 FS block 7536642 logged at sequence 38402086, journal block 26711
> (inode block for inode 7536655):
> Inode: 7536655 Type: symlink Mode: 0777 Flags: 0x0 Generation: 3532221116
> User: 0 Group: 0 Size: 3
> File ACL: 0 Directory ACL: 0
> Links: 0 Blockcount: 0
> Fragment: Address: 0 Number: 0 Size: 0
> ctime: 0x4f9fc732 -- Tue May 1 06:21:22 2012
> atime: 0x4f9fc730 -- Tue May 1 06:21:20 2012
> mtime: 0x4f9fc72f -- Tue May 1 06:21:19 2012
> dtime: 0x4f9fc732 -- Tue May 1 06:21:22 2012
> Fast_link_dest: bin
> Blocks: (0+1): 7235938 No magic number at block 28053: end of journal.
With the above inode info run the following commands
dd if=/dev/mapper/wks01-root of=recovered.file.001 bs=4096 count=1
skip=7235938 file recovered.file.001 file: ASCII text, with very long
lines
Files been recovered to recovered.file.001.
there are also some tools which can be helpful:
Recovery Tools - Command Line :
testdisk,
photorec,
extundelete
Recovery Tools - Gui :
R-Linux,
R-Studio,
UFS Explorer,
Recovery Explorer

changing file creation date by subtracting 4:00 hours

I have a file in c:/users/text.txt
This text.txt have a file creation date, how to get that text.txt creation date and subtract -4:00 using shell script
last_update=$(stat -c "%n %y" $file)
this statement is giving me the file creation date. How can i subtract -4:00 from it?
for suppose lets say text.txt file was created at 04/04/2019 4:00, i want to change that to 04/04/2019 12:00.
%y is not creation but last modification date.
Get the date as seconds since Epoch and subtract 4 hours from it, convert it to human readable form using date, change access and modification times of a file using touch:
$ stat file
File: file
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: b31ch/45852d Inode: 65386 Links: 1
Access: (0600/-rw-------) Uid: (10138/ u0_a138) Gid: (10138/ u0_a138)
Access: 2019-04-04 12:34:56.172954982 +0300
Modify: 2019-04-04 12:34:56.172954982 +0300
Change: 2019-04-04 12:34:56.172954982 +0300
Birth: -
$
$ stat -c '%y' file
2019-04-04 12:34:56.172954982 +0300
$ stat -c '%Y' file
1554370496
$ date -d #$(( $(stat -c '%Y' file) - 4*60*60 ))
Thu Apr 4 08:34:56 +03 2019
$ touch -d "$( date -d #$(( $(stat -c '%Y' file) - 4*60*60 )) )" file
$
$ stat file
File: file
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: b31ch/45852d Inode: 65386 Links: 1
Access: (0600/-rw-------) Uid: (10138/ u0_a138) Gid: (10138/ u0_a138)
Access: 2019-04-04 08:34:56.000000000 +0300
Modify: 2019-04-04 08:34:56.000000000 +0300
Change: 2019-04-04 12:37:14.492954929 +0300
Birth: -
See stat(1), date(1), and touch(1) for further information.

Linux show verbose/technical file information

Is there a GNU/Linux command that shows verbose information about a file?
Something that outputs all the (raw) technical information the filesystem has about the file, e.g. blocks used, exact create and modified timestamps, etc.
stat. Example:
echo foo > /tmp/bar ; stat /tmp/bar
Output:
File: '/tmp/bar'
Size: 4 Blocks: 8 IO Block: 4096 regular file
Device: 80bh/2059d Inode: 87 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1000/ woo) Gid: ( 1000/ woo)
Access: 2017-05-21 23:34:23.770302257 -0400
Modify: 2017-05-21 23:34:23.770302257 -0400
Change: 2017-05-21 23:34:23.770302257 -0400
Birth: -
I don't think stat gives all blocks used. hdparm can do that, almost... it shows sectors, not blocks; also the sector addresses are relative to the hard drive, not the file system:
hdparm --fibmap /tmp/bar
Output:
/tmp/bar:
filesystem blocksize 4096, begins at LBA 253288448; assuming 512 byte sectors.
byte_offset begin_LBA end_LBA sectors
0 253559088 253559095 8
For file system blocks there's filefrag:
filefrag -v /tmp/bar
Output:
Filesystem type is: ef53
File size of /tmp/bar is 4 (1 block of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 0: 33830.. 33830: 1: last,eof
/tmp/bar: 1 extent found

How to check if a locale is UTF-8?

I'm working with Yocto to create an embedded linux distribution for an ARM device (i.MX 6Quad Processors).
I've configured the list of desired locales with the variable:
IMAGE_LINGUAS = "de-de fr-fr en-gb en-gb.iso-8859-1 en-us en-us.iso-8859-1 zh-cn"
As result I've obtained a file systems that contains the following folders:
root#lam_icu:/usr/lib/locale# cd /usr/share/locale/
root#lam_icu:/usr/share/locale# ls -la
total 0
drwxr-xr-x 6 root root 416 Nov 17 2016 .
drwxr-xr-x 30 root root 2056 Nov 17 2016 ..
drwxr-xr-x 4 root root 296 Nov 17 2016 de
drwxr-xr-x 3 root root 232 Nov 17 2016 en_GB
drwxr-xr-x 4 root root 296 Nov 17 2016 fr
drwxr-xr-x 4 root root 296 Nov 17 2016 zh_CN
and:
root#lam_icu:/usr/share/locale# cd /usr/lib/locale/
root#lam_icu:/usr/lib/locale# ls -la
total 0
drwxr-xr-x 9 root root 640 Mar 13 2017 .
drwxr-xr-x 32 root root 40000 Mar 13 2017 ..
drwxr-xr-x 3 root root 1016 Mar 13 2017 de_DE
drwxr-xr-x 3 root root 1016 Mar 13 2017 en_GB
drwxr-xr-x 3 root root 1016 Mar 13 2017 en_GB.ISO-8859-1
drwxr-xr-x 3 root root 1016 Mar 13 2017 en_US
drwxr-xr-x 3 root root 1016 Mar 13 2017 en_US.ISO-8859-1
drwxr-xr-x 3 root root 1016 Mar 13 2017 fr_FR
drwxr-xr-x 3 root root 1016 Mar 13 2017 zh_CN
Which is the encoding of all non ISO-8859-1 locales? Can I assume that "en_GB" or "en_US" use the UTF-8 encoding?
I've tried to open the "LC_IDENTIFICATION" file, the result is:
Hc�������������cEnglish locale for the USAFree Software
Foundation,
Inc.http://www.gnu.org/software/libc/bug-glibc-locales#gnu.orgEnglishUSA1.02000-06-24en_US:2000en_US:2000en_US:2000en_US:2000en_US:2000en_US:2000en_US:2000en_US:2000en_US:2000en_US:2000en_US:2000en_US:2000UTF-8
At the end of the file there is something that recalls "UTF-8". Is this enough to assume that the encoding is UTF-8?
How to check if a locale is UTF-8?
LC_IDENTIFICATION doesn't tell you much:
LC_IDENTIFICATION - this is not a user-visible category, it contains information about the locale itself and is rarely useful for users or developers (but is listed here for completeness sake).
You'd have to look at the complete set of files.
There appears to be no standard command-line utility for doing this, but there is a runtime call (added a little later than the original locale functions). Here is a sample program which illustrates the function nl_langinfo:
#include <stdio.h>
#include <locale.h>
#include <langinfo.h>
int
main(int argc, char **argv)
{
int n;
for (n = 1; n < argc; ++n) {
if (setlocale(LC_ALL, argv[n]) != 0) {
char *code = nl_langinfo(CODESET);
if (code != 0)
printf("%s ->%s\n", argv[n], code);
else
printf("?%s (nl_langinfo)\n", argv[n]);
} else {
printf("? %s (setlocale)\n", argv[n]);
}
}
return 0;
}
and some output, e.g., by foo $(locale -a):
aa_DJ ->ISO-8859-1
aa_DJ.iso88591 ->ISO-8859-1
aa_DJ.utf8 ->UTF-8
aa_ER ->UTF-8
aa_ER#saaho ->UTF-8
aa_ER.utf8 ->UTF-8
aa_ER.utf8#saaho ->UTF-8
aa_ET ->UTF-8
aa_ET.utf8 ->UTF-8
af_ZA ->ISO-8859-1
af_ZA.iso88591 ->ISO-8859-1
af_ZA.utf8 ->UTF-8
am_ET ->UTF-8
am_ET.utf8 ->UTF-8
an_ES ->ISO-8859-15
an_ES.iso885915 ->ISO-8859-15
an_ES.utf8 ->UTF-8
ar_AE ->ISO-8859-6
ar_AE.iso88596 ->ISO-8859-6
ar_AE.utf8 ->UTF-8
ar_BH ->ISO-8859-6
ar_BH.iso88596 ->ISO-8859-6
The directory names you're referring to are often (but not required) to be the same as encoding names. That is the assumption made in the example program. There was a related question in How to get terminal's Character Encoding, but it has no useful answers. One is interesting though, since it asserts that
locale charmap
will give the locale encoding. According to the standard, that's not necessarily so:
The command locale charmap gives the name used in localedef -f
However, localedef attaches no special meaning to the name given in the -f option.
localedef has a different option -u which identifies the codeset, but locale (in the standard) mentions no method for displaying this information.
As usual, implementations may (or may not) treat unspecified features in different ways. The GNU C library's documentation differs in some respects from the standard (see locale and localedef), but offers no explicit options for showing the codeset name.

How to get file creation date/time in Bash/Debian?

I'm using Bash on Debian GNU/Linux 6.0. Is it possible to get the file creation date/time? Not the modification date/time.
ls -lh a.txt and stat -c %y a.txt both only give the modification time.
Unfortunately your quest won't be possible in general, as there are only 3 distinct time values stored for each of your files as defined by the POSIX standard (see Base Definitions section 4.8 File Times Update)
Each file has three distinct associated timestamps: the time of last
data access, the time of last data modification, and the time the file
status last changed. These values are returned in the file
characteristics structure struct stat, as described in <sys/stat.h>.
EDIT: As mentioned in the comments below, depending on the filesystem used metadata may contain file creation date. Note however storage of information like that is non standard. Depending on it may lead to portability problems moving to another filesystem, in case the one actually used somehow stores it anyways.
ls -i file #output is for me 68551981
debugfs -R 'stat <68551981>' /dev/sda3 # /dev/sda3 is the disk on which the file exists
#results - crtime value
[root#loft9156 ~]# debugfs -R 'stat <68551981>' /dev/sda3
debugfs 1.41.12 (17-May-2010)
Inode: 68551981 Type: regular Mode: 0644 Flags: 0x80000
Generation: 769802755 Version: 0x00000000:00000001
User: 0 Group: 0 Size: 38973440
File ACL: 0 Directory ACL: 0
Links: 1 Blockcount: 76128
Fragment: Address: 0 Number: 0 Size: 0
ctime: 0x526931d7:1697cce0 -- Thu Oct 24 16:42:31 2013
atime: 0x52691f4d:7694eda4 -- Thu Oct 24 15:23:25 2013
mtime: 0x526931d7:1697cce0 -- Thu Oct 24 16:42:31 2013
**crtime: 0x52691f4d:7694eda4 -- Thu Oct 24 15:23:25 2013**
Size of extra inode fields: 28
EXTENTS:
(0-511): 352633728-352634239, (512-1023): 352634368-352634879, (1024-2047): 288392192-288393215, (2048-4095): 355803136-355805183, (4096-6143): 357941248-357943295, (6144
-9514): 357961728-357965098
mikyra's answer is good. The fact just like what he said.
[jason#rh5 test]$ stat test.txt
File: `test.txt'
Size: 0 Blocks: 8 IO Block: 4096 regular empty file
Device: 802h/2050d Inode: 588720 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 500/ jason) Gid: ( 500/ jason)
Access: 2013-03-14 01:58:12.000000000 -0700
Modify: 2013-03-14 01:58:12.000000000 -0700
Change: 2013-03-14 01:58:12.000000000 -0700
if you want to verify, which file was created first, you can structure your file name by appending system date when you create a series of files.
Note that if you've got your filesystem mounted with noatime for performance reasons, then the atime will likely show the creation time. Given that noatime results in a massive performance boost (by removing a disk write for every time a file is read), it may be a sensible configuration option that also gives you the results you want.
Creation date/time is normally not stored. So no, you can't.
You can find creation time - aka birth time - using stat and also match using find.
We have these files showing last modified time:
$ ls -l --time-style=long-iso | sort -k6
total 692
-rwxrwx---+ 1 XXXX XXXX 249159 2013-05-31 14:47 Getting Started.pdf
-rwxrwx---+ 1 XXXX XXXX 275799 2013-12-30 21:12 TheScienceofGettingRich.pdf
-rwxrwx---+ 1 XXXX XXXX 25600 2015-05-07 18:52 Thumbs.db
-rwxrwx---+ 1 XXXX XXXX 148051 2015-05-07 18:55 AsAManThinketh.pdf
To find files created within a certain time frame using find as below.
Clearly, the filesystem knows about the birth time of a file:
$ find -newerBt '2014-06-13' ! -newerBt '2014-06-13 12:16:10' -ls
20547673299906851 148 -rwxrwx--- 1 XXXX XXXX 148051 May 7 18:55 ./AsAManThinketh.pdf
1407374883582246 244 -rwxrwx--- 1 XXXX XXXX 249159 May 31 2013 ./Getting\ Started.pdf
We can confirm this using stat:
$ stat -c "%w %n" * | sort
2014-06-13 12:16:03.873778400 +0100 AsAManThinketh.pdf
2014-06-13 12:16:04.006872500 +0100 Getting Started.pdf
2014-06-13 12:16:29.607075500 +0100 TheScienceofGettingRich.pdf
2015-05-07 18:32:26.938446200 +0100 Thumbs.db
stat man pages explains %w:
%w time of file birth, human-readable; - if unknown
ls -i menus.xml
94490 menus.xml
Here the number 94490 represents inode
Then do a:
df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg-root 4.0G 3.4G 408M 90% /
tmpfs 1.9G 0 1.9G 0% /dev/shm
/dev/sda1 124M 27M 92M 23% /boot
/dev/mapper/vg-var 7.9G 1.1G 6.5G 15% /var
To find the mounting point of the root "/" filesystem, because the file menus.xml is on '/' that is '/dev/mapper/vg-root'
debugfs -R 'stat <94490>' /dev/mapper/vg-root
The output may be like the one below:
debugfs -R 'stat <94490>' /dev/mapper/vg-root
debugfs 1.41.12 (17-May-2010)
Inode: 94490 Type: regular Mode: 0644 Flags: 0x0
Generation: 2826123170 Version: 0x00000000
User: 0 Group: 0 Size: 4441
File ACL: 0 Directory ACL: 0
Links: 1 Blockcount: 16
Fragment: Address: 0 Number: 0 Size: 0
ctime: 0x5266e438 -- Wed Oct 23 09:46:48 2013
atime: 0x5266e47b -- Wed Oct 23 09:47:55 2013
mtime: 0x5266e438 -- Wed Oct 23 09:46:48 2013
Size of extra inode fields: 4
Extended attributes stored in inode body:
selinux = "unconfined_u:object_r:usr_t:s0\000" (31)
BLOCKS:
(0-1):375818-375819
TOTAL: 2
Where you can see the creation time:
ctime: 0x5266e438 -- Wed Oct 23 09:46:48 2013
stat -c %w a.txt
%w returns the file creation(birth) date if it is available, which is rare.
Here's the link
As #mikyra explained, creation date time is not stored anywhere.
All the methods above are nice, but if you want to quickly get only last modify date, you can type:
ls -lit /path
with -t option you list all file in /path odered by last modify date.
If you really want to achieve that you can use a file watcher like inotifywait.
You watch a directory and you save information about file creations in separate file outside that directory.
while true; do
change=$(inotifywait -e close_write,moved_to,create .)
change=${change#./ * }
if [ "$change" = ".*" ]; then ./scriptToStoreInfoAboutFile; fi
done
As no creation time is stored, you can build your own system based on inotify.
Cited from https://unix.stackexchange.com/questions/50177/birth-is-empty-on-ext4/131347#131347 , the following shellscript would work to get creation time:
get_crtime() {
for target in "${#}"; do
inode=$(stat -c %i "${target}")
fs=$(df "${target}" | tail -1 | awk '{print $1}')
crtime=$(sudo debugfs -R 'stat <'"${inode}"'>' "${fs}" 2>/dev/null | grep -oP 'crtime.*--\s*\K.*')
printf "%s\t%s\n" "${target}" "${crtime}"
done
}
even better:
lsct ()
{
debugfs -R 'stat <'`ls -i "$1" | (read a b;echo -n $a)`'>' `df "$1" | (read a; read a b; echo "$a")` 2> /dev/null | grep --color=auto crtime | ( read a b c d;
echo $d )
}
lsct /etc
Wed Jul 20 19:25:48 2016
Another trick to add to your arsenal is the following:
$ grep -r "Copyright" /<path-to-source-files>/src
Generally speaking, if one changes a file they should claim credit in the “Copyright”. Examine the results for dates, file names, contributors and contact email.
example grep result:
/<path>/src/someobject.h: * Copyright 2007-2012 <creator's name> <creator's email>(at)<some URL>>

Resources