Do you know which system call the Linux file command uses to determine the type of file?
On the command line, if you use the file command it will reply with something like:
dev-1:~/$ file download.png
download.png: PNG image data, 724x 724, 8-bit/color RGBA, non-interlaced
Is this an entire routine that does that? Or is there a system call that maps to that functionality?
On the suggestion of #stark I ran the Linux strace command like the following:
$ strace file download.png
A lot of info is returned (many system calls) but I picked through it and found the core item I was looking for:
fstat(3, {st_mode=
I searched and found a man page for fstat:
These functions [lstat, fstatat] return information about a file, in the buffer pointed
to by statbuf.
There is also a call (or calls) to pread64() which reads from the file descriptor.
Hi I've some helpful Linux file commands for you hope it will be useful below
file bobs_file.txt
output = bobs_file.txt: ASCII
file textfile.tar
output = textfile.tar: POSIX tar archive (GNU)
file Picture/
output = Pictures/: directory
file iuk.png
output = iuk.png: PNG image data, 213 x 213, 8-bit/color RGB, non-interlaced
Related
According to thread:
Linux: fast creating of formatted output file (csv) from find command
there is a suggested bash command, including awk (which I don't understand):
find /mnt/sda2/ | awk 'BEGIN{FS=OFS="/"}!/.cache/ {$2=$3=""; new=sprintf("%s",$0);gsub(/^\/\/\//,"",new); printf "05;%s;/%s\n",$NF,new }' > $p1"Seagate-4TB-S2-BTRFS-1TB-Dateien-Verzeichnisse.csv"
With this command, I am able to create a csv file containing "05;file name;full path and file name" of the directory and file content of my device mounted on /mnt/sda2. Thanks again to -> tink
How must I adapt the above command to receive date(&time) and file size also?
Thank you in advance,
-Linuxfluesterer
Intent:
Does Go have the functionality (package or otherwise) to perform a special file stat on Linux akin to the command file -s <path>
Example:
[root#localhost ~]# file /proc/uptime
/proc/uptime: empty
[root#localhost ~]# file -s /proc/uptime
/proc/uptime: ASCII text
Use Case:
I have a fileglob of files in /proc/* that I need to very quickly detect if they are truly empty instead of appearing to be empty.
Using The os Package:
Code:
result,_ := os.Stat("/proc/uptime")
fmt.Println("Name:",result.Name()," Size:",result.Size()," Mode:",int(result.Mode()))
fmt.Printf("%q",result)
Result:
Name: uptime Size: 0 Mode: 292
&{"uptime" '\x00' 'Ĥ' {%!q(int64=63606896088) %!q(int32=413685520) %!q(*time.Location=&{ [] [] 0 0 <nil>})} {'\x03' %!q(uint64=4026532071) '\x01' '脤' '\x00' '\x00' '\x00' '\x00' '\x00' 'Ѐ' '\x00' {%!q(int64=1471299288) %!q(int64=413685520)} {%!q(int64=1471299288) %!q(int64=413685520)} {%!q(int64=1471299288) %!q(int64=413685520)} ['\x00' '\x00' '\x00']}}
Obvious Workaround:
There is the obvious workaround of the following. But it's a little over the top to need to call in a bash shell in order to get file stats.
output,_ := exec.Command("bash","-c","file -s","/proc/uptime").Output()
//parse output etc...
EDIT/MY PRACTICAL USE CASE:
Quickly determining which files are zero size without needing to read each one of them first.
file -s /cgroup/memory/lsf/<cluster>/*/tasks | <clean up commands> | uniq -c
6 /cgroup/memory/lsf/<cluster>/<jobid>/tasks: ASCII text
805 /cgroup/memory/lsf/<cluster>/<jobid>/tasks: empty
So in this case, I know that only those 6 jobs are running and the rest (805) have terminated. Reading the file works like this:
# cat /cgroup/memory/lsf/<cluster>/<jobid>/tasks
#
or
# cat /cgroup/memory/lsf/<cluster>/<jobid>/tasks
12352
53455
...
I'm afraid you might be confusing matters here: file is special in precisely a way it "knows" a set of heuristics to carry out its tasks.
To my knowledge, Go does not have anything like this in its standard library, and I've not came across a 3rd-party package implementing a file-like functionality (though I invite you to search by relevant keywords on http://godoc.org)
On the other hand, Go provides full access to the syscall interface of the underlying OS so when it comes to querying the OS in a way file does it, there's nothing you could not do in plain Go.
So I suggest you to just fetch the source code of file, learn what it does in its mode turned on by the "-s" command-line option and implement that in your Go code.
We'll try to have you with specific problems doing that — should you have any.
Update
Looks like I've managed to grasp the OP is struggling with: a simple check:
$ stat -c %s /proc/$$/status && wc -c < $_
0
849
That is, the stat call on a file under /proc shows it has no contents but actually reading from that file returns that contents.
OK, so the solution is simple: instead of doing a call to os.Stat() while traversing the subtree of the filesystem one should instead merely attempt to read a single byte from the file, like in:
var buf [1]byte
f, err := os.Open(fname)
if err != nil {
// do something, or maybe ignore.
// A not existing file is OK to ignore
// (the POSIX error code will be ENOENT)
// because after the `path/filepath.Walk()` fetched an entry for
// this file from its directory, the file might well have gone.
}
_, err = f.Read(buf[:])
if err != nil {
if err == io.EOF {
// OK, we failed to read 1 byte, so the file is empty.
}
// Otherwise, deal with the error
}
f.Close()
You might try to be more clever and first obtain the stat information
(using a call to os.Stat()) to see if the file is a regular file—to
not attempt reading from sockets etc.
I have a fileglob of files in /proc/* that I need to very quickly
detect if they are truly empty instead of appearing to be empty.
They are truly empty in some sense (eg. they occupy no space on file system). If you want to check whether any data can be read from them, try reading from them - that's what file -s does:
-s, --special-files
Normally, file only attempts to read and
determine the type of argument files which stat(2) reports are
ordinary files. This prevents problems, because reading special files
may have peculiar consequences. Specifying the -s option causes file
to also read argument files which are block or character special
files. This is useful for determining the filesystem types of the
data in raw disk partitions, which are block special files. This
option also causes file to disregard the file size as reported by
stat(2) since on some systems it reports a zero size for raw disk
partitions.
I use a program which create me postscript file before using ps2pdf to make it a readable pdf, i've made a program which add some string to overwrite the company new logo. (The first program can't import image file itself).
I add the string before the before-last line of the file (" showpage").
While running my program to add the logo there is no error.
With the option -dNOSAFER everything is fine, but by default it's set to -dSAFER, and an invalidfileaccess error pop, the files are 6 jpg images alone in their directory.
I don't want to make it run with the -dNOSAFER option on. As it will fully open the file system.
In the documentation I've seen that there is a "permitted path" setting, but i can't find nowhere to set this up. Is it just a command line option to set in the command launching the program ? Or is there a config file for GhostScript / ps2pdf where i can put the path to this directory as permitted path.
in this documentation :
http://www.ghostscript.com/doc/current/Use.htm
I only find
-dTTYPAUSE
Causes Ghostscript to read a character from /dev/tty, rather than
standard input, at the end of each page. This may be useful if input
is coming from a pipe. Note that -dTTYPAUSE overrides -dNOPAUSE. Also
note that -dTTYPAUSE requires opening the terminal device directly,
and may cause problems in combination with -dSAFER. Permission errors
can be avoided by adding the device to the permitted reading list
before invoking safer mode
gs -dTTYPAUSE -dDELAYSAFER -c '<< /PermitFileReading [ (/dev/tty)] >> setuserparams .locksafe' -dSAFER
The quote is just for the context but is this a way to put the permitted path ?
As gs automatically launch with the full system as readOnly there will be no difference ? There is no other find result for PermitFile in this page.
Try adding the required path to the search path with -I (Include) See Use.htm, section 8 How Ghostscript finds files. This should only be a problem if you are using 'run' or similar to read files from another location.
The section on TTYPAUSE is not relevant.
I have built a linux image for ARM on Ubuntu. I was curious to use the file command on the image file created in arch/arm/boot directory. When i execute the following the command
balaji#balaji-virtual-machine:~/meraj/linux-stable/arch/arm/boot$ ls
bootp compressed dts Image install.sh Makefile zImage
balaji#balaji-virtual-machine:~/meraj/linux-stable/arch/arm/boot$ file Image
Image: data
balaji#balaji-virtual-machine:~/meraj/linux-stable/arch/arm/boot$ file zImage
zImage: data
balaji#balaji-virtual-machine:~/meraj/linux-stable/arch/arm/boot$
It gives not much information. I would like to know if this is expected behaviour or not?
From file manpage:
The type printed will usually contain one of the words...
... "data" meaning anything else (data is usually 'binary' or non-printable).
Exceptions are well-known file formats (core files, tar archives) that
are known to contain binary data.
Also...
Any file that cannot be identified as having been written in any of
the character sets listed above is simply said to be 'data'.
I'm using ImageMagick to do some image processing from the commandline, and would like to operate on a list of files as specified in foo.txt. From the instructions here: http://www.imagemagick.org/script/command-line-processing.php I see that I can use Filename References from a file prefixed with #. When I run something like:
montage #foo.txt output.jpg
everything works as expected, as long as foo.txt is in the current directory. However, when I try to access bar.txt in a different directory by running:
montage /some_directory/#bar.txt
output2.jpg
I get:
montage: unable to open image
/some_directory/#bar.txt: No such file
or directory # blob.c/OpenBlob/2480.
I believe the issue is my syntax, but I'm not sure what to change it to. Any help would be appreciated.
Quite an old entry but it seems relatively obvious that you need to put the # before the full path:
montage #/some_directory/bar.txt output2.jpg
As of ImageMagick 6.5.4-7 2014-02-10, paths are not supported with # syntax. The # file must be in the current directory and identified by name only.
I haven't tried directing IM to pull the list of files from a file, but I do specify multiple files on the command line like this:
gm -sOutputFile=dest.ext -f file1.ppm file2.ppm file3.ppm
Can you pull the contents of that file into a variable, and then let the shell expand that variable?