I have a piece of metadata that I would like to hide in a plain text file, won't be shown by generic text editor, is there a way to do that?
Not within the file itself. For storing metadata I suggest using alternative data streams in the filesystem. In OS X (with HFS) they're called Forks and in Windows (with NTFS) they're called Alternate Data Streams. You're probably already familiar with the "Resource fork" on OS X.
...basically it's a hidden file with the same name as the original file (and moves with it), except it's accessed with special (normally illegal) characters in the file path.
Note that ADS files are not moved/copied between incompatible filesystems (e.g. from NTFS to FAT), when compressed into a zip or archived into a tarball, nor when uploaded, so never assume that your ADS file will be persistent (making it useful for storing local-machine-only metadata, such as editor preferences, etc)
On Windows, you can learn more here: http://support.microsoft.com/kb/105763
On Mac OS X, you can learn more here: https://developer.apple.com/library/mac/documentation/Carbon/Reference/File_Manager/DeprecationAppendix/AppendixADeprecatedAPI.html#//apple_ref/c/func/FSAllocateFork
On Linux, the main filesystems (extfs: ext3, ext4, etc) do not support forks in the filesystem. When reading a HFS volume on Linux you access forks by appending the fork name to the filename (e.g. "/foo/bar/rsrc" to access the resource fork of the file "/foo/bar").
Related
I am working on a web app project to block all the file executable from file upload.
Example: user can upload, txt, png, image and video files and not any executable scripts like, Perl, Python, exe, PHP, .so, .sh files.
If it is a PHP file, then I strstr for "<?php" tag, If this tag is present, then it is PHP file. How can we find the same for other script/executable files?
Edit: Some time hackers will upload the malicious files using .png or .jpg extn, so what is the pattern to check inside the files?
Rather than making your own checks you make use of an existing library and you block everything that does not register as a desired format.
Most such libraries guess the content type and encoding of a file by looking for certain signatures or magic byte sequences at specific positions within the file.
Other libraries may be more specialised and will for example only identify image or video formats.
https://www.php.net/manual/en/intro.fileinfo.php
https://github.com/ahupp/python-magic
https://docs.python.org/3/library/imghdr.html
The file programme is a command line tool for identification of file types.
After the first pass where you identify and accept only the desired file formats you should then make all files that are not rejected go through an antivirus scanner.
Depending on you use cases you may decide to strip the original file name extension and/or even the complete file name that was provided during the upload and assign the mime-type that was detected rather than rely on user provided properties.
my friends
I have a question about why file extensions are created?
I found a quote on Wikipedia
"They are commonly used to imply information about the way data might be stored in the file"
what does it mean?
File extension is an identifier which tell the operating system what kind of data and file type they are working with and what associated program opens the file.
if u have an .apk extension file, system can easily recognize it as an application file. If it is an mp4, means it's some kind of multimedia file and can be operated with multimedia applications.
They are commonly used to imply information about the way data might be stored in the file. A normal text editor uses .txt as extension when an html uses extension .html These two files stores data differently.
Basically I am trying to get list of programs in Linux which are installed and can open particular file extension .jpg for example. If not all, At-least default program should get listed.
Linux (the kernel) has no knowledge on file types to application mapping. If you want to use Gnome programs you can look at https://people.gnome.org/~shaunm/admin-guide/mimetypes-7.html. For KDE there is another mechanism. Each toolkit can define it as it likes. And the programmer can use the defaults or not. So it is simply application specific!
What do you want to achieve?
If you (double) click with a explorer/browser application on an icon or file name, exactly the explorer/browser looks for the file type. Typically this is realized via mime type dictionary. But how a program looks for the file type and maybe execute another program is only related to the programmer who writes that program. The GUI tool-chains like Gnome and KDE have a lot of support for that topic and so you have basic conformity for each family of applications.
If you want to know how a application do the job, start it with strace. But it is quite hard to dig into the huge amount of data.
Also you can take a look for xdg-open. Many programs use this helper to start applications. As an example: If you start Dolphin with strace you will find a line like lstat64("/etc/xdg", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 after clicking on a file.
you can run from command line with:
xdg-open <file-name>
You maybe also want to have a look for applications which registers for file types: /usr/share/applications/*.desktop
Here you can find in each desktop file some mime-types which are registered for the applications. E.g. for audiacity is:
MimeType=application/x-audacity-project;audio/flac;audio/x-flac;audio/basic;audio/x-aiff;audio/x-wav;application/ogg;audio/x-vorbis+ogg;
For your example with jpg:
$ xdg-mime query filetype <any-jpg-file>
image/jpeg
$ grep 'image/jpeg' -R /usr/share/applications/*
...
/usr/share/applications/mimeinfo.cache:image/jpeg2000=kde4-kolourpaint.desktop;gimp.desktop;
So you can see that gimp is one of the default applications for jpg
The place to start looking is at the mailcap (/etc/mailcap) and MIME-types, e.g., in /etc/mime.types in Debian (the filename and path will vary according to who provides it).
The mailcap file gives some rules for opening a file, while MIME-types lists the known filetypes with a tag that allows multiple applications to know about the file types.
Except for embedded or reduced-functionality systems (such as those based on busybox), you would find these files on almost every UNIX-like system.
I need a program to list all the file that are accessed/opened by a process in Linux.
It should work like this,
o/p: The full path of the files that the process is accessing.
Don't want to use 'lsof' utility or any other utility.
Is there anyway to achieve this programmatically?
If you want just the files which are accessible thru opened file descriptors by process of pid 1234, list the /proc/1234/fd/ directory (most of the entries are symlinks). You'll also get additional details thru /proc/1234/fdinfo/
Try
ls -l /proc/self/fd/
to get an idea of what these files contain.
Programatically you could use readdir(3) after opendir(3) on these directories (and also readlink(2), at least for entries in /proc/1234/fd/ ....). See also proc(5)
Notice that /proc/ is Linux specific. Some other Unixes have it (e.g. Solaris), with very different contents, properties, semantics.
If you care also about files which have been opened and closed in the past by some process, it is much more difficult. See also inotify(7) and ptrace(2)...
To convert a file path to a "canonical" absolute fiile path, use realpath(3).
I want to distribute a cross-platform application for which the executable file is slightly different, depending on the user who downloaded it. This is done by having a placeholder string somewhere in the executable that is replaced with something user-specific upon download.
The webserver that has to do these string replacements is a Linux machine. For Windows, the executable is not compressed in the installer .exe, so the string replacement is easy.
For uncompressed Mac OS X .dmg files, this is also easy. However, .dmg files that are compressed with either gzip or bzip2 are not so easy. For example, in the latter case, the compressed .dmg is not one big bzip2-compressed disk image, but instead consists of a few different bzip2-compressed parts (with different block sizes) and a plist suffix. Also, decompressing and recompressing the different parts with bzip2 does not result in the original data, so I'm guessing Apple uses some different parameters to bzip2 than the command-line tool.
Is there a way to generate a compressed .dmg from an uncompressed one on Linux (which does not have hdiutil)? Or maybe another suggestion for creating customized applications without pregenerating them? It should work without any input by the user.
I realize that I'm a bit too late here, but we wanted to do exactly the same thing and got it to work using libdmg. https://github.com/planetbeing/libdmg-hfsplus
Basically, you can use libdmg to unpack a dmg file to an uncompressed file containing a hfs+ file system, play around with the files inside the hfs+ file system, and them put it back together again as a dmg file with the correct checksums.
If you use any fancy dmg features, like showing an EULA before the image is mounted, then these will not survive the process. Background images and so on work, though.
If your web server and client support the gzip encoding, then you can deal with uncompressed files on the server, but have them compressed / decompressed on the fly by the web server / web client respectively.
e.g. apache's mod_gzip.
Otherwise maybe you can split your dmg into 3 parts:
the stuff before what you want to replace
the string you want to replace
the stuff after what you want to replace
If the gzip stream is splittable at those points, you could just concatenate the front and back onto the gzipped string you want to replace. That would let you generate it on the fly.
Release a normal, read-only, compressed dmg. Then bundle your app in a package installer with a pre-flight script that sets the variables you need.