I would like to convert a file to .dat below is my query
I have a File eg: ABC this file doesn't have an extension(when i click its propertise it says type of file: file ) I want to convert this file to a .dat by writing a unix script
Linux (and Unix) do not use the file extension to define the type of a file, though some programs to use the file extension as a guideline. Unix/Linux examines the file magic number (the first bytes) of the file to determine the file type, though the program 'file' is the best explanation of how this is done (three tests, filesystem tests, magic tests, and language tests, the first that succeeds determines the file type).
Windows makes heavy use of the file extension to determine file type, and keeps metadata which maps file extension to application(s) which understand the file.
Linux/Unix uses the file magic number, examination of the first line of the file, and hints at the file type (for human use and some program use) using the file extension.
MacOS tracks file metadata using extension, file type code and creator code (metadata kept apart from filename), although OSX is derived from a Unix-like OS, so many of the Linux/Unix notes are applicable.
Related
I am working on a web app project to block all the file executable from file upload.
Example: user can upload, txt, png, image and video files and not any executable scripts like, Perl, Python, exe, PHP, .so, .sh files.
If it is a PHP file, then I strstr for "<?php" tag, If this tag is present, then it is PHP file. How can we find the same for other script/executable files?
Edit: Some time hackers will upload the malicious files using .png or .jpg extn, so what is the pattern to check inside the files?
Rather than making your own checks you make use of an existing library and you block everything that does not register as a desired format.
Most such libraries guess the content type and encoding of a file by looking for certain signatures or magic byte sequences at specific positions within the file.
Other libraries may be more specialised and will for example only identify image or video formats.
https://www.php.net/manual/en/intro.fileinfo.php
https://github.com/ahupp/python-magic
https://docs.python.org/3/library/imghdr.html
The file programme is a command line tool for identification of file types.
After the first pass where you identify and accept only the desired file formats you should then make all files that are not rejected go through an antivirus scanner.
Depending on you use cases you may decide to strip the original file name extension and/or even the complete file name that was provided during the upload and assign the mime-type that was detected rather than rely on user provided properties.
my friends
I have a question about why file extensions are created?
I found a quote on Wikipedia
"They are commonly used to imply information about the way data might be stored in the file"
what does it mean?
File extension is an identifier which tell the operating system what kind of data and file type they are working with and what associated program opens the file.
if u have an .apk extension file, system can easily recognize it as an application file. If it is an mp4, means it's some kind of multimedia file and can be operated with multimedia applications.
They are commonly used to imply information about the way data might be stored in the file. A normal text editor uses .txt as extension when an html uses extension .html These two files stores data differently.
Basically I am trying to get list of programs in Linux which are installed and can open particular file extension .jpg for example. If not all, At-least default program should get listed.
Linux (the kernel) has no knowledge on file types to application mapping. If you want to use Gnome programs you can look at https://people.gnome.org/~shaunm/admin-guide/mimetypes-7.html. For KDE there is another mechanism. Each toolkit can define it as it likes. And the programmer can use the defaults or not. So it is simply application specific!
What do you want to achieve?
If you (double) click with a explorer/browser application on an icon or file name, exactly the explorer/browser looks for the file type. Typically this is realized via mime type dictionary. But how a program looks for the file type and maybe execute another program is only related to the programmer who writes that program. The GUI tool-chains like Gnome and KDE have a lot of support for that topic and so you have basic conformity for each family of applications.
If you want to know how a application do the job, start it with strace. But it is quite hard to dig into the huge amount of data.
Also you can take a look for xdg-open. Many programs use this helper to start applications. As an example: If you start Dolphin with strace you will find a line like lstat64("/etc/xdg", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 after clicking on a file.
you can run from command line with:
xdg-open <file-name>
You maybe also want to have a look for applications which registers for file types: /usr/share/applications/*.desktop
Here you can find in each desktop file some mime-types which are registered for the applications. E.g. for audiacity is:
MimeType=application/x-audacity-project;audio/flac;audio/x-flac;audio/basic;audio/x-aiff;audio/x-wav;application/ogg;audio/x-vorbis+ogg;
For your example with jpg:
$ xdg-mime query filetype <any-jpg-file>
image/jpeg
$ grep 'image/jpeg' -R /usr/share/applications/*
...
/usr/share/applications/mimeinfo.cache:image/jpeg2000=kde4-kolourpaint.desktop;gimp.desktop;
So you can see that gimp is one of the default applications for jpg
The place to start looking is at the mailcap (/etc/mailcap) and MIME-types, e.g., in /etc/mime.types in Debian (the filename and path will vary according to who provides it).
The mailcap file gives some rules for opening a file, while MIME-types lists the known filetypes with a tag that allows multiple applications to know about the file types.
Except for embedded or reduced-functionality systems (such as those based on busybox), you would find these files on almost every UNIX-like system.
Why is bash(.sh) script not executable by default.
I agree that while we touch any file in linux it is created for reading purpose.
But since file name extensions such as sh and csh are for execution purpose.
Won't it be ideal to touch them in an executable mode.
Question might sound redundant to, but i still thought of asking it :)
Ultimately the answer to this question is that that's not what touch was designed to do.
In fact, touch was not even designed to create files; its primary purpose in life is to change file timestamps. It is merely a side effect of that purpose, and the fact that the designers decided to be a little more generous and create the target file if it doesn't already exist (and if you didn't provide the -c option), that it allows you to create files at all.
It should also be mentioned that there are other techniques that create files, such as redirection (echo 'echo '\''I am a script.'\'';' >|script.sh;). The act of file creation is a generic one, and the whole concept of a file is a generic one. A file is just a byte stream; what goes in the byte stream is unspecified at the file abstraction layer. As #AdamGent mentioned, Windows requires certain types of executable files to have certain extensions in order to be executed properly, but even in Windows, you can put executable code in non-executable-extensioned files, and you can put non-executable content in executable-extensioned files. There's no enforcement of file name / file content correspondence at the file layer.
All of that being said, it would often be a convenience if you could easily create a script in Unix that automatically has executable permission set. I've actually written a script to allow me to edit a new file in vim, and then have its permissions set to executable after write-quitting. The reason this potential convenience has not been standardized into a utility is likely related to the concern about security; you don't want people to accidentally make files executable, because that raises the risk of security holes.
You can always write your own program to create a file and make it executable, perhaps based on the extension of the file name.
Another thing that can be added here is that even shell scripts don't always need to be executable. For example, if you write a script that is only intended to be sourced from existing shell processes (via the source or classic . builtins), then the script does not need to be executable at all. Thus, there are cases where the file extension itself does not provide enough information to determine what the appropriate permissions are for the file.
There is nothing in the file name that says a file is even a script. Common practice perhaps says that .sh and .csh are scripts but there is no such rule.
What makes a file an executable script is the magic number in the beginning of the file. The magic number #! (the shebang, which has many other names as well) means the file is a script. For example
#!/bin/bash
When you try to execute the file (it must then also be set to executable in the permissions), the Linux kernel looks at the first two bytes of the file and finds the magic number (#!). It then knows it is supposed to execute whatever comes after the Shebang with the name of the file as argument followed by the rest of the arguments passed.
If you type the following in the shell
blablabla.sh lol1 lol2
The shell recognizes that you are trying to execute something so it invokes the kernel
exec blablabla.sh lol1 lol2
The kernel finds the shebang, and it becomes
exec /bin/bash blablabla.sh lol1 lol2
By exec i mean one of the exec family system calls.
Other fun names for #! include sha-bang, hashbang, pound-bang, hash-exclam and hash-pling
Because the .sh script extension is completely arbitrary in Unix environments. Its only a convention. You can name your script whatever you like so long as it has an executable bit set. This is unlike Windows where I believe its required (.com, .exe, and I think .dll).
touch just changes the timestamp of the file. It again does not care what the file extension of the file is. In fact most tools in Unix do not care about file extension.
Doing Google’s tutorial for Protocol Buffers: https://developers.google.com/protocol-buffers/docs/javatutorial
Trying to compile it into a java file using:
protoc -I=/home/Eggy/Desktop/Test/ --java_out=/home/Eggy/Desktop/Test/addressbook.proto
Error is: Missing input file
For some reason the file in the folder is unrecognised. If I press tab the file doesn't show in cmd.
Doing it this way: Missing input file with protoc in protocol buffer and this way https://groups.google.com/forum/#!topic/protobuf/y2mxdn1vKGY
Brings up a different error:
File does not reside within any path specified using --proto_path (or -I). You must specify a --proto_path which encompasses this file. Note that the proto_path must be an exact prefix of the .proto file names -- protoc is too dumb to figure out when two paths (e.g. absolute and relative) are equivalent (it's harder than you think).
I also created the file as a text file and saved it as a .proto if that helps in Sublime Text.
Your command is slightly off. The --java_out parameter needs to specify the destination folder, not the input file, and then you need a space before you list the input file.