I have a folder that I own, but also has read permissions for the entire group. I would like to know who read/copied the contents and when. Is it possible to do this with a script or otherwise?
kernel’s audit system can be used for this
Related
I need a program to list all the file that are accessed/opened by a process in Linux.
It should work like this,
o/p: The full path of the files that the process is accessing.
Don't want to use 'lsof' utility or any other utility.
Is there anyway to achieve this programmatically?
If you want just the files which are accessible thru opened file descriptors by process of pid 1234, list the /proc/1234/fd/ directory (most of the entries are symlinks). You'll also get additional details thru /proc/1234/fdinfo/
Try
ls -l /proc/self/fd/
to get an idea of what these files contain.
Programatically you could use readdir(3) after opendir(3) on these directories (and also readlink(2), at least for entries in /proc/1234/fd/ ....). See also proc(5)
Notice that /proc/ is Linux specific. Some other Unixes have it (e.g. Solaris), with very different contents, properties, semantics.
If you care also about files which have been opened and closed in the past by some process, it is much more difficult. See also inotify(7) and ptrace(2)...
To convert a file path to a "canonical" absolute fiile path, use realpath(3).
Is there a standard way of determining file locations under Linux? Even better, are there any POSIX API's which allow the retrieval of standard file locations?
For example, how can I determine a user's home directory? Or, how can I determine the proper location for system configuration files?
I know that typically these locations would be "/home/username" or "/etc/". Should I just hardcode the paths as such?
The path to the current user's home directory is in the environment variable HOME. (I know systems where home dirs are spread over several partitions (say, /vol/vol[number]/[first letter]/[user name]) and not located in /home/.)
For other users, there's getpwent (and getpwent_r), which pull the home directory from the passwd entry.
For the other directories, there is the File System Hierarchy Standard, which most Linux distros adhere to and some other OSen as well.
I don't think there's an API for this. Thus, if a system does things differently, you're on your own -- good luck! ;-)
The current user's home directory can be found in the HOME environment variable. For other users, you can use the getpwnam or getpwuid functions (or the _r variants) to look up another specified user's home directory, among other things.
I know that you didn't ask this, however if you're looking to find the location of an executable, you can use which
I am trying to write a script or a piece of code to archive files, but I do not want to archive anything that is currently open. I need to find a way to determine what files in a directory are open. I want to use either Perl or a shell script, but can try use other languages if needed. It will be in a Linux environment and I do not have the option to use lsof. I have also had inconsistant results with fuser. Thanks for any help.
I am trying to take log files in a directory and move them to another directory. If the files are open however, I do not want to do anything with them.
You are approaching the problem incorrectly. You wish to keep files from being modified underneath you while you are reading, and cannot do that without operating system support. The best that you can hope for in a multi-user system is to keep your archive metadata consistent.
For example, if you are creating the archive directory, make sure that the number of bytes stored in the archive matches the directory. You can checksum the file contents before and after reading the filesystem and compare that with what you wrote to the archive and perhaps flag it as "inconsistent".
What are you trying to accomplish?
Added in response to comment:
Look at logrotate to steal ideas about how to handle this consistently just have it do the work for you. If you are concerned that rename of files will make processes that are currently writing them will break things, take a look at man 2 rename:
rename() renames a file, moving it
between directories if required. Any
other hard links to the file (as
created using link(2)) are unaffected.
Open file descriptors for oldpath are
also unaffected.
If newpath already exists it will be atomically replaced (subject
to a few conditions; see ERRORS
below), so that there is no point at
which another process attempting to
access newpath will find it missing.
Try ls -l /proc/*/fd/* as root.
msw has answered the question correctly but if you want to file the list of open processes, the lsof command will give it to you.
I am trying to write a bash script to do a task, I have done pretty well so far, and have it working to an extent, but I want to set it up so it's distributable to other people, and will be opening it up as open source, so I want to start doing things the "conventional" way. Unfortunately I'm not all that sure what the conventional way is.
Ideally I want a link to an in depth online resource that discusses this and surrounding topics in depth, but I'm having difficulty finding keywords that will locate this on google.
At the start of my script I set a bunch of global variables that store the names of the dirs that it will be accessing, this means that I can modify the dir's quickly, but this is programming shortcuts, not user shortcuts, I can't tell the users that they have to fiddle with this stuff. Also, I need for individual users' settings not to get wiped out on every upgrade.
Questions:
Name of settings folder: ~/.foo/ -- this is well and good, but how do I keep my working copy and my development copy separate? tweek the reference in the source of the dev version?
If my program needs to maintain and update library of data (gps tracklog data in this case) where should this directory be? the user will need to access some of this data, but it's mostly for internal use. I personally work in cygwin, and I like to keep this data on separate drive, so the path is wierd, I suspect many users could find this. for a default however I'm thinking ~/gpsdata/ -- would this be normal, or should I hard code a system that ask the user at first run where to put it, and stores this in the settings folder? whatever happens I'm going ot have to store the directory reference in a file in the settings folder.
The program needs a data "inbox" that is a folder that the user can dump files, then run the script to process these files. I was thinking ~/gpsdata/in/ ?? though there will always be an option to add a file or folder to the command line to use that as well (it processed files all locations listed, including the "inbox")
Where should the script its self go? it's already smart enough that it can create all of it's ancillary/settings files (once I figure out the "correct" directory) if run with "./foo --setup" I could shove it in /usr/bin/ or /bin or ~/.foo/bin (and add that to the path) what's normal?
I need to store login details for a web service that it will connect to (using curl -u if it matters) plan on including a setting whereby it asks for a username and password every execution, but it currently stores it plane text in a file in ~/.foo/ -- I know, this is not good. The webservice (osm.org) does support oauth, but I have no idea how to get curl to use it -- getting curl to speak to the service in the first place was a hack. Is there a simple way to do a really basic encryption on a file like this to deter idiots armed with notepad?
Sorry for the list of questions, I believe they are closely related enough for a single post. This is all stuff that stabbing at, but would like clarification/confirmation over.
Name of settings folder: ~/.foo/ -- this is well and good, but how do I keep my working copy and my development copy separate?
Have a default of ~/.foo, and an option (for example --config-directory) that you can use to override the default while developing.
If my program needs to maintain and update library of data (gps tracklog data in this case) where should this directory be?
If your script is running under a normal user account, this will have to be somewhere in the user's home directory; elsewhere, you'll have no write permissions. Perhaps ~/.foo/tracklog or something? Again, add a command line option, and also an option in the configuration file, to override this.
I'm not a fan of your ~/gpsdata default; I don't want my home directory cluttered with all sorts of directories that programs created without my consent. You see this happen on Windows a lot, and it's really annoying. (Saved games in My Documents? Get out of here!)
The program needs a data "inbox" that is a folder that the user can dump files, then run the script to process these files. I was thinking ~/gpsdata/in/ ?
As stated above, I'd prefer ~/.foo/inbox. Also with command-line option and configuration file option to change this.
But do you really need an inbox? If the user needs to run the script manually over some files, it might be better just to accept those file names on the command line. They could just be processed wherever, without having to move them to a "magic" location.
Where should the script its self go?
This is usually up to the packaging system of the particular OS you're running on. When installing from source, /usr/local/bin is a sensible default that won't interfere with package managers.
Is there a simple way to do a really basic encryption on a file like this to deter idiots armed with notepad?
Yes, there is. But it's better not to, because it creates a false sense of security. Without a master password or something, secure storage is not possible! Pidgin, for example, explicitly stores passwords in plain text, so that users won't make any false assumptions about their passwords being stored "securely". So it's best just to store them in plain text, complain if the file is world-readable, and add a clear note to the manual to warn the user what's going on.
Bottom line: don't try to reinvent the wheel. There have been thousands of scripts and programs that faced the same issues; most of them ended up adopting the same conventions, and for good reasons. Look at what they do, and mimic them instead of reinventing the wheel.
You can start with the Filesystem Hierarchy Standard. I'm not sure how well followed it is, but it does provide some guidance. In general, I try to use the following:
$HOME/.foo/ is used for user-specific settings - it is hidden
$PREFIX/etc/foo/ is for system-wide configuration
$PREFIX/foo/bin/ is for system-wide binaries
sym-links from $PREFIX/foo/bin are added to $PREFIX/bin/ for ease of use
$PREFIX/foo/var/ is where variable data would live - this is where your input spools and log files would live
$PREFIX should default to /opt/foo even though almost everyone seems to plop stuff in /usr/local by default (thanks GNU!). If someone wants to install the package in their home directory, then substitute $HOME for $PREFIX. At least that is my take on how this should all work.
How could I track changes of specific directory in UNIX? For example, I launch some utility which create some files during its execution. I want to know what exact files were created during one particular launch. Is there any simple way to get such information? Problem is that:
I cannot flush directory content after script execution
Files created with the name that has hash as a compound part. There is no possibility to get this hash from script for subsequent search.
There could be several scripts executed simultaneously, I do not want to see files created by another process in the same folder.
Please notice that I do not want to know whether directory has been changed as stated here, I need filenames which ideally could be grepped to match specific pattern.
You need to subscribe to file system change notifications.
You should use something like FAM, gamin, or inotify to detect when a file has been created, closed, etc.
You could use strace -f myscript to trace all system calls made by the script, and use grep to filter the system calls that create new files.
You could use the Linux Auditing System. Here is a howto link:
http://www.cyberciti.biz/tips/linux-audit-files-to-see-who-made-changes-to-a-file.html
You can use the script command to track the commands launched.