I'm wondering if it matters if I add a crontab to /etc/crontab or crontab -e?
I have an Ubuntu 17 and Debian 9 VM running, and I'm confused which one is the right place.
Thanks in advance!
They are not the same.
The crontab command is specific to a user. When you edit your
crontab (via crontab -e) you are really saving it to
/var/spool/cron/. I find this to be more geared toward interactive
setup/maintenance: it uses your $EDITOR. (Though I have seen tools
like whenever that will
automatically populate a user crontab.
The "system" cron files live in /etc/crontab and /etc/cron.d.
These are similar to your user's crontab, but the format has an
additional (sixth) field to specify which user to run as, and you'll
need root privileges to change these. The latter directory is often
used by tools to place a cron script into, by system installs, or your
own deployment routines.
You'll also find related system directories in /etc/, such as
cron.daily/, cron.hourly/, etc. These hold normal scripts that are
run on their respective cadence. E.g., /etc/cron.daily/logrotate
rotates system log files daily. They typically orchestrated by your
/etc/anacrontab to add some small random delay across systems.
There are a few places to look for documentation of the various pieces
of cron. The relevant man pages are:
crontab(1) -- the command
crontab(5) -- spec formatting
cron(8) -- the daemon
An alternative to cron with SystemD now is
timers.
Related
I'm running linux mint 17 and I notice that every so often my computer slows to a crawl.W When I look at top I see "/usr/bin/find / -ignore_readdir_race (..." etc. sucking up most of my memory. It runs for a really long time (several hours) and my guess is that its an automated indexing process for my hard drive.
I'm working on a project that requires me to have over 6 million audio files on a mounted SSD so another guess is that the filesystem manager is trying to index all these files for quick search. Is that the case? Is there any way to turn it off for the SSD?
The locate command reports data collected for its database by a regular cron task. You can exclude directories from the database, making the task run more quickly. According to updatedb.conf(5)
PRUNEPATHS
A whitespace-separated list of path names of directories which should not be scanned by updatedb(8). Each path name must be exactly in the form in which the directory would be reported by locate(1).
By default, no paths are skipped.
On my Debian machine for instance, /etc/updatedb.conf contains this line:
PRUNEPATHS="/tmp /var/spool /media"
You could modify your /etc/updatedb.conf to add the directories which you want to ignore. Only the top-level directory of a directory tree need be listed; subdirectories are ignored when the parent is ignored.
Further reading:
Tip of the day: Speed up `locate`
How do I get mlocate to only index certain directories?
It's a daily cron job that updates databases used by the locate command. See updatedb(8) if you want to learn more. Having six million audio files will likely cause this process to eat up a lot of CPU as it's trying to index your local filesystems.
If you don't use locate, I'd recommend simply disabling updatedb, something like this:
sudo kill -9 <PID>
sudo chmod -x /etc/cron.daily/mlocate
sudo mv /var/lib/mlocate/mlocate.db /var/lib/mlocate/mlocate.db.bak
If all else fails just remove the package.
I have a question on Zen of Unix / GNU Linux, which looks like an aha-moment for me...
Apparently ANY standard Unix/Linux program has a config file, which the most are located at /etc directory.
Can we derive the concept as follow:
1- As an application developer you should design your software which have a customiztion file (possibly located at /etc.)
2- Then, Admins or users can SET these configs based on their needs and run your program.
3- Changing the behavior of your program should ONLY depends on its config file.
If you're asking whether this is true, that tends to be the convention, yes. Keep in mind that developers are free to design their programs to run however they want to and that they tend to follow this pattern only for convenience of similarity.
Other patterns you may see:
Programs with no global settings and only per user settings may store their settings in ~/.[something], or maybe somewhere else entirely. Many programs do this AND use /etc. Bash is a good example, using /etc/profile/.bashrc for default settings and ~/.bashrc for user settings.
Very large standalone installations of some programs may package all of their files into their own .../etc, .../bin, etc.. directories, and will not use the typical system directories at all. An example of this is charybdis, an ircd, which stores everything in a folder specified at compile time (Mine lives in /var/ircd, so I have /var/ircd/etc, /var/ircd/bin, /var/ircd/lib, ...)
OSX is a certified Unix and tries not to use etc - in effect only Apple supplied programs should change /etc, they supply alternatives.
However for all OSs including Windows you do have a separate configuration/customisation file (or in Windows the registry) and there probably need to be two of these. One that is set and controlled by admins and one for changes the user makes. The former of these can use /etc for Linux see the Filesystem_Hierarchy_Standard
Please tell me the differences in information displayed by two commands jobs and ps in unix operating system?
jobs is a shell builtin. It tells you about the jobs that the current shell is managing. It can give you information that is internal to the shell, like the job numbers (which you can use in shortcuts like fg %2) and the original command line as it appeared before variable expansions.
ps is an external command which can tell you about all the processes running on the system. (By default it only shows a small subset, but there are options to select larger sets of processes to display.) It doesn't know about the shell-internal stuff.
jobs: shows the current jobs living in this terminal
as example ->
gedit &
jobs
this will show u gedit is running atm.
if you close the terminal, gedit dies too, you can use disown so it wont die.
ps is a totally different thing, its a process table display tool.
I am very new to linux and am sorry for the newbie questions.
I had a homework extra credit question that I was trying to do but failed to get it.
Q. Write a security shell script that logs the following information
for every process: User ID, time started, time ended (0 if process is
still running), whether the process has tried to access a secure file
(stored as either yes or no) The log created is called
process_security_log where each of the above pieces of information is
stored on a separate line and each entry follows immediately (that is,
there are no blank lines). Write a shell script that will examine
this log and output the User ID of any process that is still running
that has tried to access a secure file.
I started by trying to just capturing the User and echo it but failed.
output=`ps -ef | grep [*]`
set -- $output
User=$1
echo $User
The output of ps is both insufficient and incapable of producing data required by this question.
You need something like auditd, SELinux, or straight up kernel hacks (ie. fork.c) to do anything remotely in the realm of security logging.
Update
Others have made suggestions to use shell command logging, ps and friends (proc or sysfs). They can be useful, and do have their place (obviously). I would argue that they shouldn't be relied on for this purpose, especially in an educational context.
... whether the process has tried to access a secure file (stored as either yes or no)
Seems to be the one that the other answers are ignoring. I stand by my original answer, but as Daniel points out there are other interesting ways to garnish this data.
systemtap
pref
LTTng
For an educational exercise these tools will help provide a more complete answer.
Since this is homework, I'm assuming that the scenario isn't a real-world scenario, and is merely a learning exercise. The shell is not really the right place to do security auditing or process accounting. However, here are some pointers that may help you discover what you can do at the shell prompt.
You might set the bash PROMPT_COMMAND to do your process logging.
You can tail or grep your command history for use in logging.
You can use /usr/bin/script (usually found in the bsdutils package) to create a typescript of your session.
You can run ps in a loop, using subshells or the watch utility, to see what processes are currently running.
You can use pidof or pgrep to find processes more easily.
You can modify your .bashrc or other shell startup file to set up your environment or start your logging tools.
As a starting point, you might begin with something trivial like this:
$ export PROMPT_COMMAND='history | tail -n1'
56 export PROMPT_COMMAND='history | tail -n1'
$ ls /etc/passwd
/etc/passwd
57 ls /etc/passwd
and build in any additional logging data or process information that you think necessary. Hope that gets you pointed in the right direction!
Take a look at the /proc pseudo-filesystem.
Inside of this, there is a subdirectory for every process that is currently running - process [pid] has its information available in /proc/[pid]/. Inside of that directory, you might make use of /prod/[pid]/stat/ or /proc/[pid]/status to get information about which user started the process and when.
I'm not sure what the assignment means by a "secure file," but if you have some way of determining which files are secure, you get get information about open files (including their names) through /prod/[pid]/fd/ and /prod/[pid]/fdinfo.
Is /proc enough for true security logging? No, but /proc is enough to get information about which processes are currently running on the system, which is probably what you need for a homework assignment about shell scripting. Also, outside of this class you'll probably find /proc useful later for other purposes, such as seeing the mapped pages for a process. This can come in handy if you're writing a stack trace utility or want to know how they work, or if you're debugging code that uses memory-mapped files.
I am trying to figure out if it is possible to set up a cron job that will transfer all files from one folder on my server to another folder and then take a set number of files (files chosen randomly) from another folder and put it in the original folder. If so any hints on how to do this, I have no experience with cron at all, I just don't want to have to log in with ftp and do the transfers manually.
Cron is really simple, all it's doing is to run a command of your choice at the specified times of day.
In your case, you probably want to write a shell script that use rsync, scp or ftp to transfer the files, make sure that exits successfully (check exit code from transfer, stored in the $? variable), then move the set of files into the original folder.
I would use rsync and passwordless authentication via ssh keys. That's good for security, and if you want to, you can even limit the receiving side to only allow that ssh key to run rsync's server side.
If this script is called /opt/scripts/myscript.sh, and is to be run once every 10 minutes, add the following to your crontab (run crontab -e to edit your crontab):
*/10 * * * * /opt/scripts/myscript.sh
Remember that the environment variables you have available in your shell are not the same as those available when the cronjob runs, so PATH, etc, may be different. This often causes cron jobs to fail the first few times you run them (See my law on cron jobs: http://efod.se/blog/archive/2010/02/19/forsbergs-law-on-cron-jobs :-)). Any output from cron is sent via mail to the user running the cron job, which is helpful for debugging. Simple debugging writing messages to some file in /tmp/ is also often a way to get your cron jobs running.
In many cases it makes sense to run cron jobs as a special user. Don't run your cron jobs as root unless they absolutely must have root access, it's better to run things as special users that only has limited permissions in the file system.
To edit your cron file:
crontab -e
An example entry for transferring files would look like:
30 3 * * * rsync -av School/* username#server:~/School/ >| /home/username/CronLogs/school_update
the fields are: minute, hour, day, month, day of week, command
So in my example, I transfer files everyday at 3:30am by executing the rsync command listed. Note the *'s mark the fields as unused.
For a quick reference/tutorial, see: this link