I am trying to find a specific file/folder based on directory name . There are one thing i could not understand which is tabcmd -> ../bin.2012....
When i did a ls -lrt on a specific path. for e.g
cmd : ls -lrt /opt/tableau/tableau_server/packages/customer-bin.20212.21.1217.2251
it will list as
lrwxrwxrwx. 1 root root 33 Dec 23 07:56 atrdiag -> ../bin.20212.21.1217.2251/atrdiag
lrwxrwxrwx. 1 root root 32 Dec 23 07:56 tabcmd -> ../bin.20212.21.1217.2251/tabcmd
lrwxrwxrwx. 1 root root 39 Dec 23 07:56 serveractutil -> ../bin.20212.21.1217.2251/serveractutil
lrwxrwxrwx. 1 root root 34 Dec 23 07:56 odbcinst -> ../bin.20212.21.1217.2251/odbcinst
lrwxrwxrwx. 1 root root 31 Dec 23 07:56 iusql -> ../bin.20212.21.1217.2251/iusql
lrwxrwxrwx. 1 root root 30 Dec 23 07:56 isql -> ../bin.20212.21.1217.2251/isql
lrwxrwxrwx. 1 root root 37 Dec 23 07:56 custactutil -> ../bin.20212.21.1217.2251/custactutil
lrwxrwxrwx. 1 root root 29 Dec 23 07:56 tsm -> ../bin.20212.21.1217.2251/tsm
The objective is to find tabcmd path dynamically as this path will change from time to time. "I could not use whereis command as it is not registered as a system path/variable"
Alternatively, what i did was
TEST=`find /opt/tableau -type d -name "tabcmd"`
echo "Tabcmd path is : $TEST"
However with the cmd i tried ,, it is returning empty
it returned "Tabcmd path is : "
Update :
when i tried using command below
TABCMD_PATH=`find /opt/tableau/tableau_server -name "tabcmd"`
echo $TABCMD_PATH
/opt/tableau/tableau_server/packages/bin.20212.21.1217.2251/tabcmd /opt/tableau/tableau_server/packages/customer-bin.20212.21.1217.2251/tabcmd
this it will print 2 times instead of 1 and if this is the approach i should take ?
It is empty because you are finding for a directory called tabcmd, you have to find by file -type f
TEST=`find /opt/tableau -type d -name "tabcmd"`
echo "Tabcmd path is : $TEST"
And the following print two path because the file is in both paths, the file link and the symbolic link:
TABCMD_PATH=`find /opt/tableau/tableau_server -name "tabcmd"`
echo $TABCMD_PATH
/opt/tableau/tableau_server/packages/bin.20212.21.1217.2251/tabcmd /opt/tableau/tableau_server/packages/customer-bin.20212.21.1217.2251/tabcmd
You have to find the file excluding the symbolic link:
TABCMD_PATH=`find /opt/tableau/tableau_server -type f -name "tabcmd"`
echo $TABCMD_PATH
I interpret inode as a pointer to the actual place where the file is stored.
But I have problem understanding:
If I use cp file1 file2 in a place where file2 already exists, the inode doesn't change. And If there is originally a hard-link to file2, they now both point to the new file just copied here.
The only reason I can think of is that Linux interprets this as modifying
the file instead of deleting and creating a new file. I don't understand why it's designed this way?
But when I use mv file1 file2, the inode changes to the inode of file1.
You are correct in stating that cp will modify the file instead of deleting and recreating.
Here is a view of the underlying system calls as seen by strace (part of the output of strace cp file1 file2):
open("file2", O_WRONLY|O_TRUNC) = 4
stat("file2", {st_mode=S_IFREG|0664, st_size=6, ...}) = 0
stat("file1", {st_mode=S_IFREG|0664, st_size=3, ...}) = 0
stat("file2", {st_mode=S_IFREG|0664, st_size=6, ...}) = 0
open("file1", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0664, st_size=3, ...}) = 0
open("file2", O_WRONLY|O_TRUNC) = 4
fstat(4, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
read(3, "hi\n", 65536) = 3
write(4, "hi\n", 3) = 3
read(3, "", 65536) = 0
close(4) = 0
close(3) = 0
As you can see, it detects that file2 is present (stat returns 0), but then opens it for writing (O_WRONLY|O_TRUNC) without first doing an unlink.
See for example POSIX.1-2017, which specifies that the destination file shall only be unlink-ed where it could not be opened for writing and -f is used:
A file descriptor for dest_file shall be obtained by performing
actions equivalent to the open() function defined in the System
Interfaces volume of POSIX.1-2017 called using dest_file as the path
argument, and the bitwise-inclusive OR of O_WRONLY and O_TRUNC as the
oflag argument.
If the attempt to obtain a file descriptor fails and the -f option is
in effect, cp shall attempt to remove the file by performing actions
equivalent to the unlink() function defined in the System Interfaces
volume of POSIX.1-2017 called using dest_file as the path argument. If
this attempt succeeds, cp shall continue with step 3b.
This implies that if the destination file exists, the copy will succeed (without resorting to -f behaviour) if the cp process has write permission on it (not necessarily run as the user that owns the file), even if it does not have write permission on the containing directory. By contrast, unlinking and recreating would require write permission on the directory. I would speculate that this is behind the reason why the standard is as it is.
The --remove-destination option on GNU cp will make it do instead what you thought ought to be the default.
Here is the relevant part of the output of strace cp --remove-destination file1 file2. Note the unlink this time.
stat("file2", {st_mode=S_IFREG|0664, st_size=6, ...}) = 0
stat("file1", {st_mode=S_IFREG|0664, st_size=3, ...}) = 0
lstat("file2", {st_mode=S_IFREG|0664, st_size=6, ...}) = 0
unlink("file2") = 0
open("file1", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0664, st_size=3, ...}) = 0
open("file2", O_WRONLY|O_CREAT|O_EXCL, 0664) = 4
fstat(4, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
read(3, "hi\n", 65536) = 3
write(4, "hi\n", 3) = 3
read(3, "", 65536) = 0
close(4) = 0
close(3) = 0
When you use mv and the source and destination paths are on the same file filesystem, it will do an rename, and this will have the effect of unlinking any existing file at the target path. Here is the relevant part of the output of strace mv file1 file2.
access("file2", W_OK) = 0
rename("file1", "file2") = 0
In either case where an destination path is unlinked (whether explicitly by unlink() as called from cp --remove-destination, or as part of the effect of rename() as called from mv), the link count of the inode to which it was pointing will be decremented, but it will remain on the filesystem if either the link count is still >0 or if any processes have open filehandles on it. Any other (hard) links to this inode (i.e. other directory entries for it) will remain.
Investigating using ls -i
ls -i will show the inode numbers (as the first column when combined with -l), which helps demonstrate what is happening.
Example with default cp action
$ rm file1 file2 file3
$ echo hi > file1
$ echo world > file2
$ ln file2 file3
$ ls -li file*
49 -rw-rw-r-- 1 myuser mygroup 3 Jun 13 10:43 file1
50 -rw-rw-r-- 2 myuser mygroup 6 Jun 13 10:43 file2
50 -rw-rw-r-- 2 myuser mygroup 6 Jun 13 10:43 file3
$ cp file1 file2
$ ls -li file*
49 -rw-rw-r-- 1 myuser mygroup 3 Jun 13 10:43 file1
50 -rw-rw-r-- 2 myuser mygroup 3 Jun 13 10:43 file2 <=== exsting inode
50 -rw-rw-r-- 2 myuser mygroup 3 Jun 13 10:43 file3 <=== exsting inode
(Note existing inode 50 now has size 3).
Example with --remove-destination
$ rm file1 file2 file3
$ echo hi > file1
$ echo world > file2
$ ln file2 file3
$ ls -li file*
49 -rw-rw-r-- 1 myuser mygroup 3 Jun 13 10:46 file1
50 -rw-rw-r-- 2 myuser mygroup 6 Jun 13 10:46 file2
50 -rw-rw-r-- 2 myuser mygroup 6 Jun 13 10:46 file3
$ cp --remove-destination file1 file2
$ ls -li file*
49 -rw-rw-r-- 1 myuser mygroup 3 Jun 13 10:46 file1
55 -rw-rw-r-- 1 myuser mygroup 3 Jun 13 10:47 file2 <=== new inode
50 -rw-rw-r-- 1 myuser mygroup 6 Jun 13 10:46 file3 <=== existing inode
(Note new inode 55 has size 3. Unmodified inode 50 still has size 6.)
Example with mv
$ rm file1 file2 file3
$ echo hi > file1
$ echo world > file2
$ ln file2 file3
$ ls -li file*
49 -rw-rw-r-- 1 myuser mygroup 3 Jun 13 11:05 file1
50 -rw-rw-r-- 2 myuser mygroup 6 Jun 13 11:05 file2
50 -rw-rw-r-- 2 myuser mygroup 6 Jun 13 11:05 file3
$ mv file1 file2
$ ls -li file*
49 -rw-rw-r-- 1 myuser mygroup 3 Jun 13 11:05 file2 <== existing inode
50 -rw-rw-r-- 1 myuser mygroup 6 Jun 13 11:05 file3 <== existing inode
#alaniwi's answer covers what is is happening, but there's an implicit why here as well.
The reason cp works the way it does is to provide a way of replacing a file with mulitple names, having all of those names refer to the new file. When the destination of cp is a file that already exists, possibly with multiple names via hard or soft links, cp will make all of those names refer to the new file. There will be no 'orphan' references to the old file left over.
Given this command, it is pretty easy to get the 'just change the file for one name' behavior -- unlink the file first. Given just that as a primitive it would be very hard to implement the 'change all references to point to the new contents' behavior.
Of course, doing rm+cp has some race condition issues (it is two commands), which is why the install command got added in BSD unix -- it basically just does rm + cp, along with some checks to make it atomic in the rare case two people try to install to the same path simultaneously, as well as the more serious problems of someone reading from the file you're trying to install to (a problem with plain cp). Then the GNU version added options to backup the old version and various other useful bookkeeping.
An inode is a collection of metadata for a file, i.e. information about a file, in a Unix/ Unix-like filesystem. It includes permission data, last access/ modify time, file size, etc.
Notably, a file's name/ path is not part of the inode. A filename is just a human-readable identifier for an inode. A file can have one or more names, the number of which is represented in the inode by its number of "links" (hard links). The number associated with the inode, the inode number, which I believe you're interpreting as its physical location on disk, is rather simply a unique identifier for the inode. An inode does contain the location of the file on disk, but that is not the inode number.
So knowing this, the difference you're seeing is in how cp and mv function. When you cp a file you're creating a new inode with a new name and copying the contents of the old file to a new location on disk. When you mv a file all you're doing is changing one of its names. If the new name is already the name of another file, the name is disassociated with the old file (and the old file's link count is reduced by 1) and associated with the new file.
You can read more about inodes here.
I've taken a look to many different topics and did not find an answer to my problem. I created 2 bash scripts on my Ubuntu server and I'm trying to execute them periodically. It seems they are running, but they produce nothing. They are executable:
drwxr-xr-x 14 root root 4096 Mar 14 18:02 ..
-rwxr-xr-x 1 root root 2623 Apr 16 21:18 backup.pl
-rw-r--r-- 1 root root 87066352 May 10 21:37 full_site_backup-10-4-2018.tar.gz
-rwxr-xr-x 1 root root 530 May 11 20:21 checkHealth.sh
drwxr-xr-x 2 root root 4096 May 11 20:35 .
So here is one of my scripts:
#!/bin/bash
# log stdout and stderr to two different files
exec >>/var/log/test.log 2>>/var/log/test.err.log
# ...and log every command we try to execute to stderr (aka looog.err.log)
# set -x
CODE=$(curl -s -o /dev/null -I -A "myuseragent" -w "%{http_code}" https://www.xxxxxxxxxxxxx.xxx/xxxx)
DATE=$(date)
if [ $CODE -gt 300 ]
then
service mysql restart
service tomcat8 restart
>&2 echo "$DATE - KO !!!!!! code retour $CODE"
else
echo "$DATE - OK, code $CODE"
fi
and here is my sudo crontab -e :
# m h dom mon dow command
0 2 * * * root /usr/bin/perl /var/backup/backup.pl
* * * * * root /bin/sh /var/backup/checkHealth.sh
and here is my sudo tail -f /var/log/cron.log:
May 11 20:39:01 ns381471 CRON[10778]: (root) CMD (root /bin/sh /var/backup/checkHealth.sh)
May 11 20:39:26 ns381471 crontab[10823]: (root) BEGIN EDIT (root)
May 11 20:40:01 ns381471 CRON[10880]: (root) CMD (root /bin/sh /var/backup/checkHealth.sh)
May 11 20:40:01 ns381471 CRON[10879]: (root) CMD (/usr/local/rtm/bin/rtm 2 > /dev/null 2> /dev/null)
May 11 20:40:30 ns381471 crontab[10823]: (root) END EDIT (root)
May 11 20:41:01 ns381471 CRON[10974]: (root) CMD (/usr/local/rtm/bin/rtm 2 > /dev/null 2> /dev/null)
May 11 20:41:01 ns381471 CRON[10975]: (root) CMD (root /bin/sh /var/backup/checkHealth.sh)
May 11 20:42:01 ns381471 CRON[11070]: (root) CMD (/usr/local/rtm/bin/rtm 2 > /dev/null 2> /dev/null)
May 11 20:42:01 ns381471 CRON[11071]: (root) CMD (root /bin/sh /var/backup/checkHealth.sh)
Any help would be appreciated. Thanks
Ok for those who are in the same situation, my crontab was
# m h dom mon dow command
0 2 * * * root /usr/bin/perl /var/backup/backup.pl
* * * * * root /bin/sh /var/backup/checkHealth.sh
It appears that I just copied the syntax you can find on many cron tutorials about editing /etc/crontab, but in my case I did a sudo crontab -e, so I did not have to specifiy the user (root), so the working entries were :
# m h dom mon dow command
0 2 * * * /usr/bin/perl /var/backup/backup.pl
* * * * * /bin/sh /var/backup/checkHealth.sh
or simplier :
# m h dom mon dow command
0 2 * * * /var/backup/backup.pl
* * * * * /var/backup/checkHealth.sh
Specifiacally, I need to give files rw----r--
and dirs rwx--xr-x
Use umask 062.
This works because umask only unsets bits, and files aren't normally created with executable bits set in the first place:
$ umask 062
$ touch myfile; mkdir mydir
$ ls -ld myfile mydir
drwx--xr-x 1 user user 0 Dec 5 15:21 mydir
-rw----r-- 1 user user 0 Dec 5 15:21 myfile
I am running a cron-ed bash script to extract cache hits and bytes served per IP address. The script (ProxyUsage.bash) has two parts:
(uniqueIP.awk) find unique IPs and create a bash script do add up the hits and bytes
run the hits and bytes per IP
ProxyUsage.bash
#!/usr/bin/env bash
sudo gawk -f /home/maxg/scripts/uniqueIP.awk /var/log/squid3/access.log.1 > /home/maxg/scripts/pxyUsage.bash
source /home/maxg/scripts/pxyUsage.bash
uniqueIP.awk
{
arrIPs[$3]++;
}
END {
for (n in arrIPs) {
m++; # count arrIPs elements
#print "Array elements: " m;
arrAddr[i++] = n; # fill arrAddr with IPs
#print i " " n;
}
asort(arrAddr); # sort the array values
for (i = 1; i <= m; i++) { # write one command line per IP address
#printf("#!/usr/bin/env bash\n");
printf("sudo gawk -f /home/maxg/scripts/proxyUsage.awk -v v_Var=%s /var/log/squid3/access.log.1 >> /home/maxg/scripts/pxyUsage.txt\n", arrAddr[i])
}
}
pxyUsage.bash
sudo gawk -f /home/maxg/scripts/proxyUsage.awk -v v_Var=192.168.1.13 /var/log/squid3/access.log.1 >> /home/maxg/scripts/pxyUsage.txt
sudo gawk -f /home/maxg/scripts/proxyUsage.awk -v v_Var=192.168.1.14 /var/log/squid3/access.log.1 >> /home/maxg/scripts/pxyUsage.txt
sudo gawk -f /home/maxg/scripts/proxyUsage.awk -v v_Var=192.168.1.22 /var/log/squid3/access.log.1 >> /home/maxg/scripts/pxyUsage.txt
TheProxyUsage.bash script runs as scheduled and creates the pxyUsage.bash script.
However the pxyUsage.text file is not amended with the latest values when the script runs.
So far I run pxyUsage.bash every day myself, as I cannot figure out, why the result is not written to file.
Both bash scripts are set to execute. Actually the file permissions are below:
-rwxr-xr-x 1 maxg maxg 169 Mar 14 08:40 ProxySummary.bash
-rw-r--r-- 1 maxg maxg 910 Mar 15 17:15 proxyUsage.awk
-rwxrwxrwx 1 maxg maxg 399 Mar 17 06:10 pxyUsage.bash
-rw-rw-rw- 1 maxg maxg 2922 Mar 17 07:32 pxyUsage.txt
-rw-r--r-- 1 maxg maxg 781 Mar 16 07:35 uniqueIP.awk
Any hints appreciated. Thanks.
The sudo(8) command requires a pseudo-tty and you do not have one allocated under cron(8); you do have one allocated when logged in the usual way.
Instead of mucking about with sudo(8), just run the script as the correct user.
If you cannot do that, then in the root crontab, do something like this:
su - username /path/to/mycommand arg1 arg2...
This will work because root can use su(1) without neding a password.