Linux returns "select: Bad file descriptor" error after using "cat" - linux

I am running a Perl script to read from multiple files on a remote machine. My code looks something like this:
open HANDLE, "/usr/bin/ssh $host cat /home/log_1.log /home/log_2.log 2>>./errorLog.log |"
Basically I am storing the file contents into HANDLE and all errors go into the errorLog.log file. But in my errorLog.log file I keep seeing "select: Bad file descriptor". I am not sure what's causing this and Google searching did not help very much. So my questions are: what does this error mean and how can I resolve it?
EDIT: In the above command, log_2.log may not exist in the remote machine. In that case, I wanted to output the error message into errorLog.log. So when my script finds that the log_2.log file does not exist in the machine, it should log "cat: /home/log_2.log: No such file or directory" but sometimes I see "cat: /home/log_2.log: No such file or directoryselect: Bad file descriptor" (two error messages without a newline).

I see three things wrong with your code:
you aren't checking the return value of open
you are using the two argument version of open
and you are using a bareword filehandle
The first will probably tell you what is going wrong, the other two are not wrong per se, but are not Modern Perl. I would write the code like this
open my $fh, "-|", "/usr/bin/ssh $host cat /home/log_[12].log 2>>./errorLog.log"
or die "could not run remote command: $!";
Or better yet
use IPC::Open3;
use Symbol 'gensym';
my $err = gensym; #deal with stupidity in the IPC::Open3 interface
my $pid = open3 my $in, my $out, $err,
"/usr/bin/ssh", $host, "cat", "/home/log_[12].log";
while (<$out>) {
#do stuff with the stdout of ssh
}
While (<$err>) {
#do stuff with the stderr of ssh
}

Related

Retrieve underlying file of tee command

References
Fullcode of what will be discussed here:
https://github.com/djon2003/com.cyberinternauts.linux.backup
activateLogs question that solved how to log to file and screen: https://stackoverflow.com/a/70792273/214898
Limitation
Just a small reminder from the last question: this script is executed on limited environment, on a QNAP (NAS).
Background
I have a function that activate logging which now has three modes: SCREEN, DISK, BOTH. With some help (from the question of the link above), I achieve to make work the BOTH option. DISK & BOTH use a file descriptor numbered 3. The first is pointing to a file and the second to stdout.
On exit of my script (using trap), it detects if there were logged errors and send them via email.
Code
function sendErrorMailOnExit()
{
## If errors happened, then send email
local isFileDescriptor3Exist=$(command 2>/dev/null >&3 && echo "Y")
if [ "$isFileDescriptor3Exist" = "Y" ]; then
local logFile=$(readlink /proc/self/fd/3 | sed s/.log$/.err/)
local logFileSize=$(stat -c %s "$logFile")
if [ $logFileSize -gt 0 ]; then
addLog "N" "Sending error email"
local logFileName=$(basename "$logFile")
local logFileContent=$(cat "$logFile")
sendMail "Y" "QNAP - Backup error" "Error happened on backup. See log file $logFileName\n\nLog error file content:\n$logFileContent"
fi
fi
}
trap sendErrorMailOnExit EXIT
Problem
As you can see, this works well because the file descriptor #3 is using a file. But now, using the BOTH option, the file descriptor #3 is pointing to stdout and the file is written via tee. Hence my question, how could I get the location of the file of tee.
Why not only using a variable coming from my function activateLogs would you say? Because, this function relaunches the script to be able to get all the logs not caught before the function is called. Thus why using this method to retrieve the error file location.
Possible solutions, but not the best (I hope)
One way would be to pass the file location through a script
parameter, but I would prefer not do that if that can be avoid.
Another, would be to create a "fake" file descriptor #4 (probably my best solution up to now) that would always point to the file.
Does anyone have an idea?
I finally opted for the creation of a "fake" file descriptor #4 that does not nothing except pointing to the current log file.

How to capture linux command log into the file?

Let's say I have the below command.
STATE_NOT_C_COUNT=`mongo --host "${DB_HOST}" --port 27017 "${MONGO_DATABASE}" --eval "db.$MONGO_DATABASE.count({\"state\" : {"'"$ne"'":\"C\"},\"physicalTableName\":\"table_name\"},{nolock:true})" | tail -1`
When I used to run the above command, got the exception like
exception: connect failed
I want to capture this exception in into the file via the error function.
error(){
if [ "$?" -ne "0" ]; then
echo "$1" 2>&1 error_log
exit 1
fi
}
I'm using the above function like this:
error $STATE_NOT_C_COUNT
But I'm not able to capture the exception through the function in files.
What you are doing is terrible. Let the program that fails print its error messages to stderr, and ensure that stderr is pointed to the right thing. However, the major issue you are having is just lack of quotes. Try:
error "$STATE_NOT_C_COUNT"
The issue is that the command error $STATE_NOT_C_COUNT is subject to field splitting, so if $STATE_NOT_C_COUNT contains any whitespace it is split into arguments, and you are only writing the first one. Another alternative is to write echo "$#" in the function, but this will squash whitespace. However, it cannot be stressed enough that this is a terrible approach, completely against the unix philosophy. The program should write its error to stderr, and you should let them go there. Just make sure stderr is pointed where you want it. The only possible reason to capture stderr is if you want to write it to multiple locations, so you might pipe it to tee or to a syslogger, or some other message bus, but doing such a thing is questionable.

the file says it's there, but it's not really there at all

What condition could cause this?
a dentry for the file exists
-e says the file is there
cat says "No such file or directory"
opendir(my $dh, $tdir) or die "Could not open temp dir: $!";
for my $file (readdir($dh)) {
print STDERR "checking file $tdir/$file\n";
next unless -e "$tdir/$file";
print STDERR "here's the file for $tdir/$file: ";
print STDERR `cat $tdir/$file`;
die "Not all files from hub '$hid' have been collected!: $tdir/$file";
}
Output:
checking file
/tmp/test2~22363~0G0Tjv/22363~0~4~22380~0~1~Test2~Event~Note
here's the file for /tmp/test2~22363~0G0Tjv/22363~0~4~22380~0~1~Test2~Event~Note:
cat: /tmp/test2~22363~0G0Tjv/22363~0~4~22380~0~1~Test2~Event~Note: No such file or directory
IPC Fatal Error: Not all files from hub '22363~0~4' have been collected!:
I realize that theoretically some other process or thread could be stepping on my file in between cycles, but this is Perl (single-threaded) and Test2/Test::Simple's test directory, which should be limited to being managed by the master process.
What's brought me here is that I've seen similar errors in other code we have:
if(-e $cachepath) {
unlink $cachepath
or Carp::carp("cannot unlink...
which throws "cannot unlink..."
also
$cache_mtime = (stat($cachepath))[_ST_MTIME];
....
if ($cache_mtime){
open my($in), '<:raw', $cachepath
or $self->_error("LoadError: Cannot open $cachepath for reading:
which throws that "LoadError: cannot open..." exception
Those are in cases which also should only have the one process operating on the directory, and that all makes me suspect there's something going on.
This is linux kernel, fully up-to-date with vendor patches.
It turns out it really was child processes and race conditions. The newer code in Test::Simple doesn't work well with nested subtests. Anyway, thanks for allowing me to rule out some obscure filesystem thing that I didn't know about.

GetAttributes uses wrong working directory in subthread

I used File::Find to traverse a directory tree and Win32::File's GetAttributes function to look at the attributes of files found in it. This worked in a single-threaded program.
Then I moved the directory traversal into a separate thread, and it stopped working. GetAttributes failed on every file with "The system cannot find the file specified" as the error message in $^E.
I traced the problem to the fact that File::Find uses chdir, and apparently GetAttributes doesn't use the current directory. I could work around this by passing it an absolute path, but then I could run into path length limits, and long paths are definitely going to be present where this script will run, so I really need to take advantage of chdir and relative paths.
To demonstrate the problem, here is a script which creates a file in the current directory, another file in a subdirectory, chdir's to the subdirectory, and looks for the file 3 ways: system("dir"), open, and GetAttributes.
When the script is run without arguments, dir shows the subdirectory, open finds the file in the subdirectory, and GetAttributes returns its attributes successfully. When run with --thread, all the tests are done in a subthread, and the dir and open still work, but the GetAttributes fails. Then it calls GetAttributes on the file that is in the original directory (which we have chdir'ed out of) and it finds that one! Somehow GetAttributes is using the original working directory of the process - or maybe the working directory of the main thread - unlike all the other file operations.
How can I fix this? I can guarantee that the main thread won't do any chdir'ing, if that matters.
use strict;
use warnings;
use threads;
use Data::Dumper;
use Win32::File qw/GetAttributes/;
sub doit
{
chdir("testdir") or die "chdir: $!\n";
system "dir";
my $attribs;
open F, '<', "file.txt" or die "open: $!\n";
print "open succeeded. File contents:\n-------\n", <F>, "\n--------\n";
close F;
my $x = GetAttributes("file.txt", $attribs);
print Dumper [$x, $attribs, $!, $^E];
if(!$x) {
# If we didn't find the file we were supposed to find, how about the
# bad one?
$x = GetAttributes("badfile.txt", $attribs);
if($x) {
print "GetAttributes found the bad file!\n";
if(open F, '<', "badfile.txt") {
print "opened the bad file\n";
close F;
} else {
print "But open didn't open it. Error: $! ($^E)\n";
}
}
}
}
# Setup
-d "testdir" or mkdir "testdir" or die "mkdir testdir: $!\n";
if(!-f "badfile.txt") {
open F, '>', "badfile.txt" or die "create badfile.txt: $!\n";
print F "bad\n";
close F;
}
if(!-f "testdir/file.txt") {
open F, '>', "testdir/file.txt" or die "create testdir/file.txt: $!\n";
print F "hello\n";
close F;
}
# Option 1: do it in the main thread - works fine
if(!(#ARGV && $ARGV[0] eq '--thread')) {
doit();
}
# Option 2: do it in a secondary thread - GetAttributes fails
if(#ARGV && $ARGV[0] eq '--thread') {
my $thr = threads->create(\&doit);
$thr->join();
}
Eventually, I figured out that perl is maintaining some kind of secondary cwd that only applies to perl built-in operators, while GetAttributes is using the native cwd. I don't know why it does this or why it only happens in the secondary thread; my best guess is that perl is trying to emulate the unix rule of one cwd per process, and failing because the Win32::* modules don't play along.
Whatever the reason, it's possible to work around it by forcing the native cwd to be the same as perl's cwd whenever you're about to do a Win32::* operation, like this:
use Cwd;
use Win32::FindFile qw/SetCurrentDirectory/;
...
SetCurrentDirectory(getcwd());
Arguably File::Find should do this when running on Win32.
Of course this only makes the "pathname too long" problem worse, because now every directory you visit will be the target of an absolute-path SetCurrentDirectory; try to work around it with a series of smaller SetCurrentDirectory calls and you have to figure out a way to get back where you came from, which is hard when you don't even have fchdir.

egrep command with piped variable in ssh throwing No Such File or Directory error

Ok, here I'm again, struggling with ssh. I'm trying to retrieve some data from remote log file based on tokens. I'm trying to pass multiple tokens in egrep command via ssh:
IFS=$'\n'
commentsArray=($(ssh $sourceUser#$sourceHost "$(egrep "$v" /$INSTALL_DIR/$PROP_BUNDLE.log)"))
echo ${commentsArray[0]}
echo ${commentsArray[1]}
commax=${#commentsArray[#]}
echo $commax
where $v is something like below but it's length is dynamic. Meaning it can have many file names seperated by pipe.
UserComments/propagateBundle-2013-10-22--07:05:37.jar|UserComments/propagateBundle-2013-10-22--07:03:57.jar
The output which I get is:
oracle#172.18.12.42's password:
bash: UserComments/propagateBundle-2013-10-22--07:03:57.jar/New: No such file or directory
bash: line 1: UserComments/propagateBundle-2013-10-22--07:05:37.jar/nouserinput: No such file or directory
0
Thing worth noting is that my log file data has spaces in it. So, in the code piece I've given, the actual comments which I want to extract start after the jar file name like : UserComments/propagateBundle-2013-10-22--07:03:57.jar/
The actual comments are 'New Life Starts here' but the logs show that we are actually getting it till 'New' and then it breaks at space. I tried giving IFS but of no use. Probably I need to give it on remote but I don't know how should I do that.
Any help?
Your command is trying to run the egrep "$v" /$INSTALL_DIR/$PROP_BUNDLE.log on the local machine, and pass the result of that as the command to run via SSH.
I suspect that you meant for that command to be run on the remote machine. Remove the inner $() to get that to happen (and fix the quoting):
commentsArray=($(ssh $sourceUser#$sourceHost "egrep '$v' '/$INSTALL_DIR/$PROP_BUNDLE.log'"))
You should use fgrep to avoid regex special interpretation from your input:
commentsArray=($(ssh $sourceUser#$sourceHost "$(fgrep "$v" /$INSTALL_DIR/$PROP_BUNDLE.log)"))

Resources