shell script to change directory to the tail result path - linux

Question

I'm using Tail to an error happen on the log lines like:
tail -f syschecklog.log | grep "ERROR processEvent: /mnt/docs/"
and this gives results like:
01.lnxp.com 2019-03-13 07:10:24, 345 ERROR processEvent: /mnt/docs/003217899/cfo paid ¿ inv -inc 1234321
So what I do manually is to change the path using cd:
cd /mnt/docs/003217899/
Is there any script to change directory automatically? As I run another manual script to change file names for the files contained in /003217899/, those like /003217899/ are happening many times a day, and they are changing, so I need this script to automatically catch those errors, and change the path then run a file name change script.
In addition to the above, the log line has another subfolder that contains a error file name like /mnt/docs/003217899/attch/fees ¿ to be paid. How can we cd to that directory?
After Altering [Update]
grep "ERROR processEvent: /mnt/docs/" syschecklog.log | sed 's#.*ERROR processEvent: /mnt/docs/ \(/.*\)/.*#\1#' | while read -r DIR
do
BASEDIR=${DIR%/*}
if [ "$BASEDIR" != /mnt/docs/ ]
then
( cd "$BASEDIR" && find -type f -exec touch {} + | python -c 'import os, re; [os.rename(i, re.sub(r"\?", "¿", i)) for i in os.listdir(".")]' )
fi
# end of code for additional requirement
( cd "$DIR" && find -type f -exec touch {} + | python -c 'import os, re;
[os.rename(i, re.sub(r"\?", "¿", i)) for i in os.listdir(".")]' )
done
Results:
[results][1]
3rd script updated for renameFiles();
$ renameFiles()
> {
> # The next line is copied unchanged from the question. This could be improved.
> find -type f -exec touch {} + | python -c 'import os, re; [os.rename(i, re.sub(r"\?", "¿", i)) for i in os.listdir(".")]'
> }
$
$ # Two possible variants because the question was modified.
$ #
$ # To process the complete input file as it is now
$ # grep "ERROR processEvent: /mnt/docs/" syschecklog.log | ...
$ #
$ # To continuously follow the file
$ # tail -f /mnt/docs/syschecklog.log | grep "ERROR processEvent: /mnt/docs/" | ...
$
$ grep "ERROR processEvent: /mnt/docs/" syschecklog.log | sed 's#.*ERROR processEvent: \(/.*\)/.*#\1#' | while read -r DIR
> do
> # additional requirement from comment: if DIR is /mnt/docs/003217899/attch
> # the script should be run both in .../003217899 and .../attch
> BASEDIR=${DIR%/*}
> if [ "$BASEDIR" != /mnt/docs/ ]
> then
> ( cd "$BASEDIR" && renameFiles)
> fi
> # end of code for additional requirement
> ( cd "$DIR" && renameFiles)
> done
-bash: cd: /mnt/docs/001234579/Exp8888861¿_Applicant_Case_Conference_l (No such file or directory): No such file or directory
-bash: cd: /mnt/docs/001888579/¿_SENIOR_RESOLUTION_MANAGER_i(No such file or directory): No such file or directory
-bash: cd: /mnt/docs/001234579/Exp2222276¿18 from all and Treatments Inc. February 27_ 20199999(No such file or directory): No such file or directory
3rd results [3rd results][2]
-bash: cd: /mnt/docs/001234579/Exp8888861¿_Applicant_Case_Conference_l (No such file or directory): No such file or directory
-bash: cd: /mnt/docs/001888579/¿_SENIOR_RESOLUTION_MANAGER_i(No such file or directory): No such file or directory
-bash: cd: /mnt/docs/001234579/Exp2222276¿18 from all and Treatments Inc. February 27_ 20199999(No such file or directory): No such file or directory
grep results as you requested;
grep "ERROR processEvent: /mnt/docs/" syschecklog.log
01.lnxp.com 3 2019-03-14 07:04:30,446 ERROR processEvent: /mnt/docs/001111224/Exposure2178861/Email_from_LAT__18_009945_AABS¿__Summary_not_received12128050 (No such file or directory)
01.lnxp.com 3 2019-03-14 07:05:13,137 ERROR processEvent: /mnt/docs/001567890/Coop_subro_question__TO__ZED_LANDERS_¿_SENIOR__Basse12130781 (No such file or directory)
01.lnxp.com 3 2019-03-14 07:05:19,914 ERROR processEvent: /mnt/docs/001323289/Exposure2622276/OCF¿18 from All and Treatments Inc. February 27_ 201912129762 (No such file or directory)
Results of Locale
$ locale
LANG=en_CA.UTF-8
LC_CTYPE="en_CA.UTF-8"
LC_NUMERIC="en_CA.UTF-8"
LC_TIME="en_CA.UTF-8"
LC_COLLATE="en_CA.UTF-8"
LC_MONETARY="en_CA.UTF-8"
LC_MESSAGES="en_CA.UTF-8"
LC_PAPER="en_CA.UTF-8"
LC_NAME="en_CA.UTF-8"
LC_ADDRESS="en_CA.UTF-8"
LC_TELEPHONE="en_CA.UTF-8"
LC_MEASUREMENT="en_CA.UTF-8"
LC_IDENTIFICATION="en_CA.UTF-8"
LC_ALL=
Results of fgrep python yourscript | od -c -tx1
$ fgrep python invert.sh | od -c -tx1
0000000 f i n d - t y p e
20 20 20 20 66 69 6e 64 20 20 2d 74 79 70 65 20
0000020 f - e x e c t o u c h {
66 20 20 2d 65 78 65 63 20 74 6f 75 63 68 20 7b
0000040 } + | p y t h o n - c
7d 20 2b 20 7c 20 70 79 74 68 6f 6e 20 2d 63 20
0000060 ' i m p o r t o s , r e ;
27 69 6d 70 6f 72 74 20 6f 73 2c 20 72 65 3b 20
0000100 [ o s . r e n a m e ( i , r e
5b 6f 73 2e 72 65 6e 61 6d 65 28 69 2c 20 72 65
0000120 . s u b ( r " \ ? " , " 302 277 "
2e 73 75 62 28 72 22 5c 3f 22 2c 20 22 c2 bf 22
0000140 , i ) ) f o r i i n o
2c 20 69 29 29 20 66 6f 72 20 69 20 69 6e 20 6f
0000160 s . l i s t d i r ( " . " ) ] '
73 2e 6c 69 73 74 64 69 72 28 22 2e 22 29 5d 27
0000200 \n
0a
0000201
I need to change each '?' in the filename to '¿' as the system creates '?' and it shows as '¿', so have to change to that where the server can understand it!
I found that Capital A with hat is created by itself in the system, using CAT
cat invert.sh
#!/bin/bash
renameFiles()
{
find -type f -exec touch {} + | python -c 'import os, re; [os.rename(i, re.sub(r"\?", "¿", i)) for i in os.listdir(".")]'
}
grep "ERROR processEvent: /mnt/docs/" syschecklog.log | sed 's#.*ERROR processEvent: /mnt/docs/ \(/.*\)/.*#\1#' | while read -r DIR
do
BASEDIR=${DIR%/*}
if [ "$BASEDIR" != /mnt/cc-docs ]
then
( cd "$BASEDIR" && renameFiles)
fi
( cd "$DIR" && renameFiles)
results of od -c -txl, on the error file;
echo *|od -c -tx1
0000000 O C F - 2 1 I n v 2 0 8 3 5
4f 43 46 2d 32 31 20 49 6e 76 20 32 30 38 33 35
0000020 9 9 A s s e s s M e d $ 6 2
39 39 20 41 73 73 65 73 73 4d 65 64 20 24 36 32
0000040 1 . 5 0 ( H a n g Q ) ?
31 2e 35 30 20 28 48 61 6e 67 20 51 29 20 3f 20
0000060 d t d F e b 2 7 _ 2 0 1 9
64 74 64 20 46 65 62 20 32 37 5f 20 32 30 31 39
0000100 1 2 1 7 4 5 8 3 \n
31 32 31 37 34 35 38 33 0a
0000111
Checked the systems when using eco on hex encoding on ¿, its attaching  to it as below;
$echo -e '\xc2\xbf'
¿

Answer 1

Script modified again for additional requirements.
(As I did not get answers to all questions I modified the script based on the incomplete information.)
Instead of processing two directories separately the script now uses find in the parent directory (or the only directory), renames and touches all files that contain a '?' in the name. (-name '*\?*').
#! /bin/bash
# Two possible variants because the question was modified.
#
# To process the complete input file as it is now
# fgrep "ERROR processEvent: /mnt/docs/" syschecklog.log | ...
#
# To continuously follow the file
# tail -f syschecklog.log| fgrep "ERROR processEvent: /mnt/docs/" | ...
# The "LANG=C sed ..." avoids problems with invalid UTF-8 characters that do not match '.' in sed's pattern
fgrep "ERROR processEvent: /mnt/docs/" syschecklog.log | LANG=C sed 's#.*ERROR processEvent: \(/mnt/docs/[^/]*\)/.*#\1#' | while IFS= read -r DIR
do
find "$DIR" -name '*\?*' | while IFS= read -r FILE
do
NEW=$(echo "$FILE"| tr '?' $'\xBF')
mv "$FILE" "$NEW"
touch "$NEW"
done
done
Note that grep and sed will switch to buffered output when used in a pipeline. This will delay the processing of the extracted lines. You might have to disable buffering for the commands in the pipeline, see http://mywiki.wooledge.org/BashFAQ/009
2nd major update
There was a problem with invalid characters. In a UTF-8 environment sed behaves strangely when the input contains bytes that are not valid UTF-8 charactes. The pattern . does not match these invalid characters. (The example file contains a byte with the value 0xBF. See http://www.linuxproblem.org/art_21.html. Setting LANG=C for the sed command fixes this problem.
I tested my script with the grep output added to the question. I wrote this into a file somelog.log. I modified my script to use grep pattern somelog.log | ... with a local file instead of using a log file with a full path which does not exist on my test system.
After adding LANG=C to the sed command the script ran successfully with the raw input file provided as an external link.
The output is
$ grep "ERROR processEvent: /mnt/docs/" syschecklog.log | sed 's#.*ERROR processEvent: \(/.*\)/.*#\1#' | while read -r DIR; do BASEDIR=${DIR%/*}; if [ "$BASEDIR" != /mnt/docs/ ]; then ( cd "$BASEDIR" && renameFiles); fi; ( cd "$DIR" && renameFiles); done
bash: cd: /mnt/docs/001234567: No such file or directory
bash: cd: /mnt/docs/001234567/Subdir9876543: No such file or directory
bash: cd: /mnt/docs/002345678: No such file or directory
bash: cd: /mnt/docs/003456789: No such file or directory
bash: cd: /mnt/docs/003456789/Subdir8765432: No such file or directory
... (more similar lines removed)
You can see that it tried to cd into the directories from the log messages. It does not show parts of the file name. In my case it simply failed because the directories don't exist. I think the script should work.
After replacing the two cd and renameFiles commands with find... the output with my test is
find: ‘/mnt/docs/001234567’: No such file or directory
find: ‘/mnt/docs/002345678’: No such file or directory
find: ‘/mnt/docs/003456789’: No such file or directory
...