I want to get NetworkActivity_5851_*_09-04-2016.done string from NetworkActivity_5851_2326316_09-04-2016.log.gz and here the code I wrote
local file="$1"
local extension="${file##*.}"
if [ $extension = 'done' ]; then
local files=`basename $file`
files="${files#*_}"
files="${files#*_}"
files="${files%_*}"
local q=_"$files"_
local mask="${file/done/log.gz}"
mask="${mask/${q}/_*_}"
r=`ls "${mask}" | wc -l`
and it works correct, but when I run it with python script it fails. I mean r variable has wrong value.
Here is code in Python
shell = Shell(RUN_SCRIPT_2, LOGFILE)
where Shell is
class Shell():
"""
Base class for the shell script object which
is under testing.
"""
def __init__(self, path_to_script, path_to_log=None):
"""
executes shell script and store results
of STDOUT and STDERR into appropriate attributes
"""
self.path_to_log = path_to_log
# clear log file before run
if self.path_to_log:
open(self.path_to_log, 'w').close()
shell = subprocess.Popen([path_to_script],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
shell=True)
# the line below makes sure shell command execution finished
self.stdout, self.stderr = shell.communicate()
self.log_data = self.get_log_data()
and file path_to_script is
echo 'Start'
file="${SOURCE_DIR}/NetworkActivity_5851_3_09-04-2016.done"
extension="${file##*.}"
if [ $extension = 'done' ]; then
files=`basename $file`
files="${files#*_}"
files="${files#*_}"
files="${files%_*}"
q=_"$files"_
mask="${file/done/log.gz}"
mask="${mask/${q}/_*_}"
r=`ls "${mask}" | wc -l`
echo $r
if [ $r = $files ]; then
rez=0
else rez=1
fi
fi
if [[ $rez -eq 1 ]]; then
echo "Failure"
else echo "Success"
fi
echo 'Finish'
So when I run path_to_script from terminal r variable sets number of files when I run it using Python it sets 0.
You can do that using bash string-manipulation techniques alone.
$ inputString="NetworkActivity_5851_2326316_09-04-2016.log.gz"
$ substring="${inputString%%.*}" # Removing the part after the first '.'
$ [[ $substring =~ .*_([[:digit:]]+)_.* ]] && NUM=${BASH_REMATCH[1]} # Extracting the number you want to replace
$ finalString="${substring/$NUM/*}.done" # Forming the final string with the extension
$ printf "%s\n" "$finalString"
NetworkActivity_5851_*_09-04-2016.done
You can put together this logic in a shell script and run the same for multiple files you have. The above commands though can be run directly on the console.
Well, I'm based on your first paragraph. Hope I got the problem! This is my solution with Perl and regular expression with a simple command line.
~$ gunzip NetworkActivity_5851_2326316_09-04-2016.log.gz && perl -e 'while(<>){print $_ if $_ =~ /NetworkActivity_\d+_.*_\d{2}\-\d{2}-\d{4}\.done/}' NetworkActivity_5851_2326316_09-04-2016.log
Hope it helps!
Related
#! /usr/bin/bash
for var in "$#"
do
echo $var
done
simple shell script which displays each of the command
line arguments, one at a time and stops displaying command line
arguments when it gets an argument whose value is “stop”.?
IIUC, try this:
#!/bin/bash
for var in "$#"
do
if [[ "$var" == 'stop' ]]; then
exit 0
else
echo "$var"
fi
done
Process args with a while loop:
#!/usr/bin/env bash
while test -n "$1"; do
test "$1" != 'stop' && echo "$1" && shift || break
done
#!/bin/bash
#if there are no args supplied exit with 1
if [ "$#" -eq 0 ]; then
echo "Unfortunately you have not passed any parameter"
exit 1
fi
#loop over each argument
for arg in "$#"
do
if [ -f arg ]; then
echo "$arg is a file."
#iterates over the files stated in arguments and reads them $
cat $arg | while read line;
do
#should access only first line of the file
if [ head -n 1 "$arg" ]; then
process line
echo "Script has ran successfully!"
exit 0
#should access only last line of the file
elif [ tail -n 1 "$arg" ]; then
process line
echo "Script has ran successfully!"
exit 0
#if it accesses any other line of the file
else
echo "We only process the first and the last line of the file."
fi
done
else
exit 2
fi
done
#function to process the passed string and decode it in base64
process() {
string_to_decode = "$1"
echo "$string_to_decode = " | base64 --decode
}
Basically what I want this script to do is to loop over the arguments passed to the script and then if it's a file then call the function that decodes in base64 but just on the first and the last line of the chosen file. Unfortunately when I run it even with calling a right file it does nothing. I think it might be encountering problems with the if [ head -n 1 "$arg" ]; then part of the code. Any ideas?
EDIT: So I understood that I am actually just extracting first line over and over again without really comparing it to anything. So I tried changing the if conditional of the code to this:
first_line = $(head -n 1 "$arg")
last_line = $(tail -n 1 "$arg")
if [ first_line == line ]; then
process line
echo "Script has ran successfully!"
exit 0
#should access only last line of the file
elif [ last_line == line ]; then
process line
echo "Script has ran successfully!"
exit 0
My goal is to iterate through files for example one is looking like this:
MTAxLmdvdi51awo=
MTBkb3duaW5nc3RyZWV0Lmdvdi51awo=
MXZhbGUuZ292LnVrCg==
And to decode the first and the last line of each file.
To decode the first and last line of each file given to your script, use this:
#! /bin/bash
for file in "$#"; do
[ -f "$file" ] || exit 2
head -n1 "$file" | base64 --decode
tail -n2 "$file" | base64 --decode
done
Yea, as the others already said the true goal of the script isn't really clear. That said, i imagine every variation of what you may have wanted to do would be covered by something like:
#!/bin/bash
process() {
encoded="$1";
decoded="$( echo "${encoded}" | base64 --decode )";
echo " Value ${encoded} was decoded into ${decoded}";
}
(( $# )) || {
echo "Unfortunately you have not passed any parameter";
exit 1;
};
while (( $# )) ; do
arg="$1"; shift;
if [[ -f "${arg}" ]] ; then
echo "${arg} is a file.";
else
exit 2;
fi;
content_of_first_line="$( head -n 1 "${arg}" )";
echo "Content of first line: ${content_of_first_line}";
process "${content_of_first_line}";
content_of_last_line="$( tail -n 1 "${arg}" )";
echo "Content of last line: ${content_of_last_line}";
process "${content_of_last_line}";
line=""; linenumber=0;
while IFS="" read -r line; do
(( linenumber++ ));
echo "Iterating over all lines. Line ${linenumber}: ${line}";
process "${line}";
done < "${arg}";
done;
some additions you may find useful:
If the script is invoked with multiple filenames, lets say 4 different filenames, and the second file does not exist (but the others do),
do you really want the script to: process the first file, then notice that the second file doesnt exist, and exit at that point ? without processing the (potentially valid) third and fourth file ?
replacing the line:
exit 2;
with
continue;
would make it skip any invalid filenames, and still process valid ones that come after.
Also, within your process function, directly after the line:
decoded="$( echo "${encoded}" | base64 --decode )";
you could check if the decoding was successful before echoing whatever the resulting garbage may be if the line wasnt valid base64.
if [[ "$?" -eq 0 ]] ; then
echo " Value ${encoded} was decoded into ${decoded}";
else
echo " Garbage.";
fi;
--
To answer your followup question about the IFS/read-construct, it is a mixture of a few components:
read -r line
reads a single line from the input (-r tells it not to do any funky backslash escaping magic).
while ... ; do ... done ;
This while loop surrounds the read statement, so that we keep repeating the process of reading one line, until we run out.
< "${arg}";
This feeds the content of filename $arg into the entire block of code as input (so this becomes the source that the read statement reads from)
IFS=""
This tells the read statement to use an empty value instead of the real build-in IFS value (the internal field separator). Its generally a good idea to do this for every read statement, unless you have a usecase that requires splitting the line into multiple fields.
If instead of
IFS="" read -r line
you were to use
IFS=":" read -r username _ uid gid _ homedir shell
and read from /etc/passwd which has lines such as:
root:x:0:0:root:/root:/bin/bash
apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
then that IFS value would allow it to load those values into the right variables (in other words, it would split on ":")
The default value for IFS is inherited from your shell, and it usually contains the space and the TAB character and maybe some other stuff. When you only read into one single variable ($line, in your case). IFS isn't applied but when you ever change a read statement and add another variable, word splitting starts taking effect and the lack of a local IFS= value will make the exact same script behave very different in different situations. As such it tends to be a good habbit to control it at all times.
The same goes for quoting your variables like "$arg" or "${arg}" , instead of $arg . It doesn't matter when ARG="hello"; but once the value starts containing spaces suddenly all sorts of things can act different; suprises are never a good thing.
This question already has an answer here:
Why variable values are lost after terminating the loop in bash? [duplicate]
(1 answer)
Closed 2 years ago.
I would like to make a script which allow me to execute a command which inherit environment variables from any PID.
Here the script I made :
#!/bin/sh
VARS=$(cat -A /proc/1/environ | tr "^#" "\n")
COMMAND=""
# sh compatible loop on a variable containing multiple lines
printf %s "$VARS" | while IFS='\n' read -r var
do
if [ "$var" != "" ]; then
export "$var"
fi
done
exec "$#"
I though exported variables would be available for the child process (created by exec) but this is obviously not the case because sh my_script.sh printenv doesn't show environment variables which are in /proc/1/environ.
I also tried the following script :
#!/bin/sh
VARS=$(cat -A /proc/1/environ | tr "^#" "\n")
COMMAND=""
# sh compatible loop on a variable containing multiple lines
printf %s "$VARS" | while IFS='\n' read -r var
do
if [ "$var" != "" ]; then
# Replace 'VAR=var' by 'VAR="var"' for eval
# sed replace only the first occurence of the '=' due of the missing /g parameter
escaped=$(echo $var | sed -e 's/=/="/')\"
COMMAND="${COMMAND} ${escaped}"
fi
done
COMMAND="${COMMAND} $#"
eval $COMMAND
However, it looks like eval doesn't export variables even if the evaluated command looks like VAR=value my_command.
How I am supposed to achieve my needs ?
Thanks in advance
That one should work (tested on RHEL 7)
#!/bin/bash
locPROC=$1
locCMD=$2
if [[ -z $locPROC || -z $locCMD ]]; then
exit
fi
if [[ -r /proc/${locPROC}/environ ]]; then
while IFS= read -r -d '' line; do
#Making sure it's properly quoted
locVar="${line/=/=\"}\""
#You probably don't want to mess with those
if [[ ${locVar:0:1} != "_" && ${locVar} != A__z* ]]; then
eval "$locVar"
eval "export ${locVar%%=*}"
fi
done < "/proc/${locPROC}/environ"
$locCMD
else
echo "Environment file is either inexistant or unreadable"
fi
EDITED : According to comments (still use eval...got to read more :) )
Here is my code:
#!/bin/bash
if [[ $1 = "" ]]; then
exit 0
fi
array=($(cat $1))
let b=${#array[#]}-1
count=0
for i in {1..7}; do
for j in {30..37}; do
for n in {40..47}; do
if [[ $count -gt $b ]]; then
printf '\n'
printf '\e[0m'
exit 1
fi
printf '\e[%s;%s;%sm%-5s' "$i" "$j" "$n" "${array[$count]}"
printf '\e[0m'
let count=$count+1
done
printf '\n'
done
done
#printf '\n'
printf '\e[0m'
exit 0
The problem is that when I start it like this
. color.sh arg
or without argument, it just closes. I know that the reason for that is exit. Is there any way correct my code so I could start a script with dot at start and terminal wouldn't close after execution? I don't want to start it like this: ./script
Replace all exit with return.
return inside a sourced script will even work with exit codes:
$ . <(echo "echo before; return 0; echo after")
before
$ echo $?
0
$ . <(echo "echo before; return 7; echo after")
before
$ echo $?
7
When you use the dot to run a script you are "sourcing" it, which means the interpreter reads and executes all the commands in that script in the context of the current environment without spawning a subshell, as if you had typed each yourself.
That's why if you source it you can set variables in a script that will remain after it has run, whereas running it in a subshell would encapsulate them, and they would go away when the script ends.
Accordingly, if you source a script that hits an exit, it causes the calling environment to exit. Use return as Socowi suggested.
I am looking into how a particular exploit works, and I chose to look at one in the program 'chkrootkit' which allows for any user to run a malicious file as root. The source code for this vulnerable shellscript is as follows
slapper (){
SLAPPER_FILES="${ROOTDIR}tmp/.bugtraq ${ROOTDIR}tmp/.bugtraq.c"
SLAPPER_FILES="$SLAPPER_FILES ${ROOTDIR}tmp/.unlock ${ROOTDIR}tmp/httpd \
${ROOTDIR}tmp/update ${ROOTDIR}tmp/.cinik ${ROOTDIR}tmp/.b"a
SLAPPER_PORT="0.0:2002 |0.0:4156 |0.0:1978 |0.0:1812 |0.0:2015 "
OPT=-an
STATUS=0
file_port=
if ${netstat} "${OPT}"|${egrep} "^tcp"|${egrep} "${SLAPPER_PORT}">
/dev/null 2>&1
then
STATUS=1
[ "$SYSTEM" = "Linux" ] && file_port=`netstat -p ${OPT} | \
$egrep ^tcp|$egrep "${SLAPPER_PORT}" | ${awk} '{ print $7 }' |
tr -d :`
fi
for i in ${SLAPPER_FILES}; do
if [ -f ${i} ]; then
file_port=$file_port $i
STATUS=1
fi
done
if [ ${STATUS} -eq 1 ] ;then
echo "Warning: Possible Slapper Worm installed ($file_port)"
else
if [ "${QUIET}" != "t" ]; then echo "not infected"; fi
return ${NOT_INFECTED}
fi
}
I know that the reason the exploit works is because the line 'file_port=$file_port $i' will execute all files specified in $SLAPPER_FILES as the user chkrootkit is running (usually root), if $file_port is empty, because of missing quotation marks around the
variable assignment."
My question is why does the command
file_port=$file_port $i
result in execution of the file? Assuming that $i refers to the path of the file (/tmp/update)
I can see that file_port might be changed to some long netstat command in the previous if statement, is this something to do with it?
I've been trying to get my head around this all day to no avail, so at this point any help will be greatly appreciated :)
This is the one-shot variable assignment feature of any Bourne shell. Any command can be prefixed with zero or more variable assignments:
VAR1=VALUE1 VAR2=VALUE2 command arguments ...
Runs command arguments ... with the respective environment variables set for just this command. A typical use might be
EDITOR=vim crontab -e