How do I print out just the hashsum and file name with sha256sum command? I want Hashsum and just the filename instead of the full path.
Command:
sha256sum /mydir/someOtherDir/file.txt
Output:
123Hashsum /mydir/someOtherDir/file.txt
Desired Output:
123Hashsum file.txt
You can read the output into variables
read -r sha file < <(sha256sum /mydir/someOtherDir/file.txt)
Then you can read just the filename with basename
echo "$sha" "$(basename "$file")"
You can try piping to sed as below (works with absolute paths only) :
sha256sum /mydir/someOtherDir/file.txt | sed 's:/.*/::'
Related
$ ls /tmp/foo/
file1.txt file2.txt
$ ls /tmp/bar/
file20.txt
$ ls /tmp/foo/file*.txt | grep -o -E '[0-9]+' | sort -n | paste -s -d,
1,2
How to fetch the number in the filename from both the directories? in the above example, I need to get 1,2,20, its in bash shell.
UPDATE:
$ ls /tmp/foo/file*.txt /tmp/bar/file*.txt /tmp/jaz99/file*.txt /tmp/nah/file*.txt | grep -o -E '[0-9]+' | sort -n | paste -s -d,
ls: cannot access /tmp/nah/file*.txt: No such file or directory
1,2,20,30,99
in this case, it should not print 99 (as its not matched by *), and should not print the error if file not found.
You can get this done using a loop with output of find:
s=
# run a loop using find command in a process substitution
while IFS= read -d '' -r file; do
file="${file##*/}" # strip down all directory paths
s+="${file//[!0-9]/}," # remove all non-numeric characters and append comma
done < <(find /tmp/{foo,bar,nah,jaz99} -name '*.txt' -print0 2>/dev/null)
echo "${s%,}" # remove last comma from string
Output
1,2,20,30
Here's my take on this. Use arrays. No need to use external tools like sed or awk or find.
#!/usr/bin/env bash
declare -a a=()
for f in /tmp/{foo,bar,nah}/file*.txt; do
[[ $f =~ .*file([0-9]+).* ]]
a+=( ${BASH_REMATCH[1]} )
done
IFS=,
echo "${a[*]}"
The [[...]] expression populates the $BASH_REMATCH array with regex components. You can use that to extract the numbers and place them in a new temporary array, which you can express with comma separators using $IFS.
Results:
$ mkdir /tmp/foo /tmp/bar
$ touch /tmp/foo/file{1,2}.txt /tmp/bar/file20.txt
$ ./doit
1,2,20
I'm attempting to create a bash script that will grep a single file for two separate pieces of data, and print them to stdout.
So far this is what I have:
#!/bin/sh
cd /my/filePath/to/directory
APP=`grep -r --include "inputs.conf" "\[" | grep -oP '^[^\/]+'`
INPUT=`grep -r --include "inputs.conf" "\[" | grep -oP '\[[^\]]+'`
for i in $APP
do
{cd /opt/splunk/etc/deployment-apps
INPUT=`grep -r --include "inputs.conf" "\[" | grep -oP '\[[^\]]+'`
echo -n "$i | $INPUT"}
done
echo "";
exit
Which gives me an output printing the entire output of the first command (which is about 200 lines), then a |, then the other results from the second command. I was thinking I could create an array to do this, however I'm still learning bash.
This is an output example from the command without piping to grep:
TA-XA6x-Server/local/inputs.conf:[perfmon://Processor]
There are 200+ of these in a single execution, and I was looking to have the format be printed as something like this
app="TA-XA6x-Server/local/inputs.conf:" | input="[perfmon://Processor]"
There are essentially two pieces of information I'm attempting to stitch together:
the file path to the file
the contents of the file itself (the input)
Here is an example of the file path:
/opt/splunk/etc/deployment-apps/TA-XA6x-Server/local/inputs.conf
and this is an example of the inputs.conf file contents:
[perfmon://TCPv4]
The easy, mostly-working-ish approach is something like this:
#!/bin/bash
while IFS=: read -r name content; do
printf 'app="%s" | input="%s"\n' "$name" "$content"
done < <(grep -r --include "inputs.conf" "\[")
If you need to work reliably with all possible filenames (including names with colons or newlines) and have GNU grep available, consider the --null argument to grep and adjusting the read usage appropriately:
#!/bin/bash
while IFS= read -r -d '' name && IFS= read -r content; do
printf 'app="%s" | input="%s"\n' "$name" "$content"
done < <(grep -r --null --include "inputs.conf" "\[")
Is there a way to calculate a checksum on a string in Linux? The checksum commands that I have seen (cksum, md5sum, sha1sum, etc.) all require a file as input and I do not have a file. I only have a path to a location and want to calculate the checksum on that path.
echo -n 'exampleString' | md5sum
should work.
echo -n "yourstring" |md5sum
echo -n "yourstring" |sha1sum
echo -n "yourstring" |sha256sum
don't forget -n or the result will change (cuz the newline will be parsed)
I'm having some rather unusual problems using grep in a bash script. Below is an example of the bash script code that I'm using that exhibits the behaviour:
UNIQ_SCAN_INIT_POINT=1
cat "$FILE_BASENAME_LIST" | uniq -d >> $UNIQ_LIST
sed '/^$/d' $UNIQ_LIST >> $UNIQ_LIST_FINAL
UNIQ_LINE_COUNT=`wc -l $UNIQ_LIST_FINAL | cut -d \ -f 1`
while [ -n "`cat $UNIQ_LIST_FINAL | sed "$UNIQ_SCAN_INIT_POINT"'q;d'`" ]; do
CURRENT_LINE=`cat $UNIQ_LIST_FINAL | sed "$UNIQ_SCAN_INIT_POINT"'q;d'`
CURRENT_DUPECHK_FILE=$FILE_DUPEMATCH-$CURRENT_LINE
grep $CURRENT_LINE $FILE_LOCTN_LIST >> $CURRENT_DUPECHK_FILE
MATCH=`grep -c $CURRENT_LINE $FILE_BASENAME_LIST`
CMD_ECHO="$CURRENT_LINE matched $MATCH times," cmd_line_echo
echo "$CURRENT_DUPECHK_FILE" >> $FILE_DUPEMATCH_FILELIST
let UNIQ_SCAN_INIT_POINT=UNIQ_SCAN_INIT_POINT+1
done
On numerous occasions, when grepping for the current line in the file location list, it has put no output to the current dupechk file even though there have definitely been matches to the current line in the file location list (I ran the command in terminal with no issues).
I've rummaged around the internet to see if anyone else has had similar behaviour, and thus far all I have found is that it is something to do with buffered and unbuffered outputs from other commands operating before the grep command in the Bash script....
However no one seems to have found a solution, so basically I'm asking you guys if you have ever come across this, and any idea/tips/solutions to this problem...
Regards
Paul
The `problem' is the standard I/O library. When it is writing to a terminal
it is unbuffered, but if it is writing to a pipe then it sets up buffering.
try changing
CURRENT_LINE=`cat $UNIQ_LIST_FINAL | sed "$UNIQ_SCAN_INIT_POINT"'q;d'`
to
CURRENT LINE=`sed "$UNIQ_SCAN_INIT_POINT"'q;d' $UNIQ_LIST_FINAL`
Are there any directories with spaces in their names in $FILE_LOCTN_LIST? Because if they are, those spaces will need escaped somehow. Some combination of find and xargs can usually deal with that for you, especially xargs -0
A small bash script using md5sum and sort that detects duplicate files in the current directory:
CURRENT="" md5sum * |
sort |
while read md5sum filename;
do
[[ $CURRENT == $md5sum ]] && echo $filename is duplicate;
CURRENT=$md5sum;
done
you tagged linux, some i assume you have tools like GNU find,md5sum,uniq, sort etc. here's a simple example to find duplicate files
$ echo "hello world">file
$ md5sum file
6f5902ac237024bdd0c176cb93063dc4 file
$ cp file file1
$ md5sum file1
6f5902ac237024bdd0c176cb93063dc4 file1
$ echo "blah" > file2
$ md5sum file2
0d599f0ec05c3bda8c3b8a68c32a1b47 file2
$ find . -type f -exec md5sum "{}" \; |sort -n | uniq -w32 -D
6f5902ac237024bdd0c176cb93063dc4 ./file
6f5902ac237024bdd0c176cb93063dc4 ./file1
I have a directory with both uncompressed and gzipped files and want to run wc -l on this directory. wc will provide a line count value for the compressed files which is not accurate (since it seems to count newlines in the gzipped version of the file). Is there a way to create a zwc script similar to zgrep that will detected the gzipped files and count the uncompressed lines?
Try this zwc script:
#! /bin/bash --
for F in "$#"; do
echo "$(zcat -f <"$F" | wc -l) $F"
done
You can use zgrep to count lines as well (or rather the beginning of lines)
zgrep -c ^ file.txt
I use too "cat file_name | gzip -d | wc -l"