I am trying to write hashes to the metadata part of my image files. In the Exiftool Forum I saw this
exiftool FILE -rawimagedigest=`exiftool FILE -all= -o - | md5`
However, I would rather not run it manually for each file, and I do prefer SHA.
I tried this
find . -name "*" -exec sh -c '
md5hash=$(exiftool "$1" -all= -m -o - | md5)
sha256hash=$(exiftool "$1" -all= -m -o - | shasum -a 256)
exiftool -overwrite_original "$1" -FileImageMd5=$md5hash;
exiftool -overwrite_original "$1" -FileImageSha256=$sha256hash
' _ {} \;
Using the example file I created a config making it possibly to write to FileImageMd5 and FileImageSha256. However, the script only works without the line
exiftool -overwrite_original "$1" -FileImageSha256=$sha256hash
If I substitute the variable in the end with $md5hash it runs as expected.
The config file is named .ExifTool_config and placed in $HOME. It consist of the following
%Image::ExifTool::UserDefined = (
'Image::ExifTool::XMP::Main' => {
rlp => {
SubDirectory => {
TagTable => 'Image::ExifTool::UserDefined::rlp',
},
},
},
);
%Image::ExifTool::UserDefined::rlp = (
GROUPS => { 0 => 'XMP', 1 => 'XMP-rlp', 2 => 'Image' },
NAMESPACE => { 'rlp' => 'http://ns.ladekjaer.org/rlp/1.0/' },
WRITABLE => 'string',
FileUniqueId => { Writable => 'lang-alt' },
FileImageSha256 => { Writable => 'lang-alt' },
FileImageMd5 => { Writable => 'lang-alt' },
);
1; #end
Apparently the script failed due to
shasum -a 256
ending its output with
-
Since a SHA256 written in hex is always 64 characters, can this be solved by adding
| head -c 64
Thus making the script
find . -name "*" -exec sh -c '
md5hash=$(exiftool "$1" -q -all= -m -o - | md5)
sha256hash=$(exiftool "$1" -q -all= -m -o - | shasum -a 256 | head -c 64)
exiftool -overwrite_original -q "$1" -FileImageMd5=$md5hash;
exiftool -overwrite_original -q "$1" -FileImageSha256=$sha256hash
' _ {} \;
Related
I'm tryingto execute lowriter command several times in a loop:
#!/bin/sh
ROOT_DIRECTORY="docs"
function count_files()
{
ls -lR $ROOT_DIRECTORY/*.doc | wc -l
}
function get_number_folders()
{
expr $(count_files) / 100 + 1
}
function create_folders_and_fill_with_files()
{
for num in $(seq 1 $(get_number_folders))
do
#mkdir $ROOT_DIRECTORY/$num
mv `ls $ROOT_DIRECTORY/*.doc | head -100` $ROOT_DIRECTORY/$num/
convert_to_pdf $ROOT_DIRECTORY $num
done
}
function convert_to_pdf()
{
lowriter -convert-to pdf --outdir ./docs/ $1/$2/*.doc --headless
set -x
}
create_folders_and_fill_with_files
Everything "works" fine, but the command was executed only one time.
How can I execute the lowriter command all times that for need.
Thankyou
I have solved my question adding 2>&1 in the end of the convertion line;
lowriter -convert-to pdf --outdir ./docs/ $1/$2/*.doc --headless 2>&1
We want to search and delete the data files that ended with extension of .pppd
We can search the files as
find $path -type f -name '*.pppd' -delete
but how to tell to find command to filter only the data files?
Example how to verify if file is data ( by file command )
file /data/file.pppd
/data/file.pppd: data
file command from manual page
NAME
file — determine file type
SYNOPSIS
file [-bchiklLNnprsvz0] [--apple] [--mime-encoding] [--mime-type] [-e testname] [-F separator] [-f namefile] [-m magicfiles] file ...
file -C [-m magicfiles]
file [--help]
You have to launch a shell:
find "${path}" \
-type f \
-name '*.pppd' \
-exec bash -c 'test "$(file "${1}"|awk -F: "{print \$NF}")" = "data"' -- {} \; \
-print
You can use the find command with the exec option to launch an explicit subshell that runs a shell loop to compare the output type.
find "$path" -type f \
-name '*.pppd' \
-exec bash -c 'for f; do [[ $(file -b "$f") = "data" ]] && echo "$f" ; done' _ {} +
This Unix.SE answer beautifully explains how the -exec bash -c option works with the find command. To briefly explain how it works, the result of find command based on the filter conditions ( -name, -type and -path ) are passed as positional arguments to the loop run under exec bash -c '..'. The loop iterates over the argument list ( for f is analogous to for f in "$#" ) and prints only the files whose type is data. Instead of parsing the result of file, use file -b to get the type directly.
You can do it like this. you can change empty regex for a valid Bash regex like for instance ^data and the txt extension for what you want to search for :
#!/bin/bash
read -a files <<< $(find . -type f -name '*.pppd' )
for file in "${files[#]}"
do
[[ "$(file -b $file )" =~ ^empty ]] && echo $file
done
If you want to delete the file :
[[ "$(file -b $file )" =~ ^empty ]] || rm "$file"
Hope it helps!
I am very new to Bash scritping and to get some practice, I am attempting to write a script that takes in a source directory and a destination directory. The script will search the source directory and copy its structure of sub-directories into the target directory (any files will be ignored, just the directories themselves will be duplicated). The source directory can have any number of sub-directories at any depth. What would be the best way to achieve this? I have started by attempting to write a recursive function where, if a directory is found, the function is called recursively onto this directory. However, due to my lack of experience with scripting, I have been stumped.
Here is what I have so far:
#! /bin/bash
if [ ! $# -eq 2 ]; then
echo "ERROR: Script needs 2 arguments: $0 source/directory/ target/directory/"
exit
fi
SOURCE_DIR=$1
TARGET_DIR=$2
function searchForDirectory {
for FILE in ls $SOURCE_DIR/*; do #should be 'ls *current_directory*'
if [ -d $FILE ]; then
searchForDirectory #call function recursively onto $FILE
#Do stuff here to create copy of this directory in target dir
fi
done
}
if [ ! -d $SOURCE_DIR ]; then
echo "ERROR: Source directory does not exist"
exit
else
searchForDirectory
fi
I know that this is just a skeleton function, and a lot more work would need to be put into it, but I'm just looking for guidance as to whether this is the correct way to go about this before I go any further, and if so, what would be my next step? How do I pass a directory into my function?
EDIT: Here is my solution thanks to Ivan's help below
#! /bin/bash
if [ ! $# -eq 2 ]; then
echo -e "\nERROR: Script needs 2 arguments:\n$0 source/directory/ target/directory/\n"
exit
fi
function recursiveDuplication {
for file in `ls $1`; do
if [ -d $1/$file ] && [ ! -d $2/$file ]; then
mkdir $2/$file
recursiveDuplication $1/$file $2/$file
elif [[ $1/$file == *.png ]]; then
convert $1/$file $2/$file
fi
done
}
if [ ! -d $1 ]; then
echo -e "\nERROR: $1 does not exist\n"
exit
elif [ -d $2 ] && [ ! -w $2 ]; then
echo -e "\nERROR: You do not have permission to write to $2\n"
exit
elif [ ! -d $2 ]; then
echo -e "\nSUCCESS: $2 has been created"
mkdir $2
fi
recursiveDuplication $1 $2
There are two issues with this solution:
As Rany Albeg Wein explained below, using 'ls' is not a good solution - and I have seen why when there is a space in a directory/*.png name.
I am also attempting to copy any *.png file from the source to the target and convert it to a *.jpg image in the target. How can I do this? I am attempting to use ImageMagick's convert image.png image.jpg command, but do not know how to do so when the image.png is being referred to as $file?
you can simplify a lot
$ find a -type d | xargs -I d mkdir -p copy/d
copy the tree structure from directory a to new directory under copy
$ tree a
a
|-- b
| |-- c
| | `-- file4
| |-- d
| | `-- file4
| `-- file3
`-- file2
3 directories, 4 files
$ tree copy
copy
`-- a
`-- b
|-- c
`-- d
4 directories, 0 files
This solution has been tested with directory names containing special chars, a loop in the directory tree and a broken symbolic link.
There is bellow a stansard solution:
( cd source/ ; find . -depth -type d -printf "%p\0" | xargs -0 -I xxx mkdir -p ../destination/xxx )
And there is a second solution using gnu cpio:
( cd source/ ; find . -depth -type d -printf "%p\0" | cpio --null -pd ../destination )
The test:
$ mkdir -p source/a/ba/c/d/e
$ mkdir -p source/a/bb/c/d/e
$ mkdir -p source/a/b/ca/d/e
$ ln -s source/broken/link source/a/ba/c/d/e/broken
$ (cd source/a/ba && ln -s ../../a tree-loop)
$ mkdir -p source/z/"$(printf "\1\2\3\4\5\6\7\10\11\12\13\14\15\16\17\20\21\22\23\24\25\26\27\30\31\32\33\34\35\36\37\40\41\42\43\44\45\46\47\72\73\74\75\76\77\100\133\134\135\1336\137\140\173\174\175\176\177dir")"
$ touch source/a/b/"$(printf "\1\2\3\4\5\6\7\10\11\12\13\14\15\16\17\20\21\22\23\24\25\26\27\30\31\32\33\34\35\36\37\40\41\42\43\44\45\46\47\72\73\74\75\76\77\100\133\134\135\1336\137\140\173\174\175\176\177file")"
$ ls
source
$ find source -depth
source/z/??????????????????????????????? !"#$%&':;<=>?#[\][6_`{|}~?dir
source/z
source/a/ba/tree-loop
source/a/ba/c/d/e/broken
source/a/ba/c/d/e
source/a/ba/c/d
source/a/ba/c
source/a/ba
source/a/b/ca/d/e
source/a/b/ca/d
source/a/b/ca
source/a/b/??????????????????????????????? !"#$%&':;<=>?#[\][6_`{|}~?file
source/a/b
source/a/bb/c/d/e
source/a/bb/c/d
source/a/bb/c
source/a/bb
source/a
source
$ ( cd source/ ; find . -depth -type d -printf "%p\0" | xargs -0 -I xxx mkdir -p ../destination/xxx )
$ find destination/ -depth
destination/z/??????????????????????????????? !"#$%&':;<=>?#[\][6_`{|}~?dir
destination/z
destination/a/ba/c/d/e
destination/a/ba/c/d
destination/a/ba/c
destination/a/ba
destination/a/b/ca/d/e
destination/a/b/ca/d
destination/a/b/ca
destination/a/b
destination/a/bb/c/d/e
destination/a/bb/c/d
destination/a/bb/c
destination/a/bb
destination/a
destination/
The gnu cpio test:
$ rm -rf destination
$ ( cd source/ ; find . -depth -type d -printf "%p\0" | cpio --null -pd ../destination )
0 blocks
$ find destination -depth
destination/z/??????????????????????????????? !"#$%&':;<=>?#[\][6_`{|}~?dir
destination/z
destination/a/ba/c/d/e
destination/a/ba/c/d
destination/a/ba/c
destination/a/ba
destination/a/b/ca/d/e
destination/a/b/ca/d
destination/a/b/ca
destination/a/b
destination/a/bb/c/d/e
destination/a/bb/c/d
destination/a/bb/c
destination/a/bb
destination/a
destination
#!/bin/bash
# 1st argument - source dir, 2nd - destination
function rrr {
for i in `ls $1`
do
if [ -d $1/$i ]
then
mkdir $2/$i
rrr $1/$i $2/$i
fi
done
}
rrr $1 $2
I am trying to make the following if-statement
if (!(zfs list -t snapshot -o name -H | grep -q secure) && (echo $days | grep -q $day) )
so when the first command doesn't return anything and the second return true, it should be executed.
If I try
if ! zfs list -t snapshot -o name -H | grep -q secure && echo $days | grep -q $day; then
echo ok
fi
then I can't not make it fail. It always prints ok.
Question
Can anyone see what I am doing wrong?
The only way how I was able to make it output ok was zfs reporting secure and $days containing $day. Any other combination (i.e. zfs not reporting secure or $daysnot containing$day`) produced no output.
As I do not have zfs, I defined a function:
function zfs () { echo secure ; } ; days=1; day=1; if ! zfs list -t snapshot -o name -H | grep -q secure && echo $days | grep -q "$day" ; then echo ok; fi
function zfs () { echo securX ; } ; days=1; day=1; if ! zfs list -t snapshot -o name -H | grep -q secure && echo $days | grep -q "$day" ; then echo ok; fi
function zfs () { echo securX ; } ; days=0; day=1; if ! zfs list -t snapshot -o name -H | grep -q secure && echo $days | grep -q "$day" ; then echo ok; fi
function zfs () { echo secure ; } ; days=0; day=1;if ! zfs list -t snapshot -o name -H | grep -q secure && echo $days | grep -q "$day" ; then echo ok; fi
function zfs () { echo secure ; } ; days=0; day=1; if ( ! ( zfs list -t snapshot -o name -H | grep -q secure) && (echo $days | grep -q "$day" ) ); then echo ok; fi
function zfs () { echo securX ; } ; days=0; day=1; if ( ! ( zfs list -t snapshot -o name -H | grep -q secure) && (echo $days | grep -q "$day" ) ); then echo ok; fi
function zfs () { echo securX ; } ; days=1; day=1; if ( ! ( zfs list -t snapshot -o name -H | grep -q secure) && (echo $days | grep -q "$day" ) ); then echo ok; fi
function zfs () { echo secure ; } ; days=1; day=1; if ( ! ( zfs list -t snapshot -o name -H | grep -q secure) && (echo $days | grep -q "$day" ) ); then echo ok; fi
This is a workaround rather than an answer, but you could reverse the sense of the test with grep itself using -v:
if zfs list -t snapshot -o name -H | grep -qv secure && echo $days | grep -q $day; then
echo ok
fi
although this is not quite the same in the case where zfs list -t snapshot -o name -H returns nothing at all.
Below is a snippet from a larger script that exports a list of the subdirectories of a user-specified directory, and prompts the user before making directories with the same names in another user-specified directory.
COPY_DIR=${1:-/}
DEST_DIR=${2}
export DIRS="`ls --hide="*.*" -m ${COPY_DIR}`"
export DIRS="`echo $DIRS | sed "s/\,//g"`"
if [ \( -z "${DIRS}" -a "${1}" != "/" \) ]; then
echo -e "Error: Invalid Input: No Subdirectories To Output\n"&&exit
elif [ -z "${DEST_DIR}" ]; then
echo "${DIRS}"&&exit
else
echo "${DIRS}"
read -p "Create these subdirectories in ${DEST_DIR}?" ANS
if [ ${ANS} = "n|no|N|No|NO|nO" ]; then
exit
elif [ ${ANS} = "y|ye|yes|Y|Ye|Yes|YE|YES|yES|yeS|yEs|YeS" ]; then
if [ ${COPYDIR} = ${DEST_DIR} ]; then
echo "Error: Invalid Target: Source and Destination are the same"&&exit
fi
cd "${DEST_DIR}"
mkdir ${DIRS}
else
exit
fi
fi
However, the command ls --hide="*.*" -m ${COPY_DIR} also prints files in the list as well. Is there any way to reword this command so that it only prints out directories? I tried ls -d, but that doesn't work, either.
Any ideas?
You should never rely on the output of ls to provide filenames. See the following for reasons not to parse ls: http://mywiki.wooledge.org/ParsingLs
You can build a list of directories safely using GNU find's -print0 option and appending the results to an array.
dirs=() # create an empty array
while read -r -d $'\0' dir; do # read up to the next \0 and store the value in "dir"
dirs+=("$dir") # append the value in "dir" to the array
done < <(find "$COPY_DIR" -type d -maxdepth 1 -mindepth 1 ! -name '*.*') # find directories that do not match *.*
The -mindepth 1 prevents find from matching the $COPY_DIR itself.