due to migrating of batch job to Linux server I have problem finding the equivalent of the following commands in Linux:
Y drive is a map drive to the NAS drive which is also connected to Ubuntu server /NAS/CCTV . Need to search every sub folders for all .264 files
Z drive is on the Ubuntu server itself. Just move every .mp4 files here, no folder here. Path on Ubuntu is /Share/CCTV/
Its just a simple script to convert the cctv capture .264 format to mp4 and move to server to be process and delete off any h264 files and any folder thats older than 1 day, the script will schedule to run every 3 mins.
I have ffmpeg installed on the Ubuntu server, just unable to find the for each file in the folders to do the same.
Also for the last for files command that delete folder older than 1 days
FOR /r y:\ %%F in (*.h264) do c:\scripts\ffmpeg -i %%F %%F.mp4
FOR /r y:\ %%F in (*.h264) do del %%F
FOR /r y:\ %%G in (*.mp4) do move %%G Z:\
forfiles -p "Y:\" -d -1 -c "cmd /c IF #isdir == TRUE rd /S /Q #path"
Appreciate any forms of help or point me to the right guide so I can rewrite it on the Linux server. I did try to search for for loop but all show me to count number, maybe I search wrongly.
Find all .h264 files (recursively)
find /NAS/CCTV -type f -name '*.h264'
Convert all such files to .mp4
while IFS= read -d '' -r file ; do
ffmpeg -i "$file" "$file".mp4
done < <(find /NAS/CCTV -type f -name '*.h264' -print0)
Note that this will create files called like filename.h264.mp4. This matches your batch file behavior. If you would prefer to replace the extension use ffmpeg -i "$file" "${file%.*}".mp4 instead and you will get a name like filename.h264.
Also move those mp4 files to another directory
while IFS= read -d '' -r file ; do
ffmpeg -i "$file" "$file".mp4
if [[ -f $file.mp4 ]] ; then
mv -f -- "$file".mp4 /Share/CCTV
fi
done < <(find /NAS/CCTV -type f -name '*.h264' -print0)
Delete old directories (recursively)
find /NAS/CCTV -type d -not -newermt '1 day ago' -exec rm -rf {} +
Documentation.
The find command recursively lists files according to criteria you specify. Any time you need to deal with files in multiple directories or very large numbers of files it is probably what you want to use. For safety against malicious file names it's important to -print0 so file names are delimited by null rather than newline, which then requires using the IFS= read -d '' construct to interpret later.
The while read variable ; do ... done construct reads data from input and assigns each record to the named variable. This allows each matching file to be handled one at a time inside the loop. The insides of the loop should be fairly obvious.
Again find is used to select files, but in this case the files are directories. The switches -not -newer select files which are not newer (in other words, files which are older) according to their m time, the modification time, compared against t, which in this case means that the next argument is text describing a time. Here you can use any expression understood by GNU date's -d switch, so I can write in plain English and it will work as expected.
As you embark on your shell scripting journey you should keep two things by your side:
shellcheck - Always runs scripts you write through shellcheck to catch basic errors.
Bash FAQ - The bash FAQ at wooledge.org. Most of the answers to questions you have not thought of yet will be here. For example FAQ 15 is highly relevant to this question.
for f in /NAS/CCTV/*.h264; do ffmpeg -i "$f" "$f".mp4; done
rm /NAS/CCTV/*.h264
mv /NAS/CCTV/*.mp4 /Share/CCTV
find /NAS/CCTV/ -type d -ctime +1 -exec rm -rf {} \;
Related
So script2 is:
find /some_directory -type f -not -iname "*.pdf" -exec bash -c './script "{}"' \; -print > temp_file
while read line
do
mkdir -p result"$(dirname "$line")"
cp "$line" ~/result"$(dirname "$line")"/$(basename "$line" .txt).pdf
done < temp_file
rm temp_file
the ./script is file "$1"|grep -q PDF
These two combined should find .txt files that are actually PDF files and then copy them to some result directory and rename them to .pdf file. But the files should be in: result/their/original/directories/file.pdf. (if the original was some_directory/their/original/directories/file.txt, also some directories have space in them, thus a lot of "..")
It is done, it works, but the question is: how to do it without mkdir?
I've tried many things but none seem to work and every post i read it says that it can be done just by using mkdir (but because my professor demands it to be done without mkdir, I'm quite sure it can be done. I don't HAVE to do it, but I'd like to know how it can be done).
Maybe it can be done in find with piping? (some argument that would print directories first and they would be copied one by one and then files?) I've wasted a lot of time on this and it would be a shame to quit it without correct answear.
You can copy a file while creating the leading target directories with install:
install -D src/file dest/src/file
I have an IP camera that takes snapshots and nests those snapshots into multiple directories. The sub directories look something like this.
/cam_folder
|--Date
|----Hour
|------Minute
|-------->file1
|-------->file2...etc
|------Minute
|-------->file1...etc
There is a ton of sub directories because of the way it stores files since it places those snapshots within a Minute directory of the Date/Hour directories.
At any rate, there are other types of files mixed in, but I know how to use find to find all the .jpgs I need:
find /cam_folder/ -type f -name '*.jpg'
But what I need to do is rename all the .jpg files to random characters. I was able to find this which works from a single directory in a bash script:
for file in *.jpg; do
new_file="$(mktemp XXXXXXXX.jpg)"
mv -f -- "$file" "$new_file"
done
My problem is how to tie these together? I need to use find to feed these into a bash script I guess?
Is there an easier way to just walk a directory recursively renaming as I go?
find /cam_folder/ -type f -name '*.jpg' -exec sh -c '
for f; do
mv -f -- "$f" "${f%/*}/$(mktemp -u XXXXXXXX.jpg)"
done' _ {} +
find
Shell Command Language § The for Loop
Shell Command Language § Parameter Expansions
I think that meets your demand
while IFS= read -r file ; do
new_file="$(mktemp XXXXXXXX.jpg)"
mv -f -- "$file" "$new_file"
done < <(find /cam_folder/ -type f -name '*.jpg')
I'm writing a shell script on a Linux machine to be run via a crontab which is meant to move all files older than the current day to a new folder, and then tar and zip the entire folder. Seems like a simple task but for some reason, I'm running into all kinds of roadblocks. I'm new to this and self-taught so any help or redirection would be greatly appreciated.
Specific criteria for which files to archive:
All log files are in /home/tech/logs/ and all pdfs are in /home/tech/logs/pdf
All files are over a day old as indicated by the file name (file name does not include $CURRENT_DATE)
All files must be *.log or *.pdf (i.e. don't archive files that don't include $CURRENT_DATE if it isn't a log or pdf file.
Filename formatting specifics:
All the log file names are in home/tech/logs in the format NAME 00_20180510.log, and all the pdf files are in a "pdf" subdirectory (home/tech/logs/pdf) with the format NAME 00_20180510_00000000.pdf ("20180510" would be whenever the file was created and the 0's would be any number). I need to use the name rather than the file metadata for the creation date, and all files (pdf/log) whose name does not include the current date are "old". I also can't just move all files that don't contain $CURRENT_DATE in the name because it would take any non-*.pdf or *.log files with it.
Right now the script creates a new folder with a new pdf subdir for the old files (mkdir -p /home/tech/logs/$ARCHIVE_NAME/pdf). I then want to move the old logs into $ARCHIVE_NAME, and move all old pdfs from the original pdf subdirectory into $ARCHIVE_NAME/pdf.
Current code:
find /home/tech/logs -maxdepth 1 -name ( "*[^$CURRENT_DATE].log" "*.log" ) -exec mv -t "$ARCHIVE_NAME" '{}' ';'
find /home/tech/logs/pdf -maxdepth 1 -name ( "*[^$CURRENT_DATE]*.pdf" "*.pdf" ) -exec mv -t "$ARCHIVE_NAME/pdf" '{}' ';'
This hasn't been working because it treats the numbers in $CURRENT_DATE as a list of numbers to exclude rather than a literal string.
I've considered just using tar's exclude options like this:
tar -cvzPf "$ARCHIVE_NAME.tgz" --directory /home/tech/logs --exclude="$CURRENT_DATE" --no-unquote --recursion --remove-files --files-from="/home/tech/logs/"
But a) it doesn't work, and b) it would theoretically include all files that weren't *.pdf or *.log files, which would be a problem.
Am I overcomplicating this? Is there a better way to go about this?
I would go about this using bash's extended glob features, which allow you to negate a pattern:
#!/bin/bash
shopt -s extglob
mv /home/tech/logs/*!("$CURRENT_DATE")*.log "$ARCHIVE_NAME"
mv /home/tech/logs/pdf/*!("$CURRENT_DATE")*.pdf "$ARCHIVE_NAME"/pdf
With extglob enabled, !(pattern) expands to everything that doesn't match the pattern (or list of pipe-separated patterns).
Using find it should also be possible:
find /home/tech/logs -name '*.log' -not -name "*$CURRENT_DATE*" -exec mv -t "$ARCHIVE_NAME" {} +
Building on #tom-fenech answer, optimized to avoid many mv invocations:
find /home/tech/logs -maxdepth 1 -name '*.log' -not -name "*_${CURRENT_DATE?}.log" | \
xargs mv -t "${ARCHIVE_NAME?}"
An interesting feature, from processing the file thru pipes, is the ability to filter them with extra tools (aka grep :), which can (arguably) become more readable i.e. ->
find /home/tech/logs -maxdepth 1 -name '*.log' | fgrep -v "_${CURRENT_DATE?}" | \
xargs mv -t "${ARCHIVE_NAME?}"
Then similarly for the pdf ones, BTW you can "dry-run" above by just replacing mv by echo mv.
--jjo
I have multiple folders with multiple files. I need to rename those files with the same name like the folder where the file stored with "_partN" prefix.
As example,
I have a folder named as "new_folder_for_upload" which have 2 files. I need to convert the name of these 2 files like,
new_folder_for_upload_part1
new_folder_for_upload_part2
I have so many folders like above which have multiple files. I need to convert all the file names as I describe above.
Can anybody help me to find out for a single linux command or script to do this work automatically?
Assuming bash shell, and assuming you want the file numbering to restart for each subdirectory, and doing the moving of all files to the top directory (leaving empty subdirectories). Formatted as script for easier reading:
find . -type f -print0 | while IFS= read -r -d '' file
do
myfile=$(echo $file | sed "s#./##")
mydir=$(dirname "$myfile")
if [[ $mydir != $lastdir ]]
then
NR=1
fi
lastdir=${mydir}
mv "$myfile" "$(dirname "$myfile")_part${NR}"
((NR++))
done
Or as one-line command:
find . -type f -print0 | while IFS= read -r -d '' file; do myfile=$(echo $file | sed "s#./##"); mydir=$(dirname "$myfile"); if [[ $mydir != $lastdir ]]; then NR=1; fi; lastdir=${mydir}; mv "$myfile" "$(dirname "$myfile")_part${NR}"; ((NR++)); done
Beware. This is armed, and will do a bulk renaming / moving of every file in or below your current work directory. Use at your own risk.
To delete the empty subdirs:
find . -depth -empty -type d -delete
Does a program exist that will allow me to batch covert files and do the following:
a) Search for MP3's on my drive
b) transcode from 128 kbs/sec to 64 kbs/sec
c) switch from Stereo to Mono
d) save and overwrite the previous file
Or is their anyway I could write a script to perform this task on a windows desktop?
if you are on un*x, the tool of choice for batch-processing is find:
find /path/to/foo -name "*.mp3"
will give you all files matching "*.mp3" in /path/to/foo and all it's subdirectories.
sincefind is a beast, you probably want to check it's manpage.
you can pass a script to find that is called for each match. e.g. the following will do an "ls -l" on all files (excluding directories named e.g. "mysounds.mp3/"; that's what the -type f is for):
find /path/to/foo -type f -name "*.mp3" -exec ls -l \{\} \;
note, that the curly braces and the semicolon are escaped with backticks, in order to prevent the shell from interpreting those special characters.
if you have a script (named convertmp3.sh and which is sitting in your current working directory) that does an in-place conversion (overwriting the old file), you can do:
find /path/to/foo -type f -name "*.mp3" -exec ./convertmp3.sh \{\} \;
such a script could look like:
#!/bin/sh
INFILE="$1"
TMPFILE="${INFILE}.mp3"
ffmpeg -i "${INFILE}" -ac 1 -b 64k "${TMPFILE}" && mv "${TMPFILE}" "${INFILE}"
note that i'm using a temporary file here, because else ffmpeg would start overwriting the source file before it has fully read it, and thus stopping with an error.
also do not forget the quotes around the filenames, to protect against filenames with spaces and the like.
also note, that this script does not check whether the input file is already in the desired format, thus potentially re-encoding it.