I have over a thousand compressed files inside each other, named in descending order starting at 1022; and need to extract all of them. What I'm trying to do is to write a shell script that will do just that. Here's what I've got so far:
cd "PATH"
for file in PATH/*; do
filename=${PWD: -4}
extract $file
numb=$(echo $filename - 1 | bc)
echo $numb
cd "$numb"
done
The "extract" function extracts any format of compression.
The problem is the loop doesn't seem to go on inside the new directory. Can anyone help me out with this?
Related
Lets say i have multiple filesnames e.g. R014-20171109-1159.log.20171109_1159.
I want to create a shell script which creates for every given date a folder and moves the files matching the date to it.
Is this possible?
For the example a folder "20171109" should be created and has the file "R014-20171109-1159.log.20171109_1159" on it.
Thanks
This is a typical application of a for-loop in bash to iterate thru files.
At the same time, this solution utilizes GNU [ shell param substitution ].
for file in /path/to/files/*\.log\.*
do
foldername=${file#*-}
foldername=${foldername%%-*}
mkdir -p "${foldername}" # -p suppress errors if folder already exists
[ $? -eq 0 ] && mv "${file}" "${foldername}" # check last cmd status and move
done
Since you want to write a shell script, use commands. To get date, use cut cmd like ex:
cat 1.txt
R014-20171109-1159.log.20171109_1159
cat 1.txt | cut -d "-" -f2
Output
20171109
is your date and create folder. This way you can loop and create as many folders as you want
Its actually quite easy(my Bash syntax might be a bit off) -
for f in /path/to/your/files*; do
## Check if the glob gets expanded to existing files.
## If not, f here will be exactly the pattern above
## and the exists test will evaluate to false.
[ -e "$f" ] && echo $f > #grep the file name for "*.log."
#and extract 8 charecters after "*.log." .
#Next check if a folder exists already with the name of 8 charecters.
#If not { create}
#else just move the file to that folder path
break
done
Main idea is from this post link. Sorry for not providing the actual code as i havent worked anytime recently on Bash
Below commands can be put in script to achieve this,
Assign a variable with current date as below ( use --date='n day ago' option if need to have an older date).
if need to get it from File name itself, get files in a loop then use cut command to get the date string,
dirVar=$(date +%Y%m%d) --> for current day,
dirVar=$(date +%Y%m%d --date='1 day ago') --> for yesterday,
dirVar=$(echo $fileName | cut -c6-13) or
dirVar=$(echo $fileName | cut -d- -f2) --> to get from $fileName
Create directory with the variable value as below, (-p : create directory if doesn't exist.)
mkdir -p ${dirVar}
Move files to directory to the directory with below line,
mv *log.${dirVar}* ${dirVar}/
I want a script that is able to read the content of a text file which contains folder names and moves the folders from their directory to a specific folder. Here is my script:
#!/bin/bash
for i in $(cat /folder/collected/folders.txt)
do
mv /fromfilelocation/$i /folder/Collected/
done
This script is partly working as it copies only the last folder in the text file, as for the other folders it gives the error "not possible: data or directory not found" But the folder is there and according to the error the folder directory is correctly displayed.
What should I do in order to make it work correctly ??
You can use this:
#!/bin/bash
for sample in `awk '{print $1}' All_bins.txt`
do mv "$sample" All_Good_Bins
done
Use while loop instead
while read i; do
mv fromfilelocation/"$i" /folder/Collected/
done </folder/collected/folders.txt
I have 25 files in a directory. I need to amass 25000 files for testing purposes. I thought I could just replicate these files over and over until I get 25000 files. I could manually copy paste 1000 times but that seemed tedious. So I thought I could write a script to do it for me. I tried
cp * .
As a trial but I got an error that said the source and destination file are the same. If I were to automate it how would i do it so that each of the 1000 times the new files are made with unique names?
As discussed in the comments, you can do something like this:
for file in *
do
filename="${file%.*}" # get everything up to last dot
extension="${file##*.}" # get extension (text after last dot)
for i in {00001..10000}
do
cp $file ${filename}${i}${extension}
done
done
The trick for i in {00001..10000} is used to loop from 1 to 10000 having the number with leading zeros.
The ${filename}${i}${extension} is the same as $filename$i$extension but makes more clarity over what is a variable name and what is text. This way, you can also do ${filename}_${i}${extension} to get files like a_23.txt, etc.
In case your current files match a specific pattern, you can always do for file in a* (if they all are on the a + something format).
If you want to keep the extension of the files, you can use this. Assuming, you want to copy all txt-files:
#!/bin/bash
for f in *.txt
do
for i in {1..10000}
do
cp "$f" "${f%.*}_${i}.${f##*.}"
done
done
You could try this:
for file in *; do for i in {1..1000}; do cp $file $file-$i; done; done;
It will append a number to any existing files.
The next script
for file in *.*
do
eval $(sed 's/\(.*\)\.\([^\.]*\)$/base="\1";ext="\2";/' <<< "$file")
for n in {1..1000}
do
echo cp "$file" "$base-$n.$ext"
done
done
will:
take all files with extensions *.*
creates the basename and extension (sed)
in a cycle 1000 times copyes the original file to file-number.extension
it is for DRY-RUN, remove the echo if satisfied
I have a script that I created that is just a list of commands (cp's, mkdir's, rm's, etc). What this does is basically update the contents of a bunch of folders with the contents of a source folder.
Right now I have about 15 folders to be modified with about 30 commands each. So when I need to add a folder, I need to add 30 more commands and specify that folder.
Is there a way in the script to create an array of folders to change and loop through it or something?
My script right now just contains basic commands that would normally be run in the command line, so nothing advanced.
Yes, you can do something like this:
for x in "folder1" "folder2" "folder3"; do
mkdir $x
cp foobar $x
done
Better yet, use an array to hold the folder names, e.g.
arr=("folder1" "folder2" "folder3")
for x in ${arr[*]} do
mkdir $x
cp foobar $x
done
If you have specific names following a pattern, you can probably use a loop to generate that list of names automatically.
Here is one approach:
#!/bin/bash
# This function does all you clever stuff
# $1 contains the first parameter, $2 the second and so on
function my_cmds()
{
echo $1
}
# You can loop over folders like this
for i in folder1 folder2 folder3
do
# here we call a function with the string as parameter
my_cmds $i
done
# ... or like this
folders[0]="folder0"
folders[1]="folders1"
folders[2]="folders2"
for i in "${folders[#]}"
do
my_cmds $i
done
a convenient way of initializing an entire array is the
array=( element1 element2 ... elementN )
notation.
This is similar to answers using for loops, but using a here document to store the list of folders. It's like having a data file embedded in the script.
while read -r folder <&3; do
mkdir "$folder"
# etc
done 3<<EOF
folder1
folder2
folder with space
EOF
I use file descriptor 3 for the here document in case you have commands in the body of the loop that attempt to read from standard input.
I am currently trying to write a script that will loop through multiple directories. The main raw_data directory contains ~150 subdirectories (subj001, subj002,...,subj00n), each of which has several subdirectories.
How can I make sure that the output from the script given bellow will be sent back to the specific subdirectory (e.g. subj0012) the input was taken from, rather than the current directory (raw_data)?
#!/bin/bash
for dir in ~raw_data/*
do
tractor -d -r -b preproc RunStages:1
done
Thank you.
The name of the dir you want to save the output to is in $dir, right? So, just send the output there via redirection:
#!/bin/bash
for dir in ~raw_data/* ; do
tractor -d -r- b preproc RunStages:1 > $dir/output
done
You should make sure that what you are processing really is a directory, though.
You can use output of find to run a loop and append the output at desired location like this:
while read d
do
echo $d >> ~raw_data/subj0012/output
done < <(find ~raw_data -type d)