Trying to figure out how to create files(.strm) with specific content using a file list.
Example:
List.txt(content)
American Gangster (2007).strm,http://xx.x.xxx.xxx:8000/movie/xxxx/xxx/2017
Mi movie trailer (2019).strm,http://xx.x.xxx.xxx:8000/movie/xxxx/xxx/123
Peter again (2020).strm,http://xx.x.xxx.xxx:8000/movie/xxxx/xxx/5684
etc.
I was able to create the .strm files using the below script:
#!/bin/bash
filename="path/list.txt"
while IFS= read -r line
do
# echo "$line"
touch "$line"
done < "$filename"`
but this leaves me with empty files, how can I read and append the content?
Desire output:
Filename: AmericanGangster(2007).strm
Inside the file: http://xx.x.xxx.xxx:8000/movie/xxxx/xxx/2017
Thanks
You need to put the contents of the file to the file names that you created. Try
filename=path/list.txt
while IFS= read -r line; do
if [[ -e $line ]]; then ##: The -e tests if the file does exists.
echo "${line##*,}" > "${line%,*}"
fi
done < "$filename"
Since there can't be a filename with a '/' in it the -e is not needed.
filename=path/list.txt
while IFS= read -r line; do
echo "${line#*,}" > "${line%,*}"
done < "$filename"
It can be done by setting IFS to IFS=,
while IFS=, read -r file contents; do
echo "$contents" > "$file"
done < "$filename"
... Or awk.
awk -F, '{print $2 > $1}' "$filename"
The , must not appear anywhere except where it is now, otherwise all solution will break.
Related
I have a list of directories in a text file, which I am looping through. I simply want to rename the final part of the directory to a standardized name (this is the part of the directory after the final forwardslash). Is there a simple way to identify the final part of a file name and rename it in a loop?
For example, the directories are ...
/data/images/projects/MM/1/scans/16/7__ep2d_fid_basic_bold_293_Resting_State
/data/images/projects/MM/1/scans/20/7__ep2d_fid_basic_bold_293_Resting_State
/data/images/projects/MM/1/scans/03/8
I want to change them to...
/data/images/projects/MM/1/scans/16/rs
/data/images/projects/MM/1/scans/20/rs
/data/images/projects/MM/1/scans/03/rs
I can't figure out a way to do this, although it should be simple. Sorry, a newbie here.
I wrote a small script that will do the job for you.
It will read the content of an input file, remove everything after last slash + slash sign, and append /rs at the end, and then write the changes to a new file (called output.txt).
Create a script.sh and chmod +x it and add the following to it,
#!/bin/bash
# Usage: ./script.sh input.txt, where input.txt is your input file with the
# raw data you want to change.
while IFS='' read -r line
do
name=$line
echo "Before: $name"
name=`echo $name | awk -F '/' '{ for(i=1; i<=8; i++) {print $i} }' | tr '\n' '/'|sed 's/.$//'`
echo "$name/rs" >> output.txt
done < "$1"
# EOF
Run it by typing ./script.sh input.txt in the console.
Note: This is one way of doing it and it probably fits your current specific problem. It does not update the input file. Rather, it creates a new file (output.txt) with the new content.
while IFS='' read -r line
do
name=$line
echo "Before: $name"
name=`echo $name | awk -F '/' '{ for(i=1; i<=10; i++) {print $i} }' | tr '\n' '/'|sed 's/.$//'`
echo "$name/rsfMRI" > /sdata/images/projects/ASD_MM/1/datafiles/output.txt
for y in $(cat /sdata/images/projects/ASD_MM/1/datafiles/output.txt); do
mv $line $y
done
done < "$1"
I have written the code for replacing content but quite confused with replacing file name, both in the same script. I want my script to run this way:
suppose I give input : ./myscript.sh abc xyz
abc is my string to be replaced
xyz is the string to replace with
If some directory or any subdirectory or file in it has name abc, that should also change to xyz.
Here is my code:
filepath="/misc/home3/babitasandooja/fol1"
for file in $(grep -lR $1 $filepath)
do
sed -i "s/$1/$2/g" $file
echo "Modified: " $file
done
Now How should I code to replace the filename as well.
I tried:
if( *$1*==$file )
then
rename $1 $2 $filepath
fi
and
find -iname $filepath -exec mv $1 $2 \;
but any of then is not working correctly. What should I do? Which approach should I take?
Any help would be appreciated.
Thanks :)
#!/bin/bash
dir=$1 str=$2 rep=$3
while IFS= read -rd '' file; do
sed -i "s/$str/$rep/g" -- "$file"
base=${file##*/} dir=${file%/*}
[[ $base == *"$str"* ]] && mv "$file" "$dir/${base//$str/$rep}"
done < <(exec grep -ZFlR "$str" "$dir")
Usage:
bash script.sh dir string replacement
Note: rename would also rename the directory part.
grep -Z makes it produce null-delimited outputs. i.e. it produces output which is composed of filenames in which everything is separated by 0x00.
-d '' makes read read input delimited by 0x00; -r prevents
backslashes to be interpreted; and IFS= prevents word splitting with IFS.
IFS= read -rd '' makes read read input with delimiter of `0
base=${file##*/} removes the directory part. It's the same as base=$(basename "$file")
dir=${file%/*} removes the file part.
[[ $base == *"$str"* ]] checks if the filename has something that could be renamed. && makes the command that follows it execute if the previous returns zero (true) code. Think of it as a single link if statement.
"$dir/${base//$str/$rep}" forms the new filename. ${base//$str/$rep} replaces anything in $base that matches value of $str and replace it with value of $rep.
I have a txt file that has the path to xml files.now i want to read the path from the text file and print the number of tabs present in each xml file.how to do this?
here is what i have done
txt file with path
/home/user/Desktop/softwares/firefox/searchplugins/bing.xml
/home/user/Desktop/softwares/firefox/searchplugins/eBay.xml
/home/user/Desktop/softwares/firefox/searchplugins/answers.xml
/home/user/Desktop/softwares/firefox/searchplugins/wikipedia.xml
/home/user/Desktop/softwares/firefox/blocklist.xml
code to count tabs in each file
code:
#!/bin/sh
#
FILEPATH=/home/user/Desktop/softwares/firefox/*.xml
for file in $FILEPATH; do
tabs=$(tr -cd '\t' < $file | wc -c);
echo "$tabs tabs in file $file" >> /home/user/Desktop/output.txt
done
echo "Done!"
Where /home/user/Desktop/files.txt contains the list of xml files:
#!/bin/bash
while IFS= read file
do
if [ -f "$file" ]; then
tabs=$(tr -cd '\t' < "$file" | wc -c);
echo "$tabs tabs in file $file" >> "/home/user/Desktop/output.txt"
fi
done < "/home/user/Desktop/files.txt"
echo "Done!"
sudo_O has provided an excellent answer. However, there are chances that somehow, mostly due to text editor's preferences, your tabs were converted to 8 consecutive space. If you would prefer to count them as tabs too then replace the "tabs" definition as:
tabs=$(cat test.xml | sed -e 's/ \{8\}/\t/g' | tr -cd '\t' | wc -c)
Full code:
#!/bin/sh
# original file names might contain spaces
# FILEPATH=/home/user/Desktop/softwares/firefox/*.xml
# a better option would be
FIREFOX_DIR="/home/user/Desktop/softwares/firefox/"
while read file
do
if [[ -f "$file" ]]
then
tabs=$(cat test.xml | sed -e 's/ \{8\}/\t/g' | tr -cd '\t' | wc -c)
echo "$tabs tabs in file $file" >> /home/user/Desktop/output.txt
fi
done < $FIREFOX_DIR/*.xml
echo "Done!"
but this is applicable only if you prefer to count 8 consecutive spaces as tabs.
I'm obviously missing something simply, and know the problem is that it's creating a blank output which is why it can't compare. However if someone could shed some light on this it would be great - I haven't isolated it.
Ultimately, I'm trying to compare the md5sum from a list stored in a txt file, to that stored on the server. If errors, I need it to report that. Here's the output:
root#vps [~/testinggrounds]# cat md5.txt | while read a b; do
> md5sum "$b" | read c d
> if [ "$a" != "$c" ] ; then
> echo "md5 of file $b does not match"
> fi
> done
md5 of file file1 does not match
md5 of file file2 does not match
root#vps [~/testinggrounds]# md5sum file*
2a53da1a6fbfc0bafdd96b0a2ea29515 file1
bcb35cddc47f3df844ff26e9e2167c96 file2
root#vps [~/testinggrounds]# cat md5.txt
2a53da1a6fbfc0bafdd96b0a2ea29515 file1
bcb35cddc47f3df844ff26e9e2167c96 file2
Not directly answering your question, but md5sum(1):
-c, --check
read MD5 sums from the FILEs and check them
Like:
$ ls
1.txt 2.txt md5.txt
$ cat md5.txt
d3b07384d113edec49eaa6238ad5ff00 1.txt
c157a79031e1c40f85931829bc5fc552 2.txt
$ md5sum -c md5.txt
1.txt: OK
2.txt: OK
The problem that you are having is that your inner read is executed in a subshell. In bash, a subshell is created when you pipe a command. Once the subshell exits, the variables $c and $d are gone. You can use process substitution to avoid the subshell:
while read -r -u3 sum filename; do
read -r cursum _ < <(md5sum "$filename")
if [[ $sum != $cursum ]]; then
printf 'md5 of file %s does not match\n' "$filename"
fi
done 3<md5.txt
The redirection 3<md5.txt causes the file to be opened as file descriptor 3. The -u 3 option to read causes it to read from that file descriptor. The inner read still reads from stdin.
I'm not going to argue. I simply try to avoid double read from inside loops.
#! /bin/bash
cat md5.txt | while read sum file
do
prev_sum=$(md5sum $file | awk '{print $1}')
if [ "$sum" != "$prev_sum" ]
then
echo "md5 of file $file does not match"
else
echo "$file is fine"
fi
done
I want to be able to prepend a string to the beginning of each text file in a folder. How can I do this using bash on Linux?
This will do that. You could make it more efficient if you are doing the same text to each file...
for f in *; do
echo "whatever" > tmpfile
cat $f >> tmpfile
mv tmpfile $f
done
You can do it like this without a loop and cat
sed -i '1i whatever' *
if you want to back up your files, use -i.bak
Or using awk
awk 'FNR==1{$0="whatever\n"$0;}{print $0>FILENAME}' *
And you can do this using sed in 1 single command as well
for f in *; do
sed -i.bak '1i\
foo-bar
' ${f}
done
This should do the trick.
FOLDER='path/to/your/folder'
TEXT='Text to prepend'
cd $FOLDER
for i in `ls -1 $FOLDER`; do
CONTENTS=`cat $i`
echo $TEXT > $i # use echo -n if you want the append to be on the same line
echo $CONTENTS >> $i
done
I wouldn't recommending doing this if your files are very big though.
You can do this as well:
for f in *; do
cat <(echo "someline") $f > tempfile
mv tempfile $f
done
It's not much different from the 1st post but does show how to treat the output of the 'echo' statement as a file without having to create a temporay file to store the value.
You may use the ed command to do without temporary files if you like:
for file in *; do
(test ! -f "${file}" || test ! -w "${file}") && continue # sort out non-files and non-writable files
if test -s "${file}" && ! grep -Iqs '.*' "${file}"; then continue; fi # sort out binary files
printf '\n%s\n\n' "FILE: ${file}"
# cf. http://wiki.bash-hackers.org/howto/edit-ed
printf '%s\n' H 0a "foobar" . ',p' q | ed -s "${file}" # dry run (just prints to stdout)
#printf '%s\n' H 0a "foobar" . wq | ed -s "${file}" # in-place file edit without any backup
done | less
This is the easiest I have worked out.
sed -i '1s/^/Text to add then new file\n/' /file/to/change
Here is an example :
for f in *;
do
mv "$f" "whatever_$f"
done
A one-liner: rename '' string_ *