I want to add sequential number for each file and its contents in a directory. The sequential number should be prefixed with the filename and for each line of its contents should have the same number prefixed. In this manner, the sequential numbers should be generated for all the files(for names and its contents) in the sub-folders of the directory.
I have tried using maxdepth, rename, print function as a part. but it throws error saying that "-maxdepth" - not a valid option.
I have already a part of code(to print the names and contents of text files in a directory) and this logic should be appended with it.
#!bin/bash
cd home/TESTING
for file in home/TESTING;
do
find home/TESTING/ -type f -name *.txt -exec basename {} ';' -exec cat {} \;
done
P.s - print, rename, maxdepth are not working
If the name of the first file is File1.txt and its contents is mentioned as "Louis" then the output for the filename should be 1File1.txt and the content should be as "1Louis".The same should be replaced with 2 for second file. In this manner, it has to traverse through all the subfolders in the directory and print accordingly. I have already a part of code and this logic should be appended with it.
There should be fail safe if you execute cd in a script. You can execute command in wrong directory if you don't.
In your attempt, the output would be the same even without the for cycle, as for file in home/TESTING only pass home/TESTING as argument to for so it only run once. In case of
for file in home/TESTING/* this would happen else how.
I used find without --maxdepth, so it will look into all subdirectory as well for *.txt files. If you want only the current directory $(find /home/TESTING/* -type f -name "*.txt") could be replaced to $(ls *.txt) as long you do not have directory that end to .txt there will be no problem.
#!/bin/bash
# try cd to directory, do things upon success.
if cd /home/TESTING ;then
# set sequence number
let "x = 1"
# pass every file to for that find matching, sub directories will be also as there is no maxdeapth.
for file in $(find /home/TESTING/* -type f -name "*.txt") ; do
# print sequence number, and base file name, processed by variable substitution.
# basename can be used as well but this is bash built in.
echo "${x}${file##*/}"
# print file content, and put sequence number before each line with stream editor.
sed 's#^#'"${x}"'#g' ${file}
# increase sequence number with one.
let "x++"
done
# unset sequence number
unset 'x'
else
# print error on stderr
echo 'cd to /home/TESTING directory is failed' >&2
fi
Variable Substitution:
There is more i only picked this 4 for now as they similar.
${var#pattern} - Use value of var after removing text that match pattern from the left
${var##pattern} - Same as above but remove the longest matching piece instead the shortest
${var%pattern} - Use value of var after removing text that match pattern from the right
${var%%pattern} - Same as above but remove the longest matching piece instead the shortest
So ${file##*/} will take the variable of $file and drop every caracter * before the last ## slash /. The $file variable value not get modified by this, so it still contain the path and filename.
sed 's#^#'"${x}"'#g' ${file} sed is a stream editor, there is whole books about its usage, for this particular one. It usually placed into single quote, so 's#^#1#g' will add 1 the beginning of every line in a file.s is substitution, ^ is the beginning of the file, 1 is a text, g is global if you not put there the g only first mach will be affected.
# is separator it can be else as well, like / for example. I brake single quote to let variable be used and reopened the single quote.
If you like to replace a text, .txt to .php, you can use sed 's#\.txt#\.php#g' file , . have special meaning, it can replace any singe character, so it need to be escaped \, to use it as a text. else not only file.txt will be matched but file1txt as well.
It can be piped , you not need to specify file name in that case, else you have to provide at least one filename in our case it was the ${file} variable that contain the filename. As i mentioned variable substitution is not modify variable value so its still contain the filename with path.
Related
I would like to find some file types in pictures folder and I have created the following bash-script in /home/user/pictures folder:
for i in *.pdf *.sh *.txt;
do
echo 'all file types with extension' $i;
find /home/user/pictures -type f -iname $i;
done
But when I execute the bash-script, it does not work as expected for files that are located on the base directory /home/user/pictures. Instead of echo 'All File types with Extension *.sh' the command interprets the variable for base directory:
all file types with extension file1.sh
/home/user/pictures/file1.sh
all file types with extension file2.sh
/home/user/pictures/file2.sh
all file types with extension file3.sh
/home/user/pictures/file3.sh
I would like to know why echo - command does not print "All File types with Extension *.sh".
Revised code:
for i in '*.pdf' '*.sh' '*.txt'
do
echo "all file types with extension $i"
find /home/user/pictures -type f -iname "$i"
done
Explanation:
In bash, a string containing *, or a variable which expands to such a string, may be expanded as a glob pattern unless that string is protected from glob expansion by putting it inside quotes (although if the glob pattern does not match any files, then the original glob pattern will remain after attempted expansion).
In this case, it is not wanted for the glob expansion to happen - the string containing the * needs to be passed as a literal to each of the echo and the find commands. So the $i should be enclosed in double quotes - these will allow the variable expansion from $i, but the subsequent wildcard expansion will not occur. (If single quotes, i.e. '$i' were used instead, then a literal $i would be passed to echo and to find, which is not wanted either.)
In addition to this, the initial for line needs to use quotes to protect against wildcard expansion in the event that any files matching any of the glob patterns exist in the current directory. Here, it does not matter whether single or double quotes are used.
Separately, the revised code here also removes some unnecessary semicolons. Semicolons in bash are a command separator and are not needed merely to terminate a statement (as in C etc).
Observed behaviour with original code
What seems to be happening here is that one of the patterns used in the initial for statement is matching files in the current directory (specifically the *.sh is matching file1.sh file2.sh, and file3.sh). It is therefore being replaced by a list of these filenames (file1.sh file2.sh file3.sh) in the expression, and the for statement will iterate over these values. (Note that the current directory might not be the same as either where the script is located or the top level directory used for the find.)
It would also still be expected that the *.pdf and *.txt would be used in the expression -- either substituted or not, depending on whether any matches are found. Therefore the output shown in the question is probably not the whole output of the script.
Such expressions (*.blabla) changes the value of $i in the loop. Here is the trick i would do :
for i in pdf sh txt;
do
echo 'all file types with extension *.'$i;
find /home/user/pictures -type f -iname '*.'$i;
done
Let's say I have a folder like this:
my_folder
====my_sub_folder_1
====my_sub_folder_2
====my_sub_folder_3
====exclude
I would like a command that return a string like this :
["my_sub_folder_1", "my_dub_folder_2", "my_dub_folder_3"]
(Notice the exclusion of the excude folder)
The best I could is :
ls -dxm */
That return the following.
my_sub_folder_1/, my_dub_folder_2/, my_dub_folder_3/
So I'm still trying to remove the / at the end of each folder, add the [] and the "".
If it's possible I would like to do that in one line so I could diretly put in a shell variable, other wise I will put it in .sh file that will return the string I'm trying to build.
(I don't know if the last part is really possible).
Assuming you are executing the script in the directory where my_folder
belongs, how about:
while IFS= read -r -d "" f; do
ary+=("$f")
done < <(find "my_folder" -maxdepth 1 -mindepth 1 -type d -not -name "exclude" -printf "\"%f\"\0")
(IFS=","; echo "[${ary[*]}]")
[Explanations]
-printf option to find command specifies the output format. The format "\"%f\"\0"
prints the filename (excluding leading directory name) wrapped by
double quotes and followed by a NUL character \0.
The NUL character is used as a filename delimiter and the filenames
are split again in the read builtin by specifying the delimiter
to the NUL character with -d "".
Then the filenames (with double quotes) are stored in the array ary
one by one.
Finally echo "[${ary[*]}]" command prints out the elements of ary
separated by IFS. The whole output are surrounded by the square brackets [].
The last line is surrounded by parens () to be executed in the subprocess.
The purpose is just not to overwrite the current IFS.
If you save the script in my answer as my_script.sh, then you can assign
a variable MY_VAR to the output by saying:
MY_VAR=$(./my_script.sh)
echo "$MY_VAR"
# or another_command "$MY_VAR" or whatever
Alternatively you can assign the variable within the script by modifying
the last line as:
MY_VAR=$(IFS=","; echo "[${ary[*]}]")
echo "$MY_VAR"
Hope this helps.
In bash this can be done as follows, it's close but it doesn't work in one line.
Change the Internal Field Separator to be a new line rather than a space. This allows spaces in directory names to be ignored.
Then perform the following:
List the directories, one per line
Use grep to remove the directory to be excluded
Iterate over the results:
Output the directory name with the last character removed
Pipe everything to xargs to recombine into a single line and store in $var
Trim the last , from ${var} and wrap in '[]'
IFS=$'\n'
var=`for d in \`ls -d1 */ | grep -v exclude_dir \`; do echo '\"'${d::-1}'\",' ; done | xargs`
echo '['${var::-1}']'
I am having a file named temp.txt where inside this file it contains the following content
https://abcdef/12345-xyz
https://ghifdfg/5426525-abc
I need to create a directories automatically in linux by using only th number part from each line in the file.
So the output should be something like 12345 and 5426525 directories created.
Any approach on how to do this could be helpful.
This is the code that i searched and got from internet,wherein this code, new directories will be created by the file name that starts with BR and W0 .
for file in {BR,W0}*.*; do
dir=${file%%.*}
mkdir -p "$dir"
mv "$file" "$dir"
done
Assuming each URL is of the form
http[s]://any/symbols/some_digits-some_letters
Then you indeed could use the simple prefix and suffix modifiers in shell variable expansion.
${x##*/} expands to the suffix part of x that starts after the last slash /.
${y%%-*} expands to the prefix part of y before the first -.
while read x ; do
y=${x##*/}
z=${y%%-*}
mkdir $z
done < temp.txt
What I want to do here is locate any file that contains a specific string in a specific line, and remove said line, not just the string.
What I have is something along the lines of this:
find / -type f -name '*.foo' -exec sed '1/stringtodetect/d' {} \;
However this will remove everything BETWEEN line 1 and the string. given that sed argument. (sed '1,/stringtodetect/d' "$file")
Lets say I have a .php file, and I'm looking for the string 'gotcha'.
I only want to edit the file if it has the string in the FIRST line of the file, like so:
gotcha with this.
gotcha
useful text
more text
dont delete me
If I ran the script, I'd want the contents of the same file to appear as such:
List item
List item
dont delete me
Any tips?
You are using the following range address for the delete command:
1,/stringtodelete/
This means all lines from line 1 until the first occurrence of stringtodelete.
Furthermore, you need not (and should not!) iterate over the results from find. find has the -exec option for that. It executes a command for each file which has been found, passing the filename as an argument.
It should be:
find / -type f -name '*.foo' -exec sed '/stringtodetect/d' {} \;
Test the command first. Once you are sure it works, use sed -i to modify the files in place. If you want a backup you can use sed -i.backup (for example). To remove the backups once you are sure you can use find again:
find / -type -name '*.foo.backup' -delete
You need a sed script that will skip any line by number that is not the one you are interested in, and only for the line you are interested in delete the line if it matches.
sed -e1bt -eb -e:t -e/string/d < $file
-e1bt = for line 1, branch to label "t"
-eb = branch unconditionally to the end of the script (at which point it will print the line).
-e:t = define label "t"
-e/string/d = delete the line if it contains "string" - this instruction will only be reached if the unconditional branch to the end of the script was NOT taken, i.e. if the line number branch WAS taken.
Could it be that it is matching parts of a string.
If you try exact match, it might help.
Also, remove the 1, at the beginning or replace it with 0,
sed '/<stringtodetect>/d' "$file";
sed is for simple substitutions on individual lines, that is all. For anything else just use awk for simplicity, clarity, robustness, portability and all of the other desirable attributes of software:
awk '!(NR==1 && /stringtodetect/)' file
You were close. I think what you're looking for is: sed '1{/gotcha/d;}'
I need some help.....
In my .bashrc file I have a VERY useful function (It may be a bit rough and ready, and a bit hacky, but it works a treat!) that reads an input file, and uses the 'tree' function on each of the input lines to create a directory tree. this tree is then printed into an output file (along with the size of the folder).
multitree()
{
while read cheese
do
pushd . > /dev/null
pushd $cheese > /dev/null
echo -e "$cheese \n\n" >> ~/Desktop/$2.txt
tree -idf . >> ~/Desktop/$2.txt
echo -e "\n\n\n" >> ~/Desktop/$2.txt
du -sh --si >> ~/Desktop/$2.txt
echo -e "\n\n\n\n\n\n\n" >> ~/Desktop/$2.txt
popd > /dev/null
done < $1
cat ~/done
}
This is a time saver like no end, and outputs a snippet like the following:
./foo
./foo/bar
./foo/bar/1
./foo/bar/1/2
etc etc....
however, the first (and most tedious) thing I need to do is remove all entries leaving only the deepest folder path (Using the above example it would be reduced to just ./foo/bar/1/2)
Is there a way of processing the file before/after the tree function to only print the deepest levels?
I know something like python might do a better job, but my issue is I've never used python And I'm not sure the work systems would let me run python... they let us modify our own .bashrc so I'm not too worried!
Thanks in advance guys!!!!
Owen.
You could use
find . -type d -links 2
Replace . with a directory if desired.
EDIT: Explanation:
find searches a directory for files that match a given filter. In this case, the directory is ., and the filter is -type d -links 2.
-type d filters for directories
-links 2 filters for those that have two (hard) links to their name. Effectively, this filters for all directories that have no subdirectories, because only those have two: The one in their parent directory and the . link in themselves. Those with subdirectories also have the .. links in their subdirectories.
Here's a hint:
You just need to count the number of "/" characters in each line.
If the current line has fewer than the number of "/" characters in the preceding line, the preceding line would be the "deepest" directory in its part of the hierarchy.
This line, and any subsequent line with still fewer "/" characters would NOT be the deepest directory in its part of the entire directory hierarchy. As soon as you get a line with the same number of "/" characters, or greater, then you can "reset" and, once again, keep an eye out for the first line with the fewer number of "/" characters.
And, finally, you need to handle the trivial case: only one line in your tree output, the current directory has no subdirectories, so it wins by default.
Another way you can implement this is by considering the following statement:
If a directory's name also exists as an exact prefix of another directory in the list, followed by the "/" character, then it is NOT the deepest directory in its part of the hierarchy.