I have a csv file which imaging my inventory sort and uniq:
|Nivcomp Nivelliergerät
|Bosch Rotationslaser GRL 300 HV
|Renault T440
|Renault Master
|Spritzer Silo
...
The second CSV file contains the complete inventory. I would like to find and count the entries of uniq.csv in cominv.csv. So that I become a new csv file with:
|Renault T440|20
|Spritzer Silo |10
...
I try with while read line, cat and grep but it would not really work.
#!/usr/bin/env bash
file="cominv.csv"
newfile="uniq.csv"
if [[ -f $file ]]; then
cat $file | uniq > $newfile
fi
if [[ -f $newfile ]]; then
while read line
do
cat $file | grep $line | wc -l
done < $newfile
fi
Thank you for help
Silvio
Related
I have these files. Imagine that each "test" represent the name of one server:
test10.txt
test11.txt
test12.txt
test13.txt
test14.txt
test15.txt
test16.txt
test17.txt
test18.txt
test19.txt
test1.txt
test20.txt
test21.txt
test22.txt
test23.txt
test24.txt
test25.txt
test26.txt
test27.txt
test28.txt
test29.txt
test2.txt
test30.txt
test31.txt
test32.txt
test33.txt
test34.txt
test35.txt
test36.txt
test37.txt
test38.txt
test39.txt
test3.txt
test40.txt
test4.txt
test5.txt
test6.txt
test7.txt
test8.txt
test9.txt
In each txt file, I have this type of data:
2019-10-14-00-00;/dev/hd1;1024.00;136.37;/
2019-10-14-00-00;/dev/hd2;5248.00;4230.53;/usr
2019-10-14-00-00;/dev/hd3;2560.00;481.66;/var
2019-10-14-00-00;/dev/hd4;3584.00;67.65;/tmp
2019-10-14-00-00;/dev/hd5;256.00;26.13;/home
2019-10-14-00-00;/dev/hd1;1024.00;476.04;/opt
2019-10-14-00-00;/dev/hd5;384.00;0.38;/usr/xxx
2019-10-14-00-00;/dev/hd4;256.00;21.39;/xxx
2019-10-14-00-00;/dev/hd2;512.00;216.84;/opt
2019-10-14-00-00;/dev/hd3;128.00;21.46;/var/
2019-10-14-00-00;/dev/hd8;256.00;75.21;/usr/
2019-10-14-00-00;/dev/hd7;384.00;186.87;/var/
2019-10-14-00-00;/dev/hd6;256.00;0.63;/var/
2019-10-14-00-00;/dev/hd1;128.00;0.37;/admin
2019-10-14-00-00;/dev/hd4;256.00;179.14;/opt/
2019-10-14-00-00;/dev/hd3;2176.00;492.93;/opt/
2019-10-14-00-00;/dev/hd1;256.00;114.83;/opt/
2019-10-14-00-00;/dev/hd9;256.00;41.73;/var/
2019-10-14-00-00;/dev/hd1;3200.00;954.28;/var/
2019-10-14-00-00;/dev/hd10;256.00;0.93;/var/
2019-10-14-00-00;/dev/hd10;64.00;1.33;/
2019-10-14-00-00;/dev/hd2;1664.00;501.64;/opt/
2019-10-14-00-00;/dev/hd4;256.00;112.32;/opt/
2019-10-14-00-00;/dev/hd9;2176.00;1223.1;/opt/
2019-10-14-00-00;/dev/hd11;22784.00;12325.8;/opt/
2019-10-14-00-00;/dev/hd12;256.00;2.36;/
2019-10-14-06-00;/dev/hd12;1024.00;137.18;/
2019-10-14-06-00;/dev/hd1;256.00;2.36;/
2019-10-14-00-00;/dev/hd1;1024.00;136.37;/
2019-10-14-00-00;/dev/hd2;5248.00;4230.53;/usr
2019-10-14-00-00;/dev/hd3;2560.00;481.66;/var
2019-10-14-00-00;/dev/hd4;3584.00;67.65;/tmp
2019-10-14-00-00;/dev/hd5;256.00;26.13;/home
2019-10-14-00-00;/dev/hd1;1024.00;476.04;/opt
2019-10-14-00-00;/dev/hd5;384.00;0.38;/usr/xxx
2019-10-14-00-00;/dev/hd4;256.00;21.39;/xxx
2019-10-14-00-00;/dev/hd2;512.00;216.84;/opt
2019-10-14-00-00;/dev/hd3;128.00;21.46;/var/
2019-10-14-00-00;/dev/hd8;256.00;75.21;/usr/
2019-10-14-00-00;/dev/hd7;384.00;186.87;/var/
2019-10-14-00-00;/dev/hd6;256.00;0.63;/var/
2019-10-14-00-00;/dev/hd1;128.00;0.37;/admin
2019-10-14-00-00;/dev/hd4;256.00;179.14;/opt/
2019-10-14-00-00;/dev/hd3;2176.00;492.93;/opt/
2019-10-14-00-00;/dev/hd1;256.00;114.83;/opt/
2019-10-14-00-00;/dev/hd9;256.00;41.73;/var/
2019-10-14-00-00;/dev/hd1;3200.00;954.28;/var/
2019-10-14-00-00;/dev/hd10;256.00;0.93;/var/
2019-10-14-00-00;/dev/hd10;64.00;1.33;/
2019-10-14-00-00;/dev/hd2;1664.00;501.64;/opt/
2019-10-14-00-00;/dev/hd4;256.00;112.32;/opt/
I would like to create a directory for each server, create in each directory a txt file for each FS and put in these txt files each lines which correspond to the FS.
For that, I've tried loop :
#!/bin/bash
directory=(ls *.txt | cut -d'.' -f1)
for d in $directory
do
if [ ! -d $d ]
then
mkdir $d
fi
done
for i in $(cat *.txt)
do
file=$(echo $i | awk -F';' '{print $2}' | sort | uniq | cut -d'/' -f3 )
data=$(echo $i | awk -F';' '{print $2}' )
echo $i | grep -w $data >> /xx/xx/xx/xx/xx/${directory/${file}.txt
done
But this loop doesn't work properly. The directories are created but not the file inside each directory.
I would like something like :
test1/hd1.txt ( with each line which for the hd1 fs in the hd1.txt)
And same thing for each server.
Can you show me how to do that?
#!/bin/bash
for src in *.txt; do
# start a subshell so we don't need to cd back afterwards
# make "$src" be stdin before cd, so we don't need full path
# be careful that in subshell only awk reads from stdin
(
# extract server name to use as directory
dir=/xx/xx/xx/xx/xx/"${src%.txt}"
# chain with "&&" so failures don't cause bad files
mkdir -p "$dir" &&
cd "$dir" &&
awk -F \; '{ split($2, dev, "/"); print > dev[3]".txt" }'
) < "$src"
done
The awk script reads lines delimited by semi-colons.
It splits the second field on slashes to extract the device name (assumption is that the devices always have form: /dev/name
Finally, the > sends output to the relevant file.
For reference, you can make your script work by doing directory=$(...); adding the prefix to mkdir (assuming the prefix directories already exist); closing the reference ${directory}; and quoting all variable references for safety:
#!/bin/bash
directory=$(ls *.txt | cut -d'.' -f1)
for d in "$directory"
do
if [ ! -d "$d" ]
then
mkdir /xx/xx/xx/xx/xx/"$d"
fi
done
for i in $(cat *.txt)
do
file=$(echo "$i" | awk -F';' '{print $2}' | sort | uniq | cut -d'/' -f3 )
data=$(echo $i | awk -F';' '{print $2}' )
echo "$i" | grep -w "$data" >> /xx/xx/xx/xx/xx/"${directory}"/"${file}".txt
done
for file in `ls *.txt`
do
echo ${file}
directory=`echo ${file} | cut -d'.' -f1`
#echo ${directory}
if [ ! -d ${directory} ]
then
mkdir ${directory}
fi
FS=`cat ${file} | awk -F';' '{print $2}' | sort | uniq | cut -d'/' -f3`
#echo $FS
for f in $FS
do
cat ${file} |grep -w -e $f > ${directory}/${f}.txt
done
done
Explanation:
For each file in the current directory, the outer for loop will run.
In the loop for the selected file, a respective directory will be created first.
Next using the FS variable we take all the possible file systems from that selected file.
Finally, an inner loop will be run using the FS types to grep and create separate file system files in the directory.
I have 2 files: File 1 and File 2. I want to check and compare the file name in File 1 and file 2 and where they match, insert into file_1 the numbers from file_2 into a new File_3 using only awk or sed.
I have tried using
cat file.2 | while read line do
Try this:
awk 'NR==FNR { d[$NF]=$1; next } d[$NF] { print d[$NF] FS $0 }' FILE2 FILE1
you can define function by yourself to check file by ne one.
checkDiff() {
file1=$1
file2=$2
result=$3
rm -rf $result
#check Which one is big based line count
file1Count=$(cat ${file1} | wc -l)
file2Count=$(cat ${file2} | wc -l)
# to check which one need to pass to while loop
if [ $file1Count -gt $file2Count ] ; then
bigFile=$file1
smallFile=$file2
else
bigFile=$file2
smallFile=$file1
fi
#check difference line by line
cat $bigFile | while read line ; do
notpint=$(cat $smallFile | grep "$line")
if [ $? -ne 0 ]; then # if you want to print where they match change "ne" to "eq"
echo $line >> $result
fi
done
}
#
checkDiff <file1> <file2> <result filename>
I have file names like: Rv0012_gyrB.txt, Rv0001_Rv.txt
How to rename a file by retaining first 6 characters of my file name and remove rest of the characters?
My desired output should be:
Rv0012.txt and Rv0001.txt
Please let me know, how to do it using a script in Linux for multiple files.
for file in *; do
filename=${file%_*}
fileext=${file##*.}
if [ "$fileext" = "$file" ]; then
mv "$file $filename"
else
mv "$file $filename.$fileext"
fi
done
This should do it, assuming you want to separate at the first occurence of underscore.
If you want to keep first 6 characters, then this:
for file in `ls | grep .txt`;
do
extension="${file##*.}"
filename="${file%.*}"
filename=${filename:0:6}
echo $filename.$extension
mv $file $filename.$extension
done
If you want to get all characters before "_", then this will do the job
for file in `ls | grep .txt`;
do
extension="${file##*.}"
filename="${file%.*}"
filename=`echo $filename | cut -d "_" -f1`
echo $filename.$extension
mv $file $filename.$extension
done
In case you have some files without extensions, try this
for file in `ls`;
do
extension="${file##*.}"
filename="${file%.*}"
filename=`echo $filename | cut -d "_" -f1`
if [ $file == $extension ]
then
mv $file $filename
else
mv $file $filename.$extension
fi
done
This script lists the unit-*-slides.txt files in from directory to a filelist.txt file and from that file list it goes to the file and reads the file and gives the count of st^ lines to a file.but it is not counting in order for ex 1,2,3,4,.... it is counting like 10,1,2,3,4......
How to read it in order.
#!/bin/sh
#
outputdir=filelist
mk=$(mkdir $outputdir)
$mk
dest=$outputdir
cfile=filelist.txt
ofile="combine-slide.txt"
output=file-list.txt
path=/home/user/Desktop/script
ls $path/unit-*-slides.txt | sort -n -t '-' -k 2 > $dest/$cfile
echo "Generating files list..."
echo "Done"
#Combining
while IFS= read file
do
if [ -f "$file" ]; then
tabs=$(cat unit-*-slides.txt | grep "st^" | split -l 200)
fi
done < "$dest/$cfile"
echo "Combining Done........!"
Try with sort -n
tabs=$(cat $( ls unit-*-slides.txt | sort -n ) | grep "st^" | split -l 200)
sort -n means numeric sort, so output of ls is ordered by number.
I have a txt file that has the path to xml files.now i want to read the path from the text file and print the number of tabs present in each xml file.how to do this?
here is what i have done
txt file with path
/home/user/Desktop/softwares/firefox/searchplugins/bing.xml
/home/user/Desktop/softwares/firefox/searchplugins/eBay.xml
/home/user/Desktop/softwares/firefox/searchplugins/answers.xml
/home/user/Desktop/softwares/firefox/searchplugins/wikipedia.xml
/home/user/Desktop/softwares/firefox/blocklist.xml
code to count tabs in each file
code:
#!/bin/sh
#
FILEPATH=/home/user/Desktop/softwares/firefox/*.xml
for file in $FILEPATH; do
tabs=$(tr -cd '\t' < $file | wc -c);
echo "$tabs tabs in file $file" >> /home/user/Desktop/output.txt
done
echo "Done!"
Where /home/user/Desktop/files.txt contains the list of xml files:
#!/bin/bash
while IFS= read file
do
if [ -f "$file" ]; then
tabs=$(tr -cd '\t' < "$file" | wc -c);
echo "$tabs tabs in file $file" >> "/home/user/Desktop/output.txt"
fi
done < "/home/user/Desktop/files.txt"
echo "Done!"
sudo_O has provided an excellent answer. However, there are chances that somehow, mostly due to text editor's preferences, your tabs were converted to 8 consecutive space. If you would prefer to count them as tabs too then replace the "tabs" definition as:
tabs=$(cat test.xml | sed -e 's/ \{8\}/\t/g' | tr -cd '\t' | wc -c)
Full code:
#!/bin/sh
# original file names might contain spaces
# FILEPATH=/home/user/Desktop/softwares/firefox/*.xml
# a better option would be
FIREFOX_DIR="/home/user/Desktop/softwares/firefox/"
while read file
do
if [[ -f "$file" ]]
then
tabs=$(cat test.xml | sed -e 's/ \{8\}/\t/g' | tr -cd '\t' | wc -c)
echo "$tabs tabs in file $file" >> /home/user/Desktop/output.txt
fi
done < $FIREFOX_DIR/*.xml
echo "Done!"
but this is applicable only if you prefer to count 8 consecutive spaces as tabs.