I have a problem with this linux shell script.
#! /bin/bash
find /sdcard/ -type f -iname "*.srt" -print >> /sdcard/files
count=`wc -l /sdcard/files |cut -d'/' -f1`
for (( c=1; c<=$count; c++ ))
do
line=`sed -n ''$c'p' /sdcard/files`
cat "$line" | sed -e 's/č/c/g' > "$line".srt""
rm "$line"
done
rm /sdcard/files
I know this isnt the best way to do this but thats all i can do with my knowlage
As you can see it finds all srt files and then replaces all "č" charactes with "c". But it doesnt work with files i downloaded
However when i make a new file and write "č" inside (with my keyboard), it replaces it as it should. I dont understand why?
I think we discovered the cause, now the fix:
vim somefile.srt -c ":set bomb" -c ":set fileencoding=utf-8" -c ":wq"
There's also a dirty way
echo -e "\xC2\xA0" >> somefile.srt
I tried iconv tool which is supposed to do the conversion, but it didn't help.
Related
I need to write a script that checks some >20k files for some >2k search text and it needs to be flexible, so I came up with this script:
#!/bin/bash
# This script checks all files in a given directory against a list of criteria
shopt -s expand_aliases
source ~/.bashrc
TIMESTAMP=$(date "+%Y-%m-%d-%T")
ROOT_DIR=/data
PROJECT_NAME=$1
FILE_DIR=$ROOT_DIR/projects/$1/$2
RESULT_DIR=$ROOT_DIR/projects/$1/check_result
SEARCHTEXT_FILE=$ROOT_DIR/scripts/$3
OIFS="$IFS"
IFS=$'\n'
files=$(find $FILE_DIR -type f -name '*.json')
for file in $files; do
while read line; do
grep -H -o $line "$file" >> $RESULT_DIR/check_result_$TIMESTAMP.log
done < $SEARCHTEXT_FILE
done
IFS="$OIFS"
This script only produces the empty $RESULT_DIR/check_result_$TIMESTAMP.log log file with correct name.
Because the file names sometimes contain spaces I added the IFS... statements and I enclosed $file in " quotes (copied from another post).
The content of the $SEARCHTEXT_FILE is for example:
'Tel alt........'
'City ..........'
If I place an echo before the grep like this
echo grep -H -o $line "$file"
then output I get is
grep -H -o 'Tel alt........' /data/projects/DNAR/input/report-157538.json
and I can execute this line as is and get the correct result.
I tried to put various combinations of " or ' or ` or () or {} around any part of this grep command but nothing changed.
Somewhere I did read about alias and the alias set for grep is
alias grep='grep --color=auto'
After many hours of searching on the internet I couldn't find any post that helped me as most of them are covering issues around wrong quotes or inline bash issues.
What are I missing here?
The simple and obvious workaround is to remove all that complexity and simply use the features of the commands you are running anyway.
find "$FILE_DIR" -type f -name '*.json' \
-exec grep -H -o -f "$SEARCHTEXT_FILE" {} + > "$RESULT_DIR/check_result_$TIMESTAMP.log"
Notice also the quoting fixes; see When to wrap quotes around a shell variable; to avoid mishaps, you should switch to lower case for your private variables (see Correct Bash and shell script variable capitalization).
shopt -s expand_aliases
and source ~/.bashrc merely look superfluous, but could contribute to whatever problem you are trying to troubleshoot; they should basically never be part of a script you plan to use in production.
I have a bunch of files under a directory. how can I check all of them and make sure if it is a perl script or not?(they don't have .pl in the filename)
If you cannot rely on there being a valid shebang either, you might pass them to perl -c.
for f in *; do
perl -c "$f" 2>/dev/null && echo "$f is Perl"
done
If you want properly machine-readable output, maybe switch the echo to printf '%s\0' "$f" so you can pass it to xargs -0 and friends.
The obvious flaw with this is that a Perl script with an error in it will be reported as not being (valid) Perl.
Check the shebang
head -n 1 script | grep perl
Normally most command line scripts contain a shebang ie something like
#!/usr/bin/perl
They're not required if you are calling the script like this
perl script
but if you want to call them as system command they help.
find ./ -type f -exec egrep -I -l '^use strict;|^use warnings;|^sub |my \$|my \%|my \#|\->{' {} + 2>&1 \
| egrep -v 'README|\.git|\.zsh$|.sh$' \
| xargs file | grep 'ASCII' \
| awk '{print $1}' \
| sed 's/:$//'
not perfect but this will find most files with relatively modern Perl5 code in them
Since they do not have the extension, try this:
find /path/to/directory/ -type f | while read line; do if file -b "$line" | grep -i perl -q; then echo "$line is a perl file"; fi; done
I am working on a project for one of my professors and he asked me to sort a couple hundred .fits images based on their header files (specifically what star they are images of) I think that grep would be the best way to do this however I can't seam to figure out how to use grep based on the header.
I am entering:
ls | imhead *.fits | grep -E -r "PG\ 1104+243" *
to just list them out for now, once they are listed I know how to copy them into a directory.
I am new to using grep so I am unsure as to where my error lies? any help would be greatly appreciated! Thanks!
Assuming that imghead will extract the headers of the .fits as txt, you can use a simple shell script to do it:
script.sh
#!/bin/bash
grep "$1" "$2" > /dev/null 2>&1 && echo "$2"
Note that the + is a special character if you use extended regular expression, meaning if you pass the -E as in the question. A simple grep without any options should do the trick here.
Use find to exec the script on every *.fits file in the current folder:
find -maxdepth 1 -name '*.fits' -exec ./script.sh 'PG 1104+243' {} \;
If you are going to copy/move/alter or do something with the files you find, you might be better off, in terms of complexity and ease of quoting, using a loop like this:
#!/bin/bash
find . -name \*.fits -print0 | while read -d '' -r file; do
echo Checking file: $file
imhead "$file" | grep -q 'PG 1104+243'
if [ $? -eq 0 ]; then
echo Object matches: $file
fi
done
By default, vim spell checker is code aware, so it doesn't spell-check code parts of the file. In effect, in markdown it considers (pandoc multiline) tables to be codes and thus doesn't spell-check their contents.
Is it possible to override this? Or enable spell-check for the entire file including code.
As far as I'm able to determine, there is no way to tell Vim to ignore the
spellcheck suggestions in the syntax file and to just "check everything".
A fairly heavy-handed workaround is to disable syntax entirely with :syn off;
you can re-enable this with :syn on.
Specifically for Markdown, you can disable highlighting of code blocks with
:syn clear markdownCodeBlock; you can reset this with :syn on as well.
Use syntax spell
:syntax spell toplevel
See:
http://usevim.com/2013/05/10/synspell/
In that case I would contact the maintainer of the markdown syntax file and ask him/she if (s)he could fix this issue.
I created bash script fixing syntax files. IT IS NOT PERFECT BUT IT IS GOOD. It can be reversed by running it again. It adds contains=#Spell to syn match and syn region definitions in all files in given directory.
To use it:
Save the script as fix_syntax_files.sh
Give it permissions
Change path at the bottom of the script to one corresponding to your vim plugins location
Run the script
(OPTIONAL) Run script again to revert the changes
The script makes backup of all files before modification so you can assume it is safe to run it. I anyway do not take any responsibility for potential problems caused by the script.
Edit:
You can leave feedback to the script in the following repository:
https://github.com/dominikduda/config_files/blob/master/bin/fix_vim_syntax_files.sh
#!/bin/bash
function fix_file {
sed -i -e '/exe/! {s/contains=/contains=#Spell,/g}' $1
sed -i -e 's/contains=#Spell,ALL/contains=ALL/g' $1
sed -i -e 's/contains=#Spell,ALLBUT/contains=ALLBUT/g' $1
sed -i -e 's/contains=#Spell,TOP/contains=TOP/g' $1
sed -i -e 's/contains=#Spell,CONTAINED/contains=CONTAINED/g' $1
sed -i -e 's/contains=#Spell,NONE/contains=#Spell/g' $1
sed -i -e '/^ *syn match/ {/contains=/! s/$/ contains=#Spell/g}' $1
sed -i -e '/^ *syn region/ {/contains=/! s/$/ contains=#Spell/g}' $1
return 0
}
function revert_file {
mv "$1/$2.spellfix-backup" "$1/$2"
return 0
}
function fix_recursively_in_catalog {
syntax_catalogs_paths="$(find $1 -type d ! -name '*.*' -not -path '*git*' -print)"
syntax_catalogs_count="$(echo "${syntax_catalogs_paths}" | wc -l)"
echo "${syntax_catalogs_count} syntax catalogs found and will be scanned for files"
echo "${syntax_catalogs_paths}" | while read -r catalog_path ; do
echo " Scanning $catalog_path"
ls -p "${catalog_path}" | grep -v / | grep -v .spellfix-backup | grep .vim | while read -r file_name ; do
cp "${catalog_path}/${file_name}" "${catalog_path}/${file_name}.spellfix-backup"
fix_file "${catalog_path}/${file_name}"
echo " Fixing ${file_name} (backup created as ${file_name}.spellfix-backup)"
done
done
echo 'Fix done.'
echo 'Remember to REVERT FIX before updating vim plugins'
return 0
}
function revert_recursively_in_catalog {
syntax_catalogs_paths="$(find $1 -type d ! -name '*.*' -not -path '*git*' -print)"
syntax_catalogs_count="$(echo "${syntax_catalogs_paths}" | wc -l)"
echo "${syntax_catalogs_count} syntax catalogs found and will be scanned for spellfix-backup files"
echo "${syntax_catalogs_paths}" | while read -r catalog_path ; do
echo " Scanning $catalog_path"
ls -p "${catalog_path}" | grep -v / | grep -v .spellfix-backup | grep .vim | while read -r file_name ; do
revert_file "${catalog_path}" "${file_name}"
echo " Reverting ${file_name} (from file ${file_name}.spellfix-backup)"
done
done
echo 'Revert done.'
echo 'Remember to FIX AGAIN after plugins update (or set it as a post update hook)'
return 0
}
function main {
syntax_catalogs_paths="$(find $1 -type d ! -name '*.*' -not -path '*git*' -print)"
while read -r catalog_path ; do
if ls -p "${catalog_path}" | grep -v / | grep .spellfix-backup; then
echo ".spellfix-backup files found, reverting fix!"
echo "--------------------------------------------"
revert_recursively_in_catalog $1
return 0
fi
done < <(echo "${syntax_catalogs_paths}")
echo ".spellfix-backup files NOT found, fixing!"
echo "-----------------------------------------"
fix_recursively_in_catalog $1
}
main ~/PATH/TO/VIM/PLUGINS/
I am trying to make a script to append all files ending with .hash to be verified by md5deep. Files with space in their name seem to break this script.
#!/bin/bash
XVAR=""
for f in *.hash
do
XVAR="$XVAR -x $f "
done
md5deep -e $XVAR -r *
Whenever i run the script with a file called "O S.hash" i would get
O: No such file or directory
If i change XVAR="$XVAR -x $f " to XVAR="$XVAR -x \'$f\' " or XVAR="$XVAR -x \"$f\" "
md5deep will interpenetrate the input as "O instead
"O: No such file or directory
an echo of the variable in the script shows XVAR as -x 'O S.hash' or -x "O S.hash"
a manual input of the command in shell such as md5deep -e -x "O S.hash" -r * works but if its in the script the command seems to break
This is not the nicest solution, but is seems it will work:
find . -name '*.hash' -printf "-x\0%p\0" | xargs -0 md5deep -r * -e
This actually doesn't do exactly the same as the OP wanted, so here's a modification as suggested by Tim Pote and Jonathan Leffler:
find . -maxdepth 1 -name '*.hash' -printf "-x\0%p\0" | xargs -0 md5deep -r * -e
Now you know why people on Unix systems traditionally avoided file names with spaces in them (and directory names likewise) — it is a nuisance (to be polite about it) to have to program the shell to handle such names. The shell was designed for use in systems without such names. Newlines also cause much grief.
With bash, your best solution by far is to use an array to hold the elements, and then "${array[#]}" to list them; it is almost trivial:
declare -a XVAR
for file in *.hash
do
XVAR+=("-x" "$file")
done
md5deep -e "${XVAR[#]}" -r *
(Exploiting the array extension notation mentioned by Gordon Davisson. See section §6.7 'Arrays' of the bash reference manual (for Bash 4.1) for a lot of array information; see section §3.4 'Shell Parameters' for the += operator.)
If you can't use arrays for some reason, then you need a program that escapes its arguments so that the shell won't distort things. I have such a program, called escape:
XVAR=
for file in *.hash
do
name=$(escape "$file")
XVAR="$XVAR -x $file"
done
eval md5deep -e $XVAR -r *
With the eval, it is tricky to use; it works, but use arrays.