My /etc/environment looks like this:
cat /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
I wish to use a command (sed, awk, python, whatever....) that will make it look like this:
cat /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
JAVA_HOME="/usr/lib/jvm/java-6-sun"
Now the catch is, I would rather it be a 1 liner (in the fields of sed -XYZ /DoMagic/ /etc/environment), it needs to contain merging logic that is - either appends a new configuration record or update an existing one. Bottom line, it should prevent the file from looking like this: (Caused by in experienced shell scripters calling echo >> on each invocation)
cat /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
JAVA_HOME="/usr/lib/jvm/java-5-sun"
JAVA_HOME="/usr/lib/jvm/java-6-sun"
JAVA_HOME="/usr/lib/jvm/java-6-sun"
JAVA_HOME="/usr/lib/jvm/java-6-sun"
I guess this is a trick questions, because what I'm trying to avoid using custom scripts, such as
/usr/local/bin/PropUpdate /etc/environment JAVA_HOME "/usr/lib/jvm/java-6-sun"
/usr/local/bin/PropUpdate is the following script (written for the sake of example, may contain bugs. Comments are appreciated)
#!/bin/bash
# Append/Update a configuration record in a file
#
# Usage example:
# /usr/local/bin/PropUpdate /etc/environment JAVA_HOME "/usr/lib/jvm/java-6-sun"
#
# Author Maxim Veksler <maxim#vekslers.org>
# Version 0.5-2010-07-27
EXPECTED_ARGS=3
E_BADARGS=3
E_BADFILE=4
if [[ $# -ne ${EXPECTED_ARGS} ]]; then
echo "Usage: `basename $0` /path/to/config.conf ParameterName newValueText" >&2
exit $E_BADARGS
fi
CONFIGURATION_FILE="$1"
CONFIGURATION_PARAMETER="$2"
CONFIGURATION_VALUE="$3"
if [[ ! -e "${CONFIGURATION_FILE}" ]]; then
echo "Configuration file ${CONFIGURATION_FILE} does not exist" >&2
exit $E_BADFILE
fi
if [[ ! -w "${CONFIGURATION_FILE}" ]]; then
echo "Can't modify ${CONFIGURATION_FILE}" >&2
exit $E_BADFILE
fi
#########################################
## Decide what parameter we are adding ##
#########################################
__param_found=0
# First check CONFIGURATION_PARAMETER supplied by use that contains "="
if [[ ${CONFIGURATION_PARAMETER} == *=* ]]; then
# It should exist in the file, plain
if grep -qE "^${CONFIGURATION_PARAMETER}" "${CONFIGURATION_FILE}"; then
__param_found=1
SUFFIX_REGEX='[[:space:]]*'
fi
else
# OK, sophisticated user, did not send "=" with the parameter...
if grep -qE "^${CONFIGURATION_PARAMETER}[[:space:]]*=" "${CONFIGURATION_FILE}"; then
# Let's check if such configuration with Parameter + "=" exists
__param_found=1
SUFFIX_REGEX='[[:space:]]*=[[:space:]]*'
elif grep -qE "^${CONFIGURATION_PARAMETER}[[:space:]]+" "${CONFIGURATION_FILE}"; then
# If such parameter exists, at all
__param_found=1
SUFFIX_REGEX='[[:space:]]\+'
fi
fi
if [[ $__param_found == 1 ]]; then
#echo sed -i "s|^\(${CONFIGURATION_PARAMETER}${SUFFIX_REGEX}\).*$|\1${CONFIGURATION_VALUE}|g" "${CONFIGURATION_FILE}"
sed -i "s|^\(${CONFIGURATION_PARAMETER}${SUFFIX_REGEX}\).*$|\1${CONFIGURATION_VALUE}|g" "${CONFIGURATION_FILE}"
else
if [[ ${CONFIGURATION_PARAMETER} == *=* ]]; then
# Configuration parameter contains "=" in it's name, good just append
echo "${CONFIGURATION_PARAMETER}${CONFIGURATION_VALUE}" >> "${CONFIGURATION_FILE}"
else
# Try to guess if this file is a "param = value" or "param value" type of file.
if grep -qE "^[[:alnum:]]+[[:space:]]*=" "${CONFIGURATION_FILE}"; then
# Seems like a "param = value" type of file
echo "${CONFIGURATION_PARAMETER}=${CONFIGURATION_VALUE}" >> "${CONFIGURATION_FILE}"
else
# Seems like a "param value" type of file
echo "${CONFIGURATION_PARAMETER} ${CONFIGURATION_VALUE}" >> "${CONFIGURATION_FILE}"
fi
fi
fi
#cat $CONFIGURATION_FILE
Thank you,
Maxim.
-- Update: I actually kinda liked this script, so I've improved it a bit. It now seems to be production ready. Enjoy.
Instead of trying to parse /etc/environment file, you could instead create a file with your own name in /etc/profile.d/, as I described in my answer to a relevant question. Then you could just copy it over during installation, because it contains just your content. Let alone that it will make your scripts shorter.
grep -q JAVA_HOME /etc/environment || echo 'JAVA_HOME="/usr/lib/jvm/java-5-sun"' >> /etc/environment
The grep command returns 0 (true) if the pattern is found in the file. So, the above reads:
check if JAVA_HOME is set in the file
OR set JAVA_HOME in the file
0-15:49 root#noneedto ~# cat /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
0-15:49 root#noneedto ~# grep JAVA_HOME /etc/environment && echo true
1-15:49 root#noneedto ~# grep -q JAVA_HOME /etc/environment || echo 'JAVA_HOME="/usr/lib/jvm/java-5-sun"' >> /etc/environment
0-15:49 root#noneedto ~# grep JAVA_HOME /etc/environment && echo true
JAVA_HOME="/usr/lib/jvm/java-5-sun"
true
0-15:49 root#noneedto ~# grep -q JAVA_HOME /etc/environment || echo 'JAVA_HOME="/usr/lib/jvm/java-5-sun"' >> /etc/environment
0-15:49 root#noneedto ~# cat /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
JAVA_HOME="/usr/lib/jvm/java-5-sun"
As you can see, if you invoke this one-liner multiple times, subsequent invocations do not add to the file because grep returns true before you attempt to append the file.
In my Ubuntu system, my JAVA_HOME looks like this:
JAVA_HOME=/usr/lib/jvm/default-java
Looking at that file with ls -l /usr/lib/jvm/default-java I noticed this:
lrwxrwxrwx 1 root root 24 Apr 27 2012 /usr/lib/jvm/default-java -> java-1.7.0-openjdk-amd64
In other words, the path in the soft link is the only thing you have to change.
To see the list of installed Java environments, I used this ls -l ... command:
prompt$ ls -l /usr/lib/jvm
total 20
lrwxrwxrwx 1 root root 24 Apr 27 2012 default-java -> java-1.7.0-openjdk-amd64
drwxr-xr-x 4 root root 4096 Feb 23 17:54 java-1.5.0-gcj-4.8-amd64
lrwxrwxrwx 1 root root 20 Sep 2 2012 java-1.6.0-openjdk-amd64 -> java-6-openjdk-amd64
lrwxrwxrwx 1 root root 20 Jul 3 2013 java-1.7.0-openjdk-amd64 -> java-7-openjdk-amd64
drwxr-xr-x 5 root root 4096 Oct 7 2012 java-6-openjdk-amd64
drwxr-xr-x 3 root root 4096 Oct 7 2012 java-6-openjdk-common
drwxr-xr-x 5 root root 4096 Sep 21 20:06 java-7-openjdk-amd64
drwxr-xr-x 8 root root 4096 Sep 18 21:18 java-7-oracle
So now I can switch to another default with:
sudo rm /usr/lib/jvm/default-java
sudo ln -s java-7-oracle /usr/lib/jvm/default-java
And the JAVA_HOME variable will run Java 7 from Oracle.
Related
Looking for a very simple way to check whether a file/directory exists while evaluating user permissions, returning different (code) errors:
There is command test that checks for permissions but fails to provide a better return code for case where file does not exist:
$ test -r /tmp/; echo $? # 0
$ test -r /tmp/root/; echo $? # 1
$ test -r /tmp/missing/; echo $? # 1
I am looking for something similar to ls where I get a different message for different errors:
$ ls /tmp/root
ls: root: Permission denied
$ ls /tmp/missing
ls: /tmp/missing: No such file or directory
I like the differentiation but the error code is 1 in both. To properly handle each error, I have to parse stderr which is honestly a very inelegant solution.
Isn't there a better and graceful way of doing this?
Something close to a pythonic way looks something like this:
import os
os.listdir("/tmp/root/dir/") # raises PermissionError
os.listdir("/tmp/foo/") # raises FileNotFoundError
Read the manual some more. There's also -d to specifically check whether the target is a directory, and a slew of other predicates to check for symlinks, device nodes, etc.
testthing () {
if ! [[ -e "$1" ]]; then
echo "$1: not found" >&2
return 2
elif ! [[ -d "$1" ]]; then
echo "$1: not a directory" >&2
return 4
elif ! [[ -r "$1" ]]; then
echo "$1: permission denied" >&2
return 8
fi
return 0
}
Usage:
testthing "/root/no/such/directory"
Notice that [[ is a Bash built-in which is somewhat more robust and versatile than the legacy [ aka test.
It's hard to predict what the priorities should be, but if you want the comparisons in a different order, by all means go for it. It is unavoidable that the shell cannot correctly tell the precise status of a directory entry when it lacks read access to the parent directory. Maybe solve this from the caller by examining the existence and permissions of every entry in the path, starting from the root directory.
The shell and standard utilities do not provide a command that does everything you seem to want:
with a single command execution,
terminate with an exit status that reports in detail on the existence and accessability of a given path,
contextualized for the current user,
accurately even in the event that a directory prior to the last path element is untraversable (note: you cannot have this one no matter what),
(maybe) correctly for both directories and regular files.
The Python os.listdir() doesn't do all of that either, even if you exclude the applicability to regular files and traversing untraversible directories, and reinterpret what "exit status" means. However, os.listdir() and ls both do demonstrate a good and useful pattern: to attempt a desired operation and deal with any failure that results, instead of trying to predict what would happen if you tried it.
Moreover, it's unclear why you want what you say you want. The main reason I can think of for wanting information about the reason for a file-access failure is user messaging, but in most cases you get that for free by just trying to perform the wanted access. If you take that out of the picture, then it rarely matters why a given path is inaccessible. Any way around, you need to either switch to an alternative or fail.
If you nevertheless really do want something that does as much as possible of the above, then you probably will have to roll your own. Since you expressed concern for efficiency in some of your comments, you'll probably want to do that in C.
Given:
$ ls -l
total 0
-rw-r--r-- 1 andrew wheel 0 Mar 22 12:01 can_read
---xr-x--x 1 andrew wheel 0 Mar 22 12:01 root
drwxr-xr-x 2 andrew wheel 64 Mar 22 13:09 sub
Note that permissions are by user for the first three, group for the second three and other or world for the last three.
Permission Denied error is from 1) Trying to read or write a file without that appropriate permission bit set for your user or group or 2) Tying to navigate to a directory without x set or 3) Trying to execute a file without appropriate permission.
You can test if a file is readable or not for the user with the -r test:
$ [[ -r root ]] && echo 'readable' || echo 'not readable'
not readable
So if you only are concerned with user permissions, -r, -w and -x test are what you are looking for.
If you want to test permissions generally, you need to use stat.
Here is a simple example with that same directory:
#!/bin/bash
arr=(can_read root sub missing)
for fn in "${arr[#]}"; do
if [[ -e "$fn" ]]
then
p=( $(stat -f "%SHp %SMp %SLp" "$fn") )
printf "File:\t%s\nUser:\t%s\nGroup:\t%s\nWorld:\t%s\nType:\t%s\n\n" "$fn" "${p[#]}" "$(stat -f "%HT" "$fn")"
else
echo "\"$fn\" does not exist"
fi
done
Prints:
File: can_read
User: rw-
Group: r--
World: r--
Type: Regular File
File: root
User: --x
Group: r-x
World: --x
Type: Regular File
File: sub
User: rwx
Group: r-x
World: r-x
Type: Directory
"missing" does not exist
Alternatively, you can grab these values directly from the drwxr-xr-x type data with:
for fn in "${arr[#]}"; do
if [[ -e "$fn" ]]
then
p=$(stat -f "%Sp" "$fn")
typ="${p:0:1}"
user="${p:1:3}"
grp="${p:4:3}"
wrld="${p:7:3}"
else
echo "\"$fn\" does not exist"
fi
done
In either case, you can then test the individual permissions with either Bash string functions, Bash regex, or get the octal equivalents and use bit masks.
Here is an example:
for fn in "${arr[#]}"; do
if [[ -e "$fn" ]]
then
p=$(stat -f "%Sp" "$fn")
user="${p:1:3}"
ty="$(stat -f "%HT" "$fn")"
printf "%s \"$fn\" is:\n" "$ty"
[[ $user =~ 'r' ]] && echo " readable" || echo " not readable"
[[ $user =~ 'w' ]] && echo " writeable" || echo " not writeable"
[[ $user =~ 'x' ]] && echo " executable" || echo " not executable"
else
echo "\"$fn\" does not exist"
fi
done
Prints:
Regular File "can_read" is:
readable
writeable
not executable
Regular File "root" is:
not readable
not writeable
executable
Directory "sub" is:
readable
writeable
executable
"missing" does not exist
(Note: stat tends to be platform specific. This is BSD and Linux will have different format strings...)
An example of use.
for d in 1 2 3; do
if [[ -e $d ]]; then printf "%s exists" $d
[[ -r $d ]] && echo " and is readable" || echo " but is not readable"
else echo "$d does not exist"
fi
stat -c "%A %n" $d
done
1 exists and is readable
drwxr-xr-x 1
2 exists but is not readable
d--------- 2
3 does not exist
stat: cannot stat ‘3’: No such file or directory
If you absolutely have to have it in one step with differentiated exit codes, write a function. (a/b is there and has accessible permissions.)
$: stat -c "%A %n" ? . a/b # note there is no directory named 3
drwxr-xr-x 1
drwxr-xr-x 2
drwxr-xr-x a
drwxrwxrwt .
drwxr-xr-x a/b
$: doubletest() { if [[ -e "$1" ]]; then [[ -r "$1" ]] && return 0 || return 2; else return 1; fi; }
$: result=( "exists and is readable" "does not exist" "exists but is unreadable" ) # EDITED - apologies, these were out of order
$: for d in . a a/b 1 2 3; do doubletest $d; echo "$d ${result[$?]}"; done
. exists and is readable
a exists and is readable
a/b exists and is readable
1 exists and is readable
2 exists and is readable
3 does not exist
$: chmod 0000 a
$: for d in . a a/b 1 2 3; do doubletest $d; echo "$d ${result[$?]}"; done
. exists and is readable
a exists but is unreadable
a/b does not exist
1 exists and is readable
2 exists but is unreadable
3 does not exist
"does not exist" for a/b is because a does not have read permissions, so there is no way for any tool to know what does or does not exist in that directory short of using root privileges.
$ sudo stat -c "%A %n" ? . a/b # sudo shows a/b
drwxr-xr-x 1
drwxr-xr-x 2
d--------- a
drwxrwxrwt .
drwxr-xr-x a/b
In that case your problem isn't the tool, it's that the tool can't do what you are asking it to do.
PASSED=$1
if [ -f $PASSED ]; then
echo "$PASSED is a file"
ls -l $PASSED
elif [ -d $PASSED ]; then
echo "$PASSED is a directory"
ls -l $PASSED
else
"$PASSED is invalid"
fi
At the terminal when I push a file input, say demo.sh, the output is correctly printed as:
"demo.sh is a file"
rwxr-xr-x 1 system system 12 Jan 16 03:12 26 14:47 demo.sh
but for a directory, say cloud, it gives:
cloud is a directory
total 0
What should I do to rectify this?
enter image description here
As shown from the man pages for ls:
-d, --directory
list directories themselves, not their contents
With your current implementation, as you are using just -l with your directories, ls will show the contents of the directory. To list the properties of the directory only, use -d in addition to -l and so:
ls -ld $PASSED
I am running a script but there is an unusual warning:
This is what happened in my console
#whoami
root
#ls -l test.sh
-rwxr-xr-x. 1 root root 1894 Feb 2 01:58 test.sh*
#./test.sh
-bash: ./test.sh: Permission denied
Edit:
my script:
#!/bin/bash
while read pass port user ip file; do
echo "startt------------------------------------" $ip
ping $ip -c 4
if [ $? -eq 0 ]; then
echo $ip ok...
else
echo $ip failed...
fi
echo "finish------------------------------------" $ip
done <<____HERE
pass 22 root 1.1.1.1 test.txt
____HERE
any idea?
thank you
I am running the script in /tmp directory
as you see the result of ls is:
-rwxr-xr-x. 1 root root 1894 Feb 2 01:58 test.sh*
there is . after permissions which indicates that an SELinux security context applies to that file. so I copied test.sh in a directory else...
the problem was solved
ls -l /
drwxrwxrwt. 8 root root 1024 Feb 2 07:44 tmp/
I was in a directory where it might be a bad idea for executables to reside
These may work as well:
setenforce 0 | reboot
OR
echo 0 > /selinux/enforce | reboot
OR:
putting SELINUX=disabled in /etc/selinux/config and reboot (making sure to comment out anything in that file enabling selinux)
SELINUX status: sestatus
I am trying to create a script that walks a directory and renames files. I would like to be able to extract the filename and file extension separately, but if the file path contains either spaces or Swedish UTF8-characters such as ÅÄÖ, it breakes.
I've found the below shown snippet to extract the filename + extension here on SO, but as I am seeing that it works on paths with no UTF-chars or whitespace, I am thinking that I am not properly escaping my variables.
Perhaps I am doing something wrong. Any ideas on what I can do to make this work with paths with UTF8 chars and whitespace?
for file in $NAUTILUS_SCRIPT_SELECTED_FILE_PATHS; do
FULLPATH="$file"
FILENAME=${FULLPATH##*/}
FILEEXTENSION=${FILENAME##*.}
BASEDIRECTORY=${FULLPATH%$FILENAME}
#Log the vars for debugging
echo "$FULLPATH" >> ~/Desktop/log.txt
echo "$FILENAME" >> ~/Desktop/log.txt
echo "$FILEEXTENSION" >> ~/Desktop/log.txt
echo "$BASEDIRECTORY" >> ~/Desktop/log.txt
done
The problem is that the NAUTILUS_SCRIPT_SELECTED_FILE_PATH variable is new-line escaped per item.
You need to use:
while read file; do
FULLPATH="$file"
FILENAME=${FULLPATH##*/}
FILEEXTENSION=${FILENAME##*.}
BASEDIRECTORY=${FULLPATH%$FILENAME}
#Log the vars for debugging
echo "$FULLPATH" >> ~/Desktop/log.txt
echo "$FILENAME" >> ~/Desktop/log.txt
echo "$FILEEXTENSION" >> ~/Desktop/log.txt
echo "$BASEDIRECTORY" >> ~/Desktop/log.txt
done <<<"$NAUTILUS_SCRIPT_SELECTED_FILE_PATHS"
I have this in my dotfiles:
# Prefer US English and use UTF-8
export LC_ALL="en_US.UTF-8"
export LANG="en_US"
With that configuration, it seems to work:
$ ls -l
total 0
drwxr-xr-x 2 Mathias staff 68 Jan 17 11:32 test space test
drwxr-xr-x 2 Mathias staff 68 Jan 17 11:29 test©test
$ for file in *; do echo "$file"; done
test space test
test©test
i have bunch of files that needs to be renamed.
file1.txt needs to be renamed to file1_file1.txt
file2.avi needs to be renamed to file2_file2.avi
as you can see i need the _ folowed by the original file name.
there are lot of these files.
So far all the answers given either:
Require some non-portable tool
Break horribly with filenames containing spaces or newlines
Is not recursive, i.e. does not descend into sub-directories
These two scripts solve all of those problems.
Bash 2.X/3.X
#!/bin/bash
while IFS= read -r -d $'\0' file; do
dirname="${file%/*}/"
basename="${file:${#dirname}}"
echo mv "$file" "$dirname${basename%.*}_$basename"
done < <(find . -type f -print0)
Bash 4.X
#!/bin/bash
shopt -s globstar
for file in ./**; do
if [[ -f "$file" ]]; then
dirname="${file%/*}/"
basename="${file:${#dirname}}"
echo mv "$file" "$dirname${basename%.*}_$basename"
fi
done
Be sure to remove the echo from whichever script you choose once you are satisfied with it's output and run it again
Edit
Fixed problem in previous version that did not properly handle path names.
For your specific case, you want to use mmv as follows:
pax> ll
total 0
drwxr-xr-x+ 2 allachan None 0 Dec 24 09:47 .
drwxrwxrwx+ 5 allachan None 0 Dec 24 09:39 ..
-rw-r--r-- 1 allachan None 0 Dec 24 09:39 file1.txt
-rw-r--r-- 1 allachan None 0 Dec 24 09:39 file2.avi
pax> mmv '*.*' '#1_#1.#2'
pax> ll
total 0
drwxr-xr-x+ 2 allachan None 0 Dec 24 09:47 .
drwxrwxrwx+ 5 allachan None 0 Dec 24 09:39 ..
-rw-r--r-- 1 allachan None 0 Dec 24 09:39 file1_file1.txt
-rw-r--r-- 1 allachan None 0 Dec 24 09:39 file2_file2.avi
You need to be aware that the wildcard matching is not greedy. That means that the file a.b.txt will be turned into a_a.b.txt, not a.b_a.b.txt.
The mmv program was installed as part of my CygWin but I had to
sudo apt-get install mmv
on my Ubuntu box to get it down. If it's not in you standard distribution, whatever package manager you're using will hopefully have it available.
If, for some reason, you're not permitted to install it, you'll have to use one of the other bash for-loop-type solutions shown in the other answers. I prefer the terseness of mmv myself but you may not have the option.
for file in file*.*
do
[ -f "$file" ] && echo mv "$file" "${file%%.*}_$file"
done
Idea for recursion
recurse() {
for file in "$1"/*;do
if [ -d "$file" ];then
recurse "$file"
else
# check for relevant files
# echo mv "$file" "${file%%.*}_$file"
fi
done
}
recurse /path/to/files
find . -type f | while read FN; do
BFN=$(basename "$FN")
NFN=${BFN%.*}_${BFN}
echo "$BFN -> $NFN"
mv "$FN" "$NFN"
done
I like the PERL cookbook's rename script for this. It may not be /bin/sh but you can do regular expression-like renames.
The /bin/sh method would be to use sed/cut/awk to alter each filename inside a for loop. If the directory is large you'd need to rely on xargs.
One should mention the mmv tool, which is especially made for this.
It's described here: http://tldp.org/LDP/GNU-Linux-Tools-Summary/html/mass-rename.html
...along with alternatives.
I use prename (perl based), which is included in various linux distributions. It works with regular expressions, so to say change all img_x.jpg to IMAGE_x.jpg you'd do
prename 's/img_/IMAGE_/' img*jpg
You can use the -n flag to preview changes without making any actual changes.
prename man entry
#!/bin/bash
# Don't do this like I did:
# files=`ls ${1}`
for file in *.*
do
if [ -f $file ];
then
newname=${file%%.*}_${file}
mv $file $newname
fi
done
This one won't rename sub directories, only regular files.