The variables var1 and var2 in the below command needs to be replaced and replicated with over 500 unique attributes. Complete beginner in python, suggestions ?
(Plan is to read a file with 500 attributes and loop it to replicate the command with all the different attributes found in the file to a second file or print the command output in console.)
command:
dsconfig create-proxy-transformation --transformation-name Attr-Mapping_proxy01 --type attribute-mapping --set enabled:true --set source-attribute:"var1" --set target-attribute:"var2"
file = open('open.txt')
for line in file:
fields = line.strip().split()
print fields[0], fields[1]
Assign values to variables var1 and var2 in above command with for loop with appropriate field values, it will do the rest of stuff.
considering in text file
AA11 BB11 CC11 DD11
AA22 BB22 CC22 DD22
AA33 BB44 CC44 DD33
Python is so easy :)
To be more specific, split() splits the contents of a string into fields delimited by some delimiter (by default any blank character, e.g. space, tab etc.), and returns an array with the split fields. strip() strips all blank characters from the beginning and end of a line. And a file in python is an iterable object which when iterated over by the keyword in, gives the lines in the file one by one. For more information on these, you can look at http://docs.python.org/2/library/stdtypes.html#str.split , http://docs.python.org/2/library/stdtypes.html#str.strip , http://docs.python.org/2/library/stdtypes.html#bltin-file-objects .
link : Reading a file and storing values into variables in python
Related
I have a file named mail_status.txt The content of the file is as follows.
1~auth_flag~
2~download_flag~
3~copy_flag~
4~auth_flag~
5~auth_flag~
6~copy_flag~
I want to perform some operation on this file so that at the end I should be getting three variables and their respective values should be as follows:
auth_flag_ids="1,4,5"
download_flag_ids="2"
copy_flag_ids="3,6"
I am quite new to this language. Please let me know if some more details are required on this.
Thanks
If you want to generate bash variables based on the file content,
please try the following:
# read the file and extract information line by line
declare -A hash # delcare hash as an associative array
while IFS= read -r line; do
key="${line#*~}" # convert "1~auth_flag~" to "auth_flag~"
key="${key%~*}_ids" # convert "auth_flag~" to "auth_flag_ids"
hash[$key]+="${line%%~*}," # append the value to the hash
done < "mail_status.txt"
# iterate over the hash to create variables
for r in "${!hash[#]}"; do # r is assigned to "auth_flag_ids", "download_flag_ids" and "copy_flag_ids" in tern
printf -v "$r" "%s" "${hash[$r]%,}" # create a variable named "$r" and assign it to the hash value by trimming the trailing comma off
done
# check the result
printf "%s=\"%s\"\n" "auth_flag_ids" "$auth_flag_ids"
printf "%s=\"%s\"\n" "download_flag_ids" "$download_flag_ids"
printf "%s=\"%s\"\n" "copy_flag_ids" "$copy_flag_ids"
First it reads the lines of the file and extracts the variable name
and the value line by line. They are stored in an associative array hash.
Next it iterates over the keys of hash to create variables whose names are
"auth_flag_ids", "download_flag_ids" and "copy_flag_ids".
printf -v var creates a variable var. This mechanism is useful to cause an
indirect reference to a variable.
I'm not going to explain in detail about the bash specific notations
such as ${parameter#word}, ${parameter%%word} or ${!name[#]}.
You can easily find the references and well-explained documents including
the bash man page.
Hope this helps.
I have a file with 278 rows of application id combinations that look like this:
Windows Azure Active Directory 00000002-0000-0000-c000-000000000000
Microsoft.SMIT 8fca0a66-c008-4564-a876-ab3ae0fd5cff
I wrote a bit of Python regex:
lines = f.readlines()
for line in lines:
newline = re.sub('\s+', ' ', line)
print(newline)
Now I have them looking like this:
Windows Azure Active Directory 00000002-0000-0000-c000-000000000000
Microsoft.SMIT3 8fca0a66-c008-4564-a876-ab3ae0fd5cff
Microsoft Azure Workflow 00000005-0000-0000-c000-000000000000
Initially I thought I could do:
newnewline = re.sub('\s\d', '\": \"', newline)
But it was removing the \s\d (space and digit), but since the app names have spaces I wasn't sure what else I could use for a regex to match.
Example (first one missing the initial 0):
Windows Azure Active Directory": "0000002-0000-0000-c000-000000000000
Windows Azure Active Directory": "00000002-0000-0000-c000-000000000000
So I am still stuck here.
I need/want to turn all 278 of them into a Python dictionary like this.
"00000004-0000-0ff1-ce00-000000000000": "Microsoft.Lync",
"00000006-0000-0ff1-ce00-000000000000": "Microsoft.Office365Portal",
All you are doing is removing some internal whitespace and creating a dictionary where the last element in a line is the key for the rest of the line. Using regular expressions is overkill. If lines is the list of lines with which you start the question, the following assembles the dictionary you want without needing to use regular expressions:
d = {}
for line in lines:
data = line.split()
d[data[-1]] = ' '.join(data[:-1])
I'm trying to read a the csv file :
"EDP";"Picture 1";"Picture 2";"Picture 2"
"1001480210";"T244.png";"";""
I create a script to read this files :
cd ~/86829/
while IFS=';' read "EDP" "Picture 1" "Picture 2" "Picture 3"
When i run the script, i got this error :
./RUN_CopyPictures.sh: 9: read: Picture 1: bad variable name
When i change titles to "Picture1";"Picture2";"Picture2" , and the script to while IFS=';' read EDP Picture1 Picture2 Picture3
it's work
So my question is very clear :
How can i read a title of csv file with blancs inside ?
The problem is not about reading "titles" (which is usually just the first line of the CSV file).
Instead, your problem (that causes the error) is that a line read x assigns a value to the variable named x (which can then be referenced via $x).
In your example you use read "Picture 1" which effectively tries to assign a value to the variable named Picture 1.
Since bash forbids variable names with spaces (just like probably any other non-esoteric programming language), this gives you an error.
The solution is to use legible and legit variable names:
while IFS=';' read edp pic1 fusel y
do
echo "Picture 1 is ${pic1}"
echo "Picture 2 is ${fusel}"
echo "Picture 3 is ${y}"
done
There are a number of naming-schemes for variable names, common ones include all lowercase with (or without) underscores to separate words, or CamelCase.
on "titles"
Your script doesn't know anything about what you refer to as "titles".
CSV doesn't know anything about "titles" either.
CSV is simply a format that has both lines (rows) and colums, as opposed to simpler text files that only have lines.
And just as a text-file doesn't have a notion of a "heading", a CSV file has no "titles".
Popular CSV-exporting software might, however, assign a special meaning to the very first row in a CSV-file and (ab)use it for title-information (so the content is only a label, whereas the actual column-content can be something different)
In any case, the IFS=';' read ... part of your script doesn't do anything with titles; it simply extracts multiple fields from a single line if input data and assigns these fields to variables.
The name of these variables can be totally arbitrary (as long as they conform to the bash syntax for variable names), and need not have anything to do with the content of the file.
I have ~ 60K bibliographic records, which can be identified by system number. These records also hold full-text (individudal text files named by system number).
I have lists of system numbers in bunches of 5K and I need to find a way to copy only the text files from each 5K list.
All text files are stored in a directory (/fulltext) and are named something along these lines:
014776324.txt.
The 5k lists are plain text stored in separated directories (e.g. /5k_list_1, 5k_list_2, ...), where each system number matches to a .txt file.
For example: bibliographic record 014776324 matches to 014776324.txt.
I am struggling to find a way to copy into the 5k_list_* folders only the corresponding text files.
Any idea?
Thanks indeed,
Let's assume we invoke the following script this way:
./the-script.sh fulltext 5k_list_1 5k_list_2 [...]
Or more succinctly:
./the-script.sh fulltext 5k_list_*
Then try using this (totally untested) script:
#!/usr/bin/env bash
set -eu # enable error checking
src_dir=$1 # first argument is where to copy files from
shift 1
for list_dir; do # implicitly consumes remaining args
while read bibliographic record sys_num rest; do
cp "$src_dir/$sys_num.txt" "$list_dir/"
done < "$list_dir/list.txt"
done
I am running cygwin on Windows 7. I am using a signal processing tool and basically performing alignments. I had about 1200 input files. Each file is of the format given below.
input_file_ format = "AC_XXXXXX.abc"
The first step required building some kind of indexes for all the input files, this was done with the tool's build-index command and now each file had 6 indexes associated with it. Therefore now I have about 1200*6 = 7200 index files. The indexes are of the form given below.
indexes_format = "AC_XXXXXX.abc.1",
"AC_XXXXXX.abc.2",
"AC_XXXXXX.abc.3",
"AC_XXXXXX.abc.4",
"AC_XXXXXX.abc.rev.1",
"AC_XXXXXX.abc.rev.1"
Now, I need to use these indexes to perform the alignment. All the 6 indexes of each file are called together and the final operation is done as follows.
signal-processing-tool ..\path-to-indexes\AC_XXXXXX.abc ..\Query file
Where AC_XXXXXX.abc is the index associated with that particular index file. All 6 index files are called with **AC_XXXXXX.abc*.
My problem is that I need to use only the first 14 characters of the index file names for the final operation.
When I use the code below, the alignment is not executed.
for file in indexes/*; do ./tool $file|cut -b1-14 Project/query_file; done
I'd appreciate help with this!
First of all, keep in mind that $file will always start with "indexes/", so trimming first 14 characters would always include that folder name in the beginning.
To use first 14 characters in a variable, use ${file:0:14}, where 0 is the starting string index, and 14 is the length of the desired substring.
Alternatively, if you want to use cut, you need to run it in a subshell: for file in indexes/*; do ./tool $(echo $file|cut -c 1-14) Project/query_file; done I changed the arg for cut to -c for characters instead of bytes