i am trying to use sed to do some text processing in a file called host
cluster_ip = "10.223.10.21"
srv_domain = "service_domain.svc"
cmd = f"'/^.*{srv_domain}/!p;$a'{cluster_ip}'\t{srv_domain}'"
Then I am calling it like this
subprocess.call(["/usr/bin/sed", "-i", "-ne", cmd, "host"])
But i am getting this error:
/usr/bin/sed: -e expression #1, char 1: unknown command: `''
Could someone please explain me what am I doing wrong?
Thank You
I also tried using fileinput but i am unable to print print(f"{cluster_ip}\t{srv_domain}\n") to the file instead this is going to the console.
cluster_ip = "123.234.45.5"
srv_domain = "service_domain.svc"
def main():
pattern = '^.*service_domain.svc'
filename = "host1"
matched = re.compile(pattern).search
with fileinput.FileInput(filename, inplace=1) as file:
for line in file:
if not matched(line): # save lines that do not match
print(line, end='') # this goes to filename due to inplace=1
# this is getting printed in console
print(f"{cluster_ip}\t{srv_domain}\n")
main()
I suppose you want to remove the first line and add a last line. You don't need to protect arguments, it's already done by the subprocess module. so you're getting the quotes literally.
quickfix:
cmd = f"/^.*{srv_domain}/!p;$a{cluster_ip}\t{srv_domain}"
better: learn to use python to avoid calling sed in your script and make them complex and non portable. You don't even need regexes here, just substring search (which could be improved with regexes to avoid substring match but the problem is already present in the original expression)
First read your file, drop the line where srv_domain is defined, and add your last line.
Something like this, using a temporary file to hold the modified contents, then overwriting it:
with open("hosts") as fr,open("hosts2","w") as fw:
for line in fr:
if not srv_domain in line:
fw.write(line)
fw.write(f"{cluster_ip}\t{srv_domain}\n")
os.remove("hosts")
os.rename("hosts2","hosts")
Related
This is the first time I am trying to do python execution in GNU parallel.
I have the below python script. I am trying to run it in parallel with a text.txt document loading the variables. The text document has the variables one on each line.
I execute the below script with this code:
parallel --bar -a PairNames.txt python3 CreateDataTablePythonScriptv2.py
Here is the python script being executed:
import sqlite3
import sys
PairName = sys.argv[1]
print(PairName)
DTBLocation = '//mnt//c//Users//Jonathan//OneDrive - Mazars in Oman//Trading//Systems//FibMatrix//Testing Trade Analysis//SQLite//Trade Analysis.db
connection = sqlite3.connect(DTBLocation)
cursor = connection.cursor()
TableName = PairName+'_DATA'
print(TableName)
cursor.execute("""CREATE TABLE IF NOT EXISTS {}
(
Date_Time INTEGER,
Open REAL,
Max_60m_Box REAL
)""".format(TableName))
connection.commit()
connection.close()
It executes correctly the first variable just fine. But the remainder of the variables do print correctly from the print command for the PairName, but for print(TableName) I get the below displays:
GBPUSD
_DATAD
USDCHF
_DATAF
NZDJPY
_DATAY
Its weird to me that it prints the PairName just fine and correctly, but then the PairName does not show up when concating the TableName.
Also, its weird that an extra letter gets added to the end of DATA for each one. It appears that the extra letter at the end of the DATA is the last letter of the input variable. I don't know why its choping the 5 letters off and how it puts it at the end of the DATA.
I printed the tablename.
I watched this video at https://www.youtube.com/watch?v=OpaiGYxkSuQ&ab_channel=OleTange[^]
I tried moving the TableName concat to right under the PairName
I printed the type of the PairName, and it is a string
I tried seperating the varibales in the txt document by tabs and commas instead of next line
I tried assigning the "_DATA" to a variable and then concating the two objects. But it had same result:
TableEnd = '_DATA'
TableName = PairName + TableEnd
If I remove the concat of PairName+'_DATA' and just use PairName only as the TableName, then it works correctly.
Sorry if this is a simple answer, but I cannot figure it out and especially since there is not too much documentation / tutorials for a newbie on GNU Parallel in this situation. Thanks for the help!
The input file is not in DOS format (i.e. ends in a CRLF rather than just an LF)? I checked this using the FILE command:
$ file test.txt
test.txt: ASCII text, with CRLF line terminators
$
Since it was CRLF (DOS format), I converted it using tr:
Copy Codetr -d '\r' < input.file > output.file```
Sorry if this is a duplicate/basic question, I couldn't find any similar questions.
I have the following multiline string
my_txt = """
foo.exe\n
bar.exec\n
abab.exe\n
"""
(The newlines aren't actually written in my code, I put them there for clarity).
I want to match every file that ends with a .exe, (not .exec).
My regex was initially:
my_reg = re.compile(".+[.](?=exe$)")
my_matches = my_reg.finditer(my_txt)
I hoped that it would first find every character, go back until it found the ., and then check if the characters exe and a newline followed.
Only one match was found, and that was:
abab.exe.
I tried to mess around a bit, and changed the first line:
my_reg = re.compile(".+[.](?=exe$)",flags=re.MULTILINE).
This time, it successfully ran, returning
foo.
abab.
I thought re.MULTILINE wasn't supposed to interfere with the $ operator, or am I wrong about the $ operator/misusing something?
Thanks in advance!
You do need the multiline flag, otherwise $ will only match the absolute end of your input. You just need to match exe instead of using a lookahead:
my_reg = re.compile(".+[.]exe$", re.MULTILINE)
Output:
['foo.exe', 'abab.exe']
Demo
If you are trying to match the filename without the extension, you can put the period inside the lookahead:
my_reg = re.compile(r".+(?=\.exe$)", re.MULTILINE)
Output:
['foo', 'abab']
Demo
Here is a snippet from a text file I am working on.
http://pastebin.com/4Uba5i4P
I would like to use python to detect those big repeating "~ Move" lines (Which are not identical except for the "~ Move" part.), and remove all but the first and last of those lines.
How I would I start to go about this?
You could read the file line by line like this:
`## Open the file with read only permit
f = open('myTextFile.txt')
## Read the first line
line = f.readline()
## If the file is not empty keep reading line one at a time #
# till the file is empty while line: print line
line = f.readline() f.close()`
With this you could then edit this sample to test each line using a regex like this:
`if line.find("~Move") == -1:
Break;
Else:
Line=Line [5:-1]`
Though this assumes that the ~Move is all at the beginning of the line. Hope this helps, if not leave a comment and I'll try and help.
I am trying to create a script that takes a file as input, looks up all the email addresses, and writes them to a designated file.
based on other similar questions, i have ended up with this:
import re
Input = open("inputdata.txt", "r")
regex = re.compile("\b[A-Z0-9._%+-]+#[A-Z0-9.-]+\.[A-Z]{2,4}\b")
Logfile = "Result.txt"
for line in Input:
query = regex.findall(line)
for line in query:
print >>Logfile, query
what am i doing wrong? this outputs nothing.
i am guessing the main problem is "for line in query:", which i have tried changing without any luck.
Cheers!
Edit: i have changed the script as suggested below, with "print (query)" instead.
i still do not get any output.
current script is:
import re
Input = open("Inputdata.txt", "r")
regex = re.compile("\b[A-Z0-9._%+-]+#[A-Z0-9.-]+\.[A-Z]{2,4}\b")
# logfile = "Result.txt"
for line in Input:
query = regex.findall(line)
for line in query:
with open("Result.txt", "a") as logfile:
logfile.write(line)
It outputs nothing, and tells me: " NameError: name "logfile" not defined".
what causes this, and is this the reason there is no output?
Your Logfile variable is just the name of a file, not an actual file object. Also, you should use with to automatically close the file when you are done. Try this:
with open("Result.txt", "a") as logfile:
print >>logfile, "hello world"
print >>logfile, "another line"
But note that in Python 3.x, the syntax is different, as print is no longer a statement but a function:
with open("Result.txt", "a") as logfile:
print("hello world", file=logfile)
print("another line", file=logfile)
Thus, instead of redirecting print, the best choice might be to write to the file directly:
with open("Result.txt", "a") as logfile:
logfile.write("hello world\n")
logfile.write("another line\n")
I don't think, with print you can write to a file without redirecting the output to a file. The way you have used the print, I guess, you want output redirection only.
Let's say your python script is in a file test.py.
replace the line:
print >>Logfile, query
with just:
print query
And from a terminal/cmd, run the script like this:
python test.py >> Result.txt
This is called output redirection.
First year Comp Sci student here.
I have an assignment that is asking us to make a simple game using Python, which takes an input file to create the game-world (2D grid). You're then supposed to give movement commands via user input afterwards. My program reads the input file one line at a time to create the world using:
def getFile():
try:
line = input()
except EOFError:
line = EOF
return line
...after which it creates a list to represent the line, with each member being a character in the line, and then creates a list containing each of these lists (amounting to a grid with row and column coordinates).
The thing is, I later need to take input in order to move the character, and I can't do this because it still wants to read the file input, and the last line from the file is an EOF character, causing an error. Specifically the "EOF when reading a line" error.
How can I get around this?
Sounds like you are reading the file directly from stdin -- something like:
python3 my_game.py < game_world.txt
Instead, you need to pass the file name as an argument to your program, that way stdin will still be connected to the console:
python3 my_game.py game_world.txt
and then get_file looks more like:
def getFile(file_name):
with open(file_name) as fh:
for line in fh:
return line
File interaction is python3 goes like this:
# the open keyword opens a file in read-only mode by default
f = open("path/to/file.txt")
# read all the lines in the file and return them in a list
lines = f.readlines()
#or iterate them at the same time
for line in f:
#now get each character from each line
for char_in_line in line:
#do something
#close file
f.close()
line terminator for the file is by default \n
If you want something else you pass it as a parameter to the open method (the newline parameter. Default=None='\n'):
open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None)