I am trying to write an automation code for picking up different environment values for execution of my testcases based on the value I pass for environment.
Here is the code I tried :
# env.robot
*** Settings ***
Variables setup.py stage01
*** Test Cases ***
Print values
log to console ${data}
# setup.py
from robot.libraries.BuiltIn import BuiltIn
import xlrd
def get_variables(env):
file_location = "values.xlsx"
workbook = xlrd.open_workbook(file_location)
sheet = workbook.sheet_by_name(env)
print("Env : " + sheet.name)
data = [[sheet.cell_value(r, c) for c in range(sheet.ncols)] for r in range(sheet.nrows)]
print(data)
BuiltIn().log_to_console(data)
return data
Response I am getting :
[ ERROR ] Error in file 'D:\env.robot': Processing variable file 'D:\setup2.py' with arguments [ stage01 ]
failed: FileNotFoundError: [Errno 2] No such file or directory: 'values.xlsx'
values.xlsx is present in the same directory with .py and .robot file
I want to get the data from values.xlsx file based on the value of env variable and use the values in robot testcases.
Please suggest what I need to modify or any other approach would also do.
Related
I am trying an assignment to open a syslog for a server program (called ticky) that creates logs and errors and then assign the errors to a dictionary and export to a csv file to sort and host to a webpage,
I am unsure of how to access syslog files, as the course only went into sys.argv and I don't know if this can be used or if I need to figure out how to use syslog module. Once the log is opened, the regex will pull the error message and add it to the dictionary, either creating a new entry or adding value to an existing key.
Am I on the right track?
#!/usr/bin/env python3
import re
import sys
errors = {}
# log line format
# Jun 1 11:06:48 ubuntu.local ticky: ERROR: Connection to DB failed (username)
logfile = sys.argv[1]
# NOTE: Check to find correct log file
with open(logfile) as f:
for line in f:
if "ERROR:" not in line:
continue
regex_error = r"ERROR: (\d+) "
"""searches for error messages"""
error = re.search(regex_error, line)
if error is None:
continue
name = error[1]
errors[name] = errors.get(name, 0) + 1
I have this block of code that works in python script form but when I package the script to an exe using pyinstaller it always results in the program saying the config file can't be found. I put the config.ini in the same folder as the exe file.
config = configparser.ConfigParser()
configComplete = True
configExists = False
try:
open(os.path.join(sys.path[0],'config.ini'))
config.read(os.path.join(sys.path[0],'config.ini'))
destination = config['server']['ServerAddress']
key = config['server']['ApiKey']
configExists = True
except KeyError:
configComplete = False
except FileNotFoundError:
try:
open(expanduser('~/.config/octoprint-cli.ini'))
config.read(expanduser('~/.config/octoprint-cli.ini'))
destination = config['server']['ServerAddress']
key = config['server']['ApiKey']
configExists = True
except KeyError:
configComplete = False
except FileNotFoundError:
pass
I don't have python currently installed to test this on my machine, but typically when I am looking for a file that's relative to the python file location it's preferable to use:
import os
CONFIG_FILE_PATH = f"{os.path.dirname(__file__)}{os.sep}config.ini"
if os.path.exists(CONFIG_FILE_PATH): # If the file already exists
config.read(CONFIG_FILE_PATH) # Read it
else: # If a config file does not exist
# Either throw error or create fresh config
This code is an OS agnostic way of looking for a file in the same directory as the python file and the catching doesn't necessarily throw an error unless you want it to.
See if that works with pyinstaller, as I believe when I last used it this worked.
I'm running pytest to check a function that uses pdfminer to convert PDF to text. The function works when doing $ python function.py and the result is what I expect it to be. I should also point out that I'm using a stream when parsing the file (io.BytesIO) and this stream is the reason my test fails.
Running pytest the function fails with a PDFSyntaxError.
# function.py
...
from pdfminer.pdfparser import PDFParser
from pdfminer.document import PDFDocument
req = requests.get(url_pointing_to_pdf_file)
pdf = io.BytesIO(req.content)
parser = PDFParser(pdf)
document = PDFDocument(parser, password=None) # this fails
...
pytest calls the init method in pdfdocument.py (part of the pdfminer library) and stops here:
for xref in xrefs:
trailer = xref.get_trailer()
...
if 'Root' in trailer:
self.catalog = dict_value(trailer['Root'])
break
else:
raise PDFSyntaxError('No /Root object! - Is this really a PDF?')
...
And this is what pytest shows when testing the function fails:
tests/test_function.py:11:
----------------------------------------------------
.../function.py:157: in function
**document = PDFDocument(parser, password=None)**
...
E pdfminer.pdfparser.PDFSyntaxError: No /Root object! - Is this really a PDF?
lib/python3.6/site-packages/pdfminer/pdfdocument.py:583:PDFSyntaxError
Running the test with a PDF file stored in the same directory as function.py is successful, so the culprit is the io.BytesIO format of the downloaded PDF file. Since I want to use a stream with function.py I would like to know if there is a better way to do this.
I can able to rename a file without any problem/error using os.rename().
But the moment I tried to rename a file with timestamp adding to it, it throws win3 error or win123 error, tried all combinations but no luck, Could anyone help.
Successfully Ran Code :
#!/usr/bin/python
import datetime
import os
import shutil
import json
import re
maindir = "F:/Protocols/"
os.chdir(maindir)
maindir = os.getcwd()
print("Working Directory : "+maindir)
path_4_all_iter = os.path.abspath("all_iteration.txt")
now = datetime.datetime.now()
timestamp = str(now.strftime("%Y%m%d_%H:%M:%S"))
print(type(timestamp))
archive_name = "all_iteration_"+timestamp+".txt"
print(archive_name)
print(os.getcwd())
if os.path.exists("all_iteration.txt"):
print("File Exists")
os.rename(path_4_all_iter, "F:/Protocols/archive/archive.txt")
print(os.listdir("F:/Protocols/archive/"))
print(os.path.abspath("all_iteration.txt"))
Log :
E:\python.exe C:/Users/SPAR/PycharmProjects/Sample/debug.py
Working Directory : F:\Protocols
<class 'str'>
all_iteration_20180409_20:25:51.txt
F:\Protocols
File Exists
['archive.txt']
F:\Protocols\all_iteration.txt
Process finished with exit code 0
Error Code :
print(os.getcwd())
if os.path.exists("all_iteration.txt"):
print("File Exists")
os.rename(path_4_all_iter, "F:/Protocols/archive/"+archive_name)
print(os.listdir("F:/Protocols/archive/"))
print(os.path.abspath("all_iteration.txt"))
Error LOG:
E:\python.exe C:/Users/SPAR/PycharmProjects/Sample/debug.py
Traceback (most recent call last):
Working Directory : F:\Protocols
<class 'str'>
File "C:/Users/SPAR/PycharmProjects/Sample/debug.py", line 22, in <module>
all_iteration_20180409_20:31:16.txt
F:\Protocols
os.rename(path_4_all_iter, "F:/Protocols/archive/"+archive_name)
File Exists
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'F:\\Protocols\\all_iteration.txt' -> 'F:/Protocols/archive/all_iteration_20180409_20:31:16.txt'
Process finished with exit code 1
Your timestamp format has colons in it, which are not allowed in Windows filenames. See this answer on that subject:
How to get a file in Windows with a colon in the filename?
If you change your timestamp format to something like:
timestamp = str(now.strftime("%Y%m%d_%H-%M-%S"))
it should work.
You can't have : characters as part of the filename, so change
timestamp = str(now.strftime("%Y%m%d_%H:%M:%S"))
to
timestamp = str(now.strftime("%Y%m%d_%H%M%S"))
and you'll be able to rename your file.
I've been trying to use the Mafft alignment tool from Bio.Align.Applications. Currently, I've had success writing my sequence information out to temporary text files that are then read by MafftCommandline(). However, I'd like to avoid redundant steps as much as possible, so I've been trying to write to a memory file instead using io.StringIO(). This is where I've been having problems. I can't get MafftCommandline() to read internal files made by io.StringIO(). I've confirmed that the internal files are compatible with functions such as AlignIO.read(). The following is my test code:
from Bio.Align.Applications import MafftCommandline
from Bio import SeqIO
from Bio.Seq import Seq
from Bio.SeqRecord import SeqRecord
import io
from Bio import AlignIO
sequences1 = ["AGGGGC",
"AGGGC",
"AGGGGGC",
"AGGAGC",
"AGGGGG"]
longest_length = max(len(s) for s in sequences1)
padded_sequences = [s.ljust(longest_length, '-') for s in sequences1] #padded sequences used to test compatibilty with AlignIO
ioSeq = ''
for items in padded_sequences:
ioSeq += '>unknown\n'
ioSeq += items + '\n'
newC = io.StringIO(ioSeq)
cLoc = str(newC).strip()
cLocEdit = cLoc[:len(cLoc)] #create string to remove < and >
test1Handle = AlignIO.read(newC, "fasta")
#test1HandleString = AlignIO.read(cLocEdit, "fasta") #fails to interpret cLocEdit string
records = (SeqRecord(Seq(s)) for s in padded_sequences)
SeqIO.write(records, "msa_example.fasta", "fasta")
test1Handle1 = AlignIO.read("msa_example.fasta", "fasta") #alignIO same for both #demonstrates working AlignIO
in_file = '.../msa_example.fasta'
mafft_exe = '/usr/local/bin/mafft'
mafft_cline = MafftCommandline(mafft_exe, input=in_file) #have to change file path
mafft_cline1 = MafftCommandline(mafft_exe, input=cLocEdit) #fails to read string (same as AlignIO)
mafft_cline2 = MafftCommandline(mafft_exe, input=newC)
stdout, stderr = mafft_cline()
print(stdout) #corresponds to MafftCommandline with input file
stdout1, stderr1 = mafft_cline1()
print(stdout1) #corresponds to MafftCommandline with internal file
I get the following error messages:
ApplicationError: Non-zero return code 2 from '/usr/local/bin/mafft <_io.StringIO object at 0x10f439798>', message "/bin/sh: -c: line 0: syntax error near unexpected token `newline'"
I believe this results due to the arrows ('<' and '>') present in the file path.
ApplicationError: Non-zero return code 1 from '/usr/local/bin/mafft "_io.StringIO object at 0x10f439af8"', message '/usr/local/bin/mafft: Cannot open _io.StringIO object at 0x10f439af8.'
Attempting to remove the arrows by converting the file path to a string and indexing resulted in the above error.
Ultimately my goal is to reduce computation time. I hope to accomplish this by calling internal memory instead of writing out to a separate text file. Any advice or feedback regarding my goal is much appreciated. Thanks in advance.
I can't get MafftCommandline() to read internal files made by
io.StringIO().
This is not surprising for a couple of reasons:
As you're aware, Biopython doesn't implement Mafft, it simply
provides a convenient interface to setup a call to mafft in
/usr/local/bin. The mafft executable runs as a separate process
that does not have access to your Python program's internal memory,
including your StringIO file.
The mafft program only works with an input file, it doesn't even
allow stdin as a data source. (Though it does allow stdout as a
data sink.) So ultimately, there must be a file in the file system
for mafft to open. Thus the need for your temporary file.
Perhaps tempfile.NamedTemporaryFile() or tempfile.mkstemp() might be a reasonable compromise.