I have a simple python app and i'm trying to combine bunch of output messages to standardize output to the user. I've created a properties file for this, and it looks similar to the following:
[migration_prepare]
console=The migration prepare phase failed in {stage_name} with error {error}!
email=The migration prepare phase failed while in {stage_name}. Contact support!
slack=The **_prepare_** phase of the migration failed
I created a method to handle fetching messages from a Properties file... similar to:
def get_msg(category, message_key, prop_file_location="messages.properties"):
""" Get a string from a properties file that is utilized similar to a dictionary and be used in subsequent
messaging between console, slack and email communications"""
message = None
config = ConfigParser()
try:
dataset = config.read(prop_file_location)
if len(dataset) == 0:
raise ValueError("failed to find property file")
message = config.get(category, message_key).replace('\\n', '\n') # if contains newline characters i.e. \n
except NoOptionError as no:
print(
f"Bad option for value {message_key}")
print(f"{no}")
except NoSectionError as ns:
print(
f"There is no section in the properties file {prop_file_location} that contains category {category}!")
print(f"{ns}")
return f"{message}"
The method returns the F string fine, to the calling class. My question is, in the calling class if the string in my properties file contains text {some_value} that is intended to be interpolated by the compiler in the calling class using an F String with curly brackets, why does it return a string literal? The output is literal text, not the interpolated value I expect:
What I get The migration prepare phase failed while in {stage_name} stage. Contact support!
What I would like The migration prepare phase failed while in Reconciliation stage. Contact support!
I would like the output from the method to return the interpolated value. Has anyone done anything like this?
I am not sure where you define your stage_name but in order to interpolate in config file you need to use ${stage_name}
Interpolation in f-strings and configParser files are not the same.
Update: added 2 usage examples:
# ${} option using ExtendedInterpolation
from configparser import ConfigParser, ExtendedInterpolation
parser = ConfigParser(interpolation=ExtendedInterpolation())
parser.read_string('[example]\n'
'x=1\n'
'y=${x}')
print(parser['example']['y']) # y = '1'
# another option - %()s
from configparser import ConfigParser, ExtendedInterpolation
parser = ConfigParser()
parser.read_string('[example]\n'
'x=1\n'
'y=%(x)s')
print(parser['example']['y']) # y = '1'
Related
I am attempting to sort a dataframe by a column called 'GameId', which are currently of type string and when I attempt to sort the result is unexpected. I have tried the following but still return a type string.
TEST['GameId'] = TEST['GameId'].astype(int)
type('GameId')
One way to make the data life easier is using dataclasses!
from dataclasses import dataclass
# here will will be calling the dataclass decorator to send hints for data type!
#dataclass
class Columns:
channel_id : int
frequency_hz : int
power_dBmV : float
name : str
# this class will call the data class to organise the data as data.frequency data.power_dBmV etc
class RadioChannel:
radio_values = ['channel_id', 'frequency', 'power_dBmV']
def __init__(self, data): # self is 'this' but for python, it just means that you mean to reference 'this' or self instance
self.data = data # this instances data is called data here
data = Columns(channel_id=data[0], frequency=data[1], power_dBmv=data[4], name=data[3]) # now we give data var a val!
def present_data(self):
# this is optional class method btw
from rich.console import Console
from rich.table import Table
console = Console()
table = Table(title="My Radio Channels")
for item in self.radio_values:
table.add_column(item)
table.add_row(data.channel_id, data.frequency_hz, data.power_dBmv)
console.print(table)
# ignore this if its confusing
# now inside your functional part of your script
if __name__ == '__main__':
myData = []
# calling an imaginary file here to read
with open("my_radio_data_file", 'r') as myfile:
mylines = myfile.readlines()
for line in myline:
myData.append(line)
myfile.close()
#my data would look like a string ["value", value, 00, 0.0, "hello joe, from: world"]
ch1 = radioChannel(data=myData[0])
ch1.present_data()
This way you can just call the class object on each line of a data file. and print it to see if it lines up. once you get the hang of it, it starts to get fun.
I used rich console here, but it works well with pandas and normal dataframes!
dataclasses help the interpreter find its way with type hints and class structure.
Good Luck and have fun!
1) In my response body comes like json format.
2) Some expected reason ,i have changed that body json to normal text using below code and working expected way
import groovy.json.*
String js = vars.get("cAccountDetails")
def data = new JsonSlurper().parseText(js)
log.info("the value is "+ data)
vars.putObject('data', data)
3) This code meaning converted json to normal text and stored in some variable thats "data"
4) so my response stored in "data" variable .
5) From "data", how can i extract **specific data** using groovy code or some other code?
import java.util.regex.*
import java.util.regex.Matcher
import java.util.regex.Pattern
def matches = (data =~ '{accountDetails=\\[(.*)\\],')
vars.putObject('matches', matches)
The above code using for correlation purpose {"matches" VARIABLE will store extracted value}
but above code is not working ,how can i fix this issue ?
Thanks in advance!!
We cannot help you unless you share your cAccountDetails variable value and indicate what do you need to extract from it.
From the first glance you regular expression should look a little bit different, i.e.
def matches = (data =~ /accountDetails=[(.*)],/)
More information:
Apache Groovy - Find Operator
Apache Groovy - Why and How You Should Use It
So I am quite new to mocking. I think I need to mock two functions.
Function under test
def get_text_from_pdf(next_pdfs_path):
# TODO test this function
"""
pulls all text from a PDF document and returns as a string.
Parameters:
next_pdfs_path (str): file path use r"" around path.
Returns:
text (str): string of text
"""
if os.path.isfile(next_pdfs_path): # check file is a real file/filepath
try:
text = ''
with fitz.open(next_pdfs_path) as doc: # using PyMuPDF
for page in doc:
text += page.getText()
return text
except (RuntimeError, IOError):
pass
pass
test code first try
from unittest import mock
#mock.patch("content_production.fitz.open", return_value='fake_file.csv', autospec=True)
def test_get_text_from_pdf(mock_fitz_open):
assert cp.get_text_from_pdf('fake_file.csv') == 'fake_file.csv'
error
E AssertionError: assert None == 'fake_file.csv'
E + where None = <function get_text_from_pdf at 0x00000245EDF8CAF0>('fake_file.csv')
E + where <function get_text_from_pdf at 0x00000245EDF8CAF0> = cp.get_text_from_pdf
Do I need to mock both fitz.open and os.path.isfile? How could that be done if yes?
EDIT
Following njriasan feedback I have tried this
#mock.patch("content_production.os.path.isfile", return_value=True, autospec=True)
#mock.patch("content_production.fitz.Page.getText")
#mock.patch("content_production.fitz.open")
def test_get_text_from_pdf(mock_fitz_open, mock_path_isfile, mock_gettext):
mock_fitz_open.return_value.__enter__.return_value = 'test'
assert cp.get_text_from_pdf('test') == 'test'
But now getting this error.
> text += page.getText()
E AttributeError: 'str' object has no attribute 'getText'
I think there are a couple issues with what you are doing. The first problem I see is that I think you are mocking the wrong function. By mocking fitz.open(next_pdfs_path) you are still expecting:
for page in doc:
text += page.getText()
to execute properly. I'd suggest that you wrap this entire with statement and text result updating in a helper function and then mock that. If the file path doesn't actually exist on your system then you will also need to mock os.path.isfile. I believe that can be done by adding a second decorator (I don't think there is any limit).
Problem statement
I want the options supported in a python module to be overridable with an .yaml file, because in some cases there are too many options to be specified with non-default values.
I implemented the logic as follows.
parser = argparse.ArgumentParser()
# some parser.add statements that comes with default values
parser.add_argument("--config_path", default=None, type=str,
help="A yaml file for overriding parameters specification in this module.")
args = parser.parse_args()
# Override parameters
if args.config_path is not None:
with open(args.config_path, "r") as f:
yml_config = yaml.safe_load(f)
for k, v in yml_config.items():
if k in args.__dict__:
args.__dict__[k] = v
else:
sys.stderr.write("Ignored unknown parameter {} in yaml.\n".format(k))
The problem is, for some options I have specific functions/lambda expressions to convert the input strings, such as:
parser.add_argument("--tokens", type=lambda x: x.split(","))
In order to apply corresponding functions when parsing option specifications in YAML, adding so many if statements does not seem a good solution. Maintaining a dictionary that changes accordingly when new options are introduced in parser object seems redundant. Is there any solution to get the type for each argument in parser object?
If the elements that you add to the parser with add_argument start with -- then they
are actually optional and usually called options. You can find these walking over
the result of the _get_optonal_actions() method of the parser instance.
If you config.yaml looks like:
tokens: a,b,c
, then you can do:
import sys
import argparse
import ruamel.yaml
sys.argv[1:] = ['--config-path', 'config.yaml'] # simulate commandline
yaml = ruamel.yaml.YAML(typ='safe')
parser = argparse.ArgumentParser()
parser.add_argument("--config-path", default=None, type=str,
help="A yaml file for overriding parameters specification in this module.")
parser.add_argument("--tokens", type=lambda x: x.split(","))
args = parser.parse_args()
def find_option_type(key, parser):
for opt in parser._get_optional_actions():
if ('--' + key) in opt.option_strings:
return opt.type
raise ValueError
if args.config_path is not None:
with open(args.config_path, "r") as f:
yml_config = yaml.load(f)
for k, v in yml_config.items():
if k in args.__dict__:
typ = find_option_type(k, parser)
args.__dict__[k] = typ(v)
else:
sys.stderr.write("Ignored unknown parameter {} in yaml.\n".format(k))
print(args)
which gives:
Namespace(config_path='config.yaml', tokens=['a', 'b', 'c'])
Please note:
I am using the new API of ruamel.yaml. Loading this way is actually faster
than using from ruamel import yaml; yaml.safe_load() although your config files
are probably not big enough to notice.
I am using the file extension .yaml, as recommended in the official FAQ. You should do so as well, unless you cannot (e.g. if your
filesystem doesn't allow that).
I use a dash in the option string --config-path instead of an underscore, this is
somewhat more natural to type and automatically converted to an underscore to get valid
identifier name
You might want to consider a different approach where you parse sys.argv by hand for
--config-path, then set the defaults for all the options from the YAML config file and
then call .parse_args(). Doing things in that order allow you to override, on
the commandline, what has a value in the config file. If you do things your way, you always
have to edit a config file if it has all correct values except for one.
I am using SoapUI to test webservices. The following string (in xml format) is my request:
<Request>
<AC>
<AssetID>1</AssetID>
<Asset_Name>ABC</Asset_Name>
<Asset_Number>1</Asset_Number>
</AC>
<AC>
<AssetID>2</AssetID>
<Asset_Name>XYZ</Asset_Name>
<Asset_Number>2</Asset_Number>
</Ac>
</Request>
I am using the following code in a groovy script to extract value of Asset_Number for each AC (The above xml string is stored in variable strRequest):
def x = new XmlSlurper().parseText("$strRequest")
x.AC.each { AC ->
assetNum = AC."Asset_Number"
<<do something with the assetNum>>
}
However, I wish to parameterize the above code to pick up Asset_Number for various types of assets (e.g. AC, Peripheral etc). The request xml for each asset is in the same format as above. If I replace 'AC' with variable name 'requestName' in above code:
//strRequest = xml request
def requestName //I will pick up value for this from a test case property
def x = new XmlSlurper().parseText("$strRequest")
x.(requestName.toString()).each { requestName ->
assetNum = requestName."Asset_Number"
<<do something with the assetNum>>
}
it shows the following error:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: Script166.groovy: 35: The current scope already contains a variable of the name requestName # line 35, column 2. { ^ org.codehaus.groovy.syntax.SyntaxException: The current scope already contains a variable of the name requestName
I have tried a solution mentioned in another post Using a String as code with Groovy XML Parser, but it doesn't serve my purpose.
Any other ideas?
You can use
x."$requestName".each
Why XmlSlurper? In SoapUI you can use XmlHolder
import com.eviware.soapui.support.XmlHolder
def responseHolder = new XmlHolder(messageExchange.getResponseContentAsXml())
def resultFromResponse = responseHolder["here_is_your_xpath"]
assert someExpectedResult == resultFromResponse
And if you need to iterate via multiple xml nodes
resultFromResponse.each
{
assert result == resultFromResponse
}