I'm trying to sort and enumerate(after this issue is resolved) parsed data from craigslist using sorted() to sort my values by the newest posting in elapsed minutes. I've got a feeling that it's my f' string causing the issue, but I'm not sure.
Expected Output:
[5712]: Mar 26: Master Licensed Electricians
[9987]: Mar 23: Microsoft O365/Exchange Consultant:
[27101]: Mar 11: Technicians needed immediately!:
[41942]: Mar 1: IT Support Technician:
[41945]: Mar 1: Network Hardware and Support Engineer:
Resulting Output:
[5712]: Mar 26: Master Licensed Electricians
[41942]: Mar 1: IT Support Technician:
[41945]: Mar 1: Network Hardware and Support Engineer:
[9987]: Mar 23: Microsoft O365/Exchange Consultant:
[27101]: Mar 11: Technicians needed immediately!:
Code block:
def outputEntries(posts):
for post in (posts):
titleDiv = post.find('a', class_='result-title')
postTitle = titleDiv.get_text()
postURL = titleDiv.get('href')
postDate = post.find('time', title_=[]).get_text()
postTimeText = post.find('time').get('datetime')
postTime = datetime.strptime(postTimeText, '%Y-%m-%d %H:%M')
elapsedMinutes = (datetime.now() - postTime).total_seconds() / 60
rounded_EM = [math.ceil(elapsedMinutes)]
#sorted_EM = sorted(rounded_EM, key=lambda x:x[1])
#sorted(rounded_EM, key=int)
print(f'{sorted(rounded_EM, key=int)}: {postDate}: {postTitle}: {postURL}')
Comments are saved methods of sorting/enumerating for troubleshooting.
Related
I'm trying to run background task processing, redis and rabbitMQ work in separate docker containers
#dramatiq.actor(store_results=True)
def count_words(url):
try:
response = requests.get(url)
count = len(response.text.split(" "))
print(f"There are {count} words at {url!r}.")
except requests.exceptions.MissingSchema:
print(f"Message dropped due to invalid url: {url!r}")
result_backend = RedisBackend(host="172.17.0.2", port=6379)
result_broker = RabbitmqBroker(host="172.17.0.5", port=5672)
result_broker.add_middleware(Results(backend=result_backend))
dramatiq.set_broker(result_broker)
message = count_words.send('https://github.com/Bogdanp/dramatiq')
print(message.get_result(block=True))
RabbitMQ:
{"queue_name":"default","actor_name":"count_words","args":["https://github.com/Bogdanp/dramatiq"],"kwargs":{},"options":{},"message_id":"8e10b6ef-dfef-47dc-9f28-c6e07493efe4","message_timestamp":1608877514655}
Redis
1:C 22 Dec 2020 13:38:15.415 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
1:M 22 Dec 2020 13:38:15.417 * Running mode=standalone, port=6379.
1:M 22 Dec 2020 13:38:15.417 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 22 Dec 2020 13:38:15.417 # Server initialized
1:M 22 Dec 2020 13:38:15.417 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 22 Dec 2020 13:38:15.417 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled (set to 'madvise' or 'never').
1:M 25 Dec 2020 10:08:12.274 * Background saving terminated with success
1:M 26 Dec 2020 19:23:59.445 * 1 changes in 3600 seconds. Saving...
1:M 26 Dec 2020 19:23:59.660 * Background saving started by pid 24
24:C 26 Dec 2020 19:23:59.890 * DB saved on disk
24:C 26 Dec 2020 19:23:59.905 * RDB: 4 MB of memory used by copy-on-write
1:M 26 Dec 2020 19:23:59.961 * Background saving terminated with success
Error:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/dramatiq/message.py", line 147, in get_result
return backend.get_result(self, block=block, timeout=timeout)
File "/usr/local/lib/python3.6/dist-packages/dramatiq/results/backends/redis.py", line 81, in get_result
raise ResultTimeout(message)
dramatiq.results.errors.ResultTimeout: count_words('https://github.com/Bogdanp/dramatiq')
L.S,
i am trying to learn python(3) and am baffled by the behavior of time.sleep()
i run raspbian buster on a rpi3b.
#!/usr/bin/python3
import bluetooth
import sys
import time
_shuttermac ="11:22:33:44:5D:6A"
while True:
# check if the device is switched on by mac address lookup
print("Checking " + time.strftime("%a, %d %b %Y %H:%M:%S - ", time.gmtime()),end ="")
result = bluetooth.lookup_name(_shuttermac, timeout=5)
#print(result)
if (result != None):
print("Device detected by bluetooth mac lookup")
else:
print("Device with MAC " + _shuttermac + " NOT detected")
time.sleep(1) # check every 5 secs if device comes online
sys.exit()
output
Checking Thu, 25 Jun 2020 11:21:02 - Device with MAC 11:22:33:44:5D:6A NOT detected
Checking Thu, 25 Jun 2020 11:21:08 - Device with MAC 11:22:33:44:5D:6A NOT detected
Checking Thu, 25 Jun 2020 11:21:14 - Device with MAC 11:22:33:44:5D:6A NOT detected
Checking Thu, 25 Jun 2020 11:21:20 - Device with MAC 11:22:33:44:5D:6A NOT detected
Checking Thu, 25 Jun 2020 11:21:26 - Device with MAC 11:22:33:44:5D:6A NOT detected
as you can see, the output statement is 6 seconds apart, not 1 as coded
where did i go wrong?
The six second interval comes because each loop of while True does two time-consuming things when the lookup fails:
bluetooth.lookup_name(_shuttermac, timeout=5) has a 5 second timeout. If it is found quickly, it should complete faster
time.sleep(1) pauses the whole Python program for 1 second
You could shorten the timeout on bluetooth.lookup_name and the sleep() duration. This may have unintended consequences in the accuracy of the lookup (I'm not very familiar with it).
Here I am just taking value(integer) from Properties file and using the same in for loop.
Note : If I use direct number instead of "getTestCasePropertyValue" value it work as expected. Not getting how loop is looping it 50 times.
Groovy script:
def getTestCasePropertyValue = testRunner.testCase.getPropertyValue( "NumOfPayments" )
log.info(getTestCasePropertyValue )
for(i=0; i<=getTestCasePropertyValue; i++)
{
log.info("Test Print"+i)
}
Output:
Fri Mar 06 12:58:47 IST 2020:INFO:2
Fri Mar 06 12:58:47 IST 2020:INFO:Test Print0
Fri Mar 06 12:58:47 IST 2020:INFO:Test Print1
Fri Mar 06 12:58:47 IST 2020:INFO:Test Print2
Fri Mar 06 12:58:47 IST 2020:INFO:Test Print3
...
Fri Mar 06 12:58:47 IST 2020:INFO:Test Print50
Your value from the properties is a String. You will detect problems like this easier, if you use .inspect() to log things.
Also the character '2' is 50 as integer, which then the for loop conditions casts this too.
def getTestCasePropertyValue = "2"
println(getTestCasePropertyValue.inspect())
// → '2'
println(getTestCasePropertyValue as char as int)
// → 50
So best explicitly cast to a number using e.g. .toLong() on the string:
println(getTestCasePropertyValue.toLong().inspect())
// → 2
I have a strange problem using groovy, I found a workaround but I'm quite not satisfied so maybe someone will be able to help me:
I use ReadyAPI 2.8. In my test cases I have groovy steps.
In one of those, I recover a String from a previous test step and I want to do a particular processing if it matches the string "TJA470". The previous test step gives a string that is the output of a ssh command.
here is the groovy step code :
def hbox_ref = context.expand( '${get current HBox reference#hbox_ref}' )
// this returns me the data as a String
log.info hbox_ref
log.info "\"$hbox_ref\"" // to check if there is no spurious blank
log.info hbox_ref.class
log.info (hbox_ref == "TJA470") => returns false
log.info (hbox_ref.equals("TJA470")) => returns false
log.info (hbox_ref.contains("TJA470")) => returns true
here is the console result :
Fri Sep 20 16:13:17 CEST 2019: INFO: TJA470
Fri Sep 20 16:13:17 CEST 2019: INFO: "TJA470
"
Fri Sep 20 16:13:17 CEST 2019: INFO: class java.lang.String
Fri Sep 20 16:13:17 CEST 2019: INFO: false
Fri Sep 20 16:13:17 CEST 2019: INFO: false
Fri Sep 20 16:13:17 CEST 2019: INFO: true
The straighforward test is == or equals though there are differences, I use those in all the other comparisons of the same type and it works.
As you can see here the most logic cases return false and I really can't work out why.
If I do the same script in a tool like 'groovy playground' it works as expected ! :(
I'm not an expert in groovy at all and there must be something that I missed, but I find it very tricky !
If anyone can help ...
thanks
Thanks to SO I found out the problem :
with copy/pasting the console return in the question, it shows that there is a special character at the end of the text. This is not visible in SOAPUI log output ...
I added the following processing in my script :
def hbox_ref = context.expand( '${get current HBox reference#hbox_ref}' )
hbox_ref = hbox_ref.replaceAll("[^a-zA-Z0-9]+","")
or
hbox_ref = hbox_ref.replaceAll("[^\\w]+","")
this gives
log.info (hbox_ref == "TJA470") => returns true (at last !)
more elegant solution (thanks to SiKing) :
(hbox_ref.trim() == "TJA470")
instead of using replaceAll
I have a string from which I have to remove the timestamp and punctuation. And I have to remove all the digits also but responseCode value
has to be kept as is for example 400 in this case. And wherever 400 comes, it should not be removed. And I want to remove all the url's
and file name ending with tar.gz.
mystr="sun aug 19 13:02:09 2018 I_am.98189: hello please connect to the local host:8080
sun aug 19 13:02:10 2018 hey.94289: hello not able to find the file
sun aug 19 13:02:10 2018 I_am.94289: Base url for file_transfer is: abc/vd/filename.tar.gz
mon aug 19 13:02:10 2018 how_94289: $var1={
'responseCode' = '400',
'responseDate' = 'Sun, 19 Aug 2018 13:02:08 ET',
'responseContent' = 'ABC' }
mon aug 20 13:02:10 2018 hello!94289: Error performing action, failed with error code [400]
"
Expected result:
"I_am hello please connect to the local host
hello not able to find the file
Base url for file_transfer
var1
responseCode = 400
responseDate
responseContent = ABC
Error performing action, failed with error code 400
"
My Solution to remove punctuation:
punctuations = '''!=()-[]{};:'"\,<>.?##$%^&*_~'''
no_punct = ""
for char in mystr:
if char not in punctuations:
no_punct = no_punct + char
# display the unpunctuated string
print(no_punct)
Maybe:
patterns = [r"\w{3} \w{3} \d{2} \d{2}:\d{2}:\d{2} \d{4}\s*", #sun aug 19 13:02:10 2018
r"\w{3}, \d{2} \w{3} \d{4} \d{2}:\d{2}:\d{2} \w{2}\s*", #Sun, 19 Aug 2018 13:02:08 ET
r":\s*([\da-zA_Z]+\/)+([a-zA-Z0-9\.]+)", #URL
r"([a-zA-Z_!]+)[\.!_]\d+:\s*", #word[._!]number:>=0space
r":\d+",
"[/':,${}\[\]]" #punctuations
]
s = mystr
for p in patterns:
s = re.sub(p,'', s)
s = s.strip()
print(s)
Output:
hello please connect to the local host
hello not able to find the file
Base url for file_transfer is
var1=
responseCode = 400
responseDate =
responseContent = ABC
Error performing action failed with error code 400