ElementTree: selecting right tree? - python-3.x

For the data contained here: http://www.playonline.com/ff11us/polnews/news.xml
I have this Python code:
import xml.etree.ElementTree as ET
import urllib.request
rss1 = "http://www.playonline.com/ff11us/polnews/news.xml"
f = urllib.request.urlopen(rss1)
objects = ET.fromstring(f.read().decode())
print([el.attrib.get('title') for el in objects.findall('*/item')])
But it's returning a blank array. Am I selecting the wrong tree? Do I need to select a child tree or something similar?
Trying to get a grasp on the layout:
>>> for o in objects:
... print(o.tag)
...
{http://purl.org/rss/1.0/}channel
{http://purl.org/rss/1.0/}item
{http://purl.org/rss/1.0/}item
{http://purl.org/rss/1.0/}item
{http://purl.org/rss/1.0/}item
{http://purl.org/rss/1.0/}item
So selecting item I thought would work, or do I need to select that entire line?
I can select on ./{http://purl.org/rss/1.0/}item to get parent items, but how do I get the data (link) from the child items?

I think there is nothing wrong in your original code, except that it needs to use namespaces (to fully qualify an element). I added the hash of namespaces to your code. See if it works for you.
import xml.etree.ElementTree as etree
import urllib.request
pon_url = "http://www.playonline.com/ff11us/polnews/news.xml"
response = urllib.request.urlopen(pon_url)
xmlstr = response.read().decode()
root = etree.fromstring(xmlstr)
/* This is the namespaces. We use it later during 'findall' */
ns = {'ns1':'http://purl.org/rss/1.0/','ns2':'http://purl.org/dc/elements/1.1/'}
#etree.dump(root)
print("\nItems:\n")
print([i.text for i in root.findall('./ns1:item/ns1:title',ns)])
print("\nlinks:\n")
print([i.text for i in root.findall('./ns1:item/ns1:link',ns)])
print("\nDescriptions:\n")
print([i.text for i in root.findall('./ns1:item/ns1:description',ns)])
print("\nDates\n");
print([i.text for i in root.findall('./ns1:item/ns2:date',ns)])
Result:
Items:
['FINAL FANTASY XI Updated (Nov. 9)', 'All Worlds Maintenance (Nov. 9)', 'Temporary Suspension of NA GM Petition Service (Nov. 5)', 'Recovery from Fenrir World Technical Difficulties (Oct. 29)', 'Fenrir World Several Areas Technical Difficulties (Oct. 29)']
links:
['http://www.playonline.com/ff11us/polnews/news24770.shtml', 'http://www.playonline.com/ff11us/polnews/news24767.shtml', 'http://www.playonline.com/ff11us/polnews/news24765.shtml', 'http://www.playonline.com/ff11us/polnews/news24762.shtml', 'http://www.playonline.com/ff11us/polnews/news24759.shtml']
Descriptions:
['A version update was performed on FINAL FANTASY XI at the following time.<br><br>* Clients will update automatically upon launch after the date and time below. After the following time, the update will automatically begin after you press the "Play" button. After that, please follow the instructions on the screen.<br><br>[Date & Time]<br>Nov. 9, 2016 21:00 (PST)<br><br>[Affected Service]<br>FINAL FANTASY XI<br><br>[Update Details]<br>http://sqex.to/DYa', 'At the following time, we will be performing server maintenance. During this period, FINAL FANTASY XI will be unavailable.<br><br>We apologize for any inconvenience this may cause and thank you for your patience.<br><br>* This maintenance will be accompanied by a client program version update. After maintenance is complete, the update will automatically begin after you press the "Play" button. After that, please follow the instructions on the screen.<br><br>* The World Transfer Service will be unavailable starting 30 minutes before the maintenance.<br><br>*After the maintenance ends, a spike in access is expected.<br>If you encounter congestion errors such as "POL-1160" and "POL-0010" during the confirmation screen or while downloading, we apologize for the inconvenience, and we ask that you try again after waiting for some time.<br><br>To ensure a smooth version update, we ask for your understanding and cooperation.<br><br>*Update details will be announced on Nov. 9, 2016 (PST)<br><br>[Date & Time]<br>Nov. 9, 2016 21:00 to 23:00 (PST)<br>* Completion time is subject to change.<br><br>[Affected Service]<br>- FINAL FANTASY XI', 'At the following time, the North American GM petition service will be temporarily unavailable.<br><br>The European GM petition service will be operating normally and users may still place GM calls for urgent issues. However, please be aware that there may be significant delays until these GM calls can be answered.<br><br>We apologize for the inconvenience and thank you for your patience in this matter.<br><br>[Date & Time]<br>Nov. 5, 2016 1:00 to 4:00 (PDT)<br><br>[Affected Services]<br>- The North American GM Petition service for FINAL FANTASY XI<br><br>[Cause]<br>Building Maintenance', 'At the time below, players were unable to access several areas on the Fenrir World.<br><br>We are pleased to announce that the issue has been addressed. We apologize for any inconvenience this may have caused.<br><br>[Date & Time]<br>Oct. 29, 2016 from 2:58 to 4:32 (PDT)<br><br>[Details]<br>- Unable to access certain areas on the Fenrir World<br><br>[Cause]<br>- Server equipment issue<br><br>[Affected Areas]<br>Unable to access the following areas:<br>- Newton Movalpolos<br>- Abyssea - Konschtat<br>- Lufaise Meadows<br>- Monarch Linn<br>- The Garden of Ru'Hmet<br>- Dynamis - Tavnazia<br>- Aydeewa Subterrane<br>- La Vaule [S]<br>- West Ronfaure<br>- North Gustaberg<br>- South Gustaberg<br>- Cape Teriggan<br>- East Sarutabaruta<br>- Ru'Aun Gardens<br>- Fort Ghelsba<br>- Qulun Dome<br>- Castle Oztroja<br>- Castle Zvahl Keep [S]<br>- Sacrificial Chamber<br>- Throne Room<br>- Ranguemont Pass<br>- Ve'Lugannon Palace<br>- Dynamis - Windurst<br>- Dangruf Wadi<br>- Outer Horutoto Ruins<br>- Ifrit's Cauldron<br>- Qu'Bia Arena<br>- Cloister of Tremors<br>- Abyssea - Attohwa<br>- Ship bound for Selbina<br>- Jeuno-Windurst Airship<br>- Northern San d'Oria<br>- Windurst Waters<br>- Lower Jeuno', 'We are currently experiencing technical difficulties with certain areas on the Fenrir World. The issue is currently under investigation, and new updates will follow as additional information becomes available.<br><br>We apologize for any inconvenience this may be causing, and we thank you for your understanding.<br><br>[Date & Time]<br>From Oct. 29, 2016 2:58 (PDT)<br><br>[Affected]<br>- Certain areas on the Fenrir World<br><br>[Cause]<br>Under investigation<br><br>[Details]<br>Under investigation']
Dates
['Thu, 10 Nov 2016 15:06:13 JST', 'Mon, 07 Nov 2016 14:14:32 JST', 'Fri, 04 Nov 2016 17:18:30 JST', 'Sat, 29 Oct 2016 20:05:44 JST', 'Sat, 29 Oct 2016 20:05:44 JST']

Related

400 Error when trying to use jsonlines file with OpenAI answers

I uploaded a file to openai for the purpose of answers. I can't figure out how to access it without getting an error.
This is the contents of my file:
{"text":"We are focusing on improving communication to the customers."}
{"text":"The emphasis on the customer is working well."}
{"text":"Services initiatives"}
{"text":"CMM and focus on customer service."}
{"text":"We are working on CMM to focus on the customer."}
{"text":"Our emphasis on making our customers number one."}
{"text":"Customer relationships are continuing to improve."}
This is the information about the file when I query openai:
{"object":"file","id":"file-02fdY5PuZ1aO4cnd2r1PxaB7","purpose":"answers","filename":"f3f1c105-5217-4132-8950-a040b6183ed7","bytes":458,"created_at":1648396701,"status":"processed","status_details":null}
Now I ask a question at https://api.openai.com/v1/answers with this POST:
{"file":"file-02fdY5PuZ1aO4cnd2r1PxaB7","question":"What is the main theme?","search_model": "davinci","model": "davinci","examples_context": "In 2017, U.S. life expectancy was 78.6 years.","examples": [["What is human life expectancy in the United States?","The life expectancy in the United States is 78 years."]],"max_tokens": 100,"stop": ["\n", "<|endoftext|>"]}
The server returns a 400 error. If in my request I replace the "file" field with a "documents" array using same data, the request is successful.
I don't know what more to look at. If someone can suggest something or if you seen an issue in what I am trying to do, I would deeply appreciate hearing from you.
Thanks

Kite autocomplete is not giving me free suggestions - only 30 per day - no pro suggestions

I installed the Kite autocomplete for Python (Anaconda-Spyder is my main IDE , though I also use SublimeText)
And i get this pic:
Kite Pro ripping me off
(Even when I do not get any starred autocompletions of any sort ...)
Isn't it supposed to say 3 starred completions left today??
Even when i sign out and back in , uninstall Kite, clear my registry and remove temp and program files, i notice it somehow knows that I have used up some of its 30 completions and gives me only 30 per day
Even if I sign out of Kite (no way for it to know I have an account with a Pro free trial , right?? ),I still get only 30 completions per day, and after that kite is "locked" - null completions today ...
And i don't even get the Starred completions - the Pro completions
So is it a mistake on my installation ? Will adding Kite to my PATH help?? (I seriously doubt it )
Or is it a bug on the devs' side ??
Any help regarding this matter would be appreciated ...
I simply cannot stand the fact that my bro gets FREE autocompletions per day, and that too unlimited ones at that ( which I guess is the free tier of Kite , since he gets only 3 starred ones)
I think there might be a bug on Kite's side.
However, I do not face the problem mentioned above :
https://i.stack.imgur.com/259T8.png |
I receive unlimited free tier auto-completions and only 3 Pro completions.Hope this answer helps you!

Python/Selenium - How to process extracted data from App store

I am using Selenium/Python to parse reviews from Apple app store. I used the following code to extract the data for the first five reviews:
URL: https://apps.apple.com/us/app/lemonade-insurance/id1055653645#see-all/reviews
wait = WebDriverWait(driver, 5)
response_ratings = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".we-customer-review")))
response_container = []
for e in response_ratings[:5]:
response_container.append(e.get_attribute('innerHTML'))
print(response_container[0])
Then, I print the first output
I expect to have star 5 out of 5, date July 6, 2019, title Convenient and Affordable!!!!, review The Lemonade app is so easy to use as well as having affordable rates!..., and Developer Response Thanks so much for your awesome review!! We're so happy to have you in... for the first review.
How do I get the above info? Thank you in advance for the help
You can use BeautifulSoup to parse the innerHTML and get what you're looking for.
One way of doing it would be:
import re
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from bs4 import BeautifulSoup
link = 'https://apps.apple.com/us/app/lemonade-insurance/id1055653645#see-all/reviews'
stars = re.compile(r"\d out of \d")
with webdriver.Chrome() as driver:
wait = WebDriverWait(driver, 10)
driver.get(link)
elements = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".we-customer-review")))
for elem in elements:
s = BeautifulSoup(elem.get_attribute("innerHTML"), "html.parser")
review_date = s.find("time").text
review_body = s.find("p").text
review_title = s.find("h3", {"data-test-customer-review-title": ""}).text.strip()
review_stars = ''.join(re.findall(stars, str(s.find("figure"))))
dev_response = s.find_all("p", {"data-test-bidi": ""})
print(f"{review_title} | {review_date} | {review_stars}")
print(review_body)
print(dev_response[1].text if len(dev_response) > 1 else "")
print("-" * 80)
This prints:
Convenient and Affordable!!!! | 07/06/2019 | 5 out of 5
The Lemonade app is so easy to use as well as having affordable rates! It took me all of 15 minutes to sign-up, pick a coverage and a deductible. Very nice customer service as well as very informative. At anytime of any day of the week I can log onto my account and check everything as well as make any necessary changes that I may need or want. The chat option feature works fantastic! Whenever I have a question I just go to the chat feature and within seconds someone is there to help and answer all my questions regarding their services and my plan coverage. I wished they had a referral feature cause I’ve already set up a couple of family members with the company as well. They were amazed that it only took about 10-15 minutes to setup and just how affordable it is!! I’ve gotten quotes from so many other companies but the monthly payments and deductible were too expensive, I was a little hesitant at first but I said hey I should at least give it a try and so far so good!!! I’m hoping that it’ll never come to the point where I’ll actually need to file a claim but if so I feel confident that the process will be easy and stress free considering how much stress I’m going to actually have due to a burglary or theft. I have faith that we’re going to have a very long relationship. Thanks to all the developers of the Lemonade App, the name of the company is nice too!!!!
Thanks so much for your awesome review!! We're so happy to have you in our Lemonade community!
--------------------------------------------------------------------------------
Made the best lemonade I’ve ever had! | 07/01/2018 | 5 out of 5
I have been telling EVERYONE about Lemonade. I don’t know how, but, getting insurance through your company is actually FUN! I have never had so much FUN doing a chore that typically involves a boring Q & A. The app really made me feel like I was getting insurance through a friend. I smiled with the shout out to my horoscope sign after entering my birthday, I loved the “making lemonade” process when getting the quote, and (because I have a future date for the policy to start) I absolutely adore the countdown. I look at it almost daily and become even more excited for my move (and I really don’t like having to move so this is really helping). The use of unclaimed funds going to charity actually makes my heart melt. By providing freedom to choose what organization you would like to contribute to truly makes me feel like I am giving back in some way, and it is beyond noble and inspiring for you to use that money to help others turn lemons into lemonade. Also, your customer service has been impeccable. Every question I have had has been answered quickly and by a friendly representative of the company. I don’t know much about the insurance world, other than we need to have it, but, you make me want to work for you!!! Where do I sign up?
--------------------------------------------------------------------------------

Understanding GTFS service times

I have a problem with the understanding of the gtfs file format. Or maybe there is an error in that data. There is a gtfs file from a public transportation agency called "Verkehrsverbund Mittelthüringen" (VMT). This data is accessible at https://transitfeeds.com/p/verkehrsverbund-mittelth-ringen/1080/latest/routes.
For example: I have taken the trip with the ID 9782458 (trips.txt).
2841_0,45,9782458,"Erfurt, Thüringenhalle","",0,,,,
This has the service ID 45 with the specification
45,0,0,0,0,0,0,0,20191101,20200229
Additionally here are the entries for the celendar_dates.txt
45,20191104,1
45,20191111,1
45,20191118,1
45,20191125,1
45,20191202,1
45,20191209,1
45,20191216,1
45,20191105,1
45,20191112,1
45,20191119,1
45,20191126,1
45,20191203,1
45,20191210,1
45,20191217,1
45,20191106,1
45,20191113,1
45,20191120,1
45,20191127,1
45,20191204,1
45,20191211,1
45,20191218,1
45,20191107,1
45,20191114,1
45,20191121,1
45,20191128,1
45,20191205,1
45,20191212,1
45,20191219,1
45,20191101,1
45,20191108,1
45,20191115,1
45,20191122,1
45,20191129,1
45,20191206,1
45,20191213,1
45,20191220,1
Does this mean, that the service is available all times, except from the 1st November 2019 to the 29th February 2020? My Problem now is the output of the search engine tansitfeeds.com. It says the trip with the ID 9782458 is available at the 14th November 2019. Which is contary to my understanding of the data: the trip won't be available in November. Where is the clue I missed? Or is there an error in the data?
The line you pasted indicates that service ID 45 runs on zero days of the week (that's what all those zeros mean), so the start and end dates in the same line don't really mean anything.
If this service actually does run on Nov. 14, this could be represented in the calendar_dates.txt file, which is usually used to represent service changes for special dates.
EDIT: the data you added from calendar_dates.txt does indeed show that service 45 has been added for date 20191114.

Strange sybase behavior around daylight savings time (DST)

I've got a strange sybase behavior which I do not understand.
Situation
I have a table (MY_TABLE) with several columns of type smalldatetime. For illustration purposes let's assume the following table and data:
MY_TABLE||ID |TS_INIT |TS_LASTCHANGE |MY_TEXT |
||4711|3/31/2013 12:00:00 AM|3/31/2013 3:00:00 AM|someText|
TS_INIT and TS_LASTCHANGE are of type smalldatetime.
When executing the following statement I get the above result:
SELECT ID, TS_INIT, TS_LASTCHANGE MY_TEXT
FROM MY_TABLE
WHERE ID = 4711
go
My client is running in UTC+1 (Berlin) and has daylight savings time (DST) enabled.
I am not sure in what time zone the server is running and whether or not DST is enabled.
Problem
When I execute this (note that it is 03:00h):
SELECT ID, TS_INIT, TS_LASTCHANGE MY_TEXT
FROM MY_TABLE
WHERE ID = 4711 AND TS_LASTCHANGE = "2013-03-31 03:00:00:000"
go
I get NO results but when I execute this (note that it is 02:00h this time):
SELECT ID, TS_INIT, TS_LASTCHANGE MY_TEXT
FROM MY_TABLE
WHERE ID = 4711 AND TS_LASTCHANGE = "2013-03-31 02:00:00:000"
go
I do again get the above result which is saying TS_LASTCHANGE is
3/31/2013 3:00:00 AM
Note that the result prints 03:00h, even though I queried for 02:00h.
Why Is the first query returning no results even though there should be a match and why is the second query returning a result even though there should be no match?!
Note also that 3/31/2013 3:00:00 AM is the first moment in DST (at least in the year 2013) and 3/31/2013 2:00:00 AM should never ever exist at all because when transitioning from winter to summer time, the clock switches from 01:59:59 to 03:00:00 (as per this site).
Database: Adaptive Server Enterprise V15.0.3
Client: Aqua Data Studio V16.0.5
EDIT:
When querying whit the TS_INIT everything works as one would expect (only a result for 3/31/2013 12:00:00 AM)
Aqua Data Studio is written in Java.
The problem you are having has to do with the fact that Java is aware of timezones and databases don't have a concept of timezone when they store date and times. When the time comes back from the database, the database's JDBC driver puts it in a Java date and just assumes the timezone is irrelevant. The problem happens when you try to display a time which the JVM thinks is invalid, so a valid date is presented, which basically pushes the time by an hour. Daylight savings for 2015 started on March 08 2.00 AM and one of your rows contains a date which is invalid according to JVM.
This has been a known design issue with Java, and they are trying to fix this with JSR-310 for inclusion in Java SE 8. With this, they will have LocalDate, OffsetDate and ZonedDate. You can read more about it here ...
https://today.java.net/pub/a/today/2008/09/18/jsr-310-new-java-date-time-api.html#jsr-310-datetime-concepts
https://jcp.org/en/jsr/detail?id=310
http://docs.google.com/View?id=dfn5297z_8d27fnf
Workaround
The only workaround, is to probably trick the JVM by setting the timezone in the JVM to GMT. If you are running ADS 16 on Windows, and you launch ADS with the shortcut icon on the desktop (which runs datastudio.exe), then you need to modify the datastudio.ini file in your folder. Add a new entry for vmarg.5=-Duser.timezone=gmt
This link explains the location of where to find the data studio.ini
https://www.aquaclusters.com/app/home/project/public/aquadatastudio/wikibook/Documentation14/page/50/Launcher-Memory-Configuration#windows
Once you have made the change, Restart ADS. Then go to Help->About->System: and double check your user.timezone setting and make sure it is GMT. Then give it a try.
With the above change there might be side effects in the application where timezone are involved, For e.g. in the Table Data Editor->Insert Current Date&Time, which would display a GMT time ... so there would be an offset.

Resources