I have two buttons in my py code, wanted to use progressbar with my function. My function is running properly however progressbar is not working.
I have done all changes which I got from here however none of them works for me.
I am using python 3.9 and selenium 4.6.0 on Chrome. I have a script that needs to upload a file to an input, this works fine on local but fails when run on RemoteDriver. The code I am using is
driver.find_element(By.XPATH, "//input[#accept]").send_keys('path to file')
When run on RemoteDriver the error returned is
selenium.common.exceptions.WebDriverException: Message: unknown command: unknown command: session/cddd71e067d7717481fb8a635103c643/se/file
I've think it is due to this line in the remote_connection.py file in selenium
Command.UPLOAD_FILE: ('POST', "/session/$sessionId/se/file")
From the research I've done the 'se' in this case is a 'vendor_prefix' for selenium but I cannot figure out how to either configure the remote driver to use a vendor prefix or remove that from POST path that is being passed (short of pulling my own version of the code and maintaining that).
For other functional reasons I can't revert to selenium 3x (which is an option I've seen to correct this), nor can I set w3c to False. Does anyone know how to work around this particular issue; either by getting send_keys to operate as expected in this situation or using another method to upload the file? Thanks.
I am running a scraper that parses data from specified websites and populates the HTML of a webpage to be later updated. The amount of data that is being scraped and parse is rather large, and the String containing the HTML of the webpage is a couple of hundred lines long.
I am using lists of dictionaries to insert the correct values into the HTML String through concatenation. After adding a certain amount of concatenations, I tried running my script to confirm that it was working properly, and I encountered this error:
RecursionError: maximum recursion depth exceeded during compilation
When I delete the added lines which add to the String containing the HTML of the page, the script runs fine.
I tried using this solution, but nothing changed.
I also tried to separate the String concatenation into a separate .py file and then importing that file and calling the function, but it continues to throw the RecursionError.
On a third shot, I attempted to use subprocess.run(), but I received ValueErrors because dictionaries cannot be passed as command line arguments.
I am running MacOS 11.4. Any help would be greatly appreciated :)
Hi I have a script that scrapes a website based on filters, this was working fine up until the latest chrome update and chromedriver update and then one of the filters was failing to click.
I have a try and except loop that repeats for 10 tries, sometimes this is enough sometimes and other times I need > 15 clicks of the same line of code for it to eventually work.
Here is my code to initiate the webpage:
from selenium import webdriver
url='https://www.marketscreener.com/stock-exchange/calendar/finance/'
options=webdriver.ChromeOptions()
chrome_prefs = {}
options.experimental_options["prefs"]=chrome_prefs
chrome_prefs["profile.defaul_content_settings"] = {"popups":1}
options.add_argument('--disable-browser-side-navigation')
options.add_argument('--disable-infobars')
options.add_argument('--disable-extensions')
options.add_argument('--disable-popup-blocking')
options.add_argument('--disable-notifications')
driver = webdriver.Chrome(executable_path=r'foo.exe',options=options)
after I log in using my credentials I try this line of code:
driver.find_element_by_id("selCountry").click()
which opens up the country filter drop down menu. The majority of times it takes an arbitrary number >10 tries for it to eventually work.
I am using chrome version 87.0.4280.141 and its chromedriver with python 3.7.
Can anyone provide a better solution to my problem or maybe an explanation on why it suceedes after x number of tries?
I am trying to visually depict my topics in python using pyldavis. However i am unable to view the graph. Is it that we have to view the graph in the browser or will it get popped upon execution. Below is my code
import pyLDAvis
import pyLDAvis.gensim as gensimvis
print('Pyldavis ....')
vis_data = gensimvis.prepare(ldamodel, doc_term_matrix, dictionary)
pyLDAvis.display(vis_data)
The program is continuously in execution mode on executing the above commands. Where should I view my graph? Or where it will be stored? Is it integrated only with the Ipython notebook?Kindly guide me through this.
P.S My python version is 3.5.
This not work:
pyLDAvis.display(vis_data)
This will work for you:
pyLDAvis.show(vis_data)
I'm facing the same problem now.
EDIT:
My script looks as follows:
first part:
import pyLDAvis
import pyLDAvis.sklearn
print('start script')
tf_vectorizer = CountVectorizer(strip_accents = 'unicode',stop_words = 'english',lowercase = True,token_pattern = r'\b[a-zA-Z]{3,}\b',max_df = 0.5,min_df = 10)
dtm_tf = tf_vectorizer.fit_transform(docs_raw)
lda_tf = LatentDirichletAllocation(n_topics=20, learning_method='online')
print('fit')
lda_tf.fit(dtm_tf)
second part:
print('prepare')
vis_data = pyLDAvis.sklearn.prepare(lda_tf, dtm_tf, tf_vectorizer)
print('display')
pyLDAvis.display(vis_data)
The problem is in the line "vis_data = (...)".if I run the script, it will print 'prepare' and keep on running after that without printing anything else (so it never reaches the line "print('display')).
Funny thing is, when I just run the whole script it gets stuck on that line, but when I run the first part, got to my console and execute purely the single line "vis_data = pyLDAvis.sklearn.prepare(lda_tf, dtm_tf, tf_vectorizer)" this is executed in a couple of seconds.
As for the graph, I saved it as html ("simple") and use the html file to view the graph.
I ran into the same problem (I use PyCharm as IDE) The problem is that pyLDAvize is developed for Ipython (see the docs, https://media.readthedocs.org/pdf/pyldavis/latest/pyldavis.pdf, page 3).
My fix/workaround:
make a dict of lda_tf, dtm_tf, tf_vectorizer (eg., pyLDAviz_dict)dump the dict to a file (eg mydata_pyLDAviz.pkl)
read the pkl file into notebook (I did get some depreciation info from pyLDAviz, but that had no effect on the end result)
play around with pyLDAviz in notebook
if you're happy with the view, dump it into html
The cause is (most likely) that pyLDAviz expects continuous user interaction (including user-initiated "exit"). However, I rather dump data from a smart IDE and read that into jupyter, than develop/code in jupyter notebook. That's pretty much like going back to before-emacs times.
From experience this approach works quite nicely for other plotting rountines
If you received the module error pyLDA.gensim, then try this one instead:
import pyLdAvis.gensim_models
You get the error because of a new version update.