Hello I have built a program script that goes onto a website and selects a size and auto checks out an Item for me it works very well but I have 2 concerns
1.I want to have this script run faster before the script ran pretty fast (so fast that it basically added to cart and went to the checkout page before the Item could even load into the cart (which resulted in errors) and so I added there script to my code
wait = WebDriverWait(driver, 10) and this one which I mainly used to wait until the item loaded into the cart and all the "add to cart" buttons showed up
wait.until(EC.presence_of_element_located((By.NAME, 'commit')))
but I want this script to run faster I tried changing the
wait = WebDriverWait(driver, 10) into something like
wait = WebDriverWait(driver, 1) and
wait = WebDriverWait(driver, 100) but I see no difference is there anything I can do to make the script run faster?(it doesnt have to do with the wait= thing Ill take any thing I can get to even shave off milaseconds.
I am currently using the send_keys option for autofill which is PAINFULLY SLOW is there anything I can use that will fill all the stuff instanstly alltogether? ik there are some "JAVA-scripts simular to this that can do it but im not sure how to right java script or more importantly how to even combine them
Can anyone help me out I just want my selenium python chromedriver script to run as fast as possible.
Thank you.
(for my script im using select for the size and just .click() and a couple of if statements which depends on how many items they want to cart and lots of def fweuf
fweuf() (i forget what those are called lol) )
For sending values with JS you can do this:
js= "document.getElementById('YOURELEMENT').value = '" + str(YOURVALUE) + "';"
driver.execute_script(js)
Hope this helps.
Related
I wanted to create a metin2 bot, nothing complicated. I am new into that so i just started to look for tutorials on youtube to find something that would lead me to world of coding, how does it work, what language is used to write it ect. So I found a tutorial on youtube about open CV Learn Code By Gaming - OpenCV and thought that's awesome thing to start with.
I got to the point where I can easily detect the names of monsters/metins and wont have any problems in finding the right points to click. After detection started working I thought about automating clicking on the window to farm some items etc. Simple things like clicking e.g 50 pixels below the name to auto attack worked just fine but the problem was that I wanted to make more bots to maximalize farming. So when i started to add more clients I got a problem where e.g you have to hold down SPACE in one window to attack from horse and click on another window which was stopping attacking from horse. So I thought about finding some code that can basically send message directly to window without controlling a mouse or keyboard so you can run multiple bots in one time and each will do perfect meanwhile you can do anything else on pc because your mouse and keyboard aren't used.
Let's start from code I found and none worked for windows in background (even with administrator privileges). Pyautogui doesn't work in background (window has to be in foreground to be clicked on and it controls mouse so there is no point in using that.
From that code I learned that I need to find "window ID" to connect to it and send messages. When i print hWnd it shows the numbers in Terminal and code passes without any fails but does nothing except printing the window ID ( Parent Handle ). Ofc I pip installed pywin32
import win32gui
import win32api
import win32con
def click(x, y):
hWnd = win32gui.FindWindow(None, "Client")
print(hWnd)
lParam = win32api.MAKELONG(x, y)
win32api.SendMessage(hWnd, win32con.WM_LBUTTONDOWN, win32con.MK_LBUTTON, lParam)
win32api.SendMessage(hWnd, win32con.WM_LBUTTONUP, win32con.MK_LBUTTON, lParam)
click(100, 100)
Than i found another code that looked similar but used different function so first i used
wdname = 'Client'
hwnd = win32gui.FindWindow(None, wdname) # parent handle
print(hwnd)
Which printed me a window ID that i used in parameters in function
def control_click(x, y, handle, button):
l_param = win32api.MAKELONG(x, y)
if button == 'left':
win32gui.PostMessage(handle, win32con.WM_LBUTTONDOWN, win32con.MK_LBUTTON, l_param)
win32gui.PostMessage(handle, win32con.WM_LBUTTONUP, win32con.MK_LBUTTON, l_param)
elif button == 'right':
win32gui.PostMessage(handle, win32con.WM_RBUTTONDOWN, 0, l_param)
win32gui.PostMessage(handle, win32con.WM_RBUTTONUP, 0, l_param)
control_click(200, 200, 329570, button ='left')
But it still did nothing but code passed clear
Anyone have any ideas about how to make clicks/keyboard clicks in python on window in background without taking controll of mouse/keyboard? If you have any experience in automating and know better ways to create automation for clicking ect. and want to share it please do. If that can be done in another programming language also share your toughts about that of course if you want to.
Windows 10 x64
Python used: 3.9.2
All codes i found in topics were out of date that s why i am asking for help there. Thanks in advance :)
Hi I have a script that scrapes a website based on filters, this was working fine up until the latest chrome update and chromedriver update and then one of the filters was failing to click.
I have a try and except loop that repeats for 10 tries, sometimes this is enough sometimes and other times I need > 15 clicks of the same line of code for it to eventually work.
Here is my code to initiate the webpage:
from selenium import webdriver
url='https://www.marketscreener.com/stock-exchange/calendar/finance/'
options=webdriver.ChromeOptions()
chrome_prefs = {}
options.experimental_options["prefs"]=chrome_prefs
chrome_prefs["profile.defaul_content_settings"] = {"popups":1}
options.add_argument('--disable-browser-side-navigation')
options.add_argument('--disable-infobars')
options.add_argument('--disable-extensions')
options.add_argument('--disable-popup-blocking')
options.add_argument('--disable-notifications')
driver = webdriver.Chrome(executable_path=r'foo.exe',options=options)
after I log in using my credentials I try this line of code:
driver.find_element_by_id("selCountry").click()
which opens up the country filter drop down menu. The majority of times it takes an arbitrary number >10 tries for it to eventually work.
I am using chrome version 87.0.4280.141 and its chromedriver with python 3.7.
Can anyone provide a better solution to my problem or maybe an explanation on why it suceedes after x number of tries?
I made a bot to play a browser game for me. The bot clicks buttons presented on a web page to continue with a story line. And I require a help with optimising the button selection, checking & submitting process.
How can I make a Python Selenium script to get all the buttons on the page and select a specific one from the available ones.
The bot right now works as follows:
x = 0
while x < 99999:
if x < 99999:
try:
browser.get('https://censoredgame.com/city/adventure/decision/7')
time.sleep(0.4)
browser.get('https://censoredgame.com/city/adventure/')
x = x + 1
except:
pass
try:
browser.get('https://censoredgame.com/city/adventure/decision/25')
time.sleep(0.4)
browser.get('https://censoredgame.com/city/adventure/')
x = x + 1
except:
pass
<a class="btn" title="Вземете решение!" href="/city/adventure/decision/29">Подслушайте</a>
<!-- This is how a button looks, I cant use NAME,ID,CLASS -->
It is pretty simple:
/city/adventure/decision/7 = Some button, /city/adventure/decision/25 = Other button.
The way my bot works is it tries to press a button by entering a link in the browser bar and it does this very slow because it checks 15 buttons on the page.
Could you please advise me on how I could improve this process?
Nikolay, I'm rly want to help you, but I didn't get how you gonna click on the button - " by entering a link in the browser bar"? Anyway, you need to open your page, then find all elements by Xpath driver.find_elements(By.XPATH, "//a[#class='btn' and #title='Вземете решение!']"). Collect all those elements in WebElements list, then do foreach loop(I'm writing code on Java and C#, so u have to do it yourself). This loop gonna be clicking on each "Подслушайте" button on the page. If you have any questions let me know.
I'm so happy to tell you I found my way to approach the problem.
I'm sorry for everyone who didn't understand my problem and wanted to help.
I fixed my code and now its really fast. Here is a little example:
x = 0
while x < 1000:
try:
browser.find_element_by_link_text('Пълна атака').click()
browser.find_element_by_link_text('Продължи').click()
except NoSuchElementException:
pass
try:
browser.find_element_by_link_text('Изправете се срещу врага').click()
browser.find_element_by_link_text('Продължи').click()
except NoSuchElementException:
pass
etc
I feel really stupid right now. The answer was so easy but I didn't get in right away.
First time i tried to find element by link text I forgot to set except NoSuchElementException and i got problems...
Thank you all for help!
Good Afternoon :) Having a problem with my Python3 Gtk3 application and Selenium WebDriver (ChromeDriver). Also, using Linux if it matters.
Basically, the user presses a button to start the Selenium webdriver automation and then as the automation process is going, it 'SHOULD' give feedback to the user in the GUI (See Content.content_liststore.append(list(item)) and LogBox.log_text_buffer).
However, it's not adding anything into the content_liststore until after fb_driver.close() is done. In the meantime, the Gtk window just "hangs".
Now, I've been looking into multithreading in hopes of the GUI being responsive to this feedback but I've also been reading that Selenium doesn't like multithreading (but I presume thats running multiple browsers/tabs (which this is not)).
So, my question is; Is multithreading the go-to fix for getting this to work?
# ELSE IF, FACEBOOK COOKIES DO NOT EXIST, PROCEED TO LOGIN PAGE
elif os.stat('facebook_cookies').st_size == 0:
while True:
try: # look for element, if not found, refresh the webpage
assert "Facebook" in fb_driver.title
login_elem = fb_driver.find_element_by_id("m_login_email")
login_elem.send_keys(facebook_username)
login_elem = fb_driver.find_element_by_id("m_login_password")
login_elem.send_keys(facebook_password)
login_elem.send_keys(Keys.RETURN)
except ElementNotVisibleException:
fb_driver.refresh()
StatusBar.status_bar.push(StatusBar.context_id, "m_login_password element not found, trying again...")
ProblemsLog.text_buffer.set_text("Facebook has hidden the password field, refreshing page...")
else:
query_elem = fb_driver.find_element_by_name("query")
query_elem.send_keys(target)
query_elem.send_keys(Keys.RETURN)
break
m_facebook_url_remove = "query="
m_facebook_url = fb_driver.current_url.split(m_facebook_url_remove, 1)[1] # Remove string before "query="
facebook_url = "https://www.facebook.com/search/top/?q=" + m_facebook_url # Merge left-over string with the desktop url
StatusBar.status_bar.push(StatusBar.context_id, "Facebook elements found")
fb_title = fb_driver.title
fb_contents = [(target_name.title(), "Facebook", facebook_url)]
for item in fb_contents:
Content.content_liststore.append(list(item))
#progress_bar.set_fraction(0.10)
LogBox.log_text_buffer.set_text("Facebook Search Complete")
with open('facebook_cookies', 'wb') as filehandler:
pickle.dump(fb_driver.get_cookies(), filehandler)
fb_driver.close()
I've considered it not working because of the 'while' loop, but another piece of code doesn't have a loop and does the exact same thing, it waits for Selenium to finish before adding content to the GUI.
Additionally, the user can select multiple websites to do this with, so the application can first go to Facebook (do it's business then close), go to LinkedIn (do it's business then close) and so fourth. And it still waits for all the Selenium code to finish before adding anything to the Gtk GUI.
I really hope that makes sense! Thank you :)
Your`s question is the answer you are lookig for. Take a read here https://wiki.gnome.org/Projects/PyGObject/Threading
I am trying to visually depict my topics in python using pyldavis. However i am unable to view the graph. Is it that we have to view the graph in the browser or will it get popped upon execution. Below is my code
import pyLDAvis
import pyLDAvis.gensim as gensimvis
print('Pyldavis ....')
vis_data = gensimvis.prepare(ldamodel, doc_term_matrix, dictionary)
pyLDAvis.display(vis_data)
The program is continuously in execution mode on executing the above commands. Where should I view my graph? Or where it will be stored? Is it integrated only with the Ipython notebook?Kindly guide me through this.
P.S My python version is 3.5.
This not work:
pyLDAvis.display(vis_data)
This will work for you:
pyLDAvis.show(vis_data)
I'm facing the same problem now.
EDIT:
My script looks as follows:
first part:
import pyLDAvis
import pyLDAvis.sklearn
print('start script')
tf_vectorizer = CountVectorizer(strip_accents = 'unicode',stop_words = 'english',lowercase = True,token_pattern = r'\b[a-zA-Z]{3,}\b',max_df = 0.5,min_df = 10)
dtm_tf = tf_vectorizer.fit_transform(docs_raw)
lda_tf = LatentDirichletAllocation(n_topics=20, learning_method='online')
print('fit')
lda_tf.fit(dtm_tf)
second part:
print('prepare')
vis_data = pyLDAvis.sklearn.prepare(lda_tf, dtm_tf, tf_vectorizer)
print('display')
pyLDAvis.display(vis_data)
The problem is in the line "vis_data = (...)".if I run the script, it will print 'prepare' and keep on running after that without printing anything else (so it never reaches the line "print('display')).
Funny thing is, when I just run the whole script it gets stuck on that line, but when I run the first part, got to my console and execute purely the single line "vis_data = pyLDAvis.sklearn.prepare(lda_tf, dtm_tf, tf_vectorizer)" this is executed in a couple of seconds.
As for the graph, I saved it as html ("simple") and use the html file to view the graph.
I ran into the same problem (I use PyCharm as IDE) The problem is that pyLDAvize is developed for Ipython (see the docs, https://media.readthedocs.org/pdf/pyldavis/latest/pyldavis.pdf, page 3).
My fix/workaround:
make a dict of lda_tf, dtm_tf, tf_vectorizer (eg., pyLDAviz_dict)dump the dict to a file (eg mydata_pyLDAviz.pkl)
read the pkl file into notebook (I did get some depreciation info from pyLDAviz, but that had no effect on the end result)
play around with pyLDAviz in notebook
if you're happy with the view, dump it into html
The cause is (most likely) that pyLDAviz expects continuous user interaction (including user-initiated "exit"). However, I rather dump data from a smart IDE and read that into jupyter, than develop/code in jupyter notebook. That's pretty much like going back to before-emacs times.
From experience this approach works quite nicely for other plotting rountines
If you received the module error pyLDA.gensim, then try this one instead:
import pyLdAvis.gensim_models
You get the error because of a new version update.