I am trying to locate a search box with id as (search2) from a website. I have been able to successfully login to the website using the below code.
import requests
from tqdm import tqdm
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
options = webdriver.ChromeOptions()
driver = webdriver.Chrome(executable_path=r'C:\chromedriver_win32\chromedriver.exe', chrome_options=options)
driver.implicitly_wait(30)
tgt = "C:\\mypath"
profile = {"plugins.plugins_list": [{"enabled":False, "name":"Chrome PDF Viewer"}],
"download.default_directory" : tgt}
options.add_experimental_option("prefs",profile)
print(options)
driver.get("http://mylink.com/")
user=driver.find_element_by_id("username")
passw=driver.find_element_by_id("password")
user.send_keys("abc#xyz.com")
passw.send_keys("Pwd")
driver.find_element_by_xpath('/html/body/div[2]/div/div/div[2]/form/div[3]/button').click()
page=driver.find_element_by_id("search2")
print(page)
The code works perfectly till here but the moment I add the below to it I get an error
page.send_keys("abc")
The error that I get is as below.
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
What I am trying to do here is login to the website and search for some items and download the results. I have already tried using the implicitly wait options as mentioned in the code. Any help would be highly appreciated.
Adding the below piece of code did the trick. Had to make the current thread sleep while the program continues to run the next steps.
time.sleep(5)
Thanks everyone.
Related
I am trying to create a script that will login to my google slide presentations and update them. In order to do this with Selenium it appears I have to login each time which causes the 'This browser or app may not be secure' error page, shown below.
I researched online that you should be able to work around this by logging in through stackoverflow or another site that uses 'log in with google'. However my below attempt seems to still provide the same results. Following up on this, I wasn't able to find out if this is still doable or if this approach no longer works.
Could someone confirm if my approach is outdated or if there is something I've failed to grasp from my approach? Sorry if I've missed something basic, I've just started writing Python so I'm deep in the learning curve stage.
I'm using Ubuntu 20.04 if that's of any use.
Code
#!/usr/bin/env python3
import webbrowser
import pyautogui
import time
import sys
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
driver=webdriver.Firefox()
driver.implicitly_wait(30)
url='https://accounts.google.com/o/oauth2/auth/identifier?client_id=717762328687-iludtf96g1hinl76e4lc1b9a82g457nn.apps.googleusercontent.com&scope=profile%20email&redirect_uri=https%3A%2F%2Fstackauth.com%2Fauth%2Foauth2%2Fgoogle&state=%7B%22sid%22%3A1%2C%22st%22%3A%2259%3A3%3Abbc%2C16%3A7667a52eee989fc7%2C10%3A1611306467%2C16%3Ad5e975ab3e71c656%2Cb45db9f242e90237a65d3c06754d76a1f8a7bbfc93dffe8b19376a3a573f700e%22%2C%22cdl%22%3Anull%2C%22cid%22%3A%22717762328687-iludtf96g1hinl76e4lc1b9a82g457nn.apps.googleusercontent.com%22%2C%22k%22%3A%22Google%22%2C%22ses%22%3A%22451ca6a7a6c349248b04db280731cf23%22%7D&response_type=code&flowName=GeneralOAuthFlow'
driver.get(url)
username = driver.find_element_by_id("identifierId")
username.send_keys("fake.user#gmail.com")
my_btn = driver.find_element_by_id('identifierNext')
my_btn.click()
This is because automation testing frameworks are disabled by Google for security reasons
However, there are a few things you can do:
The easiest way: Open browser controlled by selenium, open stack overflow and perform authorisation manually. When you next run the script, it will log you in automatically.
Add arguments to the script that block web security:
from selenium.webdriver.firefox.options import Options
options = FirefoxOptions()
options.add_argument('--disable-web-security')
options.add_argument('--user-data-dir')
options.add_argument('--allow-running-insecure-content')
You can also do this in the terminal.
Login to stackoverflow with your google account, once logged in go to your email.
driver.get("https://accounts.google.com/signin/oauth/identifier?client_id=717762328687-iludtf96g1hinl76e4lc1b9a82g457nn."
+ "apps.googleusercontent.com&scope=profile%20email&redirect_uri=https%3A%2F%2Fstackauth"
+ ".com.com%2Fauth%2Foauth2%2Fgoogle&state=%7B%22sid%22%3A1%2C%22st%22%3A%2259%3A3%3Abbc%2C16%3Af343579f103b7116%2C10%3A1611315201%2C16%3A360adcabd84812cf%2Cc1c668aab33e5327311ff2c3a0cb356918a99d0d6cb65a0da11008340199a802%22%2C%22cdl%22%3Anull%2C%22cid%22%3A%22717762328687-iludtf96g1hinl76e4lc1b9a82g457nn.apps.googleusercontent.com%22%2C%22k%22%3A%22Google%22%2C%22ses%22%3A%225b28397546b247c7994c2d3cd480cc31%22%7D&response_type=code&flowName=GeneralOAuthFlow");
driver.find_element_by_id("identifierId")\ #entering username
.send_keys("email#gmail.com")
sleep(2)
driver.find_element_by_id("identifierNext").click() #clicking next
driver.find_element_by_name("password")\ #entering password
.send_keys("password")
driver.find_element_by_id("passwordNext").click()#clicking next
sleep(2)
driver.get("https://mail.google.com/mail/u/0/#inbox") #going to email inbox
Click Insert → Link or press Ctrl + K (Cmd + K in Mac). You can also click the Insert link button on the toolbar.and your connection problem or email id you can please refresh and restart.
I'm trying to login to Nike.com using selenium but it says "We are unable to connect to our servers right now. Please try again later. Post Request Login Blocked".
I only get this error when I try to login using selenium. When I login manually it gives no error.
Currently I'm using this code,
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--disable-blink-features")
chrome_options.add_argument("--disable-blink-features=AutomationControlled")
driver = webdriver.Chrome(options = chrome_options)
driver.get("https://www.nike.com/login")
email = driver.find_element_by_xpath("/html/body/div[2]/div[3]/div[6]/form/div[2]/input")
email.send_keys("Email Here")
password = driver.find_element_by_xpath("/html/body/div[2]/div[3]/div[6]/form/div[3]/input")
password.send_keys("Password Here")
button = driver.find_element_by_xpath("/html/body/div[2]/div[3]/div[6]/form/div[6]/input")
button.click()
Can anyone help me?
EDIT # 01
Issue is not with the code. Code is working fine. Issue is that Nike is detecting the bot and I want to avoid that.
Hi Nike site not allow to automate their websites by using tools that is reason you cant automate by selenium
You cant do that with selenium but try opening the site on your machine and doing what you need with "webbrowser" here. If this doesn't work tell me.
To login in nike.com Induce WebDriverWait() and wait for element_to_be_clickable() and following locator strategy.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get("https://www.nike.com/login")
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.NAME,"emailAddress"))).send_keys("Abdul Haseeb")
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.NAME,"password"))).send_keys("Abdul Haseeb")
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//input[#value='SIGN IN']"))).click()
Browser snapshot.
your code ran fine on my machine (although I use Java + Selenium, but that doesn't matter I believe). Now coming to your question:
You should try deleting your cookies at first by using the snippet below in some
#BeforeTest section of execution of a test case.
driver.manage().deleteAllCookies();
Next thing, you should update your chromedriver version just for a recheck.
Specifically put a wait before button click.
I'm trying to perform a Keys.ARROW_DOWN in selenium but it doesn't want to work, the code open the context menu, but the key arrow_down don't work, example of what I'm doing:
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.common.keys import Keys
import time
driver = webdriver.Chrome()
driver.get('http://www.google.com.br')
time.sleep(1)
actions = ActionChains(driver)
actions.context_click().send_keys(Keys.ARROW_DOWN).perform()
chromedriver version 83
Someone can give me a light please of what I'm doing wrong?
Thanks for the help!
You can achieve with the help of pyautogui as shown in the below code
import time
import pyautogui
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
options = webdriver.ChromeOptions()
options.add_argument( "user-data-dir=C:\\Users\\Sangeeta-Laptop\\AppData\\Local\\Google\\Chrome\\User Data\\Guest Profile");
cdriver = "C:\\Users\\Sangeeta-Laptop\\Downloads\\chromedriver_win32 (4)\\chromedriver"
driver = webdriver.Chrome(executable_path=cdriver, chrome_options=options)
driver.get('http://www.google.com.br')
time.sleep(1)
actions = ActionChains(driver)
actions.context_click().perform()
time.sleep(1)
pyautogui.press("down");
But as 0buz said in the comment, there are multiple ways to achieve your requirement. So please tell us what are you trying to achieve in detail and maybe we all will help you to resolve your issue :)
Keys.ARROW_DOWN within Context Menu
Context Menu initiated through context_click() is generally invoked on a WebElement e.g. a link.
Invoking context_click() on a element opens a browser native context menu which is a browser native operation and can't be managed by Selenium by design.
Conclusion
Using Selenium you won't be able to interact with browser native context menu items using send_keys(Keys.ARROW_DOWN), send_keys(Keys.DOWN), etc.
Reference
You can find a relevant discussion in:
Could not do Arrow down using sendKeys(Keys.ARROW_DOWN) in chrome
When you perform keys.ARROW_DOWN you should enter values into field then you can perform action.
I'm trying to automate some tedious copy / paste I do monthly from my bank's online service via Selenium and Python 3. Unfortunately, I can't get Selenium to click the log-in link.
It's the blue continue button at https://www1.bmo.com/onlinebanking/cgi-bin/netbnx/NBmain?product=5.
Strangely, when I try to click that link manually in the browser launched by Selenium, it doesn't work either - whereas it does work in a browser I launch manually.
I suspect the issue is that the bank's website is smart enough to detect that I'm automating the browser activity. Is there any way to get around that?
If not, could it be something else?
I've tried using Chrome and Firefox - to no avail. I'm using a 64 bit Windows 10 machine with Chrome 73.0.3683.103 and Firefox 66.0.
Relevant code is below.
#websites and log in information
bmo_login_path = 'https://www1.bmo.com/onlinebanking/cgi-bin/netbnx/NBmain?product=5'
bmo_un = 'fake_user_name'
bmo_pw = 'fake_password'
#Selenium setup
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
chrome_driver_path = 'C:\\Path\\To\\Driver\\chromedriver.exe'
gecko_driver_path = 'C:\\Path\\To\\Driver\\geckodriver.exe'
browswer_bmo = webdriver.Firefox(executable_path = gecko_driver_path)
#browswer_bmo = webdriver.Chrome(executable_path = chrome_driver_path)
#log into BMO
browswer_bmo.get(bmo_login_path)
time.sleep(5)
browswer_bmo.find_element_by_id('siBankCard').send_keys(bmo_un)
browswer_bmo.find_element_by_id('regSignInPassword').send_keys(bmo_pw)
browswer_bmo.find_element_by_id('btnBankCardContinueNoCache1').click()
Sending the keys works perfectly. I may actually have the wrong element ID (I was trying to test that in Chrome when I realized I couldn't click the link manually) - but I think the bigger issue is that I can't manually click the link in the browser launched by Selenium. Thank you for any ideas.
EDIT
This is a screenshot that I get of all I get when I try to click the continue button.
Ultimately the error message I get in my IDE (Jupyter Notebook) is:
TimeoutException: Message: timeout
(Session info: chrome=74.0.3729.108)
(Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729#{#29}),platform=Windows NT 10.0.17134 x86_64)
To click on the button with text as Continue you can fill up the Card Number and Password field inducing WebDriverWait for the element_to_be_clickable() and you can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = webdriver.ChromeOptions()
options.add_argument('start-maximized')
options.add_argument('disable-infobars')
options.add_argument('--disable-extensions')
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://www1.bmo.com/onlinebanking/cgi-bin/netbnx/NBmain?product=5')
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.dijitReset.dijitInputInner#siBankCard[name='FBC_Number']"))).send_keys("1234567890112233")
driver.find_element_by_css_selector("input.dijitReset.dijitInputInner#regSignInPassword[name='FBC_Password']").send_keys("fake_password")
driver.find_element_by_css_selector("span.dijitReset.dijitInline.dijitIcon.dijitNoIcon").click()
# driver.quit()
Browser Snapshot:
I was able to fix this issue and solve the problem by adding the following line below the options variables. This disables the chrome check for automation. I used the whole sale code and then added the following line in the correct location before starting the driver.
options.add_experimental_option("excludeSwitches", ['enable-automation'])
ref: https://help.applitools.com/hc/en-us/articles/360007189411--Chrome-is-being-controlled-by-automated-test-software-notification
I have the following website: https://www.kvk.nl/handelsregister/publicaties/, where I would like to retrieve the login link with Selenium, Scrapy and Python. So for the relevant function, I have the following code:
def start_requests(self):
self.driver = webdriver.Chrome(executable_path=os.path.join(os.getcwd(), "Drivers", "chromedriver.exe"))
self.driver.get(self.initial_url)
test = access_page_wait.until(expected_conditions.visibility_of_element_located((By.CSS_SELECTOR, 'a[data-ui-test-class="linkCard_toegangscode"]')))
if test.is_displayed():
print("+1")
else:
print("-1")
However, this does not seem to work, since it just waits 15 seconds and then it stops. It will never reach +1 or -1.
Now my question is, how can we point selenium to the correct element. It also does not seem to work using XPATH find_elements_by_xpath("//a[#data-ui-test-class='linkCard_toegangscode']").
Should I use another selection approach and if yes, which one?
Because there is Frame which stopping you to access the element.Switch_To iframe and then access the element.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
import os
driver = webdriver.Chrome(executable_path=os.path.join(os.getcwd(), "Drivers", "chromedriver.exe"))
driver.get("https://www.kvk.nl/handelsregister/publicaties/")
driver.switch_to.frame(0)
test=WebDriverWait(driver,10).until(expected_conditions.visibility_of_element_located((By.CSS_SELECTOR, 'a[data-ui-test-class="linkCard_toegangscode"]')))
if test.is_displayed():
print("+1")
else:
print("-1")
Try the above code.It should print what you are looking after.