#Required Boundaries
from selenium import webdriver
import time
#Opens Chrome Profile
options = webdriver.ChromeOptions()
options.add_argument("/Users/paolamohan/Library/Application Support/Google/Chrome/Default")
driver = webdriver.Chrome(executable_path=r"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",options=options)
time.sleep(2)
#Gmail
Gmail = driver.find_element_by_xpath('//*[#id="gb"]/div[2]/div[3]/div[1]/div/div[1]/a')
Gmail.click()
Here is my Code. I figured out how to make the Webdriver open up Chrome with my Chrome Profile but now I can't run anything else after it executes. For this simple example, I just want the script to click the little "Gmail" icon at the top right corner of Google's home page
I'm new to Python so any written code would be extremely useful!
Thank you
Before you can click the button Selenium needs to open the page with driver.get(url).
So you do:
#Required Boundaries
from selenium import webdriver
import time
#Opens Chrome Profile
options = webdriver.ChromeOptions()
options.add_argument("/Users/paolamohan/Library/Application Support/Google/Chrome/Default")
driver = webdriver.Chrome(executable_path=r"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",options=options)
time.sleep(2)
#Access Site
url = "https://google.de"
driver.get(url)
#Gmail
Gmail = driver.find_element_by_xpath('//*[#id="gbw"]/div/div/div[1]/div[1]/a')
Gmail.click()
Related
I want to scrape the comments off this page using beautifulsoup - https://www.x....s.com/video_id/the-suburl
The comments are loaded on click via Javascript. The comments are paginated and each page loads comments on click too. I wish to fetch all comments, for each comment, I want to get the poster profile url, the comment, no. of likes, no of dislikes, and time posted (as stated on the page).
The comments can be a list of dictionaries.
How do I go about this?
This script will print all comments found on the page:
import json
import requests
from bs4 import BeautifulSoup
url = 'https://www.x......com/video_id/gggjggjj/'
video_id = url.rsplit('/', maxsplit=2)[-2].replace('video', '')
u = 'https://www.x......com/threads/video/ggggjggl/{video_id}/0/0'.format(video_id=video_id)
comments = requests.post(u, data={'load_all':1}).json()
for id_ in comments['posts']['ids']:
print(comments['posts']['posts'][id_]['date'])
print(comments['posts']['posts'][id_]['name'])
print(comments['posts']['posts'][id_]['url'])
print(BeautifulSoup(comments['posts']['posts'][id_]['message'], 'html.parser').get_text())
# ...etc.
print('-'*80)
This would be done with Selenium. Selenium emulates a browser. Depending on your preferences you can use a chrome driver or the Firefox driver which is the geckodriver.
Here is a link on how to install the chrome webdriver:
http://jonathansoma.com/lede/foundations-2018/classes/selenium/selenium-windows-install/
Then in your code here is how you would set it up:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
# this part may change depending on where you installed the webdriver.
# You may have to define the path to the driver.
# For me my driver is in C:/bin so I do not need to define the path
chrome_options = Options()
# or '-start maximized' if you want the browser window to open
chrome_options.add_argument('--headless')
driver = webdriver.Chrome(options=chrome_options)
driver.get(your_url)
html = driver.page_source # downloads the html from the driver
Selenium has several functions that you can use to perform certain actions such as click on elements on the page. Once you find an element with selenium you can use the .click() method to interact with the element.
Let me know if this helps
I am able to open Chrome via Selenium, but am unable to simulate a key press (specifically F12, since I want to open Inspect and eventually use the mobile browser Like so) While I'm able to do it manually i.e, open Chrome and press F12, I want to be able to automate this part using Selenium. My current code looks like this -
from selenium import webdriver
import time
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument("--test-type")
options.binary_location = "/usr/bin/chromium"
driver = webdriver.Chrome('/Users/amigo/Documents/pet_projects/selenium/chromedriver')
driver.get('https://www.google.com')
ActionChains(driver).send_keys(Keys.F12).perform()
While the code runs without any errors, I do not see the inspect being opened on the browser. Any suggestions and help appreciated! Thank you in advance.
Simulating the key press for F12 resembles to opening the google-chrome-devtools.
To open the google-chrome-devtools i.e. the chrome-browser-console you have to use the ChromeOptions class to add the argument --auto-open-devtools-for-tabs argument as follows:
Code Block:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_argument("--auto-open-devtools-for-tabs")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("https://selenium.dev/documentation/en/")
print(driver.title)
Console Output:
The Selenium Browser Automation Project :: Documentation for Selenium
Browser Console Snapshot:
You can find a relevant java based discussion in How to open Chrome browser console through Selenium?
As I can not add a comment, just writing as a new answer for others. Just tried that with latest Chrome Driver (100.0.4896) and Python 3.7 -- following is working as well.
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--auto-open-devtools-for-tabs")
driver = webdriver.Chrome(ChromeDriverManager().install(), options=options)
home_page_url = "https://stackoverflow.com/"
driver.get(home_page_url)
I'm trying to automate some tedious copy / paste I do monthly from my bank's online service via Selenium and Python 3. Unfortunately, I can't get Selenium to click the log-in link.
It's the blue continue button at https://www1.bmo.com/onlinebanking/cgi-bin/netbnx/NBmain?product=5.
Strangely, when I try to click that link manually in the browser launched by Selenium, it doesn't work either - whereas it does work in a browser I launch manually.
I suspect the issue is that the bank's website is smart enough to detect that I'm automating the browser activity. Is there any way to get around that?
If not, could it be something else?
I've tried using Chrome and Firefox - to no avail. I'm using a 64 bit Windows 10 machine with Chrome 73.0.3683.103 and Firefox 66.0.
Relevant code is below.
#websites and log in information
bmo_login_path = 'https://www1.bmo.com/onlinebanking/cgi-bin/netbnx/NBmain?product=5'
bmo_un = 'fake_user_name'
bmo_pw = 'fake_password'
#Selenium setup
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
chrome_driver_path = 'C:\\Path\\To\\Driver\\chromedriver.exe'
gecko_driver_path = 'C:\\Path\\To\\Driver\\geckodriver.exe'
browswer_bmo = webdriver.Firefox(executable_path = gecko_driver_path)
#browswer_bmo = webdriver.Chrome(executable_path = chrome_driver_path)
#log into BMO
browswer_bmo.get(bmo_login_path)
time.sleep(5)
browswer_bmo.find_element_by_id('siBankCard').send_keys(bmo_un)
browswer_bmo.find_element_by_id('regSignInPassword').send_keys(bmo_pw)
browswer_bmo.find_element_by_id('btnBankCardContinueNoCache1').click()
Sending the keys works perfectly. I may actually have the wrong element ID (I was trying to test that in Chrome when I realized I couldn't click the link manually) - but I think the bigger issue is that I can't manually click the link in the browser launched by Selenium. Thank you for any ideas.
EDIT
This is a screenshot that I get of all I get when I try to click the continue button.
Ultimately the error message I get in my IDE (Jupyter Notebook) is:
TimeoutException: Message: timeout
(Session info: chrome=74.0.3729.108)
(Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729#{#29}),platform=Windows NT 10.0.17134 x86_64)
To click on the button with text as Continue you can fill up the Card Number and Password field inducing WebDriverWait for the element_to_be_clickable() and you can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = webdriver.ChromeOptions()
options.add_argument('start-maximized')
options.add_argument('disable-infobars')
options.add_argument('--disable-extensions')
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://www1.bmo.com/onlinebanking/cgi-bin/netbnx/NBmain?product=5')
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.dijitReset.dijitInputInner#siBankCard[name='FBC_Number']"))).send_keys("1234567890112233")
driver.find_element_by_css_selector("input.dijitReset.dijitInputInner#regSignInPassword[name='FBC_Password']").send_keys("fake_password")
driver.find_element_by_css_selector("span.dijitReset.dijitInline.dijitIcon.dijitNoIcon").click()
# driver.quit()
Browser Snapshot:
I was able to fix this issue and solve the problem by adding the following line below the options variables. This disables the chrome check for automation. I used the whole sale code and then added the following line in the correct location before starting the driver.
options.add_experimental_option("excludeSwitches", ['enable-automation'])
ref: https://help.applitools.com/hc/en-us/articles/360007189411--Chrome-is-being-controlled-by-automated-test-software-notification
I am trying to login on a website using the selenium webdriver in Python 3. First, I need to click the button "Inloggen", after which I need to fill in username and password and than click the (new) button "Inloggen" again.
So, I've tried to locate the first "Inloggen" button (with the code below), and tried to .click() it, but then it raises an error "selenium.common.exceptions.WebDriverException: Message: ", but without message.
from selenium import webdriver
# go to login page and sign in
driver = webdriver.Firefox()
driver.get("https://www.qassa-nl.be/")
driver.find_element_by_xpath("//a[#title='Inloggen']").click()
Secondly, if this works, I can send my login keys using the classic way I guess.
Best,
Tim
Here is the Answer to your Question:
Here is the working code block which will open the url https://www.qassa-nl.be/, click on button Inloggen, fills up email, fills up password and finally clicks on Inloggen button:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
binary = FirefoxBinary('C:\\Program Files\\Mozilla Firefox\\firefox.exe')
caps = DesiredCapabilities().FIREFOX
caps["marionette"] = True
driver = webdriver.Firefox(capabilities=caps, firefox_binary=binary, executable_path="C:\\Utility\\BrowserDrivers\\geckodriver.exe")
driver.get("https://www.qassa-nl.be/")
driver.find_element_by_xpath("//div[#id='personal_info']//a[text()='Inloggen']").click()
driver.find_element_by_xpath("//input[#id='login_username']").send_keys("debanjan")
driver.find_element_by_xpath("//input[#id='login_password']").send_keys("debanjan")
driver.find_element_by_xpath("//button[#title='Inloggen']").click()
Let me know if this Answers your Question.
I'd like to start an extension(unpacked) by its hotkey in Chrome via Webdriver. I'm not able to have the hotkey/shortkey in the running ChromeDriver instance. Can someone tell me, how to do this?
This is the code so far:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("load-extension=C:\\Users\\...\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\Extensions\\blablabla\\4.3.25_0")
browser = webdriver.Chrome(chrome_options=chrome_options)
This inits a new Chrome with the extension, but the original hotkey isn't included. My idea is to add some Option or Capability or prefs, but haven't found anything similar yet.
It would also be OK if I could create a new profile, install the extension, set the hotkey, and start Chrome with this profile, but I need those profile creating steps programatically, also.
Thanks for any feedback!
This solved the question (I eliminated any command that was not with WebDriver, for example pyautogui.hotkey, etc):
chrome_options = Options()
chrome_options.add_argument("load-extension=" + ext_folder)
browser = webdriver.Chrome(chrome_options = chrome_options)
browser.get('chrome://extensions-frame/')
browser.find_element(By.XPATH, "//a[#class='extension-commands-config']").click()
browser.find_element(By.XPATH, "//span[#class='command-shortcut-text']").send_keys(Keys.CONTROL + "m")
browser.find_element(By.ID, "extension-commands-dismiss").click()
browser.find_element(By.TAG_NAME, "body").send_keys(Keys.CONTROL + "m")