Selenium Python - How to load existed profile on chrome? - python-3.x

So, I want to use my existed profile on chrome to make easily to login and fetch some data from the website. So I tried this on my current codes but it doesn't load the profile for some reason,
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
import time
class ClientDriver:
def __init__(self):
self.options = Options()
self.options.add_argument("--lang=en_US")
self.options.add_argument("--disable-gpu")
self.options.add_argument("--no-sandbox")
self.options.add_argument("--disable-dev-shm-usage")
self.options.add_argument(
r"user-data-dir=C:\Users\User\AppData\Local\Google\Chrome\User Data\Profile 1"
)
self.options.add_argument("--profile-directory=Profile 1")
def check(self):
driver = webdriver.Chrome(
service=Service(ChromeDriverManager().install()), options=self.options
)
driver.get("chrome://version/")
time.sleep(100)
x = ClientDriver()
x.check()
As can you see on my current codes, it will redirect to chrome://version/ to check the Profile Path and it's not C:\Users\User\AppData\Local\Google\Chrome\User Data\Profile 1 but it was C:\Users\Users\AppData\Local\Google\Chrome\User Data\Profile 1\Profile 1. Can someone help me out?

You're checking chrome://version/ from your selenium test, of course you see a wrong path!
You should instead open a normal chrome window, insert chrome://version/ in the search bar, press enter and check from there your profile path.
You will see it will be something like C:\Users\User\AppData\Local\Google\Chrome\User Data\Profile 1
And so in the code you'll have to write:
self.options.add_argument(
r"user-data-dir=C:\Users\User\AppData\Local\Google\Chrome\User Data"
)
self.options.add_argument("--profile-directory=Profile 1")

Related

how to find the value in WebDriverWait(self.driver, 50).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div[data-result-id*='true']"))

Good evening to all. I am new to python and was just doing one of this exercise in Udemy course where we are tasked to prepare a program using webdriver and getting Internet speed test results on speedtest.net. I was stuck when I found some one gave solution of using:
WebDriverWait(self.driver, 50).until(EC.visibility_of_element_located((By.CSS_SELECTOR,"div[data-result-id*='true']"))
My question is how and where this value for the CSS SELECTOR can be found on the website. Please explain. Please also give as much insight as you can regarding selenium or web driver and CSS in python.
#Libraries
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
#Constants
PROMISED_DOWN = 150
PROMISED_UP = 10
TWITTER_EMAIL = "my email"
TWITTER_PASSWORD = "my password"
#Class
class InternetSpeedTwitterBot:
def __init__(self):
self.s = Service('D:\Python Related Documents and Programsweb development folder\chromedriver_win32\chromedriver.exe')
self.driver = webdriver.Chrome(service=self.s)
self.down = None
self.up = None
def get_internet_speed(self):
self.driver.get("https://www.speedtest.net/")
time.sleep(5)
go_botton = self.driver.find_element(By.CSS_SELECTOR, '.start-button a')
go_botton.click()
#speed_download = self.driver.find_element(By.CSS_SELECTOR, ".download-speed")
speed_download = self.driver.find_element(By.XPATH, '//*[#id="container"]/div/div[3]/div/div/div/div[2]/div[3]/div[3]/div/div[3]/div/div/div[2]/div[1]/div[1]/div/div[2]/span')
WebDriverWait(self.driver, 50).until(
EC.visibility_of_element_located((By.CSS_SELECTOR, "div[data-result-id*='true']"))
)
speed_results = self.driver.find_elements(
By.CSS_SELECTOR, ".result-container-speed span.result-data-large.number.result-data-value"
)
self.down, self.up = (float(result.text) for result in speed_results)
print(f"Down Speed: {self.down}, Up Speed: {self.up}" )
#go_botton.click()
def tweet_at_provider(self):
pass
#Object creation
bot = InternetSpeedTwitterBot()
#Calling methods
bot.get_internet_speed()
bot.get_internet_speed()
What I do to find the elements?:
Open the inspect/dev tool from your browser (I use chrome) using F12 or right click --> Inspect
Click on element tab
Then make Ctrl + f on the element window and new search bar will appear
As you can see that in that bar you can search elements in your page by xpath or CSS.
Click in the button located on the top left of your dev tool. Then that element will be located on your dev tool.
If in your search bar you write a xpath or css selector the element will be highlighted in your screen, in that way you will know what is the selector you can use
Some doc:
How to find elements by CSS selector?: Small guide for CSS
How to find elements by XPATH selector?: Small guide for xpath

Can you actually change the default download directory for an already open chrome session using Selenium on Python?

I'm trying to download some zips files from this page to an specific path for an already chrome browser session open using the code down below:
import time
import numpy as np
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
opt = Options() #the variable that will store the selenium options
opt.add_experimental_option("debuggerAddress", "localhost:9222") #this allows bulk-dozer to take control of your Chrome Browser in DevTools mode.
opt.add_experimental_option("prefs", {"download.default_directory": r"C:\Users\ResetStoreX\Downloads\Binance futures data\ADAUSDT-Mark_Prices_Klines_1h_Timeframe"}) #set the path to save the desired zipped data
s = Service(r'C:\Users\ResetStoreX\AppData\Local\Programs\Python\Python39\Scripts\chromedriver.exe') #Use the chrome driver located at the corresponding path
driver = webdriver.Chrome(service=s, options=opt) #execute the chromedriver.exe with the previous conditions
#Why using MarkPrices: https://support.btse.com/en/support/solutions/articles/43000557589-index-price-and-mark-price#:~:text=Index%20Price%20is%20an%20important,of%20cryptocurrencies%20on%20major%20exchanges.&text=Mark%20Price%20is%20the%20price,be%20fair%20and%20manipulation%20resistant.
if driver.current_url == 'https://data.binance.vision/?prefix=data/futures/um/daily/markPriceKlines/ADAUSDT/1h/' :
number = 2 #initialize an int variable to 2 because the desired web elements in this page starts from 2
counter = 0
the_dictionary_links = {}
while number <= np.size(driver.find_elements(By.XPATH, '//*[#id="listing"]/tr')): #iterate over the tbody array
data_file_name = driver.find_element(By.XPATH, f'//*[#id="listing"]/tr[{number}]/td[1]/a').text
if data_file_name.endswith('CHECKSUM') == False:
the_dictionary_links[data_file_name] = driver.find_element(By.XPATH, f'//*[#id="listing"]/tr[{number}]/td[1]/a').get_attribute('href')
print(f'Saving {data_file_name} and its link for later use')
counter += 1
number += 1
print(counter)
i = 0
o = 0
for i,o in the_dictionary_links.items():
driver.get(o)
print(f'Downloading {i}')
time.sleep(1.8)
And unfortunately it's not working, it throws the following error:
InvalidArgumentException: invalid argument: cannot parse capability:
goog:chromeOptions from invalid argument: unrecognized chrome option:
prefs
So, I would like to know what could have gone wrong? I coded the program above based on this solution but it only seems to work for a new chrome session, and I need the download default directory to be capable of being reset when needed for an already open session. Any ideas?

This website prevent me from signing in, what should I do

I am having error when I try to sendkeys(Keys.ENTER) if I am using python to login this blibli website. But when I try to login normally from chrome browser, I can login normally without this error.
I can input the username and password with selenium python, but then it just ignores the sendkeys(Keys.Enter) command. I have tried adding the login_button.click() command, but the selenium seems to ignore the click() command too. Then, when I tried to click the login button manually, I got an error: "Yah, ada eror nih. Coba lagi, ya." Meaning: "There is an error. Please try again, ok.".
It seems like the website can detect if I am using automated software, thus prevent me from logging into the website, eventhough it is my own account. I have attached the error screenshot. The error prevents me from clicking enter or clicking the blue login button ("Masuk").
Can anybody solve this? Thank you.
The process is complete, because I got this response:
Process finished with exit code 0
This is my code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver import ActionChains
from time import sleep
import os
CHROME_DRIVER_PATH = "C:\Development\chromedriver_win32\chromedriver.exe"
URL = "https://www.blibli.com/login"
EMAIL = os.environ['EMAIL']
BLIBLI_PASSWORD = os.environ["BLIBLI_PASSWORD"]
class Blibli:
def __init__(self):
self.driver = webdriver.Chrome(executable_path=CHROME_DRIVER_PATH)
def login(self):
self.driver.get(URL)
sleep(3)
username = self.driver.find_element_by_class_name("login__username")
username.send_keys(EMAIL)
# actions = ActionChains(self.driver)
# actions.move_to_element(username).send_keys(EMAIL)
password = self.driver.find_element_by_css_selector('input[type="password"]')
password.send_keys(BLIBLI_PASSWORD, Keys.ENTER)
login_button = self.driver.find_element_by_class_name("blu-btn")
login_button.click()
def get_data(self):
pass
bot = Blibli()
bot.login()
bot.get_data()

Fill a table or list of values in textarea in website using Splinter

I am trying to write a script to automate the search for multiple entries. Here is the input:
<mirna-1>
<mirna-2>
<mirna-3>
:
:
The website address is https://www.mirnet.ca/miRNet/faces/upload/MirUploadView.xhtml
How can I fill mirna-1,.... into miRNA list box. I have made the following attempts but failed:
gecko = os.path.normpath('~/geckodriver')
nirnet_Results = []`
browser = Browser(executable_path=gecko)
browser.visit('https://www.mirnet.ca/miRNet/faces/home.xhtml')
# click miRNA_list
xpath = '//*[#id="j_idt39:j_idt55"]'
browser.find_by_xpath(xpath).click()
time.sleep(1)
#Select Human
xpath = '//*[#id="form:j_idt34"]/div[3]/span'
browser.find_by_xpath(xpath).click()
time.sleep(1)
xpath = '//*[#id="form:j_idt34_1"]'
browser.find_by_xpath(xpath).click()
time.sleep(1)
# select miRBase ID type
xpath = '//*[#id="form:j_idt38"]/div[3]/span'
browser.find_by_xpath(xpath).click()
time.sleep(1)
xpath = '//*[#id="form:j_idt38_1"]'
browser.find_by_xpath(xpath).click()
time.sleep(1)
# select Tissue type
xpath = '//*[#id="form:sourceOpt"]/div[3]/span'
browser.find_by_xpath(xpath).click()
time.sleep(1)
xpath = '//*[#id="form:sourceOpt_1"]'
browser.find_by_xpath(xpath).click()
time.sleep(1)
# select target type
xpath = '//*[#id="form:targetOpts"]/div[3]/span'
browser.find_by_xpath(xpath).click()
time.sleep(1)
xpath = '//*[#id="form:targetOpts_1"]'
browser.find_by_xpath(xpath).click()
time.sleep(1)
# Fill miRNA list
xpath = '//*[#id="form:listData"]'
browser.find_by_xpath(xpath).click()
time.sleep(1)
for mir in known_mir:
browser.fill('form:listData',mir)
browser.fill('form:listData','\n')
I am getting success in selecting options in starting drop-down list and when I try to paste a list of names in the miRNA list, I am getting all the names like this:
<mirna-1><mirna-2><mirna-3>....
while the right format is each name should in the next line. How to do this. Kindly suggest. Thanks.
You can fill this form Using Selenium Chrome Driver and python.If you haven't use selenium before and would like use selenium.However here is the official website of selenium.
To install selenium use command line
pip install -U selenium
You need download chrome driver from following website website Chromedriver based on compatibility,such as which chrome browser supports by the chrome driver.Always prefer the latest versions of chrome driver and chrome browser.
Here is the selenium code which fills the form
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
from selenium import webdriver
import time
driver=webdriver.Chrome('path to chrome driver exe') #example : 'C:\chromedriver\chromedriver.exe'
driver.get("https://www.mirnet.ca/miRNet/faces/upload/MirUploadView.xhtml")
#select Organism here
WebDriverWait(driver,5).until(EC.element_to_be_clickable((By.ID,"form:j_idt34_label"))).click()
time.sleep(2)
element=WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.XPATH,"//div[#class='ui-selectonemenu-items-wrapper']//ul[#id='form:j_idt34_items']//li[contains(.,'H. sapiens (human)')]")))
ActionChains(driver).move_to_element(element).send_keys(Keys.UP).send_keys(Keys.DOWN).send_keys(Keys.ENTER).perform()
#select ID type here
WebDriverWait(driver,5).until(EC.element_to_be_clickable((By.ID,"form:j_idt38_label"))).click()
time.sleep(2)
element=WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.XPATH,"//div[#class='ui-selectonemenu-items-wrapper']//ul[#id='form:j_idt38_items']//li[contains(.,'miRBase ID')]")))
ActionChains(driver).move_to_element(element).send_keys(Keys.UP).send_keys(Keys.DOWN).send_keys(Keys.ENTER).perform()
#select Tissue here
WebDriverWait(driver,5).until(EC.element_to_be_clickable((By.ID,"form:sourceOpt_label"))).click()
time.sleep(2)
element=WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.XPATH,"//div[#class='ui-selectonemenu-items-wrapper']//ul[#id='form:sourceOpt_items']//li[contains(.,'Exosomes [1250]')]")))
ActionChains(driver).move_to_element(element).send_keys(Keys.UP).send_keys(Keys.DOWN).send_keys(Keys.ENTER).perform()
#select Target type here
WebDriverWait(driver,5).until(EC.element_to_be_clickable((By.ID,"form:targetOpts_label"))).click()
time.sleep(2)
element=WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.XPATH,"//div[#class='ui-selectonemenu-items-wrapper']//ul[#id='form:targetOpts_items']//li[contains(.,'Genes')]")))
ActionChains(driver).move_to_element(element).send_keys(Keys.UP).send_keys(Keys.DOWN).send_keys(Keys.ENTER).perform()
items=['mirna-1','mirna-2','mirna-3','mirna-4']
for item in items:
element=WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//textarea[#id='form:listData']")))
element.send_keys(item)
element.send_keys(Keys.RETURN)
Output:
Although I would recommend selenium for something like this, it seems you want to stick with splinter. The following fix using splinter. When reproducing this issue, each time browser.fill() is run it replaces the text already written. Since you didn't include what known_mir was in your code, I am assuming it is a list (correct me if I'm wrong). I tried combining what you wanted to put into the form and it worked.
Replace:
for mir in known_mir:
browser.fill('form:listData',mir)
browser.fill('form:listData','\n')
With:
browser.fill('form:listData', '\n'.join(mir for mir in known_mir))
All my code does is combine each of the strings in known_mir.

Automating Dropdown Menu, Select and Send Keys

I am trying to automate some functions for a webpage. Specifically my goal is to automate within the "Download posts by username" section of https://vurku.com/
Whereby I pass username info to the "username" section, click and select images from the "post type" section, and click download.
However I can not seem to pass the username keys into the username section, and am getting an error that states:
line 20, in driver_pass_username_keys
self.driver.find_element(By.XPATH, "//* [#id='collection_username']").sendkeys("guendouglas")
AttributeError: 'FirefoxWebElement' object has no attribute 'sendkeys'
I have worked through many different versions of this code, as I am new to python and selenium, but since I am rather new I am lost. This code worked when I didn't program it from an object oriented perspective. But I'm trying to practice OOP.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.select import Select
from selenium.common.exceptions import TimeoutException
class AutomateVurku:
def __init__(self,driver):
self.driver = driver
def driver_load_page(self):
self.driver.get("https://vurku.com/")
return WebDriverWait(self.driver, 10).until(EC.element_to_be_clickable((By.ID, "collection_username")))
def driver_pass_username_keys(self):
self.driver.find_element(By.XPATH, "//*[#id='collection_username']").sendkeys("guendouglas")
# def driver_select_image_dropdown(self):
if "__main__" == __name__:
driver = webdriver.Firefox(executable_path="/Users/alexandrubordei/Desktop/geckodriver")
myclass = AutomateVurku(driver)
myclass.driver_load_page()
myclass.driver_pass_username_keys()
as stated, I am receiving an error that states that FireFoxWebElement doesn't have an attribute "send keys"
I am not sure what that even means
I guess this was already answered under this question:
Here
Hope, it helps.

Resources