I was just wondering, how can i hide the fact that i'm using selenium to access epic games store site? It heavily kicks me even if i'm just browsing the site myself using webdriver. I saw similar posts about this topic on stack, but they didn't help me. Here's the settings I have now. What's wrong with them?
from fake_useragent import UserAgent
from selenium import webdriver
ua = UserAgent()
us_ag = ua.random
options = webdriver.ChromeOptions()
options.add_argument(f'user-agent={us_ag}')
options.add_argument('--ignore-ssl-errors')
options.add_argument('window-size=1280,800')
options.add_argument('--ignore-certificate-errors-spki-list')
options.add_argument('--disable-blink-features=AutomationControlled')
driver = webdriver.Chrome(options=options)
The site keeps telling me that i've done captcha wrong. Even if I try for another 10 times - he still tells me the same. There are no problems when I'm browsing with my regular browser.
Related
I am trying to close portable browser via selenium
I passed --remote-debugging-port=9222 because if I do not pass it then the program is stuck in object creation of webdriver.Chrome(). It will open the portable browser but not load the URL.
But after the URL is open I want to close the browser but driver.quit() it is not working for me. I have tried some other methods for close the browser but they do not work as well.
I want to close the specific instance of the browser which is opened by this program not other opened instances of the browser.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.binary_location = 'C:/Portable/GoogleChromePortable/GoogleChromePortable.exe'
chrome_options.add_argument("--remote-debugging-port=9222")
chrome_options.add_argument("--incognito")
chrome_options.add_argument("--profile-directory=Person 1")
driver = webdriver.Chrome(options=chrome_options,executable_path='C:/Portabl/chromedriver_win32/chromedriver.exe')
url = "https://www.google.com/"
driver.get(url)
driver.quit()
I am using:
selenium 3.141.0, windows 10, python 3.8.0, portable chrome version 93.0.4577.63 (32-bit)
Your this statement
I passed --remote-debugging-port=9222 because if I do not pass it then
the program is stuck in object creation of webdriver.Chrome()
is not correct. --remote-debugging-port=9222 looks like a port number where your application is deployed and you have used chrome option to send them to browser object.
driver.quit()
this typically should have worked, what is the error when it did not work ?
also, for closing a single instance you could do
driver.close()
see if that helps.
Is there any whatsapp or webdriver API to access WhatsApp web without scanning QR code everytime while accessing it using selenium and chrome webdriver in python?
This is What you need. This code Read QR and store it in cookies
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
jokes = ["You don't need a parachute to go skydiving. You need a parachute to go skydiving twice.",
"This is Test Message."]
options = Options()
options.add_argument("--user-data-dir=chrome-data")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome('/usr/local/bin/chromedriver', options=options)
driver.maximize_window()
driver.get('https://web.whatsapp.com') # Already authenticated
time.sleep(20)
##################### Provide Recepient Name Here ###############################
driver.find_element_by_xpath("//*[#title='MyJakartaNumber']").click()
for joke in jokes:
driver.find_element_by_xpath('//*[#id="main"]/footer/div[1]/div[2]/div/div[2]').send_keys(joke)
driver.find_element_by_xpath('//*[#id="main"]/footer/div[1]/div[3]/button/span').click()
time.sleep(10)
time.sleep(30)
driver.close()
Your "WhatsApp" and "QR Code" don't tell anything to me, however if you're testing an application which requires an extra action to sign in I don't think you will be able to perform it using Selenium as it's browser automation framework.
Web applications identify users via Cookies - special HTTP Headers containing client-side information. When you start a web browser via Selenium bindings - it kicks off a clean browser session which is not authenticated in "WhatsApp" (whatever it is)
The solutions would be in:
Authenticate in WhatsApp manually, store your browser profile somewhere and start Selenium by pointing it to the earlier storied profile folder
Authenticate in WhatsApp manually, store your browser cookies and use WebDriver.add_cookie() function to read the stored cookies into the current clean session
You can use "pywhatkit". pywhatkit is used to send messages using whatssapp web. Run:
pip install pywhatkit
and you are good to go.
Below code is creating an empty task_list.html file. It is not saving the full html content file , which is what i want . How to fix it?
from selenium import webdriver
import codecs
driver = webdriver.Firefox()
html = driver.page_source
driver.get("<our_internal_work_website>")
with open(r"C:\Users\task_list.html", "wb") as f:
f.write(html.encode('utf-8'))
Few points:
I am brand new to Python, just started few days back and troubleshooting issues as they come by searching on stackoverflow.
I am running this code on interactive shell on Powershell in Windows 10.
I manually enter the login credentials when opens . I have still not reached expertise to be able to put login details automatically.
I do not have html/css knowledge , so will google around in order to troubleshoot.
I want to use beautiful soup in order to parse the html , but it also requires login credentials and i do not know where to put the login credentials because it doesnt explicitly open the "Firefox" browser. Due to that , for now i am working with Selenium, because it can open the Firefox browser.
When I FireFox is loaded through Selenium, my browser is under remote controller and a bot image show in URL section in my browser. For deal with this problem I changed User-Agent by this code:
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_preference("general.useragent.override", "whatever you want")
driver = webdriver.Firefox(profile)
User-Agent was changed successfully but, bot image in URL section of my browser was remained.
Would you help me, please?
I used this URL for changing User-Agent:
Change user agent for selenium driver
My whole code is:
MainLink="https://blog.feedspot.com/iot_blogs/"
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
caps = DesiredCapabilities.PHANTOMJS
caps["phantomjs.page.settings.userAgent"] = "whatever you want"
driver = webdriver.Firefox()
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_preference("general.useragent.override", "whatever you want")
driver = webdriver.Firefox(profile)
agent = driver.execute_script("return navigator.userAgent")
print(agent)
driver.get(MainLink)
Your code is a little confusing. You don't need to use both phantomjs and firefox as the driver for selenium. Which one are you going to use?
As I understood, you would like to avoid being detected from the page you're interacting with. This is usually a greater concern if you're operating with a headless browser, which is the case when using phantomjs but not when using firefox without explicitly telling it to run in this mode, which apparently is your case.
If you are in fact having issues of this nature, there are many ways to try to mitigate this, starting from changing the user agent, as mentioned by you. Assuming you want to go with firefox, A code example would be:
from selenium import webdriver
profile = webdriver.FirefoxProfile()
user_agent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36'
profile.set_preference("general.useragent.override", user_agent)
driver = webdriver.Firefox(profile)
MainLink="https://blog.feedspot.com/iot_blogs/"
driver.get(MainLink)
In addition, you could set a different user agent each time you make a request, combining with changing the ip address from where the requests are originated, of course, but this is besides the point...
Hope this helps...
Depending on how "deep" you want to go, you could also add cookies copied from a trusted client:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get("http://www.example.com")
# Adds the cookie into current browser context
driver.add_cookie({"name": "key", "value": "value"})
See also: https://www.selenium.dev/documentation/en/support_packages/working_with_cookies/
I could run my code without loading page or in headless way (headless browser).
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
options = Options()
options.add_argument("--headless")
driver = webdriver.Firefox(firefox_options=options)
print("Firefox Headless Browser Invoked")
driver.get('https://blog.feedspot.com/iot_blogs/')
s=driver.find_element_by_xpath("""/html/body/div[1]/div[2]/div/div/div[1]/article/div[1]/h1""")
print(s.text)
I'm using Selenium 3.5 for Python to test some behaviors of my web app using both local pages and online ones.
Until now, I just used Chrome as main browser to test it and everything works fine. Then, I decided to use Firefox 55.0.3 using geckodriver v0.19.0.
I have the following issues:
selenium opens the local pages such as file:///PATH/index.html but it doesn't close the browser even using the function quit();
I would use some browser option such as the incognito mode, but I was not able to find a list of all available options for Firefox.
This is a snippet of my code to use selenium with Firefox.
from selenium import webdriver
url = 'http://www.yahoo.com'
#url = 'file:///PATH/index.html'
browser = webdriver.Firefox()
browser.get(url)
browser.quit()
# borser.close() -> I also tried with the close()
I also read this question for the first issue but it doesn't work with the current version of the used framework.
About the second I can only find partial solution such as this one for the incognito mode but it doesn't work as expected.
Any ideas? Is it a kind of bug in geckodriver?