I want to extract 3rd line from a text input - python-3.x

Question

I have to extract the third line of a text.
URL = 'https://simonsmith.github.io/github-user-search/#/search?q=benjamn'
browser = webdriver.Chrome()
browser.get(URL)
time.sleep(20)
content = browser.page_source
soup = BeautifulSoup(content)
for link in soup.find_all('a'):
n=link.get('href')
n = re.sub(r"\#",'',n)
print(n)
In this example I would only collect benjamn from the output and discard the rest.
OUTPUT:
/
https://developer.github.com/v3/
/benjamn
/BenjamNathan
/benjamni
/benjamnnzz
/BenjamnTal
/benjamncresnik
/benjamn1012990
/benjamnsmith
/benjamn77
/BENJAMNDO4FO
/benjamnzzzz
/benjamn25
/benjamnn
/benjamn2
/benjamnwilliams
https://github.com/simonsmith/github-user-search

Answer 1

You wrote:
for link in soup.find_all('a'):
Suppose you instead had:
links = list(soup.find_all('a'))
for link in links:
Then links[2] would contain the desired link.
Equivalently, you could use:
for i, link in enumerate(soup.find_all('a')):
and focus on the particular link where i == 2.

Answer 2

You can instead better refine your selector and use a css selector
if you use either
li .u-flex
or
[class^=User].u-flex
the first being faster; you will get only the 15 links for people. If you then use find_element_by_css_selector, you will only return the first match.
That is:
browser.find_element_by_css_selector("li .u-flex").get_attribute("href")
No need for BeautifulSoup but the equivalent is:
soup.select_one('li .u-flex')['href']

Answer 3

To fetch the value benjamn use WebdriverWait and element_to_be_clickable with following xpath.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
URL = 'https://simonsmith.github.io/github-user-search/#/search?q=benjamn'
browser = webdriver.Chrome()
browser.get(URL)
element=WebDriverWait(browser, 15).until(EC.element_to_be_clickable((By.XPATH,"(//a[starts-with(#class,'User_')]//p[starts-with(#class,'User_')])[1]")))
print(element.text)
Output Printed on console:
benjamn
To Print all the text values use following code.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
URL = 'https://simonsmith.github.io/github-user-search/#/search?q=benjamn'
browser = webdriver.Chrome()
browser.get(URL)
elements=WebDriverWait(browser, 15).until(EC.visibility_of_all_elements_located((By.XPATH,"//a[starts-with(#class,'User_')]//p[starts-with(#class,'User_')]")))
for element in elements:
print(element.text)
Output:
benjamn
BenjamNathan
benjamni
benjamnnzz
BenjamnTal
benjamncresnik
benjamn1012990
benjamnsmith
benjamn77
BENJAMNDO4FO
benjamnzzzz
benjamn25
benjamnn
benjamn2
benjamnwilliams

Answer 4

You can grab that link using selenium making use of xpath and surely not hardcoding index like the following:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
LINK = 'https://simonsmith.github.io/github-user-search/#/search?q=benjamn'
with webdriver.Chrome() as driver:
wait = WebDriverWait(driver, 10)
driver.get(LINK)
expected_link = wait.until(EC.presence_of_element_located((By.XPATH,"//a[./*[contains(#class,'username')]]")))
print(expected_link.get_attribute("href"))
Output:
https://simonsmith.github.io/github-user-search/#/benjamn