I am working on this as a free time project.
I would like to use Python 3 to login to a site (fill form and submit) and then I plan to scrape the page for some data. The reason I am strictly looking for a solution in Python 3 is because I am trying to learn more about Python and figured I would go straight to Python 3. I have seen several great looking tools such as mechanize but they only seem to support Python 2.
Planning to use this for a financial investment site but let's just use starbucks as an example.
def loginToStockSite(username, pwd):
url = "https://www.starbucks.com/account/signin"
values = {"Account.UserName" : username,
"Account.PassWord" : pwd}
data = urllib.parse.urlencode(values)
data = data.encode('utf-8')
req = urllib.request.Request(url, data)
sock = urllib.request.urlopen(req)
htmlSource = sock.read().decode('utf-8')
return htmlSource
I have gotten very confused from the various encode, decode, url opener, etc. examples. I have not found a solution that has worked for me yet.
Thanks in advance for any help with my "for fun" project.
Update
Here is the html I get back (with a bunch cut out to fit the character limit for a question)
<!DOCTYPE html>
<head>
<meta charset="utf-8" />
<title>Account Sign In | Starbucks Coffee Company</title>
<meta name="viewport" content="width=device-width, initial-scale=1" />
</head>
<div class="fields">
<div class="region size1of1">
<div class="validation_summary warning validation_medium"><h2>Please Enable Cookies to Continue</h2><p>To sign in to the Starbucks web site, please enable cookies in your web browser.</p></div>
<div class="fields">
<form action="/account/signin" class="siren region size1of2" id="accountForm" method="post">
<fieldset class="login_position">
<legend class="hidden_visually">I have a Starbucks account.</legend>
<h2 class="existing_acc_h3">I have a Starbucks account.</h2>
<div id="user_info" class="region size1of2 reset">
Hi,
<span id="info_user"></span>
</div>
<div class="size1of2">
<a id="not_me" href="#notme">Not You?</a>
</div>
</div>
</div>
</div>
<div id="connect_cont" >
<div id="text_cont" class="hidden">
<h3>Have a Starbucks account?</h3>
<p>Connect your Facebook account to your Starbucks account by logging in here.</p>
</div>
</div>
</li>
<li>
<label for="Account_UserName">Username <span class="required">*</span></label>
<label for="Account_UserName" class="hidden">Starbucks Username <span class="required">*</span></label>
<input class="field_xlarge" id="Account_UserName" maxlength="200" name="Account.UserName" type="text" value="MY_USERNAME_WAS_HERE" />
</li>
<li>
<label for="Account_PassWord">Password <span class="required">*</span></label>
<label for="Account_PassWord" class="hidden">Starbucks Password <span class="required">*</span></label>
<input class="field_xlarge password" id="Account_PassWord" maxlength="200" name="Account.PassWord" type="password" value="MY_PASSWORD_WAS_HERE" />
</li>
<li>
Forgot your username or password?
<p class="privacy_policy hidden">
Concerned about privacy?
</p>
</li>
<li class="inline push">
<input type="checkbox" id="Account.IsRememberMe" name="Account.IsRememberMe" value="True" class="checkbox" />
<label for="Account.IsRememberMe">Keep me signed in.</label>
</li>
</ol>
</fieldset>
<fieldset>
<input id="ReturnUrl" name="ReturnUrl" type="hidden" value="" />
<input id="AllowGuest" name="AllowGuest" type="hidden" value="False" />
<input id="isJavaScriptDisabled" name="isJavaScriptDisabled" type="hidden" value="True" />
<span class="button"><button type="submit">Sign In</button></span>
</fieldset>
<fieldset class="submit">
<div id="fb_container">
<div id="fb_btn_cont">
Or log in using Facebook.
<p><a class="fb_button fb_button_medium" id="connect" href="#connect"><span class="fb_button_text">Login with Facebook</span></a></p>
</div>
</div>
</fieldset>
</form>
<div class="region size1of2 block_login">
<h2>I need a Starbucks account.</h2>
<p><span class="button">Create An Account</span></p>
<p>With a Starbucks account you can register and manage your Starbucks Cards and participate in My Starbucks Rewards.</p>
<ul class ="basic">
<li>Enjoy a free drink on your birthday</li>
<li>Protect your balance if your Starbucks Card is missing or stolen.</li>
<li>Transfer money between cards.</li>
<li>Track your earnings in My Starbucks Rewards</li>
<li>Reload your Card balance automatically</li>
</ul>
</div>
</div>
<div class="fields">
<div class="region size1of1">
<ul id="breadcrumb">
<li>Card .<ul>
<li>My Starbucks Rewards .<ul>
<li>View Your Stars</li>
</ul></li>
</ul></li>
</ul>
</div>
</div>
</div>
<div id="footer">
<div class="container">
<form id="search" method="get" action="/search">
<fieldset>
<input id="searchbox" name="keywords" title="Search Keyword" maxlength="100" class="search_input" />
<span class="button button_search"><button id="submit_search_util" type="submit">Search</button></span>
</fieldset>
</form>
<div class="fields">
<div class="region size5of6 suffix1of6">
<div class="footer_categorical"><ol class="blocks blocks-five-up">
<li><h4>
Buy a Card
</h4>
</li>
<li><h4>
Manage Your Card
</h4>
<ol>
<li>Check Balance</li>
<li>Reload Your Card</li>
<li>Transfer Funds</li>
<li>View Transactions</li>
</ol>
</li>
<li><h4>
My Starbucks Rewards
</h4>
<ol>
<li>Register Your Card</li>
<li>View Your Stars</li>
<li>Keep Your Gold Benefits</li>
<li>Rewards Program Terms and Conditions</li>
</ol>
</li>
<li><h4>
Learn More
</h4>
<ol>
<li>Card Terms and Conditions</li>
<li>What is a Starbucks Card eGift?</li>
<li>Card FAQs</li>
<li>Manage Your Account</li>
<li>My Starbucks Idea</li>
</ol>
</li>
I expected to get back a page like I get when I login in using a browser and have a div with the current balance on whatever the default gift card is that is registered.
<div class="balance-amount numbers">
That is the div that is visible. I found the balance in another place in the page source also but I do not see it on the page.
<p class="card_balance numbers">
<span>$27.68</span> <span class="datestamp">3/10/2013 2:12 PM</span>
</p>
Anyways, I would like to input the login info and submit the form using Python 3 or post the data somehow to login and then (as a use case but outside what is needed to answer the question) I would pull the account balance from the html.
I did find that my username and password are apparently in their corresponding fields and in the validation summary it says
<div class="validation_summary warning validation_medium"><h2>Please Enable Cookies to Continue</h2><p>To sign in to the Starbucks web site, please enable cookies in your web browser.</p></div>
I did see some examples that dealt with cookies. Is that the problem? I will research that possible solution. In the meantime, hope this helps you help me. Thanks.
Related
I have an html code with contains
<div class="form-card">
<h2 class="fs-title" test-data="area_1">
About DigCompEdu
</h2>
<p></p>
<label for="academic_teaching" class="question" test-data="question_1">
<b>1- </b>
</label>
<small id="academic_teachingHelp" class="form-text text-muted">
</small>
<div class="form-check question-form-check ">
<div class="row">
<div class="col-1">
<input class="radio" type="radio" value="0" id="3" name="2" test-data="answer_1" />
</div>
</div>
So, I have h1 with testdata name after that i have a form, this form contains question with multin check radio .. i want to access this check , not by just call ([test-data="answer_2"])
because i have another tips and don't want to check them as this one ,
i did smth like this but it's not working :
cy.get('[test-data="area_1"]~[test-data="answer_2"]').check({force: true})
Any one have another idea ?
It might be the ~ between the selectors, I haven't seen that before.
Does it work with a space between instead?
cy.get('[test-data="area_1"] [test-data="answer_2"]').check({force: true})
If the heading is followed by the form like this
<h2 class="fs-title" test-data="area_1">
About DigCompEdu
</h2>
<div class="form-check question-form-check ">
...
then you would query like this
cy.get('[test-data="area_1"]')
.next('.form-check') // next element on page
.find('[test-data="answer_2"]') // this looks inside '.form-check'
.check({force: true})
I see the following HTML code when I import it from website throught selenium. The ids I want to access are hidden inside the body and I am not able to access them using Selenium Python.
</head>
<!--[if IE 9]>
<body style="display:none" class="ie9">
<![endif]-->
<!--[if (gt IE 9)|!(IE)]><!-->
<body style="display:none">
<!--<![endif]-->
This is the code I see in the website through inspect option.
<form action="" method="post" class="form login col-sm-6 col-sm-offset-3" data-stage="DataStore1">
<fieldset class="row">
<div class="form-group">
<label class="sr-only separator" for="idToken1">
<span>Benutzername</span>
</label>
<input type="text" id="idToken1" name="callback_0" value="" data-validator="required" required="" data-validator-event="keyup" class="form-control input-lg" placeholder="Benutzername" autofocus="">
</div>
<div class="form-group">
<label for="idToken2" class="aria-label sr-only separator">
<span>Passwort</span>
</label>
<input type="password" id="idToken2" name="callback_1" class="form-control input-lg" placeholder="Passwort" value="" data-validator="required" required="" data-validator-event="keyup">
</div>
<div class="remember-forgot clearfix">
<div class="checkbox pull-left">
<label for="remember"><input type="checkbox" name="loginRemember" id="remember"> Remember my username</label>
</div>
</div>
<div class="form-group">
<input id="loginButton_0" name="callback_2" type="submit" role="button" index="0" value="Log in" class="btn btn-lg btn-block btn-uppercase btn-primary">
</div>
ch license between the licensee and ForgeRock AS.
</fieldset>
</form>
I am using Chrome webdriver.
from selenium import webdriver
driver = webdriver.Chrome(r'C:\WebDriver\bin\chromedriver.exe')
driver.get('https://app.int.eu.acdc.fdc.swarm.audi/goldenalgo#dario/data-distribution')
a = driver.find_element_by_id("idToken1")
a.send_keys(username)
print(w.value_of_css_property("display"))
print(driver.page_source)
time.sleep(5)
driver.find_element_by_id('idToken2').send_keys(password)
driver.find_element_by_id('loginButton_0').click()
driver.close()
driver.find_element_by_id("idToken1") this doesn't work and returns the error "unable to find element with css selector error in selenium".
Thanks in advance
I'm working with Stripe API and moving from v2 to v3.
I need to confirm that this form is SAQ-A (not SAQ-AEP)
Full codes: http://jsfiddle.net/xz1db74n/
HTML:
<script src="https://js.stripe.com/v3/"></script>
<body>
<form action="//httpbin.org/post" method="POST">
<input type="hidden" name="token" />
<div class="group">
<label>
<span>Card number</span>
<div id="card-number-element" class="field"></div>
</label>
<label>
<span>Expiry date</span>
<div id="card-expiry-element" class="field"></div>
</label>
<label>
<span>CVC</span>
<div id="card-cvc-element" class="field"></div>
</label>
</div>
<button type="submit">Pay $25</button>
<div class="outcome">
<div class="error"></div>
<div class="success">
Success! Your Stripe token is <span class="token"></span>
</div>
</div>
</form>
</body>
The code you shared doesn't really detail how you are integrating exactly. The only reference is loading Stripe.js v3 without showing how you use it. Overall though, as long as you use Stripe Elements and load the page over HTTPS then you will meet the requirements for SAQ A. This covered in details in Stripe's documentation here: https://stripe.com/docs/security/guide#validating-pci-compliance
I try to create a simple contact form, but I didn't get any message, the post request is made. I am pretty new with Modx.
What I tried:
Link 1
Link 2
I created 2 Chunks (emailChunkTpl and MyEmailChunk), and in my template I call for the [[$MyEmailChunk]]. Obviously I did something wrong but I am not sure what. The code is like in the examples, but with some changes,like my email.
[[!FormIt?
&hooks=`email,FormItSaveForm `
&emailTpl=`emailChunkTpl`
&emailTo=`myEmail#gmail.com`
&emailUseFieldForSubject=`1`
]]
<form action="[[~[[*id]]]]" method="post" class="contactForm">
<div class="row input-section-child">
<div class="col input-contact">
<input value="[[!+fi.input-name]]"class="input-name" name="input-name" id="input-name" type="text" placeholder="your name" />
<input value="[[!+fi.input-email]]" class="input-email" name="input-email" id="input-email" type="text" placeholder="email address" />
</div>
<div class="col input-contact-text">
<input value="[[!+fi.input-textare]]"class="input-textare" name="input-textare" id="input-textare" type="textare" placeholder="message" />
</div>
</div>
<div class="row second-row">
<div class="col checkbox">
<input value="[[!+fi.input-checkbox]]" class="input-checkbox" name="input-checkbox" id="input-checkbox" type="checkbox" ><span class="input-checkbox-span">I agree to the Privacy policy.</span>
</div>
<div class="col">
<button type="submit" class="send-button">SEND</button>
</div>
</div>
<a href="#intro" class="back-to-top"><img src="images\arrow-back.svg" /><span>Back to top</span>
</div>
</from>
You should start with cleaning up your markup, it is not valid HTML. There is a typo in your closing form tag (/from), an extra closing div tag, there is no input type of textare as far as I know, and you are missing white space around some classes. You can validate HTML here: https://validator.w3.org/
You can test to make sure your site is setup properly to send an email. There is a snippet for this called QuickEmail, download it from the extras tab in the MODX manager.
Once you're sure your site can send email, then start with a stripped down version of the Formit call -- remove all hooks except email and get it working with that first. Then add hooks one at a time.
I got redirected after I submit the form, and I want to remain on the same page. I am not sure why I got redirected, I am not using &redirectTo for this.
I try some things but nothing worked till now.
[[!FormIt?
&hooks=`spam,email,FormItSaveForm,successMess`
&formName= `Contact Form`
&emailTpl=`emailChunkTpl`
&emailTo=`email#gmail.com`
]]
<form action="[[~[[*id]]]]" method="post" class="contactForm">
<input type="hidden" name="nospam:blank" value="" />
<div class="row input-section-child">
<div class="col input-contact">
<input value="[[!+fi.input-name]]"class="input-name" name="input-name" id="input-name" type="text" placeholder="your name" />
<span class="error-message error" > [[!+fi.error.input-name]] </span>
<input value="[[!+fi.input-email]]" class="input-email" name="input-email" id="input-email" type="text" placeholder="email address" />
<span class="error-message error" >[[!+fi.error.input-email]] </span>
</div>
<div class="col input-contact-text">
<input value="[[!+fi.input-textare]]"class="input-textare" name="input-textare" id="input-textare" type="textare" placeholder="message" />
</div>
</div>
<div class="row second-row">
<div class="col">
<button type="submit" class="send-button">SEND</button>
</div>
[[+placeholder]]
</div>
</div>
</from>
As in my answer on your other post, you should remove all the hooks and try one at a time (after you've cleaned up the markup for this form).
And what is successMess? Is that a custom snippet you've made? It's not a Formit hook that I have heard of. If it's your custom snippet you should paste the code into your question so we can see it.