I'm trying to access an internal sharepoint and I haven't been able to figure out how to do it. I've seen some examples and came up with some code but I got a 401 error out of it. I obviously replace my USERNAME and PASSWORD with my actual credentials but I still got a 401. I'm attempts to access the SharePoint and grab lists of data (That span multiple pages). Right now I'm trying to just access the site and proceed from there.
import requests
from requests.auth import HTTPBasicAuth
from requests_ntlm import HttpNtlmAuth
response = requests.get("http://share-internal.division.com/teams/alpha/Components/Lists/Types/AllItems.aspx", auth=HttpNtlmAuth('DOMAIN\\USERNAME','PASSWORD'))
print (response.status_code)
I also tried a Basic request such as
import requests
from requests.auth import HTTPBasicAuth
response = requests.get("http://share-internal.Server.com/teams/Beta/Components/Lists/Workt/AllItems.aspx", auth=HTTPBasicAuth(USERNAME, PASSWORD))
print (response.status_code)
Just to be honest: I censored some of the URL as to maintain privacy
Related
There is one public html webpage where some content is available. With urllib I am able to fetch contents. But there another version where if I am logged in as user, additional content is available. in order to fetch logged-in page I use solution provided on portal and also urllib basic auth
import urllib.request
passman = urllib.request.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, url, username, password)
authhandler = urllib.request.HTTPBasicAuthHandler(passman)
opener = urllib.request.build_opener(authhandler)
urllib.request.install_opener(opener)
res = urllib.request.urlopen(url)
res_body = res.read()
res_body.decode('utf-8')
When I check contents I do not see what is additionally available for logged-in user . I would like to know how to correct this . I am using python 3.
Thanks
I am trying to access the Youtube API via python so that I can insert videos into a playlist. To do so I have to use oauth2 to authorize myself. I have looked up many guides and videos and followed their instructions step by step but I always get the same result, saying Error 400: redirect_uri_mismatch
The code that I am using is
from google_auth_oauthlib.flow import InstalledAppFlow
flow = InstalledAppFlow.from_client_secrets_file("client_secrets.json", scopes=['https://www.googleapis.com/auth/youtube.force-ssl'])
flow.run_local_server(port=8080, prompt='consent')
credentials = flow.credentials
print(credentials.to_json())
I have also tried
from google_auth_oauthlib.flow import InstalledAppFlow
flow = InstalledAppFlow.from_client_secrets_file("client_secrets.json", scopes=['https://www.googleapis.com/auth/youtube.force-ssl'])
flow.run_console()
credentials = flow.credentials
print(credentials.to_json())
I understand that this occurs when there is a mismatch between the redirect_uri on the request side and what is listed in the console. But I have triple checked that on https://console.developers.google.com/ I have the redirect URI to be http://localhost.com:8080 and have checked to make sure that my client_secrets.json file reflects the same. I have set my JavaScript origins to also be http://localhost.com:8080. I have set my oauth clientID to be a web application as I am running this in pycharm. However, whenever I run my code I get the same error saying The redirect URI in the request, http://localhost:8080/, does not match the ones authorized for the OAuth client. To update the authorized redirect URIs, visit: https://console.developers.google.com/apis/credentials/oauthclient/${your_client_id}?project=${your_project_number}
I have looked at the other questions on stackOverflow and have tried the solutions that were suggested but none of them have worked. I have no idea why I cannot get authorization and desperately need help
I'm trying to download some files from behind a SSO (Single Sign-On) site. It seems to be SAML authenticated, that's where I'm stuck. Once authenticated I'll be able to perform API requests that return JSON, so no need to interpret/scrape.
Not really sure how to deal with that in mechanicalsoup (and relatively unfamiliar with web-programming in general), help would be much appreciated.
Here's what I've got so far:
import mechanicalsoup
from getpass import getpass
import json
login_url = ...
br = mechanicalsoup.StatefulBrowser()
response = br.open(login_url)
if verbose: print(response)
# provide the username + password
br.select_form('form[id="loginForm"]')
print(br.get_current_form().print_summary()) # Just to see what's there.
br['UserName'] = input('Email: ')
br['Password'] = getpass()
response = br.submit_selected().text
if verbose: print(response)
At this point I get a page telling me javascript is disabled and that I must click submit to continue. So I do:
br.select_form()
response = br.submit_selected().text
if verbose: print(response)
That's where I get a complaint about state information being lost.
Output:
<h2>State information lost</h2>
State information lost, and no way to restart the request<h3>Suggestions for resolving this problem:</h3><ul><li>Go back to the previous page and try again.</li><li>Close the web browser, and try again.</li></ul><h3>This error may be caused by:</h3><ul><li>Using the back and forward buttons in the web browser.</li><li>Opened the web browser with tabs saved from the previous session.</li><li>Cookies may be disabled in the web browser.</li></ul>
The only hits I've found on scraping behind SAML logins are all going with a selenium approach (and sometimes dropping down to requests).
Is this possible with mechanicalsoup?
My situation turned out to require Javascript for login. My original question about getting into SAML auth was not the true environment. So this question has not truly been answered.
Thanks to #Daniel Hemberger for helping me figure that out in the comments.
In this situation MechanicalSoup is not the correct tool (due to Javascript) and I ended up using selenium to get through authenication then using requests.
I'm trying to get data from fbo.gov, which is a government website where they post contracts that vendors can bid in. They have a document containing ways of accessing information on the site through SOAP requests, which is what I'm trying to do. Although all of the examples in that document are in PHP, I am trying to make my requests in Python, because I've never done anything with PHP before.
To make the SOAP requests in Python, I'm using zeep.
Right now, I can successfully authenticate myself through HTTP, but no matter what method I try to call, I always get the same error: This user has an inactive agency.
Here is the code I'm using to send the request
from requests import Session
from requests.auth import HTTPBasicAuth
import zeep
from zeep.transports import Transport
test = "https://fbo-test.symplicity.com"
prod = "https://fbo.gov"
session = Session()
session.auth = HTTPBasicAuth("sample_username", "sample_password")
client = zeep.Client(f"{test}/ws/fbo_api.php?wsdl", transport=Transport(session=session))
dictionary = {"notice_type": "PRESOL"}
print(client.service.getList(data=dictionary))
I realize this is a long shot, but what could be causing this error? I can't find anything even remotely related to the error anywhere on the internet.
Per the Federal Service Desk:
The FBO API is only available for government user accounts.
Some of the FBO data is available at: ftp://ftp.fbo.gov
Currently, FBO is in the process of moving to SAM, and will have a public API once the move is complete. The new API is under development, with the latest specification at: https://open.gsa.gov/api/get-opportunities-public-api/
FBO.GOV has been retired as of 11/12/2019 along with the ftp.fbo.gov bulk download, use the following instead,
https://open.gsa.gov/api/sam-entity-extracts-api/
I am trying to connect to a sharepoint site, however I get the AttributeError: 'NoneType' object has no attribute 'text'.
my code is as follows
import sharepy
s = sharepy.connect("https://example.sharepoint.com/",\
username='username', password='password')
Any idea why I get this error? I am allowed in the site, but not on the https://example.sharepoint.com/ server url, and also not allowed through python. I have tried several other ways in, using urllib.request and also requests. There my issue lies with having the HTTPError: Forbidden. The code run there is:
from sharepoint import SharePointSite, basic_auth_opener
server_url = 'https://example.sharepoint.com/'
opener = basic_auth_opener(server_url, "username",
"password")
site = SharePointSite(server_url, opener)
for sp_lists in site.lists:
print(sp_lists.id)
Seems like I have an issue with permission on the server or could it be something different?
sharepy shows this error if SharePoint returns an error message during the authentication. It is an error in the library because it doesn't know how to handle all error codes.
In your case the site url is wrong - "https://example.sharepoint.com" instead of "https://example.sharepoint.com/".
it might possible that you are missing header or other, please follow below link for more ideas on HTTP Error 403: Forbidden
https://mediatemple.net/community/products/dv/204644980/why-am-i-seeing-a-403-forbidden-error-message