I have one document on google drive and there are notes, comments, that I want to get. Can anyone say, is there a way to do it?
For example, lets start with this
import httplib2
import googleapiclient.discovery
from oauth2client.service_account import ServiceAccountCredentials
from google.oauth2 import service_account
from googleapiclient.http import MediaIoBaseDownload,MediaFileUpload
import os
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = r'C:\Users\Idensas\PycharmProjects\protest\Google\creds.json'
credentials = ServiceAccountCredentials('creds.json',['https://www.googleapis.com/auth/drive'])
httpAuth = credentials.authorize(httplib2.Http())
drive = googleapiclient.discovery.build('drive','v3')
a = drive.files().list(pageSize=10,fields='files(id, name, mimeType,parents)').execute()['files'][0]
print(a)
Here is the output, this is the file where are comments that I want to get.
{'id': '1PsV3D0CrCfTpjkbateXiLZIOsoDVV5ha_WV9FFZ2QEM', 'name': 'task', 'mimeType': 'application/vnd.google-apps.document'}
Comments can be fetch using Drive API Comments.list.
Try appending this to your code:
file_id = a['id']
try:
comments = drive.comments().list(fileId=file_id,fields='comments').execute()
for comment in comments.get('comments'):
print(comment['content'])
except errors.HttpError as error:
print('An error occurred: %s' % error)
Sample Doc:
Output:
Note: If the value of a in your code is {'id': '1PsV3D0CrCfTpjkbateXiLZIOsoDVV5ha_WV9FFZ2QEM', 'name': 'task', 'mimeType': 'application/vnd.google-apps.document'}, just append the whole code, else replace the file_id with the id of the document. Also, since you are using service account, you need to share the document to the service account email which can be found in the credentials json file. Another option is to use OAuth 2.0 Client ID which can be found in this Demo.
I'm trying to make requests to the Google API to create source repositories using a service account and his JSON key file.
Since there are no client libraries for this product, I am using the queries with Python using this documentation
https://cloud.google.com/source-repositories/docs/reference/rest
I already used a similar code to invoke my cloud-functions with success, but this time I'm block for these requests at the 401 error. I set up the GOOGLE_APPLICATION_CREDENTIALS with the JSON of my service account, give the service-account the permissions of Source Repository Administrator, but still return 401.
Here's my code
import urllib.request
import json
import urllib
import google.auth.transport.requests
import google.oauth2.id_token
body = { "name" : "projects/$my_project_name/repos/$name_repo"}
jsondata = json.dumps(body).encode("utf8")
req = urllib.request.Request('https://sourcerepo.googleapis.com/v1/projects/$my_project_name/repos')
req.add_header('Content-Type', 'application/json; charset=utf-8')
auth_req = google.auth.transport.requests.Request()
id_token = google.oauth2.id_token.fetch_id_token(auth_req, 'https://www.googleapis.com/auth/cloud-platform')
req.add_header("Authorization", f"Bearer {id_token}")
response = urllib.request.urlopen(req, jsondata)
print (response.read().decode())
I tried also using the with an API-KEY at the end of the url like this
req = urllib.request.Request('https://sourcerepo.googleapis.com/v1/projects/$my_project_name/repos?key=$my-api-key')
Thank you
I tried also using the with an API-KEY at the end of the url like this
API Keys are not supported.
Your code is using an OIDC Identity Token instead of an OAuth Acess Token.
from google.oauth2 import service_account
credentials = service_account.Credentials.from_service_account_file(
'/path/to/key.json',
scopes=['https://www.googleapis.com/auth/cloud-platform'])
request = google.auth.transport.requests.Request()
credentials.refresh(request)
// Use the following code to add the access token:
req.add_header("Authorization", f"Bearer {credentials.token}")
Maybe this is a dumb question but for web3.js there is the option to use another API service Ankr, instead of Infura. Ankr gives access to BSC network which has lower fees. I cannot seem to figure out how to connect to Ankr through python web3 as it requires authentication with a username and password. It returns false when I run the python code. I am not sure which keys I am suppose to use for web3.py, or possibly the syntax for the call is wrong, when I use the requests library everything works fine so it is not an issue with the address.
# Python Code Unsuccessful
Ankr_bsc_url = 'https............'
web3 = Web3(Web3.HTTPProvider(Ankr_bsc_url, request_kwargs={'headers': {'Username': user, 'Password': password}}))
print(web3.isConnected())
//Node.js Code web3.js Works
const web3Provider = new Web3.providers.WebsocketProvider(url, {
headers: { authorization: `Basic ${Buffer.from(`${user}:${password}`).toString('base64')}`}
})
You should save the headers on a Session object, and pass it as a parameter of HTTPProvider
from web3 import Web3
import requests
s = requests.Session()
s.headers.update({'authorization': 'Basic ZZZZ'})
# HTTPProvider:
w3 = Web3(Web3.HTTPProvider('https://apis.ankr.com/XXXX/YYYY/binance/full/main', session=s))
w3.isConnected()
In my case w3.isConnected return True
I found the method below worked well when connecting to the "Basic authentication" method which required a username and password.
Alternatively, using the "Token" method did not require a username and password and that also successfully gives you an Ankr API endpoint.
from web3 import Web3
import requests
import base64
ankr_eth_url = 'INSERT_ANKR_API_ENDPOINT'
s = requests.Session()
# Make sure to use the Project Username and not your log-in username
# myProjectUsername:password
upass = "myProjectUsername:12345678".encode("ascii")
b64 = base64.b64encode(upass).decode("ascii")
s.headers.update({'Authorization': 'Basic ' + b64})
w3 = Web3(Web3.HTTPProvider(ankr_eth_url, session=s))
print(w3.isConnected())
How do we post a GraphQL request through AWS AppSync using boto?
Ultimately I'm trying to mimic a mobile app accessing our stackless/cloudformation stack on AWS, but with python. Not javascript or amplify.
The primary pain point is authentication; I've tried a dozen different ways already. This the current one, which generates a "401" response with "UnauthorizedException" and "Permission denied", which is actually pretty good considering some of the other messages I've had. I'm now using the 'aws_requests_auth' library to do the signing part. I assume it authenticates me using the stored /.aws/credentials from my local environment, or does it?
I'm a little confused as to where and how cognito identities and pools will come into it. eg: say I wanted to mimic the sign-up sequence?
Anyways the code looks pretty straightforward; I just don't grok the authentication.
from aws_requests_auth.boto_utils import BotoAWSRequestsAuth
APPSYNC_API_KEY = 'inAppsyncSettings'
APPSYNC_API_ENDPOINT_URL = 'https://aaaaaaaaaaaavzbke.appsync-api.ap-southeast-2.amazonaws.com/graphql'
headers = {
'Content-Type': "application/graphql",
'x-api-key': APPSYNC_API_KEY,
'cache-control': "no-cache",
}
query = """{
GetUserSettingsByEmail(email: "john#washere"){
items {name, identity_id, invite_code}
}
}"""
def test_stuff():
# Use the library to generate auth headers.
auth = BotoAWSRequestsAuth(
aws_host='aaaaaaaaaaaavzbke.appsync-api.ap-southeast-2.amazonaws.com',
aws_region='ap-southeast-2',
aws_service='appsync')
# Create an http graphql request.
response = requests.post(
APPSYNC_API_ENDPOINT_URL,
json={'query': query},
auth=auth,
headers=headers)
print(response)
# this didn't work:
# response = requests.post(APPSYNC_API_ENDPOINT_URL, data=json.dumps({'query': query}), auth=auth, headers=headers)
Yields
{
"errors" : [ {
"errorType" : "UnauthorizedException",
"message" : "Permission denied"
} ]
}
It's quite simple--once you know. There are some things I didn't appreciate:
I've assumed IAM authentication (OpenID appended way below)
There are a number of ways for appsync to handle authentication. We're using IAM so that's what I need to deal with, yours might be different.
Boto doesn't come into it.
We want to issue a request like any regular punter, they don't use boto, and neither do we. Trawling the AWS boto docs was a waste of time.
Use the AWS4Auth library
We are going to send a regular http request to aws, so whilst we can use python requests they need to be authenticated--by attaching headers.
And, of course, AWS auth headers are special and different from all others.
You can try to work out how to do it
yourself, or you can go looking for someone else who has already done it: Aws_requests_auth, the one I started with, probably works just fine, but I have ended up with AWS4Auth. There are many others of dubious value; none endorsed or provided by Amazon (that I could find).
Specify appsync as the "service"
What service are we calling? I didn't find any examples of anyone doing this anywhere. All the examples are trivial S3 or EC2 or even EB which left uncertainty. Should we be talking to api-gateway service? Whatsmore, you feed this detail into the AWS4Auth routine, or authentication data. Obviously, in hindsight, the request is hitting Appsync, so it will be authenticated by Appsync, so specify "appsync" as the service when putting together the auth headers.
It comes together as:
import requests
from requests_aws4auth import AWS4Auth
# Use AWS4Auth to sign a requests session
session = requests.Session()
session.auth = AWS4Auth(
# An AWS 'ACCESS KEY' associated with an IAM user.
'AKxxxxxxxxxxxxxxx2A',
# The 'secret' that goes with the above access key.
'kwWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxgEm',
# The region you want to access.
'ap-southeast-2',
# The service you want to access.
'appsync'
)
# As found in AWS Appsync under Settings for your endpoint.
APPSYNC_API_ENDPOINT_URL = 'https://nqxxxxxxxxxxxxxxxxxxxke'
'.appsync-api.ap-southeast-2.amazonaws.com/graphql'
# Use JSON format string for the query. It does not need reformatting.
query = """
query foo {
GetUserSettings (
identity_id: "ap-southeast-2:8xxxxxxb-7xx4-4xx4-8xx0-exxxxxxx2"
){
user_name, email, whatever
}}"""
# Now we can simply post the request...
response = session.request(
url=APPSYNC_API_ENDPOINT_URL,
method='POST',
json={'query': query}
)
print(response.text)
Which yields
# Your answer comes as a JSON formatted string in the text attribute, under data.
{"data":{"GetUserSettings":{"user_name":"0xxxxxxx3-9102-42f0-9874-1xxxxx7dxxx5"}}}
Getting credentials
To get rid of the hardcoded key/secret you can consume the local AWS ~/.aws/config and ~/.aws/credentials, and it is done this way...
# Use AWS4Auth to sign a requests session
session = requests.Session()
credentials = boto3.session.Session().get_credentials()
session.auth = AWS4Auth(
credentials.access_key,
credentials.secret_key,
boto3.session.Session().region_name,
'appsync',
session_token=credentials.token
)
...<as above>
This does seem to respect the environment variable AWS_PROFILE for assuming different roles.
Note that STS.get_session_token is not the way to do it, as it may try to assume a role from a role, depending where it keyword matched the AWS_PROFILE value. Labels in the credentials file will work because the keys are right there, but names found in the config file do not work, as that assumes a role already.
OpenID
In this scenario, all the complexity is transferred to the conversation with the openid connect provider. The hard stuff is all the auth hoops you jump through to get an access token, and thence using the refresh token to keep it alive. That is where all the real work lies.
Once you finally have an access token, assuming you have configured the "OpenID Connect" Authorization Mode in appsync, then you can, very simply, drop the access token into the header:
response = requests.post(
url="https://nc3xxxxxxxxxx123456zwjka.appsync-api.ap-southeast-2.amazonaws.com/graphql",
headers={"Authorization": ACCESS_TOKEN},
json={'query': "query foo{GetStuff{cat, dog, tree}}"}
)
You can set up an API key on the AppSync end and use the code below. This works for my case.
import requests
# establish a session with requests session
session = requests.Session()
# As found in AWS Appsync under Settings for your endpoint.
APPSYNC_API_ENDPOINT_URL = 'https://vxxxxxxxxxxxxxxxxxxy.appsync-api.ap-southeast-2.amazonaws.com/graphql'
# setup the query string (optional)
query = """query listItemsQuery {listItemsQuery {items {correlation_id, id, etc}}}"""
# Now we can simply post the request...
response = session.request(
url=APPSYNC_API_ENDPOINT_URL,
method='POST',
headers={'x-api-key': '<APIKEYFOUNDINAPPSYNCSETTINGS>'},
json={'query': query}
)
print(response.json()['data'])
Building off Joseph Warda's answer you can use the class below to send AppSync commands.
# fileName: AppSyncLibrary
import requests
class AppSync():
def __init__(self,data):
endpoint = data["endpoint"]
self.APPSYNC_API_ENDPOINT_URL = endpoint
self.api_key = data["api_key"]
self.session = requests.Session()
def graphql_operation(self,query,input_params):
response = self.session.request(
url=self.APPSYNC_API_ENDPOINT_URL,
method='POST',
headers={'x-api-key': self.api_key},
json={'query': query,'variables':{"input":input_params}}
)
return response.json()
For example in another file within the same directory:
from AppSyncLibrary import AppSync
APPSYNC_API_ENDPOINT_URL = {YOUR_APPSYNC_API_ENDPOINT}
APPSYNC_API_KEY = {YOUR_API_KEY}
init_params = {"endpoint":APPSYNC_API_ENDPOINT_URL,"api_key":APPSYNC_API_KEY}
app_sync = AppSync(init_params)
mutation = """mutation CreatePost($input: CreatePostInput!) {
createPost(input: $input) {
id
content
}
}
"""
input_params = {
"content":"My first post"
}
response = app_sync.graphql_operation(mutation,input_params)
print(response)
Note: This requires you to activate API access for your AppSync API. Check this AWS post for more details.
graphql-python/gql supports AWS AppSync since version 3.0.0rc0.
It supports queries, mutation and even subscriptions on the realtime endpoint.
The documentation is available here
Here is an example of a mutation using the API Key authentication:
import asyncio
import os
import sys
from urllib.parse import urlparse
from gql import Client, gql
from gql.transport.aiohttp import AIOHTTPTransport
from gql.transport.appsync_auth import AppSyncApiKeyAuthentication
# Uncomment the following lines to enable debug output
# import logging
# logging.basicConfig(level=logging.DEBUG)
async def main():
# Should look like:
# https://XXXXXXXXXXXXXXXXXXXXXXXXXX.appsync-api.REGION.amazonaws.com/graphql
url = os.environ.get("AWS_GRAPHQL_API_ENDPOINT")
api_key = os.environ.get("AWS_GRAPHQL_API_KEY")
if url is None or api_key is None:
print("Missing environment variables")
sys.exit()
# Extract host from url
host = str(urlparse(url).netloc)
auth = AppSyncApiKeyAuthentication(host=host, api_key=api_key)
transport = AIOHTTPTransport(url=url, auth=auth)
async with Client(
transport=transport, fetch_schema_from_transport=False,
) as session:
query = gql(
"""
mutation createMessage($message: String!) {
createMessage(input: {message: $message}) {
id
message
createdAt
}
}"""
)
variable_values = {"message": "Hello world!"}
result = await session.execute(query, variable_values=variable_values)
print(result)
asyncio.run(main())
I am unable to add a comment due to low rep, but I just want to add that I tried the accepted answer and it didn't work. I was getting an error saying my session_token is invalid. Probably because I was using AWS Lambda.
I got it to work pretty much exactly, but by adding to the session token parameter of the aws4auth object. Here's the full piece:
import requests
import os
from requests_aws4auth import AWS4Auth
def AppsyncHandler(event, context):
# These are env vars that are always present in an AWS Lambda function
# If not using AWS Lambda, you'll need to add them manually to your env.
access_id = os.environ.get("AWS_ACCESS_KEY_ID")
secret_key = os.environ.get("AWS_SECRET_ACCESS_KEY")
session_token = os.environ.get("AWS_SESSION_TOKEN")
region = os.environ.get("AWS_REGION")
# Your AppSync Endpoint
api_endpoint = os.environ.get("AppsyncConnectionString")
resource = "appsync"
session = requests.Session()
session.auth = AWS4Auth(access_id,
secret_key,
region,
resource,
session_token=session_token)
The rest is the same.
Hope this Helps Everyone
import requests
import json
import os
from dotenv import load_dotenv
load_dotenv(".env")
class AppSync(object):
def __init__(self,data):
endpoint = data["endpoint"]
self.APPSYNC_API_ENDPOINT_URL = endpoint
self.api_key = data["api_key"]
self.session = requests.Session()
def graphql_operation(self,query,input_params):
response = self.session.request(
url=self.APPSYNC_API_ENDPOINT_URL,
method='POST',
headers={'x-api-key': self.api_key},
json={'query': query,'variables':{"input":input_params}}
)
return response.json()
def main():
APPSYNC_API_ENDPOINT_URL = os.getenv("APPSYNC_API_ENDPOINT_URL")
APPSYNC_API_KEY = os.getenv("APPSYNC_API_KEY")
init_params = {"endpoint":APPSYNC_API_ENDPOINT_URL,"api_key":APPSYNC_API_KEY}
app_sync = AppSync(init_params)
mutation = """
query MyQuery {
getAccountId(id: "5ca4bbc7a2dd94ee58162393") {
_id
account_id
limit
products
}
}
"""
input_params = {}
response = app_sync.graphql_operation(mutation,input_params)
print(json.dumps(response , indent=3))
main()
I think there is an issue with Gmail API and Python3.
The original code in the documentation is in Python2, but for several reasons, including that my application is already working with Python3, I was passing the code to python3.
So... after sorting several issues, including a 400 request error, (which was apparently that the auth I provided to google wasn't corretly done) I'm facing (I hope) the final issue that apparently I'm trying to read a file but
Even just doing token.read() generates the same issue.
The token.pickle file is autogenerated once you've authorized google to access your email account and that the app can send emails automatically.
I know the credentials.json file is correct, as that's the point to tell google who you are and it always is reading my credentials correctly, redirecting me to give authorization to my app.
Here's the app to send emails, I think it's pretty straightforward, I followed the documentation and looked at other issues like this one and finally got this far:
from __future__ import print_function
import pickle
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
from googleapiclient.errors import HttpError
# from httplib2 import Http
# from oauth2client import client, tools, file
import base64
from email.mime.audio import MIMEAudio
from email.mime.base import MIMEBase
from email.mime.image import MIMEImage
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
import mimetypes
from apiclient import errors
SCOPES = 'https://mail.google.com/'
def SendMessage(service, user_id, message):
"""Send an email message.
Args:
service: Authorized Gmail API service instance.
user_id: User's email address. The special value "me"
can be used to indicate the authenticated user.
message: Message to be sent.
Returns:
Sent Message.
"""
try:
message = (service.users().messages().send(userId=user_id, body=message)
.execute())
print(f'Message Id: {message["id"]}')
return message
except errors.HttpError as error:
print(f'An error occurred: {error}')
def CreateMessage(sender, to, subject, message_text):
"""Create a message for an email.
Args:
sender: Email address of the sender.
to: Email address of the receiver.
subject: The subject of the email message.
message_text: The text of the email message.
Returns:
An object containing a base64url encoded email object.
"""
message = MIMEText(message_text)
message['to'] = to
message['from'] = sender
message['subject'] = subject
message_bytes = message.as_string().encode('utf-8')
# return { 'raw': base64.urlsafe_b64encode(message.as_string()) }
# return { 'raw': base64.urlsafe_b64encode(message_bytes) }
raw = base64.urlsafe_b64encode(message_bytes)
return raw.decode()
def main():
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.pickle'):
with open('token.pickle', 'r') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('gmail', 'v1', credentials=creds)
# Create
message = CreateMessage('my-email#gmail.com', 'destination#gmail.com', 'Testing Gmail API', 'Hi GMAIL API')
# Send
SendMessage(service, "me", message)
if __name__ == '__main__':
main()
I truly don't know what to do here, if someone has already solved this issue and how.
THANKS!
In your script, when token.pickle is not created, Gmail API can be used by creating token.pickle at 1st authorization. But an error of 'raw' RFC822 payload message string or uploading message via /upload/* URL required occurs at message = (service.users().messages().send(userId=user_id, body=message).execute()). So in this case, please modify as follows.
From:
raw = base64.urlsafe_b64encode(message_bytes)
return raw.decode()
To:
raw = base64.urlsafe_b64encode(message_bytes).decode('utf-8')
return {'raw': raw}
After above modification, when your script is run as the 2nd run, when token.pickle, which has already been created, is read, an error occurs. I think that this error is in your title. In this case, please modify as follows.
From:
with open('token.pickle', 'r') as token:
To:
with open('token.pickle', 'rb') as token:
By this, I think that the script works.
By the way, if an error occurs when the code is retrieved, also please modify as follows.
From:
creds = flow.run_local_server(port=0)
To:
creds = flow.run_local_server()
Note:
About the authorization, you can see the sample script at the Quickstart of the official document.
If this was not the direction of your issue, I apologize.