Outlook REST API provides an endpoint /me/messages/:message_id:/$value for requesting "raw" emails bodies (old good MIME-encoded email bodies). However, when I try to request bodies of several emails, using the Batch API (https://outlook.office.com/api/v2.0/$batch), I get only a single body in the response (the body of the first email in request I assume). The request body example:
--07edf27e-0552-45db-9a75-add15681b61b
Content-Type: application/http
MIME-Version: 1.0
Content-Transfer-Encoding: binary
GET /api/v2.0/me/messages/AAMkADllYjZhMmZhLWRiMWQtNGY5OS1hM2NiLTA3NDZmODA4NzZiMQBGAAAAAABdR0VqIHc_T6bmj6YLprUhBwDPSIJ3m0xMRZntj-YB8UfpAAAAAAEMAADPSIJ3m0xMRZntj-YB8UfpAAOHtNrzAAA=/$value HTTP/1.1
--07edf27e-0552-45db-9a75-add15681b61b
Content-Type: application/http
MIME-Version: 1.0
Content-Transfer-Encoding: binary
GET /api/v2.0/me/messages/AAMkADllYjZhMmZhLWRiMWQtNGY5OS1hM2NiLTA3NDZmODA4NzZiMQBGAAAAAABdR0VqIHc_T6bmj6YLprUhBwDPSIJ3m0xMRZntj-YB8UfpAAAAAAEMAADPSIJ3m0xMRZntj-YB8UfpAAOHtNryAAA=/$value HTTP/1.1
--07edf27e-0552-45db-9a75-add15681b61b
Content-Type: application/http
MIME-Version: 1.0
Content-Transfer-Encoding: binary
GET /api/v2.0/me/messages/AAMkADllYjZhMmZhLWRiMWQtNGY5OS1hM2NiLTA3NDZmODA4NzZiMQBGAAAAAABdR0VqIHc_T6bmj6YLprUhBwDPSIJ3m0xMRZntj-YB8UfpAAAAAAEMAADPSIJ3m0xMRZntj-YB8UfpAAOE7qpYAAA=/$value HTTP/1.1
--07edf27e-0552-45db-9a75-add15681b61b--
And response is of 'plain\text' Content-Type. Request/Response details:
{'Content-Length': '998', 'Accept-Encoding': 'gzip, deflate', 'Accept': u'application/json', 'User-Agent': u'Nimble', 'Connection': 'keep-alive', 'Content-Type': 'multipart/mixed; boundary="e91fd766-955d-4e87-9204-b28b004d3e5e"', 'Authorization': u'Bearer eyJ0eXAiOiJK..'}
ipdb> resp.request.url
'https://outlook.office.com/api/v2.0/me/$batch'
ipdb> resp.headers
{'X-CalculatedBETarget': 'VI1PR09MB2479.eurprd09.prod.outlook.com', 'Rate-Limit-Reset': '2020-02-06T18:49:26.260Z', 'X-Powered-By': 'ASP.NET', 'Transfer-Encoding': 'chunked', 'X-RUM-Validated': '1', 'X-BackEndHttpStatus': '200, 200', 'X-Proxy-RoutingCorrectness': '1', 'X-BeSku': 'WCS5', 'Date': 'Thu, 06 Feb 2020 18:39:10 GMT', 'Rate-Limit-Limit': '10000', 'Rate-Limit-Remaining': '9999', 'X-AspNet-Version': '4.0.30319', 'Server': 'Microsoft-IIS/10.0', 'X-BEServer': 'VI1PR09MB2479', 'X-DiagInfo': 'VI1PR09MB2479', 'X-FEServer': 'VI1PR0502CA0025, MWHPR10CA0071', 'X-MailboxGuid': '9eb6a2fa-db1d-4f99-a3cb-0746f80876b1', 'X-FEProxyInfo': 'VI1PR0502CA0025.EURPRD05.PROD.OUTLOOK.COM', 'x-ms-appId': 'b27e9f81-dbc4-4d4a-923a-3eacaae45241', 'request-id': 'ff7b1bce-5d41-450f-800a-d2de6f9ae2f0', 'X-Proxy-BackendServerStatus': '200', 'X-CalculatedFETarget': 'VI1PR0502CU001.internal.outlook.com', 'Cache-Control': 'private', **'Content-Type': 'text/plain'**}
If performing the same request but without requesting "raw" bodies, the response becomes of expected content-type: 'Content-Type': 'multipart/mixed; boundary=batchresponse_e4ca25bf-f9e7-4d69-bc09-139bf2776db0'. The request body:
--006b59ee-e662-46d8-be97-a21a24cc9b08
Content-Type: application/http
MIME-Version: 1.0
Content-Transfer-Encoding: binary
GET /api/v2.0/me/messages/AAMkADllYjZhMmZhLWRiMWQtNGY5OS1hM2NiLTA3NDZmODA4NzZiMQBGAAAAAABdR0VqIHc_T6bmj6YLprUhBwDPSIJ3m0xMRZntj-YB8UfpAAAAAAEMAADPSIJ3m0xMRZntj-YB8UfpAAOHtNrzAAA= HTTP/1.1
--006b59ee-e662-46d8-be97-a21a24cc9b08
Content-Type: application/http
MIME-Version: 1.0
Content-Transfer-Encoding: binary
GET /api/v2.0/me/messages/AAMkADllYjZhMmZhLWRiMWQtNGY5OS1hM2NiLTA3NDZmODA4NzZiMQBGAAAAAABdR0VqIHc_T6bmj6YLprUhBwDPSIJ3m0xMRZntj-YB8UfpAAAAAAEMAADPSIJ3m0xMRZntj-YB8UfpAAOHtNryAAA= HTTP/1.1
--006b59ee-e662-46d8-be97-a21a24cc9b08
Content-Type: application/http
MIME-Version: 1.0
Content-Transfer-Encoding: binary
GET /api/v2.0/me/messages/AAMkADllYjZhMmZhLWRiMWQtNGY5OS1hM2NiLTA3NDZmODA4NzZiMQBGAAAAAABdR0VqIHc_T6bmj6YLprUhBwDPSIJ3m0xMRZntj-YB8UfpAAAAAAEMAADPSIJ3m0xMRZntj-YB8UfpAAOE7qpYAAA= HTTP/1.1
--006b59ee-e662-46d8-be97-a21a24cc9b08--
ipdb> resp.headers
{'X-CalculatedBETarget': 'VI1PR09MB2479.eurprd09.prod.outlook.com', 'Rate-Limit-Reset': '2020-02-06T18:49:26.260Z', 'X-Powered-By': 'ASP.NET', 'Transfer-Encoding': 'chunked', 'X-RUM-Validated': '1', 'OData-Version': '4.0', 'X-BackEndHttpStatus': '200, 200', 'X-Proxy-RoutingCorrectness': '1', 'X-BeSku': 'WCS5', 'Date': 'Thu, 06 Feb 2020 18:45:54 GMT', 'Rate-Limit-Limit': '10000', 'Rate-Limit-Remaining': '9984', 'X-AspNet-Version': '4.0.30319', 'Server': 'Microsoft-IIS/10.0', 'X-BEServer': 'VI1PR09MB2479', 'X-DiagInfo': 'VI1PR09MB2479', 'X-FEServer': 'VI1PR03CA0059, MWHPR04CA0069', 'X-MailboxGuid': '9eb6a2fa-db1d-4f99-a3cb-0746f80876b1', 'X-FEProxyInfo': 'VI1PR03CA0059.EURPRD03.PROD.OUTLOOK.COM', 'x-ms-appId': 'b27e9f81-dbc4-4d4a-923a-3eacaae45241', 'request-id': '7f5dcc0f-4109-48a1-9948-39a5ad7471e6', 'X-Proxy-BackendServerStatus': '200', 'X-CalculatedFETarget': 'VI1PR03CU002.internal.outlook.com', 'Cache-Control': 'private', 'Content-Type': 'multipart/mixed; boundary=batchresponse_e4ca25bf-f9e7-4d69-bc09-139bf2776db0'}
ipdb> resp.request.url
'https://outlook.office.com/api/v2.0/me/$batch'
Is it a bug in Outlook REST API? The same request, but to the Graph API works correctly (unfortunately we can't use the Graph API in production).
Related
Apparently there's something I don't know about HEAD requests.
Here's the URL: 'https://theweekinchess.com/assets/files/pgn/eurbli22.pgn', which I'll refer to as <URL> below.
If I curl this, I see a last-modified entry in the headers:
curl --head <URL>
HTTP/2 200
last-modified: Sun, 18 Dec 2022 18:07:16 GMT
accept-ranges: bytes
content-length: 1888745
host-header: c2hhcmVkLmJsdWVob3N0LmNvbQ==
content-type: application/x-chess-pgn
date: Wed, 11 Jan 2023 23:09:14 GMT
server: Apache
But if I make a HEAD request in Node using https, That information is missing:
https.request(<URL>, { method: 'HEAD' }, res => {
console.log([<URL>, res.headers])}).end()
This returns:
[
<URL>
{
date: 'Wed, 11 Jan 2023 23:16:15 GMT',
server: 'Apache',
p3p: 'CP="NOI NID ADMa OUR IND UNI COM NAV"',
'cache-control': 'private, must-revalidate',
'set-cookie': [
'evof3sqa=4b412b5913b38669fc928a0cca9870e4; path=/; secure; HttpOnly'
],
upgrade: 'h2,h2c',
connection: 'Upgrade, Keep-Alive',
'host-header': 'c2hhcmVkLmJsdWVob3N0LmNvbQ==',
'keep-alive': 'timeout=5, max=75',
'content-type': 'text/html; charset=UTF-8'
}
]
I tried axios instead of https:
const response = await axios.head('https://theweekinchess.com/assets/files/pgn/eurbli22.pgn');
console.log({response: response.headers})
And that works (incl. the proper MIME type):
date: 'Thu, 12 Jan 2023 20:00:48 GMT',
server: 'Apache',
upgrade: 'h2,h2c',
connection: 'Upgrade, Keep-Alive',
'last-modified': 'Sun, 18 Dec 2022 18:07:16 GMT',
'accept-ranges': 'bytes',
'content-length': '1888745',
'host-header': 'c2hhcmVkLmJsdWVob3N0LmNvbQ==',
'keep-alive': 'timeout=5, max=75',
'content-type': 'application/x-chess-pgn'
I also tried waiting for re2.on('end', console.log(res.headers)), but same output as before.
I'm going to close this issue and post it instead as a 'bug' on Node's site. I'm sure there's something that needs to be changed in how I'm executing the HEAD request.
Im trying to POST request using Python to retreive a specific File. Since the URL is behind a server with authorized access theres no use posting it here
However the form data contains a field called base64 and lengthy which I cant figure out if its a form data value or base64 encoding of post request
Here are browser parameters
General:
Request URL: http://exampleapi.com/api/Document/Export
Request Method: POST
Status Code: 200 OK
Remote Address: XX.XXX.XXX.XX:XX
Referrer Policy: no-referrer-when-downgrade
Response Headers:
Access-Control-Allow-Origin: http://example.com
Cache-Control: no-cache
Content-Disposition: attachment; filename=location-downloads.xlsx
Content-Length: 7148
Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Date: Tue, 23 Jul 2019 21:00:18 GMT
Expires: -1
Pragma: no-cache
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Request Headers :
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 10162
Content-Type: application/x-www-form-urlencoded
Cookie: abcConnection=!UA7tkC3iZCmVNGRUyRpDWARVBWk/lY6SZvgxLlaygsQKk+vuwA1NxvhwE9ph4i+3NZlKeepIfuHhUvyQjl68fhhrT9ueqMx/3mBKUDcT
DNT: 1
Host: exampleapi.com
Origin: http://example.com
Referer: http://example.com/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36
Form Data:
fileName: location-downloads.xlsx
contentType: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
base64: UEsDBAoAAAAAAAh4904AAAAAAAAAAAAAAAAJAAAAZG9jUHJvcHMvUEsDBAoAAAAIAAh490(shortened for simplicity)
Here is what I tried
url='http://example.com'
urllib3.disable_warnings()
headers = {
"Content-Type": "application/x-www-form-urlencoded",
"User-Agent": "Mozilla/5.0",
}
with requests.session() as s:
r=s.get(url,headers={"User-Agent":"Mozilla/5.0"},verify=False)
data=r.content
soup=BeautifulSoup(data,'html.parser')
form_data = {
"fileName":"location-downloads.xlsx",
"contentType":"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
}
r2=s.post('http://exampleapi.com/api/Document/Export',data=json.dumps(form_data,ensure_ascii=True).encode('utf-8'),headers=headers,verify=False)
print(r2.status_code)
Any idea how i should proceed. My status code also shows 500 for the post here
I'm trying to read emails from the Microsoft Exchange server using EWS and exchangelib in Python for an email classification problem. But I am unable to connect to the exchange server.
I've tried specifying the version, auth_type, using a certificate (which gives a ssl verify error), using the smtp address in place of the username and it still doesn't connect.
Here is my code:
from exchangelib import Credentials, Account, EWSDateTime, EWSTimeZone, Configuration, DELEGATE, IMPERSONATION, NTLM, ServiceAccount, Version, Build
USER_NAME = 'domain\\user12345'
ACCOUNT_EMAIL = john.doe#ext.companyname.com'
ACCOUNT_PASSWORD = 'John#1234'
ACCOUNT_SERVER = 'oa.company.com'
creds = Credentials(USER_NAME, ACCOUNT_PASSWORD)
config = Configuration(server=ACCOUNT_SERVER, credentials=creds)
account = Account(primary_smtp_address=ACCOUNT_EMAIL, config=config, autodiscover=False, access_type=DELEGATE)
print('connecting ms exchange server account...')
print(type(account))
print(dir(account))
account.root.refresh()
Here is the error I am getting:
TransportError: Unknown failure
Retry: 0
Waited: 10
Timeout: 120
Session: 26271
Thread: 15248
Auth type: <requests_ntlm.requests_ntlm.HttpNtlmAuth object at 0x00000259AA1BD588>
URL: https://oa.company.com/EWS/Exchange.asmx
HTTP adapter: <requests.adapters.HTTPAdapter object at 0x00000259AA0DB7B8>
Allow redirects: False
Streaming: False
Response time: 0.28100000000085856
Status code: 403
Request headers: {'User-Agent': 'python-requests/2.21.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'Keep-Alive', 'Content-Type': 'text/xml; charset=utf-8', 'Content-Length': '469', 'Authorization': 'NTLM TlRMTVNTUAADAAAAGAAYAG0AAAAOAQ4BhQAAAAwADABYAAAACQAJAGQAAAAAAAAAbQAAABAAEACTAQAANoKJ4gYBsR0AAAAP7Pyb+wBnMdrlhr4FKVqPbklDSUNJQkFOS0xURElQUlUzODE5MAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAJTLmMBLPHowOJZ46XDs+4ABAQAAAAAAAAZdRefQLNUB+Fc6Z26oxvgAAAAAAgAYAEkAQwBJAEMASQBCAEEATgBLAEwAVABEAAEAEgBIAFkARABFAFgAQwBIADAAOAAEACAAaQBjAGkAYwBpAGIAYQBuAGsAbAB0AGQALgBjAG8AbQADADQASABZAEQARQBYAEMASAAwADgALgBpAGMAaQBjAGkAYgBhAG4AawBsAHQAZAAuAGMAbwBtAAUAIABpAGMAaQBjAGkAYgBhAG4AawBsAHQAZAAuAGMAbwBtAAcACAAGXUXn0CzVAQYABAACAAAACgAQAD9EWlwiiUs304wucsxnkyQAAAAAAAAAALfelDwG05hYOMUqY/e60PY=', 'Cookie': 'ClientId=SINZWMOJKWSKDGEKASFG; expires=Fri, 26-Jun-2020 10:13:02 GMT; path=/; HttpOnly'}
Response headers: {'Cache-Control': 'private', 'Server': 'Microsoft-IIS/8.5', 'request-id': 'ae4dee8d-34e0-471c-8252-b8c1056c8ea0', 'X-CalculatedBETarget': 'pqrexch05.domain.com', 'X-DiagInfo': 'PQREXCH05', 'X-BEServer': 'PQREXCH05', 'X-AspNet-Version': '4.0.30319', 'Set-Cookie': 'exchangecookie=681afc8a0905459182363cce9a98d021; expires=Sat, 27-Jun-2020 10:13:02 GMT; path=/; HttpOnly, X-BackEndCookie=S-1-5-21-1343024091-725345543-504838010-1766210=u56Lnp2ejJqBy87Iysqem5nSy8mbnNLLyZ7H0sfIysbSy5vMz8qdzcvPnpzHgYHNz87G0s/I0s3Iq87Pxc7Mxc/N; expires=Sat, 27-Jul-2019 10:13:02 GMT; path=/EWS; secure; HttpOnly', 'Persistent-Auth': 'true', 'X-Powered-By': 'ASP.NET', 'X-FEServer': 'PQREXCH05', 'Date': 'Thu, 27 Jun 2019 10:13:01 GMT', 'Content-Length': '0'}
Request data: b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\n<s:Envelope xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"><s:Header><t:RequestServerVersion Version="Exchange2013_SP1"/></s:Header><s:Body><m:ResolveNames ReturnFullContactData="false"><m:UnresolvedEntry>ICICIBANKLTD\\IPRU38190</m:UnresolvedEntry></m:ResolveNames></s:Body></s:Envelope>'
Response data: b''
You might need to configure access policy for EWS using PowerShell.
For example (to allow all apps to use REST and EWS):
Set-OrganizationConfig -EwsApplicationAccessPolicy EnforceBlockList -EwsBlockList $null
Taken from Microsoft docs on Set-OrganizationConfig.
Please search for EwsApplicationAccessPolicy in the above link for more granular access control examples.
I use python requests to capture a website's http headers. For example, this is a response header:
{'Connection': 'keep-alive',
'Access-Control-Allow-Origin': '*', 'cache-control': 'max-age=600',
'Content-Type': 'text/html; charset=utf-8', 'Expires': 'Fri, 19 Apr
2019 03:16:28 GMT', 'Via': '1.1 varnish, 1.1 varnish', 'X-ESI': 'on',
'Verso': 'false', 'Accept-Ranges': 'none', 'Date': 'Fri, 19 Apr 2019
03:11:12 GMT', 'Age': '283', 'Set-Cookie':
'CN_xid=08f66bff-4001-4173-b4e2-71ac31bb58d7; Expires=Wed, 16 Oct 2019
03:11:12 GMT; path=/;, xid1=1; Expires=Fri, 19 Apr 2019 03:11:27 GMT;
path=/;, verso_bucket=281; Expires=Sat, 18 Apr 2020 03:11:12 GMT;
path=/;', 'X-Served-By': 'cache-iad2133-IAD, cache-gru17122-GRU',
'X-Cache': 'HIT, MISS', 'X-Cache-Hits': '1, 0', 'X-Timer':
'S1555643472.999490,VS0,VE302', 'Content-Security-Policy':
"default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src
https: data: blob:; connect-src https: data: blob:; font-src https:
data:; img-src https: data: blob:; media-src https: data: blob:;
object-src https:; script-src https: data: blob: 'unsafe-inline'
'unsafe-eval'; style-src https: 'unsafe-inline';
block-all-mixed-content; upgrade-insecure-requests; report-uri
https://l.com/csp/gq",
'X-Fastly-Device-Detect': 'desktop', 'Strict-Transport-Security':
'max-age=7776000; preload', 'Vary': 'Accept-Encoding, Verso,
Accept-Encoding', 'content-encoding': 'gzip', 'transfer-encoding':
'chunked'}
I noted that from several examples I tested, the headers I receive from requests are formatted as 'key':'value' (plz note the single colons surrounding the key and the value). However, when I check the headers from the Firefox-> Web developer -> Inspector, and choose to view the header in raw format, I do not see commas:
HTTP/2.0 200 OK date: Thu, 09 May 2019 18:49:07 GMT expires: -1
cache-control: private, max-age=0 content-type: text/html;
charset=UTF-8 strict-transport-security: max-age=31536000
content-encoding: br server: gws content-length: 55844
x-xss-protection: 0 x-frame-options: SAMEORIGIN set-cookie:
1P_JAR=2019-05-09-18; expires=Sat, 08-Jun-2019 18:49:07 GMT; path=/;
domain=.google.com alt-svc: quic=":443"; ma=2592000; v="46,44,43,39"
X-Firefox-Spdy: h2
I need to know: Does python's requests module always adds single colons? This important from me as I need to include/exclude them in my regex that is used to analyze the headers.
The issue I think you are running into is the request coming back as a dict instead of a value as firefox inspector is giving you. When you do this you could be getting mixed results if one of the value pairs has a numeric or boolean value so when doing your regex you may want to use a Try/Except if you can remove the exterior apostrophes or just use the value given.
It's not the requests module that's adding the colons. Request represents headers as a dict, but you seem to be treating them as a string. When Python converts dicts to strings, they get the colons, the commas, the quotation marks.
The right fix for your program is probably to treat the dictionary as a dictionary, not convert it into a string. But if you really want the headers in string form, you should consider using different tool, such as curl.
according to https://developers.google.com/admin-sdk/email-audit/#creating_a_mailbox_for_export I am trying to request the email audit export of an user in G Suite this way:
def requestAuditExport(account):
credentials = getCredentials()
http = credentials.authorize(httplib2.Http())
url = 'https://apps-apis.google.com/a/feeds/compliance/audit/mail/export/helpling.com/'+account
status, response = http.request(url, 'POST', headers={'Content-Type': 'application/atom+xml'})
print(status)
print(response)
And I get the following result:
{'content-length': '22', 'expires': 'Tue, 13 Dec 2016 14:19:37 GMT', 'date': 'Tue, 13 Dec 2016 14:19:37 GMT', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'x-xss-protection': '1; mode=block', 'content-type': 'text/html; charset=UTF-8', 'x-content-type-options': 'nosniff', '-content-encoding': 'gzip', 'server': 'GSE', 'status': '400', 'cache-control': 'private, max-age=0', 'alt-svc': 'quic=":443"; ma=2592000; v="35,34"'}
b'Premature end of file.'
I cannot see where the problem is, can someone please give me a hint?
Thanks in advance!
Kay
Fix it by going intp the Admin Console, Manage API client access page under Security and add the Client ID, scope needed for the Directory API. For more information, check this document.
Okay, found out what was wrong and fixed it myself. Finally it looks like this:
http = getCredentials().authorize(httplib2.Http())
url = 'https://apps-apis.google.com/a/feeds/compliance/audit/mail/export/helpling.com/'+account
headers = {'Content-Type': 'application/atom+xml'}
xml_data = """<atom:entry xmlns:atom='http://www.w3.org/2005/Atom' xmlns:apps='http://schemas.google.com/apps/2006'> \
<apps:property name='includeDeleted' value='true'/> \
</atom:entry>"""
status, response = http.request(url, 'POST', headers=headers, body=xml_data)
Not sure if it was about the body or the header. It works now and I hope it will help others.
Thanks anyway.