My Azure devops page will look like :
I have 4 pandas dataframes.
I need to create 4 sub pages in Azure devops wiki from each dataframe.
Say, Sub1 from first dataframe, Sub2 from second dataframe and so on.
My result should be in tab. The result should look like :
Is it possible to create subpages thru API?
I have referenced the following docs. But I am unable to make any sense. Any inputs will be helpful. Thanks.
https://github.com/microsoft/azure-devops-python-samples/blob/main/API%20Samples.ipynb
https://learn.microsoft.com/en-us/rest/api/azure/devops/wiki/pages/create%20or%20update?view=azure-devops-rest-6.0
To create a wiki subpage, you should use Pages - Create Or Update api, and specify the path to pagename/subpagename. Regarding how to use the api in Python, you could use Azure DevOps Python API and refer to the sample below:
def create_or_update_page(self, parameters, project, wiki_identifier, path, version, comment=None, version_descriptor=None):
"""CreateOrUpdatePage.
[Preview API] Creates or edits a wiki page.
:param :class:`<WikiPageCreateOrUpdateParameters> <azure.devops.v6_0.wiki.models.WikiPageCreateOrUpdateParameters>` parameters: Wiki create or update operation parameters.
:param str project: Project ID or project name
:param str wiki_identifier: Wiki ID or wiki name.
:param str path: Wiki page path.
:param String version: Version of the page on which the change is to be made. Mandatory for `Edit` scenario. To be populated in the If-Match header of the request.
:param str comment: Comment to be associated with the page operation.
:param :class:`<GitVersionDescriptor> <azure.devops.v6_0.wiki.models.GitVersionDescriptor>` version_descriptor: GitVersionDescriptor for the page. (Optional in case of ProjectWiki).
:rtype: :class:`<WikiPageResponse> <azure.devops.v6_0.wiki.models.WikiPageResponse>`
"""
route_values = {}
if project is not None:
route_values['project'] = self._serialize.url('project', project, 'str')
if wiki_identifier is not None:
route_values['wikiIdentifier'] = self._serialize.url('wiki_identifier', wiki_identifier, 'str')
query_parameters = {}
if path is not None:
query_parameters['path'] = self._serialize.query('path', path, 'str')
if comment is not None:
query_parameters['comment'] = self._serialize.query('comment', comment, 'str')
if version_descriptor is not None:
if version_descriptor.version_type is not None:
query_parameters['versionDescriptor.versionType'] = version_descriptor.version_type
if version_descriptor.version is not None:
query_parameters['versionDescriptor.version'] = version_descriptor.version
if version_descriptor.version_options is not None:
query_parameters['versionDescriptor.versionOptions'] = version_descriptor.version_options
additional_headers = {}
if version is not None:
additional_headers['If-Match'] = version
content = self._serialize.body(parameters, 'WikiPageCreateOrUpdateParameters')
response = self._send(http_method='PUT',
location_id='25d3fbc7-fe3d-46cb-b5a5-0b6f79caf27b',
version='6.0-preview.1',
route_values=route_values,
query_parameters=query_parameters,
additional_headers=additional_headers,
content=content)
response_object = models.WikiPageResponse()
response_object.page = self._deserialize('WikiPage', response)
response_object.eTag = response.headers.get('ETag')
return response_object
More details, you could refer to the link below:
https://github.com/microsoft/azure-devops-python-api/blob/451cade4c475482792cbe9e522c1fee32393139e/azure-devops/azure/devops/v6_0/wiki/wiki_client.py#L107
Able to achieve with rest api
import requests
import base64
import pandas as pd
pat = 'TO BE FILLED BY YOU' #CONFIDENTIAL
authorization = str(base64.b64encode(bytes(':'+pat, 'ascii')), 'ascii')
headers = {
'Accept': 'application/json',
'Authorization': 'Basic '+authorization
}
df = pd.read_csv('sf_metadata.csv') #METADATA OF 3 TABLES
df.set_index('TABLE_NAME', inplace=True,drop=True)
df_test1 = df.loc['CURRENCY']
x1 = df_test1.to_html() # CONVERTING TO HTML TO PRESERVE THE TABULAR STRUCTURE
#JSON FOR PUT REQUEST
SamplePage1 = {
"content": x1
}
#API CALLS TO AZURE DEVOPS WIKI
response = requests.put(
url="https://dev.azure.com/xxx/yyy/_apis/wiki/wikis/yyy.wiki/pages?path=SamplePag2&api-version=6.0", headers=headers,json=SamplePage1)
print(response.text)
Based on #usr_lal123's answer, here is a function that can update a wiki page or create it if it doesn't:
import requests
import base64
pat = '' # Personal Access Token to be created by you
authorization = str(base64.b64encode(bytes(':'+pat, 'ascii')), 'ascii')
def update_or_create_wiki_page(organization, project, wikiIdentifier, path):
# Check if page exists by performing a Get
headers = {
'Accept': 'application/json',
'Authorization': 'Basic '+authorization
}
response = requests.get(url=f"https://dev.azure.com/{organization}/{project}/_apis/wiki/wikis/{wikiIdentifier}/pages?path={path}&api-version=6.0", headers=headers)
# Existing page will return an ETag in their response, which is required when updating a page
version = ''
if response.ok:
version = response.headers['ETag']
# Modify the headers
headers['If-Match'] = version
pageContent = {
"content": "[[_TOC_]] \n ## Section 1 \n normal text"
+ "\n ## Section 2 \n [ADO link](https://azure.microsoft.com/en-us/products/devops/)"
}
response = requests.put(
url=f"https://dev.azure.com/{organization}/{project}/_apis/wiki/wikis/{wikiIdentifier}/pages?path={path}&api-version=6.0", headers=headers,json=pageContent)
print("response.text: ", response.text)
Related
I have written a code for locust load testing for my case, where i can do a token call and then will make feature calls as per below code.
This is working fine with single token and 'n' number of users mentioned in master
I made a token call outside class and sending it as parameter to ENV. The userclass is reading the single token and using the same for all users.
I do not want to make token call inside the class, as it generates new token in every execution.
I'm looking if there is anyway like making token calls based on number of users mentioned on master -u and using only those tokens in User class.
Please suggest me if there is any documentation pointer which i can refer for this usecase
#! /usr/bin/python3.6
import json
from locust import HttpUser, task, constant, tag, events
from locust.log import setup_logging
import os
from datetime import datetime
import requests
setup_logging("INFO", None)
#events.init_command_line_parser.add_listener
def init_parser(parser):
parser.add_argument("--resp-delay", type=str, env_var="LOCUST_RESP_DELAY", default="", help="It's working")
parser.add_argument("--resp-size", type=str, env_var="LOCUST_RESP_SIZE", default="", help="It's working")
parser.add_argument("--env-endpoint", type=str, env_var="LOCUST_ENV_ENDPOINT", default="", help="It's working")
#events.init.add_listener
def _(environment, **kw):
os.environ['resp-delay'] = environment.parsed_options.resp_delay
os.environ['resp-size'] = environment.parsed_options.resp_size
os.environ['env-endpoint'] = environment.parsed_options.env_endpoint
with open("resources/data/" + environment.parsed_options.env_endpoint + '/data.json') as f:
data = json.load(f)
cal_transaction_id = datetime.now().strftime('%Y%m%dT%H%M%S')
#retrive cliend id and client secret from bitbucket repo
dict_car_app_all = data["data"]
print("env-endpoint:" + os.environ.get("env-endpoint"))
token_url = "https://ENDPOINT/auth/token"
#retrive cliend id and client secret from bitbucket repo
token_payload = "client_id=" + dict_car_app_all[0]["client_id"] + "&client_secret=" + dict_car_app_all[0]["client_secret"]
token_headers = {
'Content-Type': 'application/x-www-form-urlencoded'
}
response = requests.request("POST", token_url, data=token_payload, headers=token_headers,
cert=( 'resources/certs/' + environment.parsed_options.env_endpoint + '/MyCERT.pem',
'resources/certs/' + environment.parsed_options.env_endpoint + '/MYCERT.key'))
result = json.loads(response.text)
token = result["access_token"]
os.environ['access_token'] = token
os.environ['cal_transaction_id'] = cal_transaction_id
#class User_1(User):
class User_0(HttpUser):
wait_time = constant(1)
host = "host.com"
#tag('tag1')
#task
def load_test_api_tag1(self):
token_0 = os.environ.get('access_token')
cal_transaction_id = os.environ.get('cal_transaction_id')
env_endpoint = os.environ.get('env-endpoint')
resp_delay = os.environ.get("resp-delay")
resp_size = os.environ.get("resp-size")
feature_headers = {
'Authorization': "Bearer " + str(token_0),
'sm_transactionID': cal_transaction_id
}
url = "https://ENDPOINT/SERVICE/mytestservice/first_test"
querystring = {"response_delay": resp_delay, "data_size": resp_size}
self.client.request("GET", url, headers=feature_headers, params=querystring,
cert = ('resources/certs/' + env_endpoint + '/MyCERT.pem',
'resources/certs/' + env_endpoint + '/MyCERT.key'))
You can generate tokens in User class's on_start method so each user generates a new token when spawning.
class MyUser(User):
def on_start(self):
#generate token here and assign an instance variable like self.token=abc
super().on_start()
there is a drawback to this though, if your user count is more than your token generating service can handle some users will not be able to spawn, the way I do in my tests is if token generating part is not a part of the system I am testing, I generate tokens beforehand and write it in some file or some external db and read them from there.
My code does following tasks on daily basis:
Create a date folder on sharepoint.DONE
Create a category folder in the date folder of sharepoint. DONE
Upload a file in respective category folder. DONE
4. Find the url of file. PENDING
Send the url on email. DONE
def upload_files(base_url, site, access_token, fileName, filePath, fileFolderUrl):
try:
fileHandle = open(filePath, 'rb')
#fileFolderUrl = f'Shared Documents/Output/{subFolder}'
requestFileUploadUrl = f"{base_url}/{site}/_api/web/GetFolderByServerRelativeUrl('{fileFolderUrl}')/Files/add(url='{fileName}',overwrite=true)"
headers = {
'Accept':'application/json; odata=verbose',
'Content-Type':'application/json; odata=verbose',
'Authorization': 'Bearer {}'.format(access_token)
}
uploadFileResult = requests.post(requestFileUploadUrl, headers=headers, data=fileHandle)
if uploadFileResult.status_code == 200:
return "success"
else:
return "failure"
except Exception as e:
print(f"Error in uploading files to sharepoint:{e}")
return "error"
Can someone please help me understand the process to obtain the url of this file that I am required to upload in dynamically created folders on sharepoint using python3+?
So,the structure is : SharedDocuments/Output/Today's Date/CategoryA/file.xlsx
One solution could be to create the URL manually as you already know the url, the site and the name of the file.
I figure out that you can create the link to share a file with this pattern :
<base_url>/<site>/<FileFolderUrl>/<File>?&web=1
For example : https://company.sharepoint.com/sites/test/Documents%20partages/testing.jpg?&web=1
It works only for people that already have access it.
I have not yet tested in an automatic solution but only by hand in an empirical way
I know it's been a long while but was scrolling through it today and thought of answering it. Hope it can be helpful to someone in the future.
def get_out_file_url(base_url, site, access_token, file_folder, fileName):
"""
These are random examples of file_folder and base_url
file_folder = 'Shared Documents/Output/Dated/Category'
base_url = 'https://company.sharepoint.com'
"""
requestFileGetUrl = f'{base_url}/sites/{site}/_api/web/GetFolderByServerRelativeUrl(\'' + fileUrl + '/'+ '\')/files?$filter'
headers = {
'Accept':'application/json; odata=verbose',
'Content-Type':'application/json; odata=verbose',
'Authorization': 'Bearer {}'.format(access_token)
}
getFileResult = requests.get(requestFileGetUrl, headers=headers)
values = getFileResult.json()
fileData = values['d']['results']
for key in fileData:
if key['Name'] == fileName:
fileURLLink = key['ServerRelativeUrl']
fileCreatedTime = key['TimeCreated']
fileLastModified = key['TimeLastModified']
fileNewUrl = base_url + urllib.parse.quote(fileURLLink)
return fileNewUrl
else:
return file_folder
How can I access Airtable's pagination by including a given offset in either the url or query?
My attempt below at including in the url has failed with the error "KeyError: 'offset'".
I've been able to access the 1st offset per the function at_page(). For simplicity's sake, I took the result from the function and hardcoded it to the variable offset_01.
Thank you!
import requests
import pandas as pd
offset_01 = 'itrCG3VDp2c34x1j1/recKTJGYMICe13iA8'
url = 'https://api.airtable.com/v0/PRIVATETABLEKEY/accounts' + \
'&offset=' + offset_01
headers = {'Authorization': 'Bearer PRIVATEAPIKEY'}
# Calling API
response = requests.get(
url,
headers=headers,
)
json_response = response.json()
# Finding AT page offset
def at_page(at_json):
df_initial = pd.DataFrame(at_json)
at_offset = df_initial['offset'][0]
return at_offset
offset = at_page(json_response)
Figured it out:
It’s the params={‘offset’: ‘itrXXXXXX/recXXXXXX’} query that allows access to the next 100 records:
# Calling API
response = requests.get(
url,
headers=headers,
params={'offset': 'itrXXXXXX/recXXXXXX'}
)
json_response = response.json()
Hope this helps someone. Cheers!
I've got a list of ~3,000 URLs I'm trying to create Google shortened links of, the idea is this CSV has a list of links and I want my code to output the shortened links in the column next to the original URLs.
I've been trying to modify the code found on this site here but I'm not skilled enough to get it to work.
Here's my code (I would not normally post an API key but the original person who asked this already posted it publicly on this site) :
import json
import pandas as pd
df = pd.read_csv('Links_Test.csv')
def shorternUrl(my_URL):
API_KEY = "AIzaSyCvhcU63u5OTnUsdYaCFtDkcutNm6lIEpw"
apiUrl = 'https://www.googleapis.com/urlshortener/v1/url'
longUrl = my_URL
headers = {"Content-type": "application/json"}
data = {"longUrl": longUrl}
h = httplib2.Http('.cache')
headers, response = h.request(apiUrl, "POST", json.dumps(data), headers)
return response
for url in df['URL']:
x = shorternUrl(url)
# Then I want it to write x into the column next to the original URL
But I only get errors at this point, before I even started figuring out how to write the new URLs to the CSV file.
Here's some sample data:
URL
www.apple.com
www.google.com
www.microsoft.com
www.linux.org
Thank you for your help,
Me
I think the issue is that you didnot include the API key in the request. By the way, the certifi package allows you to secure a connection to a link. You can get it using pip install certifi or pip urllib3[secure].
Here I create my own API key, so you might want to replace it with yours.
from urllib3 import PoolManager
import json
import certifi
sampleURL = 'http://www.apple.com'
APIkey = 'AIzaSyD8F41CL3nJBpEf0avqdQELKO2n962VXpA'
APIurl = 'https://www.googleapis.com/urlshortener/v1/url?key=' + APIkey
http = PoolManager(cert_reqs = 'CERT_REQUIRED', ca_certs=certifi.where())
def shortenURL(url):
data = {'key': APIkey, 'longUrl' : url}
response = http.request("POST", APIurl, body=json.dumps(data), headers= {'Content-Type' : 'application/json'}).data.decode('utf-8')
r = json.loads(response)
return (r['id'])
The decoding part converts the response object into a string so that we can convert it to a JSON and retrieve data.
From there on, you can store the data into another column and so on.
For the sampleUrl, I got back https(goo.gl/nujb) from the function.
I found a solution here:
https://pypi.python.org/pypi/pyshorteners
Example copied from linked page:
from pyshorteners import Shortener
url = 'http://www.google.com'
api_key = 'YOUR_API_KEY'
shortener = Shortener('Google', api_key=api_key)
print "My short url is {}".format(shortener.short(url))
I am trying to access signed envelop using REST API and I getting response from REST API but when I save the response to pdf, it doesn't show any content.
Below is a small python code that I wrote to achieve this.
import json
import requests
url = "https://demo.docusign.net/restapi/v2/accounts/[Account ID]/envelopes/[Envelope ID]/documents/1"
headers = {'X-DocuSign-Authentication': '<DocuSignCredentials><Username>[username]</Username><Password>[password]</Password><IntegratorKey>[Integrator Key]</IntegratorKey></DocuSignCredentials>'}
Response = requests.get(url, headers = headers)
file = open("agreement.pdf", "w")
file.write(Response.text.encode('UTF-8'))
file.close()
Try using httplib2, also the content is always going to come back formatted as a PDF through DocuSign, so I don't think you need to set the encoding.
import json
import requests
import httplib2
url = "https://demo.docusign.net/restapi/v2/accounts/[Account ID]/envelopes/[Envelope ID]/documents/1"
headers = {'X-DocuSign-Authentication': '<DocuSignCredentials><Username>[username]</Username><Password>[password]</Password><IntegratorKey>[Integrator Key]</IntegratorKey></DocuSignCredentials>'}
http = httplib2.Http();
response, content = http.request(url, 'GET', headers = headers)
status = response.get('status');
if (status != '200'):
print("Error: " % status); sys.exit();
file = open("agreement.pdf", "w")
file.write(content)
file.close()
Have you seen the DocuSign API Walkthroughs? These are accessible through the Developer Center, I suggest you take a look as there are code samples for 9 common API use-cases, each written in Python (and 5 other languages), including the 6th walkthrough which shows you how to query and download docs from completed envelopes.
The main problem with your code is that it's assuming the response only contains the document data, which is not correct. The response will have headers and a body, that's why you can't render your document when you simply write it all to a file.
I suggest you use httplib2, which is what the following Python sample uses:
http://iodocs.docusign.com/APIWalkthrough/getEnvelopeDocuments
Here is a snippet of the Python code but check out the above link for full code:
#
# STEP 2 - Get Envelope Document(s) Info and Download Documents
#
# append envelopeUri to baseURL and use in the request
url = baseUrl + envelopeUri + "/documents";
headers = {'X-DocuSign-Authentication': authenticateStr, 'Accept': 'application/json'};
http = httplib2.Http();
response, content = http.request(url, 'GET', headers=headers);
status = response.get('status');
if (status != '200'):
print("Error calling webservice, status is: %s" % status); sys.exit();
data = json.loads(content);
envelopeDocs = data.get('envelopeDocuments');
uriList = [];
for docs in envelopeDocs:
# print document info
uriList.append(docs.get("uri"));
print("Document Name = %s, Uri = %s" % (docs.get("name"), uriList[len(uriList)-1]));
# download each document
url = baseUrl + uriList[len(uriList)-1];
headers = {'X-DocuSign-Authentication': authenticateStr};
http = httplib2.Http();
response, content = http.request(url, 'GET', headers=headers);
status = response.get('status');
if (status != '200'):
print("Error calling webservice, status is: %s" % status); sys.exit();
fileName = "doc_" + str(len(uriList)) + ".pdf";
file = open(fileName, 'w');
file.write(content);
file.close();
print ("\nDone downloading document(s)!\n");