MS Text Analytics Cognitive Service: how to work with local database? - azure

Microsoft provides a service to analyze text data called Text Analytics Cognitive Service.
Is it possible to use this service with local database? i.e. not in Azure
I work with some large databases and as for me it can be interesting to use it for:
Language detection
Key phrase extraction
Named Entity recognition
Sentiment analysis

Once you pull your data that you would like to detect its language
from your local database, you just need to fetch it then just pass in
below method. It would analysis your value in response.
API Access Keys:
private static readonly string endpointKey = "YourEndPonitKey";
private static readonly string endpoint = "https://YourServiceURL.cognitiveservices.azure.com/text/analytics/v2.1/languages";
Code Snippet:
public async Task<object> DetectLanguageAsync(string InputFromDbOrUser)
{
try
{
DetectedLanguageResponseModel ObjDitectedLanguageResponse = new DetectedLanguageResponseModel();
//Create laguage detection request param
RequestModel objRequestModel = new RequestModel();
objRequestModel.id = "1";
objRequestModel.text = InputFromDbOrUser;
//Made Document List
List<RequestModel> Objdocuments = new List<RequestModel>();
Objdocuments.Add(objRequestModel);
//Bind Request Model
LanguageDetection objRequestList = new LanguageDetection();
objRequestList.documents = Objdocuments;
// Bind and Serialize Request Object
var serializedObject = JsonConvert.SerializeObject(objRequestList);
// Call Language Detection API
using (var client = new HttpClient())
using (var request = new HttpRequestMessage())
{
request.Method = HttpMethod.Post;
request.RequestUri = new Uri(endpoint);
request.Content = new StringContent(serializedObject, Encoding.UTF8, "application/json");
request.Headers.Add("Ocp-Apim-Subscription-Key", endpointKey);
var response = await client.SendAsync(request);
//Check status code and retrive response
if (response.IsSuccessStatusCode)
{
ResponseModel objResponse = JsonConvert.DeserializeObject<ResponseModel>(await response.Content.ReadAsStringAsync());
//Check Response List
foreach (var item in objResponse.documents)
{
//Checkings Empty Response and Return to Caller
if (objResponse.documents != null)
{
ObjDitectedLanguageResponse.Language = objResponse.documents[0].detectedLanguages[0].name;
return ObjDitectedLanguageResponse;
}
else
{
return "Sorry, I am not able to find a related topic! Would you like me to Bing Search?";
}
}
}
else
{
var result_string = await response.Content.ReadAsStringAsync();
return result_string;
}
}
return ObjDitectedLanguageResponse;
}
catch (Exception ex)
{
throw new NotImplementedException(ex.Message, ex.InnerException);
}
}
Class Used:
public class DetectedLanguage
{
public string name { get; set; }
public string iso6391Name { get; set; }
}
public class DetectedLanguageResponseModel
{
public dynamic Language { get; set; }
}
public class LanguageDetection
{
public List<RequestModel> documents { get; set; }
}
public class RequestModel
{
public string id { get; set; }
public string text { get; set; }
}
public class ResponseDocument
{
public string id { get; set; }
public List<DetectedLanguage> detectedLanguages { get; set; }
}
public class ResponseModel
{
public List<ResponseDocument> documents { get; set; }
public List<object> errors { get; set; }
}
Note: The current limit is 5,120 characters for each document; if you need to analyze larger documents, you can break them up into
smaller chunks for more you could refer official document
Hope that would help. If you need more implementation assistance please have a look on here

Related

How to commit a list of folders and files to Azure DevOps Repository using Rest API?

I am trying to automate creation of repositories and its initialization using Azure DevOps REST APIs. I am able to successfully create a repo using APIs.
How can we commit a bulk data, say a list of folders and files, that constitute the basic code structure, using REST API? In the request body of Pushes - Create , contentType can either be base64encoded or rawtext. I have used rawtext to test commit of a single file and it worked successfully. Now, I have to commit both files and folders together.
Accually, Rest API is always used to commit the documents related to the project.
If you want to commit all files in folders, you should define the paths of all files in changes. Shayki Abramczyk's comment is really helpful. Note: Git folders cannot be empty.
For example, these two paths will commit folder "content".
"item": {
"path": "/tasks/content/newtasks.md"
}
"item": {
"path": "/tasks/content/inactivetasks.md"
}
Please refer to this similar issue, Rakesh has created a function with C# to push files automatically.
public class Refs
{
public string name { get; set; }
public string objectId { get; set; }
public string oldObjectId { get; set; }
public Creator creator { get; set; }
public string url { get; set; }
}
public class Change
{
public string changeType { get; set; }
public Item item { get; set; }
public Newcontent newContent { get; set; }
}
public class CommitToAdd
{
public string comment { get; set; }
public ChangeToAdd[] changes { get; set; }
}
public class ChangeToAdd
{
public string changeType { get; set; }
public ItemBase item { get; set; }
public Newcontent newContent { get; set; }
}
public class ItemBase
{
public string path { get; set; }
}
public class Newcontent
{
public string content { get; set; }
public string contentType { get; set; }
}
// ### Implementation
//on your Program.cs file
public static class program
{
public async Task AddFileToRepository(string projectName, string repositoryId, Dictionary<string, Task<string>> blobContainer)
{
var refs = new List<Refs>() { new Refs { oldObjectId = "0000000000000000000000000000000000000000", name = Constants.DevOps.MASTER_REPO_REF_NAME } };
var changes = new List<ChangeToAdd>();
foreach (var blob in blobContainer)
{
if (!blob.Key.StartsWith(".git"))
{
ChangeToAdd changeJson = new ChangeToAdd()
{
changeType = "add",
item = new ItemBase() { path = blob.Key },
newContent = new Newcontent()
{
contentType = "rawtext",
content = blob.Value.Result
}
};
changes.Add(changeJson);
}
}
CommitToAdd commit = new CommitToAdd();
commit.comment = "commit from code";
commit.changes = changes.ToArray();
var content = new List<CommitToAdd>() { commit };
var request = new
{
refUpdates = refs,
commits = content
};
var uri = $"https://dev.azure.com/{_orgnizationName}/{projectName}/_apis/git/repositories/{repositoryId}/pushes{Constants.DevOps.API_VERSION}";
using (var client = this.HttpClient)
{
var authorizationToken = Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(string.Format("{0}:{1}", "", personalaccessToken)));
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", authorizationToken);
var requestJson = JsonConvert.SerializeObject(request);
var httpContent = new StringContent(requestJson, Encoding.ASCII, "application/json");
var response = await client.PostAsync(uri, httpContent);
if (!response.IsSuccessStatusCode)
{
throw new Exception(ApplicationMessages.FailedToAddFilesToRepository);
}
}
}
}

POST form-data with Image File using C#

I'm trying to send a post request contain data and image file. but when I send uing my code below, I getting an error. Please see my code below for on client side.
public async Task<HttpContent> PostRequestAsync(string requestUri)
{
string jsonString = string.Empty;
StringContent stringJsonContent = default(StringContent);
HttpResponseMessage requestResponse = new HttpResponseMessage();
try
{
stringJsonContent = new StringContent(jsonString, Encoding.UTF8, "application/json");
requestResponse = await this.httpClient.PostAsync(requestUri, stringJsonContent);
if (requestResponse.IsSuccessStatusCode)
{
return requestResponse?.Content;
}
else if (requestResponse.StatusCode == HttpStatusCode.InternalServerError)
{
}
else if (requestResponse.StatusCode == HttpStatusCode.Unauthorized)
{
}
}
catch (Exception ex)
{
throw ex.InnerException;
}
return requestResponse?.Content;
}
}
and on WebAPI the controller looks below
[HttpPost]
public async Task<ActionResult> UpdateProfile([FromForm] UpdateProfile updateProfile)
Model is
public class UpdateProfile:BaseModel
{
public string firstName { get; set; }
public string lastName { get; set; }
public IFormFile Image { get; set; }
}
but in POSTMAN I'm successfully upload the file using below, so that I have a feeling the there's wrong with my code in client side. Anyone can suggest what I need to add on my code to work? I'm getting an error and not able to send the request.
In your client code, you don’t use the form object to transmit data. StringContent stores the value of the form instead of the key. You can use the MultipartFormDataContent object to transfer all forms and files. In addition, I give an example code.
The code in client.
class Program
{
static async Task Main(string[] args)
{
await PostRequestAsync(#"D:\upload\1.jpg", new HttpClient(), "https://localhost:44370/api/UpdateProfile");
}
public static async Task<string> PostRequestAsync(string filePath,HttpClient _httpClient,string _url)
{
if (string.IsNullOrWhiteSpace(filePath))
{
throw new ArgumentNullException(nameof(filePath));
}
if (!File.Exists(filePath))
{
throw new FileNotFoundException($"File [{filePath}] not found.");
}
//Create form
using var form = new MultipartFormDataContent();
var bytefile = AuthGetFileData(filePath);
var fileContent = new ByteArrayContent(bytefile);
fileContent.Headers.ContentType = MediaTypeHeaderValue.Parse("multipart/form-data");
form.Add(fileContent, "Image", Path.GetFileName(filePath));
//the other data in form
form.Add(new StringContent("Mr."), "firstName");
form.Add(new StringContent("Loton"), "lastName");
form.Add(new StringContent("Names--"), "Name");
var response = await _httpClient.PostAsync($"{_url}", form);
response.EnsureSuccessStatusCode();
var responseContent = await response.Content.ReadAsStringAsync();
return responseContent;
}
//Convert file to byte array
public static byte[] AuthGetFileData(string fileUrl)
{
using (FileStream fs = new FileStream(fileUrl, FileMode.OpenOrCreate, FileAccess.ReadWrite))
{
byte[] buffur = new byte[fs.Length];
using (BinaryWriter bw = new BinaryWriter(fs))
{
bw.Write(buffur);
bw.Close();
}
return buffur;
}
}
}
The action in WebApi
[ApiController]
[Route("[controller]")]
public class ApiController:Controller
{
[HttpPost("get")]
public string get([FromForm]UpdateProfile updateProfile)
{
return "get";
}
[HttpPost("UpdateProfile")]
public async Task<ActionResult> UpdateProfile([FromForm] UpdateProfile updateProfile)
{
return Json("The result data ");
}
}
The model
public class UpdateProfile : BaseModel
{
public string firstName { get; set; }
public string lastName { get; set; }
public IFormFile Image { get; set; }
}
public class BaseModel
{
public string Name { get; set; }
}
result:

Azure tranlate API call is returning and 40100

I used this HTTP-Get request to get a Bearer token for the translation:
https://api.cognitive.microsoft.com/sts/v1.0/issueToken?Subscription-Key=1fo8xxx
Using the returned Bearer I wanted to translate a short text using this API endpoint:
https://api.cognitive.microsofttranslator.com/translate?api-version=3.0&to=de
In the header I put this:
Content-Type: application/json; charset=UTF-8.
And in the body I put this:
[
{"Text":"I would really like to drive your car around the block a few times."}
]
I am using Postman, so in the authorization tab I selected Bearer and inserted in the field next to it this:
Bearer <result from the first API call>
If I send the reqeuest I get this result:
{"error":{"code":401000,"message":"The request is not authorized because credentials are missing or invalid."}}
In case someone ever stumbles upon this, after hours of trial and error I found out you need to pass the Ocp-Apim-Subscription-Region param in the header.
Here is an example in python that I was able to run successfully.
import json
import requests
def translate(text, source_language, dest_language):
if not <Secret Key>:
return 'Error: the translation service is not configured.'
headers = {'Ocp-Apim-Subscription-Key': <Secret Key>,
'Ocp-Apim-Subscription-Region': <region>,
'Content-type': 'application/json'}
url = 'https://api.cognitive.microsofttranslator.com/translate?api-version=3.0&from={}&to={}' \
.format(source_language, dest_language)
body = [{'text': text}]
request = requests.post(url, headers=headers, json=body)
if request.status_code != 200:
return 'Error: the translation service failed.'
return json.loads(request.content.decode('utf-8-sig'))
The list of regions and other examples can be found here:
https://learn.microsoft.com/en-us/azure/cognitive-services/translator/reference/v3-0-reference
Don't be fooled by the curl example that is not using the region..
Your request needs the "OCP-Apim-Subscription-Key" header. Take a look on the official example:
using System;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
// Install Newtonsoft.Json with NuGet
using Newtonsoft.Json;
/// <summary>
/// The C# classes that represents the JSON returned by the Translator Text API.
/// </summary>
public class TranslationResult
{
public DetectedLanguage DetectedLanguage { get; set; }
public TextResult SourceText { get; set; }
public Translation[] Translations { get; set; }
}
public class DetectedLanguage
{
public string Language { get; set; }
public float Score { get; set; }
}
public class TextResult
{
public string Text { get; set; }
public string Script { get; set; }
}
public class Translation
{
public string Text { get; set; }
public TextResult Transliteration { get; set; }
public string To { get; set; }
public Alignment Alignment { get; set; }
public SentenceLength SentLen { get; set; }
}
public class Alignment
{
public string Proj { get; set; }
}
public class SentenceLength
{
public int[] SrcSentLen { get; set; }
public int[] TransSentLen { get; set; }
}
private const string key_var = "TRANSLATOR_TEXT_SUBSCRIPTION_KEY";
private static readonly string subscriptionKey = Environment.GetEnvironmentVariable(key_var);
private const string endpoint_var = "TRANSLATOR_TEXT_ENDPOINT";
private static readonly string endpoint = Environment.GetEnvironmentVariable(endpoint_var);
static Program()
{
if (null == subscriptionKey)
{
throw new Exception("Please set/export the environment variable: " + key_var);
}
if (null == endpoint)
{
throw new Exception("Please set/export the environment variable: " + endpoint_var);
}
using (var client = new HttpClient())
using (var request = new HttpRequestMessage())
{
// Build the request.
// Set the method to Post.
request.Method = HttpMethod.Post;
// Construct the URI and add headers.
request.RequestUri = new Uri(endpoint + route);
request.Content = new StringContent(requestBody, Encoding.UTF8, "application/json");
request.Headers.Add("Ocp-Apim-Subscription-Key", subscriptionKey);
// Send the request and get response.
HttpResponseMessage response = await client.SendAsync(request).ConfigureAwait(false);
// Read response as a string.
string result = await response.Content.ReadAsStringAsync();
// Deserialize the response using the classes created earlier.
TranslationResult[] deserializedOutput = JsonConvert.DeserializeObject<TranslationResult[]>(result);
// Iterate over the deserialized results.
foreach (TranslationResult o in deserializedOutput)
{
// Print the detected input language and confidence score.
Console.WriteLine("Detected input language: {0}\nConfidence score: {1}\n", o.DetectedLanguage.Language, o.DetectedLanguage.Score);
// Iterate over the results and print each translation.
foreach (Translation t in o.Translations)
{
Console.WriteLine("Translated to {0}: {1}", t.To, t.Text);
}
}
}
Console.Read();
}
https://learn.microsoft.com/en-us/azure/cognitive-services/translator/quickstart-translate?pivots=programming-language-csharp

How to Insert Data in a SharePoint List via GraphAPI?

I have SharePoint List which content a Reference No. It'd URL look like this:
https://xyz.sharepoint.com/sites/site_name/Lists/List_name/AllItems.aspx
This List content ref no. I am trying to insert this data in the list.
{
"Optimum_x0020_Case_x0020_Reference": "000777"
}
This is url I am posting the data.
https://graph.microsoft.com/v1.0/sites/xyz.sharepoint.com:/sites/site_name:/lists/List_names/items
But I am getting this error:
error": {
"code": "accessDenied",
"message": "The caller does not have permission to perform the action.",
How to solve this? Using the access I am able to create folder, sub folder and Update meta data for other document.
What is the context of what you are doing this with? Is it an app that you are using? Are you inserting data on a already existing listitem or a new item?
This is the code I had to use for my UWP App. I'm not sure if this will help you or not, but it should give you a little guidance I hope. Creating the dictionary and figuring out the XML structure were the keys things I had to piece together to get my code to work.
I declared my scopes in my App.xaml.cs
public static string[] scopes = new string[] { "user.ReadWrite", "Sites.ReadWrite.All", "Files.ReadWrite.All" };
I have a submit button that I use on my MainPage
private async void SubmitButton_Click(object sender, RoutedEventArgs e)
{
var (authResult, message) = await Authentication.AquireTokenAsync();
if (authResult != null)
{
await SubmitDataWithTokenAsync(submiturl, authResult.AccessToken);
}
}
This calls the AquireToken which I have in a class file:
public static async Task<(AuthenticationResult authResult, string message)> AquireTokenAsync()
{
string message = String.Empty;
string[] scopes = App.scopes;
AuthenticationResult authResult = null;
message = string.Empty;
//TokenInfoText.Text = string.Empty;
IEnumerable<IAccount> accounts = await App.PublicClientApp.GetAccountsAsync();
IAccount firstAccount = accounts.FirstOrDefault();
try
{
authResult = await App.PublicClientApp.AcquireTokenSilentAsync(scopes, firstAccount);
}
catch (MsalUiRequiredException ex)
{
// A MsalUiRequiredException happened on AcquireTokenSilentAsync. This indicates you need to call AcquireTokenAsync to acquire a token
System.Diagnostics.Debug.WriteLine($"MsalUiRequiredException: {ex.Message}");
try
{
authResult = await App.PublicClientApp.AcquireTokenAsync(scopes);
}
catch (MsalException msalex)
{
message = $"Error Acquiring Token:{System.Environment.NewLine}{msalex}";
}
}
catch (Exception ex)
{
message = $"Error Acquiring Token Silently:{System.Environment.NewLine}{ex}";
}
return (authResult,message);
}
I had created another class for my SharePointList
public class SharePointListItems
{
public class Lookup
{
public string SerialNumber { get; set; }
public string id { get; set; }
public override string ToString()
{
return SerialNumber;
}
}
public class Value
{
public Lookup fields { get; set; }
}
public class Fields
{
[JsonProperty("#odata.etag")]
public string ODataETag { get; set; }
public string ParameterA { get; set; }
public string ParameterB { get; set; }
public string ParameterC { get; set; }
}
public class RootObject
{
[JsonProperty("#odata.context")]
public string ODataContext { get; set; }
[JsonProperty("#odata.etag")]
public string ODataETag { get; set; }
[JsonProperty("fields#odata.context")]
public string FieldsODataContext { get; set; }
public Fields Fields { get; set; }
}
}
I used this class to create a dictionary for submitting my data to SharePoint.
public async Task<string> SubmitDataWithTokenAsync(string url, string token)
{
var httpClient = new HttpClient();
HttpResponseMessage response;
try
{
var root = new
{
fields = new Dictionary<string, string>
{
// The second string are public static strings that I
// I declared in my App.xaml.cs because of the way my app
// is set up.
{ "ParameterA", App.ParameterA },
{ "ParameterB", App.ParameterB },
{ "ParameterC", App.ParameterC },
}
};
var s = new JsonSerializerSettings { DateFormatHandling = DateFormatHandling.MicrosoftDateFormat };
var content = JsonConvert.SerializeObject(root, s);
var request = new HttpRequestMessage(HttpMethod.Post, url);
//Add the token in Authorization header
request.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", token);
request.Content = new StringContent(content, Encoding.UTF8, "application/json");
response = await httpClient.SendAsync(request);
var responseString = await response.Content.ReadAsStringAsync();
return responseString;
}
catch (Exception ex)
{
return ex.ToString();
}
}
And my submiturl is defined:
public static string rooturl = "https://graph.microsoft.com/v1.0/sites/xxxxxx.sharepoint.com,495435b4-60c3-49b7-8f6e-1d262a120ae5,0fad9f67-35a8-4c0b-892e-113084058c0a/";
string submiturl = rooturl + "lists/18a725ac-83ef-48fb-a5cb-950ca2378fd0/items";
You can also look at my posted question on a similar topic here.

Create field in SharePoint programmatically using CSOM (Not with XML)

Is it possible to create fields in SharePoint with CSOM, not using XML?
I've seen many examples using XML, but none with just setting properties for the field programmatically?
fields.Add(new **FieldCreationInformation** {
InternalName = "Test",
etc..
});
That's doable, in the following example is introduced a FieldCreationInformation class:
[XmlRoot("Field")]
public class FieldCreationInformation
{
[XmlAttribute("ID")]
public Guid Id { get; set; }
[XmlAttribute()]
public string DisplayName { get; set; }
[XmlAttribute("Name")]
public string InternalName { get; set; }
[XmlIgnore()]
public bool AddToDefaultView { get; set; }
//public IEnumerable<KeyValuePair<string, string>> AdditionalAttributes { get; set; }
[XmlAttribute("Type")]
public FieldType FieldType { get; set; }
[XmlAttribute()]
public string Group { get; set; }
[XmlAttribute()]
public bool Required { get; set; }
public string ToXml()
{
var serializer = new XmlSerializer(GetType());
var settings = new XmlWriterSettings();
settings.Indent = true;
settings.OmitXmlDeclaration = true;
var emptyNamepsaces = new XmlSerializerNamespaces(new[] { XmlQualifiedName.Empty });
using (var stream = new StringWriter())
using (var writer = XmlWriter.Create(stream, settings))
{
serializer.Serialize(writer, this, emptyNamepsaces);
return stream.ToString();
}
}
public FieldCreationInformation()
{
Id = Guid.NewGuid();
}
}
and then extension method for creating a new field:
public static class FieldCollectionExtensions
{
public static Field Add(this FieldCollection fields, FieldCreationInformation info)
{
var fieldSchema = info.ToXml();
return fields.AddFieldAsXml(fieldSchema, info.AddToDefaultView, AddFieldOptions.AddFieldToDefaultView);
}
}
Usage
var fieldInfo = new FieldCreationInformation();
fieldInfo.FieldType = FieldType.Geolocation;
fieldInfo.InternalName = "ContactsLocation";
fieldInfo.DisplayName = "Contacts Location";
ctx.Site.RootWeb.Fields.Add(fieldInfo);
ctx.ExecuteQuery();
When I add fields with CSOM/JSOM I use the method on the FieldCollection AddFieldAsXml. This requires you to build a string of xml with all of the properties for the desired field, but it works. I included an excerpt of the related cpde below:
Microsoft.SharePoint.Client.Web web = _context.Web;
FieldCollection fields = web.Fields;
_context.Load(fields);
_context.ExecuteQuery();
Field field = fields.FirstOrDefault(f => f.StaticName == _staticName);
if (field == null)
{
Field createdField = fields.AddFieldAsXml(xml, false, AddFieldOptions.AddToNoContentType);
_context.Load(createdField);
_context.ExecuteQuery();
}
Similar code is used if you would like to add a field directly to an existing list.

Resources