How can I create a zip file from array variable [fileName and FileContent]. using azure function or Azure logic app(with out any third part service) - azure

How can I create a zip file from array variable [{"fileName":"", "FileContent" :""}]. using azure function or Azure logic app(with out any third part service)

For now, Compress/Zip files is not supported in logic app(if we do not use third party service), you could upvote for this feature on feedback page.
But for azure function, we can implement it by code without third party service. I wrote a sample in my function, please refer to my function code below:
using System;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using System.IO.Compression;
namespace FunctionApp1
{
public static class Function1
{
[FunctionName("Function1")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req,
ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
FileItem[] data = JsonConvert.DeserializeObject<FileItem[]>(requestBody);
//As the request data you provided is a array with multiple file items, so here use foreach to loop each file item.
foreach (FileItem item in data)
{
log.LogInformation(item.fileName);
log.LogInformation(item.FileContent);
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
var demoFile = archive.CreateEntry(item.fileName + ".txt");
using (var entryStream = demoFile.Open())
using (var streamWriter = new StreamWriter(entryStream))
{
streamWriter.Write(item.FileContent);
}
}
//here I create the zip file in local, you can modify the code to create the zip file anywhere else you want.
using (var fileStream = new FileStream(#"D:\Temp\" + item.fileName + ".zip", FileMode.Create))
{
memoryStream.Seek(0, SeekOrigin.Begin);
memoryStream.CopyTo(fileStream);
}
}
}
string responseMessage = "complete";
return new OkObjectResult(responseMessage);
}
}
public class FileItem
{
public string fileName { get; set; }
public string FileContent { get; set; }
}
}
Running the function above and request it in postman.
Then we can see the zip file was created in the path which I specified in code.

Related

Azure Cognitive Service\Computer Visio\OCR - Can I use it into into WebSite C#

I'm trying to use Azure Ocr into my website c#.
I added the package Microsoft.Azure.CognitiveServices.Vision.ComputerVision and I wrote code, with key and endpoint of my subscription.
static string subscriptionKey = "mykey";
static string endpoint = "https://myocr.cognitiveservices.azure.com/";
private const string ANALYZE_URL_IMAGE = "https://raw.githubusercontent.com/Azure-Samples/cognitive-services-sample-data-files/master/ComputerVision/Images/printed_text.jpg";
protected void Page_Load(object sender, EventArgs e)
{
// Create a client
ComputerVisionClient client = Authenticate(endpoint, subscriptionKey);
// Analyze an image to get features and other properties.
AnalyzeImageUrl(client, ANALYZE_URL_IMAGE).Wait();
}
public static ComputerVisionClient Authenticate(string endpoint, string key)
{
ComputerVisionClient client =
new ComputerVisionClient(new ApiKeyServiceClientCredentials(key))
{ Endpoint = endpoint };
return client;
}
public static async Task AnalyzeImageUrl(ComputerVisionClient client, string imageUrl)
{
// Read text from URL
var textHeaders = await client.ReadAsync(imageUrl);
...
}
It seems all ok, but at line
var textHeaders = await client.ReadAsync(urlFile);
website crashes.
I don't understand why. No error, it's just stopped.
So I ask: azure ocr can to be use only with console app?
EDIT
The code is ok for ConsoleApp and WebApp but not working for my asp.net WEBSITE.
Could be a problem with async?
We can use OCR with web app also,I have taken the .net core 3.1 webapp in Visual Studio and installed the dependency of Microsoft.Azure.CognitiveServices.Vision.ComputerVision by selecting the check mark of include prerelease as shown in the below image:
After creating computer vision resource in Azure Portal, copied the key and endpoint from there and used inside the c# code.
using System;
using System.Collections.Generic;
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision;
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision.Models;
using System.Threading.Tasks;
using System.IO;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System.Threading;
using System.Linq;
namespace ComputerVisionQuickstart
{
class Program
{
// Add your Computer Vision subscription key and endpoint
static string subscriptionKey = "c1****b********";
static string endpoint = ".abc.cognitiveservices.azure.com/";
private const string READ_TEXT_URL_IMAGE = "https://raw.githubusercontent.com/Azure-Samples/cognitive-services-sample-data-files/master/ComputerVision/Images/printed_text.jpg";
static void Main(string[] args)
{
Console.WriteLine("Azure Cognitive Services Computer Vision - .NET quickstart example");
Console.WriteLine();
ComputerVisionClient client = Authenticate(endpoint, subscriptionKey);
// Extract text (OCR) from a URL image using the Read API
ReadFileUrl(client, READ_TEXT_URL_IMAGE).Wait();
}
public static ComputerVisionClient Authenticate(string endpoint, string key)
{
ComputerVisionClient client =
new ComputerVisionClient(new ApiKeyServiceClientCredentials(key))
{ Endpoint = endpoint };
return client;
}
public static async Task ReadFileUrl(ComputerVisionClient client, string urlFile)
{
Console.WriteLine("----------------------------------------------------------");
Console.WriteLine("READ FILE FROM URL");
Console.WriteLine();
// Read text from URL
var textHeaders = await client.ReadAsync(urlFile);
// After the request, get the operation location (operation ID)
string operationLocation = textHeaders.OperationLocation;
Thread.Sleep(2000);
// Retrieve the URI where the extracted text will be stored from the Operation-Location header.
// We only need the ID and not the full URL
const int numberOfCharsInOperationId = 36;
string operationId = operationLocation.Substring(operationLocation.Length - numberOfCharsInOperationId);
// Extract the text
ReadOperationResult results;
Console.WriteLine($"Extracting text from URL file {Path.GetFileName(urlFile)}...");
Console.WriteLine();
do
{
results = await client.GetReadResultAsync(Guid.Parse(operationId));
}
while ((results.Status == OperationStatusCodes.Running ||
results.Status == OperationStatusCodes.NotStarted));
// Display the found text.
Console.WriteLine();
var textUrlFileResults = results.AnalyzeResult.ReadResults;
foreach (ReadResult page in textUrlFileResults)
{
foreach (Line line in page.Lines)
{
Console.WriteLine(line.Text);
}
}
Console.WriteLine();
}
}
}
The above code is taken from the Microsoft Document.
I can be able to read the text inside the image successfully as shown in the below screenshot:

"Access to the path 'C:\\home\\site\\wwwroot\\dataModel.csv' is denied." in a .NET Core app, deployed in Azure

I have an ASP.NET Core API app which runs some background processes via HangFire. One of the process includes
writing a csv file onto the wwwroot folder as following:
public async Task Work(PerformContext context)
{
var latestLikes = await this.likeRepository
.All()
.Select(l => new LatestLikesServiceModel
{
UserId = l.UserId,
BeatId = l.BeatId,
})
.ToListAsync();
var modelPath = this.webHostEnvironment.ContentRootPath + "\\dataModel.csv";
using (var writer = new StreamWriter(modelPath))
using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
csv.WriteRecords(latestLikes);
}
}
In localhost it works perfectly, but when I deploy it in azure the HangFire log returns:
"System.UnauthorizedAccessException","ExceptionMessage":"Access to the path 'C:\home\site\wwwroot\dataModel.csv' is denied."
How can I resolve this issue?
After one week of researching I finally found out the solution
I should have mentioned that I use Azure DevOps for CI & CD.
By default the azure app service is deployed as a zip (which cuts the direct access to the file system) What I had to do was to change the Deployment method to a Web Deploy (Additional Deployment Options) and I finally have access to the file system.
For more descriptive information, please refer:
https://tomasherceg.com/blog/post/azure-app-service-cannot-create-directories-and-write-to-filesystem-when-deployed-using-azure-devops#comments
Below code works fine on my side:
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
namespace FunctionApp97
{
public static class Function1
{
[FunctionName("Function1")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req,
ExecutionContext ec,
ILogger log)
{
string str = ec.FunctionAppDirectory;
log.LogInformation(str);
using (var reader = new StreamReader(str+ #"\filename.txt"))
{
string line;
while ((line = reader.ReadLine()) != null)
{
log.LogInformation(line);
}
}
return new OkObjectResult(str);
}
}
}

How to encrypt xml file in Azure Logicapp?

I have a below requirement
1. XML file need to read from SharePoint and copy into Azure Blob.
2. Need to use Logic app to read the file from share point and copy into Azure Blob.
3. Before copy into Azure Blob need to encrypt the data in Logic app.
Please let me know if any suggestions.
As far as I know there is nothing out the box to help here. You will need to write an Azure Function (or a REST endpoint) to perform the encryption and call that from your Logic App.
There are many examples of such a function, you can find one for PGP here. The core function is copied below and Microsoft's guidance on calling functions from Logic Apps is here.
using System.IO;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using PgpCore;
using System.Threading.Tasks;
using Microsoft.Azure.KeyVault.Models;
using System.Net.Http;
using Microsoft.Azure.Services.AppAuthentication;
using Microsoft.Azure.KeyVault;
using System;
using System.Text;
using System.Collections.Concurrent;
using Microsoft.AspNetCore.Http.Internal;
using Microsoft.Extensions.Logging;
namespace AzureFunctionsPGPEncrypt
{
public static class PGPEncrypt
{
private static readonly HttpClient client = new HttpClient();
private static ConcurrentDictionary<string, string> secrets = new ConcurrentDictionary<string, string>();
[FunctionName(nameof(PGPEncrypt))]
public static async Task<IActionResult> RunAsync(
[HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]
HttpRequest req, ILogger log)
{
log.LogInformation($"C# HTTP trigger function {nameof(PGPEncrypt)} processed a request.");
string publicKeyBase64 = req.Query["public-key"];
string publicKeyEnvironmentVariable = req.Query["public-key-environment-variable"];
string publicKeySecretId = req.Query["public-key-secret-id"];
if (publicKeyBase64 == null && publicKeyEnvironmentVariable == null && publicKeySecretId == null)
{
return new BadRequestObjectResult("Please pass a base64 encoded public key, an environment variable name, or a key vault secret identifier on the query string");
}
if (publicKeyBase64 == null && publicKeyEnvironmentVariable != null)
{
publicKeyBase64 = Environment.GetEnvironmentVariable(publicKeyEnvironmentVariable);
}
if (publicKeyBase64 == null && publicKeySecretId != null)
{
try
{
publicKeyBase64 = await GetPublicKeyAsync(publicKeySecretId);
}
catch (KeyVaultErrorException e) when (e.Body.Error.Code == "SecretNotFound")
{
return new NotFoundResult();
}
catch (KeyVaultErrorException e) when (e.Body.Error.Code == "Forbidden")
{
return new UnauthorizedResult();
}
}
byte[] data = Convert.FromBase64String(publicKeyBase64);
string publicKey = Encoding.UTF8.GetString(data);
req.EnableRewind(); //Make RequestBody Stream seekable
Stream encryptedData = await EncryptAsync(req.Body, publicKey);
return new OkObjectResult(encryptedData);
}
private static async Task<string> GetPublicKeyAsync(string secretIdentifier)
{
if (!secrets.ContainsKey(secretIdentifier))
{
var azureServiceTokenProvider = new AzureServiceTokenProvider();
var authenticationCallback = new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback);
var kvClient = new KeyVaultClient(authenticationCallback, client);
SecretBundle secretBundle = await kvClient.GetSecretAsync(secretIdentifier);
secrets[secretIdentifier] = secretBundle.Value;
}
return secrets[secretIdentifier];
}
private static async Task<Stream> EncryptAsync(Stream inputStream, string publicKey)
{
using (PGP pgp = new PGP())
{
Stream outputStream = new MemoryStream();
using (inputStream)
using (Stream publicKeyStream = GenerateStreamFromString(publicKey))
{
await pgp.EncryptStreamAsync(inputStream, outputStream, publicKeyStream, true, true);
outputStream.Seek(0, SeekOrigin.Begin);
return outputStream;
}
}
}
private static Stream GenerateStreamFromString(string s)
{
MemoryStream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream);
writer.Write(s);
writer.Flush();
stream.Position = 0;
return stream;
}
}
}

How to parse Excel spreadsheets in Azure Logic Apps

I need to parse and extract Column information from an Excel spreadsheet using Azure Logic Apps
I have already setup the ability for my Logic App to retrieve the latest unread Emails from my Outlook. Also, my Logic App does a FOR EACH to read all attachments (from unread emails) and make sure they are Excel files (based on filename extension).
I have a basic Excel file that contains 3 columns "Product, Description, Price" I need to parse each row (only Product and Price) column.
I will add the ability to store that parsed into into my SQL table hosted on Azure.
Here you go, you could use HTTP Request as well.
using ExcelDataReader;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Extensions.Logging;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using Nancy.Json;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Data;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
namespace ConvertExcelToJSon
{
public static class Function1
{
[FunctionName("ConvertToJson")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "post", Route = null)] HttpRequest req,
ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
dynamic data = JsonConvert.DeserializeObject(requestBody);
string blobName = data?.blobName;
string[] splitBlob = blobName.Split('/');
Stream blob = await GetBlobStreamAsync(splitBlob[1], splitBlob[2] + "/" + splitBlob[3]);
DataSet ds = CreateDataSet(blob);
List<Simple> simpleList = new List<Simple>();
foreach (DataTable table in ds.Tables)
{
return (ActionResult)new OkObjectResult(DataTableToJSON(table));
}
return blobName != null
? (ActionResult)new OkObjectResult($"Hello, {blobName}")
: new BadRequestObjectResult("Please pass a name on the query string or in the request body");
}
public static string DataTableToJSON(DataTable table)
{
List<Dictionary<string, object>> list = new List<Dictionary<string, object>>();
foreach (DataRow row in table.Rows)
{
Dictionary<string, object> dict = new Dictionary<string, object>();
foreach (DataColumn col in table.Columns)
{
dict[col.ColumnName] = (Convert.ToString(row[col]));
}
list.Add(dict);
}
JavaScriptSerializer serializer = new JavaScriptSerializer();
return serializer.Serialize(list);
}
public static string AppSetting(this string Key)
{
string ret = string.Empty;
if (Environment.GetEnvironmentVariable(Key) != null)
{
ret = Environment.GetEnvironmentVariable(Key);
}
return ret;
}
public static async Task<Stream> GetBlobStreamAsync(string containerName, string blobName)
{
Stream myBlob = new MemoryStream();
if (CloudStorageAccount.TryParse("AzureWebJobsStorage".AppSetting(), out CloudStorageAccount storageAccount))
{
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = cloudBlobClient.GetContainerReference(containerName);
CloudBlob myBloab = container.GetBlobReference(blobName);
await myBloab.DownloadToStreamAsync(myBlob);
myBlob.Seek(0, SeekOrigin.Begin);
}
return myBlob;
}
public static DataSet CreateDataSet(Stream stream)
{
DataSet ds;
System.Text.Encoding.RegisterProvider(System.Text.CodePagesEncodingProvider.Instance);
IExcelDataReader reader = null;
reader = ExcelReaderFactory.CreateOpenXmlReader(stream);
ds = reader.AsDataSet(new ExcelDataSetConfiguration()
{
ConfigureDataTable = (tableReader) => new ExcelDataTableConfiguration()
{
UseHeaderRow = true,
}
});
return ds;
}
}
}
I suggest you call an Azure Function from your logic App and use the Function to convert the Excel into a JSON Object. (I am currently doing this very succesfully) I use ExcelDataReader parse to parse a Blob that the Logic App creates. Send the blob location in the request and respond back with the JSON.

A namespace cannot directly contain members in the Azure Function App

TARGET: Do the Azure Function tutorial on and copied code, but got several errors when executing locally on VS2017. I appreciate you help.
https://www.cyotek.com/blog/upload-data-to-blob-storage-with-azure-functions
ERROR 1 - related to Run:
CS0116 A namespace cannot directly contain members such as fields or methods UploadToBlobFunctionApp C:\AzureFunctions\UploadToBlobFunctionApp\UploadToBlobFunctionApp\UploadToBlobFunction.cs 15 Active
ERROR 2 - related to Task CreateBlob:
CS0116 A namespace cannot directly contain members such as fields or methods UploadToBlobFunctionApp
C:\AzureFunctions\UploadToBlobFunctionApp\UploadToBlobFunctionApp\UploadToBlobFunction.cs 45 Active
ERROR 3 - related to await CreateBlob:
CS0103 The name 'CreateBlob' does not exist in the current context UploadToBlobFunctionApp C:\AzureFunctions\UploadToBlobFunctionApp\UploadToBlobFunctionApp\UploadToBlobFunction.cs 36 Active
CODE Function.cs:
using System;
using System.Configuration;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Azure;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
HttpStatusCode result;
string contentType;
result = HttpStatusCode.BadRequest;
contentType = req.Content.Headers?.ContentType?.MediaType;
if (contentType == "application/json")
{
string body;
body = await req.Content.ReadAsStringAsync();
if (!string.IsNullOrEmpty(body))
{
string name;
name = Guid.NewGuid().ToString("n");
await CreateBlob(name + ".json", body, log);
result = HttpStatusCode.OK;
}
}
return req.CreateResponse(result, string.Empty);
}
private async static Task CreateBlob(string name, string data,
TraceWriter log)
{
string accessKey;
string accountName;
string connectionString;
CloudStorageAccount storageAccount;
CloudBlobClient client;
CloudBlobContainer container;
CloudBlockBlob blob;
accessKey = "qwertyw4VhRajxlZn9C4hTMB8oSwE4klNUsvTy9VeTCIQ11111vFVVGExDwJ+JUboFv2B79j+W6foqLWE92w==";
accountName = "mystorage";
connectionString = "DefaultEndpointsProtocol=https;AccountName=" + accountName + ";AccountKey=" + accessKey + ";EndpointSuffix=core.windows.net";
storageAccount = CloudStorageAccount.Parse(connectionString);
client = storageAccount.CreateCloudBlobClient();
container = client.GetContainerReference("functionupload");
await container.CreateIfNotExistsAsync();
blob = container.GetBlockBlobReference(name);
blob.Properties.ContentType = "application/json";
using (Stream stream = new MemoryStream(Encoding.UTF8.GetBytes(data)))
{
await blob.UploadFromStreamAsync(stream);
}
}
The example that you are referencing is using scripted functions (csx file). They are mostly used while editing code directly in Azure portal.
I think you are trying to create a precompiled application with csproj and cs files. In this case, your code should be a valid C#, i.e. all methods should be inside classes.
Have a look at this example.
You can also use attributes to mark your functions and triggers instead of authoring function.json manually, see examples here.

Resources