I would like to use python in azure function to read these excel files in this blob storage. i would like to know how to access this folder of blob in the easiest way and the python code to read the excel files.
enter image description here
The easiest way would be to use input bindings
in your function.json add bindings
{
"name": "inputblob",
"type": "blob",
"dataType": "binary",
"path": "blob/path/filename.csv",
"connection": "MyStorageConnectionAppSetting",
"direction": "in"
},
Then function
import logging
import azure.functions as func
# The input binding field inputblob can either be 'bytes' or 'str' depends
# on dataType in function.json, 'binary' or 'string'.
def main(queuemsg: func.QueueMessage, inputblob: bytes) -> bytes:
logging.info(f'Python Queue trigger function processed {len(inputblob)} bytes')
return inputblob
Related
I'm new to the Azure environment. I want to execute an .exe file by passing blob storage as an input and output parameter
below is my Json Function and PowerShell code I am not sure how to pass the input blob storage file as parameter and get the output on the blob storage.
I tried to run the below code but i am getting error
Function.json
{
"bindings": [
{
"name": "Timer",
"type": "timerTrigger",
"direction": "in",
"schedule": "0 */5 * * * *"
},
{
"name": "inputBlob",
"direction": "in",
"type": "blob",
"path": "inputfilepath/{name}",
"connection": "test23_STORAGE"
},
{
"name": "outputBlob",
"direction": "out",
"type": "blob",
"path": "outputfilepath/{name}",
"connection": " test23_STORAGE"
}
]
}
Run.ps1
# Input bindings are passed in via param block.
param($Timer,$inputBlob,$outputBlob)
# Get the current universal time in the default string format.
$currentUTCtime = (Get-Date).ToUniversalTime()
# The 'IsPastDue' property is 'true' when the current function invocation is later than scheduled.
if ($Timer.IsPastDue) {
Write-Host "PowerShell timer is running late!"
}
# Write an information log with the current time.
Write-Host "PowerShell timer trigger function ran! TIME: $currentUTCtime"
Set-Location "C:\home\site\wwwroot\TimerTrigger1"
.\Extractor.exe -F $inputBlob -E $outputBlob
Error
2022-09-10T07:09:45Z [Information] Executing 'Functions.TimerTrigger1' (Reason='This function was programmatically called via the host APIs.', Id=07f70ccc-3e39-4e98-a830-73bfda54d101)
2022-09-10T07:09:45Z [Error] Executed 'Functions.TimerTrigger1' (Failed, Id=07f70ccc-3e39-4e98-a830-73bfda54d101, Duration=7ms)
AFAIK, we cannot use multiple input triggers, but we can use multiple input/output binding of the same trigger in one trigger function.
I want to execute an .exe file by passing blob storage as an input and output parameter
This might be done in 2 ways either fetching that .exe file from the Storage blob container path and running it or using the WebJobs with CRON Tab expression for these kinds of tasks.
From the Azure Functions, define one function for triggering the .exe file at runtime to run and another function for storing the output of that .exe application in the blob storage.
I have setup a very simple azure function that should be triggered whenever an excel file is dropped into my bucket. However, nothing is happening. Any suggestions?
Here is my functions.json
{
"scriptFile": "__init__.py",
"bindings": [
{
"name": "myblob",
"type": "blobTrigger",
"direction": "in",
"path": "excel/{name}.xlsx",
"connection": "jbucket123_STORAGE"
}
]
}
Here is my init.py file. Any recommendations?
import logging
import pandas as pd
import azure.functions as func
def main(myblob: func.InputStream):
logging.info(f"Python blob trigger function processed blob \n"
f"Name: {myblob.name}\n"
f"Blob Size: {myblob.length} bytes")
df=pd.read_excel(myblob.read())
logging.info(f"{df}")
Your configuration looks correct, a few things to doublecheck:
Do you use a "general-purpose" storage account? Blob triggers are not supported for the legacy "BlobStorage" account types.
Does a container "excel" exist in your storage account and are you putting your file into it? Does your file end with .xlsx (and not e.g. .xls)?
If you test locally is the storage account connection string stored under local.settings.json like so:
{
"IsEncrypted": false,
"Values": {
"FUNCTIONS_WORKER_RUNTIME": "python",
"AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=[...]",
"jbucket123_STORAGE": "DefaultEndpointsProtocol=https;AccountName=[...]"
}
}
If you run the function in Azure, is the "jbucket123_STORAGE" property set under "Configuration > Application settings"? Did you put all dependencies (e.g. pandas, openpyxl) into your requirements.txt before you published the function?
You might also want to check the "Log stream" of your function to get more details what your function is doing.
I'm trying to create a Blob trigger Azure Function in Python that automatically split all sheets in a specific Excel file into separate .csv files onto the same Azure Blob container. My init.py and function.json files look something like this:
function.json file:
{
"scriptFile": "__init__.py",
"bindings": [
{
"name": "blobin",
"type": "blobTrigger",
"direction": "in",
"path": "folder/ExcelFile.xlsx",
"connection": "blobSTORAGE"
},
{
"name": "blobout",
"type": "blob",
"direction": "out",
"path": "folder/{name}",
"connection": "blobSTORAGE"
}
]
}
init.py file:
import logging
from xlrd import open_workbook
import csv
import azure.functions as func
def main(blobin: func.InputStream, blobout: func.Out[bytes]):
logging.info(f"Python blob trigger function processed blob \n")
try:
wb = open_workbook('ExcelFile.xlsx')
for i in range(0, wb.nsheets):
sheet = wb.sheet_by_index(i)
print(sheet.name)
with open("Sheet_%s.csv" %(sheet.name.replace(" ","")), "w") as file:
writer = csv.writer(file, delimiter = ",")
print(sheet, sheet.name, sheet.ncols, sheet.nrows)
header = [cell.value for cell in sheet.row(0)]
writer.writerow(header)
for row_idx in range(1, sheet.nrows):
row = [int(cell.value) if isinstance(cell.value, float) else cell.value
for cell in sheet.row(row_idx)]
writer.writerow(row)
blobout.set(file.read)
logging.info(f"Split sheet %s into CSV successfully.\n" %(sheet.name))
except:
print("Error")
I tried to run the pure Python code on my PC without the Azure function implementing and succeeded. However, when I deployed the function onto Azure Apps it does not "trigger" when I tried to upload the Excel file. I am thinking that the config I put is wrong but don't know how to confirm or fix it. Any suggestion?
Since you don't have problem on local, I think the problems comes from the blobSTORAGE that doesn't have setted in the environment variable.
On local, environment variable is setted in the values section in local.settings.json. But when you deploy the function app, it will get environment variable from this place:
I have a activity function that should store message in Blob storage.I can overwrite a file in blob storage but i need to store data in different name.how to do that? Azure function doesn't support dynamic binding in nodejs.
Find one workaround, see whether it's useful.
Along with blob output binding, there's an activity trigger to receive message msg, we can put self-defined blob name in msg for blob binding path to consume.
In your orchestrator function which calls Activity function
yield context.df.callActivity("YourActivity", {'body':'messagecontent','blobName':'myblob'});
Then Activity function code should be modified
context.bindings.myOutputBlob = context.bindings.msg.body;
And its function.json can use blobName as expected
{
"bindings": [
{
"name": "msg",
"type": "activityTrigger",
"direction": "in"
},
{
"name":"myOutputBlob",
"direction": "out",
"type": "blob",
"connection": "AzureWebJobsStorage",
"path": "azureblob/{blobName}"
}
],
"disabled": false
}
My flow is very simple: I want to have an azure function that runs once a day and they use its output to create a file in Dropbox.
The function does some processing and returns an object with 2 properties, a FileName and a FileContent, both are strings.:
return new AzureFunctionResponse
{
FileName = $"TestFile-{DateTimeOffset.UtcNow.ToUnixTimeMilliseconds()}",
FileContent = "This is the file content"
};
My problem is that I don't know how to use those 2 properties to setup my Dropbox connector
Here's my LogicApp flow:
I'd like to use the FileName and FileContent returned from my AzureFunction to populate the respective field in the Dropbox connector but I have no idea how to set this up. I've looked for documentation, but maybe I'm not looking at the right place because I'm not finding anything.
Also here are the bindings in my function.json file, if that can be of any help.
{
"disabled": false,
"bindings": [
{
"type": "httpTrigger",
"direction": "in",
"webHookType": "genericJson",
"name": "req"
},
{
"type": "http",
"direction": "out",
"name": "res"
}
}
Using the Parse JSON action after the function should do exactly what you need. Will parse output and make them available for you in the next step.
As alternative, you can implement the whole thing without using Logic Apps.
Make an Azure Function with Timer input trigger and Api Hub File output binding. No HTTP bindings are needed.
See this question for an example.