Azure DocumentDB, When uploading af executing - no result? - azure

In my project I am supposed to get data from openweathermap.org and put that in a collection in my DocumentDB database in Azure.
The code below works locally on my development machine, but when i upload the project, it runs and succeed (says the dashboard) but no documents are created. I can only create the documents if I run from local machine.
Why is that?
Here is my code:
public static void Main()
{
JobHost host = new JobHost();
// The following code ensures that the WebJob will be running continuously
host.Call(typeof(Program).GetMethod("saveWeatherDataToAzureDocumentDB"));
}
[NoAutomaticTrigger]
public static async void saveWeatherDataToAzureDocumentDB()
{
string endpointUrl = ConfigurationManager.AppSettings["EndPointUrl"];
string authorizationKey = ConfigurationManager.AppSettings["AuthorizationKey"];
string url = "http://api.openweathermap.org/data/2.5/weather?q=hanstholm,dk&appid=44db6a862fba0b067b1930da0d769e98";
var request = WebRequest.Create(url);
string text;
var response = (HttpWebResponse)request.GetResponse();
using (var sr = new StreamReader(response.GetResponseStream()))
{
text = sr.ReadToEnd();
}
// Create a new instance of the DocumentClient
var client = new DocumentClient(new Uri(endpointUrl), authorizationKey);
// Check to verify a database with the id=FamilyRegistry does not exist
Database database = client.CreateDatabaseQuery().Where(db => db.Id == "weatherdata").AsEnumerable().FirstOrDefault();
// If the database does not exist, create a new database
if (database == null)
{
database = await client.CreateDatabaseAsync(
new Database
{
Id = "weatherdata"
});
}
// Check to verify a document collection with the id=FamilyCollection does not exist
DocumentCollection documentCollection = client.CreateDocumentCollectionQuery(database.SelfLink).Where(c => c.Id == "weathercollection").AsEnumerable().FirstOrDefault();
// If the document collection does not exist, create a new collection
if (documentCollection == null)
{
documentCollection = await client.CreateDocumentCollectionAsync("dbs/" + database.Id,
new DocumentCollection
{
Id = "weathercollection"
});
}
//Deserialiser til et dynamisk object
if (text == "")
{
mark m = new mark() { name = "Something" };
await client.CreateDocumentAsync(documentCollection.DocumentsLink, m);
}
else
{
var json = JsonConvert.DeserializeObject<dynamic>(text);
json["id"] = json["name"] + "_" + DateTime.Now;
await client.CreateDocumentAsync(documentCollection.DocumentsLink, json);
}
}
public sealed class mark
{
public string name { get; set; }
}
UPDATE - This is what I have in my App.config
<appSettings>
<!-- Replace the value with the value you copied from the Azure management portal -->
<add key="EndPointUrl" value="https://<My account>.documents.azure.com:443/"/>
<!-- Replace the value with the value you copied from the Azure management portal -->
<add key="AuthorizationKey" value="The secret code from Azure"/>
Also, At DocumentDB Account i find the Connection string like this. AccountEndpoint=https://knoerregaard.documents.azure.com:443/;AccountKey=my secret password
How should I apply this to the WebJob?
Appriciate your help!

Related

How to mock Azure blobContainerClient.GetBlobsAsync()

I have a Azure blob container which I am accessing using below code -
var blobContainerClient = GetBlobContainer(containerName);
if (blobContainerClient != null)
{
// List all blobs in the container
await foreach (BlobItem blobItem in blobContainerClient.GetBlobsAsync())
{
queuedBlobsList.Add(new QueuedBlobs { BlobName = blobItem.Name, LastModified = blobItem.Properties.LastModified });
}
}
private BlobContainerClient GetBlobContainer(string containerName)
{
return gen2StorageClient != null
? gen2StorageClient.GetBlobContainerClient(containerName)
: gen1StorageClient.GetBlobContainerClient(containerName);
}
The clients are initialised in constructor -
public class BlobService : IBlobService
{
private readonly BlobServiceClient gen1StorageClient, gen2StorageClient;
public BlobService(BlobServiceClient defaultClient, IAzureClientFactory<BlobServiceClient> clientFactory)
{
gen1StorageClient = defaultClient;
if (clientFactory != null)
{
gen2StorageClient = clientFactory.CreateClient("StorageConnectionString");
}
}
}
And my unit test where I am setting GetBlobsAsync is like this -
But I want to add list of BlobItems to test another loop.
private static Mock<BlobContainerClient> GetBlobContainerClientMockWithListOfBlobs()
{
var blobContainerClientMock = new Mock<BlobContainerClient>("UseDevelopmentStorage=true", EnvironmentConstants.ParallelUploadContainer);
var cancellationToken = new CancellationToken();
var blobs = new List<BlobItem>();
//AsyncPageable<BlobItem> blobItems = new AsyncPageable<BlobItem>(); -- Not allowing
blobContainerClientMock.Setup(x => x.GetBlobsAsync(BlobTraits.All, BlobStates.All, null, cancellationToken)).Returns(It.IsAny<AsyncPageable<BlobItem>>());
return blobContainerClientMock;
}
I came to this question because I also had the same issue.
Based on this article
AsyncPageable<T> and Pageable<T> are classes that represent collections of models returned by the service in pages.
The method GetBlobsAsync returns an AsyncPageable.
To Create an AsyncPageable you need first to create a BlobItem Page.
To create a Page<T> instance, use the Page<T>.FromValues method, passing a list of items, a continuation token, and the Response.
So let's start creating the list of items:
var blobList = new BlobItem[]
{
BlobsModelFactory.BlobItem("Blob1"),
BlobsModelFactory.BlobItem("Blob2"),
BlobsModelFactory.BlobItem("Blob3")
};
Note: BlobItem has an internal constructor but I found in this answer that there's a BlobsModelFactory.
After having the list of blobs is time to create a Page<BlobItem>:
Page<BlobItem> page = Page<BlobItem>.FromValues(blobList, null, Mock.Of<Response>());
And finally, create the AsyncPageable<BlobItem>
AsyncPageable<BlobItem> pageableBlobList = AsyncPageable<BlobItem>.FromPages(new[] { page });
And now you are able to use this to mock GetBlobsAsync method:
blobContainerClientMock
.Setup(m => m.GetBlobsAsync(
It.IsAny<BlobTraits>(),
It.IsAny<BlobStates>(),
It.IsAny<string>(),
It.IsAny<CancellationToken>()))
.Returns(pageableBlobList);
I hope this helps others with this issue.
André

Azure SignalR Service Connection string

Trying to create a POC for azure signalr service. I found the github samples, which appeared to have some solid examples. I chose this one. Basically, my problem is that when I run the code locally, works like a champ with the localhost url, but when I try to run using an Azure SignalR Service using a url that I copied from azure portal keys which is in this format: Endpoint=<service_endpoint>;AccessKey=<access_key>;, I get an error stating that "Invalid URI: The URI scheme is not valid.". How do I transform the url from what I copy from keys and use it to connect to a signalr service?
class Program
{
private const string DefaultHubEndpoint = "Endpoint=http://someFakesrsname.service.signlar.net;AccsssKey=thisseemslikeagoodaccesskeytouseformyquestion";//"http://localhost:5000/ManagementSampleHub";
private const string Target = "Target";
private const string DefaultUser = "User";
static void Main(string[] args)
{
var app = new CommandLineApplication();
app.FullName = "Azure SignalR Management Sample: SignalR Client Tool";
app.HelpOption("--help");
var hubEndpointOption = app.Option("-h|--hubEndpoint", $"Set hub endpoint. Default value: {DefaultHubEndpoint}", CommandOptionType.SingleValue, true);
var userIdOption = app.Option("-u|--userIdList", "Set user ID list", CommandOptionType.MultipleValue, true);
app.OnExecute(async () =>
{
var hubEndpoint = hubEndpointOption.Value() ?? DefaultHubEndpoint;
var userIds = userIdOption.Values != null && userIdOption.Values.Count > 0 ? userIdOption.Values : new List<string>() { "User" };
Console.WriteLine("hubEndpoint: " + hubEndpoint);
Console.WriteLine("DefaultHubEndpoint: " + DefaultHubEndpoint);
foreach (var userId in userIds)
{
Console.WriteLine("UserId: " + userId);
}
var connections = (from userId in userIds
select CreateHubConnection(hubEndpoint, userId)).ToList();
await Task.WhenAll(from conn in connections
select conn.StartAsync());
Console.WriteLine($"{connections.Count} Client(s) started...");
Console.ReadLine();
await Task.WhenAll(from conn in connections
select conn.StopAsync());
return 0;
});
app.Execute(args);
}
static HubConnection CreateHubConnection(string hubEndpoint, string userId)
{
var url = hubEndpoint.TrimEnd('/') + $"?user={userId}";
var connection = new HubConnectionBuilder().WithUrl(url).Build();
connection.On(Target, (string message) =>
{
Console.WriteLine($"{userId}: gets message from service: '{message}'");
});
connection.Closed += async ex =>
{
Console.WriteLine(ex);
Environment.Exit(1);
};
return connection;
}
}
enter code here

Azure Graph API list users in child groups

I have a group called "Building Residents". In that group I have 2 groups for every apartment:
- Apt1a_Renters
- Apt1a_Owners
- Apt2a_Renters
- Apt2a_Owners
etc, etc...
A Person can be in the Apt2a_Owners group but also in the Apt1a_Renters group.
I want to query all members of the "Building Residents" group for users but when I do that, I just get back the groups inside of it.
Is there any way, using a single query, to list all the building residents without bringing back duplicates (same person in apt2a_owners and apt1a_renters)?
The Azure Graph REST only can get the direct members from one group. To get the member users recursively, we need to implement it ourselves. Here is a code sample wiring with C# using the Azure AD Graph library for your reference:
public void testGetMembersRecursively()
{
var accessToken="";
var tenantId="";
var groupName="";
var client = GraphHelper.CreateGraphClient(accessToken,tenantId);
var group = (Group)client.Groups.ExecuteAsync().Result.CurrentPage.First(g => g.DisplayName ==groupName);
var groupFetcher = client.Groups.GetByObjectId(group.ObjectId);
List<string> users = new List<string>();
GetMembersRecursively(groupFetcher, users);
Console.WriteLine(String.Join("\n", users.Distinct<string>().ToArray()));
}
public void GetMembersRecursively(Microsoft.Azure.ActiveDirectory.GraphClient.IGroupFetcher groupFetcher, List<string> users)
{
var membersResoult = groupFetcher.Members.ExecuteAsync().Result;
AddMember( membersResoult, users);
while (membersResoult.MorePagesAvailable)
{
membersResoult = membersResoult.GetNextPageAsync().Result;
AddMember(membersResoult, users);
}
}
public void AddMember( IPagedCollection<IDirectoryObject> membersResoult, List<string> users)
{
var members = membersResoult.CurrentPage;
foreach (var obj in members)
{
var _user = obj as Microsoft.Azure.ActiveDirectory.GraphClient.User;
if (_user != null)
users.Add(_user.DisplayName);
else
{
var groupMember = obj as Microsoft.Azure.ActiveDirectory.GraphClient.Group;
if (groupMember != null)
{
GetMembersRecursively(GetGroupById(groupMember.ObjectId), users);
}
}
}
}
GraphHelper class:
class GraphHelper
{
public static ActiveDirectoryClient CreateGraphClient(string accessToken, string tenantId)
{
string graphResourceId = "https://graph.windows.net";
Uri servicePointUri = new Uri(graphResourceId);
Uri serviceRoot = new Uri(servicePointUri, tenantId);
ActiveDirectoryClient activeDirectoryClient = new ActiveDirectoryClient(serviceRoot, async () => await Task.FromResult(accessToken));
return activeDirectoryClient;
}
}

Uploading ID to same Azure Blob Index that a file is being uploaded to using .Net SDK

I am uploading documents to my an Azure Blob Storage, which works perfect, but I want to be able to link and ID to this specifically uploaded document.
Below is my code for uploading the file:
[HttpPost]
public ActionResult Upload(HttpPostedFileBase file)
{
try
{
var path = Path.Combine(Server.MapPath("~/App_Data/Uploads"), file.FileName);
string searchServiceName = ConfigurationManager.AppSettings["SearchServiceName"];
string blobStorageKey = ConfigurationManager.AppSettings["BlobStorageKey"];
string blobStorageName = ConfigurationManager.AppSettings["BlobStorageName"];
string blobStorageURL = ConfigurationManager.AppSettings["BlobStorageURL"];
file.SaveAs(path);
var credentials = new StorageCredentials(searchServiceName, blobStorageKey);
var client = new CloudBlobClient(new Uri(blobStorageURL), credentials);
// Retrieve a reference to a container. (You need to create one using the mangement portal, or call container.CreateIfNotExists())
var container = client.GetContainerReference(blobStorageName);
// Retrieve reference to a blob named "myfile.gif".
var blockBlob = container.GetBlockBlobReference(file.FileName);
// Create or overwrite the "myblob" blob with contents from a local file.
using (var fileStream = System.IO.File.OpenRead(path))
{
blockBlob.UploadFromStream(fileStream);
}
System.IO.File.Delete(path);
return new JsonResult
{
JsonRequestBehavior = JsonRequestBehavior.AllowGet,
Data = "Success"
};
}
catch (Exception ex)
{
throw;
}
}
I have added the ClientID field to the Index(It is at the bottom), but have no idea how I am able to add this to this index. This is still al nerw to me and just need a little guidance if someone can help :
Thanks in advance.

Azure Automation Managment not able to execute run book

I am using Azure Automation Management 2.0.1. I am not able find Start method for Runbooks to execute a runbook. How do this with 2.0.1
var client = new Microsoft.Azure.Management.Automation.AutomationManagementClient(new CertificateCloudCredentials(subscriptionId, cert));
var ct = new CancellationToken();
var content = await client.Runbooks.ListByNameAsync("MyAutomationAccountName", "MyRunbookName", ct);
var firstOrDefault = content?.Runbooks.FirstOrDefault();
if (firstOrDefault != null)
{
var operation = client.Runbooks.Start("MyAutomationAccountName", new RunbookStartParameters(firstOrDefault.Id));
}
You need to use the automationManagementClient.Jobs.Create
public static JobCreateResponse Create(
this IJobOperations operations,
string resourceGroupName,
string automationAccount,
JobCreateParameters parameters
)
You can find a full sample here this would be the relevant part -
private void JobStart_Click(object sender, RoutedEventArgs e)
{
// Check for runbook name
if (String.IsNullOrWhiteSpace(RunbookName.Text) || String.IsNullOrWhiteSpace(PublishState.Text)) throw new ArgumentNullException(RunbookName.Text);
// Create job create parameters
var jcparam = new JobCreateParameters
{
Properties = new JobCreateProperties
{
Runbook = new RunbookAssociationProperty
{
// associate the runbook name
Name = RunbookName.Text
},
// pass parameters to runbook if any
Parameters = null
}
};
// create runbook job. This gives back JobId
var job = automationManagementClient.Jobs.Create(this.automationAccountName, jcparam).Job;
JobGuid.Text = JobId.Text = job.Properties.JobId.ToString();
Log.Text += (String.Format("\nJob Started for Runbook {0} , JobId {1}", RunbookName.Text, JobId.Text));
}

Resources