How to mock Azure blobContainerClient.GetBlobsAsync() - azure

I have a Azure blob container which I am accessing using below code -
var blobContainerClient = GetBlobContainer(containerName);
if (blobContainerClient != null)
{
// List all blobs in the container
await foreach (BlobItem blobItem in blobContainerClient.GetBlobsAsync())
{
queuedBlobsList.Add(new QueuedBlobs { BlobName = blobItem.Name, LastModified = blobItem.Properties.LastModified });
}
}
private BlobContainerClient GetBlobContainer(string containerName)
{
return gen2StorageClient != null
? gen2StorageClient.GetBlobContainerClient(containerName)
: gen1StorageClient.GetBlobContainerClient(containerName);
}
The clients are initialised in constructor -
public class BlobService : IBlobService
{
private readonly BlobServiceClient gen1StorageClient, gen2StorageClient;
public BlobService(BlobServiceClient defaultClient, IAzureClientFactory<BlobServiceClient> clientFactory)
{
gen1StorageClient = defaultClient;
if (clientFactory != null)
{
gen2StorageClient = clientFactory.CreateClient("StorageConnectionString");
}
}
}
And my unit test where I am setting GetBlobsAsync is like this -
But I want to add list of BlobItems to test another loop.
private static Mock<BlobContainerClient> GetBlobContainerClientMockWithListOfBlobs()
{
var blobContainerClientMock = new Mock<BlobContainerClient>("UseDevelopmentStorage=true", EnvironmentConstants.ParallelUploadContainer);
var cancellationToken = new CancellationToken();
var blobs = new List<BlobItem>();
//AsyncPageable<BlobItem> blobItems = new AsyncPageable<BlobItem>(); -- Not allowing
blobContainerClientMock.Setup(x => x.GetBlobsAsync(BlobTraits.All, BlobStates.All, null, cancellationToken)).Returns(It.IsAny<AsyncPageable<BlobItem>>());
return blobContainerClientMock;
}

I came to this question because I also had the same issue.
Based on this article
AsyncPageable<T> and Pageable<T> are classes that represent collections of models returned by the service in pages.
The method GetBlobsAsync returns an AsyncPageable.
To Create an AsyncPageable you need first to create a BlobItem Page.
To create a Page<T> instance, use the Page<T>.FromValues method, passing a list of items, a continuation token, and the Response.
So let's start creating the list of items:
var blobList = new BlobItem[]
{
BlobsModelFactory.BlobItem("Blob1"),
BlobsModelFactory.BlobItem("Blob2"),
BlobsModelFactory.BlobItem("Blob3")
};
Note: BlobItem has an internal constructor but I found in this answer that there's a BlobsModelFactory.
After having the list of blobs is time to create a Page<BlobItem>:
Page<BlobItem> page = Page<BlobItem>.FromValues(blobList, null, Mock.Of<Response>());
And finally, create the AsyncPageable<BlobItem>
AsyncPageable<BlobItem> pageableBlobList = AsyncPageable<BlobItem>.FromPages(new[] { page });
And now you are able to use this to mock GetBlobsAsync method:
blobContainerClientMock
.Setup(m => m.GetBlobsAsync(
It.IsAny<BlobTraits>(),
It.IsAny<BlobStates>(),
It.IsAny<string>(),
It.IsAny<CancellationToken>()))
.Returns(pageableBlobList);
I hope this helps others with this issue.
André

Related

Program terminates when calling WorkItemTrackingHttpClient.QueryByWiqlAsyc()

I am working on a program that gets a list of workitems in the committed state from Azure DevOps for a specific area path and iteration path. My code is based on an example found at the following link: https://learn.microsoft.com/en-us/azure/devops/integrate/quickstarts/work-item-quickstart?view=azure-devops
The issue I am running into is when QueryByWiqlAsync() is called, the program terminates and there are no errors for why it terminated. Below is the code in question. I tried calling QueryByWiqlAsync() with and without the ConfigureAwait(false) and that did not seem to make a difference. Any suggestions on what to try or what to fix are appreciated.
static async void GetWorkItemsToTaskFromADO(string tfs_project, string accessToken)
{
var credentials = new VssBasicCredential(string.Empty, accessToken);
var wiql = new Wiql()
{
Query = #"Select [Id] From WorkItems WHERE [System.TeamProject] = 'SampleADOProject' AND [System.AreaPath] = 'Sample\ADO\AreaPath' AND [System.IterationPath] = 'Sample\ADO\IterationPath' AND [System.State] = 'Committed'"
};
using (var httpClient = new WorkItemTrackingHttpClient(new Uri(tfs_project), credentials))
{
try
{
var result = await httpClient.QueryByWiqlAsync(wiql).ConfigureAwait(false);
var ids = result.WorkItems.Select(item => item.Id).ToArray();
var fields = new[] { "System.Id", "System.Title", "System.State" };
var workItems = await httpClient.GetWorkItemsAsync(ids, fields, result.AsOf).ConfigureAwait(false);
// output results to test what came back...
foreach (var workItem in workItems)
{
Console.WriteLine(
"{0}\t{1}\t{2}",
workItem.Id,
workItem.Fields["System.Title"],
workItem.Fields["System.State"]
);
}
}
catch(Exception ex)
{
Console.WriteLine(ex.Message);
Console.Read();
}
}
}

Storage quota has been exceeded for this service. You must either delete documents first, or use a higher SKU for additional quota

In azure search getting error sometimes not every time when I'm create or update index in azure.
and I'm tying post multiple data as per guideline How to index large data sets in Azure Search
is there any way to update existing data in azure? How to rebuild an index
Some azure Available tiers Available tiers
Code this simple example
private static SearchServiceClient searchServiceClient;
public virtual void CreateSearchServiceClient()
{
searchServiceClient = new SearchServiceClient(searchServiceName, new SearchCredentials("APIKEY"));
}
public virtual Index CreateIndex()
{
CreateSearchServiceClient();
Index index = new Index();
string scoringProfile = "AzureSearchScoringProfile";
// Create index if search client service is available
if (searchServiceClient != null)
{
AzureSearchItem azureSearchItem = new AzureSearchItem();
// Map index schema
var definition = new Index()
{
Name = IndexName,
Fields = FieldBuilder.BuildForType<AzureSearchItem>(),
Suggesters = new List<Suggester>() {
new Suggester()
{
Name = ConstantKeys.AzureSearchTopBarSuggestor,
SourceFields = FormatSuggesterFields()
}
}
};
if (!string.IsNullOrEmpty(scoringProfile))
{
definition.ScoringProfiles = new List<ScoringProfile>()
{
new ScoringProfile()
{
Name = scoringProfile
}
};
}
definition = SetAnalyzer(definition);
// Create index
index = searchServiceClient.Indexes.CreateOrUpdate(definition);
}
return index;
}

Azure Graph API list users in child groups

I have a group called "Building Residents". In that group I have 2 groups for every apartment:
- Apt1a_Renters
- Apt1a_Owners
- Apt2a_Renters
- Apt2a_Owners
etc, etc...
A Person can be in the Apt2a_Owners group but also in the Apt1a_Renters group.
I want to query all members of the "Building Residents" group for users but when I do that, I just get back the groups inside of it.
Is there any way, using a single query, to list all the building residents without bringing back duplicates (same person in apt2a_owners and apt1a_renters)?
The Azure Graph REST only can get the direct members from one group. To get the member users recursively, we need to implement it ourselves. Here is a code sample wiring with C# using the Azure AD Graph library for your reference:
public void testGetMembersRecursively()
{
var accessToken="";
var tenantId="";
var groupName="";
var client = GraphHelper.CreateGraphClient(accessToken,tenantId);
var group = (Group)client.Groups.ExecuteAsync().Result.CurrentPage.First(g => g.DisplayName ==groupName);
var groupFetcher = client.Groups.GetByObjectId(group.ObjectId);
List<string> users = new List<string>();
GetMembersRecursively(groupFetcher, users);
Console.WriteLine(String.Join("\n", users.Distinct<string>().ToArray()));
}
public void GetMembersRecursively(Microsoft.Azure.ActiveDirectory.GraphClient.IGroupFetcher groupFetcher, List<string> users)
{
var membersResoult = groupFetcher.Members.ExecuteAsync().Result;
AddMember( membersResoult, users);
while (membersResoult.MorePagesAvailable)
{
membersResoult = membersResoult.GetNextPageAsync().Result;
AddMember(membersResoult, users);
}
}
public void AddMember( IPagedCollection<IDirectoryObject> membersResoult, List<string> users)
{
var members = membersResoult.CurrentPage;
foreach (var obj in members)
{
var _user = obj as Microsoft.Azure.ActiveDirectory.GraphClient.User;
if (_user != null)
users.Add(_user.DisplayName);
else
{
var groupMember = obj as Microsoft.Azure.ActiveDirectory.GraphClient.Group;
if (groupMember != null)
{
GetMembersRecursively(GetGroupById(groupMember.ObjectId), users);
}
}
}
}
GraphHelper class:
class GraphHelper
{
public static ActiveDirectoryClient CreateGraphClient(string accessToken, string tenantId)
{
string graphResourceId = "https://graph.windows.net";
Uri servicePointUri = new Uri(graphResourceId);
Uri serviceRoot = new Uri(servicePointUri, tenantId);
ActiveDirectoryClient activeDirectoryClient = new ActiveDirectoryClient(serviceRoot, async () => await Task.FromResult(accessToken));
return activeDirectoryClient;
}
}

Azure DocumentDB, When uploading af executing - no result?

In my project I am supposed to get data from openweathermap.org and put that in a collection in my DocumentDB database in Azure.
The code below works locally on my development machine, but when i upload the project, it runs and succeed (says the dashboard) but no documents are created. I can only create the documents if I run from local machine.
Why is that?
Here is my code:
public static void Main()
{
JobHost host = new JobHost();
// The following code ensures that the WebJob will be running continuously
host.Call(typeof(Program).GetMethod("saveWeatherDataToAzureDocumentDB"));
}
[NoAutomaticTrigger]
public static async void saveWeatherDataToAzureDocumentDB()
{
string endpointUrl = ConfigurationManager.AppSettings["EndPointUrl"];
string authorizationKey = ConfigurationManager.AppSettings["AuthorizationKey"];
string url = "http://api.openweathermap.org/data/2.5/weather?q=hanstholm,dk&appid=44db6a862fba0b067b1930da0d769e98";
var request = WebRequest.Create(url);
string text;
var response = (HttpWebResponse)request.GetResponse();
using (var sr = new StreamReader(response.GetResponseStream()))
{
text = sr.ReadToEnd();
}
// Create a new instance of the DocumentClient
var client = new DocumentClient(new Uri(endpointUrl), authorizationKey);
// Check to verify a database with the id=FamilyRegistry does not exist
Database database = client.CreateDatabaseQuery().Where(db => db.Id == "weatherdata").AsEnumerable().FirstOrDefault();
// If the database does not exist, create a new database
if (database == null)
{
database = await client.CreateDatabaseAsync(
new Database
{
Id = "weatherdata"
});
}
// Check to verify a document collection with the id=FamilyCollection does not exist
DocumentCollection documentCollection = client.CreateDocumentCollectionQuery(database.SelfLink).Where(c => c.Id == "weathercollection").AsEnumerable().FirstOrDefault();
// If the document collection does not exist, create a new collection
if (documentCollection == null)
{
documentCollection = await client.CreateDocumentCollectionAsync("dbs/" + database.Id,
new DocumentCollection
{
Id = "weathercollection"
});
}
//Deserialiser til et dynamisk object
if (text == "")
{
mark m = new mark() { name = "Something" };
await client.CreateDocumentAsync(documentCollection.DocumentsLink, m);
}
else
{
var json = JsonConvert.DeserializeObject<dynamic>(text);
json["id"] = json["name"] + "_" + DateTime.Now;
await client.CreateDocumentAsync(documentCollection.DocumentsLink, json);
}
}
public sealed class mark
{
public string name { get; set; }
}
UPDATE - This is what I have in my App.config
<appSettings>
<!-- Replace the value with the value you copied from the Azure management portal -->
<add key="EndPointUrl" value="https://<My account>.documents.azure.com:443/"/>
<!-- Replace the value with the value you copied from the Azure management portal -->
<add key="AuthorizationKey" value="The secret code from Azure"/>
Also, At DocumentDB Account i find the Connection string like this. AccountEndpoint=https://knoerregaard.documents.azure.com:443/;AccountKey=my secret password
How should I apply this to the WebJob?
Appriciate your help!

What is the PostFileWithRequest equivalent in ServiceStack's 'New API'?

I want to post some request values alongside the multipart-formdata file contents. In the old API you could use PostFileWithRequest:
[Test]
public void Can_POST_upload_file_using_ServiceClient_with_request()
{
IServiceClient client = new JsonServiceClient(ListeningOn);
var uploadFile = new FileInfo("~/TestExistingDir/upload.html".MapProjectPath());
var request = new FileUpload{CustomerId = 123, CustomerName = "Foo"};
var response = client.PostFileWithRequest<FileUploadResponse>(ListeningOn + "/fileuploads", uploadFile, request);
var expectedContents = new StreamReader(uploadFile.OpenRead()).ReadToEnd();
Assert.That(response.FileName, Is.EqualTo(uploadFile.Name));
Assert.That(response.ContentLength, Is.EqualTo(uploadFile.Length));
Assert.That(response.Contents, Is.EqualTo(expectedContents));
Assert.That(response.CustomerName, Is.EqualTo("Foo"));
Assert.That(response.CustomerId, Is.EqualTo(123));
}
I can't find any such method in the new API, nor any overrides on client.Post() which suggest that this is still possible. Does anyone know if this is a feature that was dropped?
Update
As #Mythz points out, the feature wasn't dropped. I had made the mistake of not casting the client:
private IRestClient CreateRestClient()
{
return new JsonServiceClient(WebServiceHostUrl);
}
[Test]
public void Can_WebRequest_POST_upload_binary_file_to_save_new_file()
{
var restClient = (JsonServiceClient)CreateRestClient(); // this cast was missing
var fileToUpload = new FileInfo(#"D:/test/test.avi");
var beforeHash = this.Hash(fileToUpload);
var response = restClient.PostFileWithRequest<FilesResponse>("files/UploadedFiles/", fileToUpload, new TestRequest() { Echo = "Test"});
var uploadedFile = new FileInfo(FilesRootDir + "UploadedFiles/test.avi");
var afterHash = this.Hash(uploadedFile);
Assert.That(beforeHas, Is.EqualTo(afterHash));
}
private string Hash(FileInfo file)
{
using (var md5 = MD5.Create())
{
using (var stream = file.OpenRead())
{
var bytes = md5.ComputeHash(stream);
return BitConverter.ToString(md5.ComputeHash(stream)).Replace("-", "").ToLower();
}
}
}
None of the old API was removed from the C# Service Clients, only new API's were added.
The way you process an uploaded file inside a service also hasn't changed.

Resources