I'm migrating from the Microsoft.Azure.Documents SDK to the new Microsoft.Azure.Cosmos (v3.2.0) and am having an issue getting a continuation token back for paginated queries. In the previous SDK when you had the FeedResponse object it returned a bool for HasMoreResults as well as a ContinuationToken which I pass off to my users should they want to make a call for the next page (via an API endpoint). In the new SDK I am trying to use the GetItemQueryIterator method on my container and the only examples I see are using a while loop to get all the pages using the HasMoreResults value with no way for me to extract a ContinuationToken and just pass back the first set of results.
Here is how my code looks so far:
var feedIterator = _documentContext.Container.GetItemQueryIterator<MyDocumentModel>(query, request.ContinuationToken, options);
if (feedIterator.HasMoreResults)
{
listViewModel.HasMoreResults = true;
//listViewModel.ContinuationToken = feedIterator.ContinuationToken; (DOES NOT EXIST!)
}
The commented out line is where I would usually expect to extract the ContinuationToken from but it does not exist.
Most examples show using the iterator like so:
while (feedIterator.HasMoreResults)
{
listViewModel.MyModels.AddRange(_mapper.Map<List<MyModelListViewItem>>(await results.ReadNextAsync()));
}
But I only want to return a single page of results and pass in a continuation token if I want to get the next page.
The ContinuationToken is part of the ReadNextAsync response:
FeedResponse<MyDocumentModel> response = await feedIterator.ReadNextAsync();
var continuation = response.ContinuationToken;
The reason is that the ReadNextAsync is the moment where the service call is made, representing a page of data, and the continuation is for that particular page.
I have not tried the code but looking through the documentation, ReadNextAsync() method on FeedIterator returns you an object of type FeedResponse and that has a ContinuationToken property.
Related
I have tried different ways of searching for files in a folder, even though Azure Storage explorer clearly shows there are files present and the exact same code and configuration has worked before, ListBlogsSementedAsync returns 0 files.
Code used:
var test = await directoryInfo.ListBlobsSegmentedAsync(new BlobContinuationToken());
var fileList = await directoryInfo.ListBlobsSegmentedAsync(true, BlobListingDetails.None, take, null, null, null, new CancellationToken());
I am using Storage Accounts configured as Datalake v2.
I had the same thing happen with storage account configured as standard blob, in that case I could rename the folder and it would work again. With datalake renaming didn't work. Renaming is not really a viable workaround anyway.
I have tried using different BlobListingDetails as well, nothing worked there.
The process that is running is I have a separate job that is uploading files for processing into folders and this job lists the first X files in the folder and downloads them for further processing. This works for a while, but after a while, less than a day, ListBlobsSegmentedAsync returns 0 files. When I use Azure Storage Explorer and check the folder there are thousands of files in the folder and based on data processed this is the correct answer.
EDIT:
Implemented with continuation token:
var directoryInfo = _blobContainer.GetDirectoryReference(directory);
BlobContinuationToken blobContinuationToken = null;
var list = new List<IListBlobItem>();
do
{
var resultSegment = await directoryInfo.ListBlobsSegmentedAsync(blobContinuationToken);
// Get the value of the continuation token returned by the listing call.
blobContinuationToken = resultSegment.ContinuationToken;
list.AddRange(resultSegment.Results);
}
while (blobContinuationToken != null && list.Count < take); // Loop while the continuation token is not null.
var filePaths = list.Select(x => (x as IListBlobItem)?.Uri.ToString());
return filePaths.Where(x => !string.IsNullOrEmpty(x)).ToList();
I don't think you should do new BlobContinuationToken(). This may "confuse" the SDK. You have to pass null initially. Also, are you really iterating over the results? I.e. evaluating the resulting ContinuationToken? The first page can always be empty, you have to check the token to detect if there are any more possible results.
Why is ListBlobsSegmentedAsync only returning results on second page?
It's not at all unexpected that you can occasionally get empty pages or pages with less than the max results along with a continuation token.
https://github.com/Azure-Samples/azure-sdk-for-net-storage-blob-upload-download/blob/master/v11/Program.cs
BlobContinuationToken blobContinuationToken = null;
do
{
var resultSegment = await cloudBlobContainer.ListBlobsSegmentedAsync(null, blobContinuationToken);
// Get the value of the continuation token returned by the listing call.
blobContinuationToken = resultSegment.ContinuationToken;
foreach (IListBlobItem item in resultSegment.Results)
{
Console.WriteLine(item.Uri);
}
}
while (blobContinuationToken != null); // Loop while the continuation token is not null.
In your case you might not want to wait until the token is null, you will probably want to combine it with keeping track of the returned item count.
How to create and display a "HeroCard" within the fulfill() function of LUIS action binding using node.js ? I am following the samples provided by the microsoft(https://github.com/Microsoft/BotBuilder-Samples/tree/master/Node/blog-LUISActionBinding)
Here is that how I tried to do this...
fulfill: function (parameters, callback) {
utilities.FilterFunction(parameters.x, parameters.y).then(function (matches){
utilities.CreateCard(session, matches).then(function(cards){
var reply = new builder.Message(session)
.attachmentLayout(builder.AttachmentLayout.carousel)
.attachments(cards);
callback(util.format(reply));
});
});
}
How can I use session value in the fulfill method?...without session "utilities.CreateCard" won't work...
Since session is not available in the action's fulfill method, we can make call to only to the utilities.FilterFunction and return the result via the callback. Now in our main js file, in the fulfillReplyHandler we get the actionModel which contains the result from utilities.FilterFunction.
Now we can create the "HeroCard" using the "session" that is accessible in the fulfillReplyHandler.
I can't seem to figure out how to include the CSV file content when calling the Swagger API generated methods for the Microsoft Cognitive Services Recommendations API method Uploadacatalogfiletoamodel(modelID, catalogDisplayName, accountKey);. I've tried setting the catalogDisplayName to the full path to the catalog file, however I'm getting "(EXT-0108) Passed argument is invalid."
When calling any of the Cog Svcs APIs that require HTTP body content, how do I include the body content when the exposed API doesn't have a parameter for the body?
I guess, Swagger can't help you testing functions that need to pass data thru a form. And I guess sending the CSV content in the form data shall do the trick, if you know the proper headers.
I work with nuGet called "Microsoft.Net.Http" and code looks like
HttpContent stringContent = new StringContent(someStringYouWannaSend);
HttpContent bytesContent = new ByteArrayContent(someBytesYouWannaSend);
using (var client = new HttpClient())
using (var formData = new MultipartFormDataContent())
{
formData.Add(stringContent, "metadata", "metadata");
formData.Add(bytesContent, "bytes", "bytes");
HttpResponseMessage response = client.PostAsync(someWebApiEndPoint.ToString(), formData).Result;
if (!response.IsSuccessStatusCode)
{
return false; //LOG
}
string responseContent = response.Content.ReadAsStringAsync().Result;
jsonResult= JsonConvert.DeserializeObject<someCoolClass>(responseContent);
return true;
}
Sorry about that someVariables that can't compile. Hope you'll figure this out.
When you are basing your code on the Swagger definition you depend on the good will of the person that created that Swagger definition. Maybe it is not complete yet.
If you are working on C#, try looking at the Samples repo.
Particularly for the Uploading of the catalog there are several functions on the ApiWrapper class that might be helpful, one has this signature: public CatalogImportStats UploadCatalog(string modelId, string catalogFilePath, string catalogDisplayName), another has this other signature public UsageImportStats UploadUsage(string modelId, string usageFilePath, string usageDisplayName) (where it seems like you can point to a public url).
In your case I'd probably try the second one.
Download the sample and use the Wrapper code defined there in your project.
I am using the following code to get a list of all GitHub Enterprise users, then I am trying to suspend those no longer in AD. The Suspend function works, but User.Suspended property always returns false.
var searhRequest = new SearchUsersRequest("type:user&page="+pageNumber+"&page_size=100");
githubUsers = await client.Search.SearchUsers(searhRequest);
client.User.Administration.Suspend(userId);
Yeah, I think the problem is that we were trying to cast the return value as a user when ultimately the call that this is making behind the scenes doesn't return that data. As a work around, I've just called the get user method to get the users after I've rounded up the original results.
It can probably be done better, but here's what I have right now
Task<SearchUsersResult> task;
List<User> users = new List<User>();
int page = 1;
do
{
task = github.Search.SearchUsers(new SearchUsersRequest("type:user&repos:>=0") { Page = page, PerPage = 500 });
task.Wait();
users.AddRange(task.Result.Items.ToList<User>());
page++;
}
while (users.Count < task.Result.TotalCount);
// Get all users by login (this calls the api once for every user you have)
var tasks = users.Select(u => github.User.Get(u.Login));
// Get all unsuspended users
var activeUsers = Task.WhenAll<User>(tasks).Result.Where<User>(u => !u.Suspended).ToList();
Note that in the result of the call doesn't include the "isSuspended" data (pulled from my local enterprise instance using fiddler and then sanitized)
{"login":"User1"
"id":45
"avatar_url":"http://github.com/avatars/u/45?"
"gravatar_id":""
"url":"http://github.com/api/v3/users/User1"
"html_url":"http://github.com/User1"
"followers_url":"http://github.com/api/v3/users/User1/followers"
"following_url":"http://github.com/api/v3/users/User1/following{/other_user}"
"gists_url":"http://github.com/api/v3/users/User1/gists{/gist_id}"
"starred_url":"http://github.com/api/v3/users/User1/starred{/owner}{/repo}"
"subscriptions_url":"http://github.com/api/v3/users/User1/subscriptions"
"organizations_url":"http://github.com/api/v3/users/User1/orgs"
"repos_url":"http://github.com/api/v3/users/User1/repos"
"events_url":"http://github.com/api/v3/users/User1/events{/privacy}"
"received_events_url":"http://github.com/api/v3/users/User1/received_events"
"type":"User"
"site_admin":false
"ldap_dn":"CN=User1
OU=CompanyDEVUsers
OU=Users
OU=Company
DC=Company
DC=com"
"score":1.0}
I'm trying to consume a web service with the Web API client library. My problem is that the ReadAsAsync doesn't seem to want to fully deserailize the returned object when the submitting function uses a POST method.
If I get the response as a string and manually deserailize it works. (I get a apmsgMessage with all the fields populated)
HttpClient client = GetClient();
var response = client.PostAsJsonAsync("api/robot/Preview", ad).Result;
var msg = response.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<apmsgMessage>(msg.Result);
I originally tried the code below which returns an apmsgMessage Object, but all the fields are null.
HttpClient client = GetClient();
var response = client.PostAsJsonAsync("api/robot/Preview", ad).Result;
var msg = response.Content.ReadAsAsync<apmsgMessage>().Result;
return msg;
My question is why dosn't my orginal (the PostAsJsonAsync) return a apmsgMessage fully populated. Am I doing somethign wrong with the ReadAsAsync?
I just had the same issue, and in my case I solved it by removing the [Serializable] attribute from the class.
I don't know why this attribute conflicts with the deserialization process, but as soon as I took that out, the ReadAsAsync method worked as expected.