Writing raw JSON to CosmosDB and an Azure Function - azure

I want to take the raw JSON body from an HTTP post and write it directly into my CosmosDB.
Let's say the data looks like this:
{
"id": "123456",
"storeName": "City Bar and Grille",
"invoiceTotal": 65
}
However, the documentsOut.AddAsync command uses a format like this:
wait documentsOut.AddAsync(new
{
// create a random ID
id = System.Guid.NewGuid().ToString(),
body = sourceJson
});
And then I end up with a document that looks like this:
{
"id": "0e99d3ab-1956-4c0a-8ec1-99de5c987555",
"body": {
"id": "123456",
"storeName": "City Bar and Grille",
"invoiceTotal": 65
}
}
What I really want is to end up with this:
{
"id": "123456",
"storeName": "City Bar and Grille",
"invoiceTotal": 65
}
I'd like to drop the id = System.Guid.NewGuid().ToString() completely (which should not be hard).
How can I pass the raw JSON through without needing to add it to some parent node (such as body)?

Just to formalize my comment as an answer: You're specifically creating the body property of the Cosmos DB document you're creating:
wait documentsOut.AddAsync(new
{
// create a random ID
id = System.Guid.NewGuid().ToString(),
body = sourceJson
});
At the same time, you're ignoring the incoming ID. Since you wanted to preserve that ID, You can copy the ID over (as long as it remains unique within the partition) instead of generating a new GUID, and also grab individual properties from sourceJson to avoid the nested body element:
wait documentsOut.AddAsync(new
{
id = sourceJson.id,
storeName = sourceJson.storeName,
invoiceTotal = sourceJson.invoiceTotal
});

Using a similar example as shared by you in the question.
We can create a Model class with properties which want to store in database.
public class StoreDetails : TableEntity
{
[JsonProperty("id")]
public string Id { get; set; }
[JsonProperty("storeName")]
public string StoreName { get; set; }
[JsonProperty("invoiceTotal")]
public string InvoiceTotal { get; set; }
}
Then can create object of model and pass that object to AddAsync()method. As shown in below screenshot.
var item = new StoreDetails
{
Id = Guid.NewGuid().ToString(),
StoreName = data.StoreName,
InvoiceTotal = data.InvoiceTotal
};
await storeDetailOut.AddAsync(item);
and when we triggered post API, got response as shown below.
As finally we can see the same record is stored in CosmosDB.

Related

"Traditional" one-to-many Query with RavenDB

I know the include-feature of RavenDB. It allows me to fetch a referenced document right away in one roundtrip to the database. But my problem is: The document i fetch in the first place is not including a reference to the "other" documents. But the "other" documents have references to the current document.
Imagine a setup where we have sites across the world. Each site may trigger various alarms. Each alarm has a reference to the site via siteId.
Now i would like to get a list of all the sites including all alarms. But it looks like, this is not possible with RavenDB? Since include only accepts a "path" in the site-Document which holds an id (or an array of ids) to the referenced document.
This could be solved by providing an array of alarmIds within the site'-document and referencing this array in include. But in contrast to a lot of examples featuring stuff like an orderwithlineItemswhere the order is a self contained thing, mysite` will be running for years, collecting alarms anywhere between 0 and a million. Which seems to be a bad idea to me.
Of course i could go the other way round: Query all alarms and include the sites via sitesId. But this would not return a site that has zero alarms.
So is this just a design error on my side? To i misunderstand something? Or is it just not possible to do this in one query and prevent a "n+1 query"?
public class A
{
public string Id { get; set; }
}
public class B
{
public string Id { get; set; }
public string A { get; set; }
}
public class MultiMapIndex : AbstractMultiMapIndexCreationTask<MultiMapIndex.Result>
{
public class Result
{
public string Id { get; set; }
public IEnumerable<string> Bs { get; set; }
}
public MultiMapIndex()
{
AddMap<A>(items => from a in items
select new Result {Id = a.Id, Bs = new string[0]});
AddMap<B>(items => from b in items
select new Result {Id = b.A, Bs = new[] {b.Id}});
Reduce = results => from result in results
group result by result.Id
into g
select new Result {Id = g.Key, Bs = g.SelectMany(r => r.Bs)};
}
}
[Fact]
public async Task TestCase()
{
using var store = GetDocumentStore();
await new MultiMapIndex().ExecuteAsync(store);
using (var session = store.OpenAsyncSession())
{
await session.StoreAsync(new B {A = "a/1"}, "b/0");
await session.StoreAsync(new A(), "a/1");
await session.StoreAsync(new A(), "a/2");
await session.SaveChangesAsync();
}
WaitForIndexing(store);
using (var session = store.OpenAsyncSession())
{
var results = await session.Query<MultiMapIndex.Result, MultiMapIndex>()
.Include(r => r.Bs)
.ToArrayAsync();
var before = session.Advanced.NumberOfRequests;
var bs = session.LoadAsync<B>(results[0].Bs);
Assert.Equal(before, session.Advanced.NumberOfRequests);
}
}
If you do choose to query all Alarms, as you mention,
then you can create a Map-Reduce index on the Alarms collection which will group-by the Sites.
Then you can query this Map-Reduce index and know per Site the count of Alarms it has or doesn't have...
https://demo.ravendb.net/demos/csharp/static-indexes/map-reduce-index

Dictionary property in Cosmos DB using EF

I have a class as below and try to use EF core framework to store the below model to Cosmos DB. But the the JSON is stored as
{
"id": "e6b75f1f-0cc2-488c-9074-62e7e85c727a",
"Type ": "testType",
"TagName ": "TagName",
"DictionaryList ": {}
}
public class TestDictionary
{
public Guid Id { get; set; }
public string Type { get; set; }
public string TagName { get; set; }
public Dictionary<string,object> DictionaryList { get; set; }
}
Problem is that the DictionaryList property doesn't save the data that is passed from api. It always stores as empty object instead of data passed through dictionary
I want my Jason to be stored in cosmos as below
{
"id": "e6b75f1f-0cc2-488c-9074-62e7e85c727a",
"Type ": "testType",
"TagName ": "TagName",
"DictionaryList ": {
“Account number”: “123456”,
“Check date”: “11/20/2020”
}
}
Only collections of primitive types are supported by EF Core 6, So DictionaryList needs to be typed as Dictionary<string,string>
Dictionary<string,object> is recognized as a nested entity type, you could use it, but you'd need to explicitly configure all the keys that it can contain:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<TestDictionary>()
.OwnsOne(t => t.DictionaryList, c =>
{
c.Property<string>("Type");
c.Property<string>("TagName");
});
}
I later realized that I could using the cosmos sdk insert Dictionary<stringm, object> without any problem, and that Ef core does expose the underlying cosmos client.
So I solved the issue by access the underlying cosmos client, and use that to insert my dictionary.
var cosmosClient = dbContext.Database.GetCosmosClient();
var container = cosmosClient.GetContainer( "db", "container");
await container.CreateItemAsync(new Dictionary<string,object>);
works like a charm.
remember the Id has to be id or atleast serializer has to be told that Id has to be interpreted id

When using ServiceStack templates, is it possible to shape output of htmldump with attributes on dumped object?

The code below will output a html table with the values "Name" and "Age" in the first column. Is it possible to output something else like "Navn" for "Name" and "Alder" for "Age"? If so, how? I've tried adding attributes like [DisplayName] and [DataMember] on the properties of the Person class without any success.
using (var context = new TemplateContext().Init())
{
context.VirtualFiles.WriteFile("page.html", "{{person | htmldump}}");
var pageResult = new PageResult(context.GetPage("page"))
{
Args = {["person"] = new Person{Age = "20", Name = "Ada"}}
};
var htmlString = await pageResult.RenderToStringAsync();
Console.WriteLine(htmlString);
}
public class Person
{
public string Name { get; set; }
public string Age { get; set; }
}
ServiceStack Templates HTML Filters just renders the property name of each item, so to change the Property Names it uses you can just select a new Object using the map filter with the property names you want, e.g:
{{ people | map => { Navn:it.Name, Alder:it.Age } | htmlDump }}

How to use OData without the Entity Framework

I am interested in creating a OData wcf data service using visual studio 2012. However I do not want to use an entity model framework but rather use my scheme less nosql dataset to store and retrieve the data.
Is there a way that allows me to take control of the odata services without being tide into a specific class structure such as Microsoft's entity framework.
You can use the Microsoft OData implementation without the Entity Framework. What you do need is an implementation of IQueryable. Here's an example OData service querying an array of objects:
using System.Web.Http;
using System.Web.Http.OData;
using System.Web.Http.OData.Builder;
using System.Web.Http.OData.Query;
// GET api/values
[ActionName("FromList")]
public IList<Poco> GetFromList(ODataQueryOptions<Poco> queryOptions)
{
IQueryable<Poco> data = (
new Poco[] {
new Poco() { id = 1, name = "one", type = "a" },
new Poco() { id = 2, name = "two", type = "b" },
new Poco() { id = 3, name = "three", type = "c" }
})
.AsQueryable();
var t = new ODataValidationSettings() { MaxTop = 25 };
queryOptions.Validate(t);
var s = new ODataQuerySettings() { PageSize = 25 };
IEnumerable<Poco> results =
(IEnumerable<Poco>)queryOptions.ApplyTo(data, s);
return results.ToList();
}
public class Poco
{
public int id { get; set; }
public string name { get; set; }
public string type { get; set; }
}

'Unexpected element: XX' during deserialization MongoDB C#

I'm trying to persist an object into a MongoDB, using the following bit of code:
public class myClass
{
public string Heading { get; set; }
public string Body { get; set; }
}
static void Main(string[] args)
{
var mongo = MongoServer.Create();
var db = mongo.GetDatabase("myDb");
var col = db.GetCollection<BsonDocument>("myCollection");
var myinstance = new myClass();
col.Insert(myinstance);
var query = Query.And(Query.EQ("_id", new ObjectId("4df06c23f0e7e51f087611f7)));
var res = col.Find(query);
foreach (var doc in res)
{
var obj = BsonSerializer.Deserialize<myClass>(doc);
}
}
However I get the following exception 'Unexpected element: _id' when trying to Deserialize the document.
So do I need to Deserialize in another way?? What is the preferred way of doing this?
TIA
Søren
You are searching for a given document using an ObjectId but when you save an instance of MyClass you aren't providing an Id property so the driver will create one for you (you can make any property the id by adding the [BsonId] attribute to it), when you retrieve that document you don't have an Id so you get the deserialization error.
You can add the BsonIgnorExtraElements attribute to the class as Chris said, but you should really add an Id property of type ObjectId to your class, you obviously need the Id (as you are using it in your query). As the _id property is reserved for the primary key, you are only ever going to retrieve a single document so you would be better off writing your query like this:
col.FindOneById(new ObjectId("4df06c23f0e7e51f087611f7"));
The fact that you are deserializing to an instance of MyClass once you retrieve the document lends itself to strongly typing the collection, so where you create an instance of the collection you can do this
var col = db.GetCollection<MyClass>("myCollection");
so that when you retrieve the document using the FindOneById method the driver will take care of the deserialization for you putting it all together (provided you add the Id property to the class) you could write
var col = db.GetCollection<MyClass>("myCollection");
MyClass myClass = col.FindOneById(new ObjectId("4df06c23f0e7e51f087611f7"));
One final thing to note, as the _id property is created for you on save by the driver, if you were to leave it off your MyClass instance, every time you saved that document you would get a new Id and hence a new document, so if you saved it n times you would have n documents, which probably isn't what you want.
A slight variation of Projapati's answer. First Mongo will deserialize the id value happily to a property named Id which is more chsarp-ish. But you don't necessarily need to do this if you are just retrieving data.
You can add [BsonIgnoreExtraElements] to your class and it should work. This will allow you to return a subset of the data, great for queries and view-models.
Try adding _id to your class.
This usually happens when your class doesn't have members for all fields in your document.
public class myClass
{
public ObjectId _id { get; set; }
public string Heading { get; set; }
public string Body { get; set; }
}

Resources