Azure Table Storage: Ignoring a property of a TableEntity when using the Azure.Data.Tables package - azure

I am using the new Azure.Data.Tables library from Microsoft to deal with Azure Table Storage. With the old library when you had an entity that implemented ITableEntity and you had a property that you did not want to save to the storage table you would use the [IgnoreProperty] annotation. However, this does not seem to be available on the new library.
What would be the equivalent on the Azure.Data.Tables package or how do you now avoid saving a property to table storage now?
This is the class I want to persist:
public class MySpatialEntity : ITableEntity
{
public int ObjectId { get; set; }
public string Name { get; set; }
public int MonitoringArea { get; set; }
//This is the property I want to ignore because table storage cannot store it
public Point Geometry { get; set; }
//ITableEntity Members
public virtual string PartitionKey { get => MonitoringArea.ToString(); set => MonitoringArea = int.Parse(value); }
public virtual string RowKey { get => ObjectId.ToString(); set => ObjectId = int.Parse(value); }
public DateTimeOffset? Timestamp { get; set; }
public ETag ETag { get; set; }
}

As of version 12.2.0.beta.1, Azure.Data.Tables table entity models now support ignoring properties during serialization via the [IgnoreDataMember] attribute and renaming properties via the [DataMember(Name="<yourNameHere>")] attribute.
See the changelog here.

I don't think there's anything like [IgnoreProperty] available as of now (at least with version 12.1.0).
I found two Github issues which talk about this:
https://github.com/Azure/azure-sdk-for-net/issues/19782
https://github.com/Azure/azure-sdk-for-net/issues/15383
What you can do is create a custom dictionary of the properties you want to persist in the entity and use that dictionary for add/update operations.
Please see sample code below:
using System;
using System.Collections.Generic;
using System.Drawing;
using Azure;
using Azure.Data.Tables;
namespace SO68633776
{
class Program
{
private static string connectionString = "connection-string";
private static string tableName = "table-name";
static void Main(string[] args)
{
MySpatialEntity mySpatialEntity = new MySpatialEntity()
{
ObjectId = 1,
Name = "Some Value",
MonitoringArea = 2
};
TableEntity entity = new TableEntity(mySpatialEntity.ToDictionary());
TableClient tableClient = new TableClient(connectionString, tableName);
var result = tableClient.AddEntity(entity);
}
}
public class MySpatialEntity: ITableEntity
{
public int ObjectId { get; set; }
public string Name { get; set; }
public int MonitoringArea { get; set; }
//This is the property I want to ignore because table storage cannot store it
public Point Geometry { get; set; }
//ITableEntity Members
public virtual string PartitionKey { get => MonitoringArea.ToString(); set => MonitoringArea = int.Parse(value); }
public virtual string RowKey { get => ObjectId.ToString(); set => ObjectId = int.Parse(value); }
public DateTimeOffset? Timestamp { get; set; }
public ETag ETag { get; set; }
public IDictionary<string, object> ToDictionary()
{
return new Dictionary<string, object>()
{
{"PartitionKey", PartitionKey},
{"RowKey", RowKey},
{"ObjectId", ObjectId},
{"Name", Name},
{"MonitoringArea", MonitoringArea}
};
}
}
}

Related

The property 'Country.IdCountry' is of type 'Guid' which is not supported by the current database provider

I'm trying to use EF Core in combination with Azure CosmosDB. I'm using the following configuration:
Entities:
public class Country
{
public Guid IdCountry { get; set; }
public string Name { get; set; }
public string Code { get; set; }
public string PhoneCode { get; set; }
public IdNameReference Currency { get; set; }
}
public class IdNameReference
{
public Guid Id { get; set; }
public string Name { get; set; }
}
EntityConfiguration class:
public class CountryEntityConfiguration : IEntityTypeConfiguration<Country>
{
public void Configure(EntityTypeBuilder<Country> builder)
{
builder.ToContainer("Country");
builder.HasKey(e => e.IdCountry);
builder.HasPartitionKey(e => e.IdCountry);
builder.HasNoDiscriminator();
builder.Property(e => e.IdCountry).HasConversion<GuidToStringConverter>().ValueGeneratedOnAdd().HasValueGenerator<GuidValueGenerator>();
builder.HasOne(e => e.Currency).WithMany().Metadata.DependentToPrincipal.SetPropertyAccessMode(PropertyAccessMode.Field);
}
}
DbContext OnModelCreating:
protected override void OnModelCreating(ModelBuilder builder)
{
base.OnModelCreating(builder);
builder.ApplyConfiguration(new CurrencyEntityConfiguration());
}
And the IServiceCollection configuration:
services.AddDbContext<MyDbContext>(options => options.UseCosmos(
Environment.GetEnvironmentVariable("Db_ServiceEndpoint"),
Environment.GetEnvironmentVariable("Db_AccountKey"),
Environment.GetEnvironmentVariable("Db_DatabaseName")
)
);
But I'm still getting the following error:
The property 'Country.IdCountry' is of type 'Guid' which is not supported by the current database provider. Either change the property CLR type, or ignore the property using the '[NotMapped]' attribute or by using 'EntityTypeBuilder.Ignore' in 'OnModelCreating'
EDIT: I forgot to specify. This happens when I try to add a new item to the context collection
I'm new in CosmosDB and also in configuration of EF this way.
Do you have an idea, where the problem could be?
Okay, I figured it out, I used wrong the ValueConverter.
I used
.HasConversion<GuidToStringConversion>()
but I had to use
.HasConversion<string>()
or
.HasConversion(g => g.ToString("D"), s => new Guid(s))
I hope it will someone :)

Azure Easy Tables - Load only one column

Is there some way to get only one data column for one row from Azure Easy Tables?
For example Xamarin.Forms app will send name of item to Azure and get the item creation DateTime only.
Here's an example where we want to select just the Name Column from our Dog Table.
This sample uses the Azure Mobile Client and the Azure Mobile Client SQL NuGet Packages.
Model
using Microsoft.WindowsAzure.MobileServices;
using Newtonsoft.Json;
namespace SampleApp
{
public class Dog
{
public string Name { get; set; }
public string Breed { get; set; }
public int Age { get; set; }
[JsonProperty(PropertyName = "id")]
public string Id { get; set; }
[CreatedAt]
public DateTimeOffset CreatedAt { get; set; }
[UpdatedAt]
public DateTimeOffset UpdatedAt { get; set; }
[Version]
public string AzureVersion { get; set; }
[Deleted]
public bool IsDeleted { get; set; }
}
}
Logic
using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using System.Collections.Generic;
using Microsoft.WindowsAzure.MobileServices;
using Microsoft.WindowsAzure.MobileServices.Sync;
using Microsoft.WindowsAzure.MobileServices.SQLiteStore;
namespace SampleApp
{
public class MobileClientService
{
bool isMobileClientInitialized;
MobileServiceClient mobileClient;
public async Task<string> GetDogName(string id)
{
await InitializeMobileClient();
var dog = await mobileClient.GetSyncTable<Dog>().LookupAsync(id);
var dogName = dog.Name;
return dogName;
}
public async Task<IEnumerable<string>> GetDogNames()
{
await InitializeMobileClient();
var dogNameList = await mobileClient.GetSyncTable<Dog>().Select(x => x.Name).ToEnumerableAsync();
return dogNameList;
}
async Task InitializeMobileClient()
{
if(isMobileClientInitialized)
return;
mobileClient = new MobileServiceClient("Your Azure Mobile Client Url");
var path = Path.Combine(MobileServiceClient.DefaultDatabasePath, "app.db");
var store = new MobileServiceSQLiteStore(path);
store.DefineTable<Dog>();
//ToDo Define all remaining tables
await MobileServiceClient.SyncContext.InitializeAsync(store, new MobileServiceSyncHandler());
}
}
}

When trying to override Partition Key using Azuren Search and Azure Table Storage with .NET getting bad request

I am using Azuren Search and Azure Table Storage and with .net and i am trying to index a table and make a partition key filterable now this works fine until i try to insert something in that table where i get a BadRequest with not much of additional info.
This is my class bellow
using System;
using Microsoft.Azure.Search;
using Microsoft.Azure.Search.Models;
using Microsoft.WindowsAzure.Storage.Table;
[SerializePropertyNamesAsCamelCase]
public class Asset : TableEntity
{
public Asset(string name)
{
Name = name;
}
public Asset()
{
}
public Asset(string name, DateTimeOffset toBePublished, string pkey)
{
Name = name;
ToBePublishedDate = toBePublished;
PartitionKey = pkey;
}
[System.ComponentModel.DataAnnotations.Key]
public string Id { get; set; } = DateTimeOffset.UtcNow.ToString("O")
.Replace("+", string.Empty)
.Replace(":", string.Empty)
.Replace(".", string.Empty);
[IsFilterable, IsSortable, IsSearchable]
public new string PartitionKey { get; set; }
[IsFilterable, IsSortable, IsSearchable]
public string Name { get; set; } = "TemptAsset " + new Guid();
[IsFilterable, IsSortable]
public int? Version { get; set; } = 1;
[IsFilterable, IsSortable]
public DateTimeOffset? ToBePublishedDate { get; set; } = DateTimeOffset.UtcNow;
[IsFilterable, IsSortable]
public DateTimeOffset? ToBeRetiredDate { get; set; } = null;
[IsFilterable, IsSearchable, IsSortable]
public string Company { get; set; } = "TempCompany";
[IsFilterable, IsSortable]
public bool IsApproved { get; set; } = false;
[IsFilterable, IsSortable]
public bool IsDraft { get; set; } = true;
}
This runs and the index is created successfully see bellow
Now if i try to add an entity to that table i get a BadRequest, but do the exact same thing with commenting out the PartitionKey in my entity and this works fine.
This is how i create my index
AzureSearch.CreateAssetNameIndex(AzureSearch.CreateSearchServiceClient());
and the methods called bellow
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using AssetSynch.Models;
using Microsoft.Azure.Search;
using Microsoft.Azure.Search.Models;
public static SearchServiceClient CreateSearchServiceClient()
{
string searchServiceName = "*****";
string adminApiKey = "********";
SearchServiceClient serviceClient = new SearchServiceClient(searchServiceName,
new SearchCredentials(adminApiKey));
return serviceClient;
}
public static async void CreateAssetNameIndex(SearchServiceClient serviceClient)
{
Index definition = new Index
{
Name = "assetname",
Fields = FieldBuilder.BuildForType<Asset>()
};
await serviceClient.Indexes.CreateAsync(definition);
}
If i return the error using postman this is the exception i get
{
"innerExceptions": [
{
"requestInformation": {
"httpStatusCode": 400,
"httpStatusMessage": "Bad Request",
"serviceRequestID": "59efbc9a-0002-002c-3570-d5d55c000000",
"contentMd5": null,
"etag": null,
"requestDate": "Thu, 25 May 2017 17:05:01 GMT",
"targetLocation": 0,
"extendedErrorInformation": {
"errorCode": "PropertiesNeedValue",
"errorMessage": "The values are not specified for all properties in the entity.\nRequestId:59efbc9a-0002-002c-3570-d5d55c000000\nTime:2017-05-25T16:05:06.5197909Z",
"additionalDetails": {}
},
"isRequestServerEncrypted": false
}
}
]
}
if i remove the Partition key from my entity and re run the same code to re-create the index the same piece of code this executes successfully.
What i did noticed is that there are now 2 Partition keys on my entity one of which will remain null see image bellow and that my property does not override the original.
Is there something i am missing here?
According to your codes, I find your Asset has used new keyword to modify the base class's partition property.
But this will just hidden the base.partition not override it.
public new string PartitionKey { get; set; }
After you set the value in the Asset class, you will find it contains two partition as below:
So if the base class's partition key value is null, it will return 400 error.
So if you want to add the new entity to the table, you need set the base class(TableEntity) partition key value.
So I suggest you could change your Asset as below:
[SerializePropertyNamesAsCamelCase]
public class Asset : TableEntity
{
public Asset(string name)
{
Name = name;
base.PartitionKey = this.PartitionKey;
}
public Asset()
{
base.PartitionKey = this.PartitionKey;
}
public Asset(string name, DateTimeOffset toBePublished, string pkey)
{
Name = name;
ToBePublishedDate = toBePublished;
PartitionKey = pkey;
base.PartitionKey = this.PartitionKey;
}
[Key]
[IsFilterable]
public string Id { get; set; } = DateTimeOffset.UtcNow.ToString("O")
.Replace("+", string.Empty)
.Replace(":", string.Empty)
.Replace(".", string.Empty);
[IsFilterable, IsSortable, IsSearchable]
public new string PartitionKey { get; set; }
[IsFilterable, IsSortable, IsSearchable]
public string Name { get; set; } = "TemptAsset " + new Guid();
[IsFilterable, IsSortable]
public int? Version { get; set; } = 1;
[IsFilterable, IsSortable]
public DateTimeOffset? ToBePublishedDate { get; set; } = DateTimeOffset.UtcNow;
[IsFilterable, IsSortable]
public DateTimeOffset? ToBeRetiredDate { get; set; } = null;
[IsFilterable, IsSearchable, IsSortable]
public string Company { get; set; } = "TempCompany";
[IsFilterable, IsSortable]
public bool IsApproved { get; set; } = false;
[IsFilterable, IsSortable]
public bool IsDraft { get; set; } = true;
}
If you want to use table storage as datasource, I suggest you could refer to this article.

Entity Type 'AstNode' has no key defined

I'm porting a data model from EF4 to EF6 Code First. I'm getting the following message when the database creation is attempted. I'm at a loss to understand what is causing this. I don't have any Context, AstNode or JSParser entities. It is also not looking in the Models namespace:
var context = QPDataContext.Create();
var session = context.DataSessions.FirstOrDefault(ds => ds.DataSessionId = sessionId);
Throws this exception:
{"One or more validation errors were detected during model generation:
QPWebRater.DAL.Context: : EntityType 'Context' has no key defined. Define the key for this EntityType.
QPWebRater.DAL.AstNode: : EntityType 'AstNode' has no key defined. Define the key for this EntityType.
QPWebRater.DAL.JSParser: : EntityType 'JSParser' has no key defined. Define the key for this EntityType.
(many more similar errors snipped).
"}
Here is my database context (I've simplified it a bit):
QPWebRater.DAL.QPDataContext.cs:
using System;
using System.Collections.Generic;
using System.Data.Entity;
using System.Data.Entity.Core;
using System.Data.Entity.Validation;
using System.Diagnostics;
using System.Linq;
using System.Text.RegularExpressions;
using System.Web;
using Microsoft.Ajax.Utilities;
using QPWebRater.Models;
using QPWebRater.Utilities;
namespace QPWebRater.DAL
{
public class QPDataContext : DbContext
{
public QPDataContext()
: base("DefaultConnection")
{
Database.SetInitializer<QPDataContext>(new CreateDatabaseIfNotExists<QPDataContext>());
}
public static QPDataContext Create()
{
return new QPDataContext();
}
public DbSet<DataSession> DataSession { get; set; }
public DbSet<Document> Documents { get; set; }
public DbSet<Driver> Drivers { get; set; }
public DbSet<Location> Locations { get; set; }
public DbSet<Lookup> Lookups { get; set; }
public DbSet<Quote> Quotes { get; set; }
public DbSet<Vehicle> Vehicles { get; set; }
public DbSet<Violation> Violations { get; set; }
}
}
QPWebRater.Models.DatabaseModels.cs:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
namespace QPWebRater.Models
{
public partial class DataSession
{
public DataSession()
{
this.Vehicles = new HashSet<Vehicle>();
this.Drivers = new HashSet<Driver>();
...
}
public string DataSessionId { get; set; }
public System.DateTime Timestamp { get; set; }
...
}
public partial class Document
{
public int DocumentId { get; set; }
public int QuoteId { get; set; }
public string DocumentType { get; set; }
public string Url { get; set; }
public string Description { get; set; }
public virtual Quote Quote { get; set; }
}
public partial class Driver
{
public Driver()
{
this.Violations = new HashSet<Violation>();
}
public int DriverId { get; set; }
public string DataSessionId { get; set; }
...
}
}
I solved this by examining all of the DbSet definitions. I had pared down the data model while also upgrading it. I had removed the Lookup model class but neglected to also remove the DbSet<Lookup> Lookups { get; set; } property.
This resulted in the class being resolved as Microsoft.Ajax.Utilities.Lookup. At runtime, EntityFramework tried to add a corresponding database table which failed miserably. If you are running into a similar problem then double check the generic types in your DbSet properties.

Azure Table Storage - TableEntity map column with a different name

I am using Azure Table Storage as my data sink for my Semantic Logging Application Block. When I call a log to be written by my custom EventSource, I get columns similar to the ff.:
EventId
Payload_username
Opcode
I can obtain these columns by creating a TableEntity class that matches the column names exactly (except for EventId, for some reason):
public class ReportLogEntity : TableEntity
{
public string EventId { get; set; }
public string Payload_username { get; set; }
public string Opcode { get; set; }
}
However, I would like to store the data in these columns in differently named properties in my TableEntity:
public class ReportLogEntity : TableEntity
{
public string Id { get; set; } // maps to "EventId"
public string Username { get; set; } // maps to "Payload_username"
public string Operation { get; set; } // maps to "Opcode"
}
Is there a mapper/attribute I can make use of to allow myself to have the column name different from the TableEntity property name?
You can override ReadEntity and WriteEntity methods of interface ITableEntity to customize your own property names.
public class ReportLogEntity : TableEntity
{
public string PartitionKey { get; set; }
public string RowKey { get; set; }
public string Id { get; set; } // maps to "EventId"
public string Username { get; set; } // maps to "Payload_username"
public string Operation { get; set; } // maps to "Opcode"
public override void ReadEntity(IDictionary<string, EntityProperty> properties, OperationContext operationContext)
{
this.PartitionKey = properties["PartitionKey"].StringValue;
this.RowKey = properties["RowKey"].StringValue;
this.Id = properties["EventId"].StringValue;
this.Username = properties["Payload_username"].StringValue;
this.Operation = properties["Opcode"].StringValue;
}
public override IDictionary<string, EntityProperty> WriteEntity(OperationContext operationContext)
{
var properties = new Dictionary<string, EntityProperty>();
properties.Add("PartitionKey", new EntityProperty(this.PartitionKey));
properties.Add("RowKey", new EntityProperty(this.RowKey));
properties.Add("EventId", new EntityProperty(this.Id));
properties.Add("Payload_username", new EntityProperty(this.Username));
properties.Add("Opcode", new EntityProperty(this.Operation));
return properties;
}
}

Resources