Seeder method for Azure Database in EF Core 2 - azure

What is the proper method to seed data into an Azure Database? Currently in development I have a seeder method that inserts the first couple of users as well as products. The Users (including admin user) username and password are hardcoded into the Seed method, is this an acceptable practice?
As far as the products are concerned, I have a json file with the product names and descriptions - which in development the seeder method iterates through and inserts the data.

To answer your question "The Users (including admin user) username and password are hardcoded into the Seed method, is this an acceptable practice?"
No you should keep your password in cleartext format, though you can keep it it encrypet mode and seed it.
In EF Core 2.1, the seeding workflow is quite different. There is now Fluent API logic to define the seed data in OnModelCreating. Then, when you create a migration, the seeding is transformed into migration commands to perform inserts, and is eventually transformed into SQL that that particular migration executes. Further migrations will know to insert more data, or even perform updates and deletes, depending on what changes you make in the OnModelCreating method.
Suppose thethree classes in my model are Magazine, Article and Author. A magazine can have one or more articles and an article can have one author. There’s also a PublicationsContext that uses SQLite as its data provider and has some basic SQL logging set up.
Let take an example of single entity type.
Let’s start by seeing what it looks like to provide seed data for a magazine—at its simplest.
The key to the new seeding feature is the HasData Fluent API method, which you can apply to an Entity in the OnModelCreating method.
Here’s the structure of the Magazine type:
public class Magazine
{
public int MagazineId { get; set; }
public string Name { get; set; }
public string Publisher { get; set; }
public List<Article> Articles { get; set; }
}
It has a key property, MagazineId, two strings and a list of Article types. Now let’s seed it with data for a single magazine:
protected override void OnModelCreating (ModelBuilder modelBuilder)
{
modelBuilder.Entity<Magazine> ().HasData
(new Magazine { MagazineId = 1, Name = "MSDN Magazine" });
}
A couple things to pay attention to here: First, I’m explicitly setting the key property, MagazineId. Second, I’m not supplying the Publisher string.
Next, I’ll add a migration, my first for this model. I happen to be using Visual Studio Code for this project, which is a .NET Core app, so I’m using the CLI migrations command, “dotnet ef migrations add init.” The resulting migration file contains all of the usual CreateTable and other relevant logic, followed by code to insert the new data, specifying the table name, columns and values:
migrationBuilder.InsertData(
table: "Magazines",
columns: new[] { "MagazineId", "Name", "Publisher" },
values: new object[] { 1, "MSDN Magazine", null });
Inserting the primary key value stands out to me here—especially after I’ve checked how the MagazineId column was defined further up in the migration file. It’s a column that should auto-increment, so you may not expect that value to be explicitly inserted:
MagazineId = table.Column<int>(nullable: false)
.Annotation("Sqlite:Autoincrement", true)
Let’s continue to see how this works out. Using the migrations script command, “dotnet ef migrations script,” to show what will be sent to the database, I can see that the primary key value will still be inserted into the key column:
INSERT INTO "Magazines" ("MagazineId", "Name", "Publisher")
VALUES (1, 'MSDN Magazine', NULL);
That’s because I’m targeting SQLite. SQLite will insert a key value if it’s provided, overriding the auto-increment. But what about with a SQL Server database, which definitely won’t do that on the fly?
I switched the context to use the SQL Server provider to investigate and saw that the SQL generated by the SQL Server provider includes logic to temporarily set IDENTITY_INSERT ON. That way, the supplied value will be inserted into the primary key column. Mystery solved!
You can use HasData to insert multiple rows at a time, though keep in mind that HasData is specific to a single entity. You can’t combine inserts to multiple tables with HasData. Here, I’m inserting two magazines at once:
modelBuilder.Entity<Magazine>()
.HasData(new Magazine{MagazineId=2, Name="New Yorker"},
new Magazine{MagazineId=3, Name="Scientific American"}
);
For a complete example , you can browse through this sample repo
Hope it helps.

Related

Does jOOQ support putting names on SQL statements?

Is there a way to "tag" or put names on SQL statements in jOOQ so when I look at the Performance Insights of AWS RDS, I can see something more meaningful than the first 500 chars of the statement?
For example, Performance Insights shows that this query is taking a toll in my DB:
select "my_schema"."custs"."id", "my_schema"."custs"."other_id", "my_schema"."custs"."cid_id", "my_schema"."custs"."valid_since", "my_schema"."custs"."valid_until", "my_schema"."custs"."address", "my_schema"."custs"."address_id_1", "my_schema"."pets"."id", "my_schema"."pets"."cst_id", "my_schema"."pets"."tag", "my_schema"."pets"."name", "my_schema"."pets"."description", "my_schema"."pets"."owner", "my_schema"."pets"."created_on", "my_schema"."pets"."created_by", "my_schema"."pets"."modified_on",
But as it comes chopped, it's not straight-forward to know which jOOQ code generated this.
I would prefer to see something like this:
Customer - Pet Lookup
or:
(Customer - Pet Lookup) select "my_schema"."custs"."id", "my_schema"."custs"."other_id", "my_schema"."custs"."cid_id", "my_schema"."custs"."valid_since", "my_schema"."custs"."valid_until", "my_schema"."custs"."address", "my_schema"."custs"."address_id_1", "my_schema"."pets"."id", "my_schema"."pets"."cst_id", "my_schema"."pets"."tag", "my_schema"."pets"."name", "my_schema"."pets"."description", "my_schema"."pets"."owner", "my_schema"."pets"."created_on", "my_schema"."pets"."created_by", "my_schema"."pets"."modified_on",
There are at least two out of the box approaches to what you want to achieve, both completely vendor agnostic:
1. Use "hints"
jOOQ supports Oracle style hints using the hint() method, at least for SELECT statements. Write something like:
ctx.select(T.A, T.B)
.hint("/* my tag */")
.from(T)
.where(...)
The limitation here is the location of the hint, which is going to be right after the SELECT keyword. Not sure if this will work for your RDBMS.
2. Use an ExecuteListener
You can supply your Configuration with an ExecuteListener, which patches your generated SQL strings with whatever you need to be added:
class MyListener extends DefaultExecuteListener {
// renderEnd() is called after the SQL string is generated, but
// before the prepared statement is created, let alone executed
#Override
public void renderEnd​(ExecuteContext ctx) {
if (mechanismToDetermineIfTaggingIsNeeded())
ctx.sql("/* My tag */ " + ctx.sql());
}
}
Using regular expressions, you can place that tag at any specific location within your SQL string.

IsConcurrencyToken(true) does not actually check concurrency tokens

I have a property configured like this:
public byte[] Timestamp { get; set; }
And then in my DbContext, i use the Fluent API like so:
modelBuilder.Entity<MyClass>()
.Property(x => x.Timestamp)
.IsRowVersion()
.IsConcurrencyToken(true);
So naturally i went ahead and wrote a unit test, ensuring that an entity with a wrong timestamp set would not get saved. I used Sqlite and some custom Sql to make RowVersion work in Unit Tests but to my surprise i never got an exception. Then, i tested it in our application, and i also did not get an exception when a wrong Timestamp was set on the entity.
var myInstance = await myDbContext.Instances
.Include(x => x...)
.Include(x => x...)
.SingleAsync(x => x.Id == id);
// set other values, add new entities to relationships, aso
myInstance.Timestamp = new byte[] { 1, 2, 3, 4 };
await myDbContext.SaveChangesAsync();
I am clearly missing something here. I thought configuring IsRowVersion would be enough to force EF Core to include a WHERE Timestamp = clause in the UPDATE but it seems like thats not the case. As you can see, i also tried calling IsConcurrencyToken (even with its default value of true, just to be sure) but to no avail.
Edit: I have worked "around" it now by including the Timestamp in my SingleAsync call, but this still leaves me unsure if its still possible to not get a concurrency exception, as the Timestamp set on my entity is apparently not checked at all when saving?
This is a known behavior of EF core, as documented here:
https://github.com/aspnet/EntityFrameworkCore/issues/18505
Manually changing the value of the token is considered to be a no-op, as the original value that was queried from the database is being used for a concurrency check. Manually changing the value does nothing, as it is effectively being ignored.
It is possible to manually set the token's value and have EF Core's optimistic concurrency check fail if the database has a newer version of the entity:
db.Entry(entity).Property(nameof(Entity.ConcurrencyToken)).OriginalValue = entity.ConcurrencyToken;
As per this comment in #user604613's answer

mybatis spring batch +sybase: trying to get the database identity value after insertion to assign it to the id field in pojo

my code looks like the sample code given below.
--table create statement
CREATE TABLE LOG
(
uniqueID NUMERIC(20,0) IDENTITY,
NAME VARCHAR(20) NOT NULL,
DESCRIPTION VARCHAR(200) NOT NULL,
USR VARCHAR(20) NOT NULL
)
--pojo class
public class Log
{
private long identifier;
private String name;
private String description;
private String user;
//getters+setters......
}
--insert statement in mapper
<insert id="insertRecord" parameterType="com.xxx.yyy.zzz.model.Log" useGeneratedKeys="true" keyProperty="identifier" keyColumn="uniqueID">
INSERT INTO LOG (NAME, DESCRIPTION, USR)
VALUES (#{log.name}, #{log.description}, #{log.user})
</insert>
issue: when i try to run this code against sybase database, am getting NullPointerException. When i tried to debug it, error came from within SybStatement.class. Sorry am not able to provide entier stacktrace due to constraint in copy/paste at my work station.
I am able to run the same code against H2 database successfully. Records got inserted and "identifier" in Log object is having the identify value same as database rows.
Did you face this issue in sybase?. Please share if anyone is having code for showing the usage of "useGeneratedKeys" mybatis feature in sybase..
Note:
I am running this insert statement using MybatisBatchItemWriter.
I tried to use two different sqlsessiontemplate objects for chunk reader & chunk writer and it didn't resolve the issue.
I am using jconn3 sybase jdbc jar, mybatis 3.4.4 and mybatis-spring 1.3.1 jar.
Thanks in advance
In SQL terms, you need to do SELECT ##IDENTITY to pick up the generated value. Thecquestion is if your framework generates such SQL...

Indexing Sitecore Item security and restricting returned search results

I have several roles defined, each with different restrictions to content and media items and I would like to restrict the search results that are returned based on the access rights of the currently logged in user, rather than displaying the result and the user then presented with an "Access Denied" page. Some content will obviously be accessible to extranet\anonymous so they should be returned for all users regardless.
The security follows the standard Sitecore practices so Role inheritance (roles within roles) will be used, so it will need to take this into account also.
I couldn't see anything in the Advanced Database Crawler module that would help and I've looked through the Sitecore Search and Indexing Guide (version 6.6 and version 7) but couldn't find any information about indexing the security applied to items. The following articles have some suggestions:
How can I set up a Lucene index in Sitecore that handles security correctly?
This feels "dirty" and has the potential for performance issues, particularly when there are a large number of items returned. Also, (see in the comments) the issue with paging results.
Security (aka Permissions) and Lucene - How ? Should it be done?
The above looks more realistic, and would filter out the results based on indexed security roles, there would obviously be a need to expand the roles to handle roles within roles. My concern here would be that we would need to handle denied permissions, when we specifically need to deny/restrict access for certain roles to content items (I know this is not recommended practice, but there is a very specific need to always deny).
I'm at the planning stage at the moment so with the release of Sitecore 7 today there is also the possibility to use the updated Lucene libraries and/or SOLR if that makes life easier - assuming of course that some of the modules like WebForms for Marketers and Email Campaign Manager are updated before too long.
What are the solutions that people are using for returning search results taking into account security? Any alternatives than the linked questions above? Maybe something in Sitecore 7 that I can leverage, the updated Lucene libraries or SOLR?
I'd prefer to keep this all "out of the box" Sitecore and not use other third party search products if at all possible.
A slight alternative to the suggestion from Klaus:
In Sitecore.ContentSeach.config you'll find a pipeline called contentSearch.getGlobalSearchFilters
Processors added to this pipeline will be applied to any query, so if we drop in one that applies a filter based on roles we're good.
ComputedField
To start, we want a computed field added to our index configuration:
<fields hint="raw:AddComputedIndexField">
<field fieldName="read_roles" returnType="stringCollection">Sitecore.ContentSearch.ComputedFields.ReadItemRoles,Sitecore.ContentSearch</field>
</fields>
NOTE the stored type is a collection of strings. We'll use it to index all the names of roles that can read an item.
Implementation
We have a base abstract class to handle the extraction of item security details
public abstract class ItemPermissions: IComputedIndexField
{
public string FieldName { get; set; }
public string ReturnType { get; set; }
public object ComputeFieldValue(IIndexable indexable)
{
var indexableItem = indexable as SitecoreIndexableItem;
if (indexableItem == null) return null;
var security = indexableItem.Item.Security;
return GetPermissibles(security);
}
protected abstract object GetPermissibles(ItemSecurity security);
}
We implement the above with the abstracted method
public class ReadItemRoles : ItemPermissions
{
protected override object GetPermissibles(ItemSecurity security)
{
var roles = RolesInRolesManager.GetAllRoles();
return roles.Where(security.CanRead).Select(r => r.Name);
}
}
NOTE There's obviously a performance impact here, this will reduce your indexing speed.
To reduce the impact, only add the the computed field to the index configuration for the index that contains secured content. E.g. If your web content is only accessed by the anonymous user it will add no benefit.
Pipeline
Add the entry in to the config
<contentSearch.getGlobalSearchFilters>
<processor type="Sitecore.ContentSearch.Pipelines.GetGlobalFilters.ApplyGlobalReadRolesFilter, Sitecore.ContentSearch" />
</contentSearch.getGlobalSearchFilters>
Implementation
Implement the pipeline filter to check the roles of the context user
public class ApplyGlobalReadRolesFilter : GetGlobalFiltersProcessor
{
public override void Process(GetGlobalFiltersArgs args)
{
var query = (IQueryable<SitecoreUISearchResultItem>)args.Query;
var userRoles = Context.User.Roles.Select(r => r.Name.Replace(#"\", #"\\"));
var predicate = PredicateBuilder.True<SitecoreUISearchResultItem>();
predicate = userRoles.Aggregate(predicate, (current, role) => current.Or(i => i["read_roles"].Contains(role)));
if(predicate.Body.NodeType != ExpressionType.Constant)
args.Query = query.Filter(predicate);
}
}
Summary
Create a ComputedField that returns a list of all valid roles for a given access right
Apply a pipeline processor to contentSearch.getGlobalSearchFilters to add a query filter to each search request.
Use the PredicateBuilder class to ensure the role names are OR'ed together
The big benefit here is that you take the hit at index time and the handling of item restriction is handled through a search query as normal. No need to worry about the facet numbers or search counts being incorrect.
You can restrict the roles you are checking to compute the field and you can vary the application of the pipeline filter. You can even take out the pipeline filter and just update your queries to filter when you require it.
NOTE The biggest problem with this set up is the requirement to re-index your content when security restrictions change. Should you be applying security restrictions to users themselves, you'll have to include additional computed fields.
Edit 02/06/2013
I was just tinkering with this in a project and realised that it was AND'ing the roles in the query. If a user had multiple roles assigned then both roles would have to have declared rights to the item. I've updated the pipeline processor to use the PredicateBuilder class to OR the roles. A check is also added to ensure the predicate is not a constant, this ensures the query is updated only if we have a filter to apply.
After some more searching around, the Linq to Sitecore article pointed me to the following lines of code:
var index = SearchManager.GetIndex("sitecore_master_index");
var context = index.CreateSearchContext(SearchSecurityOptions.EnableSecurityCheck))
Digging through Sitecore.ContentSearch.dll and Sitecore.ContentSearch.LuceneProvider.dll in dotPeek decompiler and the mention of the indexing.filterIndex.outbound pipeline in the Sitecore 7 Search document I found the following code:
Sitecore.ContentSearch.LuceneProvider.LuceneSearchReults
public IEnumerable<SearchHit<TElement>> GetSearchHits()
{
for (int idx = this.startIndex; idx <= this.endIndex; ++idx)
{
Document doc = this.context.Searcher.IndexReader.Document(this.searchHits.ScoreDocs[idx].Doc, (FieldSelector) this.fieldSelector);
if (!this.context.SecurityOptions.HasFlag((Enum) SearchSecurityOptions.DisableSecurityCheck))
{
string secToken = doc.GetField("_uniqueid").StringValue;
string dataSource = doc.GetField("_datasource").StringValue;
if (!string.IsNullOrEmpty(secToken))
{
bool isExcluded = OutboundIndexFilterPipeline.CheckItemSecurity(new OutboundIndexFilterArgs(secToken, dataSource));
if (!isExcluded)
yield return new SearchHit<TElement>(this.searchHits.ScoreDocs[idx].Score, this.configuration.IndexDocumentPropertyMapper.MapToType<TElement>(doc, this.selectMethod, this.virtualFieldProcessors, this.context.SecurityOptions));
}
}
else
yield return new SearchHit<TElement>(this.searchHits.ScoreDocs[idx].Score, this.configuration.IndexDocumentPropertyMapper.MapToType<TElement>(doc, this.selectMethod, this.virtualFieldProcessors, this.context.SecurityOptions));
}
}
Sitecore.ContentSearch.Pipelines.IndexingFilters
public class ApplyOutboundSecurityFilter : OutboundIndexFilterProcessor
{
public override void Process(OutboundIndexFilterArgs args)
{
if (args.IndexableUniqueId == null || !(args.IndexableDataSource == "Sitecore"))
return;
ItemUri uri = new ItemUri(args.IndexableUniqueId);
if (args.AccessRight != AccessRight.ItemRead || Database.GetItem(uri) != null)
return;
args.IsExcluded = true;
}
}
So it looks like Sitecore 7 gives us the ability to filter the the search results using the security rights of the context user straight out of the box, albeit using a very similar method of checking the item read permissions that Mark Cassidy suggested. Which is good news, since if this is a requirement for Sitecore 6 implementations then we can easily update the Advanced Database Crawler to do the same thing.
I'm still not convinced on the performance of this though, given that Sitecore 7 comes with Item Buckets and the possibility of storing millions of items. It should be possible to create several indexes though, and only EnableSecurityCheck on indexes with security enabled content, but then we need to think about combining the results from several indexes for "global search" results and also take into acccount Boosting which will mean re-ordering the combined results.
well - your considerations seems quite on target.
The easy implementation is to check the item through the database lookup - but paging, facetting and other statistics will fail in that.
The approach we do is to index security tokens for roles - having an inclusion field for allow and an exclusion field for deny rights. You then need to build a query for this - and expand all roles in roles for the query.
There are two issues you might be facing.
One is very complex OR queries for all roles in roles memberships. Might be less then well-performing.
The other is indexing congestion - as you will need to index major parts of the contentitems when security and permissions change as then are inherited.
The alternative is to use join queries - but normally that would be bad performing.
Sitecore developers made a silly mistake, it will never work, because of that statement:
if ((args.IndexableUniqueId != null) && (args.IndexableDataSource == "Sitecore"))
as args.IndexableDataSource will always be equal to "sitecore" not "Sitecore".
I'm currently upgrading big project to latest 7.2 Update and found out that silly mistake, oh Sitecore Devs usual mistakes :)

MVC 3 EF Code-first to webhost database trouble

Im fairly new to ASP.NET MVC 3, and to coding in general really.
I have a very very small application i want to upload to my webhosting domain.
I am using entity framework, and it works fine on my local machine.
I've entered a new connection string to use my remote database instead however it dosen't really work, first of all i have 1 single MSSQL database, which cannot be de dropped and recreated, so i cannot use that strategy in my initializer, i tried to supply null in the strategy, but to no avail, my tables simply does not get created in my database and thats the problem, i don't know how i am to do that with entity framework.
When i run the application, it tries to select the data from the database, that part works fine, i just dont know how to be able to create those tabes in my database through codefirst.
I could probaly get it to work through manually recreating the tables, but i want to know the solution through codefirst.
This is my initializer class
public class EntityInit : DropCreateDatabaseIfModelChanges<NewsContext>
{
private NewsContext _db = new NewsContext();
protected override void Seed(NewsContext context)
{
new List<News>
{
new News{ Author="Michael Brandt", Title="Test News 1 ", NewsBody="Bblablabalblaaaaa1" },
new News{ Author="Michael Brandt", Title="Test News 2 ", NewsBody="Bblablabalblaaaaa2" },
new News{ Author="Michael Brandt", Title="Test News 3 ", NewsBody="Bblablabalblaaaaa3" },
new News{ Author="Michael Brandt", Title="Test News 4 ", NewsBody="Bblablabalblaaaaa4" },
}.ForEach(a => context.News.Add(a));
base.Seed(context);
}
}
As i said, im really new to all this, so excuse me, if im lacking to provide the proper information you need to answer my question, just me know and i will answer it
Initialization strategies do not support upgrade strategies at the moment.
Initialization strategies should be used to initialise a new database. all subsequent changes should be done using scripts at the moment.
the best practice as we speak is to modify the database with a script, and then adjust by hand the code to reflect this change.
in future releases, upgrade / migration strategies will be available.
try to execute the scripts statement by statement from a custom IDatabaseInitializer
then from this you can read the database version in the db and apply the missing scripts to your database. simply store a db version in a table. then level up with change scripts.
public class Initializer : IDatabaseInitializer<MyContext>
{
public void InitializeDatabase(MyContext context)
{
if (!context.Database.Exists() || !context.Database.CompatibleWithModel(false))
{
context.Database.Delete();
context.Database.Create();
var jobInstanceStateList = EnumExtensions.ConvertEnumToDictionary<JobInstanceStateEnum>().ToList();
jobInstanceStateList.ForEach(kvp => context.JobInstanceStateLookup.Add(
new JobInstanceStateLookup()
{
JobInstanceStateLookupId = kvp.Value,
Value = kvp.Key
}));
context.SaveChanges();
}
}
}
Have you tried to use the CreateDatabaseOnlyIfNotExists
– Every time the context is initialized, database will be recreated if it does not exist.
The database initializer can be set using the SetInitializer method of the Database class.If nothing is specified it will use the CreateDatabaseOnlyIfNotExists class to initialize the database.
Database.SetInitializer(null);
-
Database.SetInitializer<NewsContext>(new CreateDatabaseOnlyIfNotExists<NewsContext>());
I'm not sure if this is the exact syntax as I have not written this in a while. But it should be very similar.
If you are using a very small application, you maybe could go for SQL CE 4.0.
The bin-deployment should allow you to run SQL CE 4.0 even if your provider doesn't have the binaries installed for it. You can read more here.
That we you can actually use whatever initializer you want, since you now don't have the problem of not being able to drop databases and delete tables.
could this be of any help?

Resources