Getting an error creating a Query object in SubSonic - subsonic

I am getting the following error in one of our environments. It seems to occur when IIS is restarted, but we haven't narrowed down the specifics to reproduce it.
A DataTable named 'PeoplePassword' already belongs to this DataSet.
at System.Data.DataTableCollection.RegisterName(String name, String tbNamespace)
at System.Data.DataTableCollection.BaseAdd(DataTable table)
at System.Data.DataTableCollection.Add(DataTable table)
at SubSonic.SqlDataProvider.GetTableSchema(String tableName, TableType tableType)
at SubSonic.DataService.GetSchema(String tableName, String providerName, TableType tableType)
at SubSonic.DataService.GetTableSchema(String tableName, String providerName)
at SubSonic.Query..ctor(String tableName)
at Wad.Elbert.Data.Enrollment.FetchByUserId(Int32 userId)
Based on the stacktrace, I believe the error is happening on the second line of the method while creating the query object.
Please let me know if anyone else has this problem.
Thanks!
The code for the function is:
public static List<Enrollment> FetchByUserId(int userId)
{
List<Enrollment> enrollments = new List<Enrollment>();
SubSonic.Query query = new SubSonic.Query("Enrollment");
query.SelectList = "userid, prompt, response, validationRegex, validationMessage, responseType, enrollmentSource";
query.QueryType = SubSonic.QueryType.Select;
query.AddWhere("userId", userId);
DataSet dataset = query.ExecuteDataSet();
if (dataset != null &&
dataset.Tables.Count > 0)
{
foreach (DataRow dr in dataset.Tables[0].Rows)
{
enrollments.Add(new Enrollment((int)dr["userId"], dr["prompt"].ToString(), dr["response"].ToString(), dr["validationRegex"] != null ? dr["validationRegex"].ToString() : string.Empty, dr["validationMessage"] != null ? dr["validationMessage"].ToString() : string.Empty, (int)dr["responseType"], (int)dr["enrollmentSource"]));
}
}
return enrollments;
}

This is a threading issue.
Subsonic loads it's schema on the first call of SubSonic.DataService.GetTableSchema(...) but this is not Thread safe.
Let me demonstrate this with a little example
private static Dictionary<string, DriveInfo> drives = new Dictionary<string, DriveInfo>;
private static DriveInfo GetDrive(string name)
{
if (drives.Count == 0)
{
Thread.Sleep(10000); // fake delay
foreach(var drive in DriveInfo.GetDrives)
drives.Add(drive.Name, drive);
}
if (drives.ContainsKey(name))
return drives[name];
return null;
}
this explains well what happens, on the first call to this method the dictionary is empty
If that's the case the method will preload all drives.
For every call the requested drive (or null) is returned.
But what happens if you fire the method two times directly after the start? Then both executions try to load the drives in the Dictionary. The first one to add a drive wins the second will throw an ArgumentException (element already exists).
After the initial preload, everything works fine.
Long story short, you have two choices.
Modify subsonic source to make SubSonic.DataService.GetTableSchema(...) thread safe.
http://msdn.microsoft.com/de-de/library/c5kehkcz(v=vs.80).aspx
"Warmup" subsonic before accepting requests. The technic to achive this depends on your application design. For ASP.NET you have an Application_Start method that is only executed once during your application lifecycle
http://msdn.microsoft.com/en-us/library/ms178473(v=vs.100).aspx
So you can basically put a
var count = new SubSonic.Query("Enrollment").GetRecordCount();
in the method to force subsonic to init the table schema itself.

Related

What does the "check in memory" mean in Orchard CMS?

I tried to customize the queries executed by Orchard.ContentManagement.DefaultContentManager but the following peace of code *1 render my efforts useless:
class DefaultContentManager
{
...
public virtual ContentItem Get(int id, VersionOptions options, QueryHints hints) {
...
// implemention of the query comes here
...
*1 -> // no record means content item is not in db
if (versionRecord == null) {
// check in memory
var record = _contentItemRepository.Get(id);
if (record == null) {
return null;
}
versionRecord = GetVersionRecord(options, record);
if (versionRecord == null) {
return null;
}
}
The query is executed correctly and it does not return any data (which was my goal) but afterwards a second attempt *1 is executed to still get the content item.
Why is this part of code there? What is its purpose? Also why does the comment state check in memory and then the repository (DB table) is queried.
It's already been verified at this point that the item doesn't exist in the database, but it may have just been created from code during the same request. In that case, the nHibernate session has the item, but the database doesn't have it yet. The repository hits the session, not the db directly, so if it's there, it'll retrieve it, but that'll happen in memory.

What is the best way to recycle Domino objects in Java Beans

I use a function to get access to a configuration document:
private Document lookupDoc(String key1) {
try {
Session sess = ExtLibUtil.getCurrentSession();
Database wDb = sess.getDatabase(sess.getServerName(), this.dbname1);
View wView = wDb.getView(this.viewname1);
Document wDoc = wView.getDocumentByKey(key1, true);
this.debug("Got a doc for key: [" + key1 + "]");
return wDoc;
} catch (NotesException ne) {
if (this.DispLookupErrors)
ne.printStackTrace();
this.lastErrorMsg = ne.text;
this.debug(this.lastErrorMsg, "error");
}
return null;
}
In another method I use this function to get the document:
Document wDoc = this.lookupDoc(key1);
if (wdoc != null) {
// do things with the document
wdoc.recycle();
}
Should I be recycling the Database and View objects when I recycle the Document object? Or should those be recycled before the function returns the Document?
The best practice is to recycle all Domino objects during the scope within which they are created. However, recycling any object automatically recycles all objects "beneath" it. Hence, in your example method, you can't recycle wDb, because that would cause wDoc to be recycled as well, so you'd be returning a recycled Document handle.
So if you want to make sure that you're not leaking memory, it's best to recycle objects in reverse order (e.g., document first, then view, then database). This tends to require structuring your methods such that you do whatever you need to/with a Domino object inside whatever method obtains the handle on it.
For instance, I'm assuming the reason you defined a method to get a configuration document is so that you can pull the value of configuration settings from it. So, instead of a method to return the document, perhaps it would be better to define a method to return an item value:
private Object lookupItemValue(String configKey, itemName) {
Object result = null;
Database wDb = null;
View wView = null;
Document wDoc = null;
try {
Session sess = ExtLibUtil.getCurrentSession();
wDb = sess.getDatabase(sess.getServerName(), this.dbname1);
wView = wDb.getView(this.viewname1);
wDoc = wView.getDocumentByKey(configKey, true);
this.debug("Got a doc for key: [" + configKey + "]");
result = wDoc.getItemValue(itemName);
} catch (NotesException ne) {
if (this.DispLookupErrors)
ne.printStackTrace();
this.lastErrorMsg = ne.text;
this.debug(this.lastErrorMsg, "error");
} finally {
incinerate(wDoc, wView, wDb);
}
return result;
}
There are a few things about the above that merit an explanation:
Normally in Java, we declare variables at first use, not Table of Contents style. But with Domino objects, it's best to revert to TOC so that, whether or not an exception was thrown, we can try to recycle them when we're done... hence the use of finally.
The return Object (which should be an item value, not the document itself) is also declared in the TOC, so we can return that Object at the end of the method - again, whether or not an exception was encountered (if there was an exception, presumably it will still be null).
This example calls a utility method that allows us to pass all Domino objects to a single method call for recycling.
Here's the code of that utility method:
private void incinerate(Object... dominoObjects) {
for (Object dominoObject : dominoObjects) {
if (null != dominoObject) {
if (dominoObject instanceof Base) {
try {
((Base)dominoObject).recycle();
} catch (NotesException recycleSucks) {
// optionally log exception
}
}
}
}
}
It's private, as I'm assuming you'll just define it in the same bean, but lately I tend to define this as a public static method of a Util class, allowing me to follow this same pattern from pretty much anywhere.
One final note: if you'll be retrieving numerous item values from a config document, obviously it would be expensive to establish a new database, view, and document handle for every item value you want to return. So I'd recommend overriding this method to accept a List<String> (or String[ ]) of item names and return a Map<String, Object> of the resulting values. That way you can establish a single handle for the database, view, and document, retrieve all the values you need, then recycle the Domino objects before actually making use of the item values returned.
Here's an idea i'm experimenting with. Tim's answer is excellent however for me I really needed the document for other purposes so I've tried this..
Document doc = null;
View view = null;
try {
Database database = ExtLibUtil.getCurrentSessionAsSigner().getCurrentDatabase();
view = database.getView("LocationsByLocationCode");
doc = view.getDocumentByKey(code, true);
//need to get this via the db object directly so we can safely recycle the view
String id = doc.getNoteID();
doc.recycle();
doc = database.getDocumentByID(id);
} catch (Exception e) {
log.error(e);
} finally {
JSFUtils.incinerate(view);
}
return doc;
You then have to make sure you're recycling the doc object safely in whatever method calls this one
I have some temporary documents which exist for a while as config docs then no longer needed, so get deleted. This is kind of enforced by an existing Notes client application: they have to exist to keep that happy.
I've written a class which has a HashMap of Java Date, String and Doubles with the item name as the key. So now I have a serializable representation of the document, plus the original doc noteID so that it can be found quickly and amended/deleted when it's not needed anymore.
That means the config doc can be collected, a standard routine creates a map of all items for the Java representation, taking into account the item type. The doc object can then be recycled right away.
The return object is the Java class representation of the document, with getValue(String name) and setValue(String name, val) where val can be Double String or Date. NB: this structure has no need for rich text or attachments, so it's kept to simple field values.
It works well although if the config doc has lots of Items, it could mean holding a lot of info in memory unnecessarily. Not so in my particular case though.
The point is: the Java object is now serializable so it can remain in memory, and as Tim's brilliant reply suggests, the document can be recycled right away.

How do I force an object to release references when it's created by a new AppDomain and ConstructorInfo.Invoke?

Here's another one for releasing objects created by reflection:
We're working with a reporting tool (Active Reports 6) which creates a dll for each report.
We have lots of clients that use similar but still unique reports.
Reports are read through a web interface.
We run multiple sites, one for each client.
Our choices are:
1) Put all the reports in one big project which will be called by all the sites.
Cost: It will need to be recompiled every time we make a small change to any one report, potentially creating problems for all sites.
2) Create a whole bunch of similar little projects, with one for each site - let's say for sake of space that this creates problems, too.
3) Create a "Report Factory" which will use reflection to wire-up report dlls as needed.
We chose "3".
Problem: The final product works fine except for one thing: It won't release the report dll when done.
There is not currently a problem with the operation within a test environment, but if you try to do anything in the folder with the report dlls, you get the following error message: "This action can't be completed because the folder or a file in it is open in another program"
After research on this site and others, we realized that we needed an AppDomain for each call which can be cleanly unloaded.
After still having problems, we realized that the AppDomainSetup object needed to have a setting that allowed it to optimize for multiple users (LoaderOptimization.MultiDomain)
That didn't work.
Unfortunately, the base object (Active 6 report) can not be serialized, so we can't make a deep copy and chuck the original object.
After doing all of this, we're still experiencing problems.
Here is the code (C#):
private object WireUpReport(ReportArgs args)
{
//The parameter 'args' is a custom type (ReportArgs) which merely contains a
name/value pair collection.
object myReport = null;
string sPath = String.Empty;
string sFriendlyName = String.Empty;
sFriendlyName = System.Guid.NewGuid().ToString();
Assembly asmReport = null;
AppDomainSetup ads = null;
AppDomain adWireUp = null;
ConstructorInfo ci = null;
Type myReportType = null;
Type[] parametypes = null;
object[] paramarray = null;
object retObject = null;
try
{
//Get Report Object
sPath = GetWireUpPath(args); //Gets the path to the required dll; kept in a config file
//This parameter is used in an overloaded constructor further down
ads = new AppDomainSetup();
ads.ApplicationBase = Path.GetDirectoryName(sPath);
ads.LoaderOptimization = LoaderOptimization.MultiDomain;
adWireUp = AppDomain.CreateDomain(sFriendlyName, AppDomain.CurrentDomain.Evidence, ads);
asmReport = adWireUp.GetAssemblies()[0];
asmReport = Assembly.LoadFrom(sPath);
//Create parameters for wireup
myReportType = asmReport.GetExportedTypes()[0];
parametypes = new Type[1];
parametypes[0] = typeof(ReportArgs);
ci = myReportType.GetConstructor(parametypes);
paramarray = new object[1];
paramarray[0] = args;
//Instantiate object
myReport = ci.Invoke(paramarray);
return myReport;
}
catch (Exception ex)
{
throw ex;
}
finally
{
//Make sure Assembly object is released.
if (adWireUp != null)
{
AppDomain.Unload(adWireUp);
}
if (asmReport != null)
{
asmReport = null;
}
if (ads != null)
{
ads = null;
}
if (adWireUp != null)
{
adWireUp = null;
}
if (ci != null)
{
ci = null;
}
if (myReportType != null)
{
myReportType = null;
}
if (parametypes != null)
{
parametypes = null;
}
if (paramarray != null)
{
paramarray = null;
}
}
}
The object which is returned from this code is cast as type ActiveReports and then passed around our application.
Any help would be deeply appreciated. Thanks
Your code looks like you are seriously misunderstanding how to interact with a separate AppDomain.
Think of communicating with an AppDomain like talking to someone who's currently in another country. You know where they are, but you can't just walk over and talk to them. If you want them to do something for you, you have to open up a line of communication and tell them what you need.
The way that you open that line of communication is by defining a proxy object that can be created inside the other AppDomain and then cross the boundary back to your current AppDomain. Being able to cross the boundary requires that your object either be marked as [Serializable] or inherit from MarshalByRefObject. Because we actually want to talk to a reference in the other AppDomain and not just have a copy of it, we need the proxy to do the latter.
private class CrossDomainQuery : MarshalByRefObject
{
public void LoadDataFromAssembly(string assemblyPath)
{
var assembly = Assembly.LoadFrom(assemblyPath);
//TODO: Do something with your assembly
}
}
There is a method on the AppDomain called CreateInstanceAndUnwrap() that will create an instance of that communication object inside the other AppDomain and then hand you back a __TransparentProxy object that can be cast to the proxy type.
var crossDomainQuery = (CrossDomainQuery)adWireUp.CreateInstanceAndUnwrap(
typeof(CrossDomainQuery).Assembly.FullName,
typeof(CrossDomainQuery).FullName);
Once you have that proxy object, you can call methods on it and they will be invoked in the other AppDomain.
crossDomainQuery.LoadDataFromAssembly(assemblyPath);
So how is this different from what your current example code is doing?
Your current code does not actually execute anything useful inside the other AppDomain.
adWireUp = AppDomain.CreateDomain(sFriendlyName, AppDomain.CurrentDomain.Evidence, ads);
asmReport = adWireUp.GetAssemblies()[0];
asmReport = Assembly.LoadFrom(sPath);
This creates a new AppDomain, but then it loads all of the assemblies from that AppDomain into your current AppDomain. Additionally, it explicitly loads your report assembly into your current AppDomain.
Creating an AppDomain and calling methods on it doesn't mean that your code is executing inside of it any more than reading about another country means that you're now talking to someone inside it.
Even if you do create a proxy object and execute code inside that other AppDomain, there are a few things to be aware of.
1) Both AppDomains must be able to see the type used for the proxy, and you may have to handle AssemblyResolve events for either AppDomain manually (at least temporarily) to help resolve that.
2) AppDomains are fairly expensive to create. Generally, they are not used in situations where you need to spin something up really quickly, take some action and disappear. You should plan on either keeping them around as long as you can or be prepared to take the performance hit on every call.
3) You've said that the report type that you're instantiating is not serializable, and being able to serialize the object is a requirement for passing that type back from the other AppDomain. Defining a serializable class that can transport relevant data across the boundary and using that to pass the report data might be an option, but you'll have to determine if that works for your particular situation.
Also, as an aside, unless you have logic that depends on variables being set to null, setting everything to null in your finally does nothing useful and complicates your code.

Add or replace entity in Azure Table Storage

I'm working with Windows Azure Table Storage and have a simple requirement: add a new row, overwriting any existing row with that PartitionKey/RowKey. However, saving the changes always throws an exception, even if I pass in the ReplaceOnUpdate option:
tableServiceContext.AddObject(TableName, entity);
tableServiceContext.SaveChangesWithRetries(SaveChangesOptions.ReplaceOnUpdate);
If the entity already exists it throws:
System.Data.Services.Client.DataServiceRequestException: An error occurred while processing this request. ---> System.Data.Services.Client.DataServiceClientException: <?xml version="1.0" encoding="utf-8" standalone="yes"?>
<error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
<code>EntityAlreadyExists</code>
<message xml:lang="en-AU">The specified entity already exists.</message>
</error>
Do I really have to manually query for the existing row first and call DeleteObject on it? That seems very slow. Surely there is a better way?
As you've found, you can't just add another item that has the same row key and partition key, so you will need to run a query to check to see if the item already exists. In situations like this I find it helpful to look at the Azure REST API documentation to see what is available to the storage client library. You'll see that there are separate methods for inserting and updating. The ReplaceOnUpdate only has an effect when you're updating, not inserting.
While you could delete the existing item and then add the new one, you could just update the existing one (saving you one round trip to storage). Your code might look something like this:
var existsQuery = from e
in tableServiceContext.CreateQuery<MyEntity>(TableName)
where
e.PartitionKey == objectToUpsert.PartitionKey
&& e.RowKey == objectToUpsert.RowKey
select e;
MyEntity existingObject = existsQuery.FirstOrDefault();
if (existingObject == null)
{
tableServiceContext.AddObject(TableName, objectToUpsert);
}
else
{
existingObject.Property1 = objectToUpsert.Property1;
existingObject.Property2 = objectToUpsert.Property2;
tableServiceContext.UpdateObject(existingObject);
}
tableServiceContext.SaveChangesWithRetries(SaveChangesOptions.ReplaceOnUpdate);
EDIT: While correct at the time of writing, with the September 2011 update Microsoft have updated the Azure table API to include two upsert commands, Insert or Replace Entity and Insert or Merge Entity
In order to operate on an existing object NOT managed by the TableContext with either Delete or SaveChanges with ReplaceOnUpdate options, you need to call AttachTo and attach the object to the TableContext, instead of calling AddObject which instructs TableContext to attempt to insert it.
http://msdn.microsoft.com/en-us/library/system.data.services.client.dataservicecontext.attachto.aspx
in my case it was not allowed to remove it first, thus I do it like this, this will result in one transaction to server which will first remove existing object and than add new one, removing need to copy property values
var existing = from e in _ServiceContext.AgentTable
where e.PartitionKey == item.PartitionKey
&& e.RowKey == item.RowKey
select e;
_ServiceContext.IgnoreResourceNotFoundException = true;
var existingObject = existing.FirstOrDefault();
if (existingObject != null)
{
_ServiceContext.DeleteObject(existingObject);
}
_ServiceContext.AddObject(AgentConfigTableServiceContext.AgetnConfigTableName, item);
_ServiceContext.SaveChangesWithRetries();
_ServiceContext.IgnoreResourceNotFoundException = false;
Insert/Merge or Update was added to the API in September 2011. Here is an example using the Storage API 2.0 which is easier to understand then the way it is done in the 1.7 api and earlier.
public void InsertOrReplace(ITableEntity entity)
{
retryPolicy.ExecuteAction(
() =>
{
try
{
TableOperation operation = TableOperation.InsertOrReplace(entity);
cloudTable.Execute(operation);
}
catch (StorageException e)
{
string message = "InsertOrReplace entity failed.";
if (e.RequestInformation.HttpStatusCode == 404)
{
message += " Make sure the table is created.";
}
// do something with message
}
});
}
The Storage API does not allow more than one operation per entity (delete+insert) in a group transaction:
An entity can appear only once in the transaction, and only one operation may be performed against it.
see MSDN: Performing Entity Group Transactions
So in fact you need to read first and decide on insert or update.
You may use UpsertEntity and UpsertEntityAsync methods in the official Microsoft Azure.Data.Tables TableClient.
The fully working example is available at https://github.com/Azure-Samples/msdocs-azure-data-tables-sdk-dotnet/blob/main/2-completed-app/AzureTablesDemoApplicaton/Services/TablesService.cs --
public void UpsertTableEntity(WeatherInputModel model)
{
TableEntity entity = new TableEntity();
entity.PartitionKey = model.StationName;
entity.RowKey = $"{model.ObservationDate} {model.ObservationTime}";
// The other values are added like a items to a dictionary
entity["Temperature"] = model.Temperature;
entity["Humidity"] = model.Humidity;
entity["Barometer"] = model.Barometer;
entity["WindDirection"] = model.WindDirection;
entity["WindSpeed"] = model.WindSpeed;
entity["Precipitation"] = model.Precipitation;
_tableClient.UpsertEntity(entity);
}

Windows Azure: "An item with the same key has already been added." exception thrown on Select

I'm getting a strange error while trying to select a row from a table under Windows Azure Table Storage. The exception "An item with the same key has already been added." is being thrown even though I'm not inserting anything. The query that is causing the problem is as follows:
var ids = new HashSet<string>() { id };
var fields = new HashSet<string> {"#all"};
using (var db = new AzureDbFetcher())
{
var result = db.GetPeople(ids, fields, null);
}
public Dictionary<string, Person> GetPeople(HashSet<String> ids, HashSet<String> fields, CollectionOptions options)
{
var result = new Dictionary<string, Person>();
foreach (var id in ids)
{
var p = db.persons.Where(x => x.RowKey == id).SingleOrDefault();
if (p == null)
{
continue;
}
// do something with result
}
}
As you can see, there's only 1 id and the error is thrown right at the top of the loop and nothing is being modified.
However, I'm using "" as the Partition Key for this particular row. What gives?
You probably added an object with the same row key (and no partition key) to your DataServiceContext before performing this query. Then you're retrieving the conflicting object from the data store, and it can't be added to the context because of the collision.
The context tracks all object retrieved from the Tables. Since entities are uniquely identified by their partitionKey/rowKey combination, a context, like the tables, cannot contain duplicate partitionkey/rowkey combinations.
Possible causes of such a collison are:
Retrieving an entity, modifying it, and then retrieving it again using the same context.
Adding an entity to the context, and then retrieving one with the same keys.
In both cases, the context the encounters it's already tracking a different object which does however have the same keys. This is not something the context can sort out by itself, hence the exception.
Hope this helps. If you could give a little more information, that would be helpful.

Resources