I've been wracking my brains (such as they are) for a solution and can't suss it.
Let's say we have some code that starts a download with a [HttpResponseMessage] object. You get a task from $response.Content.CopyToAsync($downloadFileStream). You save that [Task] and you return, giving control back to the caller.
For various reasons the caller may choose to cancel the download. For that reason we use a cancellation token as well.
# '...' is unimportant detail
# HttpResponseMessage is IDisposable, as is the FileStream
$response = $httpClient.Send(...)
$downloadFileStream = [FileStream]::new(...)
$this._downloadTask = $response.Content.CopyToAsync($downloadFileStream, $cancellationToken)
return
Let's assume that after the download task is started some unhandled exception occurs, and the download class clean-up methods are never called. In that case the IDisposables (if) saved as class members are never explicitly disposed by the clean-up routines.
How then, do you ensure that both $response and $downloadFileStream are always disposed? No matter what happens? Ideally you want that to happen as soon as the download task completes.
$downloadTask.ContinueWith(...) appears to be a non-starter. [SynchronisationContext]::Current is null.
ThreadJob and multiple runspaces appear at first glance to be an option, and I've tried some of those approaches but ran into other issues, such as thread safety and passing .NET objects between runspaces.
Also I'm not sure if disposing an object in a thread other than the thread which instantiated it is considered bad form.
Ideally I'm looking for something like
$this._downloadTask = $response.Content.CopyToAsync($downloadFileStream, $cancellationToken)
.ContinueWith({$fileStream.Dispose(); $response.Dispose()}
return
But with the disposal happening on the main thread not in any worker thread? I think the /shrug emoji is appropriate right now. Spent far too long already thinking about this, and there's got to be some elegant and obvious solution. Please.
e: As requested, the basic downloader class outline.
class basicHttpDownloader
{
[Task] $_downloadTask
[void] NavigateAndBeginDownload($navigationOptions)
{
...
$response = $httpClient.Send(...)
$this.SetDownloadFilename($response)
$this._downloadTask = $response.Content.CopyToAsync($downloadFileStream,
$cancellationToken)
# looking for something like..
#.ContinueWith({$fileStream.Dispose(); $response.Dispose()}
return
# the caller is expected to call GetDownloadFilename(), verify
# that it wants to complete the download, then call either
# CancelDownload or FinishDownloadSaveTo
}
[string] GetDownloadFilename()
[void] CancelDownload()
[void] FinishDownloadSaveTo($folderPath)
}
e2: Tried this which also doesn't work :p
using namespace System.Threading.Tasks
$ErrorActionPreference = "Stop"
Set-StrictMode -Version Latest
$csharp = #"
using System;
using System.Threading;
using System.Management.Automation.Runspaces;
public class yoyo_sync : SynchronizationContext
{
public yoyo_sync() : base()
{
// parameterless
Console.WriteLine("yoyo_sync.ctor");
}
public override void Send(SendOrPostCallback d, object state)
{
Console.WriteLine("yoyo_sync.Send");
if(null == Runspace.DefaultRunspace) {
Runspace.DefaultRunspace = RunspaceFactory.CreateRunspace();
}
Console.WriteLine(Runspace.DefaultRunspace.Name);
base.Send(d, state);
}
public override void Post(SendOrPostCallback d, object state)
{
Console.WriteLine("yoyo_sync.Post");
if(null == Runspace.DefaultRunspace) {
Runspace.DefaultRunspace = RunspaceFactory.CreateRunspace();
}
Console.WriteLine(Runspace.DefaultRunspace.Name);
base.Post(d, state);
}
}
"#
$yoyo = Add-Type -PassThru -TypeDefinition $csharp
[runspace]::DefaultRunspace
[System.Threading.SynchronizationContext]::SetSynchronizationContext($yoyo::new())
$task = [Task]::FromResult(1).ContinueWith([System.Action[Task[int]]] {"blah" | Out-Host},
[TaskScheduler]::FromCurrentSynchronizationContext())
$task | Format-List
Related
I build my extbased TYPO3 extension in TYPO3 8.7 . It is a Backend-Module. In the controller, i write my own action to clone the object.
In this example, i want to clone/duplicate the object 'Campaign' and safe it with a modified title, like add the 'copy' text to the title.
But the new object should have also its own new child elements that must be exact copies.
When the action is called, i get only a copy of the Object, but no childs. Is there an example or best case how to handle this task? I did not find, even i found some questions and answers that are on the same topic, but older version. i hope that upd to date, there is a more straight forward solution. Thank you for every hint that points me to the right ideas and maybe an up to date and version example. Here is what i have i my controller. How do i implement recursiv copying of all child elements (and some childs have childs, too)?
/**
* action clone
* #param \ABC\Copytest\Domain\Model\Campaign $campaign
* #return void
* #var \ABC\Copytest\Domain\Model\Campaign $newCampaign
*/
public function cloneAction(\ABC\Copytest\Domain\Model\Campaign $campaign) {
$newCampaign = $this->objectManager->get("ABC\Copytest\Domain\Model\Campaign");
$properties = $campaign->_getProperties();
unset($properties['uid']);
foreach ($properties as $key => $value) {
$newCampaign->_setProperty($key, $value);
}
$newCampaign->_setProperty('title', $properties['title']. ' COPY');
$this->campaignRepository->add($newCampaign);
$this->addFlashMessage('Clone was created', '', \TYPO3\CMS\Core\Messaging\AbstractMessage::OK);
$this->redirect('list');
}
I am aware that this question has been answered a long time ago. But I want to provide my solution to create a deep copy for further reference. Tested on TYPO3 9.5.8.
private function deepcopy($object)
{
$clone = $this->objectManager->get(get_class($object));
$properties = \TYPO3\CMS\Extbase\Reflection\ObjectAccess::getGettableProperties($object);
foreach ($properties as $propertyName => $propertyValue) {
if ($propertyValue instanceof \TYPO3\CMS\Extbase\Persistence\ObjectStorage) {
$v = $this->objectManager->get(\TYPO3\CMS\Extbase\Persistence\ObjectStorage::class);
foreach($propertyValue as $subObject) {
$subClone = $this->deepcopy($subObject);
$v->attach($subClone);
}
} else {
$v = $propertyValue;
}
if ($v !== null) {
\TYPO3\CMS\Extbase\Reflection\ObjectAccess::setProperty($clone, $propertyName, $v);
}
}
return $clone;
}
There is one approach which tackles this usecase from a different POV, namely that request argument values without an identity are automatically put into fresh objects which can then be persisted. This basically clones the original objects. This is what you need to do:
Add a view which has fields for all properties of your object, hidden fields are fine too. This can for example be an edit view with a separate submit button to call your clone action.
Add a initializeCloneAction() and get the raw request arguments via $this->request->getArguments().
Now do unset($arguments[<argumentName>]['__identity']);, do the same for every relation your object has if you want copies instead of shared references.
Store the raw request arguments again via $this->request->setArguments($arguments).
Finally allow the creation of new objects in the property mapping configuration of your argument and possibly all relation properties.
This is how a full initializeCloneAction() could look like:
public function initializeCloneAction()
{
$arguments = $this->request->getArguments();
unset(
$arguments['campaign']['__identity'],
$arguments['campaign']['singleRelation']['__identity'],
);
foreach (array_keys($arguments['campaign']['multiRelation']) as $i) {
unset($arguments['campaign']['multiRelation'][$i]['__identity']);
}
$this->request->setArguments($arguments);
// Allow object creation now that we have new objects
$this->arguments->getArgument('campaign')->getPropertyMappingConfiguration()
->setTypeConverterOption(PersistentObjectConverter::class, PersistentObjectConverter::CONFIGURATION_CREATION_ALLOWED, true)
->allowCreationForSubProperty('singleRelation')
->getConfigurationFor('multiRelation')
->allowCreationForSubProperty('*');
}
Now if you submit your form using the clone action, your clone action will get a fully populated but new object which you can store in your repository as usual. Your cloneAction() will then be very simple:
public function cloneAction(Campaign $campaign)
{
$this->campaignRepository->add($campaign);
$this->addFlashMessage('Campaign was copied successfully!');
$this->redirect('list');
}
If you have "LazyLoadingProxy" instance in your object you need add one more conditions.
if ($propertyValue instanceof \TYPO3\CMS\Extbase\Persistence\Generic\LazyLoadingProxy) {
$objectStorage = $propertyValue->_loadRealInstance();
}
This is my solution for "deepcopy" function:
private function deepcopy($object)
{
$clone = $this->objectManager->get(get_class($object));
$properties = \TYPO3\CMS\Extbase\Reflection\ObjectAccess::getGettableProperties($object);
foreach ($properties as $propertyName => $propertyValue) {
if ($propertyValue instanceof \TYPO3\CMS\Extbase\Persistence\ObjectStorage) {
$objectStorage = $this->objectManager->get(\TYPO3\CMS\Extbase\Persistence\ObjectStorage::class);
foreach ($propertyValue as $subObject) {
$subClone = $this->deepcopy($subObject);
$objectStorage->attach($subClone);
}
} elseif ($propertyValue instanceof \TYPO3\CMS\Extbase\Persistence\Generic\LazyLoadingProxy) {
$objectStorage = $propertyValue->_loadRealInstance();
} else {
$objectStorage = $propertyValue;
}
if ($objectStorage !== null) {
\TYPO3\CMS\Extbase\Reflection\ObjectAccess::setProperty($clone, $propertyName, $objectStorage);
}
}
return $clone;
}
I think a good solution is, to emulate the backend-function.
See the code-example (german text)
http://blog.marcdesign.ch/2015/05/27/typo3-extbase-objekte-kopieren/
The general idea is to extend the TYPO3\CMS\Core\DataHandling\DataHandler and use the parent-method copyRecord. You declare your predefined backend-user to $this->BE_USER in your extend class. The obejct of your predefined backenduser can you get by using the class TYPO3\\CMS\\Backend\\FrontendBackendUserAuthentication and the known name of you predefined backenduser. Your user should have admin-rights and you should define the $BE_USER->uc_default['copyLevels']= '9999'; and declare $BE_USER->uc = $BE_USER->uc_default.
I have not checked, if the declaration $GLOBALS['PAGES_TYPES'][254]['allowedTables'] = '*'; is really needed.
The method copyRecorditself needs mainly the table-name, the uid-value, the pid-value and a language-object as parameters.The languages-object can you get $GLOBALS['lang'], which can although be generated by instanciating \TYPO3\CMS\Lang\LanguageService to $GLOBALS['lang'] and \TYPO3\CMS\Core\Charset\CharsetConverter to $GLOBALS['LANG']->csConvObj.
Sorry about my poor english.
I get strange errors when I am trying to pass around NSManagedObject through several functions. (all are in the same VC).
Here are the two functions in question:
func syncLocal(item:NSManagedObject,completionHandler:(NSManagedObject!,SyncResponse)->Void) {
let savedValues = item.dictionaryWithValuesForKeys([
"score",
"progress",
"player"])
doUpload(savedParams) { //do a POST request using params with Alamofire
(success) in
if success {
completionHandler(item,.Success)
} else {
completionHandler(item,.Failure)
}
}
}
func getSavedScores() {
do {
debugPrint("TRYING TO FETCH LOCAL SCORES")
try frc.performFetch()
if let results = frc.sections?[0].objects as? [NSManagedObject] {
if results.count > 0 {
print("TOTAL SCORE COUNT: \(results.count)")
let incomplete = results.filter({$0.valueForKey("success") as! Bool == false })
print("INCOMPLETE COUNT: \(incomplete.count)")
let complete = results.filter({$0.valueForKey("success") as! Bool == true })
print("COMPLETE COUNT: \(complete.count)")
if incomplete.count > 0 {
for pendingItem in incomplete {
self.syncScoring(pendingItem) {
(returnItem,response) in
let footest = returnItem.valueForKey("player") //only works if stripping syncScoring blank
switch response { //response is an enum
case .Success:
print("SUCCESS")
case .Duplicate:
print("DUPLICATE")
case .Failure:
print("FAIL")
}
}
} //sorry for this pyramid of doom
}
}
}
} catch {
print("ERROR FETCHING RESULTS")
}
}
What I am trying to achieve:
1. Look for locally saved scores that could not submitted to the server.
2. If there are unsubmitted scores, start the POST call to the server.
3. If POST gets 200:ok mark item.key "success" with value "true"
For some odd reason I can not access returnItem at all in the code editor - only if I completely delete any code in syncLocal so it looks like
func syncLocal(item:NSManagedObject,completionHandler:(NSManagedObject!,SyncResponse)->Void) {
completionHandler(item,.Success)
}
If I do that I can access .syntax properties in the returning block down in the for loop.
Weirdly if I paste the stuff back in, in syncLocal the completion block keeps being functional, the app compiles and it will be executed properly.
Is this some kind of strange XCode7 Bug? Intended NSManagedObject behaviour?
line 1 was written with stripped, line 2 pasted rest call back in
There is thread confinement in Core Data managed object contexts. That means that you can use a particular managed object and its context only in one and the same thread.
In your code, you seem to be using controller-wide variables, such as item. I am assuming the item is a NSManagedObject or subclass thereof, and that its context is just one single context you are using in your app. The FRC context must be the main thread context (a NSManagedObjectContext with concurrency type NSMainThreadConcurrencyType).
Obviously, the callback from the server request will be on a background thread. So you cannot use your managed objects.
You have two solutions. Either you create a child context, do the updates you need to do, save, and then save the main context. This is a bit more involved and you can look for numerous examples and tutorials out there to get started. This is the standard and most robust solution.
Alternatively, inside your background callback, you simply make sure the context updates occur on the main thread.
dispatch_async(dispatch_get_main_queue()) {
// update your managed objects & save
}
I have a big doubt. The problem is Out of Memory Exception in my class. But it seems something strange here. I have class in a dll.
public class MyClass : IDisposible
{
List<ClassA> a_classLists = new .....// new instance.
List<ClassB> b_classLists = new .....// new instance.
public string Method1(int IDValue)
{
// do here some web service call and get some XML data from it.
// parse the xml.
// Iterate through a for loop and add each node value to a_classLists
// Usually contains 10 or 15 items
Method2(); // from here calling another method
FinalSaveToDB(); // finally save the data to DB
return "";
}
private void Method2()
{
// do here some web service call and get some XML data from it.
// Iterate through a forloop.
// parse the xml. [large xml data. ie, image in binary format]
// For each loop add image binary data and other xml to b_classLists
// Usually it contains 50 or 60 such large lists.
}
private void FinalSaveToDB()
{
// using sqlbulkcopy, i am saving the data in the 2 lists to 2 different
// tables in the DB.
// Tab lock is mentioned in sqlbulk class.
// Actually 2 sqlbulkcopy class for 2 lists.
// Only 1 sql connection opens, then do the sqlbulkcopy. [there is no dataset or datareader]
// sqlconnection closes. I am using "using" clause for sqlconnection, bulkcopy etc
// these all are working fine.
}
private void Dispose()
{
// here nulling everything
// proxy null
// all the lists null....
}
}
This is the class I am instantiating 1000 times using reactive framework's Observable.Start
method as shown below...
private IObservable<string> SendEmpDetails(Employee emp)
{
using (MyClass p = new MyClass())
{
return Observable.Start(() => p.Method1(emp.ID), Scheduler.ThreadPool);
}
// here I hope it will call the Dispose and release all objects in the class.
}
// This EmployeeLists contains 1000 employee objects
EmployeeLists.ToObservable().Select(x => SendEmpDetails(x).Select(y => new { emp = x, retval = y }))
.Merge(10)
.ObserveOn(Scheduler.CurrentThread)
.Subscribe(x =>
{
SendStatus(x.retval.Item1, x.retval);
});
Even though, why i am getting out of memory exception ??? After starting the app, when it
process the 200th (or above) MyClass object, it throws error.
I forgot to mention 1 more thing, I am using VS 2010 and C# 4.0 (win7, 64 bit OS).
I need to log each activity. [ie, i need to understand the each and every process the app has gone through]. SO i declared a class [MyClass] level private string variable and assign each process details like "called this method", "got 5 records from this web service" etc.
logdata = Environment.Newline() + "This method has completed";
So the error is thrown here saying out of memory with some evalution failed.
So I turned off the string evaluation check box from Options in VS.
Again, there is no use.
So I changed the string to StringBuilder and tried to append the activity string each time.
Still no use. I dont understand what is the problem in it.
Is this because all the threads are working parallel, do they exchange the MyClass resources ??? Why the objects are not released ???
Please help me in this matter.
I grabbed System.Linq.Dynamic.DynamicQueryable from here:
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx
The issue that I am running into is in code that looks like this:
var results = dataContext.GetTable<MyClass>.Select("new (MyClassID, Name, Description)").Take(5);
It appears that if that line of code is executed by multiple threads near simultaneously, Microsoft's dynamic Linq code crashes in their ClassFactory.GetDynamicClass() method, which looks like this:
public Type GetDynamicClass(IEnumerable<DynamicProperty> properties)
{
rwLock.AcquireReaderLock(Timeout.Infinite);
try
{
Signature signature = new Signature(properties);
Type type;
if (!classes.TryGetValue(signature, out type))
{
type = CreateDynamicClass(signature.properties);
classes.Add(signature, type); // <-- crashes over here!
}
return type;
}
finally
{
rwLock.ReleaseReaderLock();
}
}
The crash is a simple dictionary error: "An item with the same key has already been added."
In Ms code, The rwLock variable is a ReadWriterLock class, but it does nothing to block multiple threads from getting inside classes.TryGetValue() if statement, so clearly, the Add will fail.
I can replicate this error pretty easily in any code that creates a two or more threads that try to execute the Select("new") statement.
Anyways, I'm wondering if anyone else has run into this issue, and if there are fixes or workarounds I can implement.
Thanks.
I did the following (requires .NET 4 or later to use System.Collections.Concurrent):
changed the classes field to a ConcurrentDictionary<Signature, Type> ,
removed all the ReaderWriterLock rwLock field and all the code referring to it,
updated GetDynamicClass to:
public Type GetDynamicClass(IEnumerable<DynamicProperty> properties) {
var signature = new Signature(properties);
return classes.GetOrAdd(signature, sig => CreateDynamicClass(sig.properties));
}
removed the classCount field and updated CreateDynamicClass to use classes.Count instead:
Type CreateDynamicClass(DynamicProperty[] properties) {
string typeName = "DynamicClass" + Guid.NewGuid().ToString("N");
...
I have been trying to follow this example (download the source code from a link on the site or here, but I keep running into an error that seems embedded in the example.
My procedure has been as follows (after installing the AppFabric SDK and other dependencies):
Download the source
Create a Service Namespace on the AppFabric.
Import the project into a new Windows Azure project with one Worker Role, make sure that it all compiles and that the default Worker Role Run() method starts and functions.
Configure the method GetInterRoleCommunicationEndpoint in InterRoleCommunicationExtension.cs with the ServiceNameSpace and IssuerSecret from my AppFabric Service Namespace (IssuerName and ServicePath stay default). This is a hard-wiring of my own parameters.
Copy/paste the initialization logic from the "SampleWorkerRole.cs" file in the demo into the OnStart() method of my project's Worker Role
Comment-out references to Tracemanager.* as the demo code does not have the Tracemanager methods implemented and they're not crucial for this test to work. There are about 7-10 of these references (just do a Find -> "Tracemanager" in entire solution).
Build successfully.
Run on local Compute Emulator.
When I run this test, during the initialization of a new InterRoleCommunicationExtension (the first piece of the inter-role communication infrastructure to be initialized, this.interRoleCommunicator = new InterRoleCommunicationExtension();), an error is raised: "Value cannot be null. Parameter name: contractType."
Drilling into this a bit, I follow the execution down to the following method in ServiceBusHostFactory.cs (one of the files from the sample):public static Type GetServiceContract(Type serviceType)
{
Guard.ArgumentNotNull(serviceType, "serviceType");
Type[] serviceInterfaces = serviceType.GetInterfaces();
if (serviceInterfaces != null && serviceInterfaces.Length > 0)
{
foreach (Type serviceInterface in serviceInterfaces)
{
ServiceContractAttribute serviceContractAttr = FrameworkUtility.GetDeclarativeAttribute<ServiceContractAttribute>(serviceInterface);
if (serviceContractAttr != null)
{
return serviceInterface;
}
}
}
return null;
}
The serviceType parameter's Name property is "IInterRoleCommunicationServiceContract," which is one of the classes of the demo, and which extends IObservable. The call to serviceType.GetInterfaces() returns the "System.IObservable`1" interface, which is then passed into FrameworkUtility.GetDeclarativeAttribute(serviceInterface);, which has the following code:
public static IList GetDeclarativeAttributes(Type type) where T : class
{
Guard.ArgumentNotNull(type, "type");
object[] customAttributes = type.GetCustomAttributes(typeof(T), true);
IList<T> attributes = new List<T>();
if (customAttributes != null && customAttributes.Length > 0)
{
foreach (object customAttr in customAttributes)
{
if (customAttr.GetType() == typeof(T))
{
attributes.Add(customAttr as T);
}
}
}
else
{
Type[] interfaces = type.GetInterfaces();
if (interfaces != null && interfaces.Length > 0)
{
foreach (object[] customAttrs in interfaces.Select(iface => iface.GetCustomAttributes(typeof(T), false)))
{
if (customAttrs != null && customAttrs.Length > 0)
{
foreach (object customAttr in customAttrs)
{
attributes.Add(customAttr as T);
}
}
}
}
}
return attributes;
}</code><br>
It is here that the issue arises. After not finding any customAttributes on the "IObservable1" type, it calls type.GetInterfaces(), expecting a return. Even though type is "System.IObservable1," this method returns an empty array, which causes the function to return null and the exception with the above message to be raised.
I am extremely interested in getting this scenario working, as I think the Publish/Subscribe messaging paradigm is the perfect solution for my application. Has anyone been able to get this demo code (from the AppFabric CAT Team itself!) working, or can spot my error? Thank you for your help.
Answered in the original blog post (see link below). Please advise if you are still experiencing problems. We are committed to supporting our samples on best effort basis.
http://blogs.msdn.com/b/appfabriccat/archive/2010/09/30/implementing-reliable-inter-role-communication-using-windows-azure-appfabric-service-bus-observer-pattern-amp-parallel-linq.aspx#comments