I have written a program that writes multiple files into a SharePoint-List using CSOM:
foreach (file in myFiles) {
UploadFileToSharePoint(file.dataq, context, url+ file.name);
}
UploadFileToSharePoint(dummy, context, url+"finish.txt");
public static void UploadFileToSharePoint(Byte[] fileStream, ClientContext cCtx, String destUrl)
{
using (MemoryStream stream = new MemoryStream(fileStream))
{
Microsoft.SharePoint.Client.File.SaveBinaryDirect(cCtx, destUrl, stream, true);
}
}
As you can see, when done uploading N files into a target folder, a last file called "finish.txt" is uploaded. Now I have an Event Receiver for that list checking if the last file is uploaded:
public override void ItemAdded(SPItemEventProperties properties)
{
try
{
if (properties.ListItem.FileSystemObjectType == SPFileSystemObjectType.File)
{
if (properties.ListItem.File != null && properties.ListItem.File.ParentFolder != null)
{
if (properties.ListItem.File.Name.Contains("finish.txt"))
{
// last file inside folder
AnalyzeFolder(properties.ListItem);
}
}
}
base.ItemAdded(properties);
}
Occasionally I receive an error
Error moving files: Microsoft.SharePoint.SPException: Save Conflict.
Your changes conflict with those made concurrently by another user. If
you want your changes to be applied, click Back in your Web browser,
refresh the page, and resubmit your changes. --->
System.Runtime.InteropServices.COMException: Save Conflict. Your
changes conflict with those made concurrently by another user.
My guess would be that this happens because the last file is uploaded before the uploads of the other files before are finished.
How can I make sure that all uploads are finished before I upload the last file to prevent that error?
This isn't exactly what I was wanting, but making the Event Receiver work synchronous by adding
<Synchronization>Synchronous</Synchronization>
to the elements.xml solved that issue.
(I won't mark this as an answer, because originally I did not want to have that event receiver synchronously, but to wait for the other events. So if someone has a better answer: Feel free to post it)
Related
my scenario is simple:i made a game using cocos2d-x and i want to download images (FB and Google play) for multi player users and show them once the download is done as texture for a button.
in ideal world, things work as expected.
things get tricky when those buttons got deleted before the download is done.
so the callback function is in weird state and then i get signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
and the app crashes
This is how i implmented it
I have a Layout class called PlayerIcon. the cpp looks like this
void PlayerIcon::setPlayer(string userName, string displayName, string avatarUrl){
try {
//some code here
downloadAvatar(_userName, _avatarUrl);
//some code here
}
catch(... ){
}
}
void PlayerIcon::downloadAvatar(std::string _avatarFilePath,std::string url) {
if(!isFileExist(_avatarFilePath)) {
try {
auto downloader = new Downloader();
downloader->onFileTaskSuccess=CC_CALLBACK_1(PlayerIcon::on_download_success,this);
downloader->onTaskError=[&](const network::DownloadTask& task,int errorCode,
int errorCodeInternal,
const std::string& errorStr){
log("error while saving image");
};
downloader->createDownloadFileTask(url,_avatarFilePath,_avatarFilePath);
}
catch (exception e)
{
log("error while saving image: test");
}
} else {
//set texture for button
}
}
void PlayerIcon::on_download_success(const network::DownloadTask& task){
_isDownloading = false;
Director::getInstance()->getScheduler()-> performFunctionInCocosThread(CC_CALLBACK_0(PlayerIcon::reload_avatar,this));
}
void PlayerIcon::reload_avatar(){
try {
// setting texture in UI thread
}
catch (...) {
log("error updating avatar");
}
}
As i said, things works fine until PlayerIcon is deleted before the download is done.
i dont know what happens when the call back of the download task point to a method of un object that s deleted (or flagged for deletion).
i looked in the downloader implementation and it doesn't provide any cancellation mechanism
and i'm not sure how to handle this
Also, is it normal to have 10% crash rate on google console for a cocos2dx game
any help is really appreciated
Do you delete de Downloader in de destructor of the PlayerIcon?
there is a destroy in the apple implementation witch is trigered by the destructor.
-(void)doDestroy
{
// cancel all download task
NSEnumerator * enumeratorKey = [self.taskDict keyEnumerator];
for (NSURLSessionDownloadTask *task in enumeratorKey)
{
....
DownloaderApple::~DownloaderApple()
{
DeclareDownloaderImplVar;
[impl doDestroy];
DLLOG("Destruct DownloaderApple %p", this);
}
In the demo code of cocos2d-x: DownloaderTest.cpp they use:
std::unique_ptr<network::Downloader> downloader;
downloader.reset(new cocos2d::network::Downloader());
instead of:
auto downloader = new Downloader();
It looks like you are building this network code as part of your scene tree. If you do a replaceScene/popScene...() call, while the async network software is running in the background, this will cause the callback to disappear (the scene will be deleted from the scene-stack) and you will get a SEGFAULT from this.
If this is the way you've coded it, then you might want to extract the network code to a global object (singleton) where you queue the requests and then grab them off the internet saving the results in the global-object's output queue (or their name and location) and then let the scene code check to see if the avatar has been received yet by inquiring on the global-object and loading the avatar sprite at this point.
Note, this may be an intermittent problem which depends on the speed of your machine and the network so it may not be triggered consistently.
Another solution ...
Or you could just set your function pointers to nullptr in your PlayerIcon::~PlayerIcon() (destructor):
downloader->setOnFileTaskSuccess(nullptr);
downloader->setOnTaskProgress(nullptr);
Then there will be no attempt to call your callback functions and the SEGFAULT will be avoided (Hopefully).
I am struggling a lot with this task. I have to download files from SFTP and then parse them. I am using Durable functions like this
[FunctionName("MainOrch")]
public async Task<List<string>> RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context, ILogger log)
{
try
{
var filesDownloaded = new List<string>();
var filesUploaded = new List<string>();
var files = await context.CallActivityAsync<List<string>>("SFTPGetListOfFiles", null);
log.LogInformation("!!!!FilesFound*******!!!!!" + files.Count);
if (files.Count > 0)
{
foreach (var fileName in files)
{
filesDownloaded.Add(await context.CallActivityAsync<string>("SFTPDownload", fileName));
}
var parsingTasks = new List<Task<string>>(filesDownloaded.Count);
foreach (var downlaoded in filesDownloaded)
{
var parsingTask = context.CallActivityAsync<string>("PBARParsing", downlaoded);
parsingTasks.Add(parsingTask);
}
await Task.WhenAll(parsingTasks);
}
return filesDownloaded;
}
catch (Exception ex)
{
throw;
}
}
SFTPGetListOfFiles: This functions connects to SFTP and gets the list of files in a folder and return.
SFTPDownload: This function is suppose to connect to SFTP and download each file in Azure Function's Tempt Storage. and return the download path. (each file is from 10 to 60 MB)
[FunctionName("SFTPDownload")]
public async Task<string> SFTPDownload([ActivityTrigger] string name, ILogger log, Microsoft.Azure.WebJobs.ExecutionContext context)
{
var downloadPath = "";
try
{
using (var session = new Session())
{
try
{
session.ExecutablePath = Path.Combine(context.FunctionAppDirectory, "winscp.exe");
session.Open(GetOptions(context));
log.LogInformation("!!!!!!!!!!!!!!Connected For Download!!!!!!!!!!!!!!!");
TransferOptions transferOptions = new TransferOptions();
transferOptions.TransferMode = TransferMode.Binary;
downloadPath = Path.Combine(Path.GetTempPath(), name);
log.LogInformation("Downloading " + name);
var transferResult = session.GetFiles("/Receive/" + name, downloadPath, false, transferOptions);
log.LogInformation("Downloaded " + name);
// Throw on any error
transferResult.Check();
log.LogInformation("!!!!!!!!!!!!!!Completed Download !!!!!!!!!!!!!!!!");
}
catch (Exception ex)
{
log.LogError(ex.Message);
}
finally
{
session.Close();
}
}
}
catch (Exception ex)
{
log.LogError(ex.Message);
_traceService.TraceException(ex);
}
return downloadPath;
}
PBARParsing: function has to get the stream of that file and process it (processing a 60 MB file might take few minutes on Scale up of S2 and Scale out with 10 instances.)
[FunctionName("PBARParsing")]
public async Task PBARParsing([ActivityTrigger] string pathOfFile,
ILogger log)
{
var theSplit = pathOfFile.Split("\\");
var name = theSplit[theSplit.Length - 1];
try
{
log.LogInformation("**********Starting" + name);
Stream stream = File.OpenRead(pathOfFile);
i want the download of all files to be completed using SFTPDownload thats why "await" is in a loop. and then i want parsing to run in parallel.
Question 1: Does the code in MainOrch function seems correct for doing these 3 things 1)getting the names of files, 2) downloading them one by one and not starting the parsing function until all files are downloaded. and then 3)parsing the files in parallel. ?
I observed that what i mentioned in Question 1 is working as expected.
Question 2: 30% of the files are parsed and for the 80% i see errors that "Could not find file 'D:\local\Temp\fileName'" is azure function removing the files after i place them ? is there any other approach i can take? If i change the path to "D:\home" i might see "File is being used by another process" error. but i haven't tried it yet. out the 68 files on SFTP weirdly last 20 ran and first 40 files were not found at that path and this is in sequence.
Question3: I also see this error " Singleton lock renewal failed for blob 'func-eres-integration-dev/host' with error code 409: LeaseIdMismatchWithLeaseOperation. The last successful renewal completed at 2020-08-08T17:57:10.494Z (46005 milliseconds ago) with a duration of 155 milliseconds. The lease period was 15000 milliseconds." does it tells something ? it came just once though.
update
after using "D:\home" i am not getting file not found errors
For others coming across this, the temporary storage is local to an instance of the function app, which will be different when the function scales out.
For such scenarios, D:\home is a better alternative as Azure Files is mounted here, which is the same across all instances.
As for the lock renewal error observed here, this issue tracks it but shouldn't cause issues as mentioned. If you do see any issue because of this, it would be best to share details in that issue.
I'm currently doing a FTP download using MFC. Is a very simple program which takes 2 inputs from user and click a download button in order to download from server. Everything is fine and im able to download it from. But i realized this program can only be executed once. Either successful or fail user has to open the .exe again to download another file. I'm a beginner in C&C++ with a simple knowledge i put OnInitDialog() at the last line of the download function hopping it will loop back and initialize again. Of course it doesn't work. Below are my current codes for the download button
BOOL CFTPDOWNLOADDlg::Log_In(char* path, char* ID, char* password {
m_pFtpConnection = NULL;
try{
// path
// ID
// password
m_pFtpConnection = m_Session.GetFtpConnection(path,
ID,password,INTERNET_INVALID_PORT_NUMBER);
}
catch(CInternetException *pEx){
pEx->ReportError(MB_ICONEXCLAMATION);
m_pFtpConnection = NULL;
pEx->Delete();
return FALSE;
}
return TRUE;
}
BOOL CFTPDOWNLOADDlg::Download(){
m_Edit3.SetWindowText("Downloading..");
m_Session.EnableStatusCallback(TRUE);
if(m_pFtpConnection->GetFile(serv_Loc,host_Loc,
FALSE,FILE_ATTRIBUTE_NORMAL,FTP_TRANSFER_TYPE_BINARY,1) != 0){
MessageBox("Download Complete");
m_Edit3.SetWindowText("");}
else{
MessageBox("Download Fail");
return FALSE;
}
// Log_out Session
m_Session.Close();
m_pFtpConnection->Close();
if(m_pFtpConnection!=NULL) delete m_pFtpConnection;
else MessageBox("Download Complete");
return TRUE;
}
BOOL CFTPDOWNLOADDlg::get_Path(){
...
...
...
sprintf(serv_Loc,"soft\\%s\\%d\\%s.zip",s_No,r_Number,r_No);
sprintf(host_Loc,"%s\\%s.zip",buff2,r_No);
return TRUE;
}
void CFTPDOWNLOADDlg::OnCancel() {
// Log_out Session
m_Session.Close();
m_pFtpConnection->Close();
if(m_pFtpConnection!=NULL)
delete m_pFtpConnection;
CDialog::OnCancel();
}
void CFTPDOWNLOADDlg::OnDLButton() {
//get path from user input
get_Path();
// start download
Download();
}
I've tried to search online, i couldn't find anything which is close. Sorry for my poor explanation.
Thank you in advance for your kindness in replying
Here is what you need to do:
You should make CInternetSession m_Session; a member of your CWinApp-derived class.
You should call m_Session.Close() in ExitInstance() method of your CWinApp-derived class.
In your CDialog-derived class you should only deal with CFtpConnection related stuff. So when user clicks on Download button you should call GetFtpConnection() and initialize your m_pFtpConnection and do the rest. When download/upload is done call m_pFtpConnection->Close(); and delete m_pFtpConnection;
Please also use CString instead of char*. There are lots of benefits like automatic UNICODE support, etc.
Please also consider using CString::Format() method instead of sprintf().
You should also consider using threads to perform upload/download tasks in a separate worker thread. Use AfxBeginThread() to start the thread. This way you'll not affect Windows message pump that is a part of main application (GUI) thread. So your GUI wont lock up while you uploading/downloading files.
I am using LWUIT ResrouceEditor(latest SVN code revision 1513) to generate a UI State machine.
I want to show a wait screen when a long running command is invoked by a user using a button on the current form. I believe I can use the asynchronous option when linking the command on the button. I have setup a form in which I have a button which should invoke the asynchronous command. In command selection for that button, I have set the action to show the wait screen form and have marked the command as asynchronous. However when I use the asynchronous option, the code shows the wait screen, but after that it throws a NullPointerException.
As per my understanding, once you mark a command as asynchronous, it will call the following methods from a different thread where you can handle its processing.
protected void asyncCommandProcess(Command cmd, ActionEvent sourceEvent);
protected void postAsyncCommand(Command cmd, ActionEvent sourceEvent);
However this methods are not getting called and it throws a NullPointerException.
When I looked at the LWUIT code, in UIBuilder.java(lineno. 2278), I see that it constructs the new thread for an asynchronous command as follows:
new Thread(new FormListener(currentAction, currentActionEvent, f)).start();
But when running it through Debugger I see that currentAction and currentActionEvent are always null. And hence when the FormListener thread starts running, it never calls the above two async command processing methods. Please see the listing of the run() method in the UIBuilder.java(line no. 2178)
public void run() {
if(currentAction != null) {
if(Display.getInstance().isEdt()) {
postAsyncCommand(currentAction, currentActionEvent);
} else {
asyncCommandProcess(currentAction, currentActionEvent);
// wait for the destination form to appear before moving back into the LWUIT thread
waitForForm(destForm);
}
} else {
if(Display.getInstance().isEdt()) {
if(Display.getInstance().getCurrent() != null) {
exitForm(Display.getInstance().getCurrent());
}
Form f = (Form)createContainer(fetchResourceFile(), nextForm);
beforeShow(f);
f.show();
postShow(f);
} else {
if(processBackground(destForm)) {
waitForForm(destForm);
}
}
}
}
In the above method, since the currentAction is null, it always goes into the else statement and since the nextForm is also null, it causes the NullPointerException.
On further look at the UIBuilder.java code, I noticed what is causing the NullPointer exception. It seems when the FormListner is created, it is passed currentAction and currentActionEvent, however they are null at that time. Instead the code should be changed as follows(starting at line 2264):
if(action.startsWith("#")) {
action = action.substring(1);
Form currentForm = Display.getInstance().getCurrent();
if(currentForm != null) {
exitForm(currentForm);
}
Form f = (Form)createContainer(fetchResourceFile(), action);
beforeShow(f);
/* Replace following with next lines for fixing asynchronous command
if(Display.getInstance().getCurrent().getBackCommand() == cmd) {
f.showBack();
} else {
f.show();
}
postShow(f);
new Thread(new FormListener(currentAction, currentActionEvent, f)).start();
*/
new Thread(new FormListener(cmd, evt, f)).start();
return;
}
Can lwuit development team take a look at the above code, review and fix it. After I made the above change, the asynchronous command processing methods were invoked.
Thank you.
Thanks for the information, its probably better to use the issue tracker for things like this (at http://lwuit.java.net).
I will make a similar change although I don't understand why you commented out the form navigation portion.
To solve your use case of a wait screen we have a much simpler solution: Next Form. Just show the wait screen and in it define the "Next Form" property.
This will trigger a background thread to be invoked (processBackground callback) and only when the background thread completes the next form will be shown.
So I was writing a quick application to sort my wallpapers neatly into folders according to aspect ratio. Everything is going smoothly until I try to actually move the files (using FileInfo.MoveTo()). The application throws an exception:
System.IO.IOException
The process cannot access the file because it is being used by another process.
The only problem is, there is no other process running on my computer that has that particular file open. I thought perhaps that because of the way I was using the file, perhaps some internal system subroutine on a different thread or something has the file open when I try to move it. Sure enough, a few lines above that, I set a property that calls an event that opens the file for reading. I'm assuming at least some of that happens asynchronously. Is there anyway to make it run synchronously? I must change that property or rewrite much of the code.
Here are some relevant bits of code, please forgive the crappy Visual C# default names for things, this isn't really a release quality piece of software yet:
private void button1_Click(object sender, EventArgs e)
{
for (uint i = 0; i < filebox.Items.Count; i++)
{
if (!filebox.GetItemChecked((int)i)) continue;
//This calls the selectedIndexChanged event to change the 'selectedImg' variable
filebox.SelectedIndex = (int)i;
if (selectedImg == null) continue;
Size imgAspect = getImgAspect(selectedImg);
//This is gonna be hella hardcoded for now
//In the future this should be changed to be generic
//and use some kind of setting schema to determine
//the sort/filter results
FileInfo file = ((FileInfo)filebox.SelectedItem);
if (imgAspect.Width == 8 && imgAspect.Height == 5)
{
finalOut = outPath + "\\8x5\\" + file.Name;
}
else if (imgAspect.Width == 5 && imgAspect.Height == 4)
{
finalOut = outPath + "\\5x4\\" + file.Name;
}
else
{
finalOut = outPath + "\\Other\\" + file.Name;
}
//Me trying to tell C# to close the file
selectedImg.Dispose();
previewer.Image = null;
//This is where the exception is thrown
file.MoveTo(finalOut);
}
}
//The suspected event handler
private void filebox_SelectedIndexChanged(object sender, EventArgs e)
{
FileInfo selected;
if (filebox.SelectedIndex >= filebox.Items.Count || filebox.SelectedIndex < 0) return;
selected = (FileInfo)filebox.Items[filebox.SelectedIndex];
try
{
//The suspected line of code
selectedImg = new Bitmap((Stream)selected.OpenRead());
}
catch (Exception) { selectedImg = null; }
if (selectedImg != null)
previewer.Image = ResizeImage(selectedImg, previewer.Size);
else
previewer.Image = null;
}
I have a long-fix in mind (that's probably more efficient anyway) but it presents more problems still :/
Any help would be greatly appreciated.
Since you are using your selectedImg as a Class scoped variable it is keeping a lock on the File while the Bitmap is open. I would use an using statement and then Clone the Bitmap into the variable you are using this will release the lock that Bitmap is keeping on the file.
Something like this.
using ( Bitmap img = new Bitmap((Stream)selected.OpenRead()))
{
selectedImg = (Bitmap)img.Clone();
}
New answer:
I looked at the line where you do an OpenRead(). Clearly, this locks your file. It would be better to provide the file path instead of an stream, because you can't dispose your stream since bitmap would become erroneous.
Another thing I'm looking in your code which could be a bad practice is binding to FileInfo. Better create a data-transfer object/value object and bind to a collection of this type - some object which has the properties you need to show in your control -. That would help in order to avoid file locks.
In the other hand, you can do some trick: why don't you show streched to screen resolution images compressing them so image size would be extremly lower than actual ones and you provide a button called "Show in HQ"? That should solve the problem of preloading HD images. When the user clicks "Show in HQ" button, loads that image in memory, and when this is closed, it gets disposed.
It's ok for you?
If I'm not wrong, FileInfo doesn't block any file. You're not opening it but reading its metadata.
In the other hand, if you application shows images, you should move to memory visible ones and load them to your form from a memory stream.
That's reasonable because you can open a file stream, read its bytes and move them to a memory stream, leaving the lock against that file.
NOTE: This solution is fine for not so large images... Let me know if you're working with HD images.
using(selectedImg = new Bitmap((Stream)selected))
Will that do it?