We have an XPages application and we serialize all pages on disk for this specific application. We already use the gzip option but it seems the serialized files are removed from disk only when the http task is stopped or restarted.
As this application is used by many different customers from different places around the globe, we try to avoid restart the server or the http task as much as possible but the drawback is that serialized files are never deleted ans so sooner or later we face a disk space problem even if the gzip serialzed files are not that big.
A secondary issue is that the http task takes quite a long time to stop because it has to remove all the serialized files.
Is there any way to have the domino server "clean" old/unused serialized files without restarting the http task ?
Currently we implemented an OS script which cleans serialized files older than tow days which is fine, but I would prefer a solution within domino.
Thanks in advance for your answers/suggestions !
Renaud
I believe the httpSessionId is used to store the file(s) on disk. You could try the following:
Alter the xsp.persistence.dir.xspstate to a friendlier location on (i.e. /temp/xspstate)
Register a SessionListener with your XPage application
Inside the SessionListener's sessionDestroyed method recursively search through the folders to find the one file or folder that matches the sessionId and delete
When the sessionDestoryed method is called in the listener any file locks should have been removed. Also note, as of right now, the seesionDestroyed method is not called right after a user logs out (see my question here: SessionListener sessionDestroyed not called)
hope this helps...
Related
Apologies Paul, this is a duplicate to the post I put on OpenNTF, however the site will not allow me to log in the last 2 days to follow up, plus the wider audience of Stack might find me someone with an identical issue.
To keep it short.
I have 1 openLog database in a folder structure, logs/xpageslog.nsf
During development, I could log to this database, for example, using Paul Withers XPages OpenLog Logger, to log uncaught exceptions with the following settings:
private String logDbName = "logs\\xpageslog.nsf"; // in OpenLogItem.java from OpenLogClass library
logDbName = "logs/xpages.nsf" // in OpenLogFunctions.ls
xsp.openlog.filepath=log/xpageslog.nsf // in xsp.properties
However, if I then change all the above, to simply go to xpageslog.nsf, in the root of the server (this is a 2nd openLog database) errors still get logged to the first database.
I've tried building, cleaning, re-compiling, all to no avail. It seem's to be that somewhere, or somehow, the references to the original database are not being overwritten.
Any ideas?
It is good practice to use restart task http instead of tell http restart. Both commands have different effects.
As confirmed in comments, this solved the problem.
Some use tell http quit followed by load http, the effect is the same as with restart task http. At the other hand, simple tell http restart does not fully initialize http task, it's kind of soft reset and I recommend not using it.
Somehow I think there's a problem with stat in linux, but I tested a regular empty folder in linux vs an icon of less than 1000 bytes in size. The test was done with Apache 2.2 and the server location is in east canada.
Webpagetest results:
http://www.webpagetest.org/result/160202_KK_9HJ/1/details/
I'm curious as to why the time to first byte for the directory listing is higher than the time to first byte for the icon by one third?
What settings do I use in linux to fix this?
The time to first byte represents the time taken to 1) send the request to the server, 2) process the request and 3) return at least some of the results from the server.
For similarly sized resources 1) and 3) should be the same so let's concentrate on 2) for now.
When you request the directory, Apache has to check if the directory contains an index.html file, if not it reads the directory, then it starts to construct the HTML page creating links to the parent directory and each file/sub directory in the directory, then it has to return the file.
When you request the ico file Apache just has to pick up the file and return it to you nice and simple.
So as you can see there is more work in the first than in the second. So I don't think this is a fair test. Compare a static index.html file to a static ico file for a fairer test and then you'll know if you have an issue.
Additionally, depending on your mpm choice, settings, server load and server history there may be a thread or process started up waiting to process the first request (fast) or the first request may have to initiate one to handle this request (slow). This is likely to be less of an issue for a second request, particularly with keep-alive enabled. See here for more details: https://serverfault.com/questions/383526/how-do-i-select-which-apache-mpm-to-use.
There is also the TCP slow start issue, which particularly affects older version of OS and software, but that is unlikely to have an impact here in the small loads you are talking about and also should affect total download time rather than TTFB. Still it's yet another reason to ensure you're running up to date software.
And finally your TTFB is mostly influenced by your hosting provider and the pipes to your server and number of hops until
It gets to Apache so, once you have chosen a hosting provider, it is mostly out of your control. Again this will usually be reflective across the board and rather than the variances you see between two requests here.
I am using Spring Integration.
In the Remote FTPS server, same file will be updated every time. Source system is not ready to create new file every time.
for everyupdate of the file. i need to download file and process it.
Require help to create filter to listen to every file update.
Use an FtpPersistentAcceptOnceFileListFilter in the filter attribute and a FileSystemPersistentAcceptOnceFileListFilter in the local-filter attribute.
These filters, as well as persisting state so it lives beyond the current execution, also compare the modified time.
However, you need to be very careful updating a file server-side, rather than replacing it with a new one for each update. It is entirely possible (and even likely) to fetch a partially updated file (fetch the file before the writer has finished updating it).
I am building an app that downloads files and it keeps some metadata related to the file in core data.
I was very intrigued by the NSURLSession download task as it will allow me to download in the background and not have to write my own queueing mechanism.
My problem is when I get the callback
- (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)downloadTask didFinishDownloadingToURL:(NSURL *)location {
What pattern or method should I use to update the associated metadata for this file after moving it to the application sandbox? Specifically, I need to set a property on the metadata that will tell the application that the file is already downloaded.
I had originally started down the path of adding a property to the download task via associated objects that will tell me the objectid of the core data object. but it started to seem very hacky and it seems like there should be a simpler method.
Does anyone have an idea? am I explaining the problem sufficiently?
Your callback contains the original NSURL. Store the URL as part of your metadata. When the download completes, fetch the metadata record for that URL and update it.
I have an indexing function named "Execute()" using IndexWriter to index my site's content. It works great if I simply called it from a web page, but failed when I have it as a delegate parameter into System.Threading.Thread. Strangely though, it always work on my local dev machine, it only fails when I uploads to a shared host.
This is the error message I got
"Lock obtain timed out: SimpleFSLock error...."
Below is the failed code (but only fails on a shared host)
Scheduler scheduler = new Scheduler();
System.Threading.Thread schedulerThread = new System.Threading.Thread(scheduler.Execute);
Below is the code that works (work both on my local machine and on shared host)
Scheduler scheduler = new Scheduler();
schedulre.Execute();
Now, some ppl said, it could be a bad left over from the previous debugging session, so before I instantiated the IndexWriter, I did
if (IndexReader.IsLocked(indexingFolder))
{
log.Debug("it is locked");
IndexReader.Unlock(FSDirectory.GetDirectory(indexingFolder));
}
else
{
log.Debug("it is not locked");
}
and guess what? my log says, it is not locked.
So now I'm pretty sure it's caused by the System.Thread.Threading, but I just have no clue as to how to fix it.
Thanks
Check that on the shared host, the thread has the same permissions to the index folder as you do on the development machine/shared host.
Update: You can find what Principal the thread is running under by interrogating the thread's CurrentPrincipal property. Though this is a read-write property, you may not have the permissions to set this property in your shared-host environment.
You might find this post helpful.
Thanks everyone and especially to Vinay for pointing me in the right direction. After much tracing, i finally decided to take a look at the source and see what's there.
In "IndexWriter", you have
Lock #lock = this.directory.MakeLock("write.lock");
if (!#lock.Obtain(this.writeLockTimeout))
which is pointed to the SimpleFSLock implementation. The culprit was
new FileStream(this.lockFile.FullName, FileMode.CreateNew).Close();
by creating a new thread, internally, it throws a system.unauthorizedaccessexception, according to msdn here
When starting a new thread, System.Security.Principal.WindowsIdentity.GetCurrent() returns the identity of the process, not necessarily the identity of the code that called Thread.Start(). This is important to remember when starting asynchronous delegates or threads in an impersonated ASP.NET thread.
If you are in ASP.NET and want the new thread to start with the impersonated WindowsIdentity, pass the WindowsIdentity to the ThreadStart method. Once in the ThreadStart method, call WindowsIdentity.Impersonate().
Such, I solved my issue by impersonate the IIS account running my application in "Execute()" function and all problems are resolved.
Thanks again to all.
Probably the worst one to try and answer this since I haven't used lucene / shared hosting, but SimpleFSLock sounds like it's locking the lucene index file by using an explicit lock file on the file system (not quite the same as locking in threading). I'd say check to make sure you have configured the proper file paths and that file permissions are set correctly.
Otherwise, hopefully someone more familiar with Lucene.net can answer.
I believe the problem is with a write lock file in the Lucene index directory.
Go and list the directory's files.
In Java Lucene, you would have seen a file named write.lock in the index directory,
meaning that the index was not properly closed last time (maybe a process was abruptly stopped). In Lucene.net, look for a similarly named empty file.
I believe the same mechanism will be used in Lucene.net.
Try finding that file, erasing it and restarting Lucene.net.