Is there a way to set the fileName prefix for the rolled over file?
I use date for log files, i.e: 21-06-2017.log
And have set the max file size to 10MB.
So when the file size reaches 10MB, rollover is performed and the current file is renamed to 21-06-2017.log.1 and so on...
I have a process which reads the files with extension .log and render it on UI
Also I'm using properties file and change the fileName programatically when my Spring loads.
Any idea how this can be achieved?
[EDIT]
Even if the RolledOver files are named as 21-06-2017_1.log, that would be fine. Also any type of prefix is acceptable, the only end criteria is that the file name should end with ".log"
Related
I have a process that periodically gets files from a server and copy them with SFTP to a local directory. It should not overwrite the file if it already exists. I know with something like Winston I can automatically rotate the log file when it fills up, but in this case I need a similar functionality to rotate files if they already exist.
An example:
The routine copies a remote file called testfile.txt to a local directory. The next time it's run the same remote file is found and copied. But now I want to rename the first testfile.txt to testfile.txt.0 so it's not overwritten. And so on - after a while I'd have a directory of files with the name testfile.txt.N and the most recent testfile.txt.
What you can do is you can append date and time on the file name that gives every filename a unique name and also helps you archive it.
For example you text.txt can be either 20170202_181921_test.txt or test_20170202_181921.txt
You can use a JavaScript Date Object to get date and time.
P.S show your code of downloading files so that I can add more to that.
I looked to TortoiseSVN and TortoiseHG... bit too much for me and I don't want to setup a structure where the files will be saved. All I want is:
When saving a file, first saves the file.
Then if there isn't in file's folder a sub-folder called "History", it creates it.
Then Copies the file to "History" and gives it the first available number suffix as "_[version]".
Then, if is no txt file, in the file's folder, called "[filename].[extension]_Notes.txt", it creates it and adds a line with:
Version: [version] " of the " [filename] " saved at: " [date]
Now how would this run from any application? Dunno. Maybe have a button in the file save dialog?
I'm asking this because I'm just one guy working and not on huge projects and 99% of the time I would just like to have a localized versioning, set on a local sub-folder and a log to track the versions and dates.
Is this super easy with Tortoise and I'm being a mule?
Cheers
Currently I am using file input plugin to go over my log archive but file input plugin is not the right solution for me because file input plugin inherently expects that file is stream of events and not as a static file. Now, this is causing a great deal of problem for me because my log archive has a 100,000 + log files and I logstash opens a handle on all these files which are never going to change.
I am facing following problems
1) Logstash fails with problem mentioned in SO
2) With those many open file handles log archival storage is getting very slow.
Does anybody know a way to let logstash know that treat files statically or once a file is processed do not keep file handle on it.
In logstash Jira bug, I was told to write my own plugin with some other suggestions which won't help me much.
Logstash file input can process static file. You need to add this configuration
file {
path => "/your/logs/path"
start_position => "beginning"
}
After adding the start_position, logstash reads the file from the beginning. Please refer here for more information. Remember that this option only modifies “first contact” situations where a file is new and not seen before. If a file has already been seen before, this option has no effect. Otherwise you have set your sincedb_path to /dev/null .
For the first question, I have answer in the comment. Please try to add the maximum file opened.
For my suggestion, You can try to write a script copy the log file to the logstash monitor path and move it out constantly. You have to estimate the time that logstash process a log file.
look out for this also turn on -v and --debug for logstash
{:timestamp=>"2016-05-06T18:47:35.896000+0530",
:message=>"_discover_file: /datafiles/server.log:
**skipping because it was last modified more than 86400.0 seconds ago**",
:level=>:debug, :file=>"filewatch/watch.rb", :line=>"330",
:method=>"_discover_file"}
solution is to touch the file or change the ignore_older setting
I have been using Node.JS file system module for performing various file related operations. I have a need to verify the file name if exists in a directory and if exists i would need to keep a suffix at the end of the file. Typically how windows does with duplicate file names..
if TestFile.txt already exists and another file with same names comes in during processing the new file should be renamed as TestFile (1).txt and next file with same name should be renamed as TestFile (2).txt.
What could be the best way to achieve this. Do i have to use a temporary array to keep all file names and traverse through for each? This is a multi threaded environment and there could be 50,000+ documents coming for processing.
Thanks a ton.
Is there a configuration in NLOG that accomplishes the following
1)A new log file should be created , when the current file exceeds a particular size for ex:- 5 MB
2)The old log files should be deleted after a configured amount of time period like for ex: - 1 day
You can find answer for your question (and examples) on page:
Size-based file archival - log files can be automatically archived by moving them to another location after reaching certain size and
Time-based file archival - log files can calso be automatically archived based on time
Try to use the second one and change log files every days. Than you can keep maximum number of archived files.