Log Search terms/keywords in Solr - search

I developed a search tool using Solr; I want to look at the search terms used by other people when they search. Please suggest me a way.
This might be a duplicate of this.
However, the Solr Jetty ./logs/*request.log files are empty in my case which I am not sure why.

I am using SOLR4.4.0 with default Jetty setup. I can see following lines in solr.log file under logs folder.
INFO - 2013-10-01 12:33:08.319; org.apache.solr.core.SolrCore; [collection1] webapp=/solr path=/terms params={timestamp=1380610985478&limit=10&terms.fl=name&q=:&wt=velocity&v.template=suggest &terms.sort=count&terms.prefix=:} status=0 QTime=1069
INFO - 2013-10-01 12:33:12.027; org.apache.solr.core.SolrCore; [collection1] webapp=/solr path=/browse params={q=:} hits=28 status=0 QTime=4796
Hope This Helps!!!!

Related

who created/coded ext4lazyinit for ext4?

I have been trying to figure out who the author of ext4lazyinit ? I tried searching on lwn.net, didn't get anything. I tried using -
┌─[shirish#debian] - [/usr/share/doc/linux-image-4.9.0-1-amd64] - [10080]
└─[$] zless changelog.linux.gz
I tried searching for pattern inside the changelog but found no mention on it on the changelog even though the changelog.linux.gz goes all the way to 2005.
Can anybody share/explain where can I get more details please ?
Lukas Czerner - i used google.
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=bfff68738f1cb5c93dab1114634cea02aae9e7ba

Rsyslogs new configuration format for input files

I'm configuring a centralized logging with rsyslog.
I have to specify a input-file with some kind of WildCard but can't find any examples of how to get it working, in the description of the official documentation here a link with an exact description seems broken.
I try to log tomcat7-logfiles that look like localhost_access_log.2015-07-15.txt.
The date in the file updates every day.
What I want to get is some kind of input(type="imfile" ...)
I tried it with:
input(type="imfile" tag="access_log" statefile="tomcat-access-log"
file="/var/log/tomcat7/localhost_access_log.*.txt")
but this is not working and I don't get what I'm doing wrong.
Here is my full code:
$ModLoad imfile
$PrivDropToGroup adm
$WorkDirectory /var/spool/rsyslog
# catalina.log
$InputFileName /var/log/tomcat7/catalina.log
$InputFileTag catalina-log
$InputFileStateFile stat-catalina-log
$InputFileSeverity info
$InputRunFileMonitor
# localhost_access_log.YYYY.MM.DD.txt
input(type="imfile" tag="access_log" statefile="tomcat-access-log" file="/var/log/tomcat7/localhost_access_log.*.txt")
The catalina-logs are working as they are supposed to, however I'm not getting any access-logs in my output.
Any help would be appreciated, please remind me if I'm doing something completely wrong or if there is a better way to do this.
It will only work as of rsyslog v8.5 or newer (not 7) and only when using inotify see here for a presentation explaing the requirements.
I forced inotify (although it is the default) with:
module(load="imfile"
mode="inotify"
)
The input is defined like this:
input(type="imfile"
File="/file/path/*.log"
Tag="taskproject:"
Facility="local3"
)
After this, it should be working

message text not available when using psloglist

I'm using psloglist to analysis the saved event log for my windows 2003 server, however, the critical information i need is not retrieved properly and "message text not available. insertion strings" is appended instead. I've been searching for long while and still unable to find any solution or the root cause, anybody come across the same and could give some help in this? Thanks.
psloglist \\localhost -d 7 application -o "Source" | find "MessageText"

Log files for Solr on Tomcat

I am using Tomcat6 to deploy Solr on CentOs. Where can I find the log files to figure out the error Solr is giving?
You need to copy the logging config from solr/ into a location on the classpath.
cp SOLR_HOME/example/resources/log4j.properties TOMCAT_HOME/conf/
Then restart your tomcat instance.
i think you can find it in this directory apache-tomcat-x.x\logs
after that search for text file called "catalina" and the date of your search will be concatenated with a dot to the catalina file for example assume that we made a search at 2012-11-05 so the text file will lock like catalina.2012-11-05
The error given by solr can be seen in catlina.out but if you want it in a separate file this is the way you can configure it.
Add the entried in logging.proeperties which is at the folder apache tomcat/conf.
6localhost.org.apache.juli.FileHandler.level = FINE
6localhost.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
6localhost.org.apache.juli.FileHandler.prefix = solr.
Register the handler at the top in the file where all the other handlers are :
6localhost.org.apache.juli.FileHandler
add the below entry at the last
org.apache.solr.level=INFO
org.apache.solr.handlers=6localhost.org.apache.juli.FileHandler
you will get a seperate solr.log file where in you will have all the solr logs.

Logging with log4j on tomcat jruby-rack for a Rails 3 application

I just spent the better part of 3 hours trying to get my Rails application logging with Log4j. I've finally got it working, but I'm not sure if what I did is correct. I tried various methods to no avail until my very last attempt. So I'm really looking for some validation here, perhaps some pointers and tips as well -- anything would be appreciated to be honest. I've summarized all my feeble methods into three attempts below. I'm hoping for some enlightenment on where I went wrong with each attempt -- even if it means I get ripped up.
Thanks for the help in advance!
System Specs
Rails 3.0
Windows Server 2008
Log4j 1.2
Tomact 6.0.29
Java 6
Attempt 1 - Configured Tomcat to Use Log4J
I basically followed the guide on the Apache Tomcat website here. The steps are:
Create a log4j.properties file in $CATALINA_HOME/lib
Download and copy the log4j-x.y.z.jar into $CATALINA_HOME/lib
Replace $CATALINA_HOME/bin/tomcat-juli.jar with the tomcat-juli.jar from the Apache Tomcat Extras folder
Copy tomcat-juli-adapters.jar from the Apache Tomcat Extras folder into $CATALINA_HOME/lib
Delete $CATALINA_BASE/conf/logging.properties
Start Tomcat (as a service)
Expected Results According to the Guide
I should have seen a tomcat.log file in my $CATALINA_BASE/logs folder.
Actual Results
No tomcat.log
Saw three of the standard logs instead
jakarta_service_20101231.log
stderr_20101231.log
stdout_20101231.log
Question
Shouldn't I have at least seen a tomcat.log file?
Attempt 2 - Use default Tomcat logging (commons-logging)
Reverted all the changes from the previous setup
Modified $CATALINA_BASE/conf/logging.properties by doing the following:
Adding a setting for my application in the handlers line: 5rails3.org.apache.juli.FileHandler
Adding Handler specific properties
5rails3.org.apache.juli.FileHandler.level = FINE
5rails3.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
5rails3.org.apache.juli.FileHandler.prefix = rails3.
Adding Facility specific properties
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/rails3].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/rails3].handlers = 4host-manager.org.apache.juli.FileHandler
Modified my web.xml by adding the following context parameter as per the Logging section of the jruby-rack README (I also modified my warbler.rb accordingly, but I opted to change the web.xml directly to test things faster).
<context-param>
<param-name>jruby.rack.logging</param-name>
<param-value>commons_logging</param-value>
</context-param>
Restarted Tomcat
Results
A log file was created (rails3.log), however there was no log information in the file.
Attempt 2A - Use Log4j with existing set up
I decided to give Log4j another whirl with this new web.xml setting.
Copied the log4j.jar into my WEB-INF/lib folder
Created a log4j.properties file and put it into WEB-INF/classes
log4j.rootLogger=INFO, R
log4j.logger.javax.servlet=DEBUG
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=${catalina.base}/logs/rails3.log
log4j.appender.R.MaxFileSize=5036KB
log4j.appender.R.MaxBackupIndex=4
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss} [%t] %-5p %c %x - %m%n
Restarted Tomcat
Results
Same as Attempt 2
NOTE: I used log4j.logger.javax.servlet=DEBUG because I read in the jruby-rack README that all logging output is automatically redirected to the javax.servlet.ServletContext#log method. So I thought this would capture it, but I was obviously wrong.
Question
Why didn't this work?
Isn't Log4J using the commons_logging API?
Attempt 3 - Tried out slf4j (WORKED)
A bit uncertain as to why Attempt 2A didn't work, I thought to myself, maybe I can't use commons_logging for the jruby.rack.logging parameter because it's probably not using commons_logging API... (but I was still not sure). I saw slf4j as an option. I have never heard of it and out of curiosity, I decided to look it up. After reading about it briefly, I thought it was as good of a shot as any and decided to try it out following the instructions here.
Continuing from the setup of Attempt 2A:
Copied slf4j-api-1.6.1.jar and slf4j-simple-1.6.1.jar into my WEB-INF/lib folder
I also copied slf4j-log4j12-1.6.1.jar into my WEB-INF/lib folder
Restarted Tomcat
And VIOLA! I now have logging information going into my rails3.log file.
So the big question is:
WTF?
Even though logging seems to be working now, I'm really not sure if what I did is right. So like I said earlier, I'm really looking for some validation more or less. I'd also appreciate any pointers/tips/advice if you have any. Thanks!
We also did various experiments and finally settled with slf4j option. Coming from Java background we knew slf4j, so we didn't go further.
<context-param>
<param-name>jruby.rack.logging</param-name>
<param-value>slf4j</param-value>
</context-param>
btw, there is no need to copy slf4j-simple-1.6.1.jar into tomcat/lib or WEB-INF/lib when using slf4j-log4jxxx.jar
It's possible that your early tests with commons-logging were showing zero bytes files because of bufferSize.
If your entries get flushed to the file when you stop Tomcat then this is indeed the case and you are falling foul of the default 8Kb buffer.
To disable, for immediate flushing try...
org.apache.juli.FileHandler.bufferSize = -1
on the specific handler...
So for localhost...
2localhost.org.apache.juli.FileHandler.bufferSize = -1

Resources