How Can I Save the Indexed Data with Elasticsearch Permanently? - search

I followed the ElasticSearch Java-Api Guide.I added Java_Home to Computer->Settings->Advanced Settings->Environment Variables.And I run ElasticSearch -> elasticsearch-service-x64.exe and run as administrator elasticsearch.bat.When I did these settings ,I wrote localhost:9200 to browser and I got information abaout my client or node I guess.After a while I wrote localhost:9200 again but this time anything showed up.
I dont know is it the reason of my problem.But I know I cant keep indexed data permanently.When I indexed , data is going away in 5 seconds and searching operation dont give me any Hits.Meanwhile please tolerate my poor english.

In contrast to SOLR, elasticsearch has no notion of a commit, so everything you feed to the server without error gets indexed.

Related

Generate auto increment sequence in logstash

I am pushing logs to Elastic Search from Logstash and then i need to get back the logs in the order they were written. Sorting by time stamp does not help because there could me multiple log statements in the same time. I followed the solution in Include monotonically increasing value in logstash field? and it worked perfectly in my windows system.
But when the code was moved to the linux production environment, logstash is not starting up. Failing with the below error
reason=>"Couldn't find any filter plugin named 'seq'. Are you sure
this is correct? Trying to load the seq filter plugin resulted in this
error: no such file to load -- logstash/filters/seq", :level=>:error}
Check if the seq.rb file is in the filter folder.
Also check if the line ending of your seq.rb are linux. If you transferred the file from a windows machine to a linux, the problem might come from here.

MongoDB "open files limit" error during repairDatabase

I'm trying to do repairDatabase of MongoDB on Ubuntu 16.04 but it fails with an error "errno:24 Too many open files" ("code" : 16818).
I've raised "ulimit -n" up to 1024000, restarted the server, but still getting the same error.
It does not seem possible to raise it higher and I'm stuck with no ideas. Please help!
We have faced similar issue. First please make sure number of file descriptors used by "mongod" process while running repairDatabase() command. You can verify this with the help of "lsof -p mongod_pid" Also please note, if you want to change "max number of process", you need to edit "/etc/security/limits.conf" file by adding entry for mongod process.
Edit:
Also there is already feature request to open file per database as currently "wiredtiger" opens one file per collection and one for index. Also one should seriously look into horizontal scaling by sharding if cost is not a serious issue.

Clearing XHProf Data not working

I got XHProf working with XHgui. I like to clear or restart fresh profiling for certain site or globally. How do i clear/reset XHprof? I assume i have to delete logs in Mongo DB but I am not familiar with Mongo and I don't know the tables it stores info.
To clear XHProf with XHGui, log into mongo db and clear the collection - results as following:
mongo
db.results.drop()
The first line log into mongo db console. The last command, drops collection results that is is going to be recreated by the XHGui on the next request that is profiled
Some other useful commands:
show collections //shows all collection
use results //the same meaning as mysql i believe
db.results.help() //to find out all commands available for collection results
I hope it helps
I have similar issue in my Drupal setup, using Devel module in Drupal.
After a few check & reading on how xhprof library save the data, i'm able to figure out where the data is being saved.
The library will check is there any defined path in php.ini
xhprof.output_dir
if there's nothing defined in your php.ini, it will get the system temp dir.
sys_get_temp_dir()
In short, print out these value to find the xhprof data:
$xhprof_dir = ini_get("xhprof.output_dir");
$systemp_dir = sys_get_temp_dir();
if $xhprof_dir doesn't return any value, check the $systemp_dir, xhprof data should be there with .xhprof extension.

pyodbc fetchall() returns no results when a column returned by the query contains too much data

Setup:I am using Python 3.3 on a Windows 2012 client.
I have a select query running using pyodbc which is not returning any results via fetchall(). I know the query works fine because i can take it out and run it from Microsoft SQL Management Studio without any issues.
I can also remove one column from the select list and the query will return results. For the database row in question, this column contains a large amount of XML data (> 10,000 characters), so it seems as though there is some buffer overflow issue going on causing fetchall() to fail, though it doesn't throw any exceptions. I have tried googling around and i have seen rumors of a config option to raise the buffer size, but i haven't been able to nail down exactly how to do it, or what a workaround would be.
Is there a configuration option that I can use, or any alternative to pyodbc.
Disclaimer: I have only been using python for about 2 weeks now so i
am still quite the noob, though i have made every attempt to research
my problems thoroughly this one has proven to be elusive:
On a side note, i tried using odbc instead of pyodbc but the same query throws this oddball error which google isn't helping me solve either
[ERROR] An exception while executing the Select query: [][Negative size passed to PyBytes_FromStringAndSize]
It seems this issue was resolved by changing my SQL connection string
FROM:
DRIVER={SQL Server Native Client 11.0}
TO:
DRIVER={SQL Server}

Apache Cassandra. UnavailableException While Trying to insert a record

I am very new to Cassandra, and some how configured it. I was following This Link
.
Everything was fine. But in the end when I am trying to insert a record, it gives me the following exception. Today since afternoon I am trying to fix this. Googled a lot, but could not reach anywhere.
Any help on this will be greatly appretiated.
[default#DEMO] set Users[1234][name] = scott;
null
UnavailableException()
at org.apache.cassandra.thrift.Cassandra$insert_result.read(Cassandra.java:16077)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at org.apache.cassandra.thrift.Cassandra$Client.recv_insert(Cassandra.java:801)
at org.apache.cassandra.thrift.Cassandra$Client.insert(Cassandra.java:785)
at org.apache.cassandra.cli.CliClient.executeSet(CliClient.java:909)
at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:222)
at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:201)
at org.apache.cassandra.cli.CliMain.main(CliMain.java:328)
[default#DEMO]
Thank you
Achyuth
this is old one, I want to share my experience.
I had same issue when I was setting the qa environment. Every thing configured fine: including : cassandra-topology.properties. But the nodetool ring display the unknown DC value for all nodes since the default is set UNKNOWN. That told me that the cassandra-topology.properties is not right in some way. After tried several things still no luck, I decided to create my own cassandra-topology.properties file and re-type every thing with vi, then it starts fine.
So if you have issue, run nodetool ring first to see if the DC set is what it should be.

Resources