Apache Nutch-2.2.1 installation - nutch

I am installing nutch2.2.1 on my centOS virtual machine and getting an error injecting the seed urls(directory name). I used this command:
/usr/share/apache-nutch-2.1/src/bin/nutch inject root/apache-nutch-2.1/src/testresources/testcrawl urls
And i got an error :
Error: Could not find or load main class org.apache.nutch.crawl.InjectorJob
Similarly, for the command
/usr/share/apache-nutch-2.1/src/bin/nutch readdb
gives me an error:
Error: Could not find or load main class org.apache.nutch.crawl.WebTableReader
What should i do to fix these errors?
I am following the tutorial from: http://wiki.apache.org/nutch/Nutch2Tutorial and followed the same steps as suggested.
Also my query also revolves around setting the path for ant. Every time i open a new session i have to set the ANT_HOME and PATH environment variable manually. And then they work all fine. Same is the case with setting JAVA_HOME.

You should go to $NUTCH_HOME/runtime/local/ directory to run the the commands.

Related

django.db.utils.DatabaseError: Error while trying to retrieve text for error ORA-01804

Q1. What versions are we using?
Ans.
Python 3.6.12
OS : CentOS 7 64-bit
DB : Oracle 18c
Django 2.2
cx_Oracle : 8.1.0
Q2. Describe the problem
Ans. While running server with "python3 manage.py runserver"
application is able to contact Oracle DB and show the Django Administration page and login also works.
But when we access the application using the Apache (HTTPD) based URL over secure SSL port, we do see the Django page and the admin page as well but Login to Admin page with Internal server error.
In the logs, we see
"django.db.utils.DatabaseError: Error while trying to retrieve text for error ORA-01804"
cx_oracle is otherwise able to connect to the database properly, another application is also using the same database behind the same httpd proxy and works fine
Q3. Show the directory listing where your Oracle Client libraries are installed (e.g. the Instant Client directory). Is it 64-bit or 32-bit?
Ans. 64-bit
Q4. Show what the PATH environment variable (on Windows) or LD_LIBRARY_PATH (on Linux) is set to?
LD_LIBRARY_PATH=/srv/vol/db/oracle/product/18.0.0/dbhome_1/lib:/lib:/usr/lib
PATH=$ORACLE_HOME/bin:/srv/vol/db/oracle/product/18.0.0/dbhome_1/lib:$PATH
Q5. Show any Oracle environment variables set (e.g. ORACLE_HOME, ORACLE_BASE).
ORACLE_HOME=/srv/vol/db/oracle/product/18.0.0/dbhome_1
TNS_ADMIN=$ORACLE_HOME/network/admin
NLS_LANG=AMERICAN_AMERICA.AL32UTF8
ORACLE_BASE=/srv/vol/db/oracle
CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/lib
Any suggestions/help is highly appreciated.
Thank you
I found the problem
So I just removed all the variable declarations from /etc/sysconfig/httpd and checked, the application was still able to access the lib files, so these were now redundant.
Then undid all variable declarations done earlier in .localsh and .localrc files for the os users. To start from scratch, and go step by step to see where it breaks.
So now, cx_Oracle was looking for the lib files in wrong directory
$ORACLE_HOME/client_1/lib
instead of
$ORACLE_HOME/lib
DPI-1047: Cannot locate a 64-bit Oracle Client library: "$ORACLE_HOME/client_1/lib/libclntsh.so: cannot open shared object file: No such file or directory". See https://cx-oracle.readthedocs.io/en/latest/user_guide/installation.html for help
I did not have any subfolder named "client_1" inside dbhome_1
so I just created a symlink client_1 that points to dbhome_1 (still unsure on this, but at least it works :) )
So, now, this error was gone but now again ORA-01804 was coming. 😑
I had read somewhere that this error can be fixed by adding "libociei.so" but I did not have one on my instance, so I generated it using these commands:-
mkdir -p $ORACLE_HOME/rdbms/install/instantclient/light
cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk igenliboci
Then I just moved this libociei.so file from
$ORACLE_HOME/instantclient to $ORACLE_HOME/lib
Now there was a new error (so.. progress 😉 ):
ORA-12546 - TNS Permission Denied.
This was easy to solve 😀
I used this command to address this :-
setsebool -P httpd_can_network_connect on
And...... That was all! It worked.

JAVA_HOME for Logstash

I am trying to setup ELK stack for my Web Services Log Monitoring.
So I have setup all the parts for ELK Stack.
I am facing one issue in Log-stash. When I am running Log-stash, I am facing error, could not load Java binary
Although the simple fix it set the JAVA_HOME in environment variable.
But I don't want to set an environment variable, but what I want to set JAVA_HOME just for Log-stash. I have tried adding in startup.options, but to enable I must run system-install. When I am running system-install, I am facing the same error again.
I have added
export JAVA_HOME=/opt/jre8
then system-install file runs, but still on starting log-stash, I am getting the same error. What should I do to resolve this error?
You can config in startup.options (logstash5.4 version):
Ex:
JAVA_HOME=/.../jdk1.8.0_121
JAVACMD=/.../jdk1.8.0_121/bin/java
Then use root role to start: system-install.
(You can use update-java-alternatives --list to list installed java versions with paths)
You can add this configuration to the file- /etc/sysconfig/logstash, this file is read during startup by logstash.
This is what you should add:
export JAVA_HOME=/opt/jre8

Creating/Migrating rBac in Yii2

I'm just getting started with Yii2 and I am following a tutorial here about setting up Yii2/AdminLTE and I am unable to finish the setup as I am getting errors in Terminal on the last step.
The part I am not able to finish is:
finaly we create rbac dbmanager with simple code, you can see in
folder "console/RbacController" with specific level for :
Admin : can do everything Editor : can edit, add and view Author : can
add and view viewer ; just viewer create rbac :
"yii migrate --migrationPath=#yii/rbac/migrations"
"yii rbac/init"
dont forget to chmod -R 777 on your web/assets if linux environment
and please free to update your setting on menu setting.
So in terminal on my mac, I tried both:
yii migrate --migrationPath=#yii/rbac/migrations
with the error: -bash: yii: command not found
and
/.yii migrate --migrationPath=#yii/rbac/migrations
with error: -bash: /.yii: No such file or directory.
I was able to complete the beginning of the tutorial, it is just this last step. I am already working in my yii2-advanced-adminlte directory
UPDATED:
Added screenshot of terminal window
Normally the yii command is located in
fro advanced template in project directory parent of backend, frontend, console and not in console
be sure of find the right dir and then accessing this try launche your command
yii migrate --migrationPath=#yii/rbac/migrations
eventually adjust your path to rbac/ migrations
Found it out, the problem was using MAMP I had to manually set the php bin to 5.5.23 in terminal because once I started down the path scaisEdge showed me I was getting a No such File or Folder error.
Eventually this is the command that worked for me (check your php MAMP path):
/Applications/MAMP/bin/php/php5.5.23/bin/php yii migrate
Source link: http://www.yiiframework.com/forum/index.php/topic/47043-error-on-using-db-migration-w-mysql#entry222568

Puppet error when using classes

I am starting using puppet to manage many servers, the problem is that whenever I try to use a class, new relic for example:
node 'mynode' {
class {'newrelic::server::linux':
newrelic_license_key => '***',
}
}
It fails, and returns the following error:
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Puppet::Parser::AST::Resource failed with error ArgumentError: Could not find declared class newrelic::server::linux at /etc/puppet/manifests/site.pp:3 on node mynode
I have installed fsalum-newrelic on the master, and everything works fine when using files, packages, services etc. What am I doing wrong?
The catalog compiler will look for class newrelic::server::linux at newrelic/manifests/server/linux.pp relative to each directory in your module path. (Note: newrelic, NOT fsalum-newrelic.) Make certain that you indeed did install the module such that such a file exists in your modulepath, and make sure that it is readable by the puppetmaster process.
Note, too, that "readable by the puppetmaster process" means more than just the ownership and permissions of the file itself. It also involves ownership and permissions of all the directories in the path to that file, and possibly other forms of access control, such as ACLs and SELinux conext and policy.
Find out where you are actually installing the new puppet forge modules using perhaps a unix utility like "locate".
Then look in the the /etc/puppet/puppet.conf at the "basemodulepath" and check that the place it is installed is in the path
Here is my basemodulepath
basemodulepath = $confdir/environments/production/modules:$confdir/environments/production/local_modules:/etc/puppet/modules
The external modules I am using are either in /etc/puppet/modules or in /etc/puppet/enviroments/production/modules

How to set path to Mono so Supervisor knows about it?

I am following this tutorial (Hosting Nancy with Nginx on Ubuntu) with one change. I am using the Spark view engine instead of the built in view engine.
Spark view engine throws an error building any view when the Nancy host is running under supervisor.
System.SystemException: Error running mcs: Cannot find the specified file
at Mono.CSharp.CSharpCodeCompiler.CompileFromFileBatch (System.CodeDom.Compiler.CompilerParameters,string[]) <0x00577>
at Mono.CSharp.CSharpCodeCompiler.CompileAssemblyFromFileBatch (System.CodeDom.Compiler.CompilerParameters,string[]) <0x00033>
at System.CodeDom.Compiler.CodeDomProvider.CompileAssemblyFromFile (System.CodeDom.Compiler.CompilerParameters,string[]) <0x0004d>
at Spark.Compiler.BatchCompiler.Compile (bool,string,string[]) <0x007f4>
at Spark.Compiler.CSharp.CSharpViewCompiler.CompileView (System.Collections.Generic.IEnumerable`1>,System.Collections.Generic.IEnumerable`1>) <0x00087>
at Spark.SparkViewEngine.CreateEntryInternal (Spark.SparkViewDescriptor,bool) <0x002af>
at Spark.SparkViewEngine.CreateEntry (Spark.SparkViewDescriptor) <0x0004f>
at Nancy.ViewEngines.Spark.SparkViewEngine/<>c__DisplayClass2.b__1 (Nancy.ViewEngines.ViewLocationResult) <0x00023>
at Nancy.ViewEngines.DefaultViewCache/<>c__DisplayClass1`1.b__0 (Nancy.ViewEngines.ViewLocationResult) <0x00023>
If I run the environment setting script /opt/mono/env.sh and then run my host from the command prompt, the website works.
I think it must be due to my Nancy host not being able to find Mono when running under supervisor.
For reference:
#env.sh
export PATH=/opt/mono/bin:$PATH
export LD_LIBRARY_PATH=/opt/mono/lib:$LD_LIBRARY_PATH
export PKG_CONFIG_PATH=/opt/mono/lib/pkgconfig:$PKG_CONFIG_PATH
I am using Mono 3.8.0 from here:
http://download.opensuse.org/repositories/home:/tpokorra:/mono/xUbuntu_14.04/
How do I set these paths so that Nancy host can find Mono when running under supervisor?
I needed to set the environment option in my supervisor configuration file.
Here is the setting that worked for the above example.
environment=PATH="/opt/mono/bin:$PATH"

Resources