Currently I am working on executing Perl script in Unix box. The Perl script internally uses BCP command for for fetching the data from MS-SQL DB. The same perl script is properly running in Windows. But when I am executing it into Unix (using Putty). It is showing following error message on console.
sh: bcp: command not found
Below is my sql command in perl script.
"bcp \"select drive_id, ilf, xcoord, ycoord
from $mdtdb\.dbo\.$mdtcentraldtdtable a where
drive_id = $driveid and not exists
(select 'x' from $mdtdb\.dbo\.$mdtcentralaudittable b where
a.drive_id = b.drive_id and a.ilf = b.ilf)\"
queryout $tempdatafile -o $bcpoutfile -S $mdtsvr -q -c -t ,
-U $user1 -P $pw1";
I am successfully able to connect with the DB.
I need your help to resolve above bcp issue.
On unix, there is a bcp tool for sybase clients. It works only for Sybase Products.
There is an open-source replacement for bcp that works with MSSQL server: freebcp But the freetds library is a bit difficult to configure, and (being a reverse engineering effort by volunteers) it works only with relatively simple datatypes.
To install it on ubuntu/debian based linux machines, run this
apt-cache search freetds
freetds-common - configuration files for FreeTDS SQL client libraries
freetds-dev - MS SQL and Sybase client library (static libs and headers)
libct4 - libraries for connecting to MS SQL and Sybase SQL servers
libdbd-freetds - Freetds database server driver for libdbi
libsybdb5 - libraries for connecting to MS SQL and Sybase SQL servers
tdsodbc - ODBC driver for connecting to MS SQL and Sybase SQL servers
freetds-bin - FreeTDS command-line utilities
libaprutil1-dbd-freetds - Apache Portable Runtime Utility Library - FreeTDS Driver
sqsh - commandline SQL client for MS SQL and Sybase servers
libqt4-sql-tds - Qt 4 FreeTDS database driver
This command is optional:
apt-file list freetds-bin
freetds-bin: /usr/bin/bsqldb
freetds-bin: /usr/bin/bsqlodbc
freetds-bin: /usr/bin/datacopy
freetds-bin: /usr/bin/defncopy
freetds-bin: /usr/bin/fisql
freetds-bin: /usr/bin/freebcp
freetds-bin: /usr/bin/osql
freetds-bin: /usr/bin/tdspool
freetds-bin: /usr/bin/tsql
These commands install most of what you need
apt-get install freetds-bin
apt-get install freetds-dev
You must then enter your connection data into configfile
/etc/freetds/freetds.conf
[OurMSSQLServer]
host = 1xx.xxx.xxx.xx
port = 1433
tds version = 8.0
Maybe something else in this configfile has to be changed. This I don't remember right now.
From here on you are on your own. Please read the freetds docs: http://www.freetds.org/userguide/
The freetds software is up-to-date (actively developed). But the dev team is very small I think.
You can also compile freetds from source, but that may take more effort.
The error is very clear: the bcp command is not found in your $PATH and therefore it fails, but the real issue is that bcp is a MS SQL server utility and you won't find it on UNIX
You might be able to get programmatic access to an MSSQL database from Perl on unix by using DBI → DBD::ODBC → Easysoft SQL Server ODBC Driver
Related
I am working on Linux server with PostgreSQL that is installed through the command line.
How could I check the version of the PostgreSQL JDBC Driver using Linux command line?
I use this command psql -V and it shows the PostgreSQL version psql (PostgreSQL) 9.5.24, but I need the PostgreSQL JDBC Driver version. What is the command the should I use?
When you run:
java -jar /path/to/driver.jar
it will report its version.
Obviously you need to replace /path/to/driver.jar with the correct location where you stored the driver when you downloaded it.
By default the file name itself contains the version. So if you downloaded postgresql-42.2.19.jar then it's highly likely its version is 42.2.19
As a side note: psql -V prints the version of the command line tool. That's not necessarily the same version as the server.
From Java, you can use code like this:
java.sql.Driver d = new org.postgresql.Driver();
System.out.println(d.getMajorVersion() + "." + d.getMinorVersion());
See the documentation for details.
So I'm going to preface this by saying that I CANNOT use sudo apt-get install sqlite3. I am doing this project in a VM on google cloud platform, and our VMs are not allowed to have external IP access because we have sensitive information on these VMs. So running that command just hangs.
I went to the sqlite website and downloaded the Precompiled Binaries for Linux and put that on google cloud storage (gcs). I am able to cp the directory from gcs to the VM, but
ultimately, I cannot figure out a way to get it to run. When I cd to the directory, I see there are 3 objects sqldiff, sqlite3, sqlite3_analyzer, of which Im including an image in case you want to reference it.
I was testing out an idea at home, where I have a pc that can dual boot linux, and I was able to run sqlite3 just find from the command when I did a sudo apt-get for it (i.e. I get
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
), but I'm having problems here. I can't seem to do anything after this within the VM I am trying to run the actual code in now. I am pretty inexperienced with ubuntu and linux systems, so maybe I'm just missing something easy.
Thank you to anyone who can offer some advice.
Have you tried building the binaries from source?
https://www.sqlite.org/howtocompile.html
This worked for me when I downloaded the source code amalgamation.
Ideally, gcc should be present on your VM
$ : gcc shell.c sqlite3.c -lpthread -ldl -o sqlite3_cli
$ : ./sqlite3_cli
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite>
I've followed the official installation guide but haven't had any luck so far. I wonder if cx_Oracle can work on AWS SageMaker's virtual environment. The steps I've used so far are:
Create a /opt/oracle directory and unzip the basic instantclient in it.
sudo yum install libaio
sudo sh -c "echo /opt/oracle/instantclient_18_3 > /etc/ld.so.conf.d/oracle-instantclient.conf" and
sudo ldconfig
And finally exported the LD_LIBRARY_PATH with: export LD_LIBRARY_PATH=/opt/oracle/instantclient_18_3:$LD_LIBRARY_PATH
When trying to run a connection inside the notebook with connection = cx_Oracle.connect(usr + '/' + pwd + '#' + url), I receive the DPI-1047 error code that says that libclntsh.so cannot be open, however that library is in the /opt/oracle folder. As another option, when running the same connection through the terminal Python console, I get the ORA-01804 error code, which says that the timezone files were not properly read, which is something I'm trying to fix also but suspect is related to cx_Oracle not finding its library folder. (Now, explain to me: why does it have to be so difficult for a billionaire company to create a decent library import and installation?)
Is there a step I'm missing? Is there a detail about AWS SageMaker that I should account for? Also, is there another option for extracting data from an oracle server through Python and AWS?
Hi and thank you for using SageMaker!
After some effort, I was finally able to figure out a sequence of steps that allowed me to query an Oracle 12 database from within a SageMaker notebook instance. Here are the steps that I took:
I created an Oracle 12 database using Amazon RDS for testing purposes. (You can of course skip this step if you already have an Oracle database available.)
I downloaded the Oracle 12 Instant Client RPM as described here. Note that you will need an Oracle Account in order to download this file.
I uploaded the RPM to my SageMaker Notebook Instance from within JupyterLab. Note that this can take 2-3 minutes to completely upload before proceeding to the next step. (I initially had problems running the installation because the upload was still in progress.)
I ran all of the following commands from the Jupyter terminal as prescribed in the Oracle instructions:
cd SageMaker
sudo yum install oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm
sudo sh -c "echo /usr/lib/oracle/12.2/client64/lib > /etc/ld.so.conf.d/oracle-instantclient.conf"
sudo ldconfig
sudo mkdir -p /usr/lib/oracle/12.2/client64/lib/network/admin
# Restart Jupyter...
sudo restart jupyter-server
I then installed the cx_Oracle library:
source activate python3
pip install cx_Oracle --upgrade
Then lastly, I created a new notebook using the conda_python3 kernel:
import cx_Oracle
db = cx_Oracle.connect("my_username/my_password#my-rds-instance.ccccccccccc.us-east-1.rds.amazonaws.com/ORCL")
# Example query
cursor = db.cursor()
for row in cursor.execute('select * from DBA_TABLES'):
print(row)
...and then it worked!
Note that it took me a few tries to figure out the exact connection string, which can differ depending on how your database is configured. Unfortunately, the error messages were hard to understand - in my case, I had the error ORA-12504: TNS:listener was not given the SERVICE_NAME in CONNECT_DATA until I had specified the /ORCL at the end of the connection string.
If you need to do these steps frequently, you can add the installation and configuration of the Oracle client in a SageMaker Lifecycle Configuration script. I haven't tested that scenario, but it might be worth a try!
One last thing, I noticed in your question that you are using the Oracle 18 client. I didn't test that exact scenario, since I only have access to an Oracle 12 database. However, the Oracle 12 client should be able to connect to an Oracle 18 database, too.
Best,
Kevin
With the help of cx_Oracle package I am trying to connect to remote Oracle database. Unfortunately I have error when try to test next python code:
import cx_Oracle;
dsn_tns=cx_Oracle.makedsn('HOST', 'PORT', service_name='NAME')
connection=cx_Oracle.connect('USERNAME', 'PASSWORD', dsn_tns)
ERROR:
cx_Oracle.Database: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor
Dependencies:
1) Ubuntu 14.04 (64 bit) (local computer)
2) Python 2.7.14 (64 bit)
3) cx_Oracle 6.4.1
4) Instant Client 18.3
5) Oracle 11g database (located in remote server)
I used official documentation to install and configurate instant client.
Official documentation say that:
Oracle Call Interface 18.3 can connect to Oracle Database 11.2 or
later.
For thats why I load basic package of instant client 18.3, then unzip it:
cd /opt/oracle
unzip instantclient-basic-linux.x64-18.3.0.0.0dbru.zip
Then I install libaio:
sudo apt-get install libaio1
In terminal I wrote next 2 command:
sudo sh -c "echo /opt/oracle/instantclient_18_3 > /etc/ld.so.conf.d/oracle-instantclient.conf"
sudo ldconfig
Inside /opt/oracle/instantclient_18_3/network/admin I put tnsnames.ora file.
After that I set environment variable in ~/.profile:
export LD_LIBRARY_PATH=/opt/oracle/instantclient_18_3:$LD_LIBRARY_PATH
export TNS_ADMIN=/opt/oracle/instantclient_18_3/network/admin
export PATH=/opt/oracle/instantclient_18_3:$PATH
What else should I have done to fix the problem?
What's wrong with instant client? Absolutely nothing from the information you supplied. You are connecting to the server (listener) but the server is not configured with the database you specified.
You can confirm this with a different client / different client host.
Go check the config on the server.
I attempted to update the version of Postgres-server on an OpenSuse server. There were a few issues and I now want to revert back to using the version of postgres-server that was installed before(9.03).
I've removed version 9.4 and I've tried starting the postgresql server but I'm getting the error:
Your database files were created by PostgreSQL version 9.0.
Could not find executables for this version.
Please install the PostgreSQL server package for version 9.0.
When I run:
rpm -qa | grep postgresql
it shows:
postgresql-docs-9.0.3-1.6.1.noarch
postgresql-server-9.0.3-1.6.1.x86_64
postgresql-contrib-9.0.3-1.6.1.x86_64
postgresql-9.0.3-1.6.1.x86_64
postgresql-devel-9.0.3-1.6.1.x86_64
So it looks like version 9.03 is already installed.
I'm not sure what to do now to get the server back to how it was and I'm panicking a bit. Can anyone suggest anything?
EDIT**
Here some more info after some useful info from Ronaldhino.
I discovered my pg_ctl file in /user/bin/pg_ctl and /data appears to be the previous data directory (I deduced this as it contain pg_hbs.conf and postgresql.conf).
As a result I've run this command as user postgres:
/usr/bin/pg_ctl start -D /data/ -l /data/pg_log/new_log.txt
I did get a message initial saying:
server starting
It appears that the server is actually running. However, the psql binary file now appears to be missing so I can't psql into the database service.
Thanks
I seems that system can not recognize where's your postgresql 9.0 location is.
You should find where's the postgresql 9.0's binary file and start Server directly.
Run command for finding where's postgresql 9.0 location
which postgres
For example:
/usr/local/pgsql/bin/
Start server
sudo /usr/local/pgsql/bin/pg_ctl start -D <<your DB dir>> -l /var/log/postgresl.log
Good luck!