So I'm going to preface this by saying that I CANNOT use sudo apt-get install sqlite3. I am doing this project in a VM on google cloud platform, and our VMs are not allowed to have external IP access because we have sensitive information on these VMs. So running that command just hangs.
I went to the sqlite website and downloaded the Precompiled Binaries for Linux and put that on google cloud storage (gcs). I am able to cp the directory from gcs to the VM, but
ultimately, I cannot figure out a way to get it to run. When I cd to the directory, I see there are 3 objects sqldiff, sqlite3, sqlite3_analyzer, of which Im including an image in case you want to reference it.
I was testing out an idea at home, where I have a pc that can dual boot linux, and I was able to run sqlite3 just find from the command when I did a sudo apt-get for it (i.e. I get
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
), but I'm having problems here. I can't seem to do anything after this within the VM I am trying to run the actual code in now. I am pretty inexperienced with ubuntu and linux systems, so maybe I'm just missing something easy.
Thank you to anyone who can offer some advice.
Have you tried building the binaries from source?
https://www.sqlite.org/howtocompile.html
This worked for me when I downloaded the source code amalgamation.
Ideally, gcc should be present on your VM
$ : gcc shell.c sqlite3.c -lpthread -ldl -o sqlite3_cli
$ : ./sqlite3_cli
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite>
Related
I've followed the official installation guide but haven't had any luck so far. I wonder if cx_Oracle can work on AWS SageMaker's virtual environment. The steps I've used so far are:
Create a /opt/oracle directory and unzip the basic instantclient in it.
sudo yum install libaio
sudo sh -c "echo /opt/oracle/instantclient_18_3 > /etc/ld.so.conf.d/oracle-instantclient.conf" and
sudo ldconfig
And finally exported the LD_LIBRARY_PATH with: export LD_LIBRARY_PATH=/opt/oracle/instantclient_18_3:$LD_LIBRARY_PATH
When trying to run a connection inside the notebook with connection = cx_Oracle.connect(usr + '/' + pwd + '#' + url), I receive the DPI-1047 error code that says that libclntsh.so cannot be open, however that library is in the /opt/oracle folder. As another option, when running the same connection through the terminal Python console, I get the ORA-01804 error code, which says that the timezone files were not properly read, which is something I'm trying to fix also but suspect is related to cx_Oracle not finding its library folder. (Now, explain to me: why does it have to be so difficult for a billionaire company to create a decent library import and installation?)
Is there a step I'm missing? Is there a detail about AWS SageMaker that I should account for? Also, is there another option for extracting data from an oracle server through Python and AWS?
Hi and thank you for using SageMaker!
After some effort, I was finally able to figure out a sequence of steps that allowed me to query an Oracle 12 database from within a SageMaker notebook instance. Here are the steps that I took:
I created an Oracle 12 database using Amazon RDS for testing purposes. (You can of course skip this step if you already have an Oracle database available.)
I downloaded the Oracle 12 Instant Client RPM as described here. Note that you will need an Oracle Account in order to download this file.
I uploaded the RPM to my SageMaker Notebook Instance from within JupyterLab. Note that this can take 2-3 minutes to completely upload before proceeding to the next step. (I initially had problems running the installation because the upload was still in progress.)
I ran all of the following commands from the Jupyter terminal as prescribed in the Oracle instructions:
cd SageMaker
sudo yum install oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm
sudo sh -c "echo /usr/lib/oracle/12.2/client64/lib > /etc/ld.so.conf.d/oracle-instantclient.conf"
sudo ldconfig
sudo mkdir -p /usr/lib/oracle/12.2/client64/lib/network/admin
# Restart Jupyter...
sudo restart jupyter-server
I then installed the cx_Oracle library:
source activate python3
pip install cx_Oracle --upgrade
Then lastly, I created a new notebook using the conda_python3 kernel:
import cx_Oracle
db = cx_Oracle.connect("my_username/my_password#my-rds-instance.ccccccccccc.us-east-1.rds.amazonaws.com/ORCL")
# Example query
cursor = db.cursor()
for row in cursor.execute('select * from DBA_TABLES'):
print(row)
...and then it worked!
Note that it took me a few tries to figure out the exact connection string, which can differ depending on how your database is configured. Unfortunately, the error messages were hard to understand - in my case, I had the error ORA-12504: TNS:listener was not given the SERVICE_NAME in CONNECT_DATA until I had specified the /ORCL at the end of the connection string.
If you need to do these steps frequently, you can add the installation and configuration of the Oracle client in a SageMaker Lifecycle Configuration script. I haven't tested that scenario, but it might be worth a try!
One last thing, I noticed in your question that you are using the Oracle 18 client. I didn't test that exact scenario, since I only have access to an Oracle 12 database. However, the Oracle 12 client should be able to connect to an Oracle 18 database, too.
Best,
Kevin
I launched an Ubuntu 18.04 VM with Azure. I installed a bunch of stuff that I need. Then, I used the dashboard to create a custom image from this machine. After that, I checked that the image was okay by launching some machines with that image. Everything seemed to be working fine.
Today, I launched a new instance with my custom image. Then I tried to install a few things with apt-get install and I get the following error (e.g. for unzip):
sudo: unable to resolve host ABCDEFG: Resource temporarily unavailable
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package unzip is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'unzip' has no installation candidate
This same thing happens for any package I try to install. After testing some basic things with my repositories, I checked the internet connection with ping. E.g. ping www.google.com which is also not working. I launched a vanilla Ubuntu 18.04 instance and I am not having these problems with that machine.
I have also tried sudo reboot but no luck with that. I did notice that when the system booted it shows the following error, also indicating that something is wrong with the internet:
Failed to connect to https://changelogs.ubuntu.com/meta-release-lts. Check your Internet connection or proxy settings
Any help is greatly appreciated.
So, after some digging around, I found this answer to something similar: https://askubuntu.com/questions/1045278/ubuntu-server-18-04-temporary-failure-in-name-resolution.
I used the following command and the internet started working again:
sudo ln -s ../run/systemd/resolve/stub-resolv.conf /etc/resolv.conf
This is a little different than the answer on askubuntu because this is on an Azure image. First, I noticed that my image was missing resolv.conf in /etc. Using ls -la /etc/resolv.conf on a different azure image, I saw that it was a symbolic link to ../run/systemd/resolve/stub-resolve.conf, so I created a link that matched this format on my machine and that fixed things.
** EDIT **
It's worth noting that when you deprovision the VM to create the custom image, it does say:
WARNING! The waagent service will be stopped.
WARNING! Cached DHCP leases will be deleted.
WARNING! root password will be disabled. You will not be able to login as root.
WARNING! /etc/resolv.conf will be deleted.
WARNING! xxxx account and entire home directory will be deleted.
I am going by the CouchDB docs install directions:
http://docs.couchdb.org/en/2.0.0/install/unix.html
I install the dependencies:
$ sudo yum install autoconf autoconf-archive automake \
curl-devel erlang-asn1 erlang-erts erlang-eunit \
erlang-os_mon erlang-xmerl help2man \
js-devel-1.8.5 libicu-devel libtool perl-Test-Harness
Configure goes fine:
$ ./configure
You have configured Apache CouchDB, time to relax. Relax.
But just as I'm starting to feel relaxed I get thrown a curve ball by gmake:
$ make release
Uncaught error in rebar_core: {error, {erl_interface,include,"code:lib_dir(erl_interface)is unable to find the erl_interface library."}}
The only clue the docs give is to try gmake if make fails but since gmake and make are the same on CentOS 7 that doesn't help.
From the output it appears to be a dependency issue but I don't pretend to understand Erlang dependencies or know anything about this particular one. I installed the packages in yum the docs suggest, I thought that should be enough.
Any help would be appreciated.
To fix all build errors I have installed erlang-reltool, gcc-c++ and erlang-erl_interface besides the packages suggested in CouchDB docs.
As a side note, I have also adapted the adduser command from CouchDB docs to:
adduser --system --no-create-home --shell /bin/bash -U -c "CouchDB Administrator" couchdb
I looked at an blog post about installing CouchDB 2.0 on CentOS 7 here:
https://medium.com/linagora-engineering/setting-up-a-couchdb-2-cluster-on-centos-7-8cbf32ae619f
They include erlang itself in the yum dependencies.
I tried this on my machine and that did the trick.
I don't know if this is the best way to go but it works.
If anyone has a better solution of installing only the needed erlang packages that CouchDB requires I'd like to hear about that.
Also if anyone managing the docs at CouchDB is listening, the install directions of the web page don't work for CentOS 7. I've tried it on two machines with the same error above.
Incidentally the instructions for setting up a couchdb user and group don't work on CentOS 7 either. To get it to work you have to know enough Linux to interpret some unusual options like setting up a system account and making it geco and interpret what they mean from the instructions that presumably work on Ubuntu and then translate that into the equivalent options that work on CentOS 7 and be pretty confident your not misconfiguring your server and creating a security vulnerability. You have to decide if you want to limit the people that are able to even install your DB to people who have those Linux skills. If you decide you do, I'm going to feel left out.
I am trying to install Google Cloud SDK on a Linux machine without any Internet access.
I am following the instructions at: https://cloud.google.com/sdk/?hl=en
I downloaded the tar file on my local machine and transferred it to the Linux machine using scp. I then ran the install.sh file and got the following error:
[me#user google-cloud-sdk]$ ./install.sh
Welcome to the Google Cloud SDK!
To help improve the quality of this product, we collect anonymized data on how
the SDK is used. You may choose to opt out of this collection now (by choosing
'N' at the below prompt), or at any time in the future by running the following
command:
gcloud config set --scope=user disable_usage_reporting true
Do you want to help improve the Google Cloud SDK (Y/n)? n
This will install all the core command line tools necessary for working with
the Google Cloud Platform.
/home/me/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py:661: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
exc_message = getattr(exc, 'message', None)
/home/me/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py:664: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
msg = u'({0}) {1}'.format(command_path_string, exc.message)
ERROR: (gcloud.components.update) Failed to fetch component listing from server. Check your network settings and try again.
I have a proxy server that I can use to access the internet from this Linux machine. I tried running install.sh as 'sh install.sh --proxy host:port' but obviously, there is no input parameter called proxy to install.sh.
How can I work around this problem?
Thanks in advance.
Exported my proxy details as "export https_proxy='...'" before running the install.sh file.
This worked for me.
Go to Advanced System settings and create a variable HTTPS_PROXY,
restart CMD.
echo %HTTPS_PROXY%
To make sure it has taken the changes into account.
Launch the install.bat
I am working on AWS services. I have an ec2 ( centos ) instance. I need to configure SQL*Plus client on this centos machine.
The server with whom I want to connect is at some remote area. The server version is oracle-se(11.2.0.2)
How can I get the client installed on the CentOS machine?
Go to Oracle Linux x86-64 instant clients download page
Download the matching client
oracle-instantclient11.2-basic-11.2.0.2.0.x86_64.rpm
oracle-instantclient11.2-sqlplus-11.2.0.2.0.x86_64.rpm
Install
rpm -ivh oracle-instantclient11.2-basic-11.2.0.2.0.x86_64.rpm
rpm -ivh oracle-instantclient11.2-sqlplus-11.2.0.2.0.x86_64.rpm
Set environment variables in your ~/.bash_profile
ORACLE_HOME=/usr/lib/oracle/11.2/client64
PATH=$ORACLE_HOME/bin:$PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib
export ORACLE_HOME
export LD_LIBRARY_PATH
export PATH
Reload your .bash_profile by simply typing source ~/.bash_profile (suggested by jbass) or Log-out user and log-in again.
Now you're ready to use SQL*Plus and connect your server. Type in :
sqlplus "username/pass#(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.2.1)(PORT=1521))(CONNECT_DATA=(SID=YOURSID)))"
The solution by #ChamaraKeragala is good, but it is unnecessary to logout/login. Instead type:
source ~/.bash_profile
For everyone still getting the following error:
sqlplus command not found
The original post refers to a set of environment variables, the most important of which is ORACLE_HOME. This is the parent directory where the oracle binaries get installed.
Depending on what version of oracle you downloaded you'll have to change the ORACLE_HOME accordingly. For example, the original question's ORACLE_HOME was set to:
ORACLE_HOME=/usr/lib/oracle/11.2/client64
My version of Oracle happens to be 12.1, so my ORACLE_HOME is set to:
ORACLE_HOME=/usr/lib/oracle/12.1/client64
If you are unsure of the version that you downloaded, you can:
cd /usr/lib/oracle after the installation and find the version.
Look at the RPM file oracle-instantclient12.1, where the bolded bits would refer to the version number.
There's a good blog post[1] on $subject. setup oracle client in ubuntu with minimum effort. Following are the main steps on how to step up the client.
In my case, I was installing rpm files using alien package.
Install alien and related packages
sudo apt-get install alien
Install oracle client packages using alien.
sudo alien -i oracle-instantclient11.2-basic-11.2.0.3.0-1.x86_64.rpm
sudo alien -i oracle-instantclient11.2-sqlplus-11.2.0.3.0-1.x86_64.rpm
In my opinion these two steps are the easiest way to install oracle client rpm's on your ubuntu system. (I'm not going to mention about export oracle specific variables as it's already clearly explained in above answers)
Hope it helps someone.
[1] http://pumuduruhunage.blogspot.com/2016/04/setup-oracle-sql-plus-client-on-aws.html
For any one who is using proxy, you'd need to add an extra line to the bash profile. At least this is what made it work for me. I'm using cntlm.
export no_proxy=
Install via zip (tried with 12_2)
First of all there is no need to set ORACLE_HOME.
Simply download the .zip files from here starting with the first one Basic: followed by SQL*Plus: and any additional zips you may need.
Extract them all under /opt/oracle
You will then have a directory: /opt/oracle/instantclient_x_y
On ubuntu I had to do also:
sudo apt install libaio1
To run:
# This can be also done by adding only the path below in: /etc/ld.so.conf.d/oracle-instantclient.conf
export LD_LIBRARY_PATH=/opt/oracle/instantclient_x_y:$LD_LIBRARY_PATH
# This can be added in ~/.profile or ~/.bashrc
export ORACLE_HOME=/opt/oracle/instantclient_x_y
/opt/oracle/instantclient_x_y/sqlplus user/pass#hostname:1521/sidorservicename
At the bottom of the the above link page there are more details.