I have excel which i am trying to import in oracle database table.
Some of the values in excel consist of for example 14:39.5 with double colon. What dataype in oracle database table i should provide to store this value ?
Currently have given varchar datatype and its throwing an error during import as :
Conversion error! Value: "00:12:01.615518000" to data type: "Number". Row ignored! Value is '00:12:01.615518000'. Cannot be converted to a decimal number object. Valid format: 'Unformatted'
You can store it as an INTERVAL DAY(0) TO SECOND(9) data type:
CREATE TABLE table_name (
time INTERVAL DAY(0) TO SECOND(9)
);
Then you can use TO_DSINTERVAL passing your value with '0 ' prepended to the start:
INSERT INTO table_name (time)
VALUES ( TO_DSINTERVAL('0 ' || '00:12:01.615518000') );
db<>fiddle here
If it is part of a date/time stamp then you could store it as DATE or TIMESTAMP if you could add the date component. Oracle doesn't have just a TIME data type.
If you can't add a date component to it, then assuming it is a time interval you could convert it to seconds or microseconds (lose the colons) and store it as a NUMBER.
If you want to maintain the exact formatting as shown, your only option is to store it as text using VARCHAR2 or something similar.
Related
I am creating a sqlite3 table that accepts records from a server. There should be one date/text column that also has a datetime DEFAULT value, so I can sync a record which times differ from the server's record.
I found a solution on this forum from here. The problem is it gives me the following error on executing the table creation script: sqlite3.OperationalError: default value of column [updated_at] is not constant.
Table is created:
cur.execute('CREATE TABLE IF NOT EXISTS emp_tb(\
emp_id INTEGER PRIMARY KEY NOT NULL,\
emp_names TEXT NOT NULL,\
emp_number TEXT NOT NULL UNIQUE,\
ent_id INTEGER NOT NULL,\
active INTEGER NOT NULL DEFAULT "0",\
updated_at TEXT NULL DEFAULT (datetime("now", "localtime")),\
syncstatus INTEGER NOT NULL DEFAULT "0")')
Should I create a trigger? or How can I have a default value in format ("YYYY-MM-DD HH:MM:SS.SSS") in case the update misses a spot?
Use single quotes (') for the datetime options. As mentioned in the comments, they will have to be escaped (because the query is delimited with single quotes).
I am trying to create an external table using Hive. Below is the Hive query I ran:
create external table trips_raw
(
VendorID int,
tpep_pickup_datetime timestamp,
tpep_dropoff_datetime timestamp
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' location '/user/taxi_trips/';
When I looked at the output from the 'trips_raw' table created by the query above, I saw that both the 'tpep_pickup_date_time' and 'tpep_dropoff_datetime' columns are 'NULL' in all rows. I have seen other threads talked about the reason being that the '1/1/2018 11:13:00 AM' timestamp format is not accepted by Hive, but problem is that's the timestamp format I have in my csv source data (as you can see from screenshot here).
I could specify those 2 timestamp columns as 'string' and Hive will be able to render them correctly, but I still would want those 2 columns to be 'timestamp' type so specifying those 2 columns as 'string' type is not a viable option here.
I had also tried the following technique using recommendation from this site (https://community.hortonworks.com/questions/55266/hive-date-time-problem.html) but had no success:
Create the 'trips_raw' table using 'string' as type for the 2 timestamp columns. This allows the resulting table to render the timestamps correctly, albeit in 'string' type. The Hive command I used is shown below:
create external table trips_raw
(
VendorID int,
tpep_pickup_datetime string,
tpep_dropoff_datetime string
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' location
'/user/taxi_trips/';
When I look at the resulting table, the dates are shown as string as you can see from this screenshot below.
But as I had mentioned earlier, I want the time columns to be in timestamp type and not string type. Therefore in the next 2 steps I tried to create a blank table and then insert the data from the table created from Step 1 but converting the string to timestamp this time.
Create an external blank table called 'trips_not_raw' using the following Hive commands:
create external table trips_not_raw
(VendorID int,
tpep_pickup_datetime timestamp,
tpep_dropoff_datetime timestamp
);
Insert data from 'trips_raw' table (which was mentioned earlier in this question), using the Hive commands below:
insert into table trips_not_raw select vendorid,
from_unixtime(unix_timestamp(tpep_pickup_datetime, 'MM/dd/yyyy HH:mm:ss
aa')) as tpep_pickup_datetime,
from_unixtime(unix_timestamp(tpep_dropoff_datetime, 'MM/dd/yyyy HH:mm:ss
aa')) as tpep_dropoff_datetime
from trips_raw;
Doing this inserts the rows into the blank table 'trips_not_raw', but the results from the 2 timestamp columns still showed as 'Null' as you can see from the screenshot below:
Is there a simple way to store the 2 time columns as 'timestamp' type and not 'string', but still be able to render them correctly in the output without seeing 'Null/None'?
I'm afraid you need to parse timestamp column and then cast string as timestamp. For example,
select cast(regexp_replace('1/1/2018 11:13:00 AM', '(\\d{1,2})/(\\d{1,2})/(\\d{4})\\s(\\d{2}:\\d{2}:\\d{2}) \\w{2}', '$3-$1-$2 $4') as timestamp)
You can create and use a macro function for convenience, e.g.,
create temporary macro parse_date (ts string)
cast(regexp_replace(ts, '(\\d{1,2})/(\\d{1,2})/(\\d{4})\\s(\\d{2}:\\d{2}:\\d{2}) \\w{2}', '$3-$1-$2 $4') as timestamp);
then use it as follows
select parse_date('1/1/2018 11:13:00 AM');
We have a schema where the partition key will be a date (yyyy-MM-dd) and we are thinking about choosing the data type between text and date for this partition key.
Does one data type provide benefits over the other and how would they differ in querying/storage?
Here is an example schema.
CREATE TABLE test.user_sessions (
sess_date date (or text),
sess_starttime timestamp,
event_type text,
total_req int,
ended_at timestamp
PRIMARY KEY (sess_date, sess_starttime)
);
Cassandra Date Type :
Value is a date with no corresponding time value; Cassandra encodes date as a 32-bit integer representing days since epoch (January 1, 1970)
Cassandra Text Type :
UTF-8 encoded string; 16 bit for each character
If you store date (yyyy-MM-dd) as date data type each entry will only take 32-bit. On the other hand if you store the date as text it will take 10*16 = 160 bit storage.
As per your comments, if you need maximum portability simply store the information as a timestamp (that is a 64-bit number) corresponding to the something like yyyy-MM-dd 00:00:00 (a truncated time stamp). You can't go wrong with an "universal" number...
I'm referring to one of the presentation slide from eBay - http://www.slideshare.net/jaykumarpatel/cassandra-data-modeling-best-practices
I want to try out the same thing. Hence, I create the following table.
CREATE TABLE ebay_event (
date text,
eventtype text,
time timeuuid,
payload text,
PRIMARY KEY((date, eventtype), time));
Then, in my PHP script, I will perform insert using the following insert statement.
insert into ebay_event(date, eventtype, time, payload) values('03031611', 'view', now(), 'additional data');
Instead of hard code value '03031611', is there a way to tell cassandra, to generate ddmmyyhh based on the now() value of timeuuid column?
No. There are no such functions available in cassandra. You will have to create it in the language you are using.
Values for the timestamp type are encoded as 64-bit signed integers
representing a number of milliseconds since the standard base time
known as the epoch: January 1 1970 at 00:00:00 GMT.
There are some functions available that can create date in YYYY-mm-dd format.
Date from timeuuid
I have to import data from Excel file to SSIS but i am facing a problem in date column,in excel sheet date format is yyyy/mm/dd and when it gets upload in database it get change into yyyy/dd/mmm format.
How to fix this?
Use the SUBSTRING function in the derived column while importing the date,
(LEN(TRIM(SUBSTRING(ReceivedDateTime,1,8))) > 0 ? (DT_DBDATE)(SUBSTRING(ReceivedDateTime,1,4) + "-" + SUBSTRING(ReceivedDateTime,5,2) + "-" + SUBSTRING(ReceivedDateTime,7,2)) : (DT_DBDATE)NULL(DT_WSTR,5))
If the Data is there then use Substring function to extract the exact date that sets in the DB or if the date does not exist then insert NULL in the DB.
I see two options:
Data Conversion Transformation to convert to a text string in
the appropriate format. Using SSIS data types.
Add a script task that converts the data type. Using VB data types
First Create Table into Your Database Using below Command
CREATE TABLE [dbo].[Manual] (
[Name] nvarchar(255),
[Location] nvarchar(255),
[Date] datetime
)
SET DATEFORMAT YDM
By using DATEFORMAT YDM ,Date Will import in YYYY/DD/MM Format .Before runnung package modify the package and at the time of Column mapping select The Check Box "Delete Rows in Destination Table" .
Then Execute the Package . It Will work .