I have a table like so..
CREATE TABLE [dbo].[Locations_Hours](
[LocationID] [int] NOT NULL,
[sun_open] [nvarchar](10) NULL,
[sun_close] [nvarchar](10) NULL,
[mon_open] [nvarchar](10) NULL,
[mon_close] [nvarchar](10) NULL,
[tue_open] [nvarchar](10) NULL,
[tue_close] [nvarchar](10) NULL,
[wed_open] [nvarchar](10) NULL,
[wed_close] [nvarchar](10) NULL,
[thu_open] [nvarchar](10) NULL,
[thu_close] [nvarchar](10) NULL,
[fri_open] [nvarchar](10) NULL,
[fri_close] [nvarchar](10) NULL,
[sat_open] [nvarchar](10) NULL,
[sat_close] [nvarchar](10) NULL,
[StoreNumber] [int] NULL,
[LocationHourID] [int] IDENTITY(1,1) NOT NULL,
CONSTRAINT [PK_Locations_Hours] PRIMARY KEY CLUSTERED
(
[LocationHourID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
And SubSonic 3 is generating a class with the following properties
int LocationID
string monopen
string monclose
string tueopen
string tueclose
string wedopen
string wedclose
string thuopen
string thuclose
string friopen
string friclose
string satopen
string satclose
string sunopen
string sunclose
int? StoreNumber
int LocationHourID
When I try to perform a query against this class like so..
var result = DB.LocationHours.Where(o => o.LocationID == _locationId);
This is the resulting SQL query that SubSonic generates.
SELECT [t0].[LocationHourID], [t0].[LocationID], [t0].[StoreNumber]
FROM [dbo].[Locations_Hours] AS t0
WHERE ([t0].[LocationID] = 4019)
I cannot figure out why SubSonic is omitting the nvarchar fields when it generates the SELECT statement. Anyone got any ideas?
I wasn't ever able to solve this problem. I ended up just executing a plain old DataReader and 'manually' populated my objects.
If anyone comes along later with an answer, I will change the accepted answer.
Related
Need help please.
I have excel files where some sheets are as follows:
enter image description here
enter image description here
I need to insert into the db table the matching columns from the excel file but for example WID and eename are both under column J and status column is in N but some sheets end at M.
When I print column names this is what I get:
ToPay,CompEntity,ProjectName,Details,Amount,Invoicenumber,PayInCurrency,PayOutCurrency,Paymentdate,PaidFrom,Addedby,Unnamed:13,Status
def readfile(self):
filepath = 'MatchPoin EUR - PT123.xlsx'
df1 = pd.read_excel(filepath, sheet_name=None, usecols="A,B,C,E,F,G,H,I,K,L,M,N")
df2 = pd.concat(df1, ignore_index=True)
return df2
def insert_to_db(self, df):
conn = self.db[0]
cursor = self.db[1]
col_list = df.columns.str.replace(' ', '')
insert_columns = ",".join(col for col in col_list)
print(insert_columns)
index_list = []
for _ in col_list:
index_list.append("%s")
index_str = ",".join(index for index in index_list)
values_list = df.values
insert_values_list = [tuple(values) for values in values_list]
num_rows_inserted = 0
# query = f"""insert into suppay_payments_stagingtable ({insert_columns}) values({index_str})"""
# print (insert_values_list)
for k, v in enumerate(insert_values_list):
try:
cursor.callproc('ins_beneficiaryfromfiles', v)
num_rows_inserted = num_rows_inserted + cursor.rowcount
conn.commit()
except Exception as e:
conn.rollback()
print(e)
logging.error(e)
My database table:
CREATE TABLE `suppay_payments_stagingtable` (
`payID` int(11) NOT NULL AUTO_INCREMENT,
`toPay` varchar(100) DEFAULT NULL,
`papayaEntityName` varchar(200) DEFAULT NULL,
`projectID` varchar(100) DEFAULT NULL,
`projectName` varchar(200) DEFAULT NULL,
`details` varchar(200) DEFAULT NULL,
`amount` decimal(10,3) DEFAULT NULL,
`invoiceNumber` varchar(50) DEFAULT NULL,
`payInCurrency` varchar(5) DEFAULT NULL,
`payOutCurrency` varchar(5) DEFAULT NULL,
`paymentDate` varchar(200) DEFAULT NULL,
`paidFrom` varchar(500) DEFAULT NULL,
`addedBy` varchar(200) DEFAULT NULL,
`WID` varchar(50) DEFAULT NULL,
`status` varchar(50) DEFAULT NULL,
`eename` varchar(50) DEFAULT NULL,
`bookKeepingNotes` varchar(200) DEFAULT NULL,
`noteDateBookeeping` date DEFAULT NULL,
`underOverPayment` varchar(200) DEFAULT NULL,
`opsNotes` varchar(200) DEFAULT NULL,
`opsNoteDate` varchar(200) DEFAULT NULL,
`sourcefilename` varchar(1000) DEFAULT NULL,
`sheetname` varchar(20) DEFAULT NULL,
`beneficiaryname` varchar(200) DEFAULT NULL,
`beneficiarycode` varchar(20) DEFAULT NULL,
`createdAt` datetime DEFAULT current_timestamp(),
`updateAt` datetime DEFAULT current_timestamp(),
`deposit` varchar(100) DEFAULT NULL,
PRIMARY KEY (`payID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
I have one Azure SQL server where I have several databases. I need to be able to query across these databases, and have at the moment solves this through external tables. A challange with this solution is that external tables does not support all the same data types as ordinary tables.
According to the following article the solution to incompatible data types are to use other similiar ones in the external table.
https://learn.microsoft.com/en-us/azure/sql-data-warehouse/sql-data-warehouse-tables-data-types#unsupported-data-types.
DDL for table in DB1
CREATE TABLE [dbo].[ActivityList](
[ID] [int] IDENTITY(1,1) NOT NULL,
[Registered] [datetime] NULL,
[RegisteredBy] [varchar](50) NULL,
[Name] [varchar](100) NULL,
[ak_beskrivelse] [ntext] NULL,
[ak_aktiv] [bit] NULL,
[ak_epost] [bit] NULL,
[Template] [text] NULL
CONSTRAINT [PK_ActivityList] PRIMARY KEY CLUSTERED
(
[ID] ASC
)WITH (STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
DDL for external table in DB2
CREATE EXTERNAL TABLE [dbo].[NEMDBreplicaActivityList]
(
[ID] [int] NOT NULL,
[Registered] [datetime] NULL,
[RegisteredBy] [varchar](50) NULL,
[Name] [varchar](100) NULL,
[ak_beskrivelse] [nvarchar](4000) NULL,
[ak_aktiv] [bit] NULL,
[ak_epost] [bit] NULL,
[Template] [varchar](900) NULL
)
WITH (DATA_SOURCE = [DS],SCHEMA_NAME = N'dbo',OBJECT_NAME = N'ActivityList')
Querying the external table NEMDBreplicaActivityList produces the following error
Error retrieving data from
server.database.windows.net.db1. The
underlying error message received was:
'PdwManagedToNativeInteropException ErrorNumber: 46723, MajorCode:
467, MinorCode: 23, Severity: 16, State: 1, ErrorInfo: ak_beskrivelse,
Exception of type
'Microsoft.SqlServer.DataWarehouse.Tds.PdwManagedToNativeInteropException'
was thrown.'.
I have tried defining the ak_beskrivelse column as other external table legal datatypes, such as varchar, with the same result.
Sadly I'm not allowed to edit the data type of columns in the db1 table.
I assume that the error is related to the data type. Any ideas how to fix it?
I solved a similar problem to this by creating a view over the source table which cast the text value as varchar(max), then pointed the external table to the view.
So:
CREATE VIEW tmpView
AS
SELECT CAST([Value] AS VARCHAR(MAX))
FROM [Sourcetable].
Then:
CREATE EXTERNAL TABLE [dbo].[tmpView]
(
[Value] VARCHAR(MAX) NULL
)
WITH (DATA_SOURCE = [myDS],SCHEMA_NAME = N'dbo',OBJECT_NAME = N'tmpView')
Creating the view and casting the text value worked perfect for me 😊
Thank you!
Created view vw_TestReport:
SELECT CAST([Report Date] AS VARCHAR(MAX)) AS [Report Date]
FROM dbo.TestReport
And created external table from view:
CREATE EXTERNAL TABLE [dbo].[TestReport](
[Report Date] [varchar](max) NULL
)
WITH (DATA_SOURCE = [REFToDB],SCHEMA_NAME = N'dbo',OBJECT_NAME = N'vw_TestReport')
I have the following code:
var statList = (from i in _dbContext.Screenshots
where EntityFunctions.TruncateTime(i.dateTimeServer.Value) >= startDate && EntityFunctions.TruncateTime(i.dateTimeServer.Value) <= endDate
group i by EntityFunctions.TruncateTime(i.dateTimeServer.Value)
into g
select new ScreenshotStatistic()
{
Date = g.Key.Value,
AllScreenshots = g.Count(),
ScreenshotsNoSilent = g.Where(p => p.version.IndexOf("silent") == 0).Count(),
ScreenshotsNoSilentWithViews = g.Where(p => p.version.IndexOf("silent") == 0 && p.viewsPage + p.viewsOriginal > 0).Count(),
ScreenshotsOnlySilent = g.Where(p => p.version.IndexOf("silent") >= 0).Count(),
ScreenshotsOnlySilentWithViews = g.Where(p => p.version.IndexOf("silent") >= 0 && p.viewsPage + p.viewsOriginal > 0).Count(),
ScreenshotsOnlyUploadViaSite = g.Where(p => p.version.IndexOf("UPLOAD_VIA_SITE") >= 0).Count(),
ScreenshotsOnlyUploadViaSiteWithViews = g.Where(p => p.version.IndexOf("UPLOAD_VIA_SITE") >= 0 && p.viewsPage + p.viewsOriginal > 0).Count()
}).ToList();
It works carefully for my local database, but I get "Operation timeout" when I try to connect to SQL Azure. As I understand, my request is not optimized. How can I do the request better?
table has the following structure:
CREATE TABLE [dbo].[Screenshots](
[id] [int] IDENTITY(1,1) NOT NULL,
[dateTimeClient] [datetime] NOT NULL,
[name] [nvarchar](500) NOT NULL,
[username] [varchar](50) NULL,
[filename] [nvarchar](50) NULL,
[description] [nvarchar](500) NULL,
[version] [varchar](50) NULL,
[lang] [varchar](50) NULL,
[dateTimeServer] [datetime] NULL CONSTRAINT [DF_Screenshots_dateTimeServer] DEFAULT (getdate()),
[isPublic] [bit] NOT NULL CONSTRAINT [DF_Screenshots_isPublic] DEFAULT ((0)),
[viewsPage] [int] NOT NULL CONSTRAINT [DF_Screenshots_viewsPage_1] DEFAULT ((0)),
[viewsThumb] [int] NOT NULL CONSTRAINT [DF_Screenshots_viewsThumb_1] DEFAULT ((0)),
[viewsOriginal] [int] NOT NULL CONSTRAINT [DF_Screenshots_viewsOriginal_1] DEFAULT ((0)),
[statusID] [int] NOT NULL,
CONSTRAINT [PK_Screenshots] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET ANSI_PADDING OFF
GO
ALTER TABLE [dbo].[Screenshots] WITH CHECK ADD CONSTRAINT [FK_Screenshots_ScreenshotStatuses] FOREIGN KEY([statusID])
REFERENCES [dbo].[ScreenshotStatuses] ([ID])
ON UPDATE CASCADE
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Screenshots] CHECK CONSTRAINT [FK_Screenshots_ScreenshotStatuses]
GO
EntityFunctions.TruncateTime is being translated into:
convert (datetime2, convert(varchar(255), [dateTimeServer], 102) , 102)
This is causing your query to be non-sargable, resulting in slow performance and timeout. I recommend making the following changes
First off, I do not see why you need to truncate the time in your where clause. I suggest you make this change:
where i.dateTimeServer.Value >= startDate && i.dateTimeServer.Value <= endDate
This will prevent the database from having to run the function on every record.
If the query still times out, I would change the group by to:
group i by new {
i.dateTimeServer.Value.Year,
i.dateTimeServer.Value.Month,
i.dateTimeServer.Value.Day
}
Functionally this is the same, but I believe it produces a more friendly translation:
DATEPART (year, dateTimeServer), DATEPART (month, dateTimeServer) etc..
Add an index to dateTimeServer
If all else fails, you will need to add a column to your table with the time portion truncated. Index it and use it in your query.
I'm trying to get my database within my application to search by phone numbers. I've got to the "Search Criteria Builder" screen, but I can't seem to get my query right.
This is the query I've tried and that is when I get the "converting from nvarchar to numeric" error:
SELECT phoneNumber, serviceDate, firstName, lastName, billingAddress, emailAddress, vehicleMake, vehicleModel, vehicleYear, vehicleColor, vehicleMileage
FROM [Table]
WHERE (phoneNumber LIKE #phoneNumber + N'%')
Here is the table created:
CREATE TABLE [dbo].[Table] (
[phoneNumber] NUMERIC (18) NOT NULL,
[serviceDate] DATE NULL,
[firstName] NCHAR (10) NULL,
[lastName] NCHAR (10) NULL,
[billingAddress] NVARCHAR (50) NULL,
[emailAddress] NVARCHAR (50) NULL,
[vehicleMake] NCHAR (10) NULL,
[vehicleModel] NCHAR (10) NULL,
[vehicleYear] INT NULL,
[vehicleColor] NCHAR (10) NULL,
[vehicleMileage] INT NULL,
CONSTRAINT [PK_Table] PRIMARY KEY CLUSTERED ([phoneNumber] ASC)
);
If the VB code is needed, I will provide it as well.
If you're going to treat phone "numbers" like strings, why are you storing them as numeric? You can do this:
WHERE CONVERT(VARCHAR(32), phoneNumber) LIKE #phonenumber + '%';
(Also note that you don't need Unicode here - there is no number with umlauts.)
But better to change the data type of the column IMHO.
Subsonic is returning 00000000-0000-0000-0000-000000000000 when I insert a record and try to get it's key after insert.
product.Save();
GUID = product.ProdID;
The record is inserted correctly with correct GUIDs.
Any idea on how to resolve this? I am using version 2.2.0.0
This is my table schema
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
SET ANSI_PADDING ON
GO
CREATE TABLE [dbo].[ISA_810_ControlTracking](
[ISAID] [uniqueidentifier] ROWGUIDCOL NOT NULL CONSTRAINT [DF_ISA_810_ControlTracking_ISAID] DEFAULT (newsequentialid()),
[ISA000_01_Authorization_Information_Qualifier] [varchar](2) NOT NULL,
[ISA000_02_Authorization_Information] [varchar](10) NOT NULL,
[ISA000_03_Security_Information_Qualifier] [varchar](2) NOT NULL,
[ISA000_04_Security_Information] [varchar](10) NOT NULL,
[ISA000_05_Interchange_Id_Qualifier] [varchar](2) NOT NULL,
[ISA000_06_Interchange_Sender_Id] [varchar](15) NOT NULL,
[ISA000_07_Interchange_Id_Qualifier] [varchar](2) NOT NULL,
[ISA000_08_Interchange_Receiver_Id] [varchar](15) NOT NULL,
[ISA000_09_Interchange_Date] [datetime] NOT NULL,
[ISA000_10_Interchange_Time] [datetime] NOT NULL,
[ISA000_11_Interchange_Control_Standards_Identifier] [varchar](1) NOT NULL,
[ISA000_12_Interchange_Control_Version_Number] [varchar](5) NOT NULL,
[ISA000_13_Interchange_Control_Number] [int] NOT NULL,
[ISA000_14_Acknowledgment_Requested] [varchar](1) NOT NULL,
[ISA000_15_Usage_Indicator] [varchar](1) NOT NULL,
[ISA000_16_Component_Element_Separator] [varchar](1) NOT NULL,
[IEA000_01_Number_Of_Included_Functional_Groups] [int] NOT NULL,
[IEA000_02_Interchange_Control_Number] [int] NOT NULL,
CONSTRAINT [PK_ISA_810_ControlTrackingIndex] PRIMARY KEY CLUSTERED
(
[ISAID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY],
CONSTRAINT [IX_ISA_810_ControlTracking] UNIQUE NONCLUSTERED
(
[ISA000_06_Interchange_Sender_Id] ASC,
[ISA000_08_Interchange_Receiver_Id] ASC,
[ISA000_13_Interchange_Control_Number] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET ANSI_PADDING OFF
Unlike IDENTITY types, applications have no way of determining the generated GUID on insert. While this is possible in T-SQL by using OUTPUT clause: INSERT ... OUTPUT inserted.$ROWGUIDCOL VALUES(...) most ORMs won't know how to do this. Given that the guid is a guid and doesn't matter who generates it, I'd recommend you generate it in the client prior of saving a new record, using UuidCreateSequential.