Linq to Entities, Operation timeout when I try to use group .. by - azure

I have the following code:
var statList = (from i in _dbContext.Screenshots
where EntityFunctions.TruncateTime(i.dateTimeServer.Value) >= startDate && EntityFunctions.TruncateTime(i.dateTimeServer.Value) <= endDate
group i by EntityFunctions.TruncateTime(i.dateTimeServer.Value)
into g
select new ScreenshotStatistic()
{
Date = g.Key.Value,
AllScreenshots = g.Count(),
ScreenshotsNoSilent = g.Where(p => p.version.IndexOf("silent") == 0).Count(),
ScreenshotsNoSilentWithViews = g.Where(p => p.version.IndexOf("silent") == 0 && p.viewsPage + p.viewsOriginal > 0).Count(),
ScreenshotsOnlySilent = g.Where(p => p.version.IndexOf("silent") >= 0).Count(),
ScreenshotsOnlySilentWithViews = g.Where(p => p.version.IndexOf("silent") >= 0 && p.viewsPage + p.viewsOriginal > 0).Count(),
ScreenshotsOnlyUploadViaSite = g.Where(p => p.version.IndexOf("UPLOAD_VIA_SITE") >= 0).Count(),
ScreenshotsOnlyUploadViaSiteWithViews = g.Where(p => p.version.IndexOf("UPLOAD_VIA_SITE") >= 0 && p.viewsPage + p.viewsOriginal > 0).Count()
}).ToList();
It works carefully for my local database, but I get "Operation timeout" when I try to connect to SQL Azure. As I understand, my request is not optimized. How can I do the request better?
table has the following structure:
CREATE TABLE [dbo].[Screenshots](
[id] [int] IDENTITY(1,1) NOT NULL,
[dateTimeClient] [datetime] NOT NULL,
[name] [nvarchar](500) NOT NULL,
[username] [varchar](50) NULL,
[filename] [nvarchar](50) NULL,
[description] [nvarchar](500) NULL,
[version] [varchar](50) NULL,
[lang] [varchar](50) NULL,
[dateTimeServer] [datetime] NULL CONSTRAINT [DF_Screenshots_dateTimeServer] DEFAULT (getdate()),
[isPublic] [bit] NOT NULL CONSTRAINT [DF_Screenshots_isPublic] DEFAULT ((0)),
[viewsPage] [int] NOT NULL CONSTRAINT [DF_Screenshots_viewsPage_1] DEFAULT ((0)),
[viewsThumb] [int] NOT NULL CONSTRAINT [DF_Screenshots_viewsThumb_1] DEFAULT ((0)),
[viewsOriginal] [int] NOT NULL CONSTRAINT [DF_Screenshots_viewsOriginal_1] DEFAULT ((0)),
[statusID] [int] NOT NULL,
CONSTRAINT [PK_Screenshots] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET ANSI_PADDING OFF
GO
ALTER TABLE [dbo].[Screenshots] WITH CHECK ADD CONSTRAINT [FK_Screenshots_ScreenshotStatuses] FOREIGN KEY([statusID])
REFERENCES [dbo].[ScreenshotStatuses] ([ID])
ON UPDATE CASCADE
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Screenshots] CHECK CONSTRAINT [FK_Screenshots_ScreenshotStatuses]
GO

EntityFunctions.TruncateTime is being translated into:
convert (datetime2, convert(varchar(255), [dateTimeServer], 102) , 102)
This is causing your query to be non-sargable, resulting in slow performance and timeout. I recommend making the following changes
First off, I do not see why you need to truncate the time in your where clause. I suggest you make this change:
where i.dateTimeServer.Value >= startDate && i.dateTimeServer.Value <= endDate
This will prevent the database from having to run the function on every record.
If the query still times out, I would change the group by to:
group i by new {
i.dateTimeServer.Value.Year,
i.dateTimeServer.Value.Month,
i.dateTimeServer.Value.Day
}
Functionally this is the same, but I believe it produces a more friendly translation:
DATEPART (year, dateTimeServer), DATEPART (month, dateTimeServer) etc..
Add an index to dateTimeServer
If all else fails, you will need to add a column to your table with the time portion truncated. Index it and use it in your query.

Related

Postgres: complex foreign key constraints

I have this schema
CREATE TABLE public.item (
itemid integer NOT NULL,
itemcode character(100) NOT NULL,
itemname character(100) NOT NULL,
constraint PK_ITEM primary key (ItemID)
);
create unique index ak_itemcode on Item(ItemCode);
CREATE TABLE public.store (
storeid character(20) NOT NULL,
storename character(80) NOT NULL,
constraint PK_STORE primary key (StoreID)
);
CREATE TABLE public.storeitem (
storeitemid integer NOT NULL,
itemid integer NOT NULL,
storeid character(20) NOT NULL,
constraint PK_STOREITEM primary key (ItemID, StoreID),
foreign key (StoreID) references Store(StoreID),
foreign key (ItemID) references Item(ItemID)
);
create unique index ak_storeitemid on StoreItem (StoreItemID);
And here is the data on those tables
insert into Item (ItemID, ItemCode,ItemName)
Values (1,'abc','abc');
insert into Item (ItemID, ItemCode,ItemName)
Values (2,'def','def');
insert into Item (ItemID, ItemCode,ItemName)
Values (3,'ghi','ghi');
insert into Item (ItemID, ItemCode,ItemName)
Values (4,'lmno','lmno');
insert into Item (ItemID, ItemCode,ItemName)
Values (5,'xyz','xyz');
insert into Store (StoreID, StoreName)
Values ('B1','B1');
insert into StoreItem (StoreItemID, StoreID, ItemID)
Values (1,'B1',1);
insert into StoreItem (StoreItemID, StoreID, ItemID)
Values (2,'B1',2);
insert into StoreItem (StoreItemID, StoreID, ItemID)
Values (3,'B1',3);
Now I created this new table
CREATE TABLE public.szdata (
storeid character(20) NOT NULL,
itemcode character(100) NOT NULL,
textdata character(20) NOT NULL,
constraint PK_SZDATA primary key (ItemCode, StoreID)
);
I want to have the foreign key constraints set so that it will fail when you try to insert record which is not in StoreItem. For example this must fail
insert into SZData (StoreID, ItemCode, TextData)
Values ('B1', 'xyz', 'text123');
and this must pass
insert into SZData (StoreID, ItemCode, TextData)
Values ('B1', 'abc', 'text123');
How do I achieve this without complex triggers but using table constraints?
I prefer solution without triggers. SZData table is just for accepting input from external world and it is for single purpose.
Also database import export must not be impacted
I figured out having a function to execute on constraint will solve this issue.
The function is_storeitem does the validation. I believe this feature can be used for even complex validations
create or replace function is_storeitem(pItemcode nchar(40), pStoreId nchar(20)) returns boolean as $$
select exists (
select 1
from public.storeitem si, public.item i, public.store s
where si.itemid = i.itemid and i.itemcode = pItemcode and s.Storeid = pStoreId and s.storeid = si.storeid
);
$$ language sql;
create table SZData
(
StoreID NCHAR(20) not null,
ItemCode NCHAR(100) not null,
TextData NCHAR(20) not null,
constraint PK_SIDATA primary key (ItemCode, StoreID),
foreign key (StoreID) references Store(StoreID),
foreign key (ItemCode) references Item(ItemCode),
CONSTRAINT ck_szdata_itemcode CHECK (is_storeitem(Itemcode,StoreID))
);
This perfectly works with postgres 9.6 or greater.

Could not add table 'SELECT('

Error given when adding query that runs in SQL developer but not in MS Query. Seems to not like my nested query.
Code I am using:
SELECT ORDER_DATE
,SALES_ORDER_NO
,CUSTOMER_PO_NUMBER
,DELIVER_TO
,STATUS
,ITEM_NUMBER
,DESCRIPTION
,ORD_QTY
,SUM(QUANTITY) AS ON_HAND
,PACKAGE_ID
,PACKAGE_STATUS
,MAX(TRAN_DATE) AS LAST_TRANSACTION
,MIN(DAYS) AS DAYS
FROM (
SELECT TRUNC(SH.ORDER_DATE) AS ORDER_DATE
,SH.SALES_ORDER_NO
,SH.CUSTOMER_PO_NUMBER
,SH.SHIP_CODE AS DELIVER_TO
,SH.STATUS
,SB.ITEM_NUMBER
,IM.DESCRIPTION
,SB.ORD_QTY
,BID.QUANTITY
,SPM.PACKAGE_ID
,CASE
WHEN SPM.SHIPPED = 'Y'
THEN 'SHIPPED'
WHEN SPM.STATUS = 'C'
THEN 'PACKED'
WHEN BID.QUANTITY IS NOT NULL
THEN 'AVAILABLE'
WHEN BID.QUANTITY IS NULL
THEN 'UNAVAILABLE'
END AS PACKAGE_STATUS
,CASE
WHEN SPM.SHIPPED = 'Y'
THEN TRUNC(SPM.BILLING_DATE)
WHEN SPM.STATUS = 'C'
THEN TRUNC(SPM.END_TIME)
WHEN BID.QUANTITY IS NOT NULL
THEN TRUNC(BID.ACTIVATION_TIME)
END AS TRAN_DATE
,CASE
WHEN SPM.SHIPPED = 'Y'
THEN ROUND(SYSDATE - SPM.BILLING_DATE, 0)
WHEN SPM.STATUS = 'C'
THEN ROUND(SYSDATE - SPM.END_TIME, 0)
WHEN BID.QUANTITY IS NOT NULL
THEN ROUND(SYSDATE - BID.ACTIVATION_TIME, 0)
END AS DAYS
FROM SO_HEADER SH
LEFT JOIN SO_BODY SB ON SB.SO_HEADER_TAG = SH.SO_HEADER_TAG
LEFT JOIN SO_PACKAGE_MASTER SPM ON SPM.PACKAGE_ID = SB.PACKAGE_ID
LEFT JOIN ITEM_MASTER IM ON IM.ITEM_NUMBER = SB.ITEM_NUMBER
LEFT JOIN V_BIN_ITEM_DETAIL BID ON BID.ITEM_NUMBER = SB.ITEM_NUMBER
WHERE SH.ORDER_TYPE = 'MSR'
AND (
SB.REASON_CODE IS NULL
OR SB.REASON_CODE NOT LIKE 'CANCEL%'
)
AND (
IM.DESCRIPTION NOT LIKE '%CKV%'
OR IM.DESCRIPTION IS NULL
AND IM.ITEM_NUMBER IS NOT NULL
)
)
WHERE PACKAGE_STATUS <> 'SHIPPED'
GROUP BY ORDER_DATE
,SALES_ORDER_NO
,CUSTOMER_PO_NUMBER
,DELIVER_TO
,STATUS
,ITEM_NUMBER
,DESCRIPTION
,ORD_QTY
,PACKAGE_ID
,PACKAGE_STATUS
ORDER BY (
CASE PACKAGE_STATUS
WHEN 'AVAILABLE'
THEN 1
WHEN 'UNAVAILABLE'
THEN 2
WHEN 'PACKED'
THEN 3
END
)
,LAST_TRANSACTION;
Is there any option I can select that will allow me to run this query?

Azure SQL db query external Azure SQL DB - PPDwManagedToNativeInteropException

I have one Azure SQL server where I have several databases. I need to be able to query across these databases, and have at the moment solves this through external tables. A challange with this solution is that external tables does not support all the same data types as ordinary tables.
According to the following article the solution to incompatible data types are to use other similiar ones in the external table.
https://learn.microsoft.com/en-us/azure/sql-data-warehouse/sql-data-warehouse-tables-data-types#unsupported-data-types.
DDL for table in DB1
CREATE TABLE [dbo].[ActivityList](
[ID] [int] IDENTITY(1,1) NOT NULL,
[Registered] [datetime] NULL,
[RegisteredBy] [varchar](50) NULL,
[Name] [varchar](100) NULL,
[ak_beskrivelse] [ntext] NULL,
[ak_aktiv] [bit] NULL,
[ak_epost] [bit] NULL,
[Template] [text] NULL
CONSTRAINT [PK_ActivityList] PRIMARY KEY CLUSTERED
(
[ID] ASC
)WITH (STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
DDL for external table in DB2
CREATE EXTERNAL TABLE [dbo].[NEMDBreplicaActivityList]
(
[ID] [int] NOT NULL,
[Registered] [datetime] NULL,
[RegisteredBy] [varchar](50) NULL,
[Name] [varchar](100) NULL,
[ak_beskrivelse] [nvarchar](4000) NULL,
[ak_aktiv] [bit] NULL,
[ak_epost] [bit] NULL,
[Template] [varchar](900) NULL
)
WITH (DATA_SOURCE = [DS],SCHEMA_NAME = N'dbo',OBJECT_NAME = N'ActivityList')
Querying the external table NEMDBreplicaActivityList produces the following error
Error retrieving data from
server.database.windows.net.db1. The
underlying error message received was:
'PdwManagedToNativeInteropException ErrorNumber: 46723, MajorCode:
467, MinorCode: 23, Severity: 16, State: 1, ErrorInfo: ak_beskrivelse,
Exception of type
'Microsoft.SqlServer.DataWarehouse.Tds.PdwManagedToNativeInteropException'
was thrown.'.
I have tried defining the ak_beskrivelse column as other external table legal datatypes, such as varchar, with the same result.
Sadly I'm not allowed to edit the data type of columns in the db1 table.
I assume that the error is related to the data type. Any ideas how to fix it?
I solved a similar problem to this by creating a view over the source table which cast the text value as varchar(max), then pointed the external table to the view.
So:
CREATE VIEW tmpView
AS
SELECT CAST([Value] AS VARCHAR(MAX))
FROM [Sourcetable].
Then:
CREATE EXTERNAL TABLE [dbo].[tmpView]
(
[Value] VARCHAR(MAX) NULL
)
WITH (DATA_SOURCE = [myDS],SCHEMA_NAME = N'dbo',OBJECT_NAME = N'tmpView')
Creating the view and casting the text value worked perfect for me 😊
Thank you!
Created view vw_TestReport:
SELECT CAST([Report Date] AS VARCHAR(MAX)) AS [Report Date]
FROM dbo.TestReport
And created external table from view:
CREATE EXTERNAL TABLE [dbo].[TestReport](
[Report Date] [varchar](max) NULL
)
WITH (DATA_SOURCE = [REFToDB],SCHEMA_NAME = N'dbo',OBJECT_NAME = N'vw_TestReport')

timeout when running report in Acumatica ERP system

I have a problem when I'm running customized report,
I used this query View to generate the report: https://drive.google.com/open?id=0B0Aenr4I_Yz9RWRSN0RtaXhoSG8
After Executing finished, report doesn't work and show the timeout error.
and then I tried to trace this query in the system, and get this query:
SELECT vTranPeriodMultiCury.[BatchNbr], vTranPeriodMultiCury.[RefNbr], vTranPeriodMultiCury.[InvoiceNbr], vTranPeriodMultiCury.[JobOrderNbr], vTranPeriodMultiCury.[Customer_Vendor_ID], vTranPeriodMultiCury.[SourceCredit], vTranPeriodMultiCury.[SourceDebit], vTranPeriodMultiCury.[BaseCredit], vTranPeriodMultiCury.[BaseDebit], vTranPeriodMultiCury.[Module], vTranPeriodMultiCury.[TranDate], vTranPeriodMultiCury.[TranType], vTranPeriodMultiCury.[Customer_Vendor_CD], vTranPeriodMultiCury.[Customer_Vendor_Name], vTranPeriodMultiCury.[TranDesc], vTranPeriodMultiCury.[Curyid], vTranPeriodMultiCury.[CuryRate], vTranPeriodMultiCury.[FinPeriodID], vTranPeriodMultiCury.[AccountCD], vTranPeriodMultiCury.[AccountDesc], vTranPeriodMultiCury.[BaseBegBalSummary], vTranPeriodMultiCury.[BaseEndBalSummary], vTranPeriodMultiCury.[SourceBegBalSummary], vTranPeriodMultiCury.[SourceEndBalSummary], vTranPeriodMultiCury.[BaseBegBalIDR], vTranPeriodMultiCury.[BaseEndBalIDR], vTranPeriodMultiCury.[SourceBegBalIDR], vTranPeriodMultiCury.[SourceEndBalIDR], vTranPeriodMultiCury.[BaseBegBalJPY], vTranPeriodMultiCury.[BaseEndBalJPY], vTranPeriodMultiCury.[SourceBegBalJPY], vTranPeriodMultiCury.[SourceEndBalJPY], vTranPeriodMultiCury.[BaseBegBalUSD], vTranPeriodMultiCury.[BaseEndBalUSD], vTranPeriodMultiCury.[SourceBegBalUSD], vTranPeriodMultiCury.[SourceEndBalUSD], Account.[AccountCD], Account.[Type], Account.[NoteID], NULL, NULL, NULL, Batch.[Module], Batch.[BatchNbr], Batch.[CuryInfoID], Batch.[NoteID], NULL, NULL, NULL FROM vTranPeriodMultiCury vTranPeriodMultiCury LEFT JOIN Account Account ON (Account.CompanyID = 2) AND [Account].[DeletedDatabaseRecord] = 0 AND (vTranPeriodMultiCury.[AccountID] = Account.[AccountID]) LEFT JOIN Batch Batch ON (Batch.CompanyID = 2) AND [Batch].[DeletedDatabaseRecord] = 0 AND (vTranPeriodMultiCury.[BatchNbr] = Batch.[BatchNbr]) WHERE (vTranPeriodMultiCury.CompanyID = 2) AND (vTranPeriodMultiCury.BranchID IS NULL OR vTranPeriodMultiCury.BranchID IN (1, 2, 3)) AND (((vTranPeriodMultiCury.[AccountCD] >= NULL OR NULL IS NULL ) AND (vTranPeriodMultiCury.[AccountCD] <= NULL OR NULL IS NULL ) AND (vTranPeriodMultiCury.[FinPeriodID] = '052017' OR '052017' IS NULL ))) ORDER BY vTranPeriodMultiCury.[AccountCD] ASC, vTranPeriodMultiCury.[Curyid] ASC, vTranPeriodMultiCury.[FinPeriodID] ASC, vTranPeriodMultiCury.[TranDate] ASC, vTranPeriodMultiCury.[BatchNbr] ASC OPTION(OPTIMIZE FOR UNKNOWN)
My question is in which part of web.config should be possible to change the timeout of executing sql query of report designer ?
Probably you should increase this executionTimeOut in Location->system.web->httpRuntime in web.config:
<httpRuntime executionTimeout="300" requestValidationMode="2.0" maxRequestLength="1048576" />
Also try to execute the Following SQL Query on your SQL Server for your Acumatica ERP's database:
use YOUR_DATABASE_NAME
go
exec sp_configure 'remote query timeout (s)',6000
RECONFIGURE WITH OVERRIDE;

Subsonic2.2 and NEWSEQUENTIALID() Primary Key Column

Subsonic is returning 00000000-0000-0000-0000-000000000000 when I insert a record and try to get it's key after insert.
product.Save();
GUID = product.ProdID;
The record is inserted correctly with correct GUIDs.
Any idea on how to resolve this? I am using version 2.2.0.0
This is my table schema
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
SET ANSI_PADDING ON
GO
CREATE TABLE [dbo].[ISA_810_ControlTracking](
[ISAID] [uniqueidentifier] ROWGUIDCOL NOT NULL CONSTRAINT [DF_ISA_810_ControlTracking_ISAID] DEFAULT (newsequentialid()),
[ISA000_01_Authorization_Information_Qualifier] [varchar](2) NOT NULL,
[ISA000_02_Authorization_Information] [varchar](10) NOT NULL,
[ISA000_03_Security_Information_Qualifier] [varchar](2) NOT NULL,
[ISA000_04_Security_Information] [varchar](10) NOT NULL,
[ISA000_05_Interchange_Id_Qualifier] [varchar](2) NOT NULL,
[ISA000_06_Interchange_Sender_Id] [varchar](15) NOT NULL,
[ISA000_07_Interchange_Id_Qualifier] [varchar](2) NOT NULL,
[ISA000_08_Interchange_Receiver_Id] [varchar](15) NOT NULL,
[ISA000_09_Interchange_Date] [datetime] NOT NULL,
[ISA000_10_Interchange_Time] [datetime] NOT NULL,
[ISA000_11_Interchange_Control_Standards_Identifier] [varchar](1) NOT NULL,
[ISA000_12_Interchange_Control_Version_Number] [varchar](5) NOT NULL,
[ISA000_13_Interchange_Control_Number] [int] NOT NULL,
[ISA000_14_Acknowledgment_Requested] [varchar](1) NOT NULL,
[ISA000_15_Usage_Indicator] [varchar](1) NOT NULL,
[ISA000_16_Component_Element_Separator] [varchar](1) NOT NULL,
[IEA000_01_Number_Of_Included_Functional_Groups] [int] NOT NULL,
[IEA000_02_Interchange_Control_Number] [int] NOT NULL,
CONSTRAINT [PK_ISA_810_ControlTrackingIndex] PRIMARY KEY CLUSTERED
(
[ISAID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY],
CONSTRAINT [IX_ISA_810_ControlTracking] UNIQUE NONCLUSTERED
(
[ISA000_06_Interchange_Sender_Id] ASC,
[ISA000_08_Interchange_Receiver_Id] ASC,
[ISA000_13_Interchange_Control_Number] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET ANSI_PADDING OFF
Unlike IDENTITY types, applications have no way of determining the generated GUID on insert. While this is possible in T-SQL by using OUTPUT clause: INSERT ... OUTPUT inserted.$ROWGUIDCOL VALUES(...) most ORMs won't know how to do this. Given that the guid is a guid and doesn't matter who generates it, I'd recommend you generate it in the client prior of saving a new record, using UuidCreateSequential.

Resources