I want to use DataSync on my current application, using IHP 0.16. I believe I have followed all the installation steps in FrontController and Routes.
I have a characters table with a user_id column connected to the users table. I have set the policy on the characters table resulting in this generated SQL:
CREATE POLICY "Users can manage their characters" ON characters USING (user_id = ihp_user_id()) WITH CHECK (user_id = ihp_user_id());
ALTER TABLE characters ENABLE ROW LEVEL SECURITY;
Trying to run this in the JavaScript console
await query("characters").fetch()
I get this error in JavaScript output:
And this error in IHP output:
Query (2.119753ms): "SELECT relrowsecurity FROM pg_class WHERE oid = ?::regclass" ["characters"]
Query (0.111442ms): "SET LOCAL ROLE ?" [Identifier {fromIdentifier = "ihp_authenticated"}]
Query (0.130888ms): "SET LOCAL rls.ihp_user_id = ?" Only {fromOnly = Just 0d7b46b1-bcb4-46a2-bf77-ad27dace8416}
FormatError {fmtMessage = "1 single '?' characters, but 3 parameters", fmtQuery = "SELECT ? FROM ??", fmtParams = ["*","characters",""]}
This seems to be another error than the row level security error in the DataSync tutorial in the IHP docs. Any idea on what causes this error?
This is a known bug in IHP v0.16.0. It's already fixed in master
It's best to use IHP DataSync with the version mentioned in the introduction text at https://ihp.digitallyinduced.com/Guide/realtime-spas.html :)
There's btw a workaround for the bug if you don't want to upgrade: You always need to specify an order by, like await query("characters").orderBy('createdAt').fetch()
Related
I'm trying to do a simple incremental update from an on-prem database as source to Azure SQL database based on a varchar column called "RP" in On-Prem database that contains "date+staticdescription" for example: "20210314MetroFactory"
1- I've created a Lookup activity called Lookup1 using a table created in Azure SQL Database and uses this Query
"Select RP from SubsetwatermarkTable"
2- I've created a Copy data activity where the source settings have this Query
"Select * from SourceDevSubsetTable WHERE RP NOT IN '#{activity('Lookup1').output.value}'"
When debugging -- I'm getting the error:
Failure type: User configuration issue
Details: Failure happened on 'Source' side.
'Type=System.Data.SqlClient.SqlException,Message=Incorrect syntax near
'[{"RP":"20210307_1Plant
1KAO"},{"RP":"20210314MetroFactory"},{"RP":"20210312MetroFactory"},{"RP":"20210312MetroFactory"},{"RP":"2'.,Source=.Net
SqlClient Data
Provider,SqlErrorNumber=102,Class=15,ErrorCode=-2146232060,State=1,Errors=[{Class=15,Number=102,State=1,Message=Incorrect
syntax near
'[{"RP":"20210311MetroFactory"},{"RP":"20210311MetroFactory"},{"RP":"202103140MetroFactory"},{"RP":"20210308MetroFactory"},{"RP":"2'.,},],'
Can anyone tell me what I am doing wrong and how to fix it even if it requires creating more activities.
Note: There is no LastModifiedDate column in the table. Also I haven't yet created the StoredProcedure that will update the Lookup table when it is done with the incremental copy.
Steve is right as to why it is failling and the query you need in the Copy Data.
As he says, you want a comma-separated list of quoted values to use in your IN clause.
You can get this more easily though - from your Lookup directly using this query:-
select stuff(
(
select ','''+rp+''''
from subsetwatermarktable
for xml path('')
)
, 1, 1, ''
) as in_clause
The sub-query gets the comma separated list with quotes around each rp-value, but has a spurious comma at the start - the outer query with stuff removes this.
Now tick the First Row Only box on the Lookup and change your Copy Data source query to:
select *
from SourceDevSubsetTable
where rp not in (#{activity('lookup').output.firstRow.in_clause})
The result of #activity('Lookup1').output.value is an array like your error shows
[{"RP":"20210307_1Plant
1KAO"},{"RP":"20210314MetroFactory"},{"RP":"20210312MetroFactory"},{"RP":"20210312MetroFactory"},{"RP":"2'.,Source=.Net
SqlClient Data
Provider,SqlErrorNumber=102,Class=15,ErrorCode=-2146232060,State=1,Errors=[{Class=15,Number=102,State=1,Message=Incorrect
syntax near
'[{"RP":"20210311MetroFactory"},{"RP":"20210311MetroFactory"},{"RP":"202103140MetroFactory"},{"RP":"20210308MetroFactory"},{"RP":"2'.,},]
However, your SQL should be like this:Select * from SourceDevSubsetTable WHERE RP NOT IN ('20210307_1Plant 1KAO','20210314MetroFactory',...).
To achieve this in ADF, you need to do something like this:
create three variables like the following screenshot:
loop your result of #activity('Lookup1').output.value and append 'item().RP' to arrayvalues:
expression:#activity('Lookup1').output.value
expression:#concat(variables('apostrophe'),item().RP,variables('apostrophe'))
3.cast arrayvalues to string and add parentheses by Set variable activity
expression:#concat('(',join(variables('arrayvalues'),','),')')
4.copy to your Azure SQL database
expression:Select * from SourceDevSubsetTable WHERE RP NOT IN #{variables('stringvalues')}
I'm working with the Azure Core API. Using the web based data explorer, I'm able to successfully run a query that has multiple aggregations across multiple partitions. Yet when I attempt to run such a query from my shell using the Azure client library for Python (pip install azure-cosmos==4.0.0), I get an error message. I've tried two variations of the query, where one included the partition key and one didn't. Both queries returned the same error message.
container = database.get_container_client('some_container')
query1 = "select c.fmonth, c.fquarter, c.fyear, sum(c.revenue) as actual_revenue__sum, sum(c.predicted_revenue_m1) as predicted_revenue__sum from c where c.fyear=2020 group by c.fmonth, c.fquarter, c.fyear"
query2 = "select c.fmonth, c.fquarter, c.fyear, sum(c.revenue) as actual_revenue__sum, sum(c.predicted_revenue_m1) as predicted_revenue__sum from c where c.date_start >='2020-01-01' and c.date_start < '2021-01-01' group by c.fmonth, c.fquarter, c.fyear"
res = container.query_items(query1, enable_cross_partition_query=True)
Error Message Returned:
CosmosHttpResponseError: (BadRequest) Message: {"Errors":["Cross partition query only supports 'VALUE ' for aggregates."]}
ActivityId: 4961b99e-7032-4eac-ae84-2c8cab03a496, Microsoft.Azure.Documents.Common/2.11.0
Errors out for aggregates on multiple partitions, with enable cross
partition query set to true, but no "value" keyword present
If you want to use aggregates on multiple partitions,select value count(1) from c,this will work.
But python sdk doesn't support group by now.
Refer to this document,.net sdk and js sdk support group by.Other sdk will support later.
Hope this can help you.
I'm getting an email regarding a Schema Issue on my SQL Azure Database and it reads something like this:
"Invalid column name 'False'" Error code : 207
However it doesn't appear to have any more information and I wouldn't even know where to start because the schema was created before I started in the project, but I suspect this is coming from a Stored Procedure or View.
This is all being reported in Azure Portal, but I have no idea how to proceed. Is there any other way to get the suggestions on Azure Portal in SSMS? I already tried Tuning Advisor and I got an error that SQL Azure was not supported.
I use this query to to search objects in this scenario. This will find every occurrence of false and should help you find the offending column. This will find every occurrence of whatever you define #searchName as.
I found this on Pinal Dave's blog, way back. This works in Azure SQL.
declare #searchName varchar(50) = 'false'
select #searchName as SearchName,
OBJECT_SCHEMA_NAME(OBJECT_ID) + '.' + OBJECT_NAME(OBJECT_ID) as ObjectName,
[definition]
from sys.sql_modules
where definition LIKE '%' + #searchName + '%'
order by OBJECT_SCHEMA_NAME(OBJECT_ID) + '.' + OBJECT_NAME(OBJECT_ID)
I created a 4,000-character VARCHAR2 field in an Oracle table. I am inserting string values into the field using LINQ to Entities with Visual Studio 2010, .NET Framework 4, and ODAC 11.2 Release 4 and Oracle Developer Tools for Visual Studio (11.2.0.3.0). When I try to insert a string value greater than 1,999 characters, I get the following inner exception:
Oracle.DataAccess.Client.OracleException
ORA-00932: inconsistent datatypes: expected - got NCLOB
However, I can insert a 4,000 character string value into the field without any issue when doing so using SQL Developer.
There is a known ODAC bug (source #2) in which there is a 2,000 character limit when saving to an XMLTYPE field, but I am not saving to an XMLTYPE field. I have Oracle.DataAccess 2.112.3.0 in my GAC, and I considered updating to release 5 (11.2.0.3.20) of the aforementioned Oracle software, but "Oracle Developer Tools for Visual Studio" is the only component that appears to have been updated from release 4, and I believe that "Oracle Data Provider for .NET 4" is the component that needs updating. In my .NET project, System.Data.Entity and System.Data.OracleClient are both runtime version 4.0.30319.
Anyway, I am just wondering if anyone else has encountered this error, and if so, if any solution has been found - aside from the one in the Oracle forum thread that is linked above that proposes using stored procedures as a workaround. Google tells me that people are encountering this error only when working with XMLTYPE fields, but I can't be the only person who is encountering this error when working with a VARCHAR2 field, can I?
(FWIW, I am also hoping to receive a response to my post as user "997340" in the Oracle forum thread that is linked above. If I receive a useful response, I'll be sure to share the knowledge on this end.)
EDIT: In case it helps, below are the two blocks in my code that are failing. I created the second block when troubleshooting the first, just to see if there was any difference. I get the exception when checking to see if the string values were already inserted (the "if" statements), and when actually inserting the string values (the "AddObject" statements).
1:
if (!(from q in db.MSG_LOG_MESSAGE where q.MESSAGE == msg select q.MESSAGE).Any())
{
db.MSG_LOG_MESSAGE.AddObject(new MSG_LOG_MESSAGE { MESSAGE = msg });
db.SaveChanges();
}
2:
if (!db.MSG_LOG_MESSAGE.Any(q => q.MESSAGE == msg))
{
db.MSG_LOG_MESSAGE.AddObject(new MSG_LOG_MESSAGE { MESSAGE = msg });
db.SaveChanges();
}
APRIL 3 UPDATE:
I was able to trace the SQL that is being sent to Oracle from the "if" statement in the first code block above. Here it is:
SELECT
CASE WHEN ( EXISTS (SELECT
1 AS "C1"
FROM "SEC"."MSG_LOG_MESSAGE" "Extent1"
WHERE ("Extent1"."MESSAGE" = :p__linq__0)
)) THEN 1 WHEN ( NOT EXISTS (SELECT
1 AS "C1"
FROM "SEC"."MSG_LOG_MESSAGE" "Extent2"
WHERE ("Extent2"."MESSAGE" = :p__linq__0)
)) THEN 0 END AS "C1"
FROM ( SELECT 1 FROM DUAL ) "SingleRowTable1" ;
Unfortunately, the DBA that I worked with did not provide me with the value of the "p_linq_0" parameter, but as previously stated, when it is over 1,999 characters, an exception occurs. (When this SQL was traced, I passed a 4,000-character string as the parameter, and of course an exception occurred.) The DBA also mentioned something about certain SQL clients - such as SQLPlus - not being able to handle VARCHAR2s over 2,000 characters. I did not entirely follow. Whether using SQLPlus, SQL Developer, or any other tool, Oracle is still going to be querying a 4,000-character VARCHAR2 field. Plus, my magic number is 1,999 characters; not 2,000 characters. Did the DBA perhaps mean there is a limitation with how many characters can be in a parameter? More importantly, when I execute this SQL in SQL Developer and I enter a 4,000-character string for the parameter, it works perfectly. So I am still utterly confused about why it is not working via LINQ to Entities. I also tried the following code in my program to run a similar query with a 4,000-character string in the "msg" variable, which worked perfectly as well:
using Oracle.DataAccess;
using Oracle.DataAccess.Client;
using System.Data;
...
OracleConnection conn = new OracleConnection("Data Source=[MASKED];User Id=[MASKED];Password=[MASKED]");
conn.Open();
OracleCommand cmd = new OracleCommand();
cmd.Connection = conn;
cmd.CommandText = "SELECT message FROM msg_log_message WHERE message = '" + msg + "'";
cmd.CommandType = CommandType.Text;
OracleDataReader dr = cmd.ExecuteReader();
dr.Read();
string result1 = dr.GetString(0);
conn.Dispose();
For now, I am still pointing fingers at ODAC being buggy as it pertains to LINQ to Entities...
The latest ODP.NET documentation - for "11.2 Release 5 Production (11.2.0.3.0)" from September 2012 - states the following known issue under the "Entity Framework Related Tips, Limitations and Known Issues" section, which addresses the error from the "if" statements in the question's code blocks:
An "ORA-00932 : inconsistent datatypes" error can be encountered if a string of 2,000 or more characters, or a byte array with 4,000 bytes or more in length, is bound in a WHERE clause of a LINQ/ESQL query. The same error can be encountered if an entity property that maps to a BLOB, CLOB, NCLOB, LONG, LONG RAW, XMLTYPE column is used in a WHERE clause of a LINQ/ESQL query.
Older ODP.NET documentation - for "Release 11.2.0.3.0 Production" from May 2011 - states the same known issue, so apparently this has been a known issue for a while.
Neither of the aforementioned documentation mentions encountering the same error from the "AddObject" statements in the question's code blocks, but that issue is very similar to another known issue for XMLType fields that is mentioned:
An "ORA-00932: inconsistent datatypes: expected - got NCLOB" error will be encountered when trying to bind a string that is equal to or greater than 2,000 characters in length to an XMLType column or parameter. [Bug 12630958]
I'm just getting started with Subsonic 3.0 ActiveRecord and am trying to implement a batch query like the one in the SubSonic docs. I'm using a batch so I can query a User and a list of the users Orders in one shot.
When I call the BatchQuery.Queue() method, adding my "select user" query, SubSonic throws the following exception:
System.InvalidOperationException : Can't decide which property to consider the Key - you can create one called 'ID' or mark one with SubSonicPrimaryKey attribute
The code is as follows:
var db = new MyDB();
var userQuery = from u in db.Users //gets user by uid
where u.uid == 1
select u;
var provider = ProviderFactory.GetProvider();
var batch = new BatchQuery(provider);
batch.Queue(userQuery); //exception here
//create and add "select users orders" query here...
First things first - Why this error? My SubSonic Users object knows it's PK. "uid" is the PK in the database and the generated code reflects this. And I thought SubSonicPrimaryKey attribute was for the SimpleRepository? Is this way of batching not for ActiveRecord?
I could ask a number of other questions, but I'll leave it at that. If anyone can help me figure out what is going on and how to issue 2 batched queries I'd be grateful!
Edit - after further investigation
I ran through the source code with the debugger. Adam is correct - the ToSchemaTable() method in Objects.cs is apparently building out my schema and failing to find a PK. At the very end, it tries to find a column property named "ID" and flags this as the PK, otherwise it throws the exception. I added a check for "UID" and this works!
Still... I'm confused. I'm admittedly a bit lost after peeling back layer after layer of the source, but it seems like this portion of code is trying to build up a schema for my table and completely ignoring my generated User class - which quite nicely identifies which column/property is the PK! It doesn't seem quite right that I'd be required to name all keys "ID" w/ ActiveRecord.
I think the answer you're looking for is that this is a really stupid bug on my part. I'm hoping to push another build next week and if you could put this on the issue list I'd really appreciate it. My apologies...
SubSonic expects your primary key to be called Id so it's getting confused. SubSonicPrimaryKey is for simple repository but I assume where that exception is being thrown is shared between the different templates. If you rename your PK to Id or id or ID your query will work.