In a specific case of deleting and reinserting rows to a table does not work as intended when replication factor is more than 1 (say 3).
I have tried using quorum for read as well as write and timestamp in the query with no success.
If the replication factor is set to 1, all works well. I have 5 nodes in my cluster.
Issue becomes visible when tried to read after the delete and insert operation. The read does not return rows that should have been inserted.
Could someone please share their thoughts on this?
Edit:
Following is what I have tried from code
Database create script:
DROP KEYSPACE IF EXISTS consistency_check;
CREATE KEYSPACE IF NOT EXISTS consistency_check WITH replication = {'class': 'NetworkTopologyStrategy', 'Cassandra': 3} AND durable_writes = true;
USE consistency_check;
CREATE TABLE IF NOT EXISTS resource (
id uuid,
col1 text,
col2 text,
col3 text,
col4 text,
PRIMARY KEY(id, col1)
);
C# Unit test code:
public class Resource
{
public Guid Id { get; set; }
public string Col1 { get; set; }
public string Col2 { get; set; }
public string Col3 { get; set; }
public string Col4 { get; set; }
}
public class MyMappings : Mappings
{
public MyMappings()
{
For<Resource>()
.TableName("resource")
.PartitionKey(u => u.Id)
.Column(u => u.Id, cm => cm.WithName("id"))
.Column(u => u.Col1, cm => cm.WithName("col1"))
.Column(u => u.Col2, cm => cm.WithName("col2"))
.Column(u => u.Col3, cm => cm.WithName("col3"))
.Column(u => u.Col4, cm => cm.WithName("col4"));
}
}
//Following test fails always
[TestMethod]
public async Task DeleteInsert()
{
var table = new Table<Resource>(_session);
_session.Execute("truncate resource");
for (int i = 0; i < 10; i++)
{
var id = Guid.NewGuid();
await table.Insert(new Resource { Id = id, Col1 = id.ToString(), Col2 = id.ToString(), Col3 = id.ToString(), Col4 = id.ToString() }).ExecuteAsync().ConfigureAwait(false);
}
var data = (await table.ExecuteAsync().ConfigureAwait(false)).ToList();
foreach (var datum in data)
{
await table.Where(e => e.Id == datum.Id).Delete().ExecuteAsync().ConfigureAwait(false);
}
foreach (var datum in data)
{
await table.Insert(datum).ExecuteAsync().ConfigureAwait(false);
}
var data1 = (await table.ExecuteAsync().ConfigureAwait(false)).ToList();
Console.WriteLine("data length: {0}", data.Count);
Console.WriteLine("data1 length: {0}", data1.Count);
Assert.IsTrue(data.Count == 10);
Assert.IsTrue(data1.Count == 10);
}
Where are you setting the consistency level? If you use quorum for reads and writes, you shouldn't get this problem.
Looks like your running the inserts and deletes asynchronously - so the delete could have run before the inserts. In the case the delete clause will not be met and not rows will be deleted.
I am not super familiar with the C# async/await construct so i might be missing something
Related
Given a simple poco
public class Model
{
[PrimaryKey]
public int ID { get; set; }
public string Description { get; set; }
}
this works fine ...
var connectionString = #"Data Source=WIN8PC\SQLEXPRESS;Initial Catalog=Test;Integrated Security=True;";
connectionFactory = new OrmLiteConnectionFactory(connectionString, SqlServerDialect.Provider);
using (var db = connectionFactory.OpenDbConnection())
{ db.DropAndCreateTable<Model>(); }
var model0 = new Model { ID = 0, Description = "Item Zero" };
var model1 = new Model { ID = 1, Description = "Item One" };
using (var db = connectionFactory.OpenDbConnection())
{ db.Save(model0, model1); }
as does this ...
model0.Description += " updated";
model1.Description += " updated";
using (var db = connectionFactory.OpenDbConnection())
{
db.Save(model0);
db.Save(model1);
}
however, this crashes with a primary key violation exception ...
model0.Description += " updated again";
model1.Description += " updated again";
using (var db = connectionFactory.OpenDbConnection())
{ db.Save(model0, model1); }
The record with ID zero is required, as this is a lookup table to replace an existing C# enum type. This is a local copy of distributed data (that I don't control), so there's no reason to have an auto-increment key.
The issue appears to be in OrmLiteWriteCommandExtensions.SaveAll() - any row with id == defaultValue is assumed to be a new item, rather than an update of an existing record. The same issue occurs in the parallel async methods too.
Is there any other way to get around this issue, other than by saving each record individually (inside a transaction). It would be preferable to save all updated records for a table in one command.
Save is a high-level API that will INSERT or UPDATE based on whether or not the Primary Key has a value. If you want to insert a default Primary Key value you can use Insert instead as seen in this Live Example on Gistlyn:
public class Model
{
[PrimaryKey]
public int ID { get; set; }
public string Description { get; set; }
}
db.DropAndCreateTable<Model>();
var model0 = new Model { ID = 0, Description = "Item Zero" };
var model1 = new Model { ID = 1, Description = "Item One" };
db.Insert(model0, model1);
var rows = db.Select<Model>();
"Inserted Rows: {0}".Print(rows.Dump());
Which outputs:
Inserted Rows: [
{
ID: 0,
Description: Item Zero
},
{
ID: 1,
Description: Item One
}
]
I'm trying to apply sorting in my MVC 5 application. I used the code here
to apply sorting. Unfortunately, its not working and it won't sort. Am I missing something? All datatypes I used are strings btw.
Here is the code:
//Controller
public ActionResult Index(string sort)
{
ViewBag.ExtSortParm = String.IsNullOrEmpty(sort) ? "ext_desc" : "";
ViewBag.DtsSortParm = sort == "DTS" ? "dts_desc" : "DTS";
var sales = from s in db.Sales1 select s;
switch (sort)
{
case "ext_desc":
sales = sales.OrderByDescending(s => s.ExtSerial);
break;
case "DTS":
sales = sales.OrderBy(s => s.DTS);
break;
case "dts_desc":
sales = sales.OrderByDescending(s => s.DTS);
break;
default:
sales = sales.OrderBy(s => s.ExtSerial);
break;
}
return View(db.Sales1.ToList());
}
And my View applied:
#Html.ActionLink("ExtSerial", "Index", new { sortOrder = ViewBag.ExtSortParm })
#Html.ActionLink("DTS", "Index", new { sortOrder = ViewBag.DtsSortParm })
Was there something I missed? DTS is a date by the way just in string type. Here is an example value: 5/11/2015 5:29:56 AM
db is my database & Sales1 is my SalesEntity
You are sorting your collection but he following line returns the original unsorted collection
return View(db.Sales1.ToList());
You need to return the sorted collection to the view using
return View(sales.ToList());
The following query works perfectly fine and populates its dropdown list. The data in the data base is stored in all uppercase, ie PALM BEACH. I want to convert it to Proper case, which obviously i can do after the fact by iterating through the returned list and reformatting BUT I should be able to do it with in the query itself. The following query works fine.
Dim citylist As List(Of String) = (From c In ctx.ziptaxes
Where c.StateID = ddlStates.SelectedIndex
Order By c.City Ascending
Select c.City).ToList()
But if i try to convert it to some thing like this, it fails
Dim citylist As List(Of String) = (From c In ctx.ziptaxes
Where c.StateID = ddlStates.SelectedIndex
Let cityname = StrConv(c.City, VbStrConv.ProperCase)
Order By cityname Ascending
Select cityname).ToList()
I've tried using culture info and String.Format(c.City, vbProperCase) too and nothing other than the original query works. Any help appreciated.
ADDENDUM:
Well some further research is telling me that .Net objects like string conversion and cultureinfo cannot be used prior to the query being run. If that's the case it explains why it isn't working. The following solves my problem BUT I would still like to know if there is way to do it within the LINQ to EF.
Dim citylist As List(Of String) = (From c In ctx.ziptaxes
Where c.StateID = ddlStates.SelectedIndex
Order By c.City Ascending
Select c.City).ToList()
If citylist.Count > 0 Then
For i As Integer = 0 To citylist.Count - 1
citylist(i) = StrConv(citylist(i).ToLower(), vbProperCase)
Next
With ddlCity
.Items.Clear()
.DataSource = citylist.Distinct()
.DataBind()
.Items.Insert(0, "Select a city")
.SelectedIndex = 0
End With
End If
You can do the conversion in your SELECT. Here's an example (with an over-simplified City name converter):
using System.Collections.Generic;
using System.Linq;
using NUnit.Framework;
namespace LinqQuestion
{
[TestFixture]
public class StackOverflowTests
{
private IEnumerable<City> _cities;
[TestFixtureSetUp]
public void Arrange()
{
_cities = new List<City>
{
new City { Id = 1, Name = "FLINT", StateId = 1 },
new City { Id = 2, Name = "SAGINAW", StateId = 1 },
new City { Id = 3, Name = "DETROIT", StateId = 1 },
new City { Id = 4, Name = "FLint", StateId = 1 }
};
}
[Test]
public void TestCountryQuery()
{
var data = _cities
.Where(c => c.StateId == 1)
.OrderBy(c => c.Name)
.Select(c => StrConv(c.Name))
.Distinct().ToList();
Assert.That(data.Count == 3);
}
private static string StrConv(string original)
{
var firstLetter = original.Substring(0, 1).ToUpper();
var theRest = original.Substring(1, original.Length - 1).ToLower();
return firstLetter + theRest;
}
}
public class City
{
public int Id { get; set; }
public int StateId { get; set; }
public string Name { get; set; }
}
}
I am very naive in cassandra & am using astyanax
CREATE TABLE employees (empID int, deptID int, first_name varchar,
last_name varchar, PRIMARY KEY (empID, deptID));
i want to get the values of query:
select * from employees where empID =2 and deptID = 800;
public void read(Integer empID, String deptID) {
OperationResult<ColumnList<String>> result;
try {
columnFamilies = ColumnFamily.newColumnFamily("employees", IntegerSerializer.get(), StringSerializer.get());
result = keyspace.prepareQuery(columnFamilies).getKey(empID).execute();
ColumnList<String> cols = result.getResult();
//Other stuff
}
how should i achieve this
As far as I can find, there isn't a super clean way to do this. You have to do it by executing a cql query and then iterating through the rows. This code is taken from the astynax examples file
public void read(int empId) {
logger.debug("read()");
try {
OperationResult<CqlResult<Integer, String>> result
= keyspace.prepareQuery(EMP_CF)
.withCql(String.format("SELECT * FROM %s WHERE %s=%d;", EMP_CF_NAME, COL_NAME_EMPID, empId))
.execute();
for (Row<Integer, String> row : result.getResult().getRows()) {
logger.debug("row: "+row.getKey()+","+row); // why is rowKey null?
ColumnList<String> cols = row.getColumns();
logger.debug("emp");
logger.debug("- emp id: "+cols.getIntegerValue(COL_NAME_EMPID, null));
logger.debug("- dept: "+cols.getIntegerValue(COL_NAME_DEPTID, null));
logger.debug("- firstName: "+cols.getStringValue(COL_NAME_FIRST_NAME, null));
logger.debug("- lastName: "+cols.getStringValue(COL_NAME_LAST_NAME, null));
}
} catch (ConnectionException e) {
logger.error("failed to read from C*", e);
throw new RuntimeException("failed to read from C*", e);
}
}
You just have to tune the cql query to return what you want. This is a bit frustrating because according to the documentation, you can do
Column<String> result = keyspace.prepareQuery(CF_COUNTER1)
.getKey(rowKey)
.getColumn("Column1")
.execute().getResult();
Long counterValue = result.getLongValue();
However I don't know what rowkey is. I've posted a question about what rowkey can be. Hopefully that will help
I have 11 records in the Menu table, only 1 with a PageID set however if I use
var test = Menu.All().Where(
x => x.WebPages.Any(
pages => pages.Roles.Contains(Roles.GetRolesForUser()[0])
I get 11 records as the SQL run is this
SELECT [t0].[CategoryID], [t0].[CreatedBy], [t0].[CreatedOn],
[t0].[ID], [t0].[ImageID], [t0].[ImageIDHover], [t0].[Locale],
[t0].[ModifiedBy], [t0].[ModifiedOn], [t0].[OrderID], [t0].[PageID],
[t0].[ParentID], [t0].[Title], [t0].[URL], [t0].[Visible]
FROM [dbo].[Menu] AS t0
WHERE EXISTS(
SELECT NULL
FROM [dbo].[WebPage] AS t1
WHERE ([t1].[Roles] LIKE '%' + 'User' + '%')
)
If I run this I get the 1 record
var test = Menu.All().Where(
x => x.WebPages.Any(
pages => pages.Roles.Contains(
Roles.GetRolesForUser()[0]) && pages.ID == x.PageID));
The SQL for this is
SELECT [t0].[CategoryID], [t0].[CreatedBy], [t0].[CreatedOn],
[t0].[ID], [t0].[ImageID], [t0].[ImageIDHover], [t0].[Locale],
[t0].[ModifiedBy], [t0].[ModifiedOn], [t0].[OrderID], [t0].[PageID],
[t0].[ParentID], [t0].[Title], [t0].[URL], [t0].[Visible]
FROM [dbo].[Menu] AS t0
WHERE EXISTS (
SELECT NULL
FROM [dbo].[WebPage] AS t1
WHERE (([t1].[Roles] LIKE '%' + 'User' + '%') AND
([t1].[ID] = [t0].[PageID]))
)
The problem with Any() is that in the SQL as long as one record exits, doesn't matter which record it will return data.
I think effectively I am wanting an UNION SQL like below but I don't know how I re-engineer that into C#/Subsonic
select m.* from menu m where pageid is null
union
select m.* from menu m
join webpage p
on p.id = m.pageid
where p.roles like '%User%'
I want to return all menu records and for those with a PageID set that the corresponding WebPage has the user's role in it. If the user's role is not in the WebPage then I don't want to see it in my results.
Here are my cutodwn classes that Subsonic generates
public partial class Menu: IActiveRecord
{
public int ID {get; set;}
public int CategoryID {get;set;}
public bool Visible {get;set;}
public int PageID {get;set;}
public IQueryable<WebPage> WebPages
{
get
{
var repo=NorthCadburyWebsite.Models.WebPage.GetRepo();
return from items in repo.GetAll()
where items.ID == _PageID
select items;
}
}
}
public partial class WebPage: IActiveRecord
{
public int ID {get;set;}
public string Roles {get;set;}
}
It sure seems like you could just add an alternative test to the where clause to do this.
var test = Menu.All()
.Where(x => x.PageID == null
|| x.WebPages
.Any(pages => pages.Roles.Contains(Roles.GetRolesForUser()[0])
&& pages.ID == x.PageID));
EDIT:
Try using the the Intersect method to see if there is an overlap between the roles in the DB and the roles for the user.
var test = Menu.All()
.Where(x => x.PageID == null
|| x.WebPages
.Any(pages => pages.Roles.Intersect(Roles.GetRolesForUser().Split(','))
&& pages.ID == x.PageID));