Cassandra has org.apache.cassandra.cql3.QueryHandler interface which provide apis to handle external queries from client.
Below api which handles prepared statment:
public ResultMessage processPrepared(CQLStatement statement, QueryState state, QueryOptions options) throws RequestExecutionException, RequestValidationException;
I want to log queryString and value passed to it, in case CQLStatement,QueryState and QueryOptions is given . How can i get it?
I Believe a person who has worked on cassandra code can help me out in this.
This would be very difficult in 2.1. With newer versions where for logging they needed this they just recreate it as well as possible. You can see how in the ReadCommand implementations, theres a name() or toCQLString() used in things like slow query logging. You could backport this and the 2 implementations of appendCQLWhereClause for ability to do similar and then build one for modification statement.
in getPrepared() you can get the rawCQLStatement from the ParsedStatement.Prepared and stash it in the thread local.
You may want to alternatively consider using a custom implementation of tracing (example) or using triggers and building a mutation logger.
Do the following:
create a class that would implement the QueryHandler interface and make Cassandra aware of it
in that class you can maintain a list of the queries (add to this list when prepare method is being called) and the current query that you will get from the list when getPrepared it's called; you can get it from the list using the MD5Digest id
when processPrepared is called you can replace the ? in the query string with the values in the QueryOptions options.getValues().
HTH
Related
I'm using spark-solr in order to perform Solr queries. However, my searches don't work as they're supposed to because for some reason the requests being generated by spark prevent the searches from being distributed. I have discovered it by looking at the Solr logs where I saw that a distrib=false parameter is added to the sent requests. When executing the queries manually (not using spark) with distrib=true the results were fine.
I was trying to set the parameters sent by spark by changing the "solr.params" value in the options dictionary (I'm using pyspark):
options = {
"collection": "collection_name",
"zkhost": "server:port",
"solr.params": "distrib=true"
}
spark.read.format("solr").options(**options).load().show()
This change did not have any effect: I still see in the logs that a distrib=false parameter is being sent. Other parameters passed through the "solr.params" key (such as fq=something) do have an effect on the results. But it looks like spark insists on sending distrib=false no matter what I do.
How do I force a distributed search through spark-solr?
The easy solution is to configure the request handler to run distributed queries using an invariant. The invariant forces the distrib parameter to have a true value even if spark-solr is trying to change it in query time. Introducing the invariant can be done by adding the following lines under the definition of your request handler entry in solrconfig.xml:
<lst name="invariants">
<str name="distrib">true</str>
</lst>
While the introduction of the invariant is going to fix the problem, I think it's kind of a radical solution. This is because the solution involves hiding a behavior in which you overload the value of a parameter. By introducing the invariant you cannot decide to set distrib to false: even if your request explicitly does so, the value of distrib would still be true. This is too risky in my opinion and that's why I'm suggesting another solution which might be harder to implement but wouldn't suffer from that flaw.
The solution is to implement a query component which is going to force distrib=true only when receiving a forceDistrib=true flag as a parameter.
public class ForceDistribComponent extends SearchComponent {
private static String FORCE_DISTRIB_PARAM = "forceDistrib";
#Override
public void prepare(ResponseBuilder rb) throws IOException {
ModifiableSolrParams params = new ModifiableSolrParams(rb.req.getParams());
if (!params.getBool(FORCE_DISTRIB_PARAM, false)) return;
params.set(CommonParams.DISTRIB, true);
params.set(FORCE_DISTRIB_PARAM, false);
rb.req.setParams(params);
}
}
After building the component you can configure solr to use it by adding the component to solrconfig.xml and set your request handler to use it.
Adding the component to solrconfig.xml is done by adding the following entry to the solrconfig.xml file:
<searchComponent name="forceDistrib" class="ForceDistribComponent"/>
Configuring the request handler to use the forceDistrib component is done by adding it to the list of components under the request handler entry. It must be the first component in the list:
<arr name="components">
<str>forceDistrib</str>
<str>query</str>
...
</arr>
This solution, while more involved than simply introducing an invariant, is much safer.
Does Automapper works with IQueryable?
I have 2 Query
IQueryable<V_ImageUpload> Query1;
IQueryable<V_ImageUpload> Query2;
Mapper.CreateMap<IQueryable<V_ImageUpload_WithReceiptBackup>, IQueryable<V_ImageUpload>>();
Query1 = Mapper.Map<IQueryable<V_ImageUpload_WithReceiptBackup>, IQueryable<V_ImageUpload>>(Query2);
Exception occured is:
System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary
I've not used AutoMapper (so this was an excuse to try it out!), but there are queryable extensions available. Both your queryables are of the same type, so I'm not exactly sure what you are trying to achieve, but perhaps something like this is what you want, which takes an IQueryable<V_ImageUpload_WithReceiptBackup> and converts it to an IQueryable<V_ImageUpload>:
IQueryable<V_ImageUpload_WithReceiptBackup> query1;
IQueryable<V_ImageUpload> query2;
// Only map the actual type, not the queryable types
Mapper.CreateMap<V_ImageUpload_WithReceiptBackup, V_ImageUpload>();
query2 = query1.Project().To<V_ImageUpload>();
The .Project().To<V_ImageUpload>() keeps it as IQueryable, while Mapper.Map would end up with a List/IEnumerable. I only tested this out with LINQ to Objects, but hopefully it works with Entity Framework, or whatever you are using.
Can someone explain what these attributes HasSideEffects and IsComposable for?
IsComposable
Indicates whether the query method allows query composition. You need to mark a query as IsComposable e.x. when you want to return a class, that contains another class and does not implement IEnumerable interface. When you won't do that, you gonna get an error:
Query operation 'XXX' cannot be marked composable since its return type
'YYY' does not implement IEnumerable of T.
HasSideEffects
Indicates whether the invoke operation has side effects. You want to use it when invokes an operation with a lot of parameteres. HasSideEffectAttributes enforce a POST invoke, that has no limit like GET do.
I need to query a table using FreeTextTable (because I need ranking), with SubSonic. AFAIK, Subsonic doesn't support FullText, so I ended up creating a simple UDF function (Table Function) which takes 2 params (keywords to search and max number of results).
Now, how can I inner join the main table with this FreeTextTable?
InlineQuery is not an option.
Example:
table ARTICLE with fields Id, ArticleName, Author, ArticleStatus.
The search can be done by one of more of the following fields: ArticleName (fulltext), Author (another FullText but with different search keywords), ArticleStatus (an int).
Actually the query is far more complex and has other joins (depending on user choice).
If SubSonic cannot handle this situation, probably the best solution is good old plain sql (so there would be no need to create an UDF, too).
Thanks for your help
ps: will SubSonic 3.0 handle this situation?
3.0 can do this for you but you'd need to make a template for it since we don't handle functions (yet) out of the box. I'll be working on this in the coming weeks - for now I don't think 2.2 will do this for you.
I realize your question is more complex than this, but you can get results from a table valued function via SubSonic 2.2 with a little massaging.
Copy the .cs file from one of your generated views into a safe folder, and then change all the properties to match the columns returned by your UDF.
Then, on your collection, add a constructor method with your parameters and have it execute an InlineQuery.
public partial class UDFSearchCollection
{
public UDFSearchCollection(){}
public UDFSearchCollection(string keyword, int maxResults)
{
UDFSearchCollection coll = new InlineQuery().ExecuteAsCollection<UDFSearchCollection>("select resultID, resultColumn from dbo.udfSearch(#keyword, #maxResults)",keyword,maxResults);
coll.CopyTo(this);
coll = null;
}
}
public partial class UDFSearch : ReadOnlyRecord<UDFSearch>, IReadOnlyRecord
{
//all the methods for read only record go here
...
}
An inner join would be a little more difficult because the table object doesn't have it's own parameters collection. But it could...
It could very well be that I'm just missing the correct vernacular in this space, but I'm looking for a particular piece of functionality in SubSonic. In NetTiers it was called a "DeepLoad". A deep load runs to the database and fetches many objects (ie. fetch this OrderDetail and all of it's LineItems) in one database call.
Again, I want to run to the data store once an build up a potentially dense object graph or related items populated by the data store.
How do I do this in SubSonic and what is it called in SubSonic?
You can do this in SubSonic 3.0 (not yet released, but almost there...) using IQueryable with lazy loading:
var db=new NorthwindDB();
var order=db.Orders.Where(x=>.xID==20).SingleOrDefault();
Assert.Equal(3,order.OrderDetails.Count());
if you're not on 3 (which requires .net 3.5) you can do this with Active record as Paul mentions - but it will make two calls.
There is no eager loading, and DeepSave in ActiveRecord only calls Save.
Here is an example with Northwind Order class foreign key method.
[Test]
public void SelectOrderDetails()
{
Order order = new Order(10250);
OrderDetailCollection details = order.OrderDetails();
Assert.IsTrue(details.Count == 3);
}