Hi im using sphinx 201 beta (r2792)
i have rt index with rt_attr_uint = rake my goal is to sort results by this attribute
whitch work as expected using
$this->sphinx->SetSortMode(SPH_SORT_EXTENDED, 'rake DESC')
but i want to also randomize this result ... lets say i have this data
(id, title, rake)
1, 'test', 3
2, 'test2', 3
3, 'ásdf', 1
4, 'rreer', 0
so after i append random
$this->sphinx->SetSortMode(SPH_SORT_EXTENDED, 'rake DESC, #random')
i would expect randomize result sorted by attribute rake descending
but im getting error:
sort-by attribute '#random' not found
whitch i dont understand because this random attribute is internal attribute
am i doing something wrong? where could be a problem? thanks
What I did is create the random number on the SQL query
SELECT ... RAND() as random_sort, ...
Added it as an attribute
sql_attr_float = random_sort
And then use it in the extended sort query
$this->sphinx->SetSortMode(SPH_SORT_EXTENDED, 'rake DESC, random_sort DESC');
From my quick tests, it looks like Sphinx only supports #random if it is the only attribute in the sort clause. So I'm afraid you're out of luck - either you just sort randomly, or you just sort by rake.
For real time randomness, index the ids and sort using expr mode with an expression like this (not sure this one is 100% random):
( ( ( id % #{pagesize} + #{rand(pagesize * 2)} ) % #{pagesize} ) / #{pagesize} )
Add an If statement to the expression above to use your rake attribute.
[http://sphinxsearch.com/docs/1.10/sorting-modes.html][1]
Related
I have the dataset where i need to fetch some articles by companies, but also in same query i need to fetch data which have company = null.
Basically fetch company = 1 or company = null
I tried something like this but filter does not recognize None and just skip.
# example 1
Article.objects.filter(company__in=[1, None])
I tried something like this and this works, but i need something like example above, because i have framework for filtering which takes dict as custom filters, so i need key value pairs.
# example 2
Article.objects.filter(Q(company_id=1) | Q(company=None))
Is there any way i achive something like example 1 or some similar solutions?
EDIT:
If im stuck with example 2 is there any way i can combine those two, i have something like this in my framework.
Article.objects.filter(**custom_filter)
custom filter is dict, for example i have currently
custom_filter = {
'app': 1,
'company': 1
}
From what I can understand you have a custom_filter this does not have an in filter (At least with null values?). And you also have a list of ids you want and perhaps you may or may not want null values too. You can do something like below, to make such queries:
company_list = [1, 2, 3] # Don't put `None` here
nulls_needed = True # Change as per your need
custom_filter = {
'app': 1,
}
Article.objects.filter(
Q(company__isnull=nulls_needed) | Q(company__in=company_list), # positional args first
**custom_filter # keyword args later
)
Use __isnull as shown
Article.objects.filter(Q(company_id=1) | Q(company__isnull=True))
Tools: Peewee 3, SQLite, Python 3
Official documentation for Peewee 3 recursive common table expression (cte):
http://docs.peewee-orm.com/en/latest/peewee/querying.html#common-table-expressions
I am storing a family tree in a simple self-referencing table called Person.
Structure (see below): id, name, parent, custom_order
Notes:
- parent field equals null if this person is an ancestor / root item, otherwise equals to id of parent record if this person is a child
- custom_order is a float number (score to determine who is the user's favourite person)
Objective:
I would like to retrieve the whole family tree and ORDER the results FIRST by parent and SECOND by custom_order.
Issue:
I managed to get the results list but the ORDER is wrong.
DB model
class Person(Model):
name = CharField()
parent = ForeignKeyField('self', backref='children', null = True)
custom_order = FloatField()
Note: if parent field is null then it's a root item
Query code
# Define the base case of our recursive CTE. This will be people that have a null parent foreign-key.
Base = Person.alias()
base_case = (Base
.select(Base)
.where(Base.parent.is_null())
.cte('base', recursive=True))
# Define the recursive terms.
RTerm = Person.alias()
recursive = (RTerm
.select(RTerm)
.join(base_case, on=(RTerm.parent == base_case.c.id)))
# The recursive CTE is created by taking the base case and UNION ALL with the recursive term.
cte = base_case.union_all(recursive)
# We will now query from the CTE to get the people
query = cte.select_from(cte.c.id, cte.c.name, cte.c.parent_id, cte.c.custom_order).order_by(cte.c.parent_id, cte.c.custom_order)
print(query.sql())
Printed query syntax
('WITH RECURSIVE "base" AS
(
SELECT "t1"."id", "t1"."name", "t1"."parent_id", "t1"."custom_order" FROM "person" AS "t1" WHERE ("t1"."parent_id" IS ?)
UNION ALL
SELECT "t2"."id", "t2"."name", "t2"."parent_id", "t2"."custom_order" FROM "person" AS "t2" INNER JOIN "base" ON ("t2"."parent_id" = "base"."id")
)
SELECT "base"."id", "base"."name", "base"."parent_id" FROM "base"
ORDER BY "base"."parent_id", "base"."custom_order"',
[None])
Root of the problem
The code posted in the question works correctly. I verified it by printing the query results to the console:
query = cte.select_from(cte.c.id, cte.c.name, cte.c.parent_id, cte.c.custom_order).order_by(cte.c.parent_id, cte.c.custom_order).dicts()
print(json.dumps(list(query), indent=4))
The problem originated from the fact that I was passing the query results to a nested python Dictionary before printing them to the console, BUT the Python Dictionary is unordered. So, no wonder the printed results were in a different order than the database results.
Solution
Use a Python Ordered Dictionary if you want to store the query results in a fixed order:
import collections
treeDictionary = collections.OrderedDict()
I am trying to sort a list that contain in each index an integer and a string. like the one in the example.
I used sort() and split but I get always the wrong ordered that I expect
def takeSecond(elem):
return elem.split("|")[2]
list = ['|val1: 0|0','|val: 0|80','|val1 0|140','|val1: 0|20','|val1: 0|90']
list.sort(key=takeSecond)
print(list)
that returns:
['|val1: 0|90','|val: 0|80','|val1: 0|20','|val1: 0|0','|val1 0|140']
and I expect to get this:
['|val1: 0|140','|val: 0|90','|val1: 0|80','|val1: 20|0','|val1 0|0']
Where is my mistake in here?
Try this:
l = ['|val1: 0|0','|val: 0|80','|val1 0|140','|val1: 0|20','|val1: 0|90']
l.sort(key=lambda x:int(x.rsplit('|')[-1]), reverse=True)
This will sort your list based on what you need. and the expected output is:
In [18]: l
Out[18]: ['|val1 0|140', '|val1: 0|90', '|val: 0|80', '|val1: 0|20', '|val1: 0|0']
In addition note that:
Do not use list as a variable name. list is a built-in name in python, you will override its functionality .
What is the correct method to have the list (countryList) be available via %s in the SQL statement?
# using psycopg2
countryList=['UK','France']
sql='SELECT * from countries WHERE country IN (%s)'
data=[countryList]
cur.execute(sql,data)
As it is now, it errors out after trying to run "WHERE country in (ARRAY[...])". Is there a way to do this other than through string manipulation?
Thanks
For the IN operator, you want a tuple instead of list, and remove parentheses from the SQL string.
# using psycopg2
data=('UK','France')
sql='SELECT * from countries WHERE country IN %s'
cur.execute(sql,(data,))
During debugging you can check that the SQL is built correctly with
cur.mogrify(sql, (data,))
To expland on the answer a little and to address named parameters, and converting lists to tuples:
countryList = ['UK', 'France']
sql = 'SELECT * from countries WHERE country IN %(countryList)s'
cur.execute(sql, { # You can pass a dict for named parameters rather than a tuple. Makes debugging hella easier.
'countryList': tuple(countryList), # Converts the list to a tuple.
})
You could use a python list directly as below. It acts like the IN operator in SQL and also handles a blank list without throwing any error.
data=['UK','France']
sql='SELECT * from countries WHERE country = ANY (%s)'
cur.execute(sql,(data,))
source:
http://initd.org/psycopg/docs/usage.html#lists-adaptation
Since the psycopg3 question was marked as a duplicate, I'll add the answer to that here too.
In psycopg3, you can not use in %s with a tuple, like you could in psycopg2. Instead you have to use ANY() and wrap your list inside another list:
conn.execute("SELECT * FROM foo WHERE id = ANY(%s)", [[10,20,30]])
Docs: https://www.psycopg.org/psycopg3/docs/basic/from_pg2.html#you-cannot-use-in-s-with-a-tuple
I'm writing a script in python and am accessing an API. I can get some information I need but where I'm stuck is with nested queries. In my code below, first_name needs to equal what would essentially be result[customer->firstname] but I cannot figure out how to get that.
What is the proper syntax to get a nested query like that?
Orders-> customer -> firstname
for result in results['orders']:
order_status_info= self_api.which_api('order_statuses/%d' % result['order_status_id'])
for customer_blocked_reason in customer_blocked_reasons:
if customer_blocked_reason in order_status_info['name']:
customer_is_blocked = True
order_id = 0
order_date = result['ordered_at']
first_name = result [??????]
Is result a dict?
You can access nested dictionaries like so:
>>> d = {"level1": {"level2": "somedata"}}
>>> d["level1"]["level2"]
'somedata'