Dynamic Django queries (or why kwargs is your friend)
A very easy way to dynamically build queries in Django is to use Python kwargs (keyword arguments).
Let’s say we have a model that looks something like this:
class Entry( models.Model ):
user = models.ForeignKey( User, related_name = 'user_relation' )
category = models.ForeignKey( Category, related_name = 'category_relation' )
title = models.CharField( max_length = 64 )
entry_text = models.TextField()
deleted_datetime = models.DateTimeField()
Our goal is to dynamically build a query as we go in a view. Using kwargs, we can easily do something like this in our view code:
kwargs = {
# you can set common filter params here
}
# will return entries which don't have a deleted_datetime
if exclude_deleted:
kwargs[ 'deleted_datetime__isnull' ] = True
# will return entries in a specific category
if category is not None:
kwargs[ 'category' ] = category
# will return entries for current user
if current_user_only:
kwargs[ 'user' ] = request.user
# will return entries where titles match some search query
if title_search_query != '':
kwargs[ 'title__icontains' ] = title_search_query
# apply all filters and fetch entries that match all criteria
entries = Entry.objects.filter( **kwargs )
Its that simple. This approach seems quite pedestrian when you think about it. However, I didn’t find any examples online which actually shows this in use. It may be useful for someone new to Django.
UPDATE (Apr 30, 2009 @ 9:39AM) Based on the comments I received, I wanted to update this post a little bit. It was mentioned that this approach may not work if you use Q objects for complex lookups. It turns out that QuerySet filter() accepts both args and kwargs. So you could actually do something like:
kwargs = { 'deleted_datetime__isnull': True }
args = ( Q( title__icontains = 'Foo' ) | Q( title__icontains = 'Bar' ) )
entries = Entry.objects.filter( *args, **kwargs )
Very cool indeed. Django never ceases to amaze me.
Django queries are chainable anyway, so you could just do:
# start with everything, or a subset with Entry.objects.filter(foo) if you like
entries = Entry.objects.all()
if exclude_deleted:
entries = entries.filter(deleted_datetime__isnull=True)
if category is not None:
entries = entries.filter(category=category)
if current_user_only:
entries = entries.filter(user=request.user)
if title_search_query != '':
entries = entries.filter(title__icontains=title_search_query)
Its true that QuerySets are lazy but then again, so am I. ;)
I like the kwargs approach a little more since it involves less keystrokes, is slightly more readable (a matter of personal taste) and the end result is essentially the same.
Must reads for anyone wondering how QuerySets work:
http://docs.djangoproject.com/en/dev/topics/db/queries/#id3
http://docs.djangoproject.com/en/dev/ref/models/querysets/#when-querysets-are-evaluated
Neat one. Though, there’s one constraint that I can see with that approach: you could only specify filter parameters that you want AND’d together (though, that’s the default anyway, yeah, and that’s what we usually want most of the time). When I have to build a filter parameter list in steps, I make great use of Q objects and create Q sets that merge into a bigger Q set that is used at the end. Also, I would love to exploit the lazy nature of QuerySet objects more often.
Ayaz,
You’re right. Q objects are really handy especially for cases where OR-ing is required. Interestingly, it turns out that QuerySet’s filter method accepts both args and kwargs. So it is possible to mix and match both Q objects and standard filter parameters together. E.g.,
kwargs = { 'deleted_datetime__isnull': True }
args = ( Q( title__icontains = 'Foo' ) | Q( title__icontains = 'Bar' ) )
entries = Entry.objects.filter( *args, **kwargs )
I am going to update this post to reflect this fact.
Very good approach. I use this in my search engine. Very useful. Just to complete what you said, you can use the same approach with the Q objects. For example:
kwargs_author = {'author__icontains': 'Vinicius'}
kwargs_title = {'title__icontains': 'Vinicius'}
Entry.objects.filter(Q(**kwargs_author) | Q(**kwargs_title))
In my case, I have a form field for each kind of search (integer, date, string, boolean, primary key, choice field). I have a search form also. So, I populate my form with the GET data and it uses a field’s method called get_criteria wich returns a dictionary with in this format (’lookup_method’: ‘value’). I pass the search string from the GET data to the get_criteria of each field and then get the dictionary returned, and pass to the Q object.
@Vinicius
Your comment was very helpful. I was looking for a way to dynamically create Q objects and you provided the answer!
Also, great name!
What about constructing dynamic Q objects and ‘OR’ing them together dynamically? The only thing I’ve been able to come up with is creating the statement as a string:
‘Q(field=blah,field2=blahblah)|Q(field3=blah,field4=blahblah)’
But I can’t seem to find a way to plug that in to the filter(). I’ve tried eval(), exec(), and compile(), but no luck. Ideas?
Brooks,
You should be able to just create your Q objects and then OR them together in a list before you pass them to .filter() as follows:
args = []
q1 = Q( field = blah, field2 = blahblah )
q2 = Q( field3 = blah, field4 = blahblah )
args.append( q1 | q2 )
MyModel.objects.filter( *args )
Very helpful, thanks!
One remark:
args = ( Q( title__icontains = ‘Foo’ ) | Q( title__icontains = ‘Bar’ ) )
in the example should read
args = ( Q( title__icontains = ‘Foo’ ) | Q( title__icontains = ‘Bar’ ) , )
or *args fails to unpack. Notice trailing comma.
Regards,
–
Vladimir