Skip to content
Apr 27 / Nizam Sayeed

Dynamic Django queries (or why kwargs is your friend)

, , , ,

A very easy way to dynamically build queries in Django is to use Python kwargs (keyword arguments).

Let’s say we have a model that looks something like this:

class Entry( models.Model ):
    user = models.ForeignKey( User, related_name = 'user_relation' )
    category = models.ForeignKey( Category, related_name = 'category_relation' )
    title = models.CharField( max_length = 64 )
    entry_text = models.TextField()
    deleted_datetime = models.DateTimeField()

Our goal is to dynamically build a query as we go in a view. Using kwargs, we can easily do something like this in our view code:

kwargs = {
    # you can set common filter params here
}

# will return entries which don't have a deleted_datetime
if exclude_deleted:
    kwargs[ 'deleted_datetime__isnull' ] = True

# will return entries in a specific category
if category is not None:
    kwargs[ 'category' ] = category

# will return entries for current user
if current_user_only:
    kwargs[ 'user' ] = request.user

# will return entries where titles match some search query
if title_search_query != '':
    kwargs[ 'title__icontains' ] = title_search_query

# apply all filters and fetch entries that match all criteria
entries = Entry.objects.filter( **kwargs )

Its that simple. This approach seems quite pedestrian when you think about it. However, I didn’t find any examples online which actually shows this in use. It may be useful for someone new to Django.

UPDATE (Apr 30, 2009 @ 9:39AM) Based on the comments I received, I wanted to update this post a little bit. It was mentioned that this approach may not work if you use Q objects for complex lookups. It turns out that QuerySet filter() accepts both args and kwargs. So you could actually do something like:

kwargs = { 'deleted_datetime__isnull': True }
args = ( Q( title__icontains = 'Foo' ) | Q( title__icontains = 'Bar' ) )
entries = Entry.objects.filter( *args, **kwargs )

Very cool indeed. Django never ceases to amaze me.

Filed under Django
  • Daniel Swarbrick

    Django queries are chainable anyway, so you could just do:


    # start with everything, or a subset with Entry.objects.filter(foo) if you like
    entries = Entry.objects.all()


    if exclude_deleted:
    entries = entries.filter(deleted_datetime__isnull=True)


    if category is not None:
    entries = entries.filter(category=category)


    if current_user_only:
    entries = entries.filter(user=request.user)


    if title_search_query != '':
    entries = entries.filter(title__icontains=title_search_query)

  • http://ayaz.wordpress.com/ ayaz

    Neat one. Though, there’s one constraint that I can see with that approach: you could only specify filter parameters that you want AND’d together (though, that’s the default anyway, yeah, and that’s what we usually want most of the time). When I have to build a filter parameter list in steps, I make great use of Q objects and create Q sets that merge into a bigger Q set that is used at the end. Also, I would love to exploit the lazy nature of QuerySet objects more often.

    • http://www.nomadjourney.com Nizam

      Ayaz,

      You’re right. Q objects are really handy especially for cases where OR-ing is required. Interestingly, it turns out that QuerySet’s filter method accepts both args and kwargs. So it is possible to mix and match both Q objects and standard filter parameters together. E.g.,


      kwargs = { 'deleted_datetime__isnull': True }
      args = ( Q( title__icontains = 'Foo' ) | Q( title__icontains = 'Bar' ) )
      entries = Entry.objects.filter( *args, **kwargs )

      I am going to update this post to reflect this fact.

  • http://meiocodigo.com Vinicius Mendes

    Very good approach. I use this in my search engine. Very useful. Just to complete what you said, you can use the same approach with the Q objects. For example:


    kwargs_author = {'author__icontains': 'Vinicius'}
    kwargs_title = {'title__icontains': 'Vinicius'}
    Entry.objects.filter(Q(**kwargs_author) | Q(**kwargs_title))

    In my case, I have a form field for each kind of search (integer, date, string, boolean, primary key, choice field). I have a search form also. So, I populate my form with the GET data and it uses a field’s method called get_criteria wich returns a dictionary with in this format (‘lookup_method’: ‘value’). I pass the search string from the GET data to the get_criteria of each field and then get the dictionary returned, and pass to the Q object.

    • http://unhub.com/vacanti Vinicius Vacanti

      @Vinicius

      Your comment was very helpful. I was looking for a way to dynamically create Q objects and you provided the answer!

      Also, great name!

    • http://philgo20.com PhilGo20

      Thanks for the example, was also trying to use dynamic Q objects. great stuff

    • airstrike

      Any chance you guys could share some of that code? I’m going to start working on something rather similar right now and I’d love to have something of a starting point instead of diving in empty handed. Thanks in advance!

  • Brooks Travis

    What about constructing dynamic Q objects and ‘OR’ing them together dynamically? The only thing I’ve been able to come up with is creating the statement as a string:

    ‘Q(field=blah,field2=blahblah)|Q(field3=blah,field4=blahblah)’

    But I can’t seem to find a way to plug that in to the filter(). I’ve tried eval(), exec(), and compile(), but no luck. Ideas?

    • http://www.nomadjourney.com Nizam

      Brooks,

      You should be able to just create your Q objects and then OR them together in a list before you pass them to .filter() as follows:


      args = []
      q1 = Q( field = blah, field2 = blahblah )
      q2 = Q( field3 = blah, field4 = blahblah )
      args.append( q1 | q2 )
      MyModel.objects.filter( *args )

    • Renato Pedigoni

      “|” is the bitwise or operator, its actual function is operator.or_(a, b) [1]

      You may want something like this:

      import operator
      field_lookups = (‘title__icontains’, ‘author__icontains’,)
      q_list = [Q(**{f:q}) for f in field_lookups]
      Model.objects.filter(reduce(operator.or_, q_list))

      [1] http://docs.python.org/library/operator.html#operator.or_

    • http://www.nomadjourney.com Nizam

      I am doing it the exact way they describe it in the Django docs:
      http://www.djangoproject.com/documentation/models/or_lookups/

  • Vladimir Dronnikov

    Very helpful, thanks!

    One remark:

    args = ( Q( title__icontains = ‘Foo’ ) | Q( title__icontains = ‘Bar’ ) )
    in the example should read
    args = ( Q( title__icontains = ‘Foo’ ) | Q( title__icontains = ‘Bar’ ) , )
    or *args fails to unpack. Notice trailing comma.

    Regards,
    –
    Vladimir

  • http://www.michelepasin.org/techblog/ mike

    thanks for the post, it helped me clarify a few things!

  • http://www.ideatech.org Mansoorulhaq

    vary helpful information. I was looking for. thanks