Relationship filtering not working or only working one time

[jdlovins]

Hi there,

Trying to get this working since it would save me a lot of time and redundant calls for IDs. I might be doing something completely wrong but it seems like on every restart it works 1 time before just ignoring the filter.

To give a brief overview of the structure (or https://git.resurfed.xyz/Resurfed/timer-web/tree/zones/maps ):

requirements.txt

astroid==1.6.3
colorama==0.3.9
dj-database-url==0.5.0
Django==2.0.5
django-filter==2.0.0.dev1
djangorestframework==3.8.2
djangorestframework-filters==git+https://github.com/philipn/django-rest-framework-filters.git
gunicorn==19.8.1
isort==4.3.4
lazy-object-proxy==1.3.1
mccabe==0.6.1
pep8==1.7.1
psycopg2==2.7.4
pylint==1.8.4
pytz==2018.4
six==1.11.0
whitenoise==3.3.1
wrapt==1.10.11

settings.py

'DEFAULT_FILTER_BACKENDS': (
        'rest_framework_filters.backends.RestFrameworkFilterBackend',
    ),

models.py

class Map(models.Model):
    """ Map model """
    name = models.CharField(unique=True, max_length=50)

    author = models.ForeignKey(
        Author,
        on_delete=models.SET_NULL,
        blank=True,
        null=True
    )

    last_played = models.DateTimeField(default=now, blank=True)
    added = models.DateTimeField(default=now, blank=True)
    hours_played = models.FloatField()

class Course(models.Model):
    """ Course model """
    map = models.ForeignKey(Map, on_delete=models.CASCADE)

    author = models.ForeignKey(
        Author,
        on_delete=models.SET_NULL,
        blank=True,
        null=True
    )

    difficulty = models.SmallIntegerField(blank=True, default=0)
    checkpoints = models.SmallIntegerField()

    priority = models.ForeignKey(
        Priority,
        on_delete=models.DO_NOTHING,
        blank=True,
        null=True
    )

    behavior = models.ForeignKey(
        Behavior,
        on_delete=models.DO_NOTHING,
        blank=True,
        null=True
    )

class Zone(models.Model):
    """ Zone model """
    course = models.ForeignKey(Course, on_delete=models.CASCADE)
    start = models.CharField(max_length=60)
    end = models.CharField(max_length=60)

views.py

class MapList(generics.ListCreateAPIView):
    queryset = Map.objects.all()
    serializer_class = MapSerializer
    filter_class = MapFilter

class CourseList(generics.ListCreateAPIView):
    queryset = Course.objects.all()
    serializer_class = CourseSerializer
    filter_class = CourseFilter

class ZoneList(generics.ListCreateAPIView):
    queryset = Zone.objects.all()
    serializer_class = ZoneSerializer
    filter_class = ZoneFilter

filters.py

import rest_framework_filters as filters
from .models import Map, Course, Zone


class MapFilter(filters.FilterSet):
    class Meta:
        model = Map
        fields = {'name': ['exact', 'contains']}


class CourseFilter(filters.FilterSet):
    map = filters.RelatedFilter(MapFilter, field_name="map", queryset=Map.objects.all())

    class Meta:
        model = Course
        fields = {'map': ['exact', 'in']}


class ZoneFilter(filters.FilterSet):
    course = filters.RelatedFilter(CourseFilter, field_name="course", queryset=Course.objects.all())

    class Meta:
        model = Zone
        fields = {
            'course': ['exact', 'in']
        }

I want to be able to call
/maps/zones?course__map__name=my_name

to return all of the zones associated with the map. The zones are directly tied to courses which then are then tied to a map.

If i restart the django process it will work one time and only return the appropriate object(s), but if i refresh it will basically just ignore the filter and act like it doesn't exist.

Sorry if i am missing something very obvious!

[rpkilby]

Hi @jdlovins. I've just briefly glanced over the code, and nothing immediately pops out at me as being unusual. At a first pass, I'd try printing out the SQL queries, seeing how they differ. If that doesn't point you in the right direction, I'd try simplifying the problem. You're currently traversing two relationships. I would reduce this to one. eg, Does /maps/courses?map__name=value work?

[jdlovins]

@rpkilby Would you be be able to give me a code snippet on how to print out the SQL statements it's running? I'm not sure how to tie that into DRF and such. Would debug toolbar be of use for that?

It's good to know nothing looks obviously wrong but also annoying that something is wrong haha. I'll try reducing it down to 1 relationship and see what comes up.

[jdlovins]

Okay i got the django debug toolbar working with DRF and can see the SQL queries now.

1. For the sake of making sure i renamed any map instances to mapp so it wouldn't conflict with the python built in. I didn't rename the model, just the foreignkey name and the RelatedFilter object.
2. if i do /maps/?name=surf_test and look at the query it does indeed have a WHERE maps_map.name = surf_test which is as expected.
3. If i go one step out and do it from course with /maps/courses/?mapp__name=surf_test and trying map__name as well it does not add the WHERE clause to the sql query so it doesn't seem to be registering the filter properly.

Would you have any ideas as to why this would happen?

[jdlovins]

Well... I fixed it, due to some stupid luck.

1. I had forgotten to keep 'django_filters', in the INSTALLED_APPS list.
2. Using the django_filters backend isntead of the django-rest-framework-filters backend works. Doesn't make a whole lot of sense to me but I'll take it. Not sure if that is a bug or not.

[rpkilby]

1. I had forgotten to keep 'django_filters', in the INSTALLED_APPS list.

That shouldn't be strictly necessary. django-filter does include templates and localization files, but they shouldn't have prevented the filter from working.

2. Using the django_filters backend isntead of the django-rest-framework-filters backend works. Doesn't make a whole lot of sense to me but I'll take it. Not sure if that is a bug or not.

I don't know why the standard backend would work, but not the drf-filters backend. All it's really doing is hooking into the subsetting behavior.

---

Just as a test, try installing the master branch, and see if that doesn't fix your issue.

[swapnilsm]

@rpkilby I tried installing from master branch. It did not fix the issue. Whereas @jdlovins's fix worked.

[rpkilby]

Hi @jdlovins and @swapnilsm, I just pushed some updates to the master branch that might fix your issue. Could you test the master? There was a bug in the browsable API that would break filtersets after form rendering (i.e., the browsable API), although it's not clear to me if this was the original issue.

[rpkilby]

Hi all, I'm going to close this, assuming that the newly updated master branch fixes the issue. Let me know if there's still a bug, and I'm more than happy to reopen the issue.