missing dataset_id parameter in views.api
Closed this issue · 6 comments
We still get the occasional 500 error when fetching datasets. This is caused by a missing dataset_id parameter in views.api, and might be related to how we're handling urls in urls.py.
Representative stack trace:
Internal Server Error: /api/places/1041
Traceback (most recent call last):
File "/Users/trevorcroxson/duwamish/env/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/trevorcroxson/duwamish/src/sa_web/views.py", line 368, in api
root = settings.SHAREABOUTS.get(dataset_id.upper() + '_SITE_URL')
UnboundLocalError: local variable 'dataset_id' referenced before assignment
[24/Jun/2016 00:36:03] "GET /api/places/1041 HTTP/1.1" 500 5925
This bug seems to affect newly-created places. I'm still not sure what triggers it, but sometimes when a new place is created it won't load from the API, even though it appears in the database. What's odd, though, is that I am still able to see the place's detail view if I directly access its url. If I reload the page, however, the place then sometimes won't appear on the map, and 500 errors will be generated.
UPDATE-- after some experimentation, I think I'm starting to see a pattern in this bug. The bug affects newly-created places, as well as places that were recently commented on. It seems that if a place was recently created or modified, there is a short period time (as long as a few minutes, maybe?) when the place and its data are present in the database but are not reliably being retrieved when we fetch the place's collection in the browser. Sometimes the entire place won't be displayed on the map, and other times the place might be displayed but recent comments won't appear on the detail view (even though they're in the database).
In general, though, things seem to stabilize over time. After a few minutes of no new activity, the place and its submission sets are reliably fetched and displayed.
@Lukeswart -- do you happen to know if the shareabouts places are cached in some way, maybe by django? I've noticed puzzling behavior: after I delete places from the dev-api's database, they can persist intermittently on the map. On some map loads, a whole set of recently deleted places will appear--data intact--as though they were never deleted. On subsequent map loads, they'll be gone, only to reappear on the next map load. At first I thought it might be some kind of front-end caching, but the data that comes back from the backbone collections' fetchAllPages() methods will intermittently contain "deleted" places (that is, places which clearly are not in the database as accessed from the dev API's admin panel), and the map will render whatever it gets back.
It's kind of a shot in the dark, but this behavior feels like it could be related to the bug in this issue. There seems to be some other process (some kind of caching? a syncing issue?) between the database and what's returned by API calls. Maybe something is going on which produces this behavior and causes recently-created or edited places to behave erratically.
I dunno...does any of that sound like it could be part of the problem?
Also, this issue apaprently doesn't affect the master branch, so whatever is happening is somehow related to fetching multiple collections from multiple urls.
Thanks for the detailed description! There is definitely some caching on our API, but I'm not sure if it is causing the problems that you described. I'll look into this and respond back asap...
@Lukeswart -- thanks again for your help tracking down the issue behind this bug the other night.
I pushed up a commit that isolates usage of the mergedActivities collection to activity-view.js, and I tweaked our ActivityView to work with this change. The activities appear to be loading correctly, except when we run the auto-update (i.e. the checkForNewActivity()) method. I believe it should check every 30 seconds for new activities, and render anything new in the activity panel, right? So far this isn't happening, and I think there might be an issue with how I've adapted checkForNewActivity(). The good news, though, is that the 500 errors have stopped! I think the culprit, as you guessed, was definitely calling fetchByIds on the mergedPlaces collection, which used the old fallback /api/places url and wreaked havoc in views.api.
I'm also trying to isolate usage of the mergedPlaces to place-list-view. If I'm understanding Marionette correctly, we're using the onAfterItemAdded callback to cache views, but Marionette expects a single collection to be passed in. Now, we're passing our object of place collections. Before this wasn't an issue since we passed PlaceListView our merged place collection (which was built outside the view), but now since we're trying to isolate the merged collection within the view we might need another approach.