python3 lifetime metrics blowing up

[tommyjcarpenter]

Using python3.4 a very simple page insights query explodes:

fb_page_conn.insights.daily(['page_impressions', 'page_fan_adds']).get()


Traceback (most recent call last):
  File "bin/run_fb.py", line 52, in <module>
    function(fb_mysql_conn,  sid_list, customer_type) 
  File "bin/run_fb.py", line 22, in insights_api_fetch_all_helper
    fb_insights_mysql.get_page_info(fb_mysql_conn, stream_id, customer_class)        
  File "/Users/tcarpenter/Development/Eclipse/research-pipeline/sumall_research/integration_wrappers/fb_insights_mysql.py", line 147, in get_page_info
    print(t.get())            
  File "/Users/tcarpenter/Development/Eclipse/research-pipeline/rpvenv/lib/python3.4/site-packages/facebookinsights/graph.py", line 208, in get
    results = self.get_rows()
  File "/Users/tcarpenter/Development/Eclipse/research-pipeline/rpvenv/lib/python3.4/site-packages/facebookinsights/graph.py", line 182, in get_rows
    results = self.get_raw()
  File "/Users/tcarpenter/Development/Eclipse/research-pipeline/rpvenv/lib/python3.4/site-packages/facebookinsights/graph.py", line 166, in get_raw
    metrics, **self.params)
  File "/Users/tcarpenter/Development/Eclipse/research-pipeline/rpvenv/lib/python3.4/site-packages/facebookinsights/utils/api.py", line 58, in all
    url = self._resolve_endpoint(segments + [relative_url], params)
  File "/Users/tcarpenter/Development/Eclipse/research-pipeline/rpvenv/lib/python3.4/site-packages/facebookinsights/utils/api.py", line 32, in _resolve_endpoint
    for key in options.keys():
RuntimeError: dictionary changed size during iteration

I traced it to the line and the code reads:

    def _resolve_endpoint(self, endpoint, options={}):
    endpoint = self._segmentize_endpoint(endpoint)
    url = "/".join(self.base + endpoint)
    # remove facepy options, retain everything 
    # that needs to end up in the querystring
    blacklist = ['path', 'page', 'retry', 'data', 'method', 'relative_url']
    for key in options.keys():
        if key in blacklist:
            del options[key]

Googling the error I found that deleting keys while iterating a dictionary fails:
http://www.quora.com/Working-in-Python-how-can-I-delete-items-while-iterating-over-a-dictionary

After fixing that, I realized there was another error:

   File "bin/run_fb.py", line 52, in <module>
    function(fb_mysql_conn,  sid_list, customer_type) 
      File "bin/run_fb.py", line 22, in insights_api_fetch_all_helper
        fb_insights_mysql.get_page_info(fb_mysql_conn, stream_id, customer_class)        
      File "/Users/tcarpenter/Development/Eclipse/research-pipeline/sumall_research/integration_wrappers/fb_insights_mysql.py", line 147, in get_page_info
        print(t.get())            
      File "/Users/tcarpenter/Development/Eclipse/research-pipeline/rpvenv/lib/python3.4/site-packages/facebookinsights/graph.py", line 208, in get
        results = self.get_rows()
      File "/Users/tcarpenter/Development/Eclipse/research-pipeline/rpvenv/lib/python3.4/site-packages/facebookinsights/graph.py", line 182, in get_rows
        results = self.get_raw()
      File "/Users/tcarpenter/Development/Eclipse/research-pipeline/rpvenv/lib/python3.4/site-packages/facebookinsights/graph.py", line 166, in get_raw
        metrics, **self.params)
      File "/Users/tcarpenter/Development/Eclipse/research-pipeline/rpvenv/lib/python3.4/site-packages/facebookinsights/utils/api.py", line 58, in all
        url = self._resolve_endpoint(segments + [relative_url], params)
      File "/Users/tcarpenter/Development/Eclipse/research-pipeline/rpvenv/lib/python3.4/site-packages/facebookinsights/utils/api.py", line 37, in _resolve_endpoint
        qs = urllib.urlencode(options)
    AttributeError: 'module' object has no attribute 'urlencode'

After again googling, I found that this code is not python3 compatible:
http://stackoverflow.com/questions/28906859/module-has-no-attribute-urlencode

I have fixed both errors and have pushed a PR.

[tommyjcarpenter]

I've made a PR for this while performing the key delete properly.

[debrouwere]

Ah yes, Python 3 generators have bitten me before, good find.