sybrenstuvel/flickrapi

AssertionError: verifier must be unicode text type

grahamwhiteuk opened this issue · 11 comments

grahamwhiteuk commented

The example login code at https://stuvel.eu/flickrapi-doc/3-auth.html#authenticating-without-local-web-server does not work

When attempting to use this example the code throws an exception:

Traceback (most recent call last):
  File "test.py", line 50, in <module>
    flickr.get_access_token(verifier)
  File "/usr/lib/python2.7/site-packages/flickrapi/core.py", line 656, in get_access_token
    self.flickr_oauth.verifier = verifier
  File "/usr/lib/python2.7/site-packages/flickrapi/auth.py", line 209, in verifier
    assert isinstance(new_verifier, six.text_type), 'verifier must be unicode text type'
AssertionError: verifier must be unicode text type

I've tried with the latest 2.4 release as well as an older 2.2.1 release.

grahamwhiteuk commented

I'm using Python 2.7 (because I'm using a legacy exif library) and just discovered the above works OK with Python 3 which may well have been what you intended. May still be worth marking the code sample as requiring Python 3 though.

sybrenstuvel commented

Maybe it's time to assume it's Py3 unless something states it's Py2?

oPromessa commented

I have login code which works on both Python 2.7 and Python 3.6.
I'll clean it up a bit in the coming week or so to possibly replace the sample you mention, if you guys want.

@sybrenstuvel When you say "Maybe it's time to assume it's Py3 unless something states it's Py2?" you simply mean, documentation-wise, correct? No actually dropping support to Python 2.7, correct?

Thing is that I still need Python 2.7 to have things running with flickrapi on systems like Synology DSM which natively (still) comes with Python 2.7.

Thanks, oPromessa

grahamwhiteuk commented

I'm running Fedora 27 which defaults to Python 2.7 and is considered a fairly bleeding edge Linux distro. I believe it's not scheduled until Fedora 32 for a full switch to Python 3 as the default which will be another 2 years time. So for the more casual Python coder I don't think it's unreasonable to expect Python 2 support to some extent. For example, you could choose to just catch the exception and warn the user they're running Python 2 rather than update the code samples.

oPromessa commented

Something along these lines... maybe too complex?

#!/usr/bin/env python
"""
    by oPromessa, 2018
"""
import sys
import os
import flickrapi

# -------------------------------------------------------------------------
# authenticate
#
# Authenticates via flickrapi on flickr.com
#
def authenticate():
    """
    Authenticate user so we can upload files.
    Assumes the cached token is not available or valid.
    """
    global nuflickr
    global xCfg

    # Instantiate nuflickr for connection to flickr via flickrapi
    nuflickr = flickrapi.FlickrAPI(xCfg["api_key"],
                                   xCfg["api_secret"],
                                   token_cache_location=xCfg["TOKEN_CACHE"])
    # Get request token
    print('Getting new token.')
    try:
        nuflickr.get_request_token(oauth_callback='oob')
    except flickrapi.exceptions.FlickrError as ex:
        sys.stderr.write('+++010 Flickrapi exception on get_request_token. '
                         'Error code/msg: [{!s}]/[{!s}]\n'
                         .format(ex.code, ex))
        sys.stderr.flush()
        sys.exit(4)
    except Exception as ex:
        sys.stderr.write('+++020 Exception on get_request_token: [{!s}]\n'
                         'Exiting...\n'
                         .format(str(sys.exc_info())))
        sys.stderr.flush()
        sys.exit(4)

    # Show url. Copy and paste it in your browser
    # Adjust parameter "perms" to to your needs
    authorize_url = nuflickr.auth_url(perms=u'delete')
    print('Copy and paste following authorizaiton URL '
          'in your browser to obtain Verifier Code.')
    print(authorize_url)

    # Prompt for verifier code from the user.
    # Python 2.7 and 3.6
    # use "# noqa" to bypass flake8 error notifications
    verifier = unicode(raw_input(  # noqa
                                 'Verifier code (NNN-NNN-NNN): ')) \
               if sys.version_info < (3, ) \
               else input('Verifier code (NNN-NNN-NNN): ')

    print('Verifier: {!s}'.format(verifier))

    # Trade the request token for an access token
    try:
        nuflickr.get_access_token(verifier)
    except flickrapi.exceptions.FlickrError as ex:
        sys.stderr.write('+++030 Flickrapi exception on get_access_token. '
                         'Error code/msg: [{!s}]/[{!s}]\n'
                         .format(ex.code, ex))
        sys.stderr.flush()
        sys.exit(5)

    print('{!s} with {!s} permissions: {!s}'.format(
          'Check Authentication',
          'delete',
          nuflickr.token_valid(perms='delete')))

    # Some debug...
    sys.stderr.write('Token Cache: [{!s}]\n', nuflickr.token_cache.token)
    sys.stderr.flush()


# -------------------------------------------------------------------------
# getCachedToken
#
# If available, obtains the flickrapi Cached Token from local file.
# returns the token
# Saves the token on the global variable nuflickr.
#
def getCachedToken():
    """
    Attempts to get the flickr token from disk.
    """
    global nuflickr
    global xCfg

    sys.stderr.write('Obtaining Cached token\n')
    sys.stderr.write('TOKEN_CACHE:[{!s}]\n'.format(xCfg["TOKEN_CACHE"]))
    nuflickr = flickrapi.FlickrAPI(xCfg["api_key"],
                                   xCfg["api_secret"],
                                   token_cache_location=xCfg["TOKEN_CACHE"])

    try:
        # Check if token permissions are correct.
        if nuflickr.token_valid(perms='delete'):
            sys.stderr.write('Cached token obtained: [{!s}]\n'
                             .format(nuflickr.token_cache.token))
            return nuflickr.token_cache.token
        else:
            sys.stderr.write('Token Non-Existant.\n')
            return None
    except BaseException:
        sys.stderr.write('+++040 Unexpected error in token_valid. [{!s}]\n'
                         .format(str(sys.exc_info())))
        sys.stderr.flush()
        raise

# -------------------------------------------------------------------------
# Global Variables + Main code
#
xCfg = { 'api_key' : u'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
         'api_secret' : u'YYYYYYYYYYYYYYYY',
         'TOKEN_CACHE' : os.path.join(os.path.dirname(sys.argv[0]), "token")}
nuflickr = None

token = getCachedToken()

if (token is None):
    authenticate()

print('Do something with Flickr...')

# Total FLickr photos count: find('photos').attrib['total'] -----------
try:
    res = nuflickr.people.getPhotos(user_id="me", per_page=1)
except flickrapi.exceptions.FlickrError as ex:
    sys.stderr.write('+++050 Flickrapi exception on getPhotos. '
                     'Error code/msg: [{!s}]/[{!s}]\n'
                     .format(ex.code, ex))
finally:
    countflickr = -1
    if (res is not None) and (not res == "" and res.attrib['stat'] == "ok"):
        countflickr = format(res.find('photos').attrib['total'])


# Total photos not on Sets/Albums on FLickr ---------------------------
# (per_page=1 as only the header is required to obtain total):
#       find('photos').attrib['total']
try:
    res = nuflickr.photos.getNotInSet(per_page=1)
except flickrapi.exceptions.FlickrError as ex:
    sys.stderr.write('+++060 Flickrapi exception on getNotInSet. '
                     'Error code/msg: [{!s}]/[{!s}]\n'
                     .format(ex.code, ex))
finally:
    countnotinsets = 0
    if (res is not None) and (not res == "" and res.attrib['stat'] == "ok"):
        countnotinsets = int(format(res.find('photos').attrib['total']))

# Print the obtained results
print('  Total photos on flickr: {!s}'.format(countflickr))
print('Total photos not in sets: {!s}'.format(countnotinsets))
sybrenstuvel commented

@grahamwhiteuk if your EXIF library still doesn't support Py3, I would seriously consider using another library. They've had 10 years to port it.

@oPromessa:

you simply mean, documentation-wise, correct? No actually dropping support to Python 2.7, correct?

I mean that, regardless of which Linux distribution has which Python installed as /usr/bin/python, the world moved on from Py2 to Py3. Py3 was introduced a decade ago. There are major libraries/softwares that no longer support Python 2 at all any more, such as Django. For new projects it's just insane to start with Py2 now, because it'll be dead in just over a year. That is why I say it's time to assume that things are Py3, unless explicitly noted it's for the very old 2.7.

Currently I'm not actively developing the FlickrAPI library, because it's dynamic enough to deal with changes made by Flickr without having to change the library itself. However, I probably will drop Py2 support when I do something more than minor fixes/additions. It'll reduce the complexity of the code, allow me to use modern Python, and thus make it easier (and more fun!) to test, use, and develop.

Thanks for the example, I'll take a good look at it when I have the time. There are a few things that catch my attention, though. I would never use sys.stderr.write(), but rather use either the logging module or use print(xxx, file=sys.stderr) instead. That'll take care of using the correct platform-dependent newlines. Also you don't need to call str(x) when it's being formatted as string anyway -- str.format() takes care of you for that.

The global statements are unnecessary, and so are many of the parentheses.

The use of the finally statement seems very weird. This is meant to always run, even when there was an exception or when the function returns in the try clause. Right now it seems to handle the 'normal' case, which is a bad idea because it also will run when a FlickrError or any other exception was received.

All in all the code may work, but I'm not a great fan of the way it was written.

  • 👍1
hugovk commented

See http://python3statement.org/ for a list of projects dropping Python 2 in or before 2020.

Here's the pip installs for flickrapi from PyPI for last month:

python_version percent download_count
2.7 53.78% 1,054
3.6 26.58% 521
3.5 7.35% 144
3.4 5.82% 114
2.6 3.47% 68
3.3 2.96% 58
3.7 0.05% 1

Source: pypinfo --start-date -57 --end-date -27 --percent --pip --markdown flickrapi pyversion

grahamwhiteuk commented

I've moved to using py3exiv2 so my code is now fully Python 3. Thanks!

oPromessa commented

@sybrenstuvel great comments.

  • Had not used logging earlier to try to keep it simple... yup! my mistake. I'm now using it
  • droped use up globals
  • dropped str from .format(
  • yes, I was using finally for the normal case also.
  • also I've played with os.environ to get api_key and api_secret so the code works without changes. Don't know if it's the correct way to go about it!

Check out the revised version...

#!/usr/bin/env python
"""
    by oPromessa, 2018
"""
import sys
import os
import flickrapi
import logging

# -----------------------------------------------------------------------------
# authenticate
#
# Authenticates via flickrapi on flickr.com
#
def authenticate(xCfg):
    """
    Authenticate user so we can upload files.
    Assumes the cached token is not available or valid.
    
    Receives dictionary with the configuration.
    Returns an instance object for the class flickrapi
    """

    # Instantiate nuflickr for connection to flickr via flickrapi
    nuflickr = flickrapi.FlickrAPI(xCfg["api_key"],
                                   xCfg["api_secret"],
                                   token_cache_location=xCfg["TOKEN_CACHE"])
    # Get request token
    logging.warning('Getting new token.')
    try:
        nuflickr.get_request_token(oauth_callback='oob')
    except flickrapi.exceptions.FlickrError as ex:
        logging.error('+++010 Flickrapi exception on get_request_token. '
                      'Error code/msg: [{!s}]/[{!s}]'
                     .format(ex.code, ex))
        sys.exit(4)
    except Exception as ex:
        logging.error('+++020 Exception on get_request_token: [{!s}]. Exiting...'
                      .format(sys.exc_info()))
        sys.exit(4)

    # Show url. Copy and paste it in your browser
    # Adjust parameter "perms" to to your needs
    authorize_url = nuflickr.auth_url(perms=u'delete')
    print('Copy and paste following authorizaiton URL '
          'in your browser to obtain Verifier Code.')
    print(authorize_url)

    # Prompt for verifier code from the user.
    # Python 2.7 and 3.6
    # use "# noqa" to bypass flake8 error notifications    
    verifier = unicode(raw_input(  # noqa
                                 'Verifier code (NNN-NNN-NNN): ')) \
               if sys.version_info < (3, ) \
               else input('Verifier code (NNN-NNN-NNN): ')

    print('Verifier: {!s}'.format(verifier))

    # Trade the request token for an access token
    try:
        nuflickr.get_access_token(verifier)
    except flickrapi.exceptions.FlickrError as ex:
        logging.error('+++030 Flickrapi exception on get_access_token. '
                         'Error code/msg: [{!s}]/[{!s}]'
                         .format(ex.code, ex))
        sys.exit(5)

    print('{!s} with {!s} permissions: {!s}'.format(
          'Check Authentication',
          'delete',
          nuflickr.token_valid(perms='delete')))

    # Some debug...
    logging.info('Token Cache: [{!s}]', nuflickr.token_cache.token)
    
    return nuflickr


# -------------------------------------------------------------------------
# getCachedToken
#
# If available, obtains the flickrapi Cached Token from local file.
# returns the token
# Saves the token on the global variable nuflickr.
#
def getCachedToken(xCfg):
    """
    Attempts to get the flickr token from disk.
    
    Receives dictionary with the configuration.
    Returns an instance object for the class flickrapi
    """

    logging.warning('Obtaining Cached token')
    logging.warning('TOKEN_CACHE:[{!s}]'.format(xCfg["TOKEN_CACHE"]))
    nuflickr = flickrapi.FlickrAPI(xCfg["api_key"],
                                   xCfg["api_secret"],
                                   token_cache_location=xCfg["TOKEN_CACHE"])

    try:
        # Check if token permissions are correct.
        if nuflickr.token_valid(perms='delete'):
            logging.warning('Cached token obtained: [{!s}]'
                            .format(nuflickr.token_cache.token))
            return nuflickr
        else:
            logging.warning('Token Non-Existant.')
            return None
    except BaseException:
        logging.error('+++040 Unexpected error in token_valid. [{!s}]'
                      .format(sys.exc_info()))
        raise

# -----------------------------------------------------------------------------
# Global Variables + Main code
#
logging.basicConfig(stream=sys.stderr,
                    level=logging.WARNING,  # Use logging.DEBUG if required
                    format='[%(asctime)s]:[%(processName)-11s]'
                    '[%(levelname)-8s]:[%(name)s] %(message)s')
 
# Define two variables within your OS enviromnt (api_key, api_secret)
# to access flickr:
#
# export api_key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
# export api_secret=YYYYYYYYYYYYYYYY
#
xCfg = { 'api_key' : os.environ['api_key'],
         'api_secret' : os.environ['api_secret'],
         'TOKEN_CACHE' : os.path.join(os.path.dirname(sys.argv[0]), "token")}

print('Connecting to Flickr...')
flickr = None
flickr = getCachedToken(xCfg)

if (flickr is None):
    flickr = authenticate(xCfg)

print('Do something with Flickr...')
# Total FLickr photos count: find('photos').attrib['total'] -------------------
try:
    res = flickr.people.getPhotos(user_id="me", per_page=1)
except flickrapi.exceptions.FlickrError as ex:
    sys.stderr.write('+++050 Flickrapi exception on getPhotos. '
                     'Error code/msg: [{!s}]/[{!s}]'
                     .format(ex.code, ex))

countflickr = -1
if (res is not None) and (not res == "" and res.attrib['stat'] == "ok"):
    countflickr = format(res.find('photos').attrib['total'])

# Total photos not on Sets/Albums on FLickr -----------------------------------
# (per_page=1 as only the header is required to obtain total):
#       find('photos').attrib['total']
try:
    res = flickr.photos.getNotInSet(per_page=1)
except flickrapi.exceptions.FlickrError as ex:
    sys.stderr.write('+++060 Flickrapi exception on getNotInSet. '
                     'Error code/msg: [{!s}]/[{!s}]'
                     .format(ex.code, ex))

countnotinsets = -1
if (res is not None) and (not res == "" and res.attrib['stat'] == "ok"):
    countnotinsets = int(format(res.find('photos').attrib['total']))

# Print the obtained results
print('  Total photos on flickr: {!s}'.format(countflickr))
print('Total photos not in sets: {!s}'.format(countnotinsets))
oPromessa commented

Ran autopep8 and pylint over the example which is now available on this gist entry.

sswam commented

I found a couple issues with flickr.authenticate_via_browser():

  1. the flickr OAuth callback seems to be using https now
  2. it makes more than one request in the callback, so need to loop accepting requests until we get the verification token

Here's a patch. The patch is not useable as it is, because I hard-coded a path to a self-signed SSL certificate.

# diff -u flickrapi/auth.py.orig flickrapi/auth.py
--- flickrapi/auth.py.orig	2020-04-08 03:28:50.763824462 +1000
+++ flickrapi/auth.py	2020-04-08 03:46:43.820911440 +1000
@@ -19,6 +19,7 @@
 import os.path
 import sys
 import six
+import ssl
 
 from requests_toolbelt import MultipartEncoder
 import requests
@@ -66,6 +67,7 @@
         self.log.info('Creating HTTP server at %s', self.local_addr)
 
         http_server.HTTPServer.__init__(self, self.local_addr, OAuthTokenHTTPHandler)
+        self.socket = ssl.wrap_socket(self.socket, certfile='/home/sam/my/snakeoil-cert-and-key.pem', server_side=True)
 
         self.oauth_verifier = None
 
@@ -84,7 +86,7 @@
     def wait_for_oauth_verifier(self, timeout=None):
         """Starts the HTTP server, waits for the OAuth verifier."""
 
-        if self.oauth_verifier is None:
+        while self.oauth_verifier is None:
             self.timeout = timeout
             self.handle_request()
  • 🎉1