Attribute Error for Storage Access Key

[Black-Milk]

Trying iq-get <qubole_api_token> <query_id> results in the following error:

Traceback (most recent call last):
  File "/Users/xexm139/anaconda3/lib/python3.7/site-packages/qds_sdk/resource.py", line 77, in __getattr__
    return self.attributes[name]
KeyError: 'storage_access_key'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/xexm139/anaconda3/bin/iq-get", line 122, in <module>
    main(args)
  File "/Users/xexm139/anaconda3/bin/iq-get", line 102, in main
    write_intermediate_results(tmpFile, hc)
  File "/Users/xexm139/anaconda3/bin/iq-get", line 27, in write_intermediate_results
    hc.get_results(fp=tmpFile, delim='\t')
  File "/Users/xexm139/anaconda3/lib/python3.7/site-packages/qds_sdk/commands.py", line 165, in get_results
    boto_conn = boto.connect_s3(aws_access_key_id=acc.storage_access_key,
  File "/Users/xexm139/anaconda3/lib/python3.7/site-packages/qds_sdk/resource.py", line 79, in __getattr__
    raise AttributeError(name)
AttributeError: storage_access_key

[c-trl]

@Black-Milk could you please confirm what version qds_sdk you're using? Having a tough time replicating this. I've tried running with qds_sdk==1.12.0, qds_sdk==1.9.1 and qds_sdk==1.4.0, and was able to download results from a recent command.

[Black-Milk]

@c-trl This is what I have for qds-sdk:

I'll see if I can try another release.

[c-trl]

Hi @Black-Milk - some findings here:

In an older version of qds_sdk, get_results seems to have some unexpected behavior (see here: https://github.com/qubole/qds-sdk-py/blob/3cdaa5b6d9536b615c8476443b49b9a3bddc524b/qds_sdk/commands.py#L139L142; The get_results method uses Account.find() to authenticate, where Account.find() uses the token provided to return AWS credentials, but here Account.find() doesn't return storage_access_key as an attribute)

(authflow in v. 1.3.4)

This is resolved in newer versions (see: https://github.com/qubole/qds-sdk-py/blob/master/qds_sdk/commands.py#L305L310).

(authflow in v. 1.9.8 and up)

The get_results method is handled a bit differently now and uses Account.credentials_rest_entity_path to return credentials instead. I recommend updating your version of qds-sdk.

[Black-Milk]

Okay, then we might want to change setup.py in your repo to pull the latest version of qds_sdk. Right now it's set up to pull version 1.3.4

[skannan-adobe]

@Black-Milk isn't 1.3.4 the latest version ?

[Black-Milk]

@skannan-adobe per @c-trl 's comment, he's suggesting we use v1.9.8 of the qds_sdk. I changed this in the setup.py file and everything works as intended. Let me know if that clarifies.

[Black-Milk]

@skannan-adobe Also, latest release versions for qds_sdk can be checked here: https://github.com/qubole/qds-sdk-py/releases

[skannan-adobe]

@Black-Milk Ah, I see thanks. I was able to get it working for iq-get-unecoded