_csv.Error: iterator should return strings, not bytes (did you open the file in text mode?)

Question

_csv.Error: iterator should return strings, not bytes (did you open the file in text mode?)

neomatic999 opened this issue 8 years ago · 13 comments

Hi there

We think we have everything correctly configured but the script always stops with the error detailed below. Can you help??

START RequestId: 89291a84-36f1-11e7-a7f4-1f479e181a6e Version: $LATEST
iterator should return strings, not bytes (did you open the file in text mode?): Error
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 2244, in lambda_handler
cred_report = get_cred_report()
File "/var/task/lambda_function.py", line 1943, in get_cred_report
for row in reader:
File "/var/lang/lib/python3.6/csv.py", line 111, in next
self.fieldnames

Answer 1 · 2017-05-17T20:23:37.000Z

Looking into this

Answer 2 · 2017-05-17T23:00:31.000Z

Hi, thank you.

We are one of your larger enterprise customers in APAC. I have also spoken to the local AWS team to help with this. I have asked them to reach out to you.

The code seems to fall over in the following section (iterating through reader object):

Answer 3 · 2017-05-18T05:00:08.000Z

Can you provide some info on the environment you are running the script on?
OS? Python version?

Answer 4 · 2017-05-18T06:57:36.000Z

Sure, I have tried a few different things. I tried running it as a Lambda function in AWS with the Python 3.6 runtime and all other values as default except for timeout which I set to 5 min.

I have also tried running it on my laptop on Win 10, with the latest version of Python.

In both examples, I get exactly the same error message.

Answer 5 · 2017-05-19T06:18:28.000Z

I have not yet test it against Python 3.6 but added it to todo.
To verify, can you modify the Lambda function and switch to Python 2.7 to test?

Answer 6 · 2017-05-20T12:41:57.000Z

Hi, yes it is fully working on Python 2.7.Thank you so much. Maybe update the docu on github? Thanks for getting back to me. From: Henrik Johansson <notifications@github.com> To: awslabs/aws-security-benchmark <aws-security-benchmark@noreply.github.com> Cc: neomatic999 <christophstrizik@yahoo.com.au>; Author <author@noreply.github.com> Sent: Friday, 19 May 2017, 16:18 Subject: Re: [awslabs/aws-security-benchmark] _csv.Error: iterator should return strings, not bytes (did you open the file in text mode?) (#28) I have not yet test it against Python 3.6 but added it to todo. To verify, can you modify the Lambda function and switch to Python 2.7 to test?— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

Answer 7 · 2017-05-22T07:30:22.000Z

Will update docs and have on the todo to validate against Python 3.6.
Thanks for reporting.

Answer 8 · 2017-05-22T07:46:47.000Z

No worries, thank you for getting back to us. We are now implementing your scripts across our accounts. I work for Origin Energy, one of the largest energy companies in Australia. I am actually the CISO but I don't mind getting my hands dirty from time to time. Thanks again. RegardsChristoph From: Henrik Johansson <notifications@github.com> To: awslabs/aws-security-benchmark <aws-security-benchmark@noreply.github.com> Cc: neomatic999 <christophstrizik@yahoo.com.au>; State change <state_change@noreply.github.com> Sent: Monday, 22 May 2017, 17:30 Subject: Re: [awslabs/aws-security-benchmark] _csv.Error: iterator should return strings, not bytes (did you open the file in text mode?) (#28) Will update docs and have on the todo to validate against Python 3.6. Thanks for reporting.— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

Answer 9 · 2017-05-22T15:58:11.000Z

Great to hear, and please send in any feature requests either here or to your account team/me.
I am currently working on some multi account support as well but no ETA.

Answer 10 · 2017-05-22T22:01:37.000Z

Will do, thank you! From: Henrik Johansson <notifications@github.com> To: awslabs/aws-security-benchmark <aws-security-benchmark@noreply.github.com> Cc: neomatic999 <christophstrizik@yahoo.com.au>; State change <state_change@noreply.github.com> Sent: Tuesday, 23 May 2017, 1:58 Subject: Re: [awslabs/aws-security-benchmark] _csv.Error: iterator should return strings, not bytes (did you open the file in text mode?) (#28) Great to hear, and please send in any feature requests either here or to your account team/me. I am currently working on some multi account support as well but no ETA.— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

Answer 11 · 2018-06-08T03:36:08.000Z

It's a python 3 thing.

Fixed it here:
https://github.com/obijan42/aws-security-benchmark/commit/2d748b1bb8d97d2fab4bc8dbc294b4b1c00442c8

Answer 12 · 2018-07-06T17:14:43.000Z

Had same error but saw this closed case and code seems to be working now thanks for comment @obijan42.

Answer 13 · 2019-12-31T07:05:44.000Z

This issue is occurring with Python 3.6, 3.8.