flake8-isort reports several import warnings without any reason

[Kamforka]

Hey there, I have these import statements at the top of my module:

import os

from flask import Blueprint, abort, current_app, jsonify
from flask.views import MethodView
from webargs import fields, missing
from webargs.flaskparser import use_kwargs
from werkzeug.utils import secure_filename

from tcst_api.models.data import DataContract, DataFile
from tcst_api.schemas.data import DataContractSchema, DataFileSchema

When I run flake8-isort on the file I get the following output:

tcst_api/views/data.py:2:1: I001 isort found an import in the wrong position
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0

Which is kinda meaningless and seems to be broken.
However if I run isort on the same content I get no errors as expected.

I'm using the following flake8 and isort versions:

flake8 - 3.5.0
flake8-isort - 2.3
isort - 4.3.4

Can you please help me with this?

[gforcada]

Hi, I'm on vacations. As soon as I'm back I will try to replicate the error and reply back.

Where d yu store your isort configuration? Could you paste it?

[Kamforka]

Hey,

I'm also on holiday so I'll paste the exact config after I get back.

However I store the isort conf in the setup.cfg file.

[Kamforka]

So my setup.cfg file content is the following:

[flake8]
ignore = D401
exclude = __pycache__, migrations
max-line-length=120

[isort]
line_length=120

Did you have the time to check the issue by the way?

[Kamforka]

Any update on this one?

[gforcada]

@Kamforka sorry I got sidetracked 😅 I'm having a look at it right now

[gforcada]

@Kamforka here is what I did, but could not reproduce your errors:

cd /tmp
virtualenv-2.7 venv
cd venv
. bin/activate
pip install flake8==3.5.0 flake8-isort==2.3 isort==4.3.4
cat > setup.cfg << EOF
[flake8]
ignore = D401
exclude = __pycache__, migrations
max-line-length=120

[isort]
line_length=120
EOF
cat test.py << EOF
import os

from flask import Blueprint, abort, current_app, jsonify
from flask.views import MethodView
from webargs import fields, missing
from webargs.flaskparser import use_kwargs
from werkzeug.utils import secure_filename

from tcst_api.models.data import DataContract, DataFile
from tcst_api.schemas.data import DataContractSchema, DataFileSchema

# so that we don't get the 'F401 import not used'
os()
Blueprint()
abort()
current_app()
jsonify()
MethodView()
missing()
fields()
use_kwargs()
secure_filename()
DataContract()
DataFile()
DataContractSchema()
DataFileSchema()
EOF

Now then if I run flake8 test.py I get:

test.py:8:1: I004 isort found an unexpected blank line in imports
test.py:9:1: I001 isort found an import in the wrong position
test.py:10:1: I001 isort found an import in the wrong position

Note that flake8-isort does quite a lot of searching to find a configuration file, see:

https://github.com/gforcada/flake8-isort/blob/master/flake8_isort.py#L74-L128

Maybe you have some other file with isort related configuration somewhere? 🤔

[Kamforka]

Thanks for looking after it.

Honestly I'm pretty sure I don't have configurations elsewhere than setup.cfg.

Well I'm using it within VS Code 1.23.1, can that cause any issue?

Note:

I put a print in flake8-isort's run method and I got the right settings file returned by search_isort_config() which is: C:\ongoing_projects\<...>\back-end\setup.cfg

[gforcada]

@Kamforka oh, you are using windows and is somewhat working? that's interesting to know, might it be that the problem? 🤔 do you have access to a linux box where to try it?

[Kamforka]

@gforcada Now that I had some time to dig into the issue I found out that the line endings causing some problem.
This is the partial content of Flake8Isort.lines in my case:

['# -*- coding: utf-8 -*-\r\n', '"""The app module, containing the app factory function."""\n', '\n', 'from flask import Flask, jsonify\n', ... ]

As you can see the first line ends with \r\n while all the others ending with a simple \n. This way the in_lines and out_lines of the sort_result will be quite different as the out_line will be separated by \r\n.

That's why I get a bunch of faulty error messages back from flake8-isort.

[Kamforka]

If I replace the original self.lines in Flake8Isort.run with this:

            with open(self.filename, 'r') as sort_file:
                lines = sort_file.readlines()

            with OutputCapture():
                sort_result = SortImports(
                    file_path=self.filename,
                    file_contents=''.join(lines),
                    check=True,
                    settings_path=settings_file,
                )

It works like a charm.

So where does the Flake8Isort object get the lines content? It seems like it gets corrupted somehow.

[Kamforka]

It seems like that the line ending bug is coming from the flake8.processor.FileProcess._readlines_py3 method. As it attaches \r\n for the first line while only \n for the rest.

What do you think how should we overcome this?

[gforcada]

@Kamforka hi!

Sorry for being so late on replying! 🙇‍♂️

If the error comes from flake8, then we should at least report it on flake8 issue tracker.

Depending on the answer there, we could patch it locally here in flake8-isort if it is an urgent matter, I would be ok with it.

[Kamforka]

Hey @gforcada,

No problem. In the meanwhile I raised an issue in the flake8 repo and they managed to fix the bug in the 3.6.0 release.

This is the related issue: https://gitlab.com/pycqa/flake8/issues/457

I checked it and it works fine now.

Thank you for your help as well. You can close this now!

[florisla]

For people looking for a quick fix, simply do:

pip install --upgrade flake8