astral-sh/ruff

Implement Pylint

charliermarsh opened this issue Β· 278 comments

This is the parent issue for tracking parity with Pylint. Below, we've enumerated all Pylint rules.

Rules that are checked off have been implemented in Ruff already (either as a Pylint rule, e.g., PLE0237, or by way of overlap with another linter like Pyflakes, as with missing-format-string-key).

Rules that are crossed out have been removed some consideration. (In most cases, crossed-out rules represent Pylint specific rules, e.g., rules related to Pylint configuration.)

At time of writing, many of the remaining rules require type inference and/or multi-file analysis, and aren't ready to be implemented in Ruff. (See: #970 (comment) for an enumeration.) If you're looking to start work on a specific rule, I'd suggest commenting in the issue, to get some input on whether Ruff is capable of supporting it at present.

For guidance on getting started, see the Contributing documentation.

Note: Don't implement rules that are part of Pylint extension until #1774 is completed. Don't implement rules that mainly target Python 2.

Error

  • abstract-class-instantiated / E0110
  • access-member-before-definition / E0203
  • assigning-non-slot / E0237 (PLE0237)
  • assignment-from-no-return / E1111
  • assignment-from-none / E1128
  • await-outside-async / E1142 (PLE1142)
  • bad-configuration-section / E0014
  • bad-except-order / E0701
  • bad-exception-cause / E0705
  • bad-format-character / E1300 (PLE1300)
  • bad-plugin-value / E0013
  • bad-reversed-sequence / E0111
  • bad-str-strip-call / E1310 (PLE1310)
  • bad-string-format-type / E1307 (PLE1307)
  • bad-super-call / E1003 (mainly targets Python 2)
  • bidirectional-unicode / E2502 (PLE2502)
  • broken-collections-callable / E6005
  • broken-noreturn / E6004
  • catching-non-exception / E0712
  • class-variable-slots-conflict / E0242
  • continue-in-finally / E0116 (PLE0116)
  • dict-iter-missing-items / E1141
  • duplicate-argument-name / E0108
  • duplicate-bases / E0241 (PLE0241)
  • format-needs-mapping / E1303 (F502)
  • function-redefined / E0102 (F811)
  • import-error / E0401
  • inconsistent-mro / E0240
  • inherit-non-class / E0239
  • init-is-generator / E0100 (PLE0100)
  • invalid-all-format / E0605 (PLE0605)
  • invalid-all-object / E0604 (PLE0604)
  • invalid-bool-returned / E0304
  • invalid-bytes-returned / E0308
  • invalid-character-backspace / E2510 (PLE2510)
  • invalid-character-carriage-return / E2511
  • invalid-character-esc / E2513 (PLE2513)
  • invalid-character-nul / E2514 (PLE2514)
  • invalid-character-sub / E2512 (PLE2512)
  • invalid-character-zero-width-space / E2515 (PLE2515)
  • invalid-class-object / E0243
  • invalid-enum-extension / E0244
  • invalid-envvar-value / E1507
  • invalid-format-returned / E0311
  • invalid-getnewargs-ex-returned / E0313
  • invalid-getnewargs-returned / E0312
  • invalid-hash-returned / E0309
  • invalid-index-returned / E0305
  • invalid-length-hint-returned / E0310
  • invalid-length-returned / E0303
  • invalid-metaclass / E1139
  • invalid-repr-returned / E0306
  • invalid-sequence-index / E1126
  • invalid-slice-index / E1127
  • invalid-slice-step / E1144
  • invalid-slots / E0238
  • invalid-slots-object / E0236
  • invalid-star-assignment-target / E0113
  • invalid-str-returned / E0307
  • invalid-unary-operand-type / E1130
  • invalid-unicode-codec / E2501
  • logging-format-truncated / E1201
  • logging-too-few-args / E1206 (PLE1206)
  • logging-too-many-args / E1205 (PLE1205)
  • logging-unsupported-format / E1200
  • method-hidden / E0202
  • misplaced-bare-raise / E0704 (PLE0704)
  • misplaced-format-function / E0119
  • missing-format-string-key / E1304 (F524)
  • missing-kwoa / E1125
  • mixed-format-string / E1302 (F506)
  • modified-iterating-dict / E4702
  • modified-iterating-set / E4703 (PLE4703)
  • no-member / E1101
  • no-method-argument / E0211
  • no-name-in-module / E0611
  • no-self-argument / E0213 (N805)
  • no-value-for-parameter / E1120
  • non-iterator-returned / E0301
  • nonexistent-operator / E0107 (B002)
  • nonlocal-and-global / E0115 (PLE0115)
  • nonlocal-without-binding / E0117 (PLE0117)
  • not-a-mapping / E1134
  • not-an-iterable / E1133
  • not-async-context-manager / E1701
  • not-callable / E1102
  • not-context-manager / E1129
  • not-in-loop / E0103 (F701, F702)
  • notimplemented-raised / E0711 (F901)
  • potential-index-error / E0643 (PLE0643)
  • raising-bad-type / E0702
  • raising-non-exception / E0710
  • redundant-keyword-arg / E1124
  • relative-beyond-top-level / E0402 (TID252)
  • repeated-keyword / E1132 (PLE1132)
  • return-arg-in-generator / E0106
  • return-in-init / E0101 (PLE0101)
  • return-outside-function / E0104 (F706)
  • singledispatch-method / E1519 (PLE1519)
  • singledispatchmethod-function / E1520 (PLE5120)
  • star-needs-assignment-target / E0114
  • syntax-error / E0001 (E999)
  • too-few-format-args / E1306 (F524)
  • too-many-format-args / E1305 (F522)
  • too-many-function-args / E1121
  • too-many-star-expressions / E0112 (F622)
  • truncated-format-string / E1301 (F501)
  • undefined-all-variable / E0603 (F822)
  • undefined-variable / E0602 (F821)
  • unexpected-keyword-arg / E1123
  • unexpected-special-method-signature / E0302 (PLE0302)
  • unhashable-member / E1143
  • unpacking-non-sequence / E0633
  • unsubscriptable-object / E1136
  • unsupported-assignment-operation / E1137
  • unsupported-binary-operation / E1131
  • unsupported-delete-operation / E1138
  • unsupported-membership-test / E1135
  • used-before-assignment / E0601
  • used-prior-global-declaration / E0118 (PLE0118)
  • yield-inside-async-function / E1700 (PLE1700)
  • yield-outside-function / E0105 (F704)

Warning

  • abstract-method / W0223
  • anomalous-backslash-in-string / W1401 (W605)
  • anomalous-unicode-escape-in-string / W1402
  • arguments-differ / W0221
  • arguments-out-of-order / W1114
  • arguments-renamed / W0237
  • assert-on-string-literal / W0129 (PLW0129)
  • assert-on-tuple / W0199 (F631)
  • attribute-defined-outside-init / W0201
  • bad-builtin / W0141
  • bad-chained-comparison / W3601
  • bad-dunder-name / W3201 (PLW3201)
  • bad-format-string / W1302
  • bad-format-string-key / W1300
  • bad-indentation / W0311
  • bad-open-mode / W1501 (PLW1501)
  • bad-staticmethod-argument / W0211 (PLW0211)
  • bad-thread-instantiation / W1506
  • bare-except / W0702 (E722)
  • binary-op-exception / W0711 (PLW0711)
  • boolean-datetime / W1502
  • broad-exception-caught / W0718
  • broad-exception-raised / W0719 (TRY002)
  • cell-var-from-loop / W0640 (B023)
  • comparison-with-callable / W0143
  • confusing-with-statement / W0124
  • consider-ternary-expression / W0160 (SIM108)
  • dangerous-default-value / W0102 (B006)
  • deprecated-argument / W4903
  • deprecated-class / W4904
  • deprecated-decorator / W4905
  • deprecated-method / W4902
  • deprecated-module / W4901
  • deprecated-typing-alias / W6001
  • differing-param-doc / W9017
  • differing-type-doc / W9018
  • duplicate-except / W0705 (B014)
  • duplicate-key / W0109 (F601)
  • duplicate-string-formatting-argument / W1308
  • duplicate-value / W0130 (B033)
  • eq-without-hash / W1641 (PLW1641)
  • eval-used / W0123 (PGH001)
  • exec-used / W0122 (S102)
  • expression-not-assigned / W0106 (B018)
  • f-string-without-interpolation / W1309 (F541)
  • fixme / W0511 (FIX001, FIX002, FIX003, FIX004)
  • forgotten-debug-statement / W1515 (T100 )
  • format-combined-specification / W1305 (F525)
  • format-string-without-interpolation / W1310 (F541)
  • global-at-module-level / W0604 (PLW0604)
  • global-statement / W0603 (PLW0603)
  • global-variable-not-assigned / W0602 (PLW0602)
  • global-variable-undefined / W0601
  • implicit-str-concat / W1404 (ISC001)
  • import-self / W0406 (PLW0406)
  • inconsistent-quotes / W1405 (Q000)
  • invalid-envvar-default / W1508 (PLW1508)
  • invalid-format-index / W1307
  • invalid-overridden-method / W0236
  • isinstance-second-argument-not-valid-type / W1116
  • keyword-arg-before-vararg / W1113 (B026)
  • logging-format-interpolation / W1202 (G001)
  • logging-fstring-interpolation / W1203 (G004)
  • logging-not-lazy / W1201 (G002)
  • lost-exception / W0150 (B012)
  • method-cache-max-size-none / W1518 (B019)
  • misplaced-future / W0410 (F404)
  • missing-any-param-doc / W9021
  • missing-format-argument-key / W1303 (F524)
  • missing-format-attribute / W1306
  • missing-param-doc / W9015
  • missing-parentheses-for-call-in-test / W0126
  • missing-raises-doc / W9006
  • missing-return-doc / W9011 (Pylint extension)
  • missing-return-type-doc / W9012
  • missing-timeout / W3101
  • missing-type-doc / W9016
  • missing-yield-doc / W9013
  • missing-yield-type-doc / W9014
  • modified-iterating-list / W4701
  • multiple-constructor-doc / W9005
  • named-expr-without-context / W0131 (PLW0131)
  • nan-comparison / W0177 (PLW0117)
  • nested-min-max / W3301 (PLW3301)
  • non-ascii-file-name / W2402 (N999)
  • non-parent-init-called / W0233
  • non-str-assignment-to-dunder-name / W1115
  • overlapping-except / W0714
  • overridden-final-method / W0239
  • pointless-exception-statement / W0133 (PLW0133)
  • pointless-statement / W0104 (B018)
  • pointless-string-statement / W0105
  • possibly-unused-variable / W0641
  • preferred-module / W0407
  • protected-access / W0212 (SLF001)
  • raise-missing-from / W0707 (TRY200)
  • raising-format-tuple / W0715
  • redeclared-assigned-name / W0128
  • redefined-builtin / W0622 (A001)
  • redefined-loop-name / W2901 (PLW2901)
  • redefined-outer-name / W0621
  • redefined-slots-in-subclass / W0244
  • redundant-returns-doc / W9008
  • redundant-u-string-prefix / W1406 (UP025)
  • redundant-unittest-assert / W1503
  • redundant-yields-doc / W9010
  • reimported / W0404 (F811)
  • self-assigning-variable / W0127 (PLW0127)
  • self-cls-assignment / W0642
  • shallow-copy-environ / W1507
  • signature-differs / W0222
  • subclassed-final-class / W0240
  • subprocess-popen-preexec-fn / W1509 (PLW1509)
  • subprocess-run-check / W1510 (PLW1510)
  • super-init-not-called / W0231
  • super-without-brackets / W0245 (PLW0245)
  • too-many-try-statements / W0717
  • try-except-raise / W0706 (TRY302)
  • unbalanced-dict-unpacking / W0644
  • unbalanced-tuple-unpacking / W0632
  • undefined-loop-variable / W0631
  • unknown-option-value / W0012
  • unnecessary-ellipsis / W2301
  • unnecessary-lambda / W0108 (PLW0108)
  • unnecessary-pass / W0107 (PIE790)
  • unnecessary-semicolon / W0301 (E703)
  • unreachable / W0101
  • unspecified-encoding / W1514 (PLW1514)
  • unused-argument / W0613 (ARG001)
  • unused-format-string-argument / W1304 (F507)
  • unused-format-string-key / W1301 (F504)
  • unused-import / W0611 (F401)
  • unused-private-member / W0238
  • unused-variable / W0612 (F841)
  • unused-wildcard-import / W0614
  • useless-else-on-loop / W0120 (PLW0120)
  • useless-param-doc / W9019
  • useless-parent-delegation / W0246
  • useless-type-doc / W9020
  • useless-with-lock / W2101 (PLW2101)
  • using-constant-test / W0125
  • using-f-string-in-unsupported-version / W2601
  • using-final-decorator-in-unsupported-version / W2602
  • while-used / W0149
  • wildcard-import / W0401 (F403)
  • wrong-exception-operation / W0716

Convention

  • bad-classmethod-argument / C0202 (`N804)
  • bad-docstring-quotes / C0198 (Q002)
  • bad-file-encoding / C2503
  • bad-mcs-classmethod-argument / C0204
  • bad-mcs-method-argument / C0203
  • compare-to-empty-string / C1901 (PLC1901)
  • compare-to-zero / C2001
  • consider-iterating-dictionary / C0201 (SIM118)
  • consider-using-any-or-all / C0501 (SIM110, SIM111)
  • consider-using-dict-items / C0206 (PLC0206)
  • consider-using-enumerate / C0200
  • consider-using-f-string / C0209
  • dict-init-mutate / C3401
  • disallowed-name / C0104
  • docstring-first-line-empty / C0199 (D210)
  • empty-docstring / C0112 (D419)
  • import-outside-toplevel / C0415 (PLC0415)
  • import-private-name / C2701 (PLC2701)
  • invalid-characters-in-docstring / C0403
  • invalid-name / C0103 (N815)
  • line-too-long / C0301 (E501)
  • misplaced-comparison-constant / C2201 (SIM300)
  • missing-class-docstring / C0115 (D101)
  • missing-final-newline / C0304 (W292)
  • missing-function-docstring / C0116 (D103)
  • missing-module-docstring / C0114 (D100)
  • mixed-line-endings / C0327
  • multiple-imports / C0410 (E401)
  • multiple-statements / C0321 (E701, E702)
  • non-ascii-module-import / C2403 (PLC2403)
  • non-ascii-name / C2401 (PLC2401)
  • single-string-used-for-slots / C0205 (PLC0205)
  • singleton-comparison / C0121 (E711 , E712)
  • superfluous-parens / C0325
  • too-many-lines / C0302 (not compatible with the formatter)
  • trailing-newlines / C0305 (W391)
  • trailing-whitespace / C0303 (W291)
  • typevar-double-variance / C0131 (PLC0131)
  • typevar-name-incorrect-variance / C0105 (PLC0105)
  • typevar-name-mismatch / C0132 (PLC0132)
  • unexpected-line-ending-format / C0328
  • ungrouped-imports / C0412 (I001)
  • unidiomatic-typecheck / C0123 (E721)
  • unnecessary-direct-lambda-call / C3002 (PLC3002)
  • unnecessary-dunder-call / C2801 (PLC2801)
  • unnecessary-lambda-assignment / C3001 (E731)
  • unneeded-not / C0113 (SIM208)
  • use-implicit-booleaness-not-comparison / C1803
  • use-implicit-booleaness-not-len / C1802
  • use-maxsplit-arg / C0207
  • use-sequence-for-iteration / C0208 (PLC0208)
  • useless-import-alias / C0414 (PLC0414)
  • wrong-import-order / C0411 (I001)
  • wrong-import-position / C0413 (E402 )
  • wrong-spelling-in-comment / C0401
  • wrong-spelling-in-docstring / C0402

Refactor

  • chained-comparison / R1716
  • comparison-of-constants / R0133 (PLR0133)
  • comparison-with-itself / R0124 (PLR0124)
  • condition-evals-to-constant / R1727
  • confusing-consecutive-elif / R5601
  • consider-alternative-union-syntax / R6003 (UP007)
  • consider-merging-isinstance / R1701 (PLR1701)
  • consider-swap-variables / R1712
  • consider-using-alias / R6002 (UP006)
  • consider-using-assignment-expr / R6103
  • consider-using-augmented-assign / R6104
  • consider-using-dict-comprehension / R1717 (C402)
  • consider-using-from-import / R0402
  • consider-using-generator / R1728 (C417)
  • consider-using-get / R1715 (SIM401)
  • consider-using-in / R1714
  • consider-using-join / R1713
  • consider-using-min-builtin / R1730 (PLR1730)
  • consider-using-max-builtin / R1731 (PLR1730)
  • consider-using-namedtuple-or-dataclass / R6101
  • consider-using-set-comprehension / R1718 (C401)
  • consider-using-sys-exit / R1722 (PLR1722)
  • consider-using-ternary / R1706 (PLR1706 )
  • consider-using-tuple / R6102
  • consider-using-with / R1732 (SIM115)
  • cyclic-import / R0401
  • duplicate-code / R0801
  • else-if-used / R5501 (PLR5501)
  • empty-comment / R2044 (PLR2044)
  • inconsistent-return-statements / R1710 (RET501, RET502)
  • literal-comparison / R0123 (F632)
  • magic-value-comparison / R2004 (PLR2004)
  • no-classmethod-decorator / R0202 (PLR0202)
  • no-else-break / R1723 (RET508)
  • no-else-continue / R1724 (RET507)
  • no-else-raise / R1720 (RET506)
  • no-else-return / R1705 (RET505)
  • no-self-use / R6301 (PLR6301)
  • no-staticmethod-decorator / R0203 (PLR0203)
  • property-with-parameters / R0206 (PLR0206)
  • redefined-argument-from-local / R1704 (PLR1704)
  • redefined-variable-type / R0204
  • simplifiable-condition / R1726
  • simplifiable-if-expression / R1719 (SIM210, SIM211)
  • simplifiable-if-statement / R1703 (SIM108)
  • simplify-boolean-expression / R1709
  • stop-iteration-return / R1708
  • super-with-arguments / R1725 (UP008)
  • too-complex / R1260 (C901)
  • too-few-public-methods / R0903
  • too-many-ancestors / R0901 (requires multifile analysis)
  • too-many-arguments / R0913 (PLR0913)
  • too-many-boolean-expressions / R0916 (PLR0916)
  • too-many-branches / R0912 (PLR0912)
  • too-many-instance-attributes / R0902 (requires #1774)
  • too-many-locals / R0914 (PLR0914)
  • too-many-nested-blocks / R1702 (PLR1702)
  • too-many-public-methods / R0904 (PLR0904)
  • too-many-return-statements / R0911 (PLR0911)
  • too-many-statements / R0915 (PLR0915)
  • trailing-comma-tuple / R1707 (COM818)
  • unnecessary-comprehension / R1721 (C416)
  • unnecessary-dict-index-lookup / R1733 (PLR1733)
  • unnecessary-list-index-lookup / R1736 (PLR1736)
  • use-a-generator / R1729 (C419)
  • use-dict-literal / R1735 (C406)
  • use-list-literal / R1734 (C405)
  • use-set-for-membership / R6201 (PLR6201)
  • useless-object-inheritance / R0205 (UP004)
  • useless-option-value / R0022
  • useless-return / R1711 (PLR1711)

Need to figure out what the right "check code" approach is here. We could just start to put these under RUF. Or we could do PYE for "Pylint Error", and so on. Or we could use PYL for all of them. In either case, though, we have to allow four-digit codes, as opposed to our current three-digit codes.

I vote for PYE as it would keep the same categories and the migration would be easier.

I'll work on unneeded-not this weekend.

I would suggest PL as the pylint prefix (PLE -> PyLint Error). I think PYE might be a bit ambiguous with PYflakes, PYdocstyle, etc.

Would absolutely love to see this happen. I know flake8 intentionally decided not to implement e.g. unexpected-keyword-arg because it required looking outside of the current file for imported functions.

I expect you've already considered this, and really hope this won't stop ruff from implementing it, but just wanted to raise it.

@relsunkaev πŸ‘ Went with PLE! The first code is checked in as PLE1142 (matching E1142 from Pylint).

@MartinBernstorff - Definitely. The multi-file stuff will come, just not as quickly.

Not sure if it should be separate issue, but looking at missing-function-docstring rule (C0116), pylint's version of it is different. It only requires that a function has docstring some in class hierarchy and does not require over-ridden methods to have a docstring. As an example,

class Foo:
  def validate(self):
    """Doc."""
    ...

class Bar(Foo):
  def validate(self):
    ...

gives no errors in pylint even though second validate is missing docstring being pylint allows inheriting one. This is very useful when implementing interface and the docstring would really be same for all things that subclass.

I'd like to get pointless-statement, unnecessary-ellipsis, and unnecessary-pass in.

A while ago I set out to find all the pylint codes make redundant by type checking.

Unfortunately that code got lost to the sands of time.

You might consider skipping those as type checking tools are very sophisticated, if you can do the effort of compiling that list

I've also thought very long and hard about the fact that pylint appears very sophisticated when it comes to types because it builds on astroid which works very hard to infer types based on code (pre-type-annotations)

Ideally ruff (or any modern linter) is type-aware to be as helpful as possible, but that requires either:

  1. A Rust-implemented type checkers for Python. Short of a large company writing and maintaining this I doubt it'll keep up with the other big dogs
  2. Having the type information be seeded to ruff. Aside from the overhead and protocol, this actually seems feasible.
dae commented

For what it's worth, the type awareness of pylint is the killer feature that keeps us using it in our project despite how painfully slow it is. Our first-party code is almost fully typed, and mypy is generally good at catching typing issues. We've found multiple instances where pylint caught an issue that mypy missed however - usually in third-party code with missing or incomplete type definitions, but occasionally also in our own code when mypy doesn't notice (eg due to the presence of __getattr__: python/mypy#6251).

Out of curiosity, have you tried pyright? It claims to have inference support for codebases that aren't fully typed.

dae commented

Mypy has a check_untyped_defs which sounds equivalent, and we use that. We currently rely on mypy's no_strict_optional which pyright has no equivalent to, so pyright is not currently usable on our codebase.

Started to look at import-error / E0401,

From what i can tell - ruff doesn't currently use any "context" about the python environment, right?
In order to add this rule - we need to use something like pkgutil - which can be slow, or implement import resolution in ruff.

@charliermarsh Is the project open to any of these options currently?

We have some support for import resolution, but not enough to power import-error right now.

What we do support is (1) a src list, which is used for first-party resolution, similar to isort and reorder-python-imports (so it detects module-level first-party imports, but doesn't do "full" resolution), (2) a namespace-packages list, for namespace packages, and (3) we do crawl the filesystem to find all __init__.py files to power some checks.

Does import-error need to be able to lookup third-party packages?

Does import-error need to be able to lookup third-party packages?

FWIW this is one of the biggest reasons we use pylint at all, despite obviously being much slower to do than just first party code.

Any plans on when and how to implement more import resolution features?

WDYT about creating an issue to gather rules that will need these capabilities?

I just went through and mapped all the checked-off rules to the relevant codes within Ruff. I'm guessing we implement more than the 86 that are checked off (at time of writing), but it's my best guess based on a single pass through the list.

It’d be really nice if ruff can understand the inline pylint ignores to make migration very easy.

It’d be really nice if ruff can understand the inline pylint ignores to make migration very easy.

this is covered in #1203

We can probbably mark bad-configuration-section as complete, it has to do with a pylint refactor

useless-option-value too, it's a warning for an issue in the pylint configuration.

bad-plugin-value is also for pylint plugins that cannot be loaded, which we do not need.

Thank you! Going to strike them out.

What's the policy with duplicate logic being checked by multiple linters? E.g. too-many-format-args seems functionally equivalent to F524 which is already implemented:

https://github.com/charliermarsh/ruff/blob/41900316186ae65b588ec0fda64c21bfee4b9956/src/rules/pyflakes/rules/strings.rs#L191

We tend towards including the Flake8 variant over the Pylint variant. We're working on enabling rule aliasing, so that we'll be able to map multiple codes to the same underlying rule (details TBD).

@charliermarsh do we have anyway of knowing the minor version of python we are on? broken-collections-callable should only be run on python 3.9.0 and 3.9.1, and I don't know of a way to detect only those two sub-versions.

@colin99d - Hmm -- not really. I think we should just skip that rule.

Just my two cents, but from working on pylint for some time we have seen the definite need to specify the expected python version, preferably with the config option py-version which is slowly becoming a standard within the python ecosystem. It might be nice to support such an option as well.

@charliermarsh, do we have any way of knowing that the name token data is a dict, particularly if the item is defined in a different file.

data = {'Paris': 2_165_423, 'New York City': 8_804_190, 'Tokyo': 13_988_129}
for city, population in data:  # [dict-iter-missing-items]
    print(f"{city} has population {population}.")

@colin99d - No, unlike Pylint we don't have that capability right now.

@charliermarsh Would love to understand the T-shirt size effort for E0611 (before I take the time to dive into the code base for contribution) and how likely it is to be part of ruff.

I guess the simplest example would be:

from requests import aaa

Would pass on ruff or flake8, on pylint, i'd get:

E0611: No name 'aaa' in module 'requests' (no-name-in-module)

@AvnerCohen - I would call that a "Large". Ruff operates under a single-file model right now, so each file is analyzed in isolated (though we do some traversals upfront to infer the package hierarchy etc.). To power something like that, we need to move to a model in which we do some analysis of the available members in each module as a first pass, then enforce lint rules as a second pass. It's within scope, but it's a significant feature more-so than a single rule, if that makes sense :)

Related to #2914 -- I've posted some more commentary in there.

The pylint rules implemented by PyCharm: #2972

The pylint rules implemented by PyCharm: #2972

Opened a new issue for PyCharm inspections covered and not covered by PyLint here, since #2972 may soon be closed: #3040

I was looking at maybe implementing E0203 along with some others that are of the same ilk, but I noticed that, at least in my noddy testing and reading around, we don't really seem to keep track of these neither per scope. Would the sensible thing be to push a new Binding onto the bindings member of the Checker in checkers/ast.rs? If so, what sort of Binding? Binding::Binding?

r3m0t commented

I'll do invalid-character-*

@charliermarsh What is the suggested approach to configure a project so as to have Ruff subsume Pylint, where both implement the same checks? Right now, we have lot of duplicate warnings.

W0123 is an overlap of PHG001, so I guess that could be ticked off?
Same goes for W0122 with S102. πŸ‘πŸ»

For convenience, this page in the Pylint documentation lists all of the Pylint rules, with links to descriptions and examples of each: https://pylint.pycqa.org/en/latest/user_guide/messages/messages_overview.html.

@charliermarsh , I've began work on implementing duplicate-argument-name / E0108, however, the parser throws an error when attempting to parse the test Python file. Does this mean it can be closed off as already implemented or is there something I can alter elsewhere? :)

cargo run -p ruff_cli -- check crates/ruff/resources/test/fixtures/pylint/duplicate_argument_name.py --no-cache --select PLE0108 
   Compiling ruff v0.0.257 (/home/jacoblatonis/Projects/ruff/crates/ruff)
   Compiling ruff_cli v0.0.257 (/home/jacoblatonis/Projects/ruff/crates/ruff_cli)
warning: `ruff` (lib) generated 3 warnings
    Finished dev [unoptimized + debuginfo] target(s) in 21.66s
     Running `target/debug/ruff check crates/ruff/resources/test/fixtures/pylint/duplicate_argument_name.py --no-cache --select PLE0108`
error: Failed to parse crates/ruff/resources/test/fixtures/pylint/duplicate_argument_name.py: duplicate argument 'first_name' in function definition at line 1 column 39

@latonis - Ah yeah, we can't support that one right now, since it's not valid Python. It's sort of debatable whether Ruff should support that.

Type checkers ought to flag that as invalid syntax. Or running the code under tests.

Maybe at the most ruff flags it can't operate because there's a syntax error at and well there's your failure

πŸ‘ Yeah Ruff would flag that there's a syntax error in that file, and would probably tell you where / what it is but I haven't tested that specific case.

The table above suggests ruff implements cell-var-from-loop, but it doesn't show up in the documentation or on the command-line with ruff check. Is it implemented under a different name, or was that a mistake and it's not actually implemented?

Ah, cell-var-from-loop is equivalent to B023. For ease of switching for pylint users, then, might be handy to alias B023 as PLW0640, if aliasing is supported.

rjarry commented

Hey folks, Thanks a lot for this awesome project!

I have scrubbed over the list of pylint checks and it looks like (at least) no-name-in-module / E0611 would require leveraging the new ImportMap feature.

Also, producing a proper implementation of no-member / E1101 seems significantly more involved.

Are there any plans to work on these? I wish I could help but I am a bit clueless about rust...

Hey folks, Thanks a lot for this awesome project!

I have scrubbed over the list of pylint checks and it looks like (at least) no-name-in-module / E0611 would require leveraging the new ImportMap feature.

Also, producing a proper implementation of no-member / E1101 seems significantly more involved.

Are there any plans to work on these? I wish I could help but I am a bit clueless about rust...

It's something I would like to do after finishing the work on detecting cyclic imports. The ImportMap only shows what a module imports, not exports, but I suppose a similar type of approach would be needed whereby after traversing a module, I guess we could have anExportMap if you will of the names available.

rjarry commented

@rjarry I have to admit, I wasn't planning on doing it, I don't know what the wider plans are for ruff, I do know that ruff uses RustPython's parser, and to be honest, I don't know how t RustPython works, but if has an analogue to .pyc files, maybe we could. But it would be something I would need to learn/understand in order to do so.

Can RustPython make a very small interpreter enough to import names?
The import rules of python are very complicated and depending on comparably bigger library _imp, _frozen_importlib etc

Performance hit from using real interpreter for just imports can be pretty high especially as some python files run logic at module level (metaclasses/init_subclass/any global code). There are major libraries like tensorflow where just importing them takes several seconds.

Can we and/or PyLint instead use CPython audit events to record imports? https://docs.python.org/3/library/audit_events.html

Linters don't run code. Any runtime construct is likely not useful here.

Why do you think that is necessarily true? Have you read #970 (comment)?

Some linters run code. Various others do not for multiple reasons. In ruff’s case one major core feature is performance. Running real imports can be very slow and for some files could immediately make ruff orders of magnitude slower. Python files are allowed to run whatever code they want at import time and some libraries do non trivial initialization when imported. It may also be mild security concern although most use cases if you lint a file you typically trust content in that file.

pyright is one static analysis tool that avoids almost all imports/runtime execution for those reasons. I think mypy is the same. Some tools do choose to accept trade offs of runtime imports and use them. Given ruff’s focus on performance I don’t think it makes sense for ruff to do them.

I'm peaking at subprocess-run-check / W1510. It's not a particularly important check, but the fix is pretty simple, to just add a new parameter to the function call check=True. But it get's a little complicated to me if the function is formatted across more than one line.
If anyone has any suggestions of any other similar fixes where a parameter is added to a method that I could use as an example, I'd like to take a stab at this simple case.

I'm peaking at subprocess-run-check / W1510. It's not a particularly important check, but the fix is pretty simple, to just add a new parameter to the function call check=True. But it get's a little complicated to me if the function is formatted across more than one line. If anyone has any suggestions of any other similar fixes where a parameter is added to a method that I could use as an example, I'd like to take a stab at this simple case.

I've not tried it, but it should be in the 'args' member of the StmtKind::FunctionDef struct
Another issue to consider is that a user might have imported subprocess under an alias, but you should test how pylint deals with it, or even look at the source for pylint to find out how it's done.

Pylint uses astroid, so I wouldn't be surprised if it understands aliases pretty well.

That being said, don't let perfect be the enemy of good enough. It isn't something that is usually aliased, and reporting on the simple case is worth it to do now, and improve later.

I don't think @charliermarsh or any of the Astral folks have mentioned how type information (or any information not easily gleamable from an AST) will make it into ruff, so doing the easy thing is likely the first pass at any of these

Pylint uses astroid, so I wouldn't be surprised if it understands aliases pretty well.

That being said, don't let perfect be the enemy of good enough. It isn't something that is usually aliased, and reporting on the simple case is worth it to do now, and improve later.

I don't think @charliermarsh or any of the Astral folks have mentioned how type information (or any information not easily gleamable from an AST) will make it into ruff, so doing the easy thing is likely the first pass at any of these

This is true, but we could track those aliases considering the Import/ImportFrom StmtKind structs have that information, we already track imports per module as it is.

@chanman3388 - Thankfully we do handle alias tracking and it's largely abstracted away in end-user code. If you grep around for calls to resolve_call_path, this returns the absolute path for an expression, so it handles aliased imports, import-froms, etc. This is what we tend to use across Ruff to check (e.g.) "Is this expression a call to subprocess.check_call?" (Note that this doesn't handle code like import subprocess; foo = subprocess; foo.check_call, we treat those as independent bindings right now; but it does handle code like import subprocess as foo; foo.check_call.)

@dciborow - Sounds like you're mostly wondering how to construct the proper fix, even in the event that the function arguments are split over multiple lines, etc. -- is that right? We have crates/ruff/src/autofix/actions.rs#remove_argument which is similar. You could write an add_or_replace_argument helper with a similar structure.

In general, I think you want to find the last argument in the call, and then insert a comma (if there's no trailing comma), and then insert the argument -- unless the argument is already present in the call, in which case you just want to replace its value. This will require some mix of looking at the AST (to see if the argument is present) and looking the token stream (to see if there's a trailing comma). The method I mentioned above does some of that kind of stuff, as an example.

@charliermarsh , thanks!

I had been working on adding auto-fixes to our vscode-pylint plugin, (this one got more complicated then my easy regex fixer can handle) but was recommending to check out Ruff, and find that it runs a lot faster inside of VS Code. I also like that I can use rust to auto-fix things outside of VSCode. So, while I have never written Rust before, excited to see what I can figure out.

A lot of good, useful Pylint lints require at least some type inference to be useful (even if it's as rudimentary as backtracking through assignments), and that's probably the thing that I'm interested in the most from Ruff.

If such an inference system starts building up, it might give the project momentum to finish through the pylint port. It'd make auto fixes much easier as well.

E0102 seems to be already covered by pyflakes F811?

no-value-for-parameter / E1120 is what I expected to be implemented, any timelines on this or how I can enable it manually(within ruff)?

Nevermind, It's not covered by Pyflakes rules so probably impossible to implement

it's not implemented right now. I'd assume it's fairly hard to implement, at least to the degree that pylint can detect.

Now that E1206, E1205 are implemented, seems like E1306, and E1305 would be good first issues if someone wants to take it up. Can we also tag this issue as good first issue @charliermarsh ? Lots of easy to implement rules here.

alonme commented

i'll take a look at E1306 and E1305

Those specific rules might actually be covered by existing Pyflakes rules. I think E1306 is the same as F522, and E1305 is the same as F524?

alonme commented

@charliermarsh yup - that looks right.
should we add anything to the code to make it clear that they also cover the pylint rules?

@alonme - For now, I'll just mark them as completed here and reference those tasks. In the future, we will mark them as aliases in the code, but there's no support for that right now.

alonme commented

cool,
taking a look at E0112

EDIT:
oh well seems like that is also covered by F622

alonme commented

and i am not sure if E6004 is worth implementing as it is only relevant in python 3.7.0 and 3.7.1, and python 3.7 will be EOL in a month or so

Calling out a few that look like good-first-rules: E1003, E0241, W0130, W0131.

Unfortunately E1003 is not as easy as it looked due to classes inheriting others.

So PLE1003 will be on-hold until we get a way to track down the inherited parents.

@nm-remarkable Hey!

Can you maybe create a separate issue/discussion about all the pylint lints that seem hard to implement? I can collaborate and help create a knowledge base for other collaborators.

Or, we can talk over text or email if that suits better.

I'll take a look at W0130 and W0131.

W0706 is already implemented as TRY302.

Hey folks, I'm interested in taking on so of this work. I have a rough draft for PLE1128 already, how would I go about pushing that up for review?

@wiwa5606 creating a fork of this repository on your own account and then making a pull request should be good

Hey folks, I'm interested in taking on so of this work. I have a rough draft for PLE1128 already, how would I go about pushing that up for review?

If you have it in a branch on your fork, then GitHub should give you the option to create a pull request. Normally you would do this from your fork. You can take a look at the merged issues higher up for what these look like. I hope that answers your question!

Thanks, that helps! Raised #4532

W0707 is an alias for TRY200

W1201is also implemented as combination of several rules logging-format G

I am looking into E1111

I would really need W1203, in the next days maybe I can start working at it

@DavideCanton Isn't W1203 already implemented as one of the logging-format rules G?

@Skylion007 oh, I didn't know there was a flake8 plugin for that too :/ thanks for your help!

Some notes about various listed codes:

  • return-arg-in-generator/E0106 probably should not be implemented as it only applies to dead versions of Python (3.3 and earlier).
  • unknown-option-value / W0012 probably shouldn't be implemented either, as it's specific to pylint error codes.
  • using-f-string-in-unsupported-version / W2601 probably shouldn't be implemented as it only applies to dead versions of Python (3.5 and earlier).
  • boolean-datetime / W1502 also only applies to dead versions of Python (3.4 and earlier).
  • bad-indentation / W0311 is implemented by E111 and the pylint example gives a true positive with this configuration:
    [tool.ruff]
    select = ["E111"]
    
  • compare-to-empty-string / C1901 has been renamed to use-implicit-booleaness-not-comparison-to-string / C1804.
  • compare-to-zero / C2001 has been renamed to use-implicit-booleaness-not-comparison-to-zero / C1805.
  • consider-using-f-string / C0209 is implemented by UP031 + UP032.

(I combined the other comments into this one, sorry for noise).

Note: bad-indentation / W0311 is implemented by E111 and the pylint example gives a true positive with this configuration:

[tool.ruff]
select = ["E111"]

Note: boolean-datetime / W1502 also only applies to dead versions of Python (3.4 and earlier).

Note: unknown-option-value / W0012 probably shouldn't be implemented either, as it's specific to pylint error codes.

I've went through and evaluated all the rules listed here that aren't yet implemented (see the previous comment for some other findings), to try and find out which rules are redundant when using mypy. Mostly this was to find out what value Pylint brings that isn't already covered by the combination of ruff + mypy, but I think this could also be useful to ruff developers when prioritizing which of the rules listed to work on.

There was one rule that is partially implemented in mypy, because Pylint seems to have its own semantics for what to consider an abstract method:

  • abstract-method

I found these rules to have an equivalent error in mypy:

  • abstract-class-instantiated
  • arguments-differ
  • assigning-non-slot
  • assignment-from-no-return
  • assignment-from-none
  • bad-exception-cause
  • bad-format-character
  • bad-reversed-sequence
  • bad-super-call
  • bad-thread-instantiation
  • catching-non-exception
  • comparison-with-callable
  • deprecated-class
  • dict-iter-missing-items
  • format-combined-specification
  • global-variable-undefined
  • import-error
  • inconsistent-mro
  • inherit-non-class
  • init-is-generator
  • invalid-class-object
  • invalid-enum-extension
  • invalid-envvar-value
  • invalid-format-returned
  • invalid-hash-returned
  • invalid-metaclass
  • invalid-overridden-method
  • invalid-repr-returned
  • invalid-sequence-index
  • invalid-slice-index
  • invalid-slots-object
  • invalid-slots
  • invalid-star-assignment-target
  • invalid-str-returned
  • invalid-unary-operand-type
  • invalid-unicode-codec
  • isinstance-second-argument-not-valid-type
  • method-hidden
  • misplaced-format-function
  • missing-format-argument-key
  • missing-format-attribute
  • missing-kwoa
  • missing-type-doc
  • missing-yield-type-doc
  • no-member
  • no-value-for-parameter
  • non-iterator-returned
  • non-str-assignment-to-dunder-name
  • nonlocal-and-global
  • not-a-mapping
  • not-an-iterable
  • not-async-context-manager
  • not-callable
  • not-context-manager
  • overridden-final-method
  • raising-bad-type
  • raising-non-exception
  • redefined-variable-type
  • redundant-keyword-arg
  • relative-beyond-top-level
  • self-cls-assignment
  • signature-differs
  • star-needs-assignment-target
  • subclassed-final-class
  • super-without-brackets
  • too-many-function-args
  • typevar-double-variance
  • typevar-name-mismatch
  • unbalanced-dict-unpacking
  • unbalanced-tuple-unpacking
  • unexpected-keyword-arg
  • unhashable-member
  • unpacking-non-sequence
  • unsubscriptable-object
  • unsupported-assignment-operation
  • unsupported-binary-operation
  • unsupported-delete-operation
  • unsupported-membership-test
  • used-before-assignment
  • using-final-decorator-in-unsupported-version
  • wrong-exception-operation

I didn't evaluate these rules:

  • deprecated-argument
  • invalid-character-carriage-return
  • invalid-characters-in-docstring

And I found these shouldn't be implemented for various reasons, see previous comment:

  • boolean-datetime
  • return-arg-in-generator
  • unknown-option-value
  • using-f-string-in-unsupported-version

I hope this is useful. Here's a repository with my categorizations, should someone be interested in the details: https://github.com/antonagestam/pylint-mypy-overlap

This PR also implements one of the rules for FixMe comments (W0511): #4681

I'm looking into unnecessary-ellipsis and unnecessary-pass.

Edit: unnecessary-ellipsis is partially covered by ellipsis-in-non-empty-class-body