scrapy/itemadapter

ADAPTER_CLASSES may not supported appendleft

Opened this issue · 2 comments

ADAPTER_CLASSES is defined as Iterable

ADAPTER_CLASSES: Iterable[Type[AdapterInterface]] = deque(

However, in the docs it's suggested to use appendleft method which is not part of the Iterable API.
This causes errors with tools like mypy:

error: "Iterable[Type[AdapterInterface]]" has no attribute "appendleft" [attr-defined]

Either the class type should be changed to List or other classes that may support appendleft or a different way to extend ADAPTER_CLASSES should be given.

ADAPTER_CLASSES was originally typed as a Deque, this was most recently changed in #71 (#74). The appendleft example works in the context of the default implementation, this is mentioned in the docs:

The default implementation uses a collections.deque to support efficient addition/deletion of adapters classes to both ends, but if you are deriving a subclass (see the section on extending itemadapter for additional information), any other iterable (e.g. list, tuple) will work.

I can think of the following, though there might be a more correct and compact solution:

ADAPTER_CLASSES: Union[Iterable[Type[AdapterInterface]], Deque[Type[AdapterInterface]]] = deque( 

I was thinking we need to keep the typing as it is, and change the documentation example to something like:

ItemAdapter.ADAPTER_CLASSES = (CustomClass, *ItemAdapter.ADAPTER_CLASSES)