Endilll/vapoursynth-preview

IndexError in toolbars?

Closed this issue · 23 comments

if index >= len(self.toolbars_names):

Exception has occurred: IndexError
exception: no description
  File "C:\Users\begna112\Documents\GitHub\vapoursynth-preview\vspreview\core\abstracts.py", line 176, in __getitem__
    raise IndexError
  File "C:\Users\begna112\Documents\GitHub\vapoursynth-preview\vspreview\main.py", line 415, in __init__
    for toolbar in self.toolbars:
  File "C:\Users\begna112\Documents\GitHub\vapoursynth-preview\vspreview\core\better_abc.py", line 25, in __call__
    instance = NativeABCMeta.__call__(cls, *args, **kwargs)
  File "C:\Users\begna112\Documents\GitHub\vapoursynth-preview\vspreview\core\bases.py", line 24, in __call__
    cls.instance = super().__call__(*args, **kwargs)
  File "C:\Users\begna112\Documents\GitHub\vapoursynth-preview\vspreview\main.py", line 842, in main
    main_window = MainWindow()
  File "C:\Users\begna112\Documents\GitHub\vapoursynth-preview\run.py", line 4, in <module>
    main()

I don't know why, but occasionally scripts will cause this error. As far as I can tell, it has nothing to do with the scripts themselves. The same scripts will work on my other PC with an identical VS/Python configuration.

Any thoughts on what might cause this?

Thanks for reporting the issue.
I agree that it has nothing to do with the script, but I don't have ideas what could went wrong there.
for loop is supposed to query self.toolbars_names length and then iterate through range(0, n). toolbars_names are not changed inside loop to render length outdated either.

return len(self.toolbars_names)

If you have a PC when such exception occurs and another one which works fine, can you try to pin down the difference between them? I can imagine memory corruption causing this, even though I don't like to blame hardware for malfunctioning software.

I suppose it's possible. I have had this issue before, but I did just install all new RAM. I'll run a memtest to be sure.

Completed a memtest86 pass (1h20m) and found no errors. So at least I dont think it's the memory itself being bad. Is there some place vsp caches which might be interfering? I've already tried deleting the .vspreview folder. It never gets to the part where it recreates that folder.

There's no such place. It crashes as if self.toolbars changes while we iterate over it. Except that self.toolbars is hardcoded and not changed in runtime at all.

Is it possible that it has something to do with the scripts being on a network storage location? It's a 10Gbps link but perhaps?

I just tried moving the exact same scripts to a local drive and it loaded no problem.

Possibly related, when running from NAS, I usually get this in output (changes based on the location of course):

C:\Users/begna112/Documents/GitHub/vapoursynth-preview\vspreview\main.py:569: ResourceWarning: unclosed file <_io.TextIOWrapper.TextIOWrapper name='\\\\rui\\Fansubbing\\DDY\\2021-Summer\\Maid Dragon S2\\06\\encode\\.vspreview\\maidoragon2_0' encoding='cp6.yml' mode='r' encoding='cp1252'>
  yaml.load(storage_path.open(), Loader=yaml.Loader)
ResourceWarning: Enable tracemalloc to get the object allocation traceback

If it's acceptable for you, run VSP under debugger until you catch this exception. This way we can inspect application's state.
To ensure you'll get dropped into debugger at the right point, put breakpoint() before

Sure. What info can I get from the debugger for you?

That ResourceWarning is a separate issue that deserves it's own issue on GitHub. It happens to everyone, not just you.

I'm interested in contents of self.toolbars_names

and index

index: 6
all_toolbars_names: ['main', 'playback', 'scening', 'pipette', 'benchmark', 'misc', 'debug'] len=7
tollbars_names:  ('playback', 'scening', 'pipette', 'benchmark', 'misc', 'debug') len=6

Thank you.
So collections with toolbar names are intact.
I found out that in Python for loop calls __getitem__() until IndexError is raised (see note in docs).
So the question is why in your case IndexError is not catched by for loop implementation and propagated further.

Can you describe your environment? e.g. OS version, Python interpreter and its version

Windows OS:

WindowsBuildLabEx                                       : 19041.1.amd64fre.vb_release.191206-1406
WindowsCurrentVersion                                   : 6.3
WindowsEditionId                                        : Professional
WindowsInstallationType                                 : Client
WindowsProductId                                        : 00330-80000-00000-AA042
WindowsProductName                                      : Windows 10 Pro
WindowsVersion                                          : 2004
OsVersion                                               : 10.0.19041

Python

PS C:\Users\begna112> python --version
Python 3.9.6

Failing command (using F5 to run):

PS E:\DDY\BDs\Shingeki no Kyojin Final Part 1\.encode>  e:; cd 'e:\DDY\BDs\Shingeki no Kyojin Final Part 1\.encode'; & 'C:\Python39\python.exe' 'c:\Users\begna112\.vscode\extensions\ms-python.python-2021.8.1105858891\pythonFiles\lib\python\debugpy\launcher' '52193' '--' 'C:/Users/begna112/Documents/GitHub/vapoursynth-preview/run.py' 'e:\DDY\BDs\Shingeki no Kyojin Final Part 1\.encode\shingeki_final1_66.py'

Tested and removing the . from .encode does not matter.

Noticed my other PC is on windows 2009, not 2004. And 21H1 is available now. I'm going to go ahead and upgrade to see if that resolves it.

Same issue on 2H21.

I'm running Windows 10 Pro 21H1 myself, and upgrading to CPython 3.9.6 from 3.9.1 doesn't help to reproduce your issues.
It feels like CPython on your machine is more sensitive (for the lack of better word) than usual. That ResourceWarning suggests this, too. Otherwise me and other users would be able to reproduce behavior you see.

How would I troubleshoot that? Reinstall python?

I copied (not moved) my entire code directory to a new folder, still on the NAS and it works from that location, while it still fails from the old one. How certain are you that the unclosed yaml is unrelated? I wonder if some filesystems are more prone to failure on unclosed files. In this case, it's a ZFS raidz2 array. If it's perhaps trying to reallocate the same block to the same file...?

Just fixed #36. Can you pull the latest master and try again?
I don't expect anything to change, though.

Okay i pulled and tested. No change. But it seems pretty likely there's something wrong with the specific files in-storage or the path to them. I literally copied .encode\ to !encode\ in the same parent directory and it works there...

I'll try deleting .encode, recopying it in and seeing how it works.

After completing the above, the problem for that folder resolved. I suspect this must be some kind of conflict between a COW filesystem or network attached storage. If I can create it again after you closed #36 , then I'll add more info here.

That's an interesting outcome.
I'll close this issue for now. Feel free to reopen it if it the problem persists.