Crypto library problem in ST4

Question

Crypto library problem in ST4

Closed this issue 2 years ago · 19 comments

MacOS 3.11 on M1 running ST 4103.

reloading python 3.3 plugin Mediawiker.mediawiker
Traceback (most recent call last):
  File "/Applications/Sublime Text.app/Contents/MacOS/Lib/python33/sublime_plugin.py", line 308, in reload_plugin
    m = importlib.import_module(modulename)
  File "./python3.3/importlib/__init__.py", line 90, in import_module
  File "<frozen importlib._bootstrap>", line 1584, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 584, in _check_name_wrapper
  File "<frozen importlib._bootstrap>", line 1022, in load_module
  File "<frozen importlib._bootstrap>", line 1003, in load_module
  File "<frozen importlib._bootstrap>", line 560, in module_for_loader_wrapper
  File "<frozen importlib._bootstrap>", line 868, in _load_module
  File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mediawiker.py", line 20, in <module>
    from .mwcommands import mw_utils as utils
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/__init__.py", line 4, in <module>
    from .mw_add_category import MediawikerAddCategoryCommand, MediawikerSetCategoryCommand
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/mw_add_category.py", line 6, in <module>
    from . import mw_utils as utils
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/mw_utils.py", line 27, in <module>
    from ..lib import browser_cookie3
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/browser_cookie3/__init__.py", line 44, in <module>
    from Crypto.Cipher import AES
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Cipher/__init__.py", line 27, in <module>
    from Crypto.Cipher._mode_ecb import _create_ecb_cipher
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Cipher/_mode_ecb.py", line 46, in <module>
    """
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/_raw_api.py", line 258, in load_pycryptodome_raw_lib
    raise OSError("Cannot load native module '%s': %s" % (name, ", ".join(attempts)))
OSError: Cannot load native module 'Crypto.Cipher._raw_ecb': Trying '_raw_ecb.cpython-33m.so': dlopen(/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/../Cipher/_raw_ecb.cpython-33m.so, 6): image not found, Trying '_raw_ecb.abi3.so': dlopen(/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/../Cipher/_raw_ecb.abi3.so, 6): image not found, Trying '_raw_ecb.so': dlopen(/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/../Cipher/_raw_ecb.so, 6): no suitable image found.  Did find:
	/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/../Cipher/_raw_ecb.so: mach-o, but wrong architecture
	/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Cipher/_raw_ecb.so: mach-o, but wrong architecture

Answer 1 · 2021-04-30T13:30:12.000Z

In the ST support discord channel, someone notes ST3 run under Rosetta, ST4 run arms native.

Answer 2 · 2021-04-30T17:42:09.000Z

I haven't possibility to build libs under arm :(

Answer 3 · 2021-04-30T19:12:51.000Z

I'm still running pandoc (Haskell) and python as X86 because of the libraries not yet working on M1/ARM. ST4 is now being distributed as a universal binary (with X86 and ARM) and perhaps I can force it to run the X86 version under Rosetta. Not sure how this would be resolved in general. MediaWiker looks to be my only plugin with this issue.

Answer 4 · 2021-04-30T19:28:13.000Z

Without Crypto binaries we have possibility to work with authorization type "login" only. I'll try to find the way to build them.

Answer 5 · 2021-06-23T21:04:29.000Z

@tosher, I was using "login" above. Still, it seemed as if the crypto libraries are being called upon. Is it because I have https on?

		"English wikipedia":
		{
			"authorization_type": "login",
			"cookies_browser": "chrome",
			"domain": "",
			"host": "en.wikipedia.org",
			"http_auth_login": "",
			"http_auth_password": "",
			"https": true,
			"is_ssl_cert_verify": true,
			"is_wikia": false,
			"oauth_access_secret": "",
			"oauth_access_token": "",
			"oauth_consumer_secret": "",
			"oauth_consumer_token": "",
			"pagepath": "/wiki/",
			"password": "FOO",
			"path": "/w/",
			"preview_custom_head":
			[
			],
			"preview_sandbox": "",
			"proxy_host": "",
			"retry_timeout": 30,
			"show_red_links": false,
			"use_http_auth": false,
			"username": "Reagle",
		},

Answer 6 · 2021-06-24T17:40:47.000Z

Maybe this dependency fires anyway, I will check it.

P.S. If you have python/venv on your mac, you could build the library by just installing this module with the same python-version - required .so file will be in site-packages.

Answer 7 · 2021-06-24T19:56:34.000Z

I just installed the M1 version of ST4 4107 and see the following when installing MediaWiker:

...
ignored packages updated to: ["Markdown", "Mediawiker", "Vintage"]
reloading settings Packages/User/Package Control.sublime-settings
reloading settings Packages/User/Preferences.sublime-settings
Package Control: The dependency 'oauthlib' is not currently installed; installing...
Package Control: The dependency 'oauthlib' has successfully been installed or updated
Package Control: The dependency 'requests-oauthlib' is not currently installed; installing...
Package Control: The dependency 'requests-oauthlib' has successfully been installed or updated
reloading settings Packages/User/Package Control.sublime-settings
ignored packages updated to: ["Markdown", "Vintage"]
reloading settings Packages/Mediawiker/MediawikerPanel.sublime-settings
reloading python 3.3 plugin Mediawiker.mediawiker
Traceback (most recent call last):
  File "/private/var/folders/2l/73vdx0sd5rvcn38yg036h6500000gp/T/AppTranslocation/1F3FAC32-2C9E-439C-A5AF-B8AC4B52CFD2/d/Sublime Text.app/Contents/MacOS/Lib/python33/sublime_plugin.py", line 308, in reload_plugin
    m = importlib.import_module(modulename)
  File "./python3.3/importlib/__init__.py", line 90, in import_module
  File "<frozen importlib._bootstrap>", line 1584, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 584, in _check_name_wrapper
  File "<frozen importlib._bootstrap>", line 1022, in load_module
  File "<frozen importlib._bootstrap>", line 1003, in load_module
  File "<frozen importlib._bootstrap>", line 560, in module_for_loader_wrapper
  File "<frozen importlib._bootstrap>", line 868, in _load_module
  File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mediawiker.py", line 20, in <module>
    from .mwcommands import mw_utils as utils
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/__init__.py", line 4, in <module>
    from .mw_add_category import MediawikerAddCategoryCommand, MediawikerSetCategoryCommand
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/mw_add_category.py", line 6, in <module>
    from . import mw_utils as utils
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/mw_utils.py", line 23, in <module>
    from . import mw_properties as p
ImportError: cannot import name mw_properties
...

Answer 8 · 2021-06-24T20:03:00.000Z

Okay, running M1 again, here's the errors on install:

startup, version: 4107 osx arm64 channel: stable
executable: /Applications/Sublime Text.app/Contents/MacOS/sublime_text
application: /Applications/Sublime Text.app
working dir: /
packages path: /Users/reagle/Library/Application Support/Sublime Text 3/Packages
state path: /Users/reagle/Library/Application Support/Sublime Text 3/Local
zip path: /Applications/Sublime Text.app/Contents/MacOS/Packages
zip path: /Users/reagle/Library/Application Support/Sublime Text 3/Installed Packages
ignored_packages: ["Markdown", "Vintage"]
pre session restore time: 0.263356
loading dictionary Packages/Language - English/en_US.dic
font face "Liberation Mono" could not be found, defaulting to "Menlo"
OpenGL Context Information:
  GL API Version: 4.1 Metal - 71.6.4
  GLSL Version: 4.10
  Vendor: Apple
  Renderer: Apple M1
startup time: 0.358551
first paint time: 0.36435
...
reloading python 3.3 plugin Mediawiker.mediawiker
Traceback (most recent call last):
  File "/Applications/Sublime Text.app/Contents/MacOS/Lib/python33/sublime_plugin.py", line 308, in reload_plugin
    m = importlib.import_module(modulename)
  File "./python3.3/importlib/__init__.py", line 90, in import_module
  File "<frozen importlib._bootstrap>", line 1584, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 584, in _check_name_wrapper
  File "<frozen importlib._bootstrap>", line 1022, in load_module
  File "<frozen importlib._bootstrap>", line 1003, in load_module
  File "<frozen importlib._bootstrap>", line 560, in module_for_loader_wrapper
  File "<frozen importlib._bootstrap>", line 868, in _load_module
  File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mediawiker.py", line 20, in <module>
    from .mwcommands import mw_utils as utils
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/__init__.py", line 4, in <module>
    from .mw_add_category import MediawikerAddCategoryCommand, MediawikerSetCategoryCommand
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/mw_add_category.py", line 6, in <module>
    from . import mw_utils as utils
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/mw_utils.py", line 27, in <module>
    from ..lib import browser_cookie3
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/browser_cookie3/__init__.py", line 44, in <module>
    from Crypto.Cipher import AES
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Cipher/__init__.py", line 27, in <module>
    from Crypto.Cipher._mode_ecb import _create_ecb_cipher
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Cipher/_mode_ecb.py", line 46, in <module>
    """
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/_raw_api.py", line 258, in load_pycryptodome_raw_lib
    raise OSError("Cannot load native module '%s': %s" % (name, ", ".join(attempts)))
OSError: Cannot load native module 'Crypto.Cipher._raw_ecb': Trying '_raw_ecb.cpython-33m.so': dlopen(/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/../Cipher/_raw_ecb.cpython-33m.so, 6): image not found, Trying '_raw_ecb.abi3.so': dlopen(/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/../Cipher/_raw_ecb.abi3.so, 6): image not found, Trying '_raw_ecb.so': dlopen(/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/../Cipher/_raw_ecb.so, 6): no suitable image found.  Did find:
	/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/../Cipher/_raw_ecb.so: mach-o, but wrong architecture
	/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Cipher/_raw_ecb.so: mach-o, but wrong architecture
...
reloading settings Packages/Mediawiker/MediawikerPanel.sublime-settings
reloading python 3.3 plugin Mediawiker.mediawiker
Traceback (most recent call last):
  File "/Applications/Sublime Text.app/Contents/MacOS/Lib/python33/sublime_plugin.py", line 308, in reload_plugin
    m = importlib.import_module(modulename)
  File "./python3.3/importlib/__init__.py", line 90, in import_module
  File "<frozen importlib._bootstrap>", line 1584, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 584, in _check_name_wrapper
  File "<frozen importlib._bootstrap>", line 1022, in load_module
  File "<frozen importlib._bootstrap>", line 1003, in load_module
  File "<frozen importlib._bootstrap>", line 560, in module_for_loader_wrapper
  File "<frozen importlib._bootstrap>", line 868, in _load_module
  File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mediawiker.py", line 20, in <module>
    from .mwcommands import mw_utils as utils
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/__init__.py", line 4, in <module>
    from .mw_add_category import MediawikerAddCategoryCommand, MediawikerSetCategoryCommand
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/mw_add_category.py", line 6, in <module>
    from . import mw_utils as utils
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/mw_utils.py", line 23, in <module>
    from . import mw_properties as p
ImportError: cannot import name mw_properties
reloading settings Packages/User/Package Control.sublime-settings
reloading settings Packages/User/Preferences.sublime-settings
generating meta info summary
error parsing lexer: Packages/HTML/HTML.sublime-syntax: syntax inheritance does not allow for mixed versions at line 3 column 1
error parsing lexer: Packages/HTML/HTML.sublime-syntax: syntax inheritance does not allow for mixed versions at line 3 column 1
generating syntax summary
Error loading scope:source.coffee: Unable to find syntax file for scope "source.coffee" in Packages/MarkdownEditing/Markdown.sublime-syntax
reloading settings Packages/User/Preferences.sublime-settings

Answer 9 · 2021-06-24T20:08:24.000Z

@tosher, which library, specifically would I build? I'm running Rosetta2 in Terminal so my Python binaries are Intel. (If I can fix this issue, I might be able to move everything to native M1.) Doesn't ST use it's own Python version?

Answer 10 · 2021-07-12T18:17:03.000Z

Hi, just wanted to check on this, any word?

Answer 11 · 2021-07-16T12:57:58.000Z

https://github.com/tosher/Mediawiker/releases/tag/v3.6.8

Answer 12 · 2021-07-16T13:03:30.000Z

Doesn't ST use it's own Python version?

Yes, own version. To build some module, needs to setup required version, than create venv and install/compile a module.

In Sublime's console:

> import sys;sys.version
'3.8.8 (default, Mar 10 2021, 13:30:47) [MSC v.1915 64 bit (AMD64)]'

Answer 13 · 2021-07-16T13:51:10.000Z

Thanks! I tried to install from Github/repo and it failed, it looks like it's still try to build the crypto. I can try to build, but I'd need specific commands as I don't understand your suggestions.

File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mediawiker.py", line 20, in <module>
    from .mwcommands import mw_utils as utils
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/__init__.py", line 4, in <module>
    from .mw_add_category import MediawikerAddCategoryCommand, MediawikerSetCategoryCommand
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/mw_add_category.py", line 6, in <module>
    from . import mw_utils as utils
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/mwcommands/mw_utils.py", line 30, in <module>
    from ..lib import browser_cookie3
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/browser_cookie3/__init__.py", line 44, in <module>
    from Crypto.Cipher import AES
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Cipher/__init__.py", line 27, in <module>
    from Crypto.Cipher._mode_ecb import _create_ecb_cipher
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Cipher/_mode_ecb.py", line 46, in <module>
    """
  File "/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/_raw_api.py", line 258, in load_pycryptodome_raw_lib
    raise OSError("Cannot load native module '%s': %s" % (name, ", ".join(attempts)))
OSError: Cannot load native module 'Crypto.Cipher._raw_ecb': Trying '_raw_ecb.cpython-33m.so': dlopen(/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/../Cipher/_raw_ecb.cpython-33m.so, 6): image not found, Trying '_raw_ecb.abi3.so': dlopen(/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/../Cipher/_raw_ecb.abi3.so, 6): image not found, Trying '_raw_ecb.so': dlopen(/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/../Cipher/_raw_ecb.so, 6): no suitable image found.  Did find:
	/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Util/../Cipher/_raw_ecb.so: mach-o, but wrong architecture
	/Users/reagle/Library/Application Support/Sublime Text 3/Packages/Mediawiker/lib/Crypto.osx.x64/Crypto/Cipher/_raw_ecb.so: mach-o, but wrong architecture

Answer 14 · 2021-07-16T18:56:24.000Z

Another try :)

https://github.com/tosher/Mediawiker/releases/tag/v3.6.9

If it will work, you could use firefox cookies.

Answer 15 · 2021-07-16T20:43:56.000Z

Excellent, thanks! I can now install the plugin on an M1.

I can also login in via login method with username and password specified. I'd prefer not to keep my password in the config file, but I never got anything else to work. For instance, setting the login to cookies and specifying a path to the cookie file (is that what were supposed to do?)

	"firefox_cookie_files":
	[
		"/Users/reagle/Library/Application Support/Firefox/Profiles/6yk15l3u.default/cookies.sqlite",
	],

yields an error:

 '''Setup new connection to "English wikipedia".'''
Connecting to "https://en.wikipedia.org" ..  done.
Login in with authorization type cookies (firefox)..  done
Anonymous connection detected, forcing new connection.. 
Connecting to "https://en.wikipedia.org" ..  done.
Login in with authorization type cookies (firefox)..  done
Page [[Talk:Joseph M. Reagle Jr.]] was opened successfully from "English wikipedia".
Anonymous connection detected, forcing new connection.. 
Connecting to "https://en.wikipedia.org" ..  done.
Login in with authorization type cookies (firefox)..  done
  >>> APIError exception for get_notifications: ('login-required', 'You must be logged in.', 'See https://en.wikipedia.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes.'), trying to reconnect.. 
  >>> Forcing new connection.. 
Connecting to "https://en.wikipedia.org" ..  done.
Login in with authorization type cookies (firefox)..  done
Anonymous connection detected, forcing new connection.. 
Connecting to "https://en.wikipedia.org" ..  done.
Login in with authorization type cookies (firefox)..  done
Anonymous connection detected, forcing new connection.. 
Connecting to "https://en.wikipedia.org" ..  done.
Login in with authorization type cookies (firefox)..  done
  >>> APIError exception for get_notifications: ('login-required', 'You must be logged in.', 'See https://en.wikipedia.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes.')

Answer 16 · 2021-07-16T20:57:52.000Z

firefox_cookie_files needs to define if the plugin not works by default (couldn't to find cookies)

In firefox you must be logged in.

    "authorization_type": "cookies",
    "cookies_browser": "firefox",

Answer 17 · 2021-07-16T22:25:11.000Z

Yes, I had those two values set too and had logged in with Firefox beforehand, not sure why it didn’t work. But I appreciate you updating the plug-in.

I wonder if we could use the keyring module to store the password? https://pypi.org/project/keyring/

Answer 18 · 2021-07-17T13:30:19.000Z

If cookies file was not found, for firefox will be an error:
BrowserCookieError exception for get_page: Failed to find Firefox cookie

Answer 19 · 2022-06-28T12:41:38.000Z

I think the libraries have caught up now.