!(import! &self lib) is not working for python libraries

after installing metta-motto with pip

!(import! &self motto)
returns : Error (import! GroundingSpace-top motto) Failed to resolve module top:motto
also
!(import! &self numpy)
is not working too. So, MeTTa can't find python libraries.

The problem can be solved by adding "include_paths"
import site
include_path = site.getsitepackages()
metta = MeTTa(env_builder=Environment.custom_env(include_paths=include_path))

We may want to add site-packages to the default include path of MeTTa

@luketpeterson , what do you think?

Originally, I was thinking that I would register a catalog object that would query the Python module system. However that's probably a couple days' work because I don't have bindings for the catalog API in C and Python yet.

hyperon-experimental/python/hyperon/runner.py

Line 320 in 53a9ad3

    
           # #LP-TODO-Next Implement a Catalog that uses the Python module-space, so that any module loaded via `pip`

The suggestion from @besSveta to use the sitepackages directory probably accomplishes essentially the same thing with only two lines of code change. Basically adding the include path automatically when the new runner is initialized, right below this line here:

hyperon-experimental/python/hyperon/runner.py

Line 116 in 53a9ad3

    
           hp.env_builder_push_fs_module_format(env_builder, _PyFileMeTTaModFmt, 5000) #5000 is an arbitrary number unlikely to conflict with the arbitrary number chosen by other formats

I think that would be ok with me, if other people who understand the "pythonic" way agree that it wouldn't have any undesirable side-effects. For example, is there a way to install a python module via pip that puts it in a different location? Or is there a way that using a different pyenv would lead to modules in the directory that are not in sync with the running Python? Anything like that to watch out for?

The other question is: do we want to use pip as the main distribution mechanism? Or do we want to use the remote git-based catalog? If a module is self-contained (it has no dependencies on other python packages / wheels except what's part of the module's own code (both MeTTa and Python code)), then using pip seems silly.

But if a MeTTa package is the top of a big Python dependency tree, then pip might be better because people expect running pip install will install packages into their python environment, but they probably don't expect that importing a module in MeTTa will change what's installed in Python.

We would like to have it at least as a hotfix, because it makes deployment of metta-motto tutorials to the metta-lang.dev site much more straightforward.

The other question is: do we want to use pip as the main distribution mechanism?

For MeTTa itself, not necesserily. This is needed for the case, when Python users want to use Python packages, which depend on MeTTa and are MeTTa packages themselves. So, they install these packages using pip, and they should work.

I have a hot-fix ready that adds the site packages to the default search path.

However in testing it on a clean system, I installed metta-motto using pip install metta-motto, and now the error is:

[2024-05-28T11:51:54Z ERROR hyperonc::util] Python error loading MeTTa module 'motto'. ModuleNotFoundError("No module named 'openai'")

So I think the metta-motto python wheel needs to pull in the openai bindings as a dependency

closing via #702

However in testing it on a clean system, I installed metta-motto using pip install metta-motto, and now the error is:

Yes, thanks, openai is added to the requirements of metta-motto, but the update has not yet been published on pypi.