How to develop

Hi, I wanted to further issue #210 and cloned the repository. The only thing I managed to do was building the package with pip install . as suggested in the readme.
What I would like to do is:

run any of the unit tests
run my own script that uses the local clone of openTSNE
change things in openTSNE and test my changes

However, I am not very experienced in python package development and I have trouble finding out what I'm doing wrong.
I tried running python -m unittest tests/test_utils.py, but it errored:

E
======================================================================
ERROR: test_utils (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_utils
Traceback (most recent call last):
  File "C:\Users\haegeldd\AppData\Local\Programs\Python\Python310\lib\unittest\loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "C:\Users\haegeldd\git\openTSNE\tests\test_utils.py", line 4, in <module>
    from openTSNE.utils import clip_point_to_disc
  File "C:\Users\haegeldd\git\openTSNE\openTSNE\__init__.py", line 1, in <module>
    from .tsne import TSNE, TSNEEmbedding, PartialTSNEEmbedding, OptimizationInterrupt
  File "C:\Users\haegeldd\git\openTSNE\openTSNE\tsne.py", line 11, in <module>
    from openTSNE import _tsne
ImportError: cannot import name '_tsne' from partially initialized module 'openTSNE' (most likely due to a circular import) (C:\Users\haegeldd\git\openTSNE\openTSNE\__init__.py)


----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)
(venv)

I also tried to create my own script in the same directory where pyproject.toml sits:

import openTSNE.tsne
from sklearn import datasets

if __name__ == '__main__':
    iris = datasets.load_iris()
    x, y = iris["data"], iris["target"]
    tsne = openTSNE.tsne.TSNE()
    embed = tsne.fit(x)

But running that errors similarly:

$ python myscript.py
Traceback (most recent call last):
  File "C:\Users\haegeldd\git\openTSNE\myscript.py", line 1, in <module>
    import openTSNE.tsne
  File "C:\Users\haegeldd\git\openTSNE\openTSNE\__init__.py", line 1, in <module>
    from .tsne import TSNE, TSNEEmbedding, PartialTSNEEmbedding, OptimizationInterrupt
  File "C:\Users\haegeldd\git\openTSNE\openTSNE\tsne.py", line 11, in <module>
    from openTSNE import _tsne
ImportError: cannot import name '_tsne' from partially initialized module 'openTSNE' (most likely due to a circular import) (C:\Users\haegeldd\git\openTSNE\openTSNE\__init__.py)
(venv)

So yeah, how do I run anything?

Hi David, thanks for your willingness to work on this issue! I have seen this error before, but I've never been able to properly reproduce, diagnose, and fix this issue. If I remember correctly, this happened to me when running tests from Pycharm, but not when I ran it from the command line. This clearly doesn't apply here since you're running it from the command line, but it's probably the same issue. Since I'm replying quite late, if you have been able to resolve the issue, I'd be keen to know how you did it.

Try running the commands from the build script one by one as see what happens:

openTSNE/azure-pipelines.yml

Lines 64 to 92 in 52ae1d6

    
                 python -m pip install --upgrade pip 
        
                 python -m pip install flake8 pytest 
        
                 python -m pip install numpy 
        
               displayName: 'Install job dependencies' 
        
             # stop the build if there are Python syntax errors or undefined names 
        
             - script: flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics 
        
               displayName: 'Check for syntax errors' 
        
             - script: pip install -vv . 
        
               displayName: 'Install package' 
        
             - script: pip install pynndescent 
        
               displayName: 'Install optional dependencies - pynndescent' 
        
             - script: pip install hnswlib 
        
               env: 
        
                 HNSWLIB_NO_NATIVE: 1  # -march=native is not available on clang, so just disable it 
        
               displayName: 'Install optional dependencies - hnswlib' 
        
             # Since Python automatically adds `cwd` to `sys.path`, it's important we remove the local folder 
        
             # containing our code from the working directory. Otherwise, the tests will use the local copy 
        
             # instead of the installed package. We can easily achieve this by renaming the source folder. 
        
             - bash: mv openTSNE src 
        
               displayName: 'Remove source files from path' 
        
             - script: pytest -v 
        
               timeoutInMinutes: 15 
        
               displayName: 'Run unit tests'

	python -m pip install --upgrade pip
	python -m pip install flake8 pytest
	python -m pip install numpy
	displayName: 'Install job dependencies'

	# stop the build if there are Python syntax errors or undefined names
	- script: flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics
	displayName: 'Check for syntax errors'

	- script: pip install -vv .
	displayName: 'Install package'

	- script: pip install pynndescent
	displayName: 'Install optional dependencies - pynndescent'

	- script: pip install hnswlib
	env:
	HNSWLIB_NO_NATIVE: 1 # -march=native is not available on clang, so just disable it
	displayName: 'Install optional dependencies - hnswlib'

	# Since Python automatically adds `cwd` to `sys.path`, it's important we remove the local folder
	# containing our code from the working directory. Otherwise, the tests will use the local copy
	# instead of the installed package. We can easily achieve this by renaming the source folder.
	- bash: mv openTSNE src
	displayName: 'Remove source files from path'

	- script: pytest -v
	timeoutInMinutes: 15
	displayName: 'Run unit tests'