It is a small library dedicated to the fast Spacy model stack, preparing data for training and training models.
Choose the language of documantation: English here
!git clone https://github.com/Lednik7/SpacyToolKit.git
!python -m pip install -r SpacyToolKit/requirements.txt
For the library to work correctly, you need to download packages
pip install spacy
pip install googletrans
pip install pymorphy2==0.8
pip install sklearn
pip install numpy
Before you run the code below, you need to install one of the models:
Russian models:
!git clone -b v2.1 https://github.com/buriy/spacy-ru.git && cp -r ./spacy-ru/ru2/.
English models:
!python -m spacy download en_core_web_sm #size 11 mb
!python -m spacy download en_core_web_md #size 48 mb
!python -m spacy download en_core_web_lg #size 746 mb
More information about models en and ru
To begin, we import the necessary functions and the main class:
from SpacyToolKit.Tools import SpacyTools, sort_doc
from SpacyToolKit.other import get_translate
import en_core_web_sm #!python -m spacy download en_core_web_sm
Now create an instance of the class:
model = SpacyTools()
Since the model works best in English, we’ll use a translation from Google:
text = model.sample_text #your text
trans = get_translate(text) #text translation into english
Now we are ready to load the text into the model and make a prediction:
model.load_text(trans)
nlp = en_core_web_sm.load()
doc = model.create(nlp)
Let's look at the result:
print(sort_doc(doc)) #filter the results
print(model.text)
Output:
Time: 1.39 - en_sm
['Python', 'Data Science', 'GitHub']
Data Analyst with work experience. He graduated from SSAU with a master's degree in mathematics.
I have experience working with various databases and in writing macros. Worked with various data analysis frameworks in Python.
He participated in the development of several systems for data analysis. There are examples of their Data Science projects on GitHub:
Fine! Now you are ready to delve into the topic.
You can see examples here
This project is licensed under the MIT License - see the LICENSE file for details