Cloud-Native Neural Search? Framework for Any Kind of Data
Jina is a neural search framework that empowers anyone to build SOTA and scalable neural search applications in minutes.
⏱️ Save time - The design pattern of neural search systems. Quickly build solutions of indexing, querying, understanding multi-/cross-modal data such as video, image, text, audio, source code, PDF.
🌩️ Local & cloud friendly - Distributed architecture, scalable & cloud-native from day one. Same developer experience on local, Docker compose, Kubernetes.
🚀 Serve, scale & share - Serve a local project with HTTP, WebSockets or gRPC endpoints in just minute. Scale your neural search applications to meet your availability and throughput requirements. Share and reuse building blocks from Hub.
🍱 Own your stack - Keep end-to-end stack ownership of your solution. Avoid integration pitfalls you get with fragmented, multi-vendor, generic legacy tools. Enjoy the integration with the neural search ecosystem including DocArray, Hub and Finetuner.
pip install jina
For Jina 2.x users, please first uninstall old Jina via pip uninstall jina
before installation. Please also read the 2to3 migration guide.
More install options including Conda, Docker, and Windows can be found here.
We promise you can build a scalable ResNet-powered image search service in 20 minutes or less, from scratch to Kubernetes. If not, you can forget about Jina.
Document, Executor and Flow are three fundamental concepts in Jina.
- Document is a data structure contains multi-modal data.
- Executor is a self-contained component and performs a group of tasks on Documents.
- Flow ties Executors together into a processing pipeline, provides scalability and facilitates deployments in the cloud.
Leveraging these three concepts, let's build a simple image search service, as a "productization" of DocArray README.
Preliminaries: install PyTorch & Torchvision-
Import what we need.
from docarray import Document, DocumentArray from jina import Executor, Flow, requests
-
Copy-paste the preprocessing step and wrap it via
Executor
:class PreprocImg(Executor): @requests async def foo(self, docs: DocumentArray, **kwargs): for d in docs: ( d.load_uri_to_image_tensor(200, 200) # load .set_image_tensor_normalization() # normalize color .set_image_tensor_channel_axis( -1, 0 ) # switch color axis for the PyTorch model later )
-
Copy-paste the embedding step and wrap it via
Executor
:class EmbedImg(Executor): def __init__(self, **kwargs): super().__init__(**kwargs) import torchvision self.model = torchvision.models.resnet50(pretrained=True) @requests async def foo(self, docs: DocumentArray, **kwargs): docs.embed(self.model)
-
Wrap the matching step into an
Executor
:class MatchImg(Executor): _da = DocumentArray() @requests(on='/index') async def index(self, docs: DocumentArray, **kwargs): self._da.extend(docs) docs.clear() # clear content to save bandwidth @requests(on='/search') async def foo(self, docs: DocumentArray, **kwargs): docs.match(self._da, limit=9) del docs[...][:, ('embedding', 'tensor')] # save bandwidth as it is not needed
-
Connect all
Executor
s in aFlow
, scale embedding to 3:f = ( Flow(port_expose=12345) .add(uses=PreprocImg) .add(uses=EmbedImg, replicas=3) .add(uses=MatchImg) )
Plot it via
f.plot('flow.svg')
and you get: -
Download the image dataset.
Pull from Cloud | Manually download, unzip and load |
---|---|
index_data = DocumentArray.pull('demo-leftda', show_progress=True) |
|
- Index image data:
with f: f.post( '/index', index_data, show_progress=True, request_size=8, ) f.block()
The full indexing on 6000 images should take ~8 mins on a Macbook Air 2020.
Now you can use a Python client to access the service:
from jina import Client
c = Client(port=12345) # connect to localhost:12345
print(c.post('/search', index_data[0])['@m']) # '@m' is the matches-selector
To switch from gRPC interface to REST API, one can simply set protocol='http'
:
with f:
...
f.protocol = 'http'
f.block()
Now you can query it via curl
:
Or go to http://0.0.0.0:12345/docs
and test requests via a Swagger UI:
You can containerize the Executors and play them in a sandbox thanks to Hub.
-
Move each
Executor
class to a separate folder with one Python file in each:PreprocImg
-> 📁preproc_img/exec.py
EmbedImg
-> 📁embed_img/exec.py
MatchImg
-> 📁match_img/exec.py
-
Create a
requirements.txt
inembed_img
as it requirestorchvision
.. ├── embed_img │ ├── exec.py # copy-paste codes of ImageEmbeddingExecutor │ └── requirements.txt # add the requirement `torchvision` └── match_img └── exec.py # copy-paste codes of IndexExecutor └── preproc_img └── exec.py # copy-paste codes of IndexExecutor
-
Push all Executors to the Hub:
jina hub push preproc_img jina hub push embed_img jina hub push match_img
You will get three Hub Executors that can be used via Sandbox, Docker container or the source.
- In particular, sandbox hosts your Executor on Jina Cloud and allows you to play with it from local:
from docarray import DocumentArray from jina import Flow index_data = DocumentArray.pull( 'demo-leftda', show_progress=True ) # Download the dataset as shown in the tutorial above f = Flow().add(uses='jinahub+sandbox://2k7gsejl') with f: print(f.post('/', index_data[:10]))
-
Now that all Executors are in container, we can easily use Docker compose to orchestrate the Flow.
f = ( Flow(port_expose=12345) .add(uses='jinahub+docker://1ylut0gf') .add(uses='jinahub+docker://258lzh3c') ) f.to_docker_compose_yaml() # By default, stored at `docker-compose.yml`
-
Now in the console do:
docker-compose up
-
Create a Kubernetes cluster and get credentials (example in GCP, more K8s providers here):
gcloud container clusters create test --machine-type e2-highmem-2 --num-nodes 1 --zone europe-west3-a gcloud container clusters get-credentials test --zone europe-west3-a --project jina-showcase
-
Create a namespace
flow-k8s-namespace
for demonstration purpose:kubectl create namespace flow-k8s-namespace
-
Generate the kubernetes configuration files using one line of code:
f.to_k8s_yaml('./k8s_config', k8s_namespace='flow-k8s-namespace')
-
You
k8s_config
folder would look like the following:k8s_config ├── executor0 │ ├── executor0-head.yml │ └── executor0.yml ├── executor1 │ ├── executor1-head.yml │ └── executor1.yml └── gateway └── gateway.yml
-
Use
kubectl
to deploy your neural search application:kubectl apply -R -f ./k8s_config
-
Run port forwarding so that you can send requests to our Kubernetes application from local CLI :
kubectl port-forward svc/gateway -n flow-k8s-namespace 12345:12345
Now we have the service up running in Kubernetes!
- 👗 Fashion image search:
jina hello fashion
- 🤖 QA chatbot:
pip install "jina[demo]" && jina hello chatbot
- 📰 Multimodal search:
pip install "jina[demo]" && jina hello multimodal
- 🍴 Fork the source of a demo to your folder:
jina hello fork fashion ../my-proj/
- Create a new Jina project:
jina new hello-jina
- Join our Slack community to chat to our engineers about your use cases, questions, and support queries.
- Join our Engineering All Hands meet-up to
discuss your use case and learn Jina's new features.
- When? The second Tuesday of every month
- Where? Zoom (see our public calendar/.ical/Meetup group) and live stream on YouTube
- Subscribe to the latest video tutorials on our YouTube channel
Jina is backed by Jina AI and licensed under Apache-2.0. We are actively hiring AI engineers, solution engineers to build the next neural search ecosystem in open source.
We welcome all kinds of contributions from the open-source community, individuals and partners. We owe our success to your active involvement.