jina: A Python repository from darshit7

English • 日本語 • français • Deutsch • Русский язык • 中文

Website • Docs • Examples • Newsletter • Hub (beta) • Dashboard (beta) • Twitter • We are Hiring

Want to build a search system backed by deep learning? You come to the right place!

Jina is the easiest way to build neural search in the cloud. It provides an one-stop solution for multi-/cross-modality search. Jina has long-term support from a full-time, venture-backed team.

⏱️ Time Saver - Bootstrapping an AI-powered search system with Jina is just minutes thing. It saves engineers months of time!

🧠 First-class AI models - Jina is a new design pattern of the neural search system, offering first-class support on state-of-the-art AI models.

🌌 Universal Search - Large-scale indexing and querying data of any kind on multiple platforms. Video, image, long/short text, music, source code you name it!

🚀 Production Ready - Cloud-native features come out-of-the-box, e.g. containerization, microservice, distributing, scaling, sharding, async IO, REST, gRPC.

🧩 Plug & Play - Extending Jina with simple Python scripts or Docker images optimized to your search domain. Check out Jina Hub for more extensions.

Install
Jina "Hello, World!" 👋🌍
Build your own Project
Tutorials
Documentation
Contributing
Community
Join Us
Roadmap
License

Install

Install from PyPi

On Linux/MacOS with Python >= 3.7:

pip install jina

To install Jina with extra dependencies, or install on Raspberry Pi please refer to the documentation.

...or Run in a Docker Container

We provide a universal Docker image (only 80MB!) that supports multiple architectures (including x64, x86, arm-64/v7/v6). Simply run:

docker run jinaai/jina --help

Jina "Hello, World!" 👋🌍

As a starter, you can try out our "Hello, World" - a simple demo of image neural search for Fashion-MNIST. No extra dependencies needed, just run:

jina hello-world

...or even easier for Docker users, no install required:

docker run -v "$(pwd)/j:/j" jinaai/jina hello-world --workdir /j && open j/hello-world.html  # replace "open" with "xdg-open" on Linux

Click here to see console output

The Docker image downloads Fashion-MNIST training and test data and tells Jina to index 60,000 images from the training set. Then it randomly samples images from the test set as queries and asks Jina to retrieve relevant results. The whole process takes about 1 minute, and it'll eventually open a webpage and show results like this:

The implementation behind it is as simple as can be:

Python API

or use YAML spec

or use Dashboard

from jina.flow import Flow

f = (Flow()
        .add(uses='encoder.yml', parallel=2)
        .add(uses='indexer.yml', shards=2, 
             separated_workspace=True))

with f:
    f.index(fashion_mnist, batch_size=1024)

!Flow
pods:
  encode:
    uses: encoder.yml
    parallel: 2
  index:
    uses: indexer.yml
    shards: 2
    separated_workspace: true

All the big words you can name: computer vision, neural IR, microservice, message queue, elastic, replicas & shards. They all happened in just one minute!

Adding Parallelism and Sharding

from jina.flow import Flow

f = (Flow().add(uses='encoder.yml', parallel=2)
           .add(uses='indexer.yml', shards=2, separated_workspace=True))

Distributing the Flow

from jina.flow import Flow

f = Flow().add(uses='encoder.yml', host='192.168.0.99')

Using Docker Container

from jina.flow import Flow

f = (Flow().add(uses='jinahub/cnn-encode:0.1')
           .add(uses='jinahub/faiss-index:0.2', host='192.168.0.99'))

Concatenating Embeddings

from jina.flow import Flow

f = (Flow().add(name='eb1', uses='BiTImageEncoder')
           .add(name='eb2', uses='KerasImageEncoder', needs='gateway')
           .join(needs=['eb1', 'eb2'], uses='_concat'))

Enabling Network Query

from jina.flow import Flow

f = Flow(port_expose=45678, rest_api=True)

with f:
    f.block()

Intrigued? Play with different options:

jina hello-world --help

Be sure to continue with our Jina 101 Guide - to understand all key concepts of Jina in 3 minutes!

Build your own Project

pip install cookiecutter && cookiecutter gh:jina-ai/cookiecutter-jina

With Cookiecutter you can easily create a Jina project from templates with one terminal command. This creates a Python entrypoint, YAML configs and a Dockerfile. You can start from there.

Tutorials

Jina 101 Concept Illustration Book, Copyright by Jina AI Limited

Jina 101: First Thing to Learn About Jina

English • 日本語 • français • Português • Deutsch • Русский язык • 中文 • عربية

Tutorials	Level
Use Flow API to Compose Your Search Workflow Orchestrate Pods to work together: sequentially and in parallel; locally and remotely	🐣
Input and Output Functions in Jina Use Jina's input and output functions	🐣
Use Dashboard to Get Insight of Jina Workflow Monitor workflows and get insights with Jina's dashboard	🐣
From BERT-as-Service to X-as-Service Extract feature vector data using any deep learning representation	🐣
Build a NLP Semantic Search System Search South Park scripts and practice with Flows and Pods	🐣
Build a Flower Image Search System Search images, define your own executors, and run them in Docker	🐣
Video Semantic Search in Scale with Prefetching and Sharding Increase performance using prefetching and sharding	🕊
Revisit "Hello, World!" in a Client-Server Architecture Run a Flow remotely and connect from a local client	🕊
Distribute Your Workflow Remotely Run Jina on remote instances and distribute your workflow	🕊
Extend Jina by Implementing Your Own Executor Implement your own ideas as Jina plugins	🕊
Run Jina Pod via Docker Container Solve complex dependencies easily with Docker containers	🕊
Google's Big Transfer Model in (Poké-)Production Search Pokemon with SOTA visual representation!	🚀
Share Your Extension with the World Share your extensions with engineers around the globe on Jina Hub	🚀

Documentation

The best way to learn Jina in depth is to read our documentation. Documentation is built on every push, merge, and release of the master branch.

Are you a "Doc"-star? Affirmative? Join us! We welcome all kinds of improvements on the documentation.

Documentation for older versions is archived here.

Contributing

We welcome all kinds of contributions from the open-source community, individuals and partners. Without your active involvement, Jina won't be successful.

Community

Slack channel - a communication platform for developers to discuss Jina
Community newsletter - subscribe to the latest updates, releases and event news of Jina
LinkedIn - get to know Jina AI as a company and find job opportunities
- follow us and interact with using hashtag #JinaSearch
Company - know more about our company and how we are fully committed to open-source!

Join Us

Jina is an open-source project. We are hiring full-stack developers, evangelists, and PMs to build the next neural search ecosystem in open source.

Roadmap

GitHub milestones lay out the path to Jina's future improvements.

We are looking for partnerships to build a Open Governance model (e.g. Technical Steering Committee) around Jina, to enable a healthy open-source ecosystem and developer-friendly culture. If you are interested, contact us at hello@jina.ai.

License

Jina is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

darshit7/jina

Contents