refactor: upgrade examples to release 2.0

Question

refactor: upgrade examples to release 2.0

jakobkruse1 opened this issue 4 years ago · 12 comments

jakobkruse1 commented 4 years ago

We want to know how examples will run with Jina 2.0.
The following tasks should be finished for each of them:

Use Jina 2.0 interface
C&P Executors from Hub and adopt to new Executor base class
Adopt Readme according to the Example Template #518
Add comments to YAML files to make them better understandable #445

Examples to be updated:

wikipedia-sentences #619
pokedex #626
multi modal PDF #629
multires-lyrics #648 Sebastian, Jakob
cross-model #631
query-while-indexing (only once dump-load approach has been agreed upon) Kelton: #665

Other tasks:

Move all other examples to legacy and update example README.
Create tutorials

We might continue with more afterwards.

Answer 1 · 2021-05-19T15:09:30.000Z

As discussed in meeting today, I'd recommend merging these uni-modal (mono-modal?) examples into pokedex, and possibly renaming the example to "image/audio/video search" or similar:

audio search
tumblr gif search

Also, if we can get a decent metadata-heavy text dataset with good chunkability and searchability, we could look at merging the text-based examples (wikipedia-*, multires-lyrics) in future

Answer 2 · 2021-05-19T15:10:37.000Z

Might also be good to rename 'wikipedia-sentences' to plain old 'text-search'. Makes it more google-able, and if/when we change the dataset we can retain the old URL

Answer 3 · 2021-05-20T07:28:05.000Z

Instead of working on cross-modal, I suggest starting with cross-modal-pdf, which is using smaller model and less data. Plus, it is equipped with proper tests.

Answer 4 · 2021-05-20T07:30:57.000Z

I share the same view with @alexcg1, we need to rename it just example-audio-search, example-video-search, example-text-search. As we are moving to 2.0, I suggest pushing the proposal we had before that we should split the examples into separated repos.

Answer 5 · 2021-05-20T08:17:28.000Z

Who can handle that @nan-wang ? I might have some capacity today to get started

Answer 6 · 2021-05-25T12:50:31.000Z

Do we have a list somewhere of

what we're updating?
what we're merging?
what we're deleting?

If not, I'll create a table in this thread and we can discuss

@nan-wang

Answer 7 · 2021-05-25T12:56:09.000Z

I share the same view with @alexcg1, we need to rename it just example-audio-search, example-video-search, example-text-search. As we are moving to 2.0, I suggest pushing the proposal we had before that we should split the examples into separated repos.

I suggest we wait until all other example work is finished before doing this. If we change names mid-stream people are going to have a nightmare properly merging

Answer 8 · 2021-05-25T14:11:23.000Z

I will take on the multi-modal-pdf @nan-wang

Answer 9 · 2021-06-02T13:10:06.000Z

@alexcg1 @nan-wang @theUnkownName updated the ticket with some tasks from the meeting

Answer 10 · 2021-07-30T13:57:34.000Z

@nan-wang @CatStark @florian-hoenicke Some of these examples have been upgraded on an older version of jina2.0. We should use hub executors in all examples and update all of them to the latest jina version. Do you agree?

Answer 11 · 2021-07-30T15:08:15.000Z

I really like the idea of using Hub Executors - many folks learn Jina by adapting the examples, so if Hub is baked in then they'll keep using Hub stuff (except for when they need to build their own Executors, which they can then push to Hub 😄 )

Answer 12 · 2021-08-03T07:50:25.000Z

We should use hub executors in all examples and update all of them to the latest jina version. Do you agree?

Sure, let's do it.