DownloadTask should download all referenced schemas

[ferozed]

Maybe I am understanding this wrong... but shouldnt the DownloadSchema task have an option to download all referenced schemas of a subject?

For eg, assume the following schema registered in the registry.

$ curl -s http://localhost:8081/subjects/topic-three-value/versions/1 | jq
{
  "subject": "topic-three-value",
  "version": 1,
  "id": 3,
  "references": [
    {
      "name": "com.zillow.streamz.data.schema_one",
      "subject": "topic-one-value",
      "version": 1
    }
  ],
  "schema": "\"com.zillow.streamz.data.schema_one\""
}

If I download this schema, and try to generate a java POJO from it, it wont work since it cant find the com.zillow.streamz.data.schema_one type. However if I download the schema that has that type and put it in the same directory, then POJO generation succeeds.

Given that downloadSchema task is used as a precursor to using GenerateAvroJavaTask from gradle-avro-plugin, shouldnt the downloadSchemaTask have an option to download the dependencies specified in the schema references?

[ImFlog]

Hello @ferozed,
Thank you for using this plugin :)
This seems to be a valid use case, I think we can add a flag to the download task to download the referenced schemas.
I will work on this task early january after my holidays.
Of course if you want to submit a PR for this I can help !

[ferozed]

Hi! Yeah, I would love to submit a PR. Let me take a stab at it.

[ferozed]

Here is the MR: #156

[ImFlog]

I have a small issue with the plugin publishing (an error on Gradle side it seems) but I didn't forget to release this.

[ImFlog]

This is released in https://github.com/ImFlog/schema-registry-plugin/releases/tag/v1.13.0.
During my final tests on the example directory, I think that there is currently a limitation on the feature because we are not recursively downloading the references.
Apart from that I can confirm that it works.

I will fill another issue to improve this (if you want to iterate on doing a PR help yourself 🙂 ).

Thank you again for this 🙇

[ferozed]

Thank you very much, @ImFlog ! Really appreciate your help.

[ferozed]

During my final tests on the example directory, I think that there is currently a limitation on the feature because we are not recursively downloading the references.

Ah.. hadnt thought of that...