neo4j-contrib/neo4j-streams

Local setup of Kafka to Neo4j by using Docker

maheshtati441 opened this issue · 4 comments

maheshtati441 commented

Guidelines

Please note that GitHub issues are only meant for bug reports/feature requests. If you have questions on how to use Streams, please ask on StackOverflow instead of creating an issue here.

Expected Behavior (Mandatory)

When an event is generated from Kafka, the event should be updated in Neo4j by the cypher query defined in the Neo4j Sink Connector

Actual Behavior (Mandatory)

I couldn't see the connect clusters and the tab to create schema and unable to identify the issue.

How to Reproduce the Problem

  1. Run the docker-compose.yml file and the required images will be installed in the Docker and browse for localhost:9021
  2. By using the Maven clean install, able to get the zip file and placed under the Plugins folder where the docker-compose.yml is there.
  3. Check for the connect-plugins and able to see them through Postman but not in the connect under the control center (localhost:9021)
  4. Run the curl command to create the sink connector and check in the control center and it will not be found eventhough it got created. But it can be found by querying through Postman.
  5. Unable to find to create the schema under the Topic created(my-topic)

Simple Dataset (where it's possibile)

//Insert here a set of Cypher statements that helps us to reproduce the problem

Steps (Mandatory)

  1. Started the docker
    image

2.Showing the docker containers in which Schema-Registry is in exited state after the docker run
image

  1. In the Confluent (localhost:9021), under the connect there are no connectors found
    image

  2. If I request for connect-plugins through Postman, I can see the kafka connectors
    [
    {
    "class": "org.apache.kafka.connect.mirror.MirrorCheckpointConnector",
    "type": "source",
    "version": "1"
    },
    {
    "class": "org.apache.kafka.connect.mirror.MirrorHeartbeatConnector",
    "type": "source",
    "version": "1"
    },
    {
    "class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
    "type": "source",
    "version": "1"
    },
    {
    "class": "streams.kafka.connect.sink.Neo4jSinkConnector",
    "type": "sink",
    "version": "2.0.2"
    },
    {
    "class": "streams.kafka.connect.source.Neo4jSourceConnector",
    "type": "source",
    "version": "2.0.2"
    }
    ]

  3. By using the curl command, I'm able to post and able to see the sink connectors through Postman
    image
    By using (localhost:8083/connectors/), I can see the sink connector getting created.
    image

  4. Under the Schema section of the Topic which is created, I don't see any option to create a schema.
    image

  5. If I query for the subjects and Topics I'm not getting any response.
    image

image

Screenshots (where it's possibile)

image
image
image
image
image
image
image
image

Specifications (Mandatory)

Currently used versions

Versions

  • OS: Windows 10
  • Neo4j: neo4j:4.3-enterprise
  • Neo4j-Streams: 4.1
OneCricketeer commented

Firstly, I recommend not using Control Center. It will not help you debug the problem(s).

Your schema registry (port 8081) is simply down, as shown by your Docker Desktop image (and it's not clear you're using Avro data, anyway for the Connector). You should use docker-compose logs schema-registry to see why that could be (perhaps your machine doesn't have enough memory to run everything, and removing Control Center would be a step towards fixing that)

Also, port 9092 is not an HTTP address, so using POSTMan on it wont work.

mroiter-larus commented

Hi @maheshtati441,

this not seems a Neo4j Streams related issue. I suppose there is a misconfiguration in your docker-compose.yml.
If you can share the docker-compose file we can try to help you.

maheshtati441 commented
ali-ince commented

He @maheshtati441, is this still a problem and if so, would you please post the docker compose file into the github issue so that we can access it?