Netflix/mantis

How to build or install PushRequestEventSourceJob into development environment

jpittis opened this issue · 0 comments

I've been trying my best to run the PushRequestEventSourceJob in my Mantis development environment. No luck so far. What's the recommended way to accomplish this? Any help would be greatly appreciated. I believe the documentation on this is either out of date or incomplete.

What I've tried so far

  1. Uploading via the UI (CORS issue)
  2. Adjusting agent build to include the jobs by default (gradle dependency issue)
  3. Trying to hack in the artifacts with a docker file (idk, at this point I'm grasping at straws)

Uploading via the UI

I've attempting building the artifacts like so:

./gradlew mantis-source-jobs:mantis-source-job-publish:mantisZipArtifact

But when I drag and drop them into the shared UI, I get CORS errors. I tried installing the https://github.com/Netflix/mantis-ui locally, but it turns out it's a totally different UI vs the one published at https://netflix.github.io/mantis-ui, and it's pre-populated with a bunch of mock data rather than letting me point it at my development Mantis API server.

Adjusting agent build to include the jobs by default

Afaik the SineFunction job is installed via the agent's build.gradle. So I tried adding source-job-publish there as well (and associated dependencies):

def mantisSourcePublishJob = project(":mantis-source-jobs:mantis-source-job-publish")
def mantisSourcePublishJobBuildDir = mantisSourcePublishJob.buildDir
def mantisSourcePublishJobMantisZipArtifact = mantisSourcePublishJob.tasks.named("mantisZipArtifact")

This gives an unexpected error that the task doesn't exist:

A problem occurred evaluating project ':mantis-server:mantis-server-agent'.
> Task with name 'mantisZipArtifact' not found in project ':mantis-source-jobs:mantis-source-job-publish'.

Even though when I call the task directly, it definitely does exist... some kind of dependency ordering issue maybe:

./gradlew :mantis-source-jobs:mantis-source-job-publish:mantisZipArtifact

Trying to hack in the artifacts with a docker file

The agent dockerfile I've come up with looks something like this:

from azul/zulu-openjdk:8-latest as builder
workdir mantis
copy ./mantis .
run ./gradlew mantis-source-jobs:mantis-source-job-publish:mantisZipArtifact

from netflixoss/mantisagent:latest
workdir /apps/mantis/mantis-server-agent
copy --from=builder ./mantis/mantis-source-jobs/mantis-source-job-publish/build/distributions/* ./mantis-artifacts/storage/

The master dockerfile looks something like this (where PushRequestEventSourceJob is essentially a copy of SharedMrePublishEventSource):

from netflixoss/mantiscontrolplaneserver:latest
workdir /apps/mantis/mantis-control-plane-server
copy ./PushRequestEventSourceJob ./job-clusters/.

I end up getting weird service loading errors when I kick-off the job, demonstrating I didn't install it correctly:

2024-01-07 20:41:41 ERROR TaskExecutor:417 - Failed to submit task
java.util.NoSuchElementException
	at java.base/java.util.ServiceLoader$2.next(ServiceLoader.java:1318)
	at java.base/java.util.ServiceLoader$2.next(ServiceLoader.java:1306)
	at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403)
	at io.mantisrx.server.agent.SingleTaskOnlyFactory.getRuntimeTaskInstance(SingleTaskOnlyFactory.java:34)
	at io.mantisrx.server.agent.TaskExecutor.prepareTask(TaskExecutor.java:405)
	at io.mantisrx.server.agent.TaskExecutor.lambda$submitTask$7(TaskExecutor.java:382)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
 	at java.base/java.lang.Thread.run(Thread.java:840)
2024-01-07 20:41:41 INFO  TaskStatusUpdateHandlerImpl:47 - onStatusUpdate for status: {"jobId":"PushRequestEventSourceJob-1","stageNum":1,"workerIndex":0,"workerNumber":1,"type":"INFO","message":"stage 1 worker index=0 number=1 failed during initialization","state":"Failed","hostname":null,"timestamp":1704660101343,"reason":"Normal","payloads":[]}
2024-01-07 20:41:41 ERROR AgentV2Main:65 - Task null failed