Not able to start Geoserver because of Memory leak

Question

Not able to start Geoserver because of Memory leak

krishnaglodha opened this issue a year ago · 34 comments

krishnaglodha commented a year ago

What is the bug or the crash?

I'm trying to run docker-compose.yml but getting following error

Steps to reproduce the issue

I created docker-compose.yml file with 2 Geoserver images

Versions

2.20.6

Additional context

No response

krishnaglodha commented a year ago

on it

Answer 1 · 2023-10-10T19:12:51.000Z

How did you build the image. Using the latest repo?

Answer 2 · 2023-10-14T13:57:23.000Z

@krishnaglodha I tested this with a local build and the error I got is

kartozageoserver-geoserver-prod-1  | org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'pagedUniquePPIO' defined in URL [jar:file:/usr/local/tomcat/webapps/geoserver/WEB-INF/lib/gs-wps-core-2.20.6.jar!/applicationContext.xml]: Instantiation of bean failed; nested exception is java.lang.NoClassDefFoundError: org/json/simple/parser/JSONParser
kartozageoserver-geoserver-prod-1  | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1334)
kartozageoserver-geoserver-prod-1  | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1232)
kartozageoserver-geoserver-prod-1  | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
kartozageoserver-geoserver-prod-1  | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
kartozageoserver-geoserver-prod-1  | 	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
kartozageoserver-geoserver-prod-1  | 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
kartozageoserver-geoserver-prod-1  | 	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
kartozageoserver-geoserver-prod-1  | 	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
kartozageoserver-geoserver-prod-1  | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)

This can be easily solved by running:

-v  ./foo.sh:/docker-entrypoint-geoserver.d/foo.sh

where the contents are

#!/usr/bin/env bash

wget http://www.java2s.com/Code/JarDownload/json-simple/json-simple-1.1.1.jar.zip

unzip json-simple-1.1.1.jar.zip  

mv json-simple-1.1.1.jar /usr/local/tomcat/webapps/geoserver/WEB-INF/lib/

Answer 3 · 2023-10-23T04:06:52.000Z

How did you build the image. Using the latest repo?

I'm using 2.20

Answer 4 · 2023-10-23T06:41:11.000Z

did you try the solution I gave you?

Answer 5 · 2023-10-23T07:16:59.000Z

I tried, but still no luck.

I believe I'm already having json-simple jar

Answer 6 · 2023-10-23T07:19:49.000Z

I will take a look, Last time I build the 2.20.6 locally and that fix worked. Are you pulling from dockerhub.

Answer 7 · 2023-10-26T12:45:07.000Z

HI @NyakudyaA I'm trying with 2.20.7 https://github.com/krishnaglodha/docker-geoserver/tree/cluster/2.20.7 , the goal is to put it in clustering

Answer 8 · 2023-10-26T12:54:21.000Z

@krishnaglodha I am able to start that container with that specific tag without any issues. I suggest if you need to test if clustering is working you can update the action in https://github.com/kartoza/docker-geoserver/blob/develop/.github/workflows/deploy-image.yaml#L26 and https://github.com/kartoza/docker-geoserver/blob/develop/.github/workflows/build-latest.yaml#L20 to add that specific tag. This will test the clustering is working and other tests

Answer 9 · 2023-10-26T15:47:40.000Z

Are you suggesting to build it from scratch

Answer 10 · 2023-10-26T15:50:01.000Z

I pulled from dockerhub and it worked. I don't know why you keep getting errors

Answer 11 · 2023-10-26T15:51:22.000Z

Will it be possible for us to have a quick meet ? me and @francbartoli are working on clustering thing, but are facing some issues for last 1 month.

Answer 12 · 2023-10-26T15:53:51.000Z

HI @NyakudyaA I'm trying with 2.20.7 https://github.com/krishnaglodha/docker-geoserver/tree/cluster/2.20.7 , the goal is to put it in clustering

@francbartoli can you check this setup and confirm ?

Answer 13 · 2023-10-26T15:54:17.000Z

I can give you pointers, no problem with that but hopefully, it will be less than five minutes

Answer 14 · 2023-10-26T15:54:55.000Z

Have you tested with the https://github.com/krishnaglodha/docker-geoserver/tree/cluster/2.20.7 ?

Answer 15 · 2023-10-26T15:57:10.000Z

Do you want me to run https://github.com/krishnaglodha/docker-geoserver/blob/cluster/2.20.7/clustering/docker-compose.yml with the 2.20.7 image?

Answer 16 · 2023-10-26T15:58:21.000Z

Yes please, it'd be great help :)

Our setup includes

Db
active MQ broker
Master GS
Node GS
Geofence DB
Load balancing

Answer 17 · 2023-10-26T16:02:27.000Z

But in your branch are those things setup. It looks like there is only 1,3,4 up

Answer 18 · 2023-10-26T16:04:32.000Z

can you check now ?

Answer 19 · 2023-10-26T16:07:05.000Z

it's there now, will test and give feedback

Answer 20 · 2023-10-26T16:08:27.000Z

Thanks a lot ! I'm available for anything needed

Answer 21 · 2023-10-28T08:33:44.000Z

Hi, @NyakudyaA You had any luck ?

Answer 22 · 2023-10-28T13:46:43.000Z

Will try it this weekend, will give feedback by COB tommorow

Answer 23 · 2023-10-28T14:31:02.000Z

Thanks a lot, Really appreciate it Thank you, Krishna G. Lodha Personal email : ***@***.*** Office email : ***@***.***

…

On 28 Oct 2023 at 19:16 +0530, kartoza/docker-geoserver ***@***.***>, wrote: Will try it this weekend, will give feedback by COB tommorow

Answer 24 · 2023-10-28T14:51:15.000Z

@krishnaglodha Can you give this branch a test run? https://github.com/kartoza/docker-geoserver/tree/external

Answer 25 · 2023-10-30T04:15:02.000Z

@NyakudyaA just to be clear, you mean I should try https://github.com/kartoza/docker-geoserver/tree/external/clustering this setup ?

Answer 26 · 2023-10-30T05:11:11.000Z

@NyakudyaA just to be clear, you mean I should try https://github.com/kartoza/docker-geoserver/tree/external/clustering this setup ?

Yes, use docker-compose-external.yml

Answer 27 · 2023-10-30T11:16:21.000Z

my docker logs is stuck in infinite loop

Answer 28 · 2023-10-30T12:08:50.000Z

Still getting memory leak issue

version: "3.9"

# Common template for ActiveMQ services below
x-common-activemq:
  &default-common-activemq
  image: kartoza/activemq-docker:2.23.2
  build: activemq-docker
  restart: on-failure

services:

  db:
    image: kartoza/postgis:16-3.4
    volumes:
      - geo-db-data:/var/lib/postgresql
    ports:
      - "5432"
    environment:
      - POSTGRES_DB=gis,data,sample
      - POSTGRES_USER=docker
      - POSTGRES_PASS=docker
      - ALLOW_IP_RANGE=0.0.0.0/0
      - FORCE_SSL=TRUE
    restart: on-failure
    healthcheck:
      test: "PGPASSWORD=docker pg_isready -h 127.0.0.1 -U docker -d gis"

  broker1:
    << : *default-common-activemq
    environment:
      - JAVA_HOME=/opt/java/openjdk
      - HOST=db
      - POSTGRES_PORT=5432
      - POSTGRES_DB=gis
      - POSTGRES_USER=docker
      - POSTGRES_PASS=docker
      - ACTIVEMQ_SERVER_URI=tcp://broker1:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600&jms.useAsyncSend=false&transport.daemon=true
      - ACTIVEMQ_SERVER_DISCOVERYURI=multicast://default
      - SSL_MODE=ALLOW
    depends_on:
      db:
        condition: service_healthy
    healthcheck:
      test: netstat -ltn | grep -c ":61616"
      interval: 30s
      timeout: 10s
      retries: 10

  master:
    image: kartoza/geoserver:2.23.2
    ports:
      - "8081:8080"
    environment:
      - CLUSTERING=true
      - CLUSTER_DURABILITY=false
      - DB_BACKEND=POSTGRES
      - HOST=db
      - POSTGRES_PORT=5432
      - POSTGRES_DB=gis
      - POSTGRES_USER=docker
      - POSTGRES_PASS=docker
      - SSL_MODE=ALLOW
      - BROKER_URL==tcp://broker1:61616
      - READONLY=disabled
      - TOGGLE_MASTER=true
      - TOGGLE_SLAVE=false
      - EMBEDDED_BROKER=disabled
      - RUN_AS_ROOT=TRUE
      - GEOSERVER_ADMIN_USER=admin
      - GEOSERVER_ADMIN_PASSWORD=myawesomegeoserver
      - STABLE_EXTENSIONS=css-plugin,importer-plugin,ogr-wfs-plugin,ogr-wps-plugin,wps-cluster-hazelcast-plugin,wps-download-plugin
      - COMMUNITY_EXTENSIONS=activeMQ-broker-plugin,gdal-wcs-plugin,gdal-wps-plugin,geopkg-plugin,hz-cluster-plugin,jms-cluster-plugin
    volumes:
      - geoserver-cluster-data:/opt/geoserver/data_dir


  node:
    image: kartoza/geoserver:2.23.2
    ports:
      - "8082:8080"
    environment:
      - CLUSTERING=true
      - CLUSTER_DURABILITY=false
      - DB_BACKEND=POSTGRES
      - HOST=db
      - POSTGRES_PORT=5432
      - POSTGRES_DB=gis
      - POSTGRES_USER=docker
      - POSTGRES_PASS=docker
      - SSL_MODE=ALLOW
      - BROKER_URL=tcp://broker1:61616
      - READONLY=enabled
      - TOGGLE_MASTER=false
      - TOGGLE_SLAVE=true
      - EMBEDDED_BROKER=disabled
      - RUN_AS_ROOT=TRUE
      - GEOSERVER_ADMIN_USER=admin
      - GEOSERVER_ADMIN_PASSWORD=myawesomegeoserver
      - STABLE_EXTENSIONS=css-plugin,importer-plugin,ogr-wfs-plugin,ogr-wps-plugin,wps-cluster-hazelcast-plugin,wps-download-plugin
      - COMMUNITY_EXTENSIONS=activeMQ-broker-plugin,gdal-wcs-plugin,gdal-wps-plugin,geopkg-plugin,hz-cluster-plugin,jms-cluster-plugin
    volumes:
      - geoserver-cluster-data:/opt/geoserver/data_dir

volumes:
   geoserver-cluster-data:
   geo-db-data:

I made some changes to script

Answer 29 · 2023-10-30T12:18:57.000Z

@krishnaglodha This works on my side. Are you starting with a clean volume? If I get a chance I will add this to github workflows to test the implementation as a scenario test

Answer 30 · 2023-10-30T15:38:09.000Z

Is there any way to get commercial support ?

Answer 31 · 2023-10-31T02:32:12.000Z

@krishnaglodha You can look at https://kartoza.com/product/support

Answer 32 · 2023-11-01T10:22:14.000Z

Cluster config works, but while creating style, it gives following error

I was able to replicate Workspace, layer, stores , but only facing issue with styles

Answer 33 · 2023-11-01T10:48:51.000Z

@krishnaglodha The download in https://github.com/kartoza/docker-geoserver/blob/develop/scripts/start.sh#L227-L230 seems to be inconsistent. Sometimes it downloads and sometimes it doesn't. I suggest you manually download those files and mount them to

-v ./setup-jars:/settings
-v ./missing-jars.sh:/docker-entrypoint-geoserver.d/missing-jars.sh

And then you just have a logic in missing-jars.sh to copy the .jars to /usr/local/tomcat/webapps/geoserver/WEB_INF/lib/