JVM heap size isn't adjusted based on memory constraints

[ebroder]

I ran into issues with tasks consistently getting OOM killed when I created my ECS cluster using c3.8xlarge instances.

As I understand it, the default JVM behavior is to set the max heap size to 25% of the total system memory. However, when run under a docker memory constraint, it has no ability to detect that less memory than usual is available. On a c3.8xlarge instance, for example, this results in a max heap size of 15G, in spite of the container capping memory usage at 512M. With a max heap that large, the JVM is pretty lazy about GCs, so it runs out of memory within a minute or two.

This should be pretty easy to fix by just passing an -Xmx option explicitly (say, have start_connector.sh look for a JAVA_OPTS envvar?)

[dymaws]

Hi,

As a part of the improvement process for cross-region replication library, we have refactored the library completely and released an updated version. As a result, the previous version involving multiple components (ECS included) has been deprecated and I suggest you use the new libraries as command-line tools instead. The code has also been refactored down to the fundamental components only, stripping away other complicated and unnecessary dependencies. This should make the library much easier to understand, and a lot less cumbersome to manage as a client-side solution.

Thank you!