If the order of keys is inconsistent during deserialization, OOM will occur.
Closed this issue · 6 comments
linking12 commented
first deserialization, if key order is:
key1
key2
key3
sencond deserialization, if key order is:
key2
key3
key1
when do deserialize multiple times use compactmap , will trigger OOM
vlsi commented
Do you mean serialize-deserialize within the same JVM or across JVMs?
linking12 commented
@vlsi in my case , it is across JVMs;
but i write a simple code to test this scenes, it will reappear
linking12 commented
oom simple code(keys are out of order):
public class CompactRandomOrder {
private static final int ENTRY_CARD = 1000;
private static final int LOOP_COUNT = 100000;
private static final int MAP_SIZE = 50;
private static Random random = new Random(ENTRY_CARD);
public static void main(String[] args) throws InterruptedException {
List<String> keyList = new ArrayList<>();
List<String> valueList = new ArrayList<>();
for (int i = 0; i < ENTRY_CARD; i++) {
keyList.add(i + "#keyKeykeyKeykeyKeykeyKeykeyKeykeyKeykeyKeykeyKeykeyKey" + random.nextInt());
valueList.add(i + "#valueValuevalueValuevalueValuevalueValuevalueValue" + random.nextInt());
}
int idx;
List<Map> listMap = new ArrayList<>(LOOP_COUNT);
for (int i = 0; i < LOOP_COUNT; i++) {
CompactHashMap map = new CompactHashMap();
for (int j = 0; j < MAP_SIZE; j++) {
idx = Math.abs(random.nextInt()) % ENTRY_CARD;
map.put(keyList.get(idx), valueList.get(idx));
}
listMap.add(map);
}
System.out.println("has filled");
Thread.sleep(200000);
}
}
normal code (keys are sequential)
public class CompactOrderly {
private static final int ENTRY_CARD = 1000;
private static final int LOOP_COUNT = 100000;
private static final int MAP_SIZE = 50;
private static Random random = new Random(ENTRY_CARD);
@SneakyThrows
public static void main(String[] args) {
List<String> keyList = new ArrayList<>();
List<String> valueList = new ArrayList<>();
for (int i = 0; i < ENTRY_CARD; i++) {
keyList.add(i + "#keyKeykeyKeykeyKeykeyKeykeyKeykeyKeykeyKeykeyKeykeyKey" + random.nextInt());
valueList.add(i + "#valueValuevalueValuevalueValuevalueValuevalueValue" + random.nextInt());
}
int idx;
List<Map> listMap = new ArrayList<>(LOOP_COUNT);
for (int i = 0; i < LOOP_COUNT; i++) {
CompactHashMap map = new CompactHashMap();
idx = Math.abs(random.nextInt()) % (ENTRY_CARD - MAP_SIZE);
for (int j = idx; j < ENTRY_CARD; j++) {
if (j - idx >= 50) {
break;
}
map.put(keyList.get(j), valueList.get(j));
}
listMap.add(map);
}
System.out.println("has filled");
Thread.sleep(200000);
}
}

vlsi commented
I've published 2.0.1, and it should appear on Central soon.
vlsi commented
Thanks for figuring out the root cause.