Invalid memory allocation

Question

Invalid memory allocation

den-krupsky opened this issue 2 years ago · 18 comments

Hello. I tried to use library and it works incorrectly in my case. Sometimes I get this exception:

java.lang.IllegalArgumentException: Invalid allocation size : -651755510
	at java.base/jdk.internal.foreign.Utils.checkAllocationSizeAndAlign(Utils.java:169)
	at java.base/java.lang.foreign.MemorySegment.ofAddress(MemorySegment.java:892)
	at org.jnetpcap.util.PcapReceiver.lambda$commonArrayHandler$0(PcapReceiver.java:125)
	at org.jnetpcap.Pcap0_4$ProxyPcapHandler.callback(Pcap0_4.java:79)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
	at org.jnetpcap.internal.ForeignDowncall.invokeInt(ForeignDowncall.java:91)
	at org.jnetpcap.Pcap0_4.loop(Pcap0_4.java:725)
	at org.jnetpcap.util.PcapReceiver.commonArrayHandler(PcapReceiver.java:117)
	at org.jnetpcap.Pcap0_4.loop(Pcap0_4.java:738)

Sometimes I get this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000208d2a4fa0c, pid=20428, tid=7964
#
# JRE version: Java(TM) SE Runtime Environment (19.0.2+7) (build 19.0.2+7-44)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (19.0.2+7-44, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
# Problematic frame:
# v  ~StubRoutines::jbyte_disjoint_arraycopy 0x00000208d2a4fa0c

My environment:

Windows 11 x64
JDK 19
Npcap 1.72

My VM options:
-Djava.library.path=C:\Windows\System32 --enable-native-access=ALL-UNNAMED --enable-preview
Code that I tried to run:

public class OfflineCapture {

    public static void main(String[] args) throws PcapException {
        try(Npcap pcap = Npcap.openOffline("C:\\dump.pcap")) {
            pcap.loop(1, OfflineCapture::handle, "user");
        }
    }

    protected static void handle(String msg, PcapHeader header, byte[] packet) {
        System.out.println("Capture: " + header.captureLength() + ". Wire: " + header.wireLength() + ". Data: " + packet.length);
    }

}

P.S. Sometime I had these errors in "live" capturing

Answer 1 · 2023-03-29T19:33:25.000Z

I can present pcap file to you in private channel for testing

Answer 2 · 2023-03-29T20:13:59.000Z

Den, thank you for letting me know.

Please upload your pcap file(s) using the following private upload link.

Answer 3 · 2023-03-29T21:07:39.000Z

BTW, I have just rewritten that part of the capture code, see Pull #27 for details.

I also have a pretty good idea as to what is causing this. I will just make sure that this does not sneak through with the next update very soon. Your pcap files will help with that.

I'm releasing 3 new repo modules and jNetPcap update this week which will provide protocol level support on top of basic jNetPcap API.

Answer 4 · 2023-03-30T10:32:02.000Z

I sent you pcap file. I hope it will help you to find a problem

Den, thank you for letting me know.

Please upload your pcap file(s) using the following private upload link.

Answer 5 · 2023-03-30T13:33:45.000Z

Thanks got it. I will get back to you. The new update will fix this.

Answer 6 · 2023-03-31T18:54:30.000Z

Hi Den,
Can you please try this new build and let me know if it fixes your issue?

jnetpcap-2.0.0+bug28.zip

Also I opened up a new Pull Request for this issue.

Edit: I'm also attaching updated javadocs for this build. This is because of small API changes which differ from the main branch. So if you need it:

jnetpcap-2.0.0+bug28-javadocs.zip

Answer 7 · 2023-04-02T03:09:54.000Z

Hello, thank you for your work. I'm sorry for long feedback
I ran multiple time wih updated jar-library from zip and it still fails:

java.lang.IllegalStateException: invalid length [-28884] from PcapHeaderABI [PCAP_HEADER_PADDED_LE]
	at org.jnetpcap.internal.PcapHeaderABI.validateLength(PcapHeaderABI.java:77)
	at org.jnetpcap.internal.PcapHeaderABI.captureLength(PcapHeaderABI.java:189)
	at org.jnetpcap.PcapHeader.captureLength(PcapHeader.java:99)
	at org.jnetpcap.Pcap0_4.lambda$loop$3(Pcap0_4.java:740)
	at org.jnetpcap.internal.StandardPcapDispatcher.nativeCallback(StandardPcapDispatcher.java:163)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
	at org.jnetpcap.internal.ForeignDowncall.invokeInt(ForeignDowncall.java:91)
	at org.jnetpcap.internal.StandardPcapDispatcher.loopNative(StandardPcapDispatcher.java:136)
	at org.jnetpcap.Pcap0_4.loop(Pcap0_4.java:736)

java.lang.IllegalStateException: invalid length [36184] from PcapHeaderABI [PCAP_HEADER_PADDED_LE]
	at org.jnetpcap.internal.PcapHeaderABI.validateLength(PcapHeaderABI.java:77)
	at org.jnetpcap.internal.PcapHeaderABI.wireLength(PcapHeaderABI.java:258)
	at org.jnetpcap.PcapHeader.wireLength(PcapHeader.java:132)
	at ...OfflineCapture.handle(OfflineCapture.java:16)
	at org.jnetpcap.Pcap0_4.lambda$loop$3(Pcap0_4.java:746)
	at org.jnetpcap.internal.StandardPcapDispatcher.nativeCallback(StandardPcapDispatcher.java:163)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
	at org.jnetpcap.internal.ForeignDowncall.invokeInt(ForeignDowncall.java:91)
	at org.jnetpcap.internal.StandardPcapDispatcher.loopNative(StandardPcapDispatcher.java:136)
	at org.jnetpcap.Pcap0_4.loop(Pcap0_4.java:736)

java.lang.IllegalStateException: invalid length [49045] from PcapHeaderABI [PCAP_HEADER_PADDED_LE]
	at org.jnetpcap.internal.PcapHeaderABI.validateLength(PcapHeaderABI.java:77)
	at org.jnetpcap.internal.PcapHeaderABI.wireLength(PcapHeaderABI.java:258)
	at org.jnetpcap.PcapHeader.wireLength(PcapHeader.java:132)
	at ...OfflineCapture.handle(OfflineCapture.java:16)
	at org.jnetpcap.Pcap0_4.lambda$loop$3(Pcap0_4.java:746)
	at org.jnetpcap.internal.StandardPcapDispatcher.nativeCallback(StandardPcapDispatcher.java:163)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
	at org.jnetpcap.internal.ForeignDowncall.invokeInt(ForeignDowncall.java:91)
	at org.jnetpcap.internal.StandardPcapDispatcher.loopNative(StandardPcapDispatcher.java:136)
	at org.jnetpcap.Pcap0_4.loop(Pcap0_4.java:736)

Edit: Once I got successful result

Answer 8 · 2023-04-02T11:37:11.000Z

Ok, the PCAP_HEADER_PADDED_LE is the correct ABI for AMD64 architecture. There is only one other place where the bug can exist, to do with byte swapping when reading offline files using pcap. I will have another update for you today.

Also notice no more VM crashes :) Your feedback is helping a lot.

Answer 9 · 2023-04-02T21:31:28.000Z

Yeah, VM no longer crashed. I will wait updates

Answer 10 · 2023-04-02T21:35:01.000Z

Almost done. Found and replicated the problem exactly.

Answer 11 · 2023-04-03T03:10:37.000Z

Try this update.

jnetpcap-2.0.0+bug28-2.zip
jnetpcap-2.0.0+bug28-2-javadocs.zip

Let me know.

Answer 12 · 2023-04-04T07:55:47.000Z

I did several code running. It seems 2.0.0+bug28-2 is working correctly. There weren't VM crashes or exceptions
Great work!

Answer 13 · 2023-04-04T08:09:42.000Z

Remains to fix openLive sessions))

Exception in thread "main" org.jnetpcap.PcapHeaderException$OutOfRangeException: invalid length [671242056] from PcapHeaderABI [PCAP_HEADER_PADDED_LE]
wireLength: PCAP_HEADER_COMPACT_LE(76) PCAP_HEADER_PADDED_LE(20) CMP_BE(+12=0x4C000000) PAD_BE(+20=0x14000000) Buf[p=0 l=24 c=24]
	at org.jnetpcap.internal.PcapHeaderABI.validateLength(PcapHeaderABI.java:190)
	at org.jnetpcap.internal.PcapHeaderABI.wireLength(PcapHeaderABI.java:366)
	at org.jnetpcap.PcapHeader.wireLength(PcapHeader.java:152)
	at ...Clazz.lambda$main$0(Clazz.java:42)
	at org.jnetpcap.Pcap0_4.lambda$loop$3(Pcap0_4.java:745)
	at org.jnetpcap.internal.StandardPcapDispatcher.nativeCallback(StandardPcapDispatcher.java:186)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
	at org.jnetpcap.internal.ForeignDowncall.invokeInt(ForeignDowncall.java:91)
	at org.jnetpcap.internal.StandardPcapDispatcher.loopRaw(StandardPcapDispatcher.java:165)
	at org.jnetpcap.internal.StandardPcapDispatcher.loopNative(StandardPcapDispatcher.java:152)
	at org.jnetpcap.Pcap0_4.loop(Pcap0_4.java:735)
	at ...Clazz.main(Clazz.java:42)

Exception in thread "main" org.jnetpcap.PcapHeaderException$OutOfRangeException: invalid length [-1378656552] from PcapHeaderABI [PCAP_HEADER_PADDED_LE]
wireLength: PCAP_HEADER_COMPACT_LE(60) PCAP_HEADER_PADDED_LE(20) CMP_BE(+12=0x3C000000) PAD_BE(+20=0x14000000) Buf[p=0 l=24 c=24]
	at org.jnetpcap.internal.PcapHeaderABI.validateLength(PcapHeaderABI.java:190)
	at org.jnetpcap.internal.PcapHeaderABI.wireLength(PcapHeaderABI.java:366)
	at org.jnetpcap.PcapHeader.wireLength(PcapHeader.java:152)
	at ...Clazz.lambda$main$0(Clazz.java:42)
	at org.jnetpcap.Pcap0_4.lambda$loop$3(Pcap0_4.java:745)
	at org.jnetpcap.internal.StandardPcapDispatcher.nativeCallback(StandardPcapDispatcher.java:186)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
	at org.jnetpcap.internal.ForeignDowncall.invokeInt(ForeignDowncall.java:91)
	at org.jnetpcap.internal.StandardPcapDispatcher.loopRaw(StandardPcapDispatcher.java:165)
	at org.jnetpcap.internal.StandardPcapDispatcher.loopNative(StandardPcapDispatcher.java:152)
	at org.jnetpcap.Pcap0_4.loop(Pcap0_4.java:735)
	at ...Clazz.main(Clazz.java:42)

I run next code:

        try (Pcap pcap = Npcap.create(device)) {
            pcap.activate();
            pcap.loop(0, (PcapHandler.OfArray<String>) (msg, header, packet) -> System.out.println("Capture: " + header.captureLength() + ". Wire: " + header.wireLength() + ". Data: " + packet.length), "user");
        } catch (PcapException ex) {
            System.err.println("Failed:" + ex.getMessage());
            throw new RuntimeException(ex);
        }

Answer 14 · 2023-04-04T12:27:19.000Z

Try this as a work around with bug28-2:

// Set cmd line property: -Dorg.jnetpcap.abi=PCAP_HEADER_COMPACT_LE or
System.setProperty("org.jnetpcap.abi", "PCAP_HEADER_COMPACT_LE");

try (Pcap pcap = Npcap.create(device)) {
            pcap.activate();
            pcap.loop(0, (PcapHandler.OfArray<String>) (msg, header, packet) -> System.out.println("Capture: " + header.captureLength() + ". Wire: " + header.wireLength() + ". Data: " + packet.length), "user");
        } catch (PcapException ex) {
            System.err.println("Failed:" + ex.getMessage());
            throw new RuntimeException(ex);
        }
}

Answer 15 · 2023-04-04T12:51:24.000Z

Also this was an easy fix and here the 3rd bug package.
jnetpcap-2.0.0-bug28-3.zip

The main javadocs have been updated so you can view them online now.

As soon as this is squared away, and I properly run all unit tests on my Win machine and I will make the next release. Hopefully today or tomrrow.

I'm staging a lot of modules in the background and they will very soon be made public and released which will greatly expand the basic Pcap functionality. Lots of stuff in the queue to be released.

Screenshot:

Answer 16 · 2023-04-04T13:06:10.000Z

The jNetPcap Pro module is a bit more fun to use than the basic jnetpcap module.

Here is one of the examples:

private static final String PCAP_FILE = "pcaps/LAN-1.pcapng";

private final static Ip4 ip4 = new Ip4();
private final static Tcp tcp = new Tcp();
private final static Ip4OptRouter router = new Ip4OptRouter();

/**
 * Pcap packet capture example using jNetPcap.
 * 
 * @param args
 * @throws PcapException
 * @throws IOException
 */
public static void main(String[] args) throws PcapException {

	/* Make sure we have a compatible Pcap runtime installed */
	Pcap.checkPcapVersion(Pcap.VERSION);

	/*
	 * Automatically close Pcap resource when done and checks the client and
	 * installed runtime API versions to ensure they are compatible.
	 */
	try (PcapPro pcap = PcapPro.openOffline(PCAP_FILE)) {

		/* Enable Ip Fragment tracking and reassembly */
		pcap
				.enableIpfTracking(true)
				.enableIpfReassembly(true);

		/* Compile packet filter to capture TCP packets only */
		BpFilter filter = pcap.compile("tcp", true);

		/* Set our packet filter and start the capture */
		pcap.setFilter(filter);

		/* 10 is packet count, nextPacket is packet handler and user obj at the end */
		pcap.dispatch(1, PcapExample2::nextPacket, "Hello World");
	}
}

private static void nextPacket(String message, Packet packet) {

	// If present, printout ip4 header
	if (packet.hasHeader(ip4))
		System.out.println(ip4);

	// If present, printout ip4 router alert option header
	if (ip4.isBound() && ip4.hasExtension(router))
		System.out.println(router);

	// If present, printout ip4 header
	if (packet.hasHeader(ip4) && ip4.hasExtension(router))
		System.out.println(router);

	// If present, printout tcp header
	if (packet.hasHeader(tcp))
		System.out.println(tcp);

}

Note PcapPro extends Pcap.

Answer 17 · 2023-04-04T13:21:44.000Z

Try this as a work around with bug28-2:

// Set cmd line property: -Dorg.jnetpcap.abi=PCAP_HEADER_COMPACT_LE or
System.setProperty("org.jnetpcap.abi", "PCAP_HEADER_COMPACT_LE");

try (Pcap pcap = Npcap.create(device)) {
            pcap.activate();
            pcap.loop(0, (PcapHandler.OfArray<String>) (msg, header, packet) -> System.out.println("Capture: " + header.captureLength() + ". Wire: " + header.wireLength() + ". Data: " + packet.length), "user");
        } catch (PcapException ex) {
            System.err.println("Failed:" + ex.getMessage());
            throw new RuntimeException(ex);
        }
}

It helped

Answer 18 · 2023-04-06T19:39:15.000Z

You can download the latest release for a clean install and API with this bug fix.

All the other modules have been checked in and are undergoing cleanup and review before becoming "public" repos. Almost complete with the process.

Thanks a lot for all your feedback and testing.