SuffolkLITLab/EfileProxyServer

Fully parallelize code updater

BryceStevenWilley opened this issue · 0 comments

There are 3 stages to downloading codes for courts: getting the URLs with getPolicy, downloading the codes from their URL, and putting the data from the XML into a postgres db.

Steps 1 and 2 can be parallelized, and step 3 technically can, but we get errors thrown in the JAXB unmarshaller, specifically because the default thread factory doesn't use the correct class path (?) to load the class definitions, which JAX uses to create instances of your object from XML.

For other resources, see:

Exception in thread "main" java.lang.NullPointerException
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
	at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:564)
	at java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:591)
	at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:689)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:765)
	at edu.suffolk.litlab.efspserver.codes.CodeUpdater.streamDownload(CodeUpdater.java:281)
	at edu.suffolk.litlab.efspserver.codes.CodeUpdater.downloadSpeedTest(CodeUpdater.java:558)
	at edu.suffolk.litlab.efspserver.codes.CodeUpdater.executeCommand(CodeUpdater.java:582)
	at edu.suffolk.litlab.efspserver.codes.CodeUpdater.main(CodeUpdater.java:614)
Caused by: java.lang.NullPointerException: Cannot read field "expectText" because the return value of "com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallingContext$State.access$600(com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallingContext$State)" is null
	at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallingContext.expectText(UnmarshallingContext.java:609)
	at com.sun.xml.bind.v2.runtime.unmarshaller.StAXStreamConnector.handleCharacters(StAXStreamConnector.java:296)
	at com.sun.xml.bind.v2.runtime.unmarshaller.StAXStreamConnector.bridge(StAXStreamConnector.java:161)
	at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:385)
	at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:364)
	at edu.suffolk.litlab.efspserver.codes.CodeUpdater.lambda$streamDownload$1(CodeUpdater.java:291)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
	at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)

Couldn't progress past that.