Large LS agents result in an XML exception on import

Question

Large LS agents result in an XML exception on import

Closed this issue 3 years ago · 3 comments

This specifically happens when the LotusScript text length goes beyond what a WORD can contain.

java.util.concurrent.ExecutionException: java.lang.RuntimeException: Exception while importing element Code/Agents/Lengthy LotusScript Agent.lsa
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.openntf.nsfodp.compiler.servlet.ODPCompilerServlet.doPost(ODPCompilerServlet.java:216)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:713)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
	at org.eclipse.equinox.http.registry.internal.ServletManager$ServletWrapper.service(ServletManager.java:180)
	at org.eclipse.equinox.http.servlet.internal.ServletRegistration.handleRequest(ServletRegistration.java:90)
	at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:111)
	at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:59)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
	at com.ibm.domino.xsp.adapter.osgi.OSGIModule.invokeServlet(OSGIModule.java:167)
	at com.ibm.domino.xsp.adapter.osgi.OSGIModule.access$0(OSGIModule.java:153)
	at com.ibm.domino.xsp.adapter.osgi.OSGIModule$1.invokeServlet(OSGIModule.java:134)
	at com.ibm.domino.xsp.adapter.osgi.AbstractOSGIModule.invokeServletWithNotesContext(AbstractOSGIModule.java:181)
	at com.ibm.domino.xsp.adapter.osgi.OSGIModule.doService(OSGIModule.java:128)
	at com.ibm.domino.xsp.adapter.osgi.OSGIService.doService(OSGIService.java:418)
	at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.doService(LCDEnvironment.java:357)
	at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.service(LCDEnvironment.java:313)
	at com.ibm.domino.xsp.bridge.http.engine.XspCmdManager.service(XspCmdManager.java:272)
Caused by: java.lang.RuntimeException: Exception while importing element Code/Agents/Lengthy LotusScript Agent.lsa
	at org.openntf.nsfodp.compiler.ODPCompiler.lambda$9(ODPCompiler.java:578)
	at org.openntf.nsfodp.compiler.ODPCompiler$$Lambda$124/0x0000000000000000.accept(Unknown Source)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:497)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:487)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:241)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
	at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:497)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:487)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:241)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
	at org.openntf.nsfodp.compiler.ODPCompiler.importBasicElements(ODPCompiler.java:572)
	at org.openntf.nsfodp.compiler.ODPCompiler.compile(ODPCompiler.java:365)
	at org.openntf.nsfodp.compiler.ODPCompiler.compile(ODPCompiler.java:300)
	at org.openntf.nsfodp.compiler.servlet.ODPCompilerServlet.lambda$3(ODPCompilerServlet.java:213)
	at org.openntf.nsfodp.compiler.servlet.ODPCompilerServlet$$Lambda$39/0x0000000000000000.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at lotus.domino.NotesThread.run(Unknown Source)
Caused by: java.lang.RuntimeException: javax.xml.bind.UnmarshalException
 - with linked exception:
[org.xml.sax.SAXParseException; lineNumber: 147; columnNumber: 31; XML document structures must start and end within the same entity.]
	at org.openntf.nsfodp.compiler.dxl.DxlImporterLog.forXml(DxlImporterLog.java:48)
	at org.openntf.nsfodp.compiler.ODPCompiler.importDxl(ODPCompiler.java:847)
	at org.openntf.nsfodp.compiler.ODPCompiler.lambda$9(ODPCompiler.java:575)
	... 32 more
Caused by: javax.xml.bind.UnmarshalException
 - with linked exception:
[org.xml.sax.SAXParseException; lineNumber: 147; columnNumber: 31; XML document structures must start and end within the same entity.]
	at javax.xml.bind.helpers.AbstractUnmarshallerImpl.createUnmarshalException(AbstractUnmarshallerImpl.java:335)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.createUnmarshalException(UnmarshallerImpl.java:563)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:249)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:214)
	at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:157)
	at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:214)
	at org.openntf.nsfodp.compiler.dxl.DxlImporterLog.forXml(DxlImporterLog.java:45)
	... 34 more
Caused by: org.xml.sax.SAXParseException; lineNumber: 147; columnNumber: 31; XML document structures must start and end within the same entity.
	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:400)
	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
	at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1473)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.endEntity(XMLDocumentFragmentScannerImpl.java:899)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.endEntity(XMLDocumentScannerImpl.java:559)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.endEntity(XMLEntityManager.java:1410)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.load(XMLEntityScanner.java:1916)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.scanLiteral(XMLEntityScanner.java:1187)
	at com.sun.org.apache.xerces.internal.impl.XMLScanner.scanAttributeValue(XMLScanner.java:988)
	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanAttribute(XMLNSDocumentScannerImpl.java:447)
	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:250)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2784)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:243)
	... 38 more

Answer 1 · 2021-05-19T17:34:47.000Z

In testing, I observed this in remote compilation against Windows but not in local compilation on macOS.

Answer 2 · 2021-05-19T17:40:23.000Z

The XML exception seems to be a knock-on effect. The lower-level trouble is a lengthy sequence of:

<fatalerror source='NotesInputSource' line='159' column='295'>Invalid character (Unicode: 0x0)</fatalerror>

However, the XML returned by the C function stops abruptly, presumably due to string-length limits in the call. I'm not sure what can be done about that, but the specific exception is worth noting.

Answer 3 · 2021-05-19T17:50:45.000Z

I'm concerned that this may be trouble with reading from the ZIP directly. If I first do a Files.copy to copy the DXL to the filesystem and then do a Files.newInputStream on that, it works. It's not too much trouble to make such a copy, but it bodes poorly for other elements. Still, I'll start by adding the workaround here and seeing how it shakes out in the future.