digest correct in some xml files but not others
SmartLayer opened this issue · 12 comments
First, I experimented signing a simple one-liner xml file using xmlsectool, and verified it using xmldsigjs, xmlsectool, xmlsec1, all verifies correctly.
Then, I signed a multi-page XML file with multi-namespace using xmlsectool. xmlsectool and xmlsec1 verifies it correctly but xmldsigjs gives digest mismatch error:
"XMLJS0013: Cryptographic error: Invalid digest for uri ''.
Calculated digest is **** but the xml to validate supplies digest ****"
It's a lot of work to go through the multi-page multi-namespace XML to find out what exactly triggered the miscalculation of digest, but one thing for sure, it's not because of any variation in the <dsig:Signature> since there are no variations like number of sigatures or number of certificates or difference in <dsig:Reference>. There were only differences in the content being signed.
Here is the working case:
$ printf '<?xml version="1.0" encoding="UTF-8"?>\n<text>hello world</text>' > hello-world.xml
$ /opt/xmlsectool-2.0.0/xmlsectool.sh --sign --keyInfoKeyName 'Shong Wang' --digest SHA-256 --signatureAlgorithm http://www.w3.org/2001/04/xmldsig-more#rsa-sha256 --inFile hello-world.xml --outFile hello-world-signed.xml --keystore shong.wang.p12 --keystoreType PKCS12 --key 1 --keyPassword shong.wang --signaturePosition LAST
INFO XMLSecTool - Reading XML document from file 'hello-world.xml'
INFO XMLSecTool - XML document parsed and is well-formed.
INFO XMLSecTool - XML document successfully signed
INFO XMLSecTool - XML document written to file /home/weiwu/IdeaProjects/TokenScript/examples/EntryToken/hello-world-signed.xml
The verification works:
$ node test-xmldsigjs.js
Signature status: true
Where test-xmldsignjs.js is slightly modified from examples to check hello-world-signed.xml
Here is the non-working case, signed with the same key/certificates, the test file is well-formed and validated against corresponding schemas:
Signing:
$ /opt/xmlsectool-2.0.0/xmlsectool.sh --sign --keyInfoKeyName 'Shong Wang' --digest SHA-256 --signatureAlgorithm http://www.w3.org/2001/04/xmldsig-more#rsa-sha256 --inFile long-test.xml --outFile long-test-signed.xml --keystore shong.wang.p12 --keystoreType PKCS12 --key 1 --keyPassword shong.wang --signaturePosition LAST
INFO XMLSecTool - Reading XML document from file 'long-test.xml'
INFO XMLSecTool - XML document parsed and is well-formed.
INFO XMLSecTool - XML document successfully signed
INFO XMLSecTool - XML document written to file /home/weiwu/IdeaProjects/TokenScript/examples/EntryToken/long-test-signed.xml
Resulting file:
Verifying using xmlsectool (SUCCESS):
$ /opt/xmlsectool-2.0.0/xmlsectool.sh --verifySignature --keyInfoKeyName 'Shong Wang' --digest SHA-256 --signatureAlgorithm http://www.w3.org/2001/04/xmldsig-more#rsa-sha256 --inFile long-test-signed.xml --keystore shong.wang.p12 --keystoreType PKCS12 --key 1 --keyPassword shong.wang
INFO XMLSecTool - Reading XML document from file 'long-test-signed.xml'
INFO XMLSecTool - XML document parsed and is well-formed.
INFO XMLSecTool - XML document signature verified.
Verifying using xmlsec1 (SUCCESS):
$ xmlsec1 --verify long-test-signed.xml
OK
SignedInfo References (ok/all): 1/1
Manifests References (ok/all): 0/0
Verifying using xmldsigjs.js (file name updated to long-test-signed.xml) (FAILURE)
test-xmldsigjs.js.txt
$ node test-xmldsigjs.js
XmlError {
prefix: 'XMLJS',
code: 13,
name: 'XmlError',
message: "XMLJS0013: Cryptographic error: Invalid digest for uri ''. Calculated digest is RsL4D/BpAQQ5k5MBbyhv+Ez7TJbBAXVwecUyit8w7pU= but the xml to validate supplies digest IaBPprnL25MstZmfJOg9R8iV+Ktu/qba6n3WxmhU1TQ=",
stack: "Error: XMLJS0013: Cryptographic error: Invalid digest for uri ''. Calculated digest is RsL4D/BpAQQ5k5MBbyhv+Ez7TJbBAXVwecUyit8w7pU= but the xml to validate supplies digest IaBPprnL25MstZmfJOg9R8iV+Ktu/qba6n3WxmhU1TQ=\n" +
' at new XmlError (/home/weiwu/IdeaProjects/tokenscript/node_modules/xml-core/dist/index.js:216:22)\n' +
' at SignedXml.ValidateReferences (/home/weiwu/IdeaProjects/tokenscript/node_modules/xmldsigjs/build/index.js:2708:23)\n' +
' at async SignedXml.Verify (/home/weiwu/IdeaProjects/tokenscript/node_modules/xmldsigjs/build/index.js:2403:21)'
}
My first thought is that there is some digest calculating issue when the file content has mixed namespaces. What is the underlying library that calculates the digest for xmldsigjs?
C14N is hardest part of XMLDSIG, there are lots of different nuances that could result in this.
The way we would debug is use another implementation, for example the .NET one, changing but by bit until we figured out how to get the same hash.
We will find time to look at this but can not commit to a timeline.
@rmhrisk Thanks. Since I use Java mostly, I can dump the result of Java C14N. It's native support for XMLDSig for quite some years and sing the same tune as xmlsec C library. If you can show me a few lines of how to dump the canonicalised XML from JavaScript, I can do a byte-to-byte comparison with the Java's dump and find the culprit there. How do you think?
I had a look at the files and experimented a bit I found that the issue is in the referenced files; token.en.shtml and enter.en.shtml. In these files are segments of CDATA. Whenever anything (even if it is just a single whitespace) is written in such a segment the verification fails. Thus it seems that the different libraries handle CDATA differently.
Since everything in CDATA is user-decided I guess a quick fix is simply to avoid using CDATA. In fact, since even a non-special character messes up the verification it seems that XMLDSIGjs might not support CDATA at all? But I am not sure about this.
I do recall there being some historic issue with CDATA; it’s entirely possible we don’t C14N it right.
@colourful-land You can print to console incoming data here https://github.com/PeculiarVentures/xmldsigjs/blob/master/src/signed_xml.ts#L358 or here https://github.com/PeculiarVentures/xmldsigjs/blob/master/src/algorithm.ts#L45
@colourful-land it looks like this particular document works fine in browser but not in node; that seems to be a function of xmldom's handling of CDATA.
@colourful-land it looks like this particular document works fine in browser but not in node; that seems to be a function of xmldom's handling of CDATA.
Later this month I set out to find the exact cause of this problem by stripping off all CDATA, then I will close this issue and start another with a specific test-case.
SGTM!
SGTM!
OKay, I stripped all CDATA and still, the code fails on 1/2 of the files I sent to test. On the other hand, all these signed XML files verify correctly in Java and TCL.
https://github.com/AlphaWallet/TokenScript/tree/xmldsig-verification-examples/xmldsig
So this problem is not limited to or caused by CDATA.
The above link provided how to reproduce the test, including a xmldsigverify.js which verifies all files given as command-line parameters; then a suite of test-files which you can get with git clone.
Ah, spent more than 5 hours to narrow down this problem to a simple 3-line XML test case that can demonstrate the problem. Let's close this issue and move the discussion to the new issue where the test case is presented. #47