Executing PatchHandle against non-existing Document does not throw an Exception

[Zipunrar]

So we can address your issue, please include the following:

Version of MarkLogic Java Client API

5.5.4

Version of MarkLogic Server

10.0-9.2

Java version

Java 8 (Oracle and OpenJDK)

OS and version

Windows 10

Input: Some code to illustrate the problem, preferably in a state that can be independently reproduced on our end

I created a simple test that will show the behavior:

import com.marklogic.client.DatabaseClient;
import com.marklogic.client.DatabaseClientFactory;
import com.marklogic.client.DatabaseClientFactory.DigestAuthContext;
import com.marklogic.client.DatabaseClientFactory.SecurityContext;
import com.marklogic.client.ResourceNotFoundException;
import com.marklogic.client.document.DocumentMetadataPatchBuilder;
import com.marklogic.client.document.DocumentPatchBuilder;
import com.marklogic.client.document.DocumentPatchBuilder.Position;
import com.marklogic.client.document.XMLDocumentManager;
import com.marklogic.client.io.Format;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringRunner;

/**
 * This test shows an issue with executing document patches against non-existing documents.
 *
 * <p>The expected behavior is to throw an {@link ResourceNotFoundException} in all cases.
 * However the actual behavior is, that only for document patches an exception is thrown.
 * For metadata patches nothing happens and the problem is ignored silently.</p>
 *
 * @see <a href="https://jira.din.de/browse/XSCORE-1231">XSCORE-1231</a>
 *
 * @author WIB
 * @version 1.0
 * @since 2022-10-26
 */
@RunWith (SpringRunner.class)
public class PatchNonExistingDocumentTest
{
    @Value ("${marklogic.host}")
    private String host;
    @Value ("${marklogic.port}")
    private int port;
    @Value ("${marklogic.username}")
    private String username;
    @Value ("${marklogic.password}")
    private String password;

    private DatabaseClient client;

    private XMLDocumentManager manager;

    @Before
    public void setUp()
    {
        final SecurityContext context = new DigestAuthContext(this.username, this.password);

        this.client = DatabaseClientFactory.newClient(this.host, this.port, context);

        this.manager = this.client.newXMLDocumentManager();
    }

    @After
    public void tearDown()
    {
        this.client.release();
    }

    @Test(expected = ResourceNotFoundException.class)
    public void testPatch_NonExistingDocument()
    {
        final DocumentPatchBuilder builder = this.manager.newPatchBuilder();

        builder.insertFragment("/a", Position.LAST_CHILD, "<foo>bar</foo>");

        this.manager.patch("unknown", builder.build());
    }

    @Test(expected = ResourceNotFoundException.class)
    public void testPatch_CollectionOfNonExistingDocument()
    {
        final DocumentMetadataPatchBuilder builder = this.manager.newPatchBuilder(Format.XML);

        builder.addCollection("anything");

        this.manager.patch("unknown", builder.build());
    }

    @Test(expected = ResourceNotFoundException.class)
    public void testPatch_PropertyOfNonExistingDocument()
    {
        final DocumentMetadataPatchBuilder builder = this.manager.newPatchBuilder(Format.XML);

        builder.addPropertyValue("junit", "foobar");

        this.manager.patch("unknown", builder.build());
    }
}

The tests testPatch_CollectionOfNonExistingDocument and testPatch_PropertyOfNonExistingDocument will fail.

Actual output: What did you observe? What errors did you see? Can you attach the logs? (Java logs, MarkLogic logs)

The operations will throw an ResourceNotFoundException for patching documents but not for patching metadata only.

Expected output: What specifically did you expect to happen?

The operations should behave consistent, by throwing a ResourceNotFoundException because we are patching something that
does not exist.

Alternatives: What else have you tried, actual/expected?

Nothing. Could be temporarly fixed by executing an "exists" request against the target URI before executing the patch.

[rjrudin]

Thanks for the tests @Zipunrar , I'm trying them out now.

[rjrudin]

@Zipunrar Thanks again for the test, made debugging very easy.

I haven't identified why yet, but "unknown" seems to be a special case. If I change that to "unknown.xml" or even "unknownn", I get the expected 404 and ResourceNotFoundException from ML. But the specific value "unknown" results in a 204 No Content being returned.

Can you try your test with "unknown.xml" and "unknownn" (or any word besides "unknown") as the URI? Curious if you get the same results.

[Zipunrar]

Hi @rjrudin ,

very interesting. Never thought the URI could lead to such strange behavior. I tried some different values and get other results depending on the value of the URI.

• "unknown", "unknown.xml", "foobar", "unknownn" produces failed tests for both property and collection patching
• "foo/bar.xml", "check", "simple/test/with/random/uri.json" however does only fail for the property patch not for the collection or document patch
• I didn't find values to let only the collection patching fail
• In all cases the patching document test never fails and the expected exception will be thrown

[rjrudin]

Revisiting this - for patching document properties, I verified that the underlying server functions - xdmp:document-set-properties and xdmp:document-add-properties - do not throw an error if the URI is not found. So the third test that invokes addPropertyValue is behaving correctly.

However, I'm able to produce the same behavior in qconsole - i.e. xdmp:document-set-collections will fail for some URIs but not throw an error for other URIs. If I run a "clear" operation on the database, then the function will fail for all URIs. So it appears to be a server issue where URIs perhaps associated with deleted documents do not always result in an error being thrown. Will investigate further - but I don't think this has anything to do with the Java Client or the REST API.

Will leave this open until I isolate the server issue.

[rjrudin]

@Zipunrar I think I know the explanation. As noted above, performing any "properties" operations on a URI will succeed and results in a properties fragment for the URI being created - even though there's not a document for that URI. So if you start with an empty database, you still won't see any documents in qconsole, but you will see a fragment count of 1 in the Admin UI for that database.

Once the URI is associated with a properties fragment, all metadata operations will succeed on that URI. If you clear the database, that will delete the properties fragment, and then all metadata operations (including PATCH requests) will throw an error as expected.

With your test, if you were to run it twice, the test that adds a collection would not throw an error because the test that adds a property would have resulted in the URI being created with a properties fragment associated with it.

Thus, I do not believe there is a bug here - the behavior with the Java Client and REST API is consistent with how the server functions work when a URI is involved that only has a properties fragment associated with it.