[issue]: lunr-search doesn't pickup on heading elements that are non-direct children of '.markdown'

[Jeandcc]

Here's a demo codebase used to replicate the issue: https://github.com/Jeandcc/wrapped-heading-issue-w-docusaurus-lunr-search

As you can see on this file, we swizzled the DocItem 'component' so we can render some content on the documentation page based on the Front Matter data as well, and not only on the markdown of the page.

// /src/theme/DocItem/Content/index.js
export default function DocItemContent({ children }) {
  const { frontMatter } = useDoc();
  // (...)

  const headingFromFM = frontMatter?.heading || "";
  const descriptionFromFM = frontMatter?.description || "";
  const wrappedHeadingFromFM = frontMatter?.wrappedHeading || "";
  const wrappedDescriptionFromFM = frontMatter?.wrappedDescription || "";

  return (
    <div className={clsx(ThemeClassNames.docs.docMarkdown, "markdown")}>
      { /* (...) */ }

      <MDXContent>
        {headingFromFM && <h2>{headingFromFM}</h2>}
        {descriptionFromFM && <p>{descriptionFromFM}</p>}

        <div>
          {wrappedHeadingFromFM && <h2>{wrappedHeadingFromFM}</h2>}
          {wrappedDescriptionFromFM && <p>{wrappedDescriptionFromFM}</p>}
        </div>

        {children}
      </MDXContent>
    </div>
  );
}

Here's the example doc used to render Front Matter content as well as MD content.

The problem that we are currently facing is that docusaurus-lunr-search only pickups the heading elements that are direct children of the .markdown element. Headings that are encapsulated by other elements, such as divs, will not get picked up for indexing by lunr-search.

Here's "visual proof":

Visual preview of the documentation page

Searching for top-level heading generated with FrontMatter and Swizzling:

Searching for non-top-level heading generated with FrontMatter and Swizzling:

---

After browsing the docusaurus-lunr-search codebase, I believe the fix for this issue will touch on these lines of the codebase:

docusaurus-lunr-search/src/html-to-doc.js

Lines 109 to 118 in 44be1f0

	for (const node of markdown.children) {
	if (is(node, sectionHeaderTest)) {
	if (currentSection) {
	emitCurrent()
	}
	currentSection = node
	} else if (currentSection) {
	contentsAcc += getContent(node) + ' '
	}
	}

[Jeandcc]

Hello @praveenn77 , any chance you could have a quick look here? I'd be willing to work on a PR if you're too busy, but it would be great to get some insights from you first.

Thanks

[praveenn77]

Hi @Jeandcc , thanks for reporting this issue,
I have made a change and publsihed a new version 2.4.2 now you can search your FrontMatter element by adding an attribute data-search-children to the wrapper elements.

Hope this helps,
Thanks.

[Jeandcc]

Hi @praveenn77 , thanks a lot for the code you built for this functionality!

I took the freedom to further develop this feature, aiming at reducing the workload for developers in case they have headings on different levels of hierarchy.

With the previous approach, devs would have to add the data-search-children attribute to all divs that contained headings.

With the PR #117 that I opened, this would not be necessary, as the code would look through all direct and indirect children of the element with the data-search-children attribute.

Could you please take a look at it, and let me know in case of questions or requests for tweaks?

Thanks!