Linkcheck Builder: False Positive "Anchor not found" Errors with Encoded Characters in Fragment Identifiers

[nrdlngr]

Describe the bug

Linkcheck Builder: False Positive "Anchor not found" Errors with Encoded Characters in Fragment Identifiers

Problem Description

The Sphinx linkcheck builder incorrectly reports "Anchor not found" errors for URLs with encoded characters in fragment identifiers (anchors), despite these URLs working correctly in web browsers.

Current Behavior

When encountering a URL with percent-encoded characters in the anchor/fragment (e.g., https://example.com/page#standard-input%2Foutput-stdio), the linkcheck builder:

1. Extracts the fragment: standard-input%2Foutput-stdio
2. Decodes it to: standard-input/output-stdio
3. Searches for an HTML element with id="standard-input/output-stdio" or name="standard-input/output-stdio"
4. Reports a broken link when the element isn't found

Expected Behavior

The linkcheck builder should recognize URLs with encoded characters in fragments that work in browsers. This includes cases where:

1. The HTML contains the encoded form in the id attribute: id="standard-input%2Foutput-stdio"
2. The browser handles the URL correctly through its own anchor matching rules

Example

URL being checked:

https://example.com/page#standard-input%2Foutput-stdio

Sphinx error:

broken link: Anchor 'standard-input/output-stdio' not found

Actual HTML in target page:

<element id="standard-input%2Foutput-stdio">...</element>

This URL works correctly in web browsers but fails in Sphinx linkcheck.

Affected Code

The issue appears to be in the following modules:

• sphinx/builders/linkcheck.py in HyperlinkAvailabilityCheckWorker._check_uri()
• sphinx/builders/linkcheck.py in AnchorCheckParser.handle_starttag()

Impact

This issue causes false positive "broken link" reports in documentation that references modern web pages with encoded characters in anchor IDs, leading to:

• Failed CI/CD pipelines due to false positive link checks
• Documentation teams needing to create workarounds or ignore valid URLs
• Extra maintenance overhead to verify links manually

Environment Information

sphinx-build --bug-report

Note: Please include the output of this command when filing the actual issue

Possible Solution Approach

Enhance anchor checking to consider both the decoded anchor (current behavior) and the original encoded anchor as valid matches. This would better align with browser behavior while maintaining backward compatibility.

Additional Context

• Modern web platforms often use URL-safe encodings in element IDs
• Browsers handle both encoded and decoded fragment matching
• This impacts documentation referencing web frameworks, APIs, and various technical documentation sites

How to Reproduce

When encountering a URL with percent-encoded characters in the anchor/fragment (e.g., https://example.com/page#standard-input%2Foutput-stdio), the linkcheck builder:

1. Extracts the fragment: standard-input%2Foutput-stdio
2. Decodes it to: standard-input/output-stdio
3. Searches for an HTML element with id="standard-input/output-stdio" or name="standard-input/output-stdio"
4. Reports a broken link when the element isn't found

Environment Information

Platform:              darwin; (macOS-15.5-arm64-arm-64bit)
Python version:        3.12.7 (main, Oct  8 2024, 14:30:16) [Clang 16.0.0 (clang-1600.0.26.3)])
Python implementation: CPython
Sphinx version:        8.3.0+/fd7eef97b
Docutils version:      0.21.2
Jinja2 version:        3.1.6
Pygments version:      2.19.1

Sphinx extensions

Additional context

I already have a fix implemented that I'd like to contribute via a PR. I've tested this locally and it works for my implementation.

[jayaddison]

Browsers handle both encoded and decoded fragment matching

I have a theory about why this works in web browsers, and I think it is arguably due to a bug/shortcut.

Browsers will need/want to support unicode anchor targets, and that means that they need to support URLs where those anchor names are percent-encoded (aka URLencoded).

Browsers -- unlike the Sphinx linkcheck builder -- necessarily load the entire HTML of the page to produce a result. While doing so they likely build an index of id / name targets found in the HTML.

Building an index suitable for single-equality presence checks by a fragment anchor identifier (the part in the browser URL address after the # character) could be done easily by natively URLencoding each id / name attribute value as it is found. This wouldn't double-encode existing URLencoded values, since those are already ASCII.

Modern web platforms often use URL-safe encodings in element IDs

This seems to be unusual; I don't think that URLencoding content within an HTML document is good practice, and the HTML5 spec doesn't seem to mention any logic related to that for anchor attributes.

This impacts documentation referencing web frameworks, APIs, and various technical documentation sites

@nrdlngr can you share any examples of web platforms and/or Sphinx ecosystem components that produce HTML with percent-encoded anchor values?

[nrdlngr]

The specific site that I am linking to was here:

https://modelcontextprotocol.io/docs/concepts/transports#standard-input%2Foutput-stdio

[jayaddison]

The specific site that I am linking to was here:

https://modelcontextprotocol.io/docs/concepts/transports#standard-input%2Foutput-stdio

Thanks @nrdlngr. Unless this behaviour is widespread/commonplace, I think that this is a bug in the HTML documentation on the modelcontextprotocol.io website.

It seems they build their documentation using mintlify: https://github.com/modelcontextprotocol/modelcontextprotocol/blob/6a464549441df4e4a8ef09d9dc6b20e134bfba1e/README.md?plain=1#L9-L10

...and that theory is somewhat-supported by the fact that I found two </html> closing tags in the markup of the transports page -- matching an existing issue report at mintlify/starter#59

As far as I can tell, although Mintlify provide many components as open source software, it is not possible to install and locally build a Mintlify project without using their hosted services -- and so it's unclear to me where the code/logic producing the (in my opinion, incorrectly) URLencoded id attribute is.

Basically if we could figure out where to find that code, then I think it would be straightforward to report it as a problem and/or fix it in the Mintlify codebase.

If we add search of multiple anchor variations to the Sphinx linkcheck builder, then it's arguably slightly less efficient (across all future invocations), but also it's a feature that would be difficult to withdraw -- and we'd implictly be permitting other websites to begin making the same -- again in my opinion -- mistake.

I think this particular error is rare and so I would be optimistic that we can trace down cases where it happens and report/fix those. Do you know of any other websites where this occurs?

I'll also try to confirm my theory that web browsers are using the URLencoded version of the anchor target id/names as a lowest-common-denominator (potentially explaining why browsers do seem to handle this quirk, unexpectedly perhaps).

[jayaddison]

As far as I can tell, although Mintlify provide many components as open source software, it is not possible to install and locally build a Mintlify project without using their hosted services -- and so it's unclear to me where the code/logic producing the (in my opinion, incorrectly) URLencoded id attribute is.

I forgot to mention: as supporting context, it may be helpful to know that the relevant transports page markup (and standard I/O section heading) can be found at: https://github.com/modelcontextprotocol/modelcontextprotocol/blob/6a464549441df4e4a8ef09d9dc6b20e134bfba1e/docs/docs/concepts/transports.mdx?plain=1#L54

[jayaddison]

I'm now also wondering whether there are existing Web Platform Tests to cover scenarios such as this. I'll try to add any relevant findings about that too.

[jayaddison]

I'm now also wondering whether there are existing Web Platform Tests to cover scenarios such as this. I'll try to add any relevant findings about that too.

This appears relevant; other tests in the same directory may be also: https://github.com/web-platform-tests/wpt/blob/d5212ae2a12687d7abad0db6ae39252474776aca/html/browsers/browsing-the-web/scroll-to-fragid/scroll-frag-percent-encoded.html

Is it a bug that the top position in that test file for the id="do%20not%20go%20here" anchor target is 400? I think the intent was for each anchor to have a unique position, so that the tests can unambiguously determine whether the browser-under-test has scrolled...

[jayaddison]

I'm now also wondering whether there are existing Web Platform Tests to cover scenarios such as this. I'll try to add any relevant findings about that too.

This appears relevant; other tests in the same directory may be also: https://github.com/web-platform-tests/wpt/blob/d5212ae2a12687d7abad0db6ae39252474776aca/html/browsers/browsing-the-web/scroll-to-fragid/scroll-frag-percent-encoded.html

Is it a bug that the top position in that test file for the id="do%20not%20go%20here" anchor target is 400? I think the intent was for each anchor to have a unique position, so that the tests can unambiguously determine whether the browser-under-test has scrolled...

I've added a question about the test fixture and whether it could be buggy at: https://github.com/web-platform-tests/wpt/pull/2595/files#r2142279143

That test case could help us to determine what the expected and actual behaviour of browsers is in relation to this kind of percent encoding, and whether the browser should indeed follow such URLencoded fragment identifiers to URLencoded anchors within an HTML document.

[jayaddison]

Modern web platforms often use URL-safe encodings in element IDs

This seems to be unusual; I don't think that URLencoding content within an HTML document is good practice, and the HTML5 spec doesn't seem to mention any logic related to that for anchor attributes.

Ok; I may be mistaken about all of this. The HTML5 spec -- specifically the section with title "Scrolling to a fragment" (7.4.6.4 in the version I'm reading, last-updated 2025-06-09; permalink) -- does, in the algorithm to find the indicated target, mention a lookup of potentialIndicatedElement before performing percent-decoding. I interpret that as: the raw -- and potentially percent-encoded, as in the MCP example we are investigating here -- will be sought as an identified element in the HTML.

So: my apologies. It may be that this is indeed per HTML spec -- despite the fact that I think it's unnecessary and unusual -- although unexpectedly, perhaps more-efficient, per the HTML5 algorithm -- for the MCP homepage to use percent-encoding within attribute values.

Even so: I do think there could be a quirk/bug in the WPT test, and I will attempt to follow up on that separately.

Given all this, I will attempt to review the linked PR #13621 soon.

Edit: add hyperlink and permalink.

[jayaddison]

Ok; I may be mistaken about all of this. The HTML5 spec -- specifically the section with title "Scrolling to a fragment" (7.4.6.4 in the version I'm reading, last-updated 2025-06-09; permalink) -- does, in the algorithm to find the indicated target, mention a lookup of potentialIndicatedElement before performing percent-decoding. I interpret that as: the raw -- and potentially percent-encoded, as in the MCP example we are investigating here -- will be sought as an identified element in the HTML.

Indeed: the commit where this behaviour was codified in the WhatWG HTML5 spec is: whatwg/html@c230f55

(the commit message helps by describing the reasoning for the change -- other browsers had already performed literal-value-search before attempting percent-decoded-value-search)

[jayaddison]

Initially I was apprehensive and uncertain that this should be considered a bug in Sphinx.

That's because I didn't expect HTML documents to contain any percent-encoded content; I felt that the HTML and URI specifications were distinct and that the form and content of each datatype (HTML documents, and URIs) did not overlap.

I do still think that it is generally unusual and incorrect to included percent-encoded content in HTML documents -- but the difference-in-behaviour identified here has uncovered a genuine difference between Sphinx's linkcheck anchor discovery compared to web browsers. And, since what that utility is intended to do is to confirm whether hyperlinks will navigate the user to referenced content sections within a document -- as a browser would -- then I think a fix is required in the code.

As a result, I've begun review of the linked PR in #13621.