GeezerVison is showing github.com — Add utility functions to test for legal XML characters and names · Issue #139489 · python/cpython
Add utility functions to test for legal XML characters and names #139489DescriptionFeature or enhancement
It is well known fact that some characters (such as < or &) must be escaped in XML and HTML, and that attribute values must be quoted. It is less known fact (unless you specially looked for it) that not all characters can be included in XML, even if escaped. For example, XML cannot contain the null character.
There is also restriction on names of elements and attributes. They cannot contain <, >, /, !, ?, spaces and many other characters. But unlike to Python identifiers, -, :, ., etc are acceptable. The list of valid and invalid characters is pretty long. In the case when the user input is used for element or attribute names without validation, this can even lead to XML injection vulnerability (CVE-2025-9375)
So, I think that it would be useful to provide standard functions to validate XML characters and names in the stdlib. xml.sax.saxutils looks an appropriate place, it already has escape() and quoteattr() utilities.
We can also provide functions to "sanitize" XML characters, similar to sanitize_xml() in Lib/test/libregrtest/utils.py, but more general. This is similar to old issue #63014.
Now, the problem is that there are two standards of XML: 1.0 and 1.1. The former is much more popular. And they have different definitions of legal characters. There are also restricted characters in XML 1.1 which cannot be used in "well-formed" documents and parsed entities. There is also a set of characters (version depending) using which legal but is discouraged. Should we have several functions or several parameters to specify the XML version and other options?
https://www.w3.org/TR/xml/#charsets
https://www.w3.org/TR/xml11/#charsets
Fortunately, the syntax for names is the same in XML 1.0 and 1.1.
https://www.w3.org/TR/xml/#NT-Name
https://www.w3.org/TR/xml11/#NT-Name
We can also add a similar set of functions for HTML.
Linked PRs
Activityserhiy-storchaka commented on Oct 2, 2025 picnixz commented on Oct 2, 2025
Should we have several functions or several parameters to specify the XML version and other options
If the options are common to standards, we can have one function with many parameters and one "version='...'" parameter. I don't think we'll see XML 2.0 soon as there doesn't seem to be any consensus on that, but I think it's better if we have a simple switch. By default, I would use version="1.0" (or we could even have a default_version module-wide configuration that would parametrize such functions if version=None but that could be a separate feature).
If the options are not common, let's have different functions, but also one utility function to ease usage:
def escape_xml_1_1(string, /, a=None, b=None): ...
def escape_xml_1_0(string, /, x=None, y=None, z=None): ...
def escape_xml(string, /, *args, version='1.0', **kwargs):
if version == "1.0": return escape_xml_1_0(string, *args, **kwargs)
if version == "1.1": return escape_xml_1_1(string, *args, **kwargs)
raise ValueError(f"unknown version {version}")mpkocher commented on Oct 2, 2025 Sounds like a reasonable approach.
Functions in Python are first-class citizens and are useful to leverage.
def escape_xml(string, /, *args, version='1.0', **kwargs):
funcs = {"1.0": escape_xml_1_0, "1.1": escape_xml_1_1}
if f := funcs.get(version):
return f(string, *args, **kwargs)
raise ValueError(f"unknown version {version}. Supported versions are {list(funcs.keys())}")serhiy-storchaka commented on Oct 9, 2025 I initially planned to add these functions in the xml.sax.saxutils module, because it already contains few other helper functions. But @vstinner proposed to create a new xml.utils module for the new function or put them directly in the xml top-level package. And I cannot find strong arguments against this. Yes, xml was a pure package which only served as a namespace for other XML related packages, but these functions are not actually specific to SAX parsing (as well as existing functions escape(), unescape() and quoteattr()). serhiy-storchaka commented on Oct 9, 2025 serhiy-storchaka commented on May 5, 2026 Since Expat, and therefore the Python stdlib does not support XML 1.1, it would be simpler to only support XML 1.0. Support of XML 1.1 adds complexity, and we cannot be sure that API matches the user's need. If there is demand for such feature, we can add it later (either as separate functions or as new parameters).
#149412 only supports XML 1.0. I have two other branches for support of XML 1.1 with different interfaces. serhiy-storchaka commented on May 6, 2026 The new functions are used in #149468. And I am planning to use them in more places. JelleZijlstra commented on May 8, 2026 The new is_valid_name function is in xml.__all__ as of 3.15.0b1, but is_valid_text is not. Is that intentional?
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|