New lxml version breaks XML entity references

[zimri-leisher]

F´ Version	3.5.1
Affected Component	fprime-util build

---

Problem Description

3.5.0 introduced the following line to requirements.txt:
lxml==5.3.0
Previously, the default version was 4.9.3. This is now causing issues because some of the lxml settings have changed, which broke usage of a specific XML feature: entity references.

Entity references in XML allow you to "include" other XML documents. This is extremely useful for modular TlmPacketizer based deployments. It looks like this:

<!DOCTYPE doc [
<!ENTITY fsw_core SYSTEM "../../common/topology/core/fsw_core_packets.xml">
]>
<packets name="packets" namespace="randomNamespace" size="3500">

    <import_topology>deployment/Top/deploymentTopologyAppAi.xml</import_topology>

    &fsw_core;
</packets>

This effectively imports whatever text is inside of the fsw_core_packets.xml.
This doesn't work in the 5.3.0 version of lxml, because the default XML parser has entity resolving disabled. The fix is on line 263 of fprime/Autocoders/Python/bin/tlm_packet_gen.py:

       if not os.path.isfile(xml_filename):
            raise TlmPacketParseIOError("File %s does not exist!" % xml_filename)

        fd = open(xml_filename, "r")
        xml_parser = etree.XMLParser(remove_comments=True, load_dtd=True, resolve_entities=True, no_network=True) # < add load_dtd=True, resolve_entities=True, and no_network=True
        element_tree = etree.parse(fd, parser=xml_parser)
        channel_size_dict = None

I think they disabled this by default because there are some possible security vulnerabilities with network access. However, I think passing no_network=True should be enough to mitigate these.

Context / Environment

Execute fprime-util version-check and share the output.

Operating System: Linux
CPU Architecture: x86_64
Platform: Linux-5.15.133.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Python version: 3.10.12
CMake version: 3.22.1
Pip version: 22.0.2
Pip packages:
    fprime-tools==3.5.1
    fprime-gds==3.5.1
    fprime-fpp-*==2.2.1
Project submodules:
    https://github.com/nasa/fprime.git @ v3.5.1

[zimri-leisher]

Made a PR to fix this ^

[LeStarch]

We closed the PR as it is likely superseded, but I will keep this issue open until that work is delivered. The link to the PR (for recovery) is:

#3112

[zimri-leisher]

Okay, understood. We've reverted since so not experiencing any issues. Looking forward to FPP packet spec!