Better markdown integration

[RossBencina]

The following areas relating to prapti markdown have been identified for improvement:

• The use of ### level 3 headings for delimiting message sections has been criticised as arbitrary. Furthermore, if your view of a Markdown document is as an outline structure of nested headings, then any use of headings as message delimiters may be confusing. For example, we use L3 headings as delimiters, but we do not terminate a message section when an L2 heading is encountered. Possible improvement:

  • Allow any heading level to stand in for a message delimiter
  • Consider alternative ways of parsing out a message sequence from the heading hierarchy (e.g. allow L2 to start a hidden section after encountering an L3 message heading)

• Right now, the parser considers any line that matches our regex as a heading. In fact we should not be recognising message delimiter headings inside fenced code blocks.

• We may want to treat roles in message headings as case insensitive. E.g. "User" and "USER" might also be considered reasonable presentation in some settings (e.g. OpenAI docs display example conversations using User: and System: headings.)

• HTML <!-- --> comments are part of the markdown spec. We may want to interpret them as "commenting out" sections of the file, and not send these parts to the LLM.

• Don't match % config lines inside HTML comments or fenced code blocks (the latter is somewhat important when using prapti to document prapti configurations)