Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wip: mdoc reader #10225

Draft
wants to merge 81 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
68ec788
Start Mdoc lexer
silby Sep 9, 2024
b4bf4f4
Start mdoc reader
silby Sep 11, 2024
419ec78
Register the mdoc reader
silby Sep 11, 2024
f44074e
Checkpoint
silby Sep 11, 2024
b301ee0
stash pandoc.cabal nopub
silby Sep 11, 2024
fe1659d
Use standard spaceChar
silby Sep 11, 2024
35900e9
Consume spaces when lexing mdoc control tokens
silby Sep 11, 2024
5038334
Backtrack delim lexing
silby Sep 11, 2024
fd2ee1b
Delete some traces
silby Sep 11, 2024
d88e71d
Remove skip to end
silby Sep 11, 2024
eec5178
Lex blank mdoc lines
silby Sep 11, 2024
f4383b9
Handle blanks in mdoc parser
silby Sep 12, 2024
353943c
Comment upon parseStr's output
silby Sep 12, 2024
5a56e5d
Remove unused/redundant imports
silby Sep 12, 2024
754ffc8
Make more progress on inlines/paras
silby Sep 12, 2024
e1eddf6
Add parsing of code blocks from mdoc
silby Sep 12, 2024
9e44bb6
tktk: Unify roff escapes
silby Sep 12, 2024
a714f9d
Comment upon my approach
silby Sep 12, 2024
4431651
Add Xr parser
silby Sep 12, 2024
759d327
Lex callable macros in parsed control lines
silby Sep 13, 2024
784a873
Correct comments
silby Sep 13, 2024
1e8f7b7
Parse Nm macro (outside Sh NAME)
silby Sep 13, 2024
57c9709
Parse multiple inline macros per control line
silby Sep 13, 2024
34a83d9
Add a deprecated macro synonym
silby Sep 13, 2024
a3a892a
Plan for special handling of AUTHORS
silby Sep 14, 2024
4b3ecb8
Correct pasted comment
silby Sep 14, 2024
91f24a0
Add macros for line-scoped enclosures
silby Sep 14, 2024
497477d
Add Qq
silby Sep 15, 2024
7131923
Extract spacify function
silby Sep 15, 2024
1f5da9a
Rework enclosures and support multiline ones
silby Sep 15, 2024
10bdda8
fixup: qq
silby Sep 15, 2024
41f1454
Add Op macro
silby Sep 15, 2024
03d82ba
Add partial Ns support
silby Sep 15, 2024
ffe16cc
Lex called macros better
silby Sep 16, 2024
87097af
Lex inner quotes in quoted args correctly
silby Sep 16, 2024
fbd18a2
Handle the Ns macro
silby Sep 16, 2024
f988804
fixup: Remove Do trace
silby Sep 16, 2024
83a523d
Lex quoted delimiters on control lines
silby Sep 16, 2024
6ed3d83
Start Mdoc tests
silby Sep 16, 2024
12f4c49
Use lineEnclosure to parse Sh/Ss
silby Sep 16, 2024
3691891
Add Oo/Op macro
silby Sep 16, 2024
f8d263f
Handle Ns at end of control line correctly
silby Sep 16, 2024
e6d6dd9
Add the No macro
silby Sep 16, 2024
d319de0
Lex quoted callable macros as macros
silby Sep 16, 2024
30e3fce
Convert some more Ns tests from mandoc
silby Sep 16, 2024
608bbb2
Add more mandoc regress tests
silby Sep 16, 2024
bb3db9c
Add remaining multiline enclosures
silby Sep 16, 2024
07ea56e
Fix actual multiline enclosures
silby Sep 16, 2024
ee6be32
Remove YAGNI parseLit
silby Sep 16, 2024
ebef1a2
fixup: remaining multiline enclosures
silby Sep 16, 2024
8664d8e
Handle middle delimiters (i.e. the pipe)
silby Sep 16, 2024
35cd77b
Add middle punctuation tests from mandoc regress
silby Sep 16, 2024
f3b8fee
fixup: Spacing enclosures
silby Sep 16, 2024
81504ea
Add Lk support for hyperlinks
silby Sep 16, 2024
ecbb1db
Add Mt macro
silby Sep 17, 2024
309e864
Add .No test
silby Sep 17, 2024
2f28750
Add Pa (path) macro
silby Sep 19, 2024
f921ba8
Add Fl, Ev, Cm and tests
silby Sep 19, 2024
79c34c9
fixup Ev: Move test to the right section
silby Sep 19, 2024
86af568
Refine Fl parser
silby Sep 20, 2024
5a2196e
Extract helper for inline macros that produce Code
silby Sep 20, 2024
162f48d
Zap useless comments
silby Sep 20, 2024
2cf842d
Remove useless do
silby Sep 20, 2024
91e1730
Add .Fl Fl support
silby Sep 20, 2024
2a1688b
Add Ar macro
silby Sep 20, 2024
c9d1530
Add more easy macros
silby Sep 20, 2024
9b63a48
Zap comment alluding to an abandoned helper
silby Sep 20, 2024
8fe57d3
Add In macro (non-SYNOPSIS version)
silby Sep 21, 2024
dbf31cb
Ignore .Ns at start of line in mdoc lexer
silby Sep 24, 2024
b8b7411
Add Ap macro
silby Sep 24, 2024
f7d07be
Add a couple more easy codelike macros
silby Sep 24, 2024
3464d9c
tktktk: Sm support
silby Sep 25, 2024
3b74c82
Abandon whitespace-preservation in text lines
silby Sep 26, 2024
681e1d3
Skip to the end of the SYNOPSIS for now
silby Sep 26, 2024
9d956b8
Reformat
silby Sep 26, 2024
7e0f712
Fix .Nm parsing and insertion of manual name
silby Sep 26, 2024
638d84a
Skip unknown macros in block context
silby Sep 26, 2024
6ed129f
Parse display blocks and 1-line literals
silby Sep 26, 2024
2d94ad6
Move skipUnknownMacro to outer blocks
silby Sep 27, 2024
2589dbb
Add Dv macro
silby Sep 27, 2024
2147b2f
Add bullet, ordered, and definition lists
silby Sep 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion pandoc.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -522,6 +522,7 @@ library
split >= 0.2 && < 0.3,
syb >= 0.1 && < 0.8,
tagsoup >= 0.14.6 && < 0.15,
template-haskell,
temporary >= 1.1 && < 1.4,
texmath >= 0.12.8.10 && < 0.13,
text >= 1.1.1.0 && < 2.2,
Expand All @@ -547,6 +548,7 @@ library
hs-source-dirs: src

exposed-modules: Text.Pandoc,
Text.Pandoc.Readers.Roff,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exposed this for my lexroff.hs thing which I didn't commit, move back to other-modules before merge

Text.Pandoc.App,
Text.Pandoc.Data,
Text.Pandoc.Options,
Expand Down Expand Up @@ -588,6 +590,8 @@ library
Text.Pandoc.Readers.EPUB,
Text.Pandoc.Readers.Muse,
Text.Pandoc.Readers.Man,
Text.Pandoc.Readers.Mdoc,
Text.Pandoc.Readers.Mdoc.Lex,
Text.Pandoc.Readers.FB2,
Text.Pandoc.Readers.DokuWiki,
Text.Pandoc.Readers.Ipynb,
Expand Down Expand Up @@ -699,6 +703,7 @@ library
Text.Pandoc.Readers.LaTeX.Parsing,
Text.Pandoc.Readers.LaTeX.SIunitx,
Text.Pandoc.Readers.LaTeX.Table,
Text.Pandoc.Readers.Mdoc.Macros,
Text.Pandoc.Readers.Typst.Parsing,
Text.Pandoc.Readers.Typst.Math,
Text.Pandoc.Readers.ODT.Base,
Expand All @@ -721,8 +726,8 @@ library
Text.Pandoc.Readers.Org.ParserState,
Text.Pandoc.Readers.Org.Parsing,
Text.Pandoc.Readers.Org.Shared,
Text.Pandoc.Readers.Roff.Escape,
Text.Pandoc.Readers.Metadata,
Text.Pandoc.Readers.Roff,
Text.Pandoc.Writers.Docx.OpenXML,
Text.Pandoc.Writers.Docx.StyleMap,
Text.Pandoc.Writers.Docx.Table,
Expand Down Expand Up @@ -822,6 +827,7 @@ test-suite test-pandoc
Tests.Readers.Muse
Tests.Readers.Creole
Tests.Readers.Man
Tests.Readers.Mdoc
Tests.Readers.FB2
Tests.Readers.DokuWiki
Tests.Writers.Native
Expand Down Expand Up @@ -858,3 +864,8 @@ benchmark benchmark-pandoc
deepseq
-- we increase heap size to avoid benchmarking garbage collection:
ghc-options: -rtsopts -with-rtsopts=-A8m -threaded

executable lexroff
import: common-executable
main-is: lexroff.hs
build-depends: pandoc, text
Comment on lines +867 to +871
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Private test thingy, need to zap this.

3 changes: 3 additions & 0 deletions src/Text/Pandoc/Readers.hs
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ module Text.Pandoc.Readers
, readEPUB
, readMuse
, readMan
, readMdoc
, readFB2
, readIpynb
, readCSV
Expand Down Expand Up @@ -106,6 +107,7 @@ import Text.Pandoc.Readers.TWiki
import Text.Pandoc.Readers.Txt2Tags
import Text.Pandoc.Readers.Vimwiki
import Text.Pandoc.Readers.Man
import Text.Pandoc.Readers.Mdoc
import Text.Pandoc.Readers.CSV
import Text.Pandoc.Readers.CslJson
import Text.Pandoc.Readers.BibTeX
Expand Down Expand Up @@ -168,6 +170,7 @@ readers = [("native" , TextReader readNative)
,("rtf" , TextReader readRTF)
,("typst" , TextReader readTypst)
,("djot" , TextReader readDjot)
,("mdoc" , TextReader readMdoc)
]

-- | Retrieve reader, extensions based on format spec (format+extensions).
Expand Down
Loading
Loading