Feedback request for Miller 6 docs

[johnkerl]

Within the context of the Go port (README, tracking issue, and responses from the 2021 Miller survey) I'd like to hear some thoughts on Miller 6 docs.

• URL is https://johnkerl.org/miller6. (Ultimately these will replace https://miller.readthedocs.io/en/latest, once Miller 6 is released but not yet)
• Styling is (I believe) much better for desktop as well as mobile -- in terms of access to menu/navbar
• There's more, and gentler, up-front material for non-experts; expert-level material is pushed farther down
• Many big pages have been split into smaller ones -- including the non-DSL and DSL reference pages, those two are now many. (Maybe too many -- ?)
• Also note not all content has been updated for Miller 6 -- there are statements in some pages which are true for Miller 5, not true for Miller 6 anymore. Also several spots I've marked with "TODO". In short this is a good draft but not a final one by any means.

Anyway, those are my perspectives -- what about yours?

[aborruso]

Hi @johnkerl this is great, thank you!!

Some points:

• MkDocs is a really great tool. I'm using it for my Italian Miller guide (still too short to show you);
  • there is a really great theme which I like a lot more than RTD one. It's Material https://squidfunk.github.io/mkdocs-material/
  • I have discovered this theme studying csvtk documentation https://bioinf.shenwei.me/csvtk/
• a great thing, using something like MkDocs, is to collaborate on documentation because it is very very easy;
• I love that you have inserted more, and gentler, up-front material for non-experts. I am that kind of user, and I think it's important that a gem like Miller also contains this type of content;
• if you don't change the style, there are things like this to fix

Thank you very much

[johnkerl]

I have some points about this new Material version. Can I add it here in the next days?

Yes, please! :D

[johnkerl]

I love this project and I try to give my contribution. I'm sorry I can't do great things.

@aborruso you have done countless great things! :D

[trantor]

As a feedback, I do like the theme quite a bit.
On another note, I've seen a lot of activity over the last few weeks and I've been quite annoyed with myself for not having being able to lend a hand in any way given my daily activities.
Do you need some help with proofreading the docs (I am very good at spotting inconsistencies) or some other kind of feedback on code @johnkerl , by any chance?

[johnkerl]

@trantor yes I would love that!!!!!! Whatever you have time for. :)

One caveat ... I'm midway through working through https://github.com/johnkerl/miller/blob/main/docs6/docs/proofreads.txt ... so if you don't mind looking and finding things that I may already be editing, go right ahead! Or if you'd rather be more careful about duplication of effort, maybe hold off a couple weeks -- totally up to you.

Code -- similarly; there are many things I feel 'ready for review' in Miller 6 but lots of things marked TODO. Any feedback you have would be great. The caveat for docs6 is that known to-do items are in the single docs6/docs/proofreads.txt; for the Go code, the to-do items are distributed throughout the source code with either TODO or xxx.

[NikosAlexandris]

@aborruso I experimented a bit and the CSS-wrap method will work fine :)

When I select a line-wrapped bit in the browser and copy, then paste into the terminal, it pastes as a single line (as desired):

Byt the way, the colorised output looks awesome -- really useful and easier to read on-screen, given the dark terminal background.-

[NikosAlexandris]

Looks awesome! May I propose a Glossary page too? A reference to have acronyms fully spelled out and descriptive sentences. A tutorial and examples are really the Thing. Yet a glossary is a quick overview to grasp the terminology. And there are quite some keywords there to list.

[NikosAlexandris]

Really @johnkerl the new docs is just fantastic -- makes reading so much more enjoyable.

[johnkerl]

Awesome @NikosAlexandris thank you!! :D

[johnkerl]

@NikosAlexandris https://johnkerl.org/miller6/glossary/index.html is a start ... please let me know what you think of the general direction.

[NikosAlexandris]

I needed this, quite many times! :-)

Just my personal preference of learning/understanding, is to really have an acronym fully spelled out in the glossary. Like, it took me days to finally understand Domain-Specific-Language. And, maybe even still today I am confused about the D in the DKVP. Though, it's clear to me now, and at least, that KVP stands for key-value-pair :-)

[johnkerl]

Sorry @NikosAlexandris -- DKVP is for 'delimited key-value pairs' -- fixed in the glossary -- thank you! :)

[johnkerl]

Also @NikosAlexandris was it you that asked about multicore -- ?

https://johnkerl.org/miller6/cpu/

[NikosAlexandris]

Maybe indirectly in #637 (?) -- Thank you Anyway :-)

[johnkerl]

Found it, it was a response in the survey :)

[NikosAlexandris]

Additional Markdown tables for the data-conversion keystroke-savers: https://gist.github.com/NikosAlexandris/65a88b4df50c33f16ff2dba16c8ef474

[johnkerl]

@NikosAlexandris https://johnkerl.org/miller6/reference-main-flag-list/index.html#format-conversion-keystroke-saver-flags 💥

[aborruso]

@NikosAlexandris https://johnkerl.org/miller6/reference-main-flag-list/index.html#format-conversion-keystroke-saver-flags 💥

Fantastic!!!

[NikosAlexandris]

The Data-conversion keystroke-savers section seems currently to not use a new/Go version of miller, so mlr help format-conversion fails.

[johnkerl]

@NikosAlexandris the docs are produced by code on a branch which I haven't merged to head yet ... sorry for the confusion. :^/

[NikosAlexandris]

If such comments cause you to waste more time, rather than save and be useful, let me please know -- I do know it's stuff under construction. I will be more conservative then to report such minor issues.

[johnkerl]

@NikosAlexandris not at all!! This is all super-helpful, more please! :) I let you about the branch because I felt bad about having confused you . :)

[NikosAlexandris]

Great. I can create mini Pull-Requests also, hopefully saving some of your precious time.

[NikosAlexandris]

@johnkerl I read through https://johnkerl.org/miller6/programming-language/#out-of-stream-variables-begin-and-end and I was quite pleased with the new style, and (obviously after having spent some time with Miller by now) started to digest things faster. One thing I miss reading, to know it can work or even how it will work, is using put with some existing field names to create a/some new field/s, then use the newly created fields in a subsequent statement/s. Like, what I am trying to do in #651 (comment) but just a simple case.

The intro tells us, and shows, how to write a simple statement, that multiple statements are to be separated by ; and that multi-line statements are possible too. Then, except for the out-of-stream variables (which indeed and naturally appear twice or more (?) in a statement), there is no single simple example that demonstrates that a new field can be reused further in a subsequent statement. I think such a simple example deserves to be presented in the intro.

Example:

➜  echo 'a=3' |mlr.go --d2c put '$b = $a - 1 ; $c = $a - $b'
a,b,c
3,2,1

Sorry if I missed to find it or please ignore if you see not fit for it in the intro.

---

ps- Off-topic on Miller syntax + GitHub: would be cool to be able to apply Miller's syntax highlighting in issues and elsewhere in this repository via:
```miller
..
```

[johnkerl]

@NikosAlexandris how is this?

https://johnkerl.org/miller6/10min/#computing-new-fields

[NikosAlexandris]

Feels natural to have it right after the 1st put example.