Uploading documentation to Hackage, hyperlinking source

[ghost]

Hackage from time to time fails to build docs for one reason or another. Would it be possible to add a command to upload docs with hyperlinked source (I think it's a basic courtesy for documentation readers, but stack haddock doesn't do that by default, and I can't find any option to make it do this), and upload it to Hackage using the login information used for uploading packages?

[borsboom]

Source will be hyperlinked if the hscolour executable is available (stack build hscolour --copy-bins is an easy way). I wonder if a warning to this effect might be a good idea.

The documentation generated by stack haddock isn't quite in the right format for hackage (the links to other packages aren't right, since it generates relative links that work locally), but it might be possible to make links that work locally as well as on hackage with a slight adjustment of the location of the haddocks (they'd need to go in an extra docs subdirectory).

[borsboom]

In terms of CLI for this, it probably should be possible to upload haddocks independently of, or together with, uploading a package. Making this part of stack upload is probably the right place. Perhaps stack upload --with-haddock would upload the package and haddocks, whereas stack upload --only-haddock would only upload the haddock.

[jdnavarro]

👍

[phadej]

Few resources:

• https://github.com/ekmett/lens/blob/master/scripts/hackage-docs.sh
• https://github.com/ndmitchell/neil

I'll might have time to look into this issue

[phadej]

I tried this one way and another, and cleanest approach feels to be to do something like:

cabal configure --builddir .../dist-hackage ...
cabal configure --builddir .../dist-hackage haddock ...
tar --format=ustar .../dist-hackage/doc ... # actually programmatically so can have right directory structure without moving files around

and have this functionality as build --hackage-docs

Is this approach ok to @borsboom ?

[phadej]

So far I've been using https://github.com/phadej/binary-orphans/blob/master/hackage-docs.sh script

[borsboom]

The docs generated by stack haddock are almost in the right format already, they just need a tweak to the paths and href URLs. So I think the best approach is to make stack haddock output have the right format in the first place, and then add the functionality to stack upload to tar and upload the output of stack haddock.

[denibertovic]

@borsboom it would be nice if "--with-haddock" was actually the default.

[cblp]

@denibertovic, no, Hackage must work, and --with-haddock is only a crutch.

[denibertovic]

@cblp I don't understand. If we upload the docs with the package then hackage doesn't need to generate if for us. Or did I misunderstand how this works?

[cblp]

@denibertovic, I mean it is normal if Hackage builds docs, and it is not if user has to build and upload docs.

[denibertovic]

Well that's not much of a reason... :/ I mean we can change that.... I don't see a reason why not.

[tfausak]

It would be nice if Hackage reliably built documentation. Unfortunately it doesn't. I frequently see documentation either fail to build or take several days to build. It's been like this for years, so it seems like waiting for Hackage to work properly isn't a realistic option.

[denibertovic]

I agree with this. And I'm sure we could get the hackage folks on board here and figure out a solution that works for everybody.

[cocreature]

I’ve taken a look at the steps that seem necessary and it doesn’t look too hard:

1. Add an option to generate haddock suitable for hackage. cabal-install 1.24 has a --for-hackage option so it would make sense to use the same name. This option needs to change html-location and contents-location in Execute.hs to 'package/$pkg-$version/docs and /package/$pkg-$version. These docs should go in a separate directory because a) they are not useful locally so they should not override the existing docs and b) it makes it easy to put them in a tar archive.
2. Add an option to upload to also upload the docs. This should be the default imho, since the whole idea of letting hackage build docs is silly when you can just build them locally and upload them.

@borsboom Since you are assigned to this: Do you have any plans to tackle this soon? IMHO this is one of the most important issues since the docs situation on hackage is catastrophic atm. Otherwise I would give it a shot, but I’m short on time and don’t know the stack codebase very well so it could take some time.

[luigy]

I’ve taken a look at the steps that seem necessary and it doesn’t look too hard:

Indeed, just wanted to add that neil is kept updated and we should just borrow/adapt what it is doing :)
http://neilmitchell.blogspot.co.uk/2014/10/fixing-haddock-docs-on-hackage.html

[meditans]

This would a very useful feature to have. Currently, the user has to install cabal, an external ghc with usual settings, and use either the script or the neil tool. +1

[idontgetoutmuch]

I would like this feature.

[ChrisPenner]

Any news on this issue? I still have pretty common problems with haddocks and would love a stack upload --with-haddock 😄

[colonelpanic8]

What's the status of this issue? What needs to be done?

[ChrisPenner]

BTW I've adapted a script to allow me to just run haddock-up in a project and it'll build and upload docs; not a perfect solution but it's helped me so far.

You can grab it here if you like: https://github.com/ChrisPenner/dotfiles/blob/master/bin/haddock-up

[dbaynard]

@IvanMalison I suspect @phadej is no longer working on this, so I'll change the label.

[ProofOfKeags]

How on earth do stack users upload packages to hackage today if the stack process can't correctly generate the docs? Do people just use cabal? It seems like I have to be missing something obvious.

[tfausak]

I have published many packages using both Cabal and Stack. I don't think I've ever manually uploaded documentation.

[ProofOfKeags]

I have published many packages using both Cabal and Stack. I don't think I've ever manually uploaded documentation.

For some reason when I use stack to do the dist/upload the docs do not show. Not sure what I'm doing wrong.

[yutotakano]

I tend to manually upload the documentation afterwards since mine has some dependencies not in the system -- and even then, because of this issue, the hyperlinks that are created with "--haddock-hyperlink-source" are wrong and do not work when uploaded. Hopefully this issue is resolved sometime soon, as the route to solution by cocreature looks solid (admittedly now 6 years ago).

[mpilgrem]

Closing, as I think this is addressed by #6270 (stack haddock --haddock-for-hackage).