Update Appendix A for finance-quote 1.63

[pkryger]

A few weeks ago @bpschuck asked for help with updating Gnucash for recent version of finance-quote. This is a complitentary PR to Gnucash/gnucash#2033.

I decided to move modules that are not a part of the list into a new section for deprecated sources. IIUC this is still usefull information for a case when an old .gnucash file (or a database) has been configured with one of the deprecated sources.

[jralls]

I think this needs to be in the wiki instead of the manual: It changes a lot and we're terrible at keeping it up to date.

[fellen]

I think this needs to be in the wiki instead of the manual: It changes a lot and we're terrible at keeping it up to date.

I had created this appendix before we started the wiki.
It is easy done, while updating engine/gnc-commodity.cpp like in Gnucash/gnucash#2033.

OTOH one could think about adding more infos like the URL, API key requirement, … already in the documentation of the perl sources and populate corresponding fields in QuoteSourceList.

[jralls]

It is easy done, while updating engine/gnc-commodity.cpp like in Gnucash/gnucash#2033.

We haven't done a great job of keeping gnc-commodity.cpp up to date either.

[fellen]

A very good start! Please check my notes.

[Edit:] Should we add version infos like introduced in FQ version x.y?
Perhaps we should add classifications like They offer shares and/or funds and/or ForEx, Indices, Options, …

[fellen]

Region missing, usually the TLD of the URL, mapping com, mil, … to us.
Perhaps we should add a term like multi or supraregional to the global players?

[pkryger]

Sorry, I forgot to replay earlier.

I intentionally left the region out here and in a few other places (marketwatch, stockdata, twelvedata). This was to follow what has been done in financeapi and, to some degree, goldmoney. I guess adding some extra term here should be followed in other places, right?

As for using term multi: I'd refrain from doing that, as there are already "Multiple Quote Sources" and this may cause confusion. global perhaps?

[jralls]

The TLD isn't useful, there are too many supposedly country TLDs that have been commercialized. supranational seems a bit legalistic and global implies a bit wider scope than is the case. How about multnational, a term widely used in the US for companies doing business in more than one country?

[fellen]

@jralls I wonder if we should add storages for this new envars in preferences.

[pkryger]

Let me know when that happens and I will update this document.

[pkryger]

@jralls @fellen thank you for taking a time and reviewing this.

I applied most of the suggestions from comments (and resolved them). I did that as individual commits to help with spotting what has actually been changed since the previous read. Note that I also changed a structure, which rendered another large change 🤷‍♂️.

Please let me know if this has reached a state good enough start Wiki port. Also, could you please let me know what's need to be done to perform such a move.

[jralls]

Wiki port...

Do you have a wiki account already? If so you can just add the list here. If not you'll have to request an account.

[fellen]

Wiki port

To avoid redundancy you should IMHO only update the content of <SyntaxHighlight lang="console">

[jralls]

To avoid redundancy you should IMHO only update the content of

Since that's the actual output from gnucash-cli -Q info, no.

To avoid redundancy we're going to remove this appendix because it's too hard to keep it up to date.

[bpschuck]

On 10/19/24 04:20, Przemysław Kryger wrote: As for using term |multi|: I'd refrain from doing that, as there are already "Multiple Quote Sources" and this may cause confusion. |global| perhaps?

PK, I believe you are losing sight of what the F::Q code refers to them as, "failover" methods. In some ways, these are "virtual" methods. Sort of like round-robin DNS load balancing. Hypothetically, I have FOOB defined to use source "nasdaq". F::Q will try to use the various modules where "nasdaq" is advertised. Either one of the methods that "nasdaq" is an alias for is successful, or F::Q tries all of them (currently 8) an finally returns with a success=0. A module where the method can also be called using "nyse", "nasdaq", "usa", or "amex", isn't "global". I would change the table to something like the following, after of course a quick description of how they are used. Failover Methods Failover Method - Possible Source Method nyse - alphavantage, financeapi, fool, googleweb, ... nasdaq - alphavantage, financeapi, fool, googleweb, ... india - bseindia, indiamutual, nseindia . . . Bruce S.

[bpschuck]

On 10/19/24 10:38, John Ralls wrote: To avoid redundancy we're going to remove this appendix because it's too hard to keep it up to date.

The bane of my existence at the day job. Supporting some legacy applications that this was the obvious choice of long gone developers. To paraphrase, "we're not going to write documentation because it would be too hard to keep up to date"... Hence we've had some outages where we had to spend hours reverse engineering before we could identify the issue at hand. Seriously though, once it is pretty much caught up it should be relatively easy to keep up to date. As long as there is an effort to. But like you said earlier John, perhaps moving it to the GnuCash Wiki is a much better alternative. Then again, it is something that should be more of F::Q doc, and not GnuCash. Not exactly something that is fit for POD, and the GitHub supplied Wiki isn't exactly one that is full featured. While I did recently register finance-quote.org, I'm not going to pay for a hosting service out of my own pocket. Hell, I'm not sure how much longer I'll be playing F::Q gatekeeper... Bruce S.

[jralls]

@bpschuck

I believe you are losing sight of what the F::Q code refers to them as, "failover" methods. In some ways, these are "virtual" methods. Sort of like round-robin DNS load balancing.

It's not about that, it's about sources like AlphaVantage and Yahoo_JSON that provide quotes from more than one country. What F::Q calls failover sources are listed in a gnucash array named multiple_quote_sources, hence @pkryger's aversion to "multi" for the likes of AlphaVantage.

Then again, it is something that should be more of F::Q doc, and not GnuCash.

There's no reason it can't be in both. It could be in POD but few GnuCash users are going to read POD. The Github-provided wiki might be limited compared to MediaWiki, but it's certainly good enough for this purpose. On the other hand you're welcome to use GnuCash's wiki (you've already got an account) and place links to it from Finance::Quote.

[fellen]

Today I found a few minor issues — mostly from our ruleset.

[fellen]

@bpschuck Should we add a few words about your policy on removing and perhaps reenabling?

[fellen]

Questions: Do they all support the ISIN as symbol or are they using different national IDs like german WKN?
If they use ticker symbols are they defined somewhere globally or has each market its own list?
A few Examples

[pkryger]

I think the content of this PR is in a reasonable condition to start porting it to GnuCash Wiki. I will close this PR once I am done with the port and I will provide a link from Wiki to here for future archeological excursions.

[fellen]

Just some more polishing:

[pkryger]

So I exported contents of this chapter into Wiki, please see https://wiki.gnucash.org/wiki/Online_Quotes#Finance::Quote_Sources and https://wiki.gnucash.org/wiki/Online_Quotes#Details_of_Some_Finance::Quote_Sources.

I am closing this PR and will open a new one, to remove this file.

The gist of how the export was made

I used automated export with:

pandoc -f docbook -t mediawiki ../C/manual/tips-appendix.docbook -o tips-appendix.wiki

And later edited it manually to (note: regexps as in Emacs):

• increase nesting level s/^\(=+\)\( [a-z0-9 :-]+\)\(=+\)"/\1=\2\3=/,
• converted most of links to inline s/\[\(https?://.+\)\]/\1/ (note: not all!),
• updated API Keys procedures (manually),
• compressed tables
  • removed empty lines s/^\([|!].*\)\n\n/\1\n/
  • converted to single row in markup, where applicable:
    • first 4 columns: s/^([!|])( .)\n([!|])( .)\n([!|])( .)\n([!|])( .)/\1\2 \3\3\4 \5\5\6 \7\7\8/`
    • then 3 columns: s/^\([!|]\)\( .*\)\n\([!|]\)\( .*\)\n\([!|]\)\( .*\)/\1\2 \3\3\4 \5\5\6/

[pkryger]

Closing - this section has been exported to https://wiki.gnucash.org/wiki/Online_Quotes#Finance::Quote_Sources

[fellen]

Several reasons

1. Thanks to the Great Firewall users in CN can not access wikis.
2. primary the appendix is documenting tables in GnuCash's engine.
3. Until now part of the documentation policy was: the Docu team moves parts from the wiki to docs and replace them by a link. Common users can still add new developments to the wiki. Do we really want to revert that?

[jralls]

@fellen, knock it off. There is no "Doc Team" and you darn well know it.

The fact that there are tables for this in GnuCash is an implementation detail that has no business being in the Manual.

The PRC's blocking of Wikipedia is well documented, but that article says nothing about blocking all wikis. @xuxinhang are you able to access https://wiki.gnucash.org/wiki/Online_Quotes?

Besides, it's pretty much impossible to keep this up to date considering that F::Q is on a different release cadence from GnuCash and the various websites that F::Q depends upon do breaking changes pretty much at random.

[fellen]

The first column GnuCash Name in https://wiki.gnucash.org/wiki/Online_Quotes#Finance::Quote_Sources only exist after an entry was added in gnc-commodity.cpp.
An wiki update without a change in gnc-commodity.cpp is pointless.

Perhaps we can get a recent useful table by Doxygen? File, existing page QuoteSourceType

[jralls]

Perhaps we can get a recent useful table by Doxygen? File, existing page QuoteSourceType

For who? The Doxygen docs are for developers, who care about the structure and the functions. It's rare that the values would be important for anything other than testing.

[xuxinhang]

The PRC's blocking of Wikipedia is well documented, but that article says nothing about blocking all wikis. @xuxinhang are you able to access https://wiki.gnucash.org/wiki/Online_Quotes?

The wiki.gnucash.org is accessible via my ISP.

China's Firewall works on the black-list containing DNS names of some famous companies, so it doesn't care which software the website is hosted on. In fact, most of overseas websites are accessible to Chinese visitors, excluding popular social media and news media websites.