be able to scan also pages where the button 'Termin berlinweit suchen' is not available

[larswillrich]

the current version works only well for services you can do in any 'burgeramt' in berlin.
e.g. 1) https://service.berlin.de/dienstleistung/120686/

Nevertheless, for some services it's limited to only one office and the button is not available like in the example above. Anyway there are buttons available e.g.
multiple choses: 2) https://service.berlin.de/dienstleistung/318962/#wohin
one single button but with the information of the location in the URL:
3) https://service.berlin.de/dienstleistung/318962/standort/326487/

If I try to use one of the pages 2 and 3, I get an error message, that the button cannot be found.
DEBUG buergeramt-sniper.loader - Loading :get https://service.berlin.de/dienstleistung/318962/standort/326487/ 12:23:57.964 [main] DEBUG buergeramt-sniper.loader - {} 12:23:58.433 [main] ERROR buergeramt-sniper.api - Unable to find out the calendar page for https://service.berlin.de/dienstleistung/318962/standort/326487/ 12:23:58.436 [main] DEBUG buergeramt-sniper.scheduler - Stopping scheduler... Exception in thread "main" java.lang.IllegalArgumentException: No implementation of method: :take! of protocol: #'clojure.core.async.impl.protocols/ReadPort found for class: nil, compiling:(/private/var/folders/bg/gfpy0ytn2_l87xwfsg84tn200000gn/T/form-init6551450872928220009.clj:1:125) at clojure.lang.Compiler.load(Compiler.java:7239) at clojure.lang.Compiler.loadFile(Compiler.java:7165) at clojure.main$load_script.invoke(main.clj:275) at clojure.main$init_opt.invoke(main.clj:280) at clojure.main$initialize.invoke(main.clj:308) at clojure.main$null_opt.invoke(main.clj:343) at clojure.main$main.doInvoke(main.clj:421) at clojure.lang.RestFn.invoke(RestFn.java:421) at clojure.lang.Var.invoke(Var.java:383) at clojure.lang.AFn.applyToHelper(AFn.java:156) at clojure.lang.Var.applyTo(Var.java:700) at clojure.main.main(main.java:37) Caused by: java.lang.IllegalArgumentException: No implementation of method: :take! of protocol: #'clojure.core.async.impl.protocols/ReadPort found for class: nil at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:554) at clojure.core.async.impl.protocols$eval8088$fn__8089$G__8079__8096.invoke(protocols.clj:15) at clojure.core.async$_LT__BANG__BANG_.invoke(async.clj:92) at buergeramt_sniper.core$_main.doInvoke(core.clj:121) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.lang.Var.invoke(Var.java:379) at user$eval16670.invoke(form-init6551450872928220009.clj:1) at clojure.lang.Compiler.eval(Compiler.java:6782) at clojure.lang.Compiler.eval(Compiler.java:6772) at clojure.lang.Compiler.load(Compiler.java:7227) ... 11 more

I would be more then happy to see a feature of having this also enabled.
Which of the both webpages 2 or 3 I mentioned is more easy to integrate?

If you point me to the code place, I may can try to work on a PR.

Looking forward to hear from you! Great work and really helpful until now!

[dryewo]

Hello Lars!

I'm glad this software is still useful for somebody and I'm happy to help you to improve it.
I myself unfortunately stopped using and maintaining it some time ago, as well as I stopped doing Clojure.

It looks like the error comes from the fact that the initial page parser cannot extract the URL of the calendar from the service root page. The page is parsed by this function:
https://github.com/dryewo/buergeramt-sniper/blob/master/src/buergeramt_sniper/scraper.clj#L56
It has a specific DOM path (to the button) where it expects the URL to be. It seems that on pages with other button the DOM path looks different, that's why this function cannot be found.

One way to improve it would be to make the function more flexible and try multiple DOM paths to find one that works - you can start with that and see if it succeeds further. You can use dry run mode in order to avoid getting an actual appointment while debugging.