Add web OSINT

[hasninbl]

Dear Taranis-NG Team,
First of all, thank you for this platform - Taranis-NG.
Secondly, I want to add OSINT source web, https://www.cvedetails.com/cisa-known-exploited-vulnerabilities/kev-1.html
but I didn't achieve the collect of the information .

[milankowww]

Hello, try the attached file
securityscorecard-cisa-known-exploited.json
or manually add the source using the following selectors:

• starting URL: https://www.cvedetails.com/cisa-known-exploited-vulnerabilities/kev-1.html
• landing page
  • next page button: xpath://*[@id=\"pagingt\"]/a[i[contains(@class, \"fa-chevron-right\")]]
  • article link: css:h3 > a
• article page
  • title: css:.page-header
  • short description: css:#cvedetailssummary
  • full text: css:main or pick some other part that you wish to crawl
  • published: xpath://div[span[contains(@class, \"ssc-text-secondary\") and contains(text(), \"Published\")]]

[hasninbl]

Hello, I imported the attached file, but the Taranis-NG did not display any information.
for the second way manually, can you explain more:

thank you

[hsninbil]

Dear @milankowww ,

you find below the logs of the error:
Requesting title page: https:// https://www.cvedetails.com/cisa-known-exploited-vulnerabilities/kev-1.html
collectors_1 | [2024-03-28 12:57:53 +0100] [810] [DEBUG] [None] COLLECTOR web 'SecurityScorecard CVEdetails - Known Exploited Vulnerabilities': Error obtaining title page
collectors_1 | [2024-03-28 12:57:55 +0100] [1] [ERROR] Worker (pid:1715) was sent SIGTERM!
collectors_1 | [2024-03-28 12:57:55 +0100] [810] [DEBUG] [None] Traceback (most recent call last):
collectors_1 | File "/app/collectors/web_collector.py", line 495, in __browse_title_page
collectors_1 | browser.get(index_url)
collectors_1 | File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 356, in get
collectors_1 | self.execute(Command.GET, {"url": url})
collectors_1 | File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute
collectors_1 | self.error_handler.check_response(response)
collectors_1 | File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
collectors_1 | raise exception_class(message, screen, stacktrace)
collectors_1 | selenium.common.exceptions.InvalidArgumentException: Message: Malformed URL: URL constructor: https:// https://www.cvedetails.com/cisa-known-exploited-vulnerabilities/kev-1.html is not a valid URL.
collectors_1 | Stacktrace:
collectors_1 | RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8
collectors_1 | WebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:192:5
collectors_1 | InvalidArgumentError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:383:5
collectors_1 | GeckoDriver.prototype.navigateTo@chrome://remote/content/marionette/driver.sys.mjs:837:11

Can you help me for fix it ? and if I have source json how can I add it i.e. https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities_schema.json ?
Thank you in advance

[hasninbl]

@milankowww
any advise about this request?

[Progress1]

It looks you have put wrong link:
Requesting title page: https:// https://www.cvedetails.com/cisa-known-exploited-vulnerabilities/kev-1.html
Do you see https:// two times?
Check this field.. Maybe you put sting divided with enter? (delete field, edit and save link again)