Merge branch 'update-news-for-1498' of github.com:ArneBab/fred-stagin…

…g-1 into next

NEWS.md

@@ -5,38 +5,235 @@ next:
 
 1498:
 
-- Support Schema hypha[net]
-- Update Sharesite to 0.5.1
-- Deprecated custom datetime handling class CurrentTimeUTC
-- ⚠ The Windows installer is currently not signed!
-- Shorten description of sharesite freesite for english
-- gzip: replace test workaround by fixing the output of the compressor — thanks to Bombe for the SingleOffsetReplacingOutputStream!
-- replace GregorianCalendar with java.time.OffsetDateTime in CurrentTimeUTC — thanks to Veniamin Fernandes
-- CSS: Support word-wrap: anywhere and CSS selector focus-within.
-- Move UPnP2 to normal plugins. It does not seem broken, but UPnP does
-- Move Library plugin to advanced plugins because new users tend to get lost with it
-- html-filter: allow summary and details html element. Thanks to naejadu
-- 💄 Add a bit of styling to the plugin list (for winterfacey) — thanks to Bombe
-- 🔥 Remove option that to load plugins from central server — thanks to Bombe!
-- fix: invalid max store size showed bytes with GiB suffix
-- polish: show datastore size warning with GiB suffix
-- Refine ClientRequestSelectorTest, PersistentJobRunnerImplTest, SplitFileFetcherStorageTest, and SplitFileInserterStorageTest — thanks to Veniamin Fernandes
-- Change deprecated jcenter() maven repo to mavenCentral() — thanks to Veniamin Fernandes
-- 🐛 Fix JarClassLoader’s ability to work with ServiceLoader — thanks to Bombe!
-- refactor the browser warning wizard page
-- fix the flag size of nepal — thanks to Percept0r@NYZkOs7eQ…!
-- Switch swiss flag to civil and state ensign — thanks to Percept0r@NY
-- bookmarks: Add Opennet SeedNodes stats site
-- Reorder starting bookmarks: FFS → clean spider → Index of Indexes
-- Re-order default software bookmarks by ease of use from fproxy
-- Add generate media site to the default bookmarks
-- Show the noderef in basic-mode: it is now robust enough
-- Added tests for PebbleUtils — thanks to Bombe!
-- 🐛 Fix NPE when subsets are not initialized — thanks to Bombe
-- reduce logging for too many excluded sub-arrays thanks to Hiina
-- disable setting for new-load-management (NLM broke nodes)
-- add utility to disable a config option, thanks to Bombe
-- merge debian package as default build action thanks to DC*/desyncr! This resolves one of our high impact tasks.
+# Short changelog:
+
+This release resolves the last blocker for Freenet / Hyphanet 0.8 by
+providing an official Debian package. Additionally it optimizes the
+networking and data transfer core and provides many improvements for
+website authors and user experience.
+
+
+Starting with this release, Freenet / Hyphanet has an official Debian
+package built automatically via github actions. This was the most
+important [high-impact-task][] and the last release blocker of version
+0.8 in our [Roadmap][]. Big thanks go to DC*!
+
+With this finally realized, the next step is to get in contact with
+the many privacy focussed distributions which build on Debian to make
+`hyphanet-fred` available where it is most important. Once this is
+done, tools which build on Hyphanet — like FMS, but also jSite and
+tools from pyFreenet — can be packaged to work out of the box, using
+Hyphanet as an ordinary background service. That’s a step towards
+Hyphanet as decentralized, privacy-preserving communication backend for
+other applications.
+
+Another step towards this is accepting the Schema hypha[net] to
+simplify writing browser extensions that forward hypha:-links to
+Hyphanet.
+
+
+The networking layer was optimized significantly. Searching packet
+types is often stopped early and common or cheaper checks are done
+before less common or time-consuming checks. This gives significant
+reductions of CPU load, especially for very fast nodes.
+
+Juiceman fixed a bug limiting MTU to 1280 where not needed.
+
+And recently failed and data not found cooldown times were reduced to
+5 minutes and 3 minutes, reducing one of the big annoyances when
+accessing a site quickly after upload.
+
+On the data transfer layer, healing was optimized. After 1495 strongly
+increased the amount of healing to keep large files available for
+longer, 1498 specializes healing to keys close to the node location.
+This reduces healing per file, but improves privacy, because healing
+inserts are then more similar to forwarding — they mostly send data
+close to the nodes location — and it reduces the network load of
+healing, because the specialized healing inserts need fewer hops to
+reach the optimal storage location in the network.
+
+
+In addition to these changes deep down, there are a number of directly
+visible improvements.
+
+The plugins KeepAlive and Sharesite are updated (the latter now uses
+the new Night Zen Garden style). The UPnP2 plugin is now visible in
+simple mode. It can replace UPnP and should work better. On the
+flipside the Library plugin is moved to advanced plugins, because it
+does not work reliably enough.
+
+The plugin list is easier to navigate by removing the defunct option
+to download plugins from the clearnet and by adding better styling.
+Downloading from the clearnet was an unnecessary privacy risk since
+we’ve been bundling essential plugins with the installer for a few
+years now.
+
+
+The noderef for friend-to-friend connections is shown in simple mode
+again, because it is robust enough with the changes in recent years.
+This should remove a barrier to adding direct connections and enabling
+fully confidential messages between friends.
+
+There are new configuration options to allow connecting via local
+services. That’s a step towards making it easy to add a second layer
+of security, for example confining connections to a local network.
+Thanks goes to s7r for these changes!
+
+When bandwidth detection fails, the upload bandwidth now defaults to
+160KiB/s. Also the NLM config is now disabled statically. This was a
+testing setup which could still be active in old nodes, but it would
+break connectivity nowadays.
+
+
+The default bookmarks include the Opennet SeedNodes statistics,
+the generate media site to create decentralized streaming sites, and
+the high-impact-tasks. The bookmarks are also re-ordered to be a
+better match for newcomers. Starting category: first steps, clean
+spider, Index of Indexes. For the software category ordered by ease of
+use from fproxy.
+
+For website authors, more CSS elements, selectors and combinators
+(`:checked`, `word-wrap: anywhere`, `focus-within`, `^=`, `$=`, `*=`,
+`>`, `+`, `~`) and additional HTML elements (`summary`, `details`,
+`<meta name="Viewport"...>`) are available. This strongly expands the
+possibilities of websites authors in Hyphanet, because Javascript or
+webassembly are no viable options in an environment where a privacy
+breach could put people at risk. We’ve seen with Java applets, that
+untrusted code will always break out of its containment. The CSS
+improvements in contrast provide a safe way to enable limited
+interactivity.
+
+Streaming support via m3u lists was improved to allow accessing
+segments of up to 200MiB.
+
+And using `-1` as version in a USK now properly finds version `0`, if
+this is the only existing version.
+
+
+There were a number of Java 21 fixes, including all our tests (thanks
+to Bombe!), and improvement to the github actions (thanks to
+AHOHNMYC).
+
+In addition to that there was a lot of polish. Bert Massop and
+Veniamin Fernandes replaced our homegrown CurrentTimeUTC with modern
+Java options. Alex fixed the pronoun used in strings. Bombe added
+getters for all direct field access in the node. Hiina reduced logging
+level of store warnings so no unneeded backtraces are created for node
+with large stores and Juiceman updated code to use more modern
+structures.
+
+Time-dependence of compressor selection was removed. This caused
+non-determinism for inserts and could cause keys to be
+non-reproducible on systems with faster or slower network.
+
+
+And finally the new [exe signing workflow][] we built to fulfill the
+requirements of SignPath, our new windows installer signing provider
+for the upcoming releases, runs the [verify-build script][] on every
+release to ensure that the jar we release has actually been built from
+the sources. This provides a second safety net, in addition to
+anonymous users running the script and posting the results (thanks to
+all who did this — please keep it up, otherwise people have to fully
+trust github). The release is not yet byte-by-byte reproducible,
+because the jar MANIFEST defines among other info the exact java
+version used to compile it, and the java version available differs by
+distribution and time, so it would get harder over time to verify the
+build.
+
+
+A special thanks goes to Bombe for many careful reviews!
+
+[high-impact-task]: https://github.com/hyphanet/wiki/wiki/High-Impact-tasks
+[Roadmap]: https://github.com/hyphanet/wiki/wiki/Roadmap
+[exe signing workflow]: https://github.com/hyphanet/sign-windows-installer
+[verify-build script]: https://github.com/hyphanet/scripts/blob/master/verify-build
+
+# Full changelog:
+
+
+## high impact tasks
+merge debian package as default build action thanks to DC*/desyncr! This resolves one of our high impact tasks.
+
+## Plugins
+Update KeepAlive to commit 86e47a101f26fd1d3be0437681a043aa4ae3f22c
+Update Sharesite to 0.5.1
+Move UPnP2 to normal plugins. It does not seem broken, but UPnP does
+Move Library plugin to advanced plugins because new users tend to get lost with it
+💄 Add better styling to the plugin list in winterfacey to make it easier to understand at a glance — thanks to Bombe
+🔥 Remove option to load plugins from central server — thanks to Bombe! This was an unnecessary privacy risk, since we’re already bundling essential plugins with the installer, and it made plugin handling harder to understand.
+
+## Bookmarks
+Add high-impact-tasks to bookmarks
+Add generate media site to the default bookmarks
+Add Opennet SeedNodes stats site
+Reorder starting bookmarks: FFS → clean spider → Index of Indexes
+Reorder default software bookmarks by ease of use from fproxy
+Disable activelink for Index of Indexes (workaround, because it fails)
+
+## Optimize networking and transfer layer
+break early when condition is met — thanks to Juiceman
+Check the HashCode before equals. This saves ~20% method-runtime.
+Re-order or’ed MessageFilters so the most likely is checked first
+specialize healing to keys close to the node
+fix healing decision: do not divide 0-1 by MAX_VALUE — thanks to Bombe for the review!
+Reduce recently failed and data not found wait times
+
+## Filters
+CSS: Fix: checked only the first char of the key part of CSS selectors, Add test that would catch too lax filtering.
+CSS: Support pseudo-element checked. This enables limited interactivity via CSS.
+CSS: Support the attribute selectors ^= $= *=, Add tests.
+CSS: Support Combinators > + and ~, add test for ~ and simplify the implementation
+CSS: Support word-wrap: anywhere and CSS selector focus-within.
+HTML: allow summary and details html element. Thanks to naejadu
+HTML: accept <meta name="Viewport" ...>, thanks to torusrxxx
+
+## Constants
+Show the noderef in basic-mode: it is now robust enough
+
+## Configuration
+accepting localhost in NodeIPPortDetector and allowBindToLocalhost configurable — thanks to s7r!
+Provide static methods for simpler boolean config creation
+Increase default bandwidth to 160KiB upload, when detection fails
+disable setting for new-load-management (NLM broke nodes)
+add utility to disable a config option, thanks to Bombe
+
+## Misc
+add m3u-player insertion test: is added at end of body
+[CI] Update actions, fix actions cache
+♻️ add and use getters and setters for access to node fields
+Increase max transparent passthrough to 200MiB links in m3u-lists.
+Remove time-dependence of compressor selection. This caused
+non-determinism for inserts and could cause keys to be
+non-reproducible.
+improve date object construction in CurrentTimeUTC.get()
+Support Schema hypha[net] to simplify writing browser extensions that forward hypha://-links to Hyphanet.
+polish: show datastore size warning with GiB suffix
+Remove hash generation to native big integer to reduce dependencies. This had come in when merging an old pull request and added a new dependency without need.
+
+## Polish
+Replace indexOf with .contains()
+Change more string comparisons into .isEmpty() checks
+Capitalize L in literal longs
+Add missing Global flag to GetFailed FCP message. Thanks to oishii and Bombe!
+Fix links to mailing lists and IRC in CONTRIBUTING.md — thanks to Juiceman!
+Deprecated custom datetime handling class CurrentTimeUTC
+replace GregorianCalendar with java.time.OffsetDateTime in CurrentTimeUTC — thanks to Veniamin Fernandes
+Refine ClientRequestSelectorTest, PersistentJobRunnerImplTest, SplitFileFetcherStorageTest, and SplitFileInserterStorageTest — thanks to Veniamin Fernandes
+Change deprecated jcenter() maven repo to mavenCentral() — thanks to Veniamin Fernandes
+Added tests for PebbleUtils — thanks to Bombe!
+🐛 Fix NPE when subsets are not initialized — thanks to Bombe
+reduce logging for too many excluded sub-arrays thanks to Hiina
+fix the flag size of nepal — thanks to Percept0r@NYZkOs7eQ…!
+Switch swiss flag to civil and state ensign — thanks to Percept0r@NY
+
+## Fixes
+Thanks to Bombe all our tests work again on Java 21!
+🐛 Fix JarClassLoader’s ability to work with ServiceLoader — thanks to Bombe!
+gzip: replace test workaround by fixing the output of the compressor — thanks to Bombe for the SingleOffsetReplacingOutputStream!
+fix: invalid max store size showed bytes with GiB suffix
+
+
+Also a special thanks to Bombe for many careful reviews!
 
 
 1497: