M20 PTU Support, Multiple iMet-4 Fixes

This release brings the following updates:

• Decoders: M20 Pressure, Temperature, and Humidity support. Note that not all M20 sondes have a pressure sensor. Thanks to DF9DQ and @rs1729!
• Decoders: The iMet-4 decode chain should now gracefully handle up to 4 different iMet-4 sondes observed on the same frequency.
• Other: Small tweaks to some log levels.

There are no configuration file changes in this release.

Note: We are aware that new installations may display the following message on startup:

The WebSocket transport is not available, you must install a WebSocket server that is compatible with your async mode to enable it. See the documentation for details. (further occurrences of this error will be logged with level INFO)

This can be safely ignored. The web interface should still continue to work as usual. Eventually we will figure out how to suppress it...

The pre-built Docker Image is now the recommended way to setup radiosonde_auto_rx on a new station. Using Docker avoids issues with updates to dependencies, and allows for fixes relating to system packages to be easily pushed out to users.

Please report any problems encountered either on the mailing list, or by raising an issue on this repository.

PLEASE DO NOT USE THE GITHUB GENERATED RELEASE ZIP FILES - USE THE INSTALLATION/UPDATE STEPS DESCRIBED IN THE WIKI

Telemetry-derived transmit frequency, iMet-54 frame CRC, PS-15 sub-type support.

This release brings the following updates:

• Decoders: We now report the telemetry-derived transmit frequency for RS41, RS92 and iMS-100 sondes to SondeHub under the new tx_frequency optional field. This will allow identification of receivers with poorly-calibrated oscillators.
• Decoders: iMet-54 decoder updated to make use of a reverse-engineered frame CRC (more great work from @rs1729 !)
• Decoders: M10 decoder now only outputs the number-of-GPS-sats field for non-gTop M10s.
• Decoders: auto_rx now recognises the Graw DFM sub-type code of 0x7 as the PS-15 PilotSonde.
• Other: Some tweaks of logging levels.

There are no configuration file changes in this release.

Note: We are aware that new installations may display the following message on startup:

The WebSocket transport is not available, you must install a WebSocket server that is compatible with your async mode to enable it. See the documentation for details. (further occurrences of this error will be logged with level INFO)

This can be safely ignored. The web interface should still continue to work as usual. Eventually we will figure out how to suppress it...

The pre-built Docker Image is now the recommended way to setup radiosonde_auto_rx on a new station. Using Docker avoids issues with updates to dependencies, and allows for fixes relating to system packages to be easily pushed out to users.

Please report any problems encountered either on the mailing list, or by raising an issue on this repository.

PLEASE DO NOT USE THE GITHUB GENERATED RELEASE ZIP FILES - USE THE INSTALLATION/UPDATE STEPS DESCRIBED IN THE WIKI

iMet-50, 'new' LMS6-403 type support

This release brings the following updates:

• Decoders: Support for iMet-50 Pilot-sondes. These use the same telemetry format as the iMet-54, though without sensor data, and have been observed in Israel.
• Decoders: Changed references to LMS6-400 to LMS6-403 (better matches with manufacturer naming)
• Decoders: Support for 'new' LMS6-403 sondes, with slightly different framing format. These sondes have been observed to be launched from Vandenburg Space Force base, and stop transmitting shortly after burst. We are unsure if these are a new variant of LMS6 sondes, or the same sondes with a firmware modification.
• Docker: Minor changes to fsk_demod to support a switch to Alpine Linux in the future.

There are no configuration file changes in this release.

Note: We are aware that new installations may display the following message on startup:

The WebSocket transport is not available, you must install a WebSocket server that is compatible with your async mode to enable it. See the documentation for details. (further occurrences of this error will be logged with level INFO)

This can be safely ignored. The web interface should still continue to work as usual. Eventually we will figure out how to suppress it...

The pre-built Docker Image is now the recommended way to setup radiosonde_auto_rx on a new station. Using Docker avoids issues with updates to dependencies, and allows for fixes relating to system packages to be easily pushed out to users.

Please report any problems encountered either on the mailing list, or by raising an issue on this repository.

PLEASE DO NOT USE THE GITHUB GENERATED RELEASE ZIP FILES - USE THE INSTALLATION/UPDATE STEPS DESCRIBED IN THE WIKI

M20 APRS Support, Improved LMS6-1680 Decoding

This release brings the following updates:

• APRS: M20 telemetry can now be uploaded to APRS-IS (e.g. radiosondy.info)
• APRS: The recommended comment string has been changed to include the full sonde serial number. Refer to the configuration file change-log: https://github.com/projecthorus/radiosonde_auto_rx/wiki/Configuration-File-Changelog#version-155
• APRS: Removed the option to upload telemetry as position reports instead of APRS objects. The convention for telemetry uploading is to use Objects, and the position report option was causing more trouble than it was worth.
• Decoders: The LMS6-1680 decode chain has been improved, with a good few dB of improved sensitivity. There is also the option to enable an alternate decode chain which gives slightly higher performance again, which can be selected between in a new configuration file option (refer change-log linked above). However, this decode chain uses a considerable amount of CPU, so is only recommended for use on a RPI 3B+ or better.
• Scanning: Detection of LMS6-1680 sondes has been improved somewhat through the use of a wider FM filter bandwidth.
• Web Interface: Instead of the serial number in the table being a link, there are now small icons next to each serial number, linking to Sondehub and radiosondy.info.
• Historical Display: Table now shows Last Height, Maximum Height, and Maximum range for each sonde.
• Historical Display: The coverage map now sets any bearings with no sonde data to the station location and 0 range. This has the effect of making the coverage map a little 'spiky', but is a more accurate representation of the data.
• Docker: Fixes to running auto_rx on the Pi Zero W (Note that the Zero W is right on the bare edge of having enough CPU grunt to run auto_rx).

Note: We are aware that new installations may display the following message on startup:

The WebSocket transport is not available, you must install a WebSocket server that is compatible with your async mode to enable it. See the documentation for details. (further occurrences of this error will be logged with level INFO)

This can be safely ignored. The web interface should still continue to work as usual. Eventually we will figure out how to suppress it...

The pre-built Docker Image is now the recommended way to setup radiosonde_auto_rx on a new station. Using Docker avoids issues with updates to dependencies, and allows for fixes relating to system packages to be easily pushed out to users.

Please report any problems encountered either on the mailing list, or by raising an issue on this repository.

PLEASE DO NOT USE THE GITHUB GENERATED RELEASE ZIP FILES - USE THE INSTALLATION/UPDATE STEPS DESCRIBED IN THE WIKI

Minor Fixes & Updates, Docker rtl_fm performance improvements

This release brings the following updates:

• Docker Container: Switch to compiling rtl-sdr from source, to resolve an issue where the Debian rtl-sdr package uses very high CPU on recent Raspbian kernel versions (>5.0.0). Note: This affects native installations of auto_rx too, but can't be fixed as easily - If you are noticing extremely high CPU usage on your station (rtl_fm using ~70% CPU, when it should be using ~11%), it is recommended that you uninstall the system-provided rtl-sdr packages and compile/install rtl-sdr from source. Refer Section 4.2.1 of the auto_rx installation instructions
• Web Interface: Adjusted window detection width for mobile/desktop browsers. Should help get most table fields back when clearing cookies on the web interface.
• Historical Display: Re-named Plot Launches/Landings to Plot Last RX / Plot First RX.
• Historical Display: Added Plot Coverage button.
• Historical Display: Added Last Height & Maximum Range fields to table.
• RS41 Decoding: Relaxed RS41 serial number regex to permit older radiosondes (now allows letters C-Z), and any serial number, to allow research groups re-launching RS41s with modified serials to be received by auto_rx.
• RS92 Decoding: Handle error when downloading GPS Almanac correctly.
• Configuration File: Added 'save_raw_hex' debugging option, which saves hexadecimal frame data for a select number of sonde types to the log directory while decoding. Refer Config File Changelog
• Configuration File: Added a minimum time limit to the SondeHub upload rate (10 seconds). Uploading faster than this results in higher database load (and higher running costs!) for little benefit.

The pre-built Docker Image is now the recommended way to setup radiosonde_auto_rx on a new station. Using Docker avoids issues with updates to dependencies, and allows for fixes relating to system packages to be easily pushed out to users (e.g. the rtl-sdr issues).

IMPORTANT UPDATE NOTES:

• Once again, if you haven't updated to Python >3.6 - now is the time (auto_rx will no longer work on Python 2.7 or 3.5.). If your station is running on Python 2, please either do a fresh install with the latest Raspbian. It is highly recommended that you use the Docker installation referred to above.

If you are upgrading from an old release and are encountering issues with the web interface, please refer to the release notes for v1.5.1 for information on the Python package updates required: https://github.com/projecthorus/radiosonde_auto_rx/releases/tag/v1.5.1

Please report any problems encountered either on the mailing list, or by raising an issue on this repository.

PLEASE DO NOT USE THE GITHUB GENERATED RELEASE ZIP FILES - USE THE INSTALLATION/UPDATE STEPS DESCRIBED IN THE WIKI

Historical Data View and Skew-T Plots

This release brings the much-requested ability to display historical log files! A huge thanks to @LukePrior for their work bringing us this new feature.

The new 'historical view', available from the top-right of the main auto_rx web interface provides the following functions:

• Plot the flight paths of selected radiosonde flights.
• Plot the landing locations of all received radiosonde flights.
• Generate a Skew-T plot for a selected radiosonde flight (if temperature/humidity data is available). This uses the skewt-js library from dfelix.
• Download selected logs as a Zip file.

You can find a guide to the new web interface on the auto_rx wiki here: https://github.com/projecthorus/radiosonde_auto_rx/wiki/Web-Interface-Guide

Skew-T plots are also available on the sondehub.org/card/ page used to display historical data from the SondeHub database. We hope to add Skew-T plots to the SondeHub tracker sometime in the near future.

Note: If you are using a 'native' install of auto_rx, you will need to install the 'semver' Python package by running: sudo pip3 install semver

Finally, we have now turned off the 'v1' SondeHub interface. radiosonde_auto_rx stations will no longer appear on tracker.habhub.org, nor will radiosonde telemetry be available via tracker.habhub.org. All radiosonde tracking is now located on https://tracker.sondehub.org/ and stations must be running at least v1.5.1 for their telemetry and station to appear on the map.

The pre-built Docker Image is now the recommended way to setup radiosonde_auto_rx on a new station. Using Docker avoids issues with updates to dependencies, like is occurring in this release.

IMPORTANT UPDATE NOTES:

• Once again, if you haven't updated to Python >3.6 - now is the time (auto_rx will no longer work on Python 2.7 or 3.5.). If your station is running on Python 2, please either do a fresh install (ideally with the latest version of Raspbian), or follow the Python 2 -> 3 upgrade guide here: https://github.com/projecthorus/radiosonde_auto_rx/wiki/Performing-Updates#upgrading-a-station-from-python-2-to-python-3

If you are upgrading from an old release and are encountering issues with the web interface, please refer to the release notes for v1.5.1 for information on the Python package updates required: https://github.com/projecthorus/radiosonde_auto_rx/releases/tag/v1.5.1

Please report any problems encountered either on the mailing list, or by raising an issue on this repository.

PLEASE DO NOT USE THE GITHUB GENERATED RELEASE ZIP FILES - USE THE INSTALLATION/UPDATE STEPS DESCRIBED IN THE WIKI

Bugfix for 1.5.2

Fixes issues with station.cfg backwards compatibility (blacklist was being interpreted as the new only_scan list). If you hav re-named your configuration file entries to the new names there will be no issues.

More Web Interface Improvements, Habitat DB Deprecated

This release of auto_rx brings the following new features & changes:

We have now cut-over to the new SondeHub v2 database! This means no more overloading of the Habitat database, which was intended for only a small number of amateur high-altitude balloon launches. The Habitat uploader has now been disabled permanently in this release. Habitat will stop accepting uploads from auto_rx stations around the end of May.

The Sondehub Tracker has been switched over to use this new database, with the following changes:

• Update rates are increased, with 1-second resolution telemetry available for sondes currently in the air (telemetry over larger time periods is decimated to avoid overloading browser clients).
• The telemetry graphs will work for all provided telemetry fields (temperature, pressure, humidity, battery voltage, satellites)
• The Signal-to-noise ratio observed by each received is shown next to their callsign in the list of radiosondes.
• The radiosonde type is shown next to the serial number in the list of radiosondes.
• Chase Car position uploading has been updated to use the new SondeHub listeners API. Chasemapper has also been updated to be able to use this API to show chase car positions on the new tracker.

The Sondehub v2 database is currently handling ~4 million telemetry packets per day, with peaks above 200 packets per second during the major radiosonde launch times. Running this database system is definitely not cost-free, so if you'd like to help support the ongoing running of the Sondehub DB and tracker, we have a Patreon page ( https://patreon.com/sondehub ) and a PayPal donate URL ( https://sondehub.org/go/donate .

Other updates in this release include:

• More improvements to the Web Interface (Thanks @LukePrior !). There are now some basic controls over the scanning & decoding system available from a button at the bottom of the Settings pane. These allow the user to halt the scanner, and manually stop/start a decoder on a provided frequency. I expect this functionality will mainly be useful for testing purposes, as the current scanning system works pretty well. These controls are only functional if web_control is set to True in the station.cfg file, and a web_password is set. Note: Stations that have their web interface exposed to the internet should use this feature with caution - no guarantee of web interface security is provided! You may need to Shift-Refresh the page, or clear your browser cache for full functionality after updating.

• Fixes to SNR reports from radiosonde signals that are fading. SNRs are now calculated using a 'peak hold'.

• Fixes to handling of telemetry from DFM radiosondes when the serial number has not yet been received. (Should stop the 'xxxxxxxx' sondes from appearing on the map)

• Fixed to e-mail notifications for radiosonde types that don't report ascent/descent rates.

• Update the URLs in the email notifications and the web interface to work with the new Sondehub v2 database. The main change here is the stripping off the sonde type prefix that is used on some serial numbers (DFM, M10/20, iMet...).

• Map movement in the web interface has been smoothed. This does increase CPU load in the browser - let us know if you experience issues on low-powered systems.

• Re-named the white/black/grey-list entries in the configuration file to have more sensible names: only_scan, always_scan and never_scan.

Refer to the Configuration File Changelog for details on what has changed. As always, auto_rx should be backwards-compatible with old station.cfg files, though you will see warnings on startup.

The pre-built Docker Image is now the recommended way to setup radiosonde_auto_rx on a new station. Using Docker avoids issues with updates to dependencies, like is occurring in this release.

IMPORTANT UPDATE NOTES:

• Once again, if you haven't updated to Python >3.6 - now is the time (auto_rx will no longer work on Python 2.7 or 3.5.). If your station is running on Python 2, please either do a fresh install (ideally with the latest version of Raspbian), or follow the Python 2 -> 3 upgrade guide here: https://github.com/projecthorus/radiosonde_auto_rx/wiki/Performing-Updates#upgrading-a-station-from-python-2-to-python-3

Please report any problems encountered either on the mailing list, or by raising an issue on this repository.

PLEASE DO NOT USE THE GITHUB GENERATED RELEASE ZIP FILES - USE THE INSTALLATION/UPDATE STEPS DESCRIBED IN THE WIKI

Updated Web Interface, SondeHub Station Info, Dropped Python <3.6 support.

This release of auto_rx brings the following new features & changes:

• Support for Python versions older than 3.6 has been dropped completely. Users running Python 2.7 or 3.5 will encounter an error on startup and auto_rx will exit. Time to upgrade to Python 3! In particular, Python >=3.6 is required. This means that stations running Raspbian Stretch (which comes with Python 3.5) will need to update to the latest Raspbian distribution - Raspbian Buster.
• Thanks to the work of @LukePrior, a revamped web interface! This brings a much cleaner and customisable web interface, with better mobile support and a dark mode!
• Updated the socket.io JS library, and removed the version pinning for flask-socketio. You will need to update your flask-socketio version for the web interface to continue working. Upgrading can be performed by running sudo pip3 install -U flask-socketio . You will also likely need to clear your browser cache to get the new socketio Javascript library (e.g. hold down Shift and click the page refresh button).
• Additions to the SondeHub v2 uploader system, such that station locations appear on the v2 tracker map even if no telemetry has been uploaded recently.
• Added an optional station contact e-mail field. This information is only made available to the SondeHub admins (myself and Michaela), and will only be used if an issue with your station is noticed (or if we need your help observing something interesting!).
• Added some additional checks on the date/time reported in radiosonde telemetry.
• Fixes to the Docker image generation for armv6/armv7 systems.

The pre-built Docker Image is now the recommended way to setup radiosonde_auto_rx on a new station. Using Docker avoids issues with updates to dependencies, like is occurring in this release.

IMPORTANT UPDATE NOTES:

• Once again, if you haven't updated to Python >3.6 - now is the time (auto_rx will no longer work on Python 2.7 or 3.5.). If your station is running on Python 2, please either do a fresh install (ideally with the latest version of Raspbian), or follow the Python 2 -> 3 upgrade guide here: https://github.com/projecthorus/radiosonde_auto_rx/wiki/Performing-Updates#upgrading-a-station-from-python-2-to-python-3

Please report any problems encountered either on the mailing list, or by raising an issue on this repository.

PLEASE DO NOT USE THE GITHUB GENERATED RELEASE ZIP FILES - USE THE INSTALLATION/UPDATE STEPS DESCRIBED IN THE WIKI

Intermet Systems iMet-54 and Meteo-Radiy MRZ Support, RS92 PTU, SondeHub v2 DB Uploader

This release brings the following changes thanks to the hard work of rs1729:

• Support for the Intermet iMet-54 radiosonde, which is currently used in South Africa.
• Support for PTU data decoding for RS92-SGP and RS92-NGP sondes.
• Support for the Meteo-Radiy MRZ-H1 (Commonly used in Russia)

Also:

• Addition of the SondeHub DB uploader. This uploads all the telemetry that your station receives to a new ElasticSearch-based database that we're calling 'Sondehub v2'! We're working hard behind the scenes to make best use of this data, including building up a separate version of the habhub tracker which will use this new DB instead of Habitat. For now, data is uploaded to this DB so we have data to work with to help development. This data is also now available via the AWS Open Data program - see below for more information.
• Added checks to ensure decoder and auto_rx versions match. If you haven't re-built the decoders (./build.sh) you will get a critical error when a decoder is started up. Make sure you rebuild after each update!

Thanks to @snh, auto_rx is also now available as a Docker image! See here for more information: https://github.com/projecthorus/radiosonde_auto_rx/wiki/Docker

With the addition of the new Sondehub v2 uploader, we will be starting to disable uploading of telemetry from older auto_rx versions (1.4.1 and older). Users running older versions will soon start to see notices in their logs reminding them to update, and in a few months will have all telemetry uploads blocked. Please update!

This release took far too long to get out the door... I'll try and push out updates more regularly from now on.

Telemetry Data Licensing & AWS Open Data
We have recently had a few requests from auto_rx users wishing to make (non-commercial!) use of the radiosonde flight path data gathered by auto_rx users into the SondeHub database, and the question of data licensing has arisen. Additionally, we have been accepted into the Amazon Web Services Open Data program - making our collected radiosonde telemetry available by yet another mechanism. More information on this is available here.

To clarify the licensing situation of this data, we have chosen to use a very permissive license - Creative Commons BY-SA 2.0. This is a similar license to what other telemetry gathering systems (e.g. ADS-B systems, SatNOGS) use, and ensures that the project and it's contributors (you!) are attributed appropriately if any data is used.

By continuing to use auto_rx and uploading telemetry to Habitat and Sondehub, you agree to release the telemetry data you gather under this license. With the upgraded database we will be providing more ways to access the data collected by the whole auto_rx network - some initial information is available here.

IMPORTANT UPDATE NOTES:

• Once again, if you haven't updated to Python 3 - now is the time (Sondehub uploading will not work under Python 2). If your station is running on Python 2, please either do a fresh install (ideally with the latest version of Raspbian), or follow the Python 2 -> 3 upgrade guide here: https://github.com/projecthorus/radiosonde_auto_rx/wiki/Performing-Updates#upgrading-a-station-from-python-2-to-python-3

Please report any problems encountered either on the mailing list, or by raising an issue on this repository.

PLEASE DO NOT USE THE GITHUB GENERATED RELEASE ZIP FILES - USE THE INSTALLATION/UPDATE STEPS DESCRIBED IN THE WIKI