Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nrfcloud coap: use downloader library #19908

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

maxd-nordic
Copy link
Contributor

@maxd-nordic maxd-nordic commented Jan 14, 2025

Rework the implementation for nRF Cloud CoAP downloads to use downloader instead of a custom implementation. This also adds some missing features to downloader and cleans up the previous code from fota_download and nrf_cloud_download.
The old implementation had issues with resumption which should be resolved by this. Also, by reducing custom code, maintainability should be improved.

@github-actions github-actions bot added the changelog-entry-required Update changelog before merge. Remove label if entry is not needed or already added. label Jan 14, 2025
@NordicBuilder
Copy link
Contributor

NordicBuilder commented Jan 14, 2025

CI Information

To view the history of this post, clich the 'edited' button above
Build number: 11

Inputs:

Sources:

sdk-nrf: PR head: 41264acb5915f862fe18edb02e3adde1adf2571f

more details

sdk-nrf:

PR head: 41264acb5915f862fe18edb02e3adde1adf2571f
merge base: f5c6c05dff1fbc9c49d1adee831be0d2fe94f02f
target head (main): c0fb88a06d165723699eb7f32cc41e466794a13d
Diff

Github labels

Enabled Name Description
ci-disabled Disable the ci execution
ci-all-test Run all of ci, no test spec filtering will be done
ci-force-downstream Force execution of downstream even if twister fails
ci-run-twister Force run twister
ci-run-zephyr-twister Force run zephyr twister
List of changed files detected by CI (19)
include
│  ├── net
│  │  ├── downloader.h
│  │  │ fota_download.h
subsys
│  ├── net
│  │  ├── lib
│  │  │  ├── downloader
│  │  │  │  ├── Kconfig
│  │  │  │  ├── src
│  │  │  │  │  ├── transports
│  │  │  │  │  │  ├── coap.c
│  │  │  │  │  │  │ http.c
│  │  │  ├── fota_download
│  │  │  │  ├── Kconfig
│  │  │  │  ├── src
│  │  │  │  │  │ fota_download.c
│  │  │  ├── nrf_cloud
│  │  │  │  ├── Kconfig.nrf_cloud_coap
│  │  │  │  ├── Kconfig.nrf_cloud_fota
│  │  │  │  ├── coap
│  │  │  │  │  ├── include
│  │  │  │  │  │  │ nrf_cloud_coap_transport.h
│  │  │  │  │  ├── src
│  │  │  │  │  │  │ nrf_cloud_coap_transport.c
│  │  │  │  ├── include
│  │  │  │  │  │ nrf_cloud_download.h
│  │  │  │  ├── src
│  │  │  │  │  ├── nrf_cloud_download.c
│  │  │  │  │  │ nrf_cloud_fota_poll.c
tests
│  ├── subsys
│  │  ├── net
│  │  │  ├── lib
│  │  │  │  ├── downloader
│  │  │  │  │  │ CMakeLists.txt
│  │  │  │  ├── fota_download
│  │  │  │  │  │ CMakeLists.txt
│  │  │  │  ├── lwm2m_client_utils
│  │  │  │  │  │ CMakeLists.txt
│  │  │  │  ├── lwm2m_fota_utils
│  │  │  │  │  │ CMakeLists.txt
│  │  │  │  ├── mcumgr_smp_client
│  │  │  │  │  │ CMakeLists.txt

Outputs:

Toolchain

Version: 342151af73
Build docker image: docker-dtr.nordicsemi.no/sw-production/ncs-build:342151af73_912848a074

Test Spec & Results: ✅ Success; ❌ Failure; 🟠 Queued; 🟡 Progress; ◻️ Skipped; ⚠️ Quarantine

  • ◻️ Toolchain - Skipped: existing toolchain is used
  • ✅ Build twister
    • sdk-nrf test count: 345
  • 🟠 Integration tests
    • 🟠 test-fw-nrfconnect-boot
    • 🟠 test-fw-nrfconnect-nrf-iot_serial_lte_modem
    • 🟠 test-fw-nrfconnect-nrf-iot_zephyr_lwm2m
    • 🟠 test-fw-nrfconnect-nrf-iot_samples
    • 🟠 test-fw-nrfconnect-nrf-iot_lwm2m
    • 🟠 test-fw-nrfconnect-nrf-iot_thingy91
    • 🟠 test-fw-nrfconnect-nrf-iot_mosh
    • 🟠 test-fw-nrfconnect-nrf-iot_positioning
    • 🟠 test-sdk-mcuboot
    • ⚠️ test-fw-nrfconnect-fw-update
    • ⚠️ test-fw-nrfconnect-nrf-iot_cloud
Disabled integration tests
    • desktop52_verification
    • doc-internal
    • test_ble_nrf_config
    • test-fw-nrfconnect-apps
    • test-fw-nrfconnect-ble_mesh
    • test-fw-nrfconnect-ble_samples
    • test-fw-nrfconnect-chip
    • test-fw-nrfconnect-fem
    • test-fw-nrfconnect-nfc
    • test-fw-nrfconnect-nrf-iot_libmodem-nrf
    • test-fw-nrfconnect-nrf_crypto
    • test-fw-nrfconnect-ps
    • test-fw-nrfconnect-rpc
    • test-fw-nrfconnect-rs
    • test-fw-nrfconnect-tfm
    • test-fw-nrfconnect-thread
    • test-fw-nrfconnect-zigbee
    • test-low-level
    • test-sdk-audio
    • test-sdk-dfu
    • test-sdk-find-my
    • test-sdk-pmic-samples
    • test-sdk-sidewalk
    • test-sdk-wifi
    • test-secdom-samples-public

Note: This message is automatically posted and updated by the CI

@NordicBuilder
Copy link
Contributor

You can find the documentation preview for this PR at this link.

Note: This comment is automatically posted by the Documentation Publish GitHub Action.

@maxd-nordic maxd-nordic force-pushed the downloader-nrfcloud-coap branch 2 times, most recently from ff29a72 to 48a0dae Compare January 15, 2025 14:10
@maxd-nordic
Copy link
Contributor Author

note: CONFIG_DOWNLOADER_STACK_SIZE=4096 is needed when integrating this, the normal stack size is too small.

@maxd-nordic maxd-nordic force-pushed the downloader-nrfcloud-coap branch 2 times, most recently from 575a09a to c6ee5da Compare January 17, 2025 14:10
@maxd-nordic maxd-nordic marked this pull request as ready for review January 17, 2025 14:14
@maxd-nordic maxd-nordic requested review from a team as code owners January 17, 2025 14:14
@maxd-nordic maxd-nordic changed the title Downloader nrfcloud coap nrfcloud coap: use downloader library Jan 17, 2025
@maxd-nordic maxd-nordic force-pushed the downloader-nrfcloud-coap branch from c6ee5da to 2b2c4d8 Compare January 17, 2025 15:00
@maxd-nordic
Copy link
Contributor Author

@maxd-nordic maxd-nordic force-pushed the downloader-nrfcloud-coap branch from 2b2c4d8 to f9b7ac7 Compare January 17, 2025 16:05
}

/* we are already connected using the given socket */
client->sock = socket;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe lock around these two lines

* Callback to do client authentication.
* This is called after connecting.
*/
int (*auth_cb)(int sock);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The callback returns an int, but it does not seem to be checked by the code.
I suggest this is either made void or that it's documented what the auth_cb should return and that the return is handled by the code.

return err;
}
LOG_DBG("dl->file: %s", dl->file);
strncpy(file, dl->file, sizeof(file));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
strncpy(file, dl->file, sizeof(file));
strncpy(file, dl->file, sizeof(file) - 1);
file[FILENAME_SIZE - 1] = '\0\;

To ensure null-termination. It might be safe in this case to not do this, but then I think it warrants a comment just to make it obvious that it has been considered as safe

subsys/net/lib/downloader/src/transports/coap.c Outdated Show resolved Hide resolved
@@ -448,6 +465,11 @@ static int dl_coap_connect(struct downloader *dl)

coap->new_data_req = true;

/* run auth callback if set */
if (coap->auth_cb != NULL) {
coap->auth_cb(coap->sock.fd);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
coap->auth_cb(coap->sock.fd);
err = coap->auth_cb(coap->sock.fd);

Feels like this should be returned or at least checked + logged

subsys/net/lib/fota_download/src/fota_download.c Outdated Show resolved Hide resolved
@maxd-nordic maxd-nordic force-pushed the downloader-nrfcloud-coap branch from f9b7ac7 to 44bc70a Compare January 20, 2025 10:13
Replace some magic numbers in downloader:coap with macros.
This also makes it look more similar to the http transport code.

Signed-off-by: Maximilian Deubel <[email protected]>
Add support for Proxy-URI options to downloader.
This is defined in RFC7252 and we need it for nRF Cloud.

Signed-off-by: Maximilian Deubel <[email protected]>
Add support for an authentication callback to the downloader library.
The callback is called after connecting and allows to authenticate
the current socket.
nRF Cloud uses authentication in this way for their CoAP backend.

Signed-off-by: Maximilian Deubel <[email protected]>
CoAP URLs were parsed twice, leading to errors.
Now, the backend doesn't try to treat the file name as a URL anymore.

Signed-off-by: Maximilian Deubel <[email protected]>
Integrate the improved downloader library into nRF Cloud.
Instead of using a custom downloader backend,
use downloader for coap downloads.
Also,
remove external_download_client support since it isn't used anymore.

Signed-off-by: Maximilian Deubel <[email protected]>
@maxd-nordic maxd-nordic force-pushed the downloader-nrfcloud-coap branch from 44bc70a to 41264ac Compare January 20, 2025 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog-entry-required Update changelog before merge. Remove label if entry is not needed or already added.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants