Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: T114 (rev 1) lfs out of space #5648

Open
eiginn opened this issue Dec 22, 2024 · 10 comments
Open

[Bug]: T114 (rev 1) lfs out of space #5648

eiginn opened this issue Dec 22, 2024 · 10 comments
Labels
bug Something isn't working

Comments

@eiginn
Copy link

eiginn commented Dec 22, 2024

Category

Other

Hardware

Heltec Mesh Node T114

Firmware Version

2.5.15.79da236

Description

Crash loop on start complaining about free space. I have it still in this state if I can provide anything. Serial output attached.

Relevant log output

[19:03:49.494] tio v3.7
[19:03:49.494] Press ctrl-t q to quit
[19:03:51.214] Connected to /dev/serial/by-id/usb-Heltec_HT-n5262_0615010964025919-if00
@INFO  | ??:??:?? 1

//\ E S H T /\ S T / C

INFO  | ??:??:?? 1 Wait for peripherals to stabilize
DEBUG | ??:??:?? 2 Filesystem files:
DEBUG | ??:??:?? 2  prefs (directory)
DEBUG | ??:??:?? 2    module.proto (94 Bytes)
DEBUG | ??:??:?? 2    channels.proto (110 Bytes)
DEBUG | ??:??:?? 2    config.proto (174 Bytes)
DEBUG | ??:??:?? 2    db.proto.tmp (23947 Bytes)
DEBUG | ??:??:?? 2  adafruit (directory)
DEBUG | ??:??:?? 2  bond_prph (directory)
DEBUG | ??:??:?? 2  bond_cntr (directory)
DEBUG | ??:??:?? 2 Power::lipoInit lipo sensor is not ready yet
DEBUG | ??:??:?? 2 Use analog input 4 for battery level
INFO  | ??:??:?? 2 Scan for i2c devices
DEBUG | ??:??:?? 2 Scan for I2C devices on port 2
DEBUG | ??:??:?? 2 Scan for I2C devices on port 1
INFO  | ??:??:?? 2 No I2C devices found
DEBUG | ??:??:?? 2 acc_info = 0
INFO  | ??:??:?? 2 S:B:69,2.5.15.79da236
INFO  | ??:??:?? 2 Build timestamp: 1732889685
DEBUG | ??:??:?? 2 Reset reason: 0x0
DEBUG | ??:??:?? 2 Set random seed 3696110524
INFO  | ??:??:?? 2 Init NodeDB
ERROR | ??:??:?? 2 Could not open / read /prefs/db.proto
WARN  | ??:??:?? 2 Devicestate 0 is old, discard
INFO  | ??:??:?? 2 Install default DeviceState
DEBUG | ??:??:?? 2 Initial packet id 1943059448
DEBUG | ??:??:?? 2 Partially randomized packet id 1959957497
DEBUG | ??:??:?? 2 Use nodenum 0x65698ef0
INFO  | ??:??:?? 2 Load /prefs/config.proto
INFO  | ??:??:?? 2 Loaded /prefs/config.proto successfully
INFO  | ??:??:?? 2 Loaded saved config version 23
INFO  | ??:??:?? 2 Load /prefs/module.proto
INFO  | ??:??:?? 2 Loaded /prefs/module.proto successfully
INFO  | ??:??:?? 2 Loaded saved moduleConfig version 23
INFO  | ??:??:?? 2 Load /prefs/channels.proto
INFO  | ??:??:?? 2 Loaded /prefs/channels.proto successfully
INFO  | ??:??:?? 2 Loaded saved channelFile version 23
DEBUG | ??:??:?? 2 cleanupMeshDB purged 0 entries
DEBUG | ??:??:?? 2 Use nodenum 0x65698ef0
INFO  | ??:??:?? 2 Adding node to database with 1 nodes and 154176 bytes free!
DEBUG | ??:??:?? 2 Expand short PSK #1
INFO  | ??:??:?? 2 Wanted region 1, using US
DEBUG | ??:??:?? 2 Coerce telemetry to min of 30 minutes on defaults
INFO  | ??:??:?? 2 Save /prefs/db.proto
lfs warn:314: No more free space 224
ERROR | ??:??:?? 2 Error: can't encode protobuf io error
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->blo
[19:03:52.773] Disconnected
@eiginn eiginn added the bug Something isn't working label Dec 22, 2024
@fifieldt
Copy link
Contributor

As a comparison, here's a T114 (rev 1) built with today's firmware with 46 nodes:

DEBUG | ??:??:?? 1  config.proto (171 Bytes)
DEBUG | ??:??:?? 1  channels.proto (203 Bytes)
DEBUG | ??:??:?? 1  module.proto (110 Bytes)
DEBUG | ??:??:?? 1  db.proto (5556 Bytes)

so at least we can say that config/channels/module files are normal.

How many nodes in the node DB?

@eiginn
Copy link
Author

eiginn commented Dec 22, 2024

Since it's in a crash loop I don't know how to pull that info from it. Any tips?

@fifieldt
Copy link
Contributor

Even, a rough guess would be useful - 20? 80? 100?

@eiginn
Copy link
Author

eiginn commented Dec 22, 2024

I'd say 80 to 90 nodes

@fifieldt
Copy link
Contributor

Thanks! OK, so at the upper end. The firmware version you're running should have included the fix to limit to 80 nodes, which we put in to avoid filesystem out of space errors.

@fifieldt
Copy link
Contributor

Potentially related to #5491

@analogman2 analogman2 mentioned this issue Dec 29, 2024
Open
@analogman2
Copy link

I am experiencing the same or similar issue on a RAK Wisblock 4631. The node crashed after a week and the last time I checked >150 nodes. This is the second time I have experienced this issue on 2.5.16, both times I ran nrf_erase before flashing. Upgrading to 2.5.18 today.

DEBUG | ??:??:?? 1 Filesystem files:
DEBUG | ??:??:?? 1 prefs (directory)
DEBUG | ??:??:?? 1 channels.proto (147 Bytes)
DEBUG | ??:??:?? 2 module.proto (118 Bytes)
DEBUG | ??:??:?? 2 config.proto (169 Bytes)
DEBUG | ??:??:?? 2 db.proto.tmp (27377 Bytes)
DEBUG | ??:??:?? 2 adafruit (directory)
DEBUG | ??:??:?? 2 bond_prph (directory)
DEBUG | ??:??:?? 2 bond_cntr (directory)
DEBUG | ??:??:?? 2 Power::lipoInit lipo sensor is not ready yet
DEBUG | ??:??:?? 2 Use analog input 5 for battery level
INFO | ??:??:?? 2 Scan for i2c devices
DEBUG | ??:??:?? 2 Scan for I2C devices on port 1
INFO | ??:??:?? 2 No I2C devices found
DEBUG | ??:??:?? 2 acc_info = 0
INFO | ??:??:?? 2 S:B:9,2.5.16.f81d3b0
INFO | ??:??:?? 2 Build timestamp: 1733662250
DEBUG | ??:??:?? 2 Reset reason: 0x0
DEBUG | ??:??:?? 2 Set random seed 1102827228
INFO | ??:??:?? 2 Init NodeDB
ERROR | ??:??:?? 2 Could not open / read /prefs/db.proto
WARN | ??:??:?? 2 Devicestate 0 is old, discard
INFO | ??:??:?? 2 Install default DeviceState
DEBUG | ??:??:?? 2 Initial packet id 913518860
DEBUG | ??:??:?? 2 Partially randomized packet id 544149773
DEBUG | ??:??:?? 2 Use nodenum 0x9e76f91f
INFO | ??:??:?? 2 Load /prefs/config.proto
INFO | ??:??:?? 2 Loaded /prefs/config.proto successfully
INFO | ??:??:?? 2 Loaded saved config version 23
INFO | ??:??:?? 2 Load /prefs/module.proto
INFO | ??:??:?? 2 Loaded /prefs/module.proto successfully
INFO | ??:??:?? 2 Loaded saved moduleConfig version 23
INFO | ??:??:?? 2 Load /prefs/channels.proto
INFO | ??:??:?? 2 Loaded /prefs/channels.proto successfully
INFO | ??:??:?? 2 Loaded saved channelFile version 23
DEBUG | ??:??:?? 2 cleanupMeshDB purged 0 entries
DEBUG | ??:??:?? 2 Use nodenum 0x9e76f91f
INFO | ??:??:?? 2 Adding node to database with 1 nodes and 153976 bytes free!
DEBUG | ??:??:?? 2 Expand short PSK https://github.com/meshtastic/firmware/pull/1
INFO | ??:??:?? 2 Wanted region 1, using US
INFO | ??:??:?? 2 Save /prefs/db.proto
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
lfs warn:314: No more free space 224
ERROR | ??:??:?? 2 Error: can't encode protobuf io error
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: block < lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count
ERROR | ??:??:?? 2 LFS assert: head >= 2 && head <= lfs->cfg->block_count #75ish more times
ERROR | ??:??:?? 2 LFS asser

@fifieldt
Copy link
Contributor

Thanks for the logs, very helpful.

@SpudGunMan
Copy link

write erase didnt help flashing older code hasn't helped yet

@cracky22
Copy link

cracky22 commented Jan 5, 2025

Isn’t this a similar problem to #5491

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@fifieldt @eiginn @SpudGunMan @analogman2 @cracky22