Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📢 Meshtastic Rust needs a new platform owner / maintainer 📢 #25

Open
rcarteraz opened this issue Dec 13, 2024 · 18 comments
Open

📢 Meshtastic Rust needs a new platform owner / maintainer 📢 #25

rcarteraz opened this issue Dec 13, 2024 · 18 comments

Comments

@rcarteraz
Copy link
Member

There are many missing features and other quality of life issues that need to be addressed. This repo has been without a dedicated maintainer for a while now and is in danger of soon becoming deprecated.

Please respond to this issue if you are interested in taking on this role or helping out in some specific way.

@lukipuki
Copy link
Collaborator

I've got maintainer rights from @ajmcquilkin a while ago, the last few PRs were merged by me: https://github.com/meshtastic/rust/pulls?q=is%3Apr+is%3Aclosed

Though the need for this crate in my own project is low, so I'm not dedicating as much time to it as would be ideal. A second (or third) maintainer would be great.

@krant
Copy link
Contributor

krant commented Dec 18, 2024

I'd be happy to help. What are the major issues with the repo now? Besides new release overdue I'm not seeing what is missing, but may be my use case is small.

@lukipuki
Copy link
Collaborator

@krant there's the BLE support which a lot of people asked for, I have a WIP branch that I'd like to merge in about a month.

Also I'm not sure if work is needed to support encrypted messages.

And you can always improve code ergonomics, e.g. many people struggled with PacketRouter.

These are just a few things that multiple people asked in Discord from the top of my head.

@rcarteraz
Copy link
Member Author

I've got maintainer rights from @ajmcquilkin a while ago, the last few PRs were merged by me: https://github.com/meshtastic/rust/pulls?q=is%3Apr+is%3Aclosed

Though the need for this crate in my own project is low, so I'm not dedicating as much time to it as would be ideal. A second (or third) maintainer would be great.

We can work on getting Crates access. We really need someone to own this and keep it current, that's really what we're looking for, otherwise we'll have to shift focus elsewhere.

@krant
Copy link
Contributor

krant commented Dec 22, 2024

Full featured Rust CLI on a par with the python one could be something that could attract more users (no need to install Python and dependencies). I can take it if there is a desire.

@rcarteraz
Copy link
Member Author

Full featured Rust CLI on a par with the python one could be something that could attract more users (no need to install Python and dependencies). I can take it if there is a desire.

We'd be 100% supportive of this idea!

@krant
Copy link
Contributor

krant commented Dec 25, 2024

Awesome, I'll prepare a PR with a bare bones implementation and we can decide where to go from there.

@ralphtheninja
Copy link
Contributor

👋 I was thinking about implementing a crate for meshtastic and obviously there was one already! This is great news. I'd be happy to dive in as well.

Full featured Rust CLI on a par with the python one could be something that could attract more users (no need to install Python and dependencies). I can take it if there is a desire.

This would be awesome.

@geoffreygarrett
Copy link
Contributor

geoffreygarrett commented Jan 10, 2025

Hi all! I'd love to help contribute to this project. I have some local work from a couple months ago that might address some much needed changes, most of them are in a sort of "blueprint" phase as starting point, ready for implementation. Some of what I worked on locally:

  • improved trait system for transports
  • improved message/packet framing to account for serial messages (this was the first thing that broke the moment I tried to use the repo)
  • starting point for a CLI for v1 and v2, the idea was that v1 should be a drop-in replacement for the python CLI, and v2 is something more idiomatic in design
  • python bindings via pyo3

Bluetooth was something I spent some time looking at too, and started working on feature gating it on a per OS basis. At one point I was also trying to get it working via WASM.

Seeing as though I'm a little late to the party, let me know how I can contribute here and coordinate with what you all have started working on. If nothing is currently in the pipeline, I would start off by making a PR with the much needed structure changes for the project if that's agreeable with everyone!

I'm really happy to see that there's some renewed interest. I'm a little sad I didn't see this activity a bit sooner.

@rcarteraz
Copy link
Member Author

Hi all! I'd love to help contribute to this project. I have some local work from a couple months ago that might address some much needed changes, most of them are in a sort of "blueprint" phase as starting point, ready for implementation. Some of what I worked on locally:

  • improved trait system for transports
  • improved message/packet framing to account for serial messages (this was the first thing that broke the moment I tried to use the repo)
  • starting point for a CLI for v1 and v2, the idea was that v1 should be a drop-in replacement for the python CLI, and v2 is something more idiomatic in design
  • python bindings via pyo3

Bluetooth was something I spent some time looking at too, and started working on feature gating it on a per OS basis. At one point I was also trying to get it working via WASM.

Seeing as though I'm a little late to the party, let me know how I can contribute here and coordinate with what you all have started working on. If nothing is currently in the pipeline, I would start off by making a PR with the much needed structure changes for the project if that's agreeable with everyone!

I'm really happy to see that there's some renewed interest. I'm a little sad I didn't see this activity a bit sooner.

We'd love to have you start contributing! You have some incredible ideas and we'd be happy to see them come to fruition. I know @krant mentioned also wanting to work on a Rust CLI, maybe the two of you can coordinate based on what you both already have completed?

@rcarteraz
Copy link
Member Author

Hi everyone,

Thank you all for the enthusiasm and interest in contributing to this project—it’s great to see so much renewed activity here! As you know, maintaining a project like this requires consistent effort to address issues, implement new features, and ensure long-term viability.

While expressing interest is a great first step, the best way to demonstrate your readiness to take on a larger role, such as a maintainer, is through consistent contributions. This could mean submitting PRs, actively participating in discussions, or tackling some of the pressing needs in the repo, such as BLE support, CLI improvements, or other quality-of-life enhancements.

We’re excited about the ideas being shared—particularly the work on a Rust CLI and the structural improvements mentioned by @geoffreygarrett and @krant. It would be fantastic to see those ideas come to life! Coordination between contributors will also be key to avoid duplicate efforts and maximize impact.

Once we see sustained contributions, we can move forward with formalizing roles like maintainer or platform owner. Thank you again for stepping up and helping move this project forward!

@krant
Copy link
Contributor

krant commented Jan 11, 2025

@geoffreygarrett Great point about idiomatic CLI v2, it would be nice to have alongside with python-compatible v1, may be even in the same binary (we could use multicall feature of clap for this). Let me come up with v1 PR (I already have CLI tool used in closed-source project) - if you bring trait/API improvements sooner, we'll manage to sync them somehow.

@rcarteraz thanks for the guidance, all points have been taken into account 🙂

@geoffreygarrett
Copy link
Contributor

Hey everyone,

Full disclaimer: I learned (the hard way) that bite-sized commits and better local branch management would have saved me from piling up so many changes before going on holiday. Unfortunately, I didn’t wrap things up in time, so my local project isn’t fully functional at the moment. Now that there’s renewed activity, I’m trying to separate what’s genuinely useful from my own experiments.

Here’s a quick overview of the structural changes I was envisioning (my local tree basically looks like this):

meshtastic-rust
├── apps
│   └── cli
├── bindings
│   └── python
├── Cargo.lock
├── Cargo.toml
├── crates
│   ├── meshtastic
│   ├── meshtastic-core
│   ├── meshtastic-device
│   ├── meshtastic-network
│   └── meshtastic-protocol
├── docs
│   ├── book
│   └── book.toml
├── flake.nix
└── LICENSE

Sub-crate highlights:

  • meshtastic-core: Core LoRa radio calculations and Meshtastic config utilities in Rust, featuring From implementations into the generated protobuf models.
  • meshtastic-device: Essentially the original src from upstream, minus TypeScript types. Partial support for discovery over TCP, HTTP, Serial (via HWIDs from meshtastic/firmware), etc.
  • meshtastic-protocol: Holds protobuf definitions plus QOL traits for node IDs:
    fn main() -> Result<(), meshtastic_protocol::NodeIdError> {
        // Creating a NodeId from a u32
        let node_id = NodeId::try_from(0x7c3b4eeb)?;
        println!("NodeId: {}", node_id); // Displays as "!7c3b4eeb"
    
        // Parsing a NodeId from a string
        let parsed_node_id = NodeId::from_str("!7c3b4eeb")?;
        assert_eq!(node_id, parsed_node_id);
    
        // Checking if a NodeId is a broadcast address
        assert!(!node_id.is_broadcast());
    
        Ok(())
    }
  • meshtastic-network: Early work on TUN/VPN support across multiple platforms (not just Windows):
    meshtastic-network
    ├── Cargo.toml
    ├── sketch.md
    ├── src
    │   ├── address
    │   ├── bin
    │   ├── error.rs
    │   ├── gateway
    │   ├── interface
    │   ├── lib.rs
    │   ├── models
    │   └── protocol
    └── tests
        ├── tun_service.rs
        ├── tun_test_2.rs
        └── tun_tests.rs
    

@krant, I like the single-binary idea. The Clap flatten approach was my attempt at letting serde figure out whether to invoke V1 or V2 commands. It seemed promising but might be redundant if we go with explicit subcommands or multicall.

Currently, apps/cli/src/bin/meshtastic/mod.rs looks like this:

mod common;
mod v1;
mod v2;

use clap::Parser;
use test_rs::Result;
use tracing::debug;

use common::{init_i18n, init_logging, CommonOpts};
use v1::CliV1;
use v2::CliV2;

#[derive(Parser, Debug)]
#[command(version)]
pub struct Cli {
    #[command(flatten)]
    common: CommonOpts,

    #[command(subcommand)]
    command: Option<v2::Commands>,

    // V1 Legacy flags
    #[command(flatten)]
    legacy: v1::LegacyOpts,
}

enum CommandType {
    V1(CliV1),
    V2(CliV2),
}

impl Cli {
    fn into_command_type(self) -> CommandType {
        if self.legacy.has_operations() {
            CommandType::V1(CliV1 {
                common: self.common,
                legacy: self.legacy,
            })
        } else if let Some(command) = self.command {
            CommandType::V2(CliV2 {
                common: self.common,
                command,
            })
        } else {
            CommandType::V1(CliV1 {
                common: self.common,
                legacy: self.legacy,
            })
        }
    }
}

#[tokio::main]
pub async fn main() -> Result<()> {
    let cli = Cli::parse();
    init_logging(cli.common.log_level, cli.common.log_file.as_ref());
    init_i18n(cli.common.locale);

    match cli.into_command_type() {
        CommandType::V1(v1_cli) => {
            debug!("Running in V1 mode: {v1_cli:?}");
            v1::handle_command(v1_cli).await
        }
        CommandType::V2(v2_cli) => {
            debug!("Running in V2 mode: {v2_cli:?}");
            v2::handle_command(v2_cli).await
        }
    }
}

Legacy to Modern Command Mapping Examples
To put some more precise words to "idiomatic", it was something more hierarchical that I intended to achieve. From my notes I had the following:

// Legacy: --info
// Modern: info

// Legacy: --nodes
// Modern: nodes

// Legacy: --set FIELD VALUE
// Modern: config set FIELD VALUE

// Legacy: --configure FILE
// Modern: config import FILE

// Legacy: --sendtext TEXT --dest DEST --ch-index INDEX --ack
// Modern: message send TEXT --to DEST --channel INDEX --ack

I was also in the middle of an abstract ProtocolCommand trait for async radio commands, but then pivoted to some web/app integration where I plan to use this Rust code. I ended up going down a rabbit hole with TUN cross-platform support as well, but I’m happy to revise or drop any part if it doesn’t fit the repo’s needs.

Let me know what you think—I’m open to any feedback!

@geoffreygarrett
Copy link
Contributor

Regarding the restructuring, @rcarteraz and @krant, I’ll open a PR linked to an issue so we can all discuss the details more easily. Thank you both for your guidance!

@krant
Copy link
Contributor

krant commented Jan 12, 2025

That's impressive @geoffreygarrett, I especially like the idea of separate meshtastic-protocol (or meshtastic-protobuf?) crate - it would allow to ship pre-generated protobuf bindings and drop build-time dependency from prost-build and protoc-bin-vendored, which seems to be one of the pain points of the project. Of course we shall keep the possibility to generate them from .proto files at build stage behind the feature-flag (like it is done in https://github.com/tikv/rust-prometheus for example).
On the other hand excessive crates granularity could create more friction - we should seek for a balance and if some functionality would be easier to have behind a feature flag, then let's go for it.
Regarding CLI v1/v2 structure - I'd better postpone it until we have python-compatible v1, which is by itself quite a challenge. Let's not complicate it prematurely. Until then we could work out v2 specification in separate issue and gather the feedback from the community.
As for global restructuring, it feels like we don't have yet critical mass of code/features which require that. Overall we're already have quite a massive TODO list here:

  • BLE support (@lukipuki)
  • Python-compatible CLI (@krant)
  • TUN/VPN support (@geoffreygarrett)
  • QoL improvements (???)
  • Discovery via HWIDs (???)
  • Pre-generated protobuf bindings (???)

Let's come up with incremental PRs and when/if they merge, code restructuring could be re-evaluated more naturally.

@clayrosenthal
Copy link
Contributor

clayrosenthal commented Jan 16, 2025

Hey, I've just been dabbling with meshtastic recently after I got a radio. I didn't realize there was already a rust client, but I was taking a crack at one and I got pre-generated protobufs working. I'm no rust expert, but I can try and make a PR.

edit: made the PR #30

@clayrosenthal
Copy link
Contributor

I'm trying to learn more rust, so I'd be happy to take a crack at some of the QoL improvements or other features

@krant
Copy link
Contributor

krant commented Jan 16, 2025

Hi @clayrosenthal, nice project, that's nice to see the ecosystem grows! Pre-generated protobuf PR would be very welcoming, especially if it is possible to hide prost-build dependency behind the feature-flag. The idea is to make project buildable on hosts without protoc, but those who want to generate by themselves just do something like cargo --features gen.
While I was writing this comment you've already create the PR, wow, that was lightning fast. 🙂
I'll leave my comments there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants