Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rust deserealize fails with Stack Overflow #2341

Open
poborin opened this issue Oct 7, 2024 · 8 comments
Open

Rust deserealize fails with Stack Overflow #2341

poborin opened this issue Oct 7, 2024 · 8 comments
Labels
awaiting Waiting for responses, PR, further discussions, upstream release, etc bug Something isn't working

Comments

@poborin
Copy link

poborin commented Oct 7, 2024

Describe the bug

I have a fairly simple data structure:

data structure
use flutter_rust_bridge::frb;
use log::{error, info};
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct TrainingPlan {
    pub(crate) cycles: Vec<CycleElement>,
    /// traning plan title
    pub(crate) title: String,
}

impl super::common_traits::Deserialize for TrainingPlan {
    #[frb(sync)]
    fn deserialize(content: String) -> Result<Self, String> {
        info!("<!> deserializing training plan: {}", content);
        match serde_json::from_str(&content) {
            Ok(plan) => Ok(plan),
            Err(e) => Err(e.to_string())
        }
    }
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct CycleElement {
    /// Type of cycle
    pub(crate) cycle_type: CycleType,
    /// End date of the cycle or phase
    pub(crate) end_date: String,
    /// Name or description of the cycle or phase
    pub(crate) name: String,
    pub(crate) phases: Vec<PhaseElement>,
    pub(crate) sessions: Vec<SessionElement>,
    /// Start date of the cycle or phase
    pub(crate) start_date: String,
    /// Nested cycles for finer planning
    pub(crate) sub_cycles: Vec<CycleElement>,
    pub(crate) target: Target,
}

/// Type of cycle
#[derive(Serialize, Deserialize, Clone, Debug)]
#[serde(rename_all = "kebab-case")]
pub enum CycleType {
    #[serde(rename = "in-season")]
    InSeason,
    Macrocycle,
    Mesocycle,
    Microcycle,
    #[serde(rename = "off-season")]
    OffSeason,
    #[serde(rename = "post-season")]
    PostSeason,
    #[serde(rename = "pre-season")]
    PreSeason,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct PhaseElement {
    /// Training phase
    pub(crate) phase: Phase,
    /// Number of weeks in the phase
    pub(crate) weeks: u8,
}

/// Training phase
#[derive(Serialize, Deserialize, Clone, Debug)]
#[serde(rename_all = "kebab-case")]
pub enum Phase {
    #[serde(rename = "in-season")]
    InSeason,
    #[serde(rename = "off-season")]
    OffSeason,
    #[serde(rename = "post-season")]
    PostSeason,
    #[serde(rename = "pre-season")]
    PreSeason,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct SessionElement {
    /// Day of the week for the training session
    pub(crate) day: Day,
    /// Duration of the training session
    pub(crate) duration: String,
    /// Focus areas of the training session
    pub(crate) focus_areas: String,
    /// Intensity level of the training session
    pub(crate) intensity: String,
    /// Additional notes for the training session
    pub(crate) notes: String,
    /// Focus areas for recovery sessions (e.g., flexibility, relaxation)
    pub(crate) recovery_focus: String,
    /// Specific details or exercises for rehabilitation
    pub(crate) rehab_details: String,
    /// Start time of the training session
    pub(crate) start_time: String,
    /// Type of training session
    #[serde(rename = "type")]
    pub(crate) coordinat_type: String,
}

/// Day of the week for the training session
#[derive(Serialize, Deserialize, Clone, Debug)]
#[serde(rename_all = "snake_case")]
pub enum Day {
    Friday,
    Monday,
    Saturday,
    Sunday,
    Thursday,
    Tuesday,
    Wednesday,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct Target {
    /// Name of individual athlete or team
    pub(crate) name: String,
    /// Target audience type
    #[serde(rename = "type")]
    pub(crate) target_type: Type,
}

/// Target audience type
#[derive(Serialize, Deserialize, Clone, Debug)]
#[serde(rename_all = "snake_case")]
pub enum Type {
    Individual,
    Team,
}

#[cfg(test)]
mod tests {
    use crate::api::types::common_traits::Deserialize;
    use crate::api::types::training_plan::TrainingPlan;
    use cargo_metadata;
    use cargo_metadata::MetadataCommand;
    use serde_json;
    use std::fs;

    #[test]
    fn test_json_deserialization() {
        let metadata = MetadataCommand::new()
            .no_deps()
            .exec()
            .unwrap();

        // Construct the full path to the file
        let file_path = metadata.workspace_root
            .join("test_data")
            .join("training_plan_generated.json");

        println!("{file_path:?}");

        // Read the contents of the file
        let traning_plan = fs::read_to_string(file_path).unwrap();

        // Attempt to deserialize JSON into TrainingPlan struct
        let result = TrainingPlan::deserialize(traning_plan);

        // Assert that deserialization was successful
        assert!(result.is_ok(), "JSON deserialization failed: {result:?}");

        // Further assertions can be added here to validate the deserialized data
        let training_plan = result.unwrap();
        println!("{training_plan:?}");
        // For example:
        // assert_eq!(training_plan.title, PlanTitle("Example Plan".to_string()));
    }
}

and the corresponding test data

test JSON
{
  "title": "[Your Club Name] Rowing Team Annual Plan",
  "cycles": [
    {
      "cycle_type": "macrocycle",
      "name": "Annual Training Cycle",
      "start_date": "[Please confirm the specific start date]",
      "end_date": "[Please confirm the specific end date]",
      "phases": [
        {
          "phase": "pre-season",
          "weeks": 12
        },
        {
          "phase": "in-season",
          "weeks": 24
        },
        {
          "phase": "post-season",
          "weeks": 4
        }
      ],
      "sub_cycles": [],
      "sessions": [
        {
          "day": "tuesday",
          "type": "ergo",
          "focus_areas": "Endurance building, technique improvement",
          "start_time": "06:00",
          "duration": "1 hour",
          "intensity": "moderate to high",
          "notes": "",
          "recovery_focus": "",
          "rehab_details": ""
        },
        {
          "day": "thursday",
          "type": "on-water",
          "focus_areas": "Technique work, race pace intervals",
          "start_time": "06:00",
          "duration": "1.5 hours",
          "intensity": "moderate",
          "notes": "",
          "recovery_focus": "",
          "rehab_details": ""
        },
        {
          "day": "saturday",
          "type": "on-water",
          "focus_areas": "Long endurance rows, consistent pacing",
          "start_time": "07:00",
          "duration": "2 hours",
          "intensity": "low to moderate",
          "notes": "",
          "recovery_focus": "",
          "rehab_details": ""
        },
        {
          "day": "monday",
          "type": "gym",
          "focus_areas": "Core stability, strength building, functional movements",
          "start_time": "17:00",
          "duration": "1 hour",
          "intensity": "moderate",
          "notes": "",
          "recovery_focus": "",
          "rehab_details": ""
        }
      ],
      "target": {
        "type": "team",
        "name": "Rowing Team of 20 Members"
      }
    }
  ]
}

So, it's a small and simple document and the data structure. However, when I execute the deserialize() method in the web app e.g.

final test = TrainingPlan.deserialize(content: jsonString);

It fails with the Stack Overflow exception

PanicException(EXECUTE_SYNC_ABORT Stack Overflow pkg/rust_lib_fusion_app.js 301:22
frb_pde_ffi_dispatcher_sync
packages/flutter_rust_bridge/src/generalized_frb_rust_binding/_web.dart 37:12  pdeFfiDispatcherSync
packages/flutter_rust_bridge/src/codec/pde.dart 24:37                          pdeCallFfi
packages/fusion_app/src/rust/frb_generated.dart 903:16                         <fn>
packages/flutter_rust_bridge/src/main_components/handler.dart 25:24            executeSync
packages/fusion_app/src/rust/frb_generated.dart 899:20                         crateApiTypesTrainingPlanTrainingPlanDeserialize
packages/fusion_app/src/rust/api/types/training_plan.dart 225:12               deserialize
packages/fusion_app/src/ui/assistant/programme.dart 18:33                      _loadTestData
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50             <fn>
dart-sdk/lib/async/zone.dart 1661:54                                           runUnary
dart-sdk/lib/async/future_impl.dart 163:18                                     handleValue
dart-sdk/lib/async/future_impl.dart 861:44                                     handleValueCallback
dart-sdk/lib/async/future_impl.dart 890:13                                     _propagateToListeners
dart-sdk/lib/async/future_impl.dart 666:5                                      [_completeWithValue]
dart-sdk/lib/async/future_impl.dart 736:7                                      callback
dart-sdk/lib/async/schedule_microtask.dart 40:11                               _microtaskLoop
dart-sdk/lib/async/schedule_microtask.dart 49:5                                _startMicrotaskLoop
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7             <fn>
)

Steps to reproduce

  1. create a assets/test_data/training_plan.json at the root of the project.
  2. add assets to pubspec.yaml
  assets:
    - assets/test_data/training_plan.json
  1. load the test data in the widget builder:
Future<TrainingPlan?> _loadTestData() async {
    try {
      final jsonString =
          await rootBundle.loadString('assets/test_data/training_plan.json');
      print("<!> $jsonString");

      return TrainingPlan.deserialize(content: jsonString);
    } catch (e) {
      print('Error loading test data: $e');
      return null;
    }
  }
  1. run a web app: flutter run --web-header=Cross-Origin-Opener-Policy=same-origin --web-header=Cross-Origin-Embedder-Policy=require-corp

Logs

adding logs fails an issue submission. I can provide logs upon request

Expected behavior

No response

Generated binding code

No response

OS

MacOS

Version of flutter_rust_bridge_codegen

2.4.0

Flutter info

[✓] Flutter (Channel stable, 3.24.3, on macOS 14.0 23A344 darwin-arm64, locale en-AU)
    • Flutter version 3.24.3 on channel stable at /opt/homebrew/Caskroom/flutter/3.24.3/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 2663184aa7 (4 weeks ago), 2024-09-11 16:27:48 -0500
    • Engine revision 36335019a8
    • Dart version 3.5.3
    • DevTools version 2.37.3

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at /Users/poborin/Library/Android/sdk
    • Platform android-34, build-tools 33.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.4)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15F31d
    • CocoaPods version 1.15.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2024.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)

[✓] VS Code (version 1.93.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.98.0

[✓] Connected device (3 available)
    • macOS (desktop)                 • macos                 • darwin-arm64   • macOS 14.0 23A344 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad • darwin         • macOS 14.0 23A344 darwin-arm64
    • Chrome (web)                    • chrome                • web-javascript • Google Chrome 129.0.6668.90

[✓] Network resources
    • All expected network resources are available.

• No issues found!

Version of clang++

19.1.0

Additional context

No response

@poborin poborin added the bug Something isn't working label Oct 7, 2024
@poborin poborin changed the title deserealize issue deserealize fails Oct 7, 2024
@poborin poborin changed the title deserealize fails Rust deserealize fails with Stack Overflow Oct 7, 2024
@fzyzcjy
Copy link
Owner

fzyzcjy commented Oct 7, 2024

Hi, could you please firstly check whether it is related to frb or your JSON deserialize code? And could you please show Rust stacktrace during the panic

@fzyzcjy fzyzcjy added the awaiting Waiting for responses, PR, further discussions, upstream release, etc label Oct 7, 2024
@poborin
Copy link
Author

poborin commented Oct 7, 2024

I don't think this is a rust deserealize issue, as the Rust test succeeds. The data structure provides a test case that checks it.

How can I add a stacktrace logging? I tried following in lib.rs?

use flutter_rust_bridge::{frb, setup_default_user_utils};
use std::backtrace::Backtrace;
use std::cell::Cell;
use std::env;

pub mod api;
mod frb_generated;

thread_local! {
    static BACKTRACE: Cell<Option<Backtrace>> = const { Cell::new(None) };
}

#[frb(init)]
pub fn init_app() {
    env::set_var("RUST_BACKTRACE", "1");
    setup_default_user_utils();

    std::panic::set_hook(Box::new(|_| {
        let trace = Backtrace::capture();
        BACKTRACE.set(Some(trace));
    }));
}

and then the catch in deserealize:

impl TrainingPlan {
    #[frb(sync)]
    pub fn test_deserialize(content: String) -> Result<Self, String> {
        info!("<!> deserializing training plan: {}", content);

        let result = panic::catch_unwind(|| {
            serde_json::from_str(&content)
        });

        match result {
            Ok(Ok(plan)) => Ok(plan),
            Ok(Err(e)) => {
                error!("Deserialization error: {}", e.to_string());
                Err(e.to_string())
            }
            Err(e) => {
                let b = BACKTRACE.take().unwrap();
                error!("at panic:\n{}", b);
                let err = format!("A panic occurred during deserialization: {e:?}");
                error!("{}", err);
                Err(err)
            }
        }
    }
}

No matter what, I don't get a stack trace.

@poborin
Copy link
Author

poborin commented Oct 7, 2024

update:

  1. I created a fn test_panic() just to see if I get a backtrace and it works.
  2. I tried async #[frb] fn deserialize(...) and it fails with a different error:
<!> test async
Error loading test data: PanicException(RangeError: Maximum call stack size exceeded)
Failed to initialize: Uncaught RangeError: Maximum call stack size exceeded
  1. Meanwhile, the sync deserialize fails without a Rust stack trace.

@fzyzcjy
Copy link
Owner

fzyzcjy commented Oct 7, 2024

stacktrace: https://cjycode.com/flutter_rust_bridge/guides/how-to/stack-trace (e.g. ensure you create a blank new project with default generated code etc)

Based on your info, I guess there may be another reason: It is not because json deserialization, but because returning the nesting type. Again, with stack trace it will be clear to see what is going on.

It would be great to have a minimal reproducible sample (e.g. if it is the nesting type problem, then we can produce a dozen line of code that reproduce it).

@poborin
Copy link
Author

poborin commented Oct 7, 2024

I created a minimal library to reproduce an error https://github.com/poborin/frb_deserialize

a command to execute:

flutter run --web-header=Cross-Origin-Opener-Policy=same-origin --web-header=Cross-Origin-Embedder-Policy=require-corp -d chrome

@fzyzcjy
Copy link
Owner

fzyzcjy commented Oct 8, 2024

Great! However, "minimal reproducible sample" often means to reduce to a minimal complexity. For example, in https://github.com/poborin/frb_deserialize/blob/main/rust/src/api/training_plan.rs, maybe it can be as short as:

pub struct A {
  field: Vec<A>,
}

pub fn f() -> A {
  A { field: vec![A { field: vec![]]}
}

i.e. remove whatever does not cause the problem, such as unneeded fields, json deserialization, etc

@poborin
Copy link
Author

poborin commented Oct 9, 2024

I have updated the repo. It turns that deserializer fails with u8:

use flutter_rust_bridge::frb;
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct TrainingPlan {
    pub(crate) weeks: u8,
}

impl TrainingPlan {
    #[frb(sync)]
    pub fn test_deserialize(content: String) -> Result<Self, String> {
        serde_json::from_str(&content).map_err(|e| e.to_string())
        
    }
}
{
  "weeks": 10
}

@fzyzcjy
Copy link
Owner

fzyzcjy commented Oct 9, 2024

Hmm, do you mean it is a serde_json::from_str error, or an error in frb?

To isolate the problem, try

    #[frb(sync)]
    pub fn test_deserialize(content: String) -> Result<Self, String> {
        Ok(Self {weeks: 10})
    }

and also try to serde_json::from_str in pure Rust tests (maybe Rust wasm if you are in web)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting Waiting for responses, PR, further discussions, upstream release, etc bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants