Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Viaduct replacement demo #6018

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Viaduct replacement demo #6018

wants to merge 1 commit into from

Conversation

bendk
Copy link
Contributor

@bendk bendk commented Dec 21, 2023

This shows off a potential viaduct replacement that uses new UniFFI features.

The 1000ft view is that:

Check out the fairy-bridge README and the demo README for details. Execute run-demo.py to test it out yourself.

The UniFFI features are still a WIP. This is currently using a branch in my repo. The current plan for getting these into UniFFI main is:

The Desktop plan needs to be explored more. I believe there should be a way to use Necko in Rust code, but that needs to be verified.

I made this using UniFFI proc-macros, to test out the technology and I have to say it was very pleasant to work with.

@bendk bendk added WIP Work in progress discuss labels Dec 21, 2023
@bendk bendk changed the title Viadict replacement demo Viaduct replacement demo Dec 21, 2023
@bendk bendk force-pushed the fairy-bridge branch 2 times, most recently from 88045a9 to 92d1861 Compare December 21, 2023 19:33
@codecov-commenter
Copy link

codecov-commenter commented Dec 21, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (main@a8c15ba). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6018   +/-   ##
=======================================
  Coverage        ?   22.64%           
=======================================
  Files           ?      330           
  Lines           ?    29814           
  Branches        ?        0           
=======================================
  Hits            ?     6750           
  Misses          ?    23064           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@bendk bendk force-pushed the fairy-bridge branch 2 times, most recently from 0810d75 to 11a0ba5 Compare December 21, 2023 19:52
Copy link
Member

@mhammond mhammond left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Far out, this is fantastic - it's small and simple. I guess seeing what the necko backend looks like is where the fun starts? Either way, I think this is worth pursuing.


## Cookies / State

Cookies are explicitly not supported at the moment, adding them would require a separate security
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know you copy/pasted this, but can we tighten this more? eg, "Cookies and state are outside the scope of this library. Any such functionality is the responsibility of the consumer"


## Name

`fairy-bridge` is named after the Fairy Bridge (Xian Ren Qiao) -- the largest known natural bridge in the world, located in northwestern Guangxi Province, China.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

love it :)

pub struct BackendSettings {
// Connection timeout (in ms)
#[uniffi(default = None)]
pub connect_timeout: Option<u32>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, I'm sure you copied these, but I wonder if these are actually appropriate defaults for timeouts? If someone neglects to think about this, it seems better the request times out too soon than never times out? Or maybe make them a non-option - I'm not sure it ever makes sense to say "it's fine if this request literally never completes" - much like we are implicitly also not allowing "it's fine if redirects never complete" below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great point, but we'll need a UniFFI fix for this: mozilla/uniffi-rs#1925

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While that PR makes sense, do we really need that here? ie, can we just make those values u32? I think that would offer ~600 hours of timeout?

(I don't really care, it's more just random thoughts about the API rather than a concrete suggestion)

mod error;
pub mod headers;
mod request;
#[cfg(feature = "reqwest")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there doesn't appear to be a feature of this name? (I was going to ask whether it should be, say, "backend-reqwest" or similar, even though it really doesn't matter :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The feature is implicit because of the optional dependency. But "backend-reqwest" sounds better to me too.

* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */

use once_cell::sync::OnceCell;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be now be using std::cell::OnceCell?

#[derive(uniffi::Record)]
pub struct Request {
pub method: Method,
pub url: String,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

many of your fns use Url - should we try and use that here and leverage custom types?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes a lot of sense, but I'm slightly worried about conversion errors. If Request.url is an Url, then Response.url maybe also should be. But that means that a request could fail if the foreign code returns an invalid URL. Maybe that's bad? I'm not really sure about this one.

/// this to fail hard with an easy to track down panic, than for e.g. `sync`
/// to fail with a JSON parse error (which we'd probably attribute to
/// corrupt data on the server, or something).
pub fn json<T: ?Sized + serde::Serialize>(mut self, val: &T) -> Self {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the trait bounds are cool, but I wonder if, for all the reasons you mention in the comment, this should just take serde_json::Value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried that out, but I felt like it made the consumer code kind of awkward. I do find this docstring a bit weird though. I'm going to update the trait bounds a bit and return a Result. I think it's more natural for consumers to deal with serialization errors than have the function panic.

pub fn json<T: ?Sized + serde::Serialize>(mut self, val: &T) -> Self {
self.body =
Some(serde_json::to_vec(val).expect("Rust component bug: serde_json::to_vec failure"));
if !self.headers.contains_key(headers::CONTENT_TYPE) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would seem odd if the existing header had anything but app/json? (Not sure what you should do in that case - if it turns out a use-case for this was legitimate, even logging would be a PITA. OTOH, if there were no valid use-cases, then this would probably result in hard-to-track-down errors which naively might also be attributed to the server instead of the app

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Let's just force the header to be application/json. json() is just a convenience method after all, the user can always serialize the data themselves.

pub fn init_reqwest_backend(settings: BackendSettings) -> Result<(), FairyBridgeError> {
// Create a multi-threaded runtime, with 1 worker thread.
//
// The difference between this and the current thread worker, is that it will manage the worker
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what "the current thread worker" is in this context - the current thread is likely to just be a normal thread and not associated with tokio at all, right? (well, tbh I don't really understand this comment at all :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a tokio thing. The runtime always piggybacks on the current thread. You can spawn tasks on the worker, but nothing happens until you call block_on, which then blocks the current thread while the tasks execute. I thought it would be the right runtime, since we only need one thread, but the execution model is very different. I'll try to update this comment to make it more clear.

* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */

use std::collections::HashMap;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like it might be fine for this to be in lib.rs? (Also fine not to be, I'm just throwing comments out there as they fleetingly enter and leave my brain :)

@bendk
Copy link
Contributor Author

bendk commented Jan 3, 2024

Thanks for the review! I just updated a bunch of code based on the suggestions. I think you're right that the real interesting part will come from integrating with Necko. If the UniFFI bits are the boring part, I guess that's a good sign.

@bendk bendk force-pushed the fairy-bridge branch 8 times, most recently from 8272beb to dac259b Compare January 11, 2024 20:06
@bendk bendk force-pushed the fairy-bridge branch 2 times, most recently from 476e13f to c003902 Compare May 21, 2024 16:40
@bendk bendk force-pushed the fairy-bridge branch 2 times, most recently from 51e7bb6 to d1d6654 Compare June 21, 2024 15:45
This shows off a potential viaduct replacement that uses new UniFFI
features. Check out `components/fairy-bridge/README.md` and
`examples/fairy-bridge-demo/README.md` for details.  Execute
`examples/fairy-bridge-demo/run-demo.py` to test it out yourself.

The UniFFI features are still a WIP.  This is currently using a branch
in my repo.  The current plan for getting these into UniFFI main is:
  - Get the `0.26.0` release out the door
  - Merge PR mozilla#1818 into `main`
  - Merge my `async-trait-interfaces` branch into main (probably using a
    few smaller PRs)

There is a C++ backend using libcurl.  I think that necko shouldn't be
any harder than this.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discuss WIP Work in progress
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants