Proposal and Disscussion of a Plugin System

[PixelboysTM]

Hi everybody, I recently came across this project and really liked the approach. I spend the last few days thinking about a potential Plugin system, including a plugin API, and wanted to discuss my thoughts and define a set of goals such a system should have.

To put one thing first, I would like to also work on a concrete implementation of such system but don't want to decide the design of the system all by myself. Therefore, this discussion.

Requirements (non-exhaustive, only my opinion)

• flexible: The system should allow a variety of different features and provide a safe abstraction around server functionalities.
• easy: Writing a basic plugin should be pretty straight forward and not bloated with verbose initialization logic or abstract Types.
• safe: unsafe rust is easy to mess up, therefore the plugin API should take place in at best 100% safe rust

Simple Implementation proposal

I thought of a plugin as an OS independent file that, if placed in the correct folder, will be automatically loaded by the server.
Generally suitable would be a shared library, which is not OS independent. But Rust allows for cross-compilation.
Therefore, I propose a Plugin file similar to a plugin.jar will be a zip file (if we want an extra file extension) with the following content:

plugin_name.zip (plugin_name.pumpkin)
 ├ plugin.toml
 ├ libplugin.so // linux build lib
 ├ libplugin.dylib // macos build lib
 ├ plugin.dll // window build lib
 └ assets/ //Potential extra files like config or lang

The plugin.toml file provides basic configuration for the plugin, which allows the server a save loading of plugins. A sample file could look something like this, with more :

[plugin]
name = "A really cool Plugin" # The human freindly name of the plugin
identifier = "cool_plugin" # an internal unique identifier of a plugin
version = "0.1.42" # The version of the plugin. 
supported_platforms = ["MacOS", "Linux", "Windows"] # A list of which OSs are supported. Says which build lib files are present, to ensure we only try to open present files.

Plugin loading

I thought about providing plugin functionality in a separate crate but don't think this is easily achievable because we will run into a circular dependency. The main pumpkin server code needs to call Plugin hooks for events etc. and the plugin crate needs all the types from the pumpkin server such as players etc. And as I can see, functionality is integrated, for example for the player in the main pumpkin crate.

I would propose to moving all module definitions from main.rs into its lib.rs to be able to expose a unique interface.
Create a Plugin module with a plugin manager, which is initialized at startup and manages all plugin related tasks.

On startup, the Plugin Manager looks in a predefined plugins directory and optionally in a by an environment variable defined extra directory (useful for developing plugins). The manager looks for .zip and .pumpkin file that include a valid plugin.toml. If a valid toml is present and the server runs one of the plugins supported platforms. We load the associated library file using the libloading crate and load the plugin.

Each Plugin must at least contain two things:

1. a struct that implements the Default and the PumpkinPlugin trait defined by the plugin API, this trait provides a safe and unified way to set up and initialize a plugin

pub trait PumpkinPlugin: Default {
   fn init(&mut self) {}
   fn post_init(&mut self) {}
   ...
}

// in plugin project.
#[derive(Default]
struct MyPlugin;

impl PumpkinPlugin for MyPlugin {
   fn init(&mut self) {
      println!("Hey, my plugin was initialized");
   }
}

2. a plugin loader function which is the entry point and used to receive the Plugins PumpkinPlugin struct. This should be abstracted away to a singular macro call:

#define_plugin(MyPlugin);

// expands to:
#[no_mangle]
pub fn pumpkin_plugin_entry_point() -> Box<dyn Plugin> {
   Box::new(MyPlugin::default())
}

This describes a basic loading and initializing of Plugins with a straightforward implementation. I have additional thoughts on event handling and Property exposing and I will write on these topics in a separate post in this discussion so this one doesn't span any longer.

I am looking forward to your thoughts on the topic.
<3 Tim

[neeleshpoli]

I like the idea of using a zip file for the plugin. However, I think we should use a different file extension from .zip as that could confuse some users. Maybe a zip file but with the .pplugin (for pumpkin plugin) extension?

[PixelboysTM]

also a great idea i like .pplugin

[PixelboysTM]

I also found a thread on the discord mentioning the planned usage of extism. I plan on looking into it to evaluate how suitable this is with the here proposed plans.

[neeleshpoli]

That's not the plan anymore, since extism increases compile times significantly

[PixelboysTM]

oh okay good to know. At first glance that also made the code a lot more inefficient and unsafer, because data needed to be piped through an binary intermediate format.

[Snowiiii]

You should look at #110

[Snowiiii]

TL;DR Extism is very heavy (Indexing & Compile time). And the API is a pain to use

[PixelboysTM]

Forking now to start some work

[PixelboysTM]

Further discussion is happening primaraly at #269