Shared Server Unix Scripts #18
Replies: 8 comments 13 replies
-
|
This sounds like an interesting idea! The documentation states
For sandboxed applications (in this case, I would presume extensions), will another app populate the applicationScriptsDirectory with the correct scripts? Or how would that work? |
Beta Was this translation helpful? Give feedback.
-
|
The script does need to be installed somehow, either by the user directly or a non-sandboxed process. This definitely annoying. One idea I had was a global registry. So, if you happen to have a non-sandboxed process that is doing this, it can consult the list and install them to everyone else's locations. I dunno if this is actually a good idea though. |
Beta Was this translation helpful? Give feedback.
-
|
I have done something similar for Haskell for Mac by providing a |
Beta Was this translation helpful? Give feedback.
-
|
Another question that came up: what language should be used for the scripts? Distributing binaries on macOS is really problematic, so I think a scripting language is a must. Apart from that, I think anything could work, as long as it is uses an agreed-upon script API. Personally, I'm going to use bash. I really hate bash. It's just terrible. But, it has almost no environmental dependencies, and all other languages I have used for scripting (Ruby, Python) do. This means you may make a script that works for some users, but eventually someone will have an unusual Python install and it won't work. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, bash would have my preference, too. (This is what I used in the past also.) |
Beta Was this translation helpful? Give feedback.
-
The script that invokes a language server cannot be shared as it needs to be in the Application Scripts directory of one specific application, but everything else can be in a shared location, where it is invoked by the executable-specific script. (The latter could simply be a shell script setting some paths and doing an exec on the actual language server launcher in a shared location.) |
Beta Was this translation helpful? Give feedback.
-
|
Right of course! Because it won't be sandboxed! This is brilliant. |
Beta Was this translation helpful? Give feedback.
-
|
Generally, I do like your idea. Sharing that work would be great. |
Beta Was this translation helpful? Give feedback.
-
|
Does it need to be a separate script entirely? For example, in Zed, they define this struct: pub struct PluginLspAdapter {
name: WasiFn<(), String>,
fetch_latest_server_version: WasiFn<(), Option<String>>,
fetch_server_binary: WasiFn<(PathBuf, String), Result<LanguageServerBinary, String>>,
cached_server_binary: WasiFn<PathBuf, Option<LanguageServerBinary>>,
initialization_options: WasiFn<(), String>,
language_ids: WasiFn<(), Vec<(String, String)>>,
executor: Arc<Background>,
runtime: Arc<Mutex<Plugin>>,
}Then an extension would fill in the values with their code. A lot of the language servers are just installed through NPM, so they just call that command: async fn fetch_server_binary(
&self,
version: Box<dyn 'static + Send + Any>,
container_dir: PathBuf,
_: &dyn LspAdapterDelegate,
) -> Result<LanguageServerBinary> {
let version = version.downcast::<String>().unwrap();
let server_path = container_dir.join(SERVER_PATH);
if fs::metadata(&server_path).await.is_err() {
self.node
.npm_install_packages(
&container_dir,
&[("vscode-json-languageserver", version.as_str())],
)
.await?;
}
Ok(LanguageServerBinary {
path: self.node.binary_path().await?,
arguments: server_binary_arguments(&server_path),
})
}Cant we do something similar where an extension just has to implement a protocol in Swift? // Already using this struct in CodeEdit
struct LanguageServerBinary {
let execPath: String
let args: [String]
let env: [String: String]?
}
protocol LanguageServer {
func fetchBinary() -> LanguageServerBinary
func checkForUpdates() -> Bool
func getCurrentVersion() -> String
}We can then use |
Beta Was this translation helpful? Give feedback.
-
|
Here's the NodeRuntime code that also installs NPM to the user's system if it doesnt exist. https://github.com/zed-industries/zed/blob/main/crates/node_runtime/src/node_runtime.rs Using NPM as a package manager for LSP binaries might be a good idea here too. What are your thoughts? |
Beta Was this translation helpful? Give feedback.
-
|
Is Zed a sandboxed app? |
Beta Was this translation helpful? Give feedback.
-
|
I would be shocked if Zed is sandboxed. |
Beta Was this translation helpful? Give feedback.
-
Can you quantify this? Examples of ones that do not, just off the top of my head: Rust, Ruby, Go, Swift, Elixir. But, even if lots of others do require NPM, it does not seem directly related to dealing with sandboxing restrictions.
I'm open to it! I don't currently understand what advantages it offers. It might help make for easier scripts, but then someone has to maintain the NPM modules, which I do not know how to do. It seems easier to use whatever install instructions the servers publish themselves. And if those require NPM, then that's what would be used.
If I'm understanding right, an "extension" here is an application-specific thing. The whole idea here is to make something that lots of applications can share. I also wanted to make sure you were aware of ProcessEnv. It has some additional capabilities you will need for executing non-sandboxed servers. In particular the |
Beta Was this translation helpful? Give feedback.
-
|
Seems like a cool idea! I haven't used |
Beta Was this translation helpful? Give feedback.
-
|
This is a good question. I think the answer is yes! Its API has all that is needed to run and interact with long-running processes. And @mchakravarty already integrated it for use with LanguageClient. But I have not personally used it. I'm in the process of experimenting now! |
Beta Was this translation helpful? Give feedback.
-
|
These days, there is one maybe inconvenient aspect about using this for a long-running process. When the process runs, macOS puts a gear icon in your menubar, where you can see which scripts are running and you can stop them. In principle, you should also be able to just launch a language server through a script and then connect to it via a unix socket (instead of stdin/stdout); i.e., it then effectively runs in the background instead as a subprocess. Then, the script is no longer long-running, but you need a mechanism to clean up stale language servers. I didn't try to do this, but I can't see a reason why it won't work. |
Beta Was this translation helpful? Give feedback.
-
|
huh. Yeah, that's an important detail for sure... I haven't been testing this with a sandboxed process, yet, so that's why I didn't notice. |
Beta Was this translation helpful? Give feedback.
-
Hey @mattmassicotte thanks for notifying me about these exciting developments. How are the experiments going? The point about "stale" servers touches on an intuition of mine about this ... LSPService / LocalService has the "advantage" (I feel) that it's an executable under my control that keeps running even when the LSP Server or other executable doesn't run anymore and even if it lost the connection to the LSP server or executable. Isn't what we really need the ability to freely run any command in a shell instead of running only one specific script? We want to be able to kill disconnected LSP servers and re-launch crashed ones, right? Could we use NSUserUnixTask in that generalized way, maybe by passing commands to run as argument to the script? But that would create a new shell session for each command ... 🤪 I guess I haven't quite catched up yet to the state of this discussion 😅 |
Beta Was this translation helpful? Give feedback.
-
|
I have been little distracted, but things are going really well. I'm pretty sure this approach will work, though it does come with trade-offs. That user-visible UI is a problem for sure. I believe you can have all the control you want here without needing an external service. You are under complete control of the process running via This approach is not at all about controlling the servers, which I think is a higher-level task. It is about installing, upgrading, and launching them in a way that is sandbox-compatible. |
Beta Was this translation helpful? Give feedback.
-
|
I threw together a basic repo to hold the scripts, documentation, as well as a Swift package for interaction/installation https://github.com/ChimeHQ/LanguageServerScripts. Currently non-functional. |
Beta Was this translation helpful? Give feedback.
-
|
I've added some more functionality and documentation to that repo, including a proof-of-concept that can start up gopls (the Go language server) via symlinks to a shared global (well, per-user...) location. It was incredibly difficult to figure out how to re-construct the user's shell environment in Bash, but I think I managed to get it. I have this working with |
Beta Was this translation helpful? Give feedback.
-
The concept of using a (Unix Task)[https://developer.apple.com/documentation/foundation/nsuserunixtask] to run language servers was introduced to me by @mchakravarty. I wasn't even aware of the system before! It is the first time I've ever seen a true sandbox-compatible solution that is explicitly available to all applications, App Store-delivered included.
The thing is, installing and running language servers is, in general, very complex. I think I would be amazing to have a shared pool of scripts which abstracts the process. The idea is a uniform interface for these actions:
Maybe others?
One big limitation of the Application Script system is the directories are scoped to the executable. So, it seems like it would be impossible to have a shared installation location. But I still think such an approach would be beneficial. Does anyone have any thoughts on this?
@Wouter01 @FastestMolasses
Beta Was this translation helpful? Give feedback.
All reactions