Support for macOS with Apple Silicon

[tonyfettes]

Sorry for opening an issue for such a supporting problem.

I'm running the tests of this project using pytest on macOS with Apple Silicon, but I got any error saying that only linux-x64 is the only supported platform for now. Will there be any plan to support macOS? Or can I get a guidance on how to add support for such platform?

Thank you in advance.

[LakshyAAAgrawal]

Hey, thanks for your interest!

multilspy itself supports all the platforms, win, linux and macOS, but it depends on whether the language server is available for your platform or not.

Different languages use different language servers, for instance, Java uses Eclipse JDT.LS, Python uses JEDI-language-server, etc. Each of these language server binaries need to be installed from their respective sources, and configuration performed. the directory src/monitors4codegen/multilspy/language_servers organises the code for setting up different language servers. Since I have not been able to test multilspy on all platforms for all languages, I had added an error message at src/monitors4codegen/multilspy/language_servers/rust_analyzer/rust_analyzer.py#L52-L55, or equivalent line in other language server files, so once the support is added/tested, this line can be removed.

For multilspy to work for a particular language on a platform, the server needs to be set up for the specific platform. For example, the file src/monitors4codegen/multilspy/language_servers/rust_analyzer/runtime_dependencies.json specifies the url from which the binaries for rust language server can be fetched for different supported platforms. To add a new platform, you can add the url for the binary for your platform by reviewing https://github.com/rust-lang/rust-analyzer/releases/ and making a new entry in the src/monitors4codegen/multilspy/language_servers/rust_analyzer/runtime_dependencies.json file. The runtime_dependencies file is used in the setup_runtime_dependencies method in file src/monitors4codegen/multilspy/language_servers/rust_analyzer/rust_analyzer.py#L42-L79, which downloads and extracts the binary, and returns the path to the executable file within the extracted directory.

The other language server directories are organised in a similar manner, having a json file named runtime_dependencies.json specifying the url for fetching binaries, and a setup_runtime_dependencies method, to do the setup. Kindly let me know which language you would like support for, and I can help you set it up.

[tonyfettes]

It works! Thank you for your detailed response, it's super helpful.

[LakshyAAAgrawal]

Hey @tonyfettes! Thank you very much for letting me know.

May I request you to please create a PR with the changes you made based on the above comments to get it working on Apple Silicon? It would be very useful to others. I am not able to do it since I don't have a device with apple silicon.

[TsukimiRini]

hi, i'm trying to access support for macOS with Intel silicon, and I did simiar modification with pr#18.
As a result, I can successfully launch multispy, but there is something wrong when I tried to request completions from eclipse-jdtls via multispy.
When I request completions involving cross-module identifiers (e.g. accessing a member field of an object which belongs to other modules), multispy always returns an empty list.
But when I request in-file or in-module identifier completions, it works.
Meanwhile, I also implemented a simple lsp client myself, sending the same requests to it and the client can return appropriate responses; furthermore, I transferred the multispy code to run on linux and it also works.
I wonder whether @tonyfettes encountered the same situation?

[LakshyAAAgrawal]

@TsukimiRini , Thanks a lot for checking out multilspy. Kindly look into the following:

1. Please have a look at the allow_incomplete flag in the request_completions method: https://github.com/microsoft/monitors4codegen/blob/022c65efb19cf6046d0b67960ca232ae7a351af4/src/monitors4codegen/multilspy/language_server.py#L467-L469
   Please check if setting it to True provides the expected results. If it does, I will work on improved documentation for the same.
2. If it does not work, could you please let me know how are you launching the server and requesting from your custom client? Maybe that will give me an idea about what changes to make to get this working in Apple Silicon.
3. Finally, I request you to kindly provide the scenario you tested as a (failing)-testcase similar to testcases in https://github.com/microsoft/monitors4codegen/blob/main/tests/multilspy/test_multilspy_java.py, so that I can reproduce the behavior on my end and try to fix the issue.

[TsukimiRini]

Hi @LakshyAAAgrawal , thank you for your detailed and timely reply!
I tried the first point you mentioned and sadly it still doesn't work, but I have something else to share with you to figure out whether I have made something wrong.
I printed out the multispy log and found that every time any request is sent, a bunch of errors keep being repeated, so the log appears to be:

INFO:multilspy:{"time": "2023-12-28 13:42:57", "level": "INFO", "caller_file": "eclipse_jdtls.py", "caller_name": "start_server", "caller_line": 368, "message": "Starting EclipseJDTLS server process"}
INFO:multilspy:{"time": "2023-12-28 13:42:57", "level": "INFO", "caller_file": "eclipse_jdtls.py", "caller_name": "start_server", "caller_line": 372, "message": "Sending initialize request from LSP client to LSP server and awaiting response"}
INFO:multilspy:{"time": "2023-12-28 13:43:00", "level": "INFO", "caller_file": "eclipse_jdtls.py", "caller_name": "window_log_message", "caller_line": 354, "message": "LSP: window/logMessage: {\"type\": 3, \"message\": \"2023\u5e7412\u670828\u65e5 \u4e0b\u53481:43:00 Main thread is waiting\"}"}
INFO:multilspy:{"time": "2023-12-28 13:43:00", "level": "INFO", "caller_file": "eclipse_jdtls.py", "caller_name": "window_log_message", "caller_line": 354, "message": "LSP: window/logMessage: {\"type\": 3, \"message\": \"2023\u5e7412\u670828\u65e5 \u4e0b\u53481:43:00 >> initialize\"}"}
...
INFO:multilspy:{"time": "2023-12-28 13:43:00", "level": "INFO", "caller_file": "eclipse_jdtls.py", "caller_name": "window_log_message", "caller_line": 354, "message": "LSP: window/logMessage: {\"type\": 1, \"message\": \"2023\u5e7412\u670828\u65e5 \u4e0b\u53481:43:00 Exception occurred executing command line.\\nCannot run program \"/usr/libexec/java_home\": error=2, No such file or directory\\njava.io.IOException: Cannot run program \"/usr/libexec/java_home\": error=2, No such file or directory
...
INFO:multilspy:{"time": "2023-12-28 13:43:00", "level": "INFO", "caller_file": "eclipse_jdtls.py", "caller_name": "window_log_message", "caller_line": 354, "message": "LSP: window/logMessage: {\"type\": 3, \"message\": \"2023\u5e7412\u670828\u65e5 \u4e0b\u53481:43:00 Setting runtime JavaSE-17-[my_project_root]/monitors4codegen/src/monitors4codegen/multilspy/language_servers/eclipse_jdtls/static/vscode-java/extension/jre/17.0.8.1-macosx-x86_64 as default global VM\"}"}
INFO:multilspy:{"time": "2023-12-28 13:43:00", "level": "INFO", "caller_file": "eclipse_jdtls.py", "caller_name": "window_log_message", "caller_line": 354, "message": "LSP: window/logMessage: {\"type\": 1, \"message\": \"2023\u5e7412\u670828\u65e5 \u4e0b\u53481:43:00 Runtime at \"[my_project_root]/monitors4codegen/src/monitors4codegen/multilspy/language_servers/eclipse_jdtls/static/vscode-java/extension/jre/17.0.8.1-macosx-x86_64\" is not compatible with the \"JavaSE-17\" environment\"}"}
INFO:multilspy:{"time": "2023-12-28 13:43:00", "level": "INFO", "caller_file": "eclipse_jdtls.py", "caller_name": "window_log_message", "caller_line": 354, "message": "LSP: window/logMessage: {\"type\": 3, \"message\": \"2023\u5e7412\u670828\u65e5 \u4e0b\u53481:43:00 JVM Runtimes changed, saving new configuration\"}"}

I suppose that maybe my JAVA environment is broken, so I try to fix everything in the environment related to JAVA, e.g. /usr/libexec/java_home, $JAVA_HOME. However, this error still exists.
I'm not familiar with the vscode-java plugin. Please point it out if I made something wrong. Thank you!

[TsukimiRini]

As for my custom client, it's a primitive one which communicates with LS via socket. I run an LS and make it listen on a certain port. Then I launch a client sending requests to the server in the format of RPC JSON object.
I think there is no particular difference in the way of LS launching between this simple client and multispy. Maybe the critical part lies in that when I use my custom client, I set up the environment mannually to launch LS, by setting the $JAVA_HOME env variable. Also, I directly run jdtls without vscode-java.

[TsukimiRini]

Ohhh..Sorry to bother you guys. It turns out that though I have checked /usr/libexec/java_home and $JAVA_HOME, I have wrongly put my JDK path behind /usr/bin in $PATH, which only contains an old version of JAVA.
It's solved! I'm happy to find multispy can also adapts to darwin-x64 easily! Thank you for your help! @LakshyAAAgrawal

[LakshyAAAgrawal]

@TsukimiRini, I am glad that multilspy worked for your usecase.

Did supporting your usecase require any changes to multilspy? I will soon be reviewing the PR microsoft/monitors4codegen#18 by @tonyfettes. If your usecase required any changes beyond that, I would be glad if you could contribute the changes to multilspy!