[PATCH] Compiler additions for Language Server Protocol
I am currently working on an A2 LSP Language Server for Language Server Protocol (to be used in VS Code and other IDEs).
The current prototype is available at <https://github.com/schierlm/OberonEmulator/tree/wip-lsp/LSPServer/A2> (in particular, LSPScannerMetadata and LSPSyntaxTreeWalker may also be useful to other editor/IDEs, outside of the context of LSP language servers, if they want to navigate to definition/references and/or reformat the source code).
Attached is a patch that includes the changes I had to make to the Fox compiler to make this possible. If you want it split into more parts, or other changes, tell me and I will rebase.
Basic workflow is:
-
LSP server dumps file to disk and calls compiler, providing parameters "-p=Win64 --objectFile='' --symbolFile=Textual -w" and a new callback. A small fix in argument parsing is needed, so that platform arguments (from -p=Win64) do not create errors if object file (from --objectFile='') does not support them.
-
Compiler calls FoxScanner and FoxParser. FoxScanner supports a new ScannerDiagnostics callback, which is used to report scanned symbols and their positions back to the LSP server.
-
At some points (particular for document formatting support, distinguising PROCEDURE types from PROCEDURE declarations, and marking boundaries for semantic signature help), a distinction is needed between different usages of the same symbol. Therefore, a concept of subSymbol is introduced in the scanner (there are only 12 subsymbols so far), which can be signaled by the parser back to the scanner.
-
The parser builds the parse tree. This parse tree is augmented with a few more Position objects (e.g. position of: end repeat identifier, export markers, prefix+suffix of qualified types), which are needed to locate these objects when navigating to definition/references.
-
New SourceDiagnostics callback in Compiler module is called first time, reporting the parse tree to the LSP, which uses it to get the module name and imports for dependency tracking.
-
Compiler calls FoxSemanticChecker, which resolves identifiers
-
Compiler calls SourceDiagnostics callback again, this time with resolved identifiers. Source tree is traversed by LSPSyntaxTreeWalker to extract positions of definitions, references, and outline scopes.
-
Compiler calls SourceDiagnostics when creating symbol file. LSP server will use this to parse the symbol file (FoxTextualSymbolFile) and use LSPSyntaxTreeWalker to match positions of source file to positions of symbol file. This is needed as the syntax tree for imported modules will include the positions from symbol file and not from object file. It will also use the callback to detect if the symbol file has changed, which will cause an event to the IDE so that it will (ask to) reanalyze all dependant modules.
For auto-completion, the workflow is similar, except that FoxSemanticChecker has got a new "FindScopeAtPosition" feature, which will report the scope that was used to try to resolve the identifier at a given position. Similar to FindPC in the backend. LSP server can then traverse the scope and search for identifiers that match the already entered prefix. Also, a flag is needed to run the semantic checker even if parse errors occurred.
Reformatting is a bit simpler, as the process stops after scanning/parsing, using LSPScannerMetadata class to obtain required spacing information for the symbols, which gets reported back to the editor.
All those are currently working well enough to be used as a daily driver. There are still a few bugs, but I presume the majority to be in my LSP code and not in the compiler patches attached.
(from redmine: issue id 115, created on 2022-01-23 by root)
- Uploads:
- CompilerChanges.patch Compiler changes