Loz Update: Now with Local LLM Support and Natural Language Linux Commands
I've been developing the Loz project in my spare time for nearly a year. It's now ready for others to use, so I'm excited to share the latest updates.
Support for Local LLM
Initially, Loz only supported the OpenAI API. A friend asked about integrating open-source LLMs like Llama2, which isn't straightforward to set up on a user's computer. I discovered a project called Ollama that simplifies the installation of various open-source LLMs with a single command. It supports GPU or CPU and is compatible with Linux, macOS, and Windows, making it an ideal solution for Loz. I refactored Loz's codebase, introduced an abstract class named LLMService, and added subclasses such as OpenAiAPI and OllamaAPI. Now users can effortlessly switch between LLMs in interactive mode using the config api command.
An advantage is the ability to compare outputs across different LLM engines. I've tested Llama2, CodeLlama, and GPT-3 for generating Git messages. Llama2 tends to be unstable with long token sizes, producing verbose and inconsistent outputs. Sometimes it even includes code diffs in the generated commit messages, indicating that Llama2 needs fine-tuning for this task.
Execute Linux Commands Using Natural Language
I've added functionality to execute Linux commands via natural language. Loz first determines whether the prompt is for generating a Linux command. If so, it returns the command(s) in JSON format, making it easy for Loz to access the command. If not, the prompt is forwarded to the OpenAI API and processed as a regular prompt.
The generated JSON varies but is generally understandable, containing command or commands objects and args object. Loz interprets these objects to construct executable Linux commands. However, there are challenges, such as when the tools referenced by the command are not installed on the local system. While enhancing the detail in prompts can improve the output, the results may still be unpredictable. Therefore, the continuous refinement of Git message and Linux command generation remains an ongoing effort.
Here are some examples of executing Linux commands with Loz:
Find the largest file in the current directory:
loz "find the largest file in the current directory"
-rw-rw-r-- 1 foo bar 9020257 Jan 31 19:49 ./node_modules/typescript/lib/typescript.jsCheck if Apache2 is running:
loz "check if apache2 is running on this system"
● apache2.service - The Apache HTTP ServerDetect GPUs in the system:
loz "Detect GPUs on this system"
00:02.0 VGA compatible controller: Intel Corporation Device a780 (rev 04)CLI for LLM
In the future, familiarity with command-line tools for Linux, Windows, or MacOS might become unnecessary, as we will interact directly with a built-in LLM within the operating system. Loz serves as a precursor to this envisioned future.