LEMONADE(1)

NAME
lemonade - Local LLM serving with GPU and NPU acceleration

DESCRIPTION
lemonade [-h] [-i INPUT] [-d CACHE_DIR] [-m [TRACK_INTERVAL]]

    [--power-hwinfo [WARMUP_PERIOD]] [--power-agt [WARMUP_PERIOD]] [--tools [tool --tool-args [tool --tool-args...]]]

Tools for evaluating and deploying LLMs.

Read this to learn the command syntax: https://github.com/lemonade-sdk/lemonade/blob/main/docs/README.md

  positional arguments:

--tools [tool --tool-args [tool --tool-args...]]
    Run `lemonade TOOL -h` to learn more about each tool.

Tools that can start a sequence:
    * huggingface-load: Load an LLM in PyTorch using huggingface transformers * llamacpp-load: Wrap llama.cpp models with an API * oga-load: Load model in onnxruntime-genai (OGA)

Tools that go into a sequence:
    * llamacpp-bench: Benchmark an LLM in llama.cpp * accuracy-mmlu: Measure accuracy with Massive Multitask Language Understanding (MMLU) * accuracy-humaneval: Measure coding accuracy with HumanEval * accuracy-perplexity: Measure perplexity score * lm-eval-harness: Evaluate model using lm-evalharness * llm-prompt: Prompt an LLM and print the result * huggingface-bench: Benchmark a huggingface-style PyTorch LLM * oga-bench: Benchmark an LLM in onnxruntime-genai (OGA)

Management tools:
    * report: Export statistics from each lemonade run to a CSV file * cache: Manage the build cache * version: Print the lemonade version number * system-info: Print system and device information

  options:

-h, --help
    show this help message and exit

-i, --input INPUT
    The input that will be evaluated by the starting tool (e.g., huggingface checkpoint)

-d, --cache-dir CACHE_DIR
    Cache directory where tool results are stored (default: /home/supermario/.cache/lemonade)

-m, --memory [TRACK_INTERVAL]
    Track memory usage and plot the results. Optionally, set the tracking interval in seconds (default: 0.25)

--power-hwinfo [WARMUP_PERIOD]
    Track power consumption using the HWiNFO application and plot the results. HWiNFO is a commercial product from a third party (https://www.hwinfo.com/) and should be acquired/licensed appropriately. Optionally, set the warmup period in seconds (default: 5). If the application is not installed at C:\Program Files\HWiNFO64\HWiNFO64.exe, set the HWINFO_PATH environment variable to point at it. This is a Windows only feature and Lemonade must be run from a CMD window with Administrator privileges.

--power-agt [WARMUP_PERIOD]
    Track power consumption using the AGT application and plot the results. AGT is an internal AMD tool. Optionally, set the warmup period in seconds (default: 5). Set the AGT_PATH environment variable to point to the AGT executable. This is a Windows only feature and Lemonade must be run from a CMD window with Administrator privileges.

    [--power-hwinfo [WARMUP_PERIOD]] [--power-agt [WARMUP_PERIOD]] [--tools [tool --tool-args [tool --tool-args...]]]

SEE ALSO
The full documentation for lemonade is maintained as a Texinfo manual. If the info and lemonade programs are properly installed at your site, the command

    info lemonade

should give you access to the complete manual.