🚀Inference

run model locally

This section provides detailed information about the nexaai commands used to run various models locally. These commands support text generation, image generation, and vision-language models, automatic speech recognition model, text-to-speech model etc.

$ nexaai -h 
usage: nexaai [-h] {onnx,gen-text,gen-image,vlm} ...

Nexa CLI tool for handling various model operations.

positional arguments:
  {onnx,gen-text,gen-image,vlm,pull,remove,list,login,whoami,logout}
                        sub-command help
    onnx                Run ONNX models for inference.
    gen-text            Run a GGUF model locally for text generation.
    gen-image           Run a GGUF model locally for image generation.
    vlm                 Run a GGUF model locally for VLM inference.
    asr                 Run a GGUF model locally for voice inference.

options:
  -h, --help            show this help message and exit

By default nexaai will run gguf models.

To run onnx models, simply add onnx afternexaai:

$ nexaai onnx -h 
usage: nexaai onnx [-h] {gen-text,gen-image,asr,tts} ...

positional arguments:
  {gen-text,gen-image,asr,tts}
                        ONNX sub-command help
    gen-text            Run ONNX model for text generation.
    gen-image           Run ONNX model for image generation.
    asr                 Run ONNX model for auto-speech-recognition.
    tts                 Run ONNX model for text-to-speech generation.

options:
  -h, --help            show this help message and exit

Please check the ONNX page for more details.