Quantized LLMs for Clinical Report and Question Answering in Retinal Disease Assessment

Large language models have demonstrated strong capabilities across a wide range of text-generation, typically relying on architectures with billions of parameters. Although this generality enables high performance, using such large models for narrowly defined clinical tasks becomes computationally inefficient. This work presents a compact LLaMA model fine-tuned on diabetic retinopathy data for generating retinal clinical reports and supporting a question-answering workflow. To enable deployment in CPU-restricted environments, the model was evaluated in its original form and in a Q4-quantized GGUF version. Experimental results show that the quantized model preserved high semantic similarity with the non-quantized model (BERTScore F1=0.9033) while nearly doubling inference speed (1.44 tokens/s vs. 0.72 tokens/s). These findings indicate that compact and quantized LLMs can produce coherent clinical reports and QA outputs without the computational burden of larger architectures, making them suitable for small clinics and regional centers with limited hardware resources.