Low-Cost LoRA Fine-Tuning and Deployment-Efficiency Profiling of a 0.5B-Parameter Language Model for Closed-Vocabulary Structured Extraction on Consumer Apple Silicon | Zenodo
Skip to main
You are using an outdated browser. Please upgrade your browser to improve your experience.
Published July 25, 2026
| Version v1
Preprint
Open
Low-Cost LoRA Fine-Tuning and Deployment-Efficiency Profiling of a 0.5B-Parameter Language Model for Closed-Vocabulary Structured Extraction on Consumer Apple Silicon
Authors/Creators
Deshpande, Tanmay<br>(Researcher)1
Show affiliations
1.
Independent Researcher
Description
We present a fully reproducible case study in fine-tuning and deploying a sub-billion-parameter language model for closed-vocabulary structured extraction on commodity hardware. Using low-rank adaptation (LoRA), we fine-tune Qwen2.5-0.5B-Instruct to extract a strict four-field JSON object (sentiment, a bounded topic set, a price-mention flag, and an implied 1–5 rating) from unstructured mobile-application reviews, and we evaluate it with a fully deterministic parse-and-compare protocol that uses no model-based judge. Training is performed entirely on an Apple M4 Mac mini using the Metal Performance Shaders (MPS) backend in 32-bit precision; because 4-bit bitsandbytes quantization is CUDA-only, this configuration deviates from canonical QLoRA and the associated quantization-recovery control cannot be produced, a limitation we state explicitly rather than omit. On a held-out test set of 200 examples, fine-tuning improves the unweighted aggregate score from 62.3% to 76.6% (+14.2 points), with the largest gain on the closed-vocabulary topic field (26.1% to 73.5%). We then profile the quantized deployment (4-bit weights, served via Ollama) using a measurement methodology adapted from the low-bit-inference literature: peak-memory decomposition, key–value (KV) cache-quantization tolerance, the memory-bandwidth throughput ceiling, and energy per token. The central deployment finding is a qualitative collapse under aggressive KV-cache quantization: an 8-bit KV cache is near-lossless (77.95% to 77.02%), whereas a 4-bit KV cache reduces aggregate accuracy to 0.38% and valid-JSON parse rate to 10.5%. We discuss the label-circularity threat that limits the interpretation of three of the four fields, and we release all code, data splits, model artifacts, and raw measurements
Files
paper.pdf
Files<br>(340.1 kB)
Name<br>Size
Download all
paper.pdf
md5:fe19e4965e6fd2b72bf1c01c00f123c1
340.1 kB
Preview
Download
Additional details
Related works
Is supplemented by
Software:
https://github.com/deshpandetanmay/qlora-structured-extraction
(URL)
Model:
https://huggingface.co/tanmaydeshpande/qlora-app-review-extraction
(URL)
Dataset:
https://huggingface.co/datasets/tanmaydeshpande/app-review-extraction-splits
(Other)
Software:
https://ollama.com/tanmayavinashdeshpande/qlora-app-review-extraction
(URL)
Dates
Created
2026-07-24
Software
Repository URL
https://github.com/deshpandetanmay/qlora-structured-extraction
Programming language
Python
Development Status
Active
References
E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685, 2021.
T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer. QLoRA: Efficient Finetuning of Quantized LLMs. arXiv:2305.14314, 2023.
Qwen Team. Qwen2.5 Technical Report. arXiv:2412.15115, 2024.
G. Grano, A. Di Sorbo, F. Mercaldo, C. A. Visaggio, G. Canfora, and S. Panichella. Android Apps and User Feedback: A Dataset for Software Evolution and Quality Improvement. Proc. 2nd ACM SIGSOFT International Workshop on App Market Analytics (WAMA), 2017.
T. Dettmers, M. Lewis, Y. Belkada, and L. Zettlemoyer. LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale. arXiv:2208.07339, 2022.
S. Mangrulkar, S. Gugger, L. Debut, Y. Belkada, S. Paul, and B. Bossan. PEFT: State-of-the-art Parameter-Efficient Fine-Tuning methods. Hugging Face, 2022. https://github.com/huggingface/peft
PrismML. Bonsai 27B: Full 27B-class reasoning in binary and ternary transformer weights. Technical report, 2026. https://github.com/PrismML-Eng/Bonsai-demo
W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica. Efficient Memory Management for Large Language Model Serving with PagedAttention. Proc. SOSP, 2023.
G. Gerganov et al. llama.cpp: LLM inference in C/C++. 2023–2026. https://github.com/ggml-org/llama.cpp
19
Views
Downloads
Show more details
All versions<br>This version
Views
Total views
19
19
Downloads
Total downloads
Data volume
Total data volume
340.1 kB<br>340.1 kB
More info on how stats are collected....
Versions
External resources
Indexed in
OpenAIRE
Communities
Keywords and subjects
Keywords
parameter-efficient fine-tuning
KV-cache quantization
LoRA
Apple Silicon
small language models
Details
DOI
DOI...