Train your own LLM? Here’s what actually happens | Exasol

Skip to content

Watch our demo

Contact us

Watch our demo

Home » Blogs » Train your own LLM? Here’s what actually happens

Large Language Models feel almost effortless to use today. You send a prompt, get an answer, and move on. But behind that simplicity lies a process that is anything but trivial-and often widely underestimated.

In conversations with colleagues and customers, one question kept coming up: “What does it actually take to build an LLM?” Not use one. Not fine-tune one. But build one end-to-end.

So I decided to find out.

Instead of relying on cloud-scale infrastructure, I took a more constrained (and realistic) approach: build a small LLM on a single notebook, understand every step of the process, and push it all the way to production-like usage. The goal wasn’t to compete with state-of-the-art models-it was to make the mechanics visible. And that’s where things get interesting.

Because training a model is only half the story. The real question is: how do you actually use it in a data-driven environment? In this case, directly where the data lives in the database. In this article, I’ll walk through the full journey:

What it takes to train a simple LLM (and where it fails),

Why fine-tuning beats starting from scratch in most cases,

And how to run inference directly inside the Exasol database , enabling parallel processing of large text datasets without moving data around.

This is not a production blueprint. It’s a hands-on exploration of how LLMs really work – from raw data to SQL query.

If you’ve ever wondered what’s behind the curtain of modern AI – and how it connects to real data systems – this is for you.

Engineering the Training Pipeline

Before training the LLM, there’s a more practical question to solve: how do you actually run the process? You need scripts – or ideally a full application-that orchestrates everything from data ingestion to deployment.

Here’s where things get interesting. Instead of writing the application from scratch, I took a different route: I asked an AI to build it for me. This approach is often called vibe coding. In simple terms, rather than focusing on syntax, you focus on intent and iterate from there.

Over multiple sessions, I guided an AI coding assistant to develop a complete training application. The goal was ambitious: cover the entire lifecycle of an LLM, including dataset handling, training, evaluation, and deployment. For the runtime environment, I chose the Apple ecosystem, specifically a machine powered by Apple Silicon chips. The integrated GPU provides a significant performance advantage for local model training, making it a practical choice for experimentation. The coding was done in Xcode, Apple’s development environment, which integrates perfectly with Anthropic’s Claude Code and OpenAI Codex. The entire development was handled in one place.

The resulting application serves as a control center for the entire LLM pipeline, enabling users to define a neural network architecture or choose a pre-trained model, load and preprocess training datasets, train the model with configurable parameters, evaluate its performance through checkpoints or intermediate states, deploy the trained model, and run inference directly within an Exasol environment.

Interestingly, building this application required relatively little time compared to the effort required to develop and train the models themselves, highlighting how AI-assisted development is reshaping modern software engineering practices.

The screenshot below captures the application during an active training phase, presenting real-time log output, interval-based metrics collected between model checkpoints, GPU utilization data, and visualizations that track training progress and convergence behavior.

Framing the Problem Space

Defining the use case means focusing on the system’s practical objective: building a large language model that can answer questions based on previously unseen text. In technical terms, this involves enabling generalization over unstructured data so the model can interpret and extract meaning without requiring task-specific retraining for each new input. From a business perspective, this translates into faster, more consistent access to insights hidden within large volumes of information.

A clear example is a portfolio manager operating in financial markets, whose daily workflow often involves reviewing hundreds or even thousands of documents, such as earnings reports, regulatory disclosures, press releases, and analyst briefings. While the underlying questions tend to remain straightforward, identifying the relevant answers across such a fragmented and high-volume information landscape is time-consuming and error-prone.

An LLM designed for question answering addresses this challenge by automatically processing unstructured text and surfacing the key insights, effectively reducing manual effort while...