Fine-Tuning

Fine-tuning continues the training of a pre-trained model using additional task-specific examples. This changes internal model parameters.

The result is an adapted model version intended to reproduce a required behaviour, task or output format more reliably.

When fine-tuning is useful

Fine-tuning can be useful when a model needs to show consistent behaviour across many requests.

Examples include a stable output format for downstream systems, specialised classification or language conventions that can be demonstrated through suitable training examples.

Fine-tuning is generally unsuitable for current company knowledge. Changes in facts require further training, and individual statements are difficult to trace to a specific source.

A knowledge base connected at runtime is often easier to update and audit for this purpose.

Fine-tuning versus RAG

RAG and fine-tuning address different needs. RAG supplies the model at runtime with relevant information from an updatable knowledge base.

Fine-tuning changes the model and targets behaviour that should remain consistent across many requests.

The two approaches can be combined. An adapted model may control output format and behaviour, while RAG provides current or internal information as traceable context.

Cost and risk

The amount and type of training data required depend heavily on the task, model and method. Clean preparation, sufficient coverage and separate evaluation are essential.

Incorrect or unbalanced examples can reinforce unwanted behaviour that becomes visible only through systematic testing.

A fine-tuning result is tied to a specific base-model version. Moving to a new model requires the data, training process and evaluation to be reviewed and, where necessary, repeated.

This maintenance effort should be included in the technical and commercial plan from the outset.