Towards Expert-Level Financial Narration With Large Language Models
AS6 Level 5
MR6, AS6-05-10
Abstract:
Financial narration turns market data into the analytical narratives that guide investment decisions. That work is still done by hand. It remains expensive, confined to well-resourced institutions, and too slow to keep pace with the markets it describes. Large Language Models (LLMs) are a natural candidate for automating it. Current LLMs possess deep financial knowledge, enough to pass professional financial examinations, yet on real market data they still produce narratives that are fluent but shallow, generic, and often factually wrong. The shortfall is therefore one of execution, not knowledge.
This thesis characterizes the shortfall by the level at which generation operates. Generation at the structural level fixes what a report should cover, then fills each section with whatever reasoning the model defaults to. Generation at the analytical level does what an expert does: it selects the analytical operations the data warrants and executes them systematically against the data. Moving LLM generation to that level requires closing three gaps in turn, and we contribute a system for each.
The diagnostic gap comes first: existing benchmarks score fluency and descriptive accuracy, not analytical depth or specificity, so the shortfall stays hidden beneath fluent prose. We build DataTales to supply the missing measurement, pairing 4.9k professional financial reports with comprehensive market data. DataTales quantifies the shortfall: no model tested exceeds 30% factual accuracy, and all retreat to simple operations rather than the data-specific ones expert analysis requires. A coordination gap follows: end-to-end generation asks a single pass to retrieve data, compute values, reason, and write all at once, so synthesis stays shallow. Our framework KAHAN separates those steps into hierarchical analytical stages grounded in programmatic computation, raising quality by 20–25% at 98% factuality. Even after closing the coordination gap, an adaptation gap remains: a language model samples the most probable analysis, not the one the data warrants, so the reasoning stays generic. AnalysisBank conditions that choice on the data. It distills expert reports into a library of Analyses — each pairing a data signal with the analytical move it calls for; and at inference, retrieves the moves matched to the signals the input contains. Novel, data-grounded insight rises 1.7–3.7×, while structural-level methods converge to a ceiling.
Taken together, the three contributions show that the analytical level is reached by how the analysis is executed, not by how much the model knows. Both frameworks hold outside finance: KAHAN improves narrative quality by 25–31% at 95.7% factuality on Parkinson’s Disease gait analysis, and on SciGen, a scientific writing benchmark, AnalysisBank leads on analytical rather than descriptive content (65.7%) and matches or beats the expert reference in 65.1% of judged comparisons. The thesis therefore contributes a transferable strategy for expert, data-grounded generation: structure the analysis into levels rather than produce it in a single pass, and draw the analytical moves themselves from expert practice rather than from model defaults.