Fin-R1 on AIBussin — AI applications, systems and books

Uncovering Reasoning in LLMs with Sparse Autoencoders

Thu, 27 Mar 2025 11:45:33 +0000

Summary

Large Language Models (LLMs) like DeepSeek-R1 show remarkable reasoning abilities, but how these abilities are internally represented has remained a mystery. This paper explores the mechanistic interpretability of reasoning in LLMs using Sparse Autoencoders (SAEs) — a tool that decomposes LLM activations into human-interpretable features. In this post, we’ll:

• Explain the SAE architecture used • Compute and visualize ReasonScore • Explore feature steering with sample completions • Provide live visualizations using Python + Streamlit

Optimizing Prompt Generation with MARS and DSPy

Mon, 24 Mar 2025 15:45:33 +0000

🕒 TL;DR

We explore MARS, a multi-agent prompt optimizer using Socratic dialogue.
We implement it using DSPy + Fin-R1 + EDGAR giving us an end-to-end financial reasoning pipeline.
We deploy the whole thing to Hugging Face Spaces with a Gradio UI.

🌟 Introduction

Prompt engineering has become the defining skill of the Large Language Model (LLM) era a delicate balance between science and art. Crafting the perfect prompt often feels like an exercise in intuition, trial, and error. But what if we could take the guesswork out of the process? What if prompts could optimize themselves?

Fin-R1: a Financial Reasoning LLM with Reinforcement Learning and CoT

Fri, 21 Mar 2025 03:27:30 +0000

Introduction

Fin-R1 is a new model specifically fine-tuned for financial reasoning, with performance that beats much larger models like DeepSeek-R1.

This post will use this model and compare it with phi3 across various tasks.

phi3 for comparison

Phi-3: a lightweight, general-purpose model known for its efficiency and strong reasoning performance at smaller parameter scales. It serves as a great baseline for assessing how domain-specific tuning in Fin-R1 improves financial understanding and response structure.