This repository contains a basic demo of using linear probes for confidence estimation. It is based off of this paper from Meta Research. Confidence Calibration for LLM Predictions.
Everything you need is in /notebooks. You can copy this the notebooks to google colab or run them locally.
If running locally you need to install uv. Instructions are here.
You should also use a clean virtual enviironment for the kernel.
uv venv
uv pip install ipykernel