http://dylanler.github.io/tags/metacognition/Recent content in Metacognition on Dylan LerHugo -- 0.133.0en-usThu, 27 Aug 2026 10:00:00 -0700http://dylanler.github.io/posts/the-edge-of-knowing/Sat, 18 Jul 2026 09:31:00 -0700http://dylanler.github.io/posts/the-edge-of-knowing/The dangerous answer is not always the wrong one. It is the wrong one delivered with enough confidence to stop the search. This month I revisited two experiments in the repository. One measures whether models admit uncertainty across factual, reasoning, ambiguous, boundary, and impossible questions. The other samples the same model repeatedly to measure agreement and entropy. Together they test a practical claim: Uncertainty becomes useful when we measure both confidence within one answer and disagreement across several answers.http://dylanler.github.io/posts/metacognition-when-llms-know-they-dont-know/Sun, 14 Dec 2025 10:00:00 -0800http://dylanler.github.io/posts/metacognition-when-llms-know-they-dont-know/“I don’t know” might be the most important thing an AI can learn to say. This experiment tests whether LLMs have calibrated uncertainty—knowing when they’re likely to be wrong and expressing appropriate confidence levels. The results reveal systematic patterns of overconfidence and appropriate humility. The Experiment We presented 250 questions across 5 categories: Factual recall: Known facts with clear answers Reasoning puzzles: Logic problems with determinable solutions Ambiguous questions: Multiple valid interpretations Knowledge boundaries: Questions near training cutoff Impossible questions: No correct answer exists For each question, models provided: