Abdullah Shafique, AI Engineer — Building an LLM Eval Framework That Catches Bad Answers
By Abdullah Shafique, Agentic AI Engineer, based in Sahiwal District, Punjab, Pakistan. I build autonomous agents, RAG systems, and LLM applications. An LLM that sounds confident is not the same as an LLM that is right. That is why I built an LLM Eval Framework — a small, practical way to test what my agents and RAG systems actually produce, before I trust them in a real task. What is an LLM eval framework? An eval framework is a set of test cases with expected behaviour, run against an LLM system, with the results scored and compared over time. Instead of reading a few outputs and guessing, you run the same tests after every change and see whether the system got better, worse, or stayed the same. My framework follows that idea: define the test, run the system, score the output, keep the record. Why I built it I was already building a RAG Document Intelligence System and an Autonomous Research Agent. Both of them generate text from retrieved documents and web sources. Reading a h...