Your AI agent works great… until it doesn't. AgentEval catches the failures before your users do. Your AI agent works great… until it doesn't. AgentEval...

Groups Conversations All groups and messages Sign in Machine Learning, Deep Learning and AI Conversations About Privacy • Terms     Stop Hallucinating, Start Evaluating — A Tour of AgentEval 0 views Eyal Estrin unread, 6:40 AM (22 minutes ago)    to https://techcommunity.microsoft.com/blog/educatordeveloperblog/stop-hallucinating-start-evaluating-%E2%80%94-a-tour-of-agenteval/4519068 Eyal Estrin Author | Cloud Architect | AWS • Azure • GCP Insights Social: @eyalestrin Connect: https://linktr.ee/eyalestrin Blog: https://security-24-7.com Reply all Reply to author Forward

LangChain's Eval Engineering Skill inspects your agent's repo and traces, proposes evals through user interviews, and outputs runnable Harbor tasks.

I did not get to this opinion from a whitepaper. I got to it because I am building an open-source project around the problem, and the build keeps arguing back. I am working on AgentEval Forge right now, an OSS evaluation lab for agents. The original idea sounded straightforward enough: scenario packs, adversarial cases, trajectory scoring, regression tracking, cost and latency analysis. In my head, it was going to be a more serious, more structured version of evaluation work I had already done elsewhere. That was the first mistake. I had already spent time building model-eval and workload-eval style systems. I knew what it meant to compare models for real tasks, how to design rubrics, how to think about speed versus cost, and how quickly a neat score can become misleading if the harness is weak. I assumed agent evaluation would be an extension of that same world. The deeper I got into AgentEval Forge, the less true that felt. Model evaluation asks whether the answer is good. Agent evaluation has to ask whether the system behaved well enough to trust. Those are not the same question. ## The comparison that made it click A clean model-eval setup is something I know how to reason about. You have an input, an output, and some way of scoring quality. Sometimes that is exact match. Sometimes it is a rubric. Sometimes it is an LLM judge. Sometimes it is a benchmark harness. However fancy the setup gets, the center of gravity is still pretty stable: did the model produce a good answer for this task? That is already hard enough. I have seen enough brittle scoring and false confidence to know that even model evaluation can go wrong in ways that look scientific. A rubric can reward the wrong thing. A binary check can miss something obviously useful. A benchmark can feel objective while still pulling you toward the wrong optimization target. So I am not pretending model eval is solved. But with agents, the shape of the problem changes. The thing being evaluated is no longer just an answer. It is a workflow. It is a sequence of decisions. It is tool choice, retries, intermediate state, cost, recovery behavior, and sometimes policy adherence. The moment you let a system do more than respond once, the final output stops being the whole story. That sounds obvious when you say it plainly. In practice, I think a lot of teams still use answer-scoring habits on systems that have already moved past that category. ## What changed for me while building AgentEval Forge The real shift happened when I tried to define what AgentEval Forge would actually have to score. At first, I thought the answer would be simple: run scenario packs, score the outputs, compare versions, done. But the more I looked at real agent behavior, the more that started to feel naive. An agent can produce a decent final answer while doing a lot of things badly on the way there. It can choose the wrong tool first and recover by luck. It can loop more than it should. It can spend five times the tokens a better path would spend. It can take a risky step that a human reviewer would never approve, then still back into something that looks acceptable at the end. If I only score the final answer, I might mark that run as a success. That is the trap. The path matters. That is the sentence I keep coming back to while building this. The path matters just as much as the endpoint, and sometimes more, because the path is where cost, safety, and trust actually live. That challenge shows up in very practical ways. I am not just trying to score whether an agent got the answer right. I am thinking about cases where it picked the wrong tool first, passed the task only because it recovered late, crossed a boundary it should not have crossed, spent too many steps on something simple, or regressed only inside one narrow scenario family while the average score still looked fine. That is the kind of mess that makes agent evaluation feel much more like systems work than answer grading. ## I had already seen hints of this in earlier work This was not the first time I had felt the edges of the problem. In earlier harness and field-study work, especially when running systems against real workloads instead of toy examples, I kept finding failures that would never have shown up in a neat prompt-eval setup. That experience made me much less impressed by demo-path confidence. A system can look strong in a controlled evaluation and then become erratic as soon as the environment gets irregular. Real repos. Real logs. Real commit histories. Real naming mess. Real ambiguity. That is where edge cases show up. That is where tool misuse shows up. That is where “technically correct” and “safe to ship” start to drift apart. I think that history is part of why AgentEval Forge took the shape it did in my head. I was not trying to build a leaderboard generator. I was trying to build something that feels closer to release discipline for agents. Not: did this run look good? More like: if I change this system, what got better, what got worse, what got more expensive, and what became riskier even if the top-line score improved? That feels much more like engineering than benchmarking. ## The five things I now think agent evaluation has to cover The more I think about it, the more I come back to five dimensions. First, you still need final correctness. If the agent does not solve the task, the rest only matters so much. Second, you need trajectory quality. How did it get there? Was the sequence of decisions sensible, efficient, and stable? Or did it succeed in a way you would never want repeated in production? Third, you need to evaluate tool behavior. Did it choose the right tools? Did it overuse them? Did it take expensive or risky actions unnecessarily? Did it rely on accidental recovery? Fourth, you need safety and policy adherence. A successful output reached through an unsafe path should not count as a clean win. Fifth, you need cost, latency, and regressions. Did the new version get slower? More expensive? More erratic? Did it improve accuracy while making operational behavior worse? Did it just change style and still get celebrated as progress? That last one matters a lot to me because I think it is where teams fool themselves most easily. They see difference and call it improvement. Those are not the same thing. ## Why I do not think the current eval ecosystem is enough I am not arguing that the current eval ecosystem is useless. Far from it. The LangChain piece on evaluation-driven development gets a core thing right: once you see failures in production, those failures need to feed offline evals, and every change should be tested against them. That loop is healthy. It is exactly the kind of discipline I want more teams to adopt. And from a different angle, Birgitta Böckeler's writing on agentic coding captures something else I think matters: the most dangerous failures often live in longer feedback loops. Maintainability. Team friction. Brute-force fixes. Misdiagnosis. Overbuilt solutions. Those costs do not always show up in the immediate output, but they absolutely show up later. That is why I do not think the issue is that existing tools are bad. I think the issue is that many of our evaluation habits were formed in a prompt-and-answer world. Agent systems are messier. They behave more like orchestrated software than isolated text generation. Once you accept that, your evaluation system has to grow up too. ## Even the eval can lie to you This is the part I do not want to over-neaten. Your evaluation setup can be wrong in ways that feel rigorous. I have already seen enough of that in model-eval and harness work to be wary of any system that sounds more certain than it really is. You can undercount useful behavior. You can make your binary checks too brittle. You can optimize for what is easy to score rather than what actually matters. So now the challenge has two layers. The agent is hard to evaluate, and the evaluation system itself becomes another system you have to design carefully, calibrate honestly, and distrust a little. That is not a reason to avoid serious evaluation. If anything, it is the opposite. It means evaluation has to be treated like product and engineering work, not just like reporting. ## Why I am still optimistic about building this For all of that, I do not feel cynical about the problem. If anything, building AgentEval Forge has made me more convinced that this is worth doing well. I want scenario packs. I want adversarial cases. I want trajectory scoring. I want regression tracking that can tell me which change made the system more useful and which one only made it look cleaner in a demo. I want an evaluation system that helps answer a release question, not just a research question. Did this version get better? Better how? At what cost? And what got riskier even if the headline score went up? That is the kind of question I want tooling to help with. ## The bigger thing I think teams are about to learn I suspect a lot of teams are still doing with agent evaluation what many teams did with AI-assisted coding more broadly: borrowing habits from an earlier problem and hoping they scale. Sometimes they do. Often they do not. And then the bill shows up later, in regressions, strange production behavior, expensive paths, unsafe tool use, or a team that keeps changing the system without ever being able to say whether it actually improved. That is why this topic feels worth writing about while I am still in the middle of building the OSS for it. The build itself is sharpening the argument for me. Every design decision keeps pushing me toward the same conclusion: model evaluation asks whether the answer is good. Agent evaluation has to ask whether the system behaved well enough to trust. That is a harder problem. I think it is also going to become one of the defining engineering problems of serious agent work. I will write more about this once the OSS is public. Some of these challenges are already turning into concrete design decisions in AgentEval Forge, and in the coming weeks I want to share which parts we solved cleanly, which parts stayed messy, and how the tradeoffs changed once the code had to work as a real system instead of an idea. I will share the GitHub repo publicly when that is ready. ## Open for discussion This is still an active build and an active line of thinking for me, so I would genuinely like pushback on it. * Where did answer scoring stop being enough for your agent work? * Are you evaluating the path yet, or still mostly the final output? * What has been hardest to score well: tool use, regressions, safety, cost, or something else? * And the big one: are we building evaluation systems that improve release confidence, or just better ways to feel rigorous?

OpenJiuwen Studio backend: FastAPI server, SQLAlchemy ORM, plugin manager, evaluation CLI (agenteval)

Scorecard AI provides essential evaluation and deployment features for your LLM product. We automate testing to help you reduce costs and ship faster.

Scorecard AI provides essential evaluation and deployment features for your LLM product. We automate testing to help you reduce costs and ship faster.

AgentEval is a new initiative to provide the legal market with an open-source collection of genAI benchmarks that can be freely used, and also is building a community to share data, ideas and protocols for evaluating legal AI tools. Heading the project is **Darius Emrani** , who is the CEO of Scorecard, a startup focused on supporting LLM-based product development. He told **Artificial Lawyer** : ‘The idea is to provide a lot of benchmarks and best practices. We want to get people involved and become the trusted source for AI benchmarks.’ This site asked why he’s chosen to focus on legal tech. Emrani said that the idea is to help ‘essential services’, and that includes health and finance, with AI needs. Central to that is assessing accuracy – or developing best practices around accuracy – and so that’s where AgentEval comes in. He added that the agentic aspect of AI was also a key component in terms of sectors understanding accuracy. Emrani also stressed that the goal is to remain open-source and community-driven, and (see community access link) he would be keen to engage with LITIG and other projects around the world focused on legal AI benchmarking. The organisation stated that: ‘Because some benchmarking efforts often rely on proprietary datasets, closed methodologies, and restricted access, it can be difficult for researchers and developers to reproduce results, compare models fairly, and refine systems. ‘At the same time, we’ve seen successful open evaluation frameworks — from NIST and ISO standards to initiatives like MLCommons, LLMSYS Chatbot Arena and LegalBench — showing that collaborative, open-source approaches lead to better benchmarking practices.’ They went on to say that this helps: * **‘Law firms** — Gain a clear, standardized way to compare legal AI solutions and select the best tools for their needs. * **Legal AI vendors** — Understand their performance relative to competitors and improve their models based on objective, industry-standard benchmarks. * **Academics & Policymakers** — Access insights into how AI systems perform in real-world legal applications, ensuring responsible deployment and regulation. * **The broader AI industry** — By making benchmarks and best practices freely available, Agent Eval gives smaller startups, research institutions, and independent developers access to the same high-quality evaluation resources as well-funded private companies.’ And here’s a summary of their goals: **Why we builtAgentEval.org** **Mission** To establish a trusted, open source for sharing AI benchmarks and best practices that drive transparency and continuous improvement in AI evaluation. **Vision** A future where open collaboration and shared data standards accelerate responsible AI innovation, making evaluation methodologies accessible and verifiable for everyone. **Why Open Benchmarks?** **Transparency & Trust** Open-sourcing our benchmarks and methodologies allows anyone to inspect, validate, and contribute to our evaluation processes. **Community-driven innovation** An open platform invites contributions from a broad community, leading to more robust and diverse evaluation practices. **Industry adoption** Open-source tools and standards are more likely to be adopted by academic institutions, industry players, and public agencies. **Non-profit and collaborative alignment** Emphasizing open source aligns perfectly with our mission to move the industry forward through shared knowledge and collective effort. You can find more info here. — **Why This Matters – The AL View** The need for some clarity on genAI accuracy came to the attention of everyone last summer after the ‘Stanford Debacle’, when a group of researchers claimed to have exposed major failings in well-known genAI legal research tools. Since then things have evolved. Artificial Lawyer, and others, raised the idea of setting up some sort of shared approach to genAI accuracy. AL suggested this could range from a ‘Kite mark’ to a set of protocols to help buyers and sellers approach this issue. LITIG and others have got things in motion, and private companies are getting involved as well. But, the challenge remains: what are we trying to achieve here? What does ‘there’ look like when it comes to gauging AI accuracy? Is it just a single benchmark test, or several tests, owned by one entity? Is it a basket of different benchmarks? Is it open-source and free for all to use? Moreover, as AL has suggested, do we need to **think more in terms of a ‘compass’ approach**? I.e. that because foundation models are moving so fast and that any company scoring X on any specific benchmark made by Y entity will immediately try to refine their outputs to improve them – as happened after the Stanford study – then is there any lasting value relying on just a single test approach to benchmarking? Moreover, one of the most insightful things that came out of the Stanford fallout was the point made by **Jeff Pfeifer** at LexisNexis about the need to focus on ‘answer usefulness’. I.e. to some extent the value of any AI output is in the ‘eye of the beholder’. So, perhaps instead what we need is a general direction to head towards, and an idea of what ‘good’ looks like **based on a basket of different benchmarks**. I.e. for X type of task, we should expect Y level of results, with those outputs put in the context of the task, e.g. case law research is different to summarising testimony, and both are very different to red-lining a basic NDA. But the overall approach is holistic, open-source, and community-driven. Does that mean that we don’t need benchmarks? No. Not at all. Quite the opposite. Rather the point is that legal work is subjective because it’s all based on language, reasoning and interpretation. Measuring AI accuracy is not like measuring the speed of a car going down a road. So, we need multiple pathways and viewpoints to assess accuracy and therefore a more principle-based approach that accommodates this is more flexible and can evolve as the sector evolves as well. To conclude, AL welcomes this open-source, community-based approach, that seeks to build a general consensus on what ‘good’ looks like, along with protocols for how to approach genAI tools. Within that ecosystem of evaluation are benchmarks – but they are a multi-prong approach to assessment – and it’s accepted, in fact expected, that the results for each company will evolve as rapidly as the state of the art is. Hence, we return to the compass approach. Any road, that’s AL’s view. There is much more to come in terms of genAI accuracy benchmarking. Watch this space! **Richard Tromans, Founder, Artificial Lawyer** — ### Share this: * * Tweet * Email * *

Google introduces a coding agent skill that automates the agent quality improvement loop using model-based AutoRaters developed with DeepMind. The skill orchestrates a five-stage evaluation flywheel (prepare data, run inference, grade, analyze failures, optimize) where the optimizer and evaluator st

Your AI agent works great… until it doesn't. AgentEval catches the failures before your users do. Your AI agent works great… until it doesn't. AgentEval...

AgentEval is a new initiative to provide the legal market with an open-source collection of genAI benchmarks that can be freely used, and also is building a community to share data, ideas and proto…