20/ One useful operational detail in this Chess LLM project is making ingestion and scheduled refreshes explicit rather than automatic on app startup. This protects a limited embedding quota while still allowing cached knowledge and live lookups to support the chat experience. #llmzoomcamp
September 10, 2026 at 4:29 PM
20/ One useful operational detail in this Chess LLM project is making ingestion and scheduled refreshes explicit rather than automatic on app startup. This protects a limited embedding quota while still allowing cached knowledge and live lookups to support the chat experience. #llmzoomcamp
19/ For my third peer evaluation, I reviewed a Chess LLM RAG application. A strong design choice is separating curated chess knowledge in Qdrant from live Chess.com data, so player ratings, streamers, and leaderboards can remain current at query time. #llmzoomcamp
Chess.com - Play Chess Online - Free Games
Play chess online for free on Chess.com with over 250 million members from around the world. Have fun playing with friends or challenging the computer!
Chess.com
September 10, 2026 at 4:19 PM
19/ For my third peer evaluation, I reviewed a Chess LLM RAG application. A strong design choice is separating curated chess knowledge in Qdrant from live Chess.com data, so player ratings, streamers, and leaderboards can remain current at query time. #llmzoomcamp
18/ Reproducible containerisation requires more than a `docker-compose.yml` file. A fresh clone should build without depending on untracked model directories or generated database files; required artifacts should be packaged or created automatically. #llmzoomcamp
September 10, 2026 at 3:45 PM
18/ Reproducible containerisation requires more than a `docker-compose.yml` file. A fresh clone should build without depending on untracked model directories or generated database files; required artifacts should be packaged or created automatically. #llmzoomcamp
17/ For my second peer evaluation, I reviewed a Coffee Assistant RAG application. The use of ONNX for local embeddings is a considered architectural choice that avoids the full PyTorch stack and helps keep Docker images manageable. #llmzoomcamp
September 10, 2026 at 3:35 PM
17/ For my second peer evaluation, I reviewed a Coffee Assistant RAG application. The use of ONNX for local embeddings is a considered architectural choice that avoids the full PyTorch stack and helps keep Docker images manageable. #llmzoomcamp
16/ A solid architectural pattern in this Government Scheme RAG: implementing document-level deduplication before passing chunks to the LLM. It's a highly effective way to guarantee evidence diversity and cut down on redundant context tokens. #llmzoomcamp
September 10, 2026 at 2:39 PM
16/ A solid architectural pattern in this Government Scheme RAG: implementing document-level deduplication before passing chunks to the LLM. It's a highly effective way to guarantee evidence diversity and cut down on redundant context tokens. #llmzoomcamp
15/ Round 3 peer review: a Government Scheme RAG. Deploying containerized Streamlit apps to Google Cloud Run is a practical setup for MVPs. It keeps infrastructure simple, handles scale-to-zero, and makes testing live links straightforward. #llmzoomcamp
September 10, 2026 at 1:45 PM
15/ Round 3 peer review: a Government Scheme RAG. Deploying containerized Streamlit apps to Google Cloud Run is a practical setup for MVPs. It keeps infrastructure simple, handles scale-to-zero, and makes testing live links straightforward. #llmzoomcamp
14/ Containerization with Docker, optional monitoring dashboard, and clear run instructions so reviewers can reproduce the full RAG stack locally. The aim is a clean, end-to-end project that’s easy to run and evaluate. #llmzoomcamp
September 5, 2026 at 5:34 PM
14/ Containerization with Docker, optional monitoring dashboard, and clear run instructions so reviewers can reproduce the full RAG stack locally. The aim is a clean, end-to-end project that’s easy to run and evaluate. #llmzoomcamp
13/ DER RegCheck now has a full reviewer workflow: About, Ask, Evidence, Review, and Monitoring tabs. Answers are cached by question + config, feedback and manual scores are stored in PostgreSQL, and a 10-question Tier 2 set supports realistic RAG-quality review. #llmzoomcamp
September 5, 2026 at 1:47 PM
13/ DER RegCheck now has a full reviewer workflow: About, Ask, Evidence, Review, and Monitoring tabs. Answers are cached by question + config, feedback and manual scores are stored in PostgreSQL, and a 10-question Tier 2 set supports realistic RAG-quality review. #llmzoomcamp
12/ Added answer-generation evaluation: 24 fixed questions, 3 prompts, deterministic citation validation, and an LLM judge (groundedness, relevance, completeness, citation quality, uncertainty). Only v3_few_shot_grounded_rag hit 100% citation validity, so it’s the runtime prompt. #llmzoomcamp
September 5, 2026 at 12:22 PM
12/ Added answer-generation evaluation: 24 fixed questions, 3 prompts, deterministic citation validation, and an LLM judge (groundedness, relevance, completeness, citation quality, uncertainty). Only v3_few_shot_grounded_rag hit 100% citation validity, so it’s the runtime prompt. #llmzoomcamp
11/ Built deterministic citation validation for answers: citations must refer only to supplied evidence labels, material claims must be cited, and unknown labels fail closed. This prevents hallucinated sources and keeps the tool in “research support, not compliance advice” territory. #llmzoomcamp
September 5, 2026 at 9:22 AM
11/ Built deterministic citation validation for answers: citations must refer only to supplied evidence labels, material claims must be cited, and unknown labels fail closed. This prevents hallucinated sources and keeps the tool in “research support, not compliance advice” territory. #llmzoomcamp
10/ Evaluated query-rewrite variants (original, expanded, HyDE, HyDE+expanded) over via PostgreSQL. Expansion gave a tiny composite gain (~0.003) but added ~36s latency per query, so runtime uses the original query with vector retrieval + reranking. Engineering > micro-optimisations. #llmzoomcamp
September 4, 2026 at 10:54 PM
10/ Evaluated query-rewrite variants (original, expanded, HyDE, HyDE+expanded) over via PostgreSQL. Expansion gave a tiny composite gain (~0.003) but added ~36s latency per query, so runtime uses the original query with vector retrieval + reranking. Engineering > micro-optimisations. #llmzoomcamp
9/ Moved DER RegCheck retrieval evaluation onto the deployed PostgreSQL + pgvector path. 7,500 evaluation records across lexical, vector, hybrid, and reranked variants. Historical v1/v2 results are kept as baselines; v3 is now the authoritative basis for production retrieval selection. #llmzoomcamp
September 4, 2026 at 10:22 PM
9/ Moved DER RegCheck retrieval evaluation onto the deployed PostgreSQL + pgvector path. 7,500 evaluation records across lexical, vector, hybrid, and reranked variants. Historical v1/v2 results are kept as baselines; v3 is now the authoritative basis for production retrieval selection. #llmzoomcamp
8/ For DER RegCheck, retrieval settings are determined by an evaluation pipeline testing multiple alphas and weightings. The benchmarks demonstrate that tuning hybrid search and applying reranking improves top-k quality over the vector baseline. #llmzoomcamp
September 1, 2026 at 11:33 AM
8/ For DER RegCheck, retrieval settings are determined by an evaluation pipeline testing multiple alphas and weightings. The benchmarks demonstrate that tuning hybrid search and applying reranking improves top-k quality over the vector baseline. #llmzoomcamp
7/ For DER RegCheck, I wanted the ingestion to be a fully repeatable flow: manifest → load docs → structural chunking → embedding → PostgreSQL + pgvector. Because all scripts and configs are versioned, the entire vector database can be rebuilt from scratch reproducibly. #llmzoomcamp
August 26, 2026 at 3:10 AM
7/ For DER RegCheck, I wanted the ingestion to be a fully repeatable flow: manifest → load docs → structural chunking → embedding → PostgreSQL + pgvector. Because all scripts and configs are versioned, the entire vector database can be rebuilt from scratch reproducibly. #llmzoomcamp
6/ The real advantage of structural chunking is better retrieval. Chunks carry metadata like source file and doc title, which lets the system go beyond basic semantic similarity. It also enables strict filtering by document or policy type, improving retrieval for complex regulations. #llmzoomcamp
August 26, 2026 at 1:26 AM
6/ The real advantage of structural chunking is better retrieval. Chunks carry metadata like source file and doc title, which lets the system go beyond basic semantic similarity. It also enables strict filtering by document or policy type, improving retrieval for complex regulations. #llmzoomcamp
5/ Chunking is more than window size; it is an architectural choice. I use structural chunking, not fixed windows. Headings, lists and tables guide each chunk so evidence and citations stay together and technical requirements remain in their original context, like the tariff section. #llmzoomcamp
August 25, 2026 at 1:06 PM
5/ Chunking is more than window size; it is an architectural choice. I use structural chunking, not fixed windows. Headings, lists and tables guide each chunk so evidence and citations stay together and technical requirements remain in their original context, like the tariff section. #llmzoomcamp
4/ Extraction matters: I’m reviewing the source files for broken headings, repeated headers, table structures, and reading-order issues. Bad source text creates retrieval problems later, so cleaning happens before any embeddings or indexes. #llmzoomcamp
August 25, 2026 at 11:23 AM
4/ Extraction matters: I’m reviewing the source files for broken headings, repeated headers, table structures, and reading-order issues. Bad source text creates retrieval problems later, so cleaning happens before any embeddings or indexes. #llmzoomcamp
3/ Before chunking, I’m documenting each source: title, file type (PDF/HTML/JSON), section coverage, and intended audience (developers, utilities, regulators). This manifest makes the corpus reproducible and easier to update if the scope changes. #llmzoomcamp
August 25, 2026 at 3:16 AM
3/ Before chunking, I’m documenting each source: title, file type (PDF/HTML/JSON), section coverage, and intended audience (developers, utilities, regulators). This manifest makes the corpus reproducible and easier to update if the scope changes. #llmzoomcamp
2/ For the corpus, I’m using a curated set of DER interconnection documents: CPUC Rule 21 overview, SCE interconnection handbook, Rule 21 tariff, testing & certification instructions, and SIWG recommendations. Small, authoritative, and specific enough to evaluate retrieval properly. #llmzoomcamp
August 25, 2026 at 2:23 AM
2/ For the corpus, I’m using a curated set of DER interconnection documents: CPUC Rule 21 overview, SCE interconnection handbook, Rule 21 tariff, testing & certification instructions, and SIWG recommendations. Small, authoritative, and specific enough to evaluate retrieval properly. #llmzoomcamp
Project 3 of #llmzoomcamp. After AI-security guidance and ATT&CK mapping, I’m shifting to energy: a RAG system for DER (distributed energy resources) interconnection rules. The problem: product developers drown in long, overlapping tariffs and handbooks. Retrieval should make this more navigable.
August 23, 2026 at 12:24 PM
Project 3 of #llmzoomcamp. After AI-security guidance and ATT&CK mapping, I’m shifting to energy: a RAG system for DER (distributed energy resources) interconnection rules. The problem: product developers drown in long, overlapping tariffs and handbooks. Retrieval should make this more navigable.
20/ This review highlighted a RAG operations concern: refreshing raw data is not enough if the embedding cache, BM25 index, and search backend are not rebuilt in the same workflow. Keeping ingestion and index refresh synchronized prevents stale retrieval results. #llmzoomcamp
August 20, 2026 at 12:21 PM
20/ This review highlighted a RAG operations concern: refreshing raw data is not enough if the embedding cache, BM25 index, and search backend are not rebuilt in the same workflow. Keeping ingestion and index refresh synchronized prevents stale retrieval results. #llmzoomcamp
19/ Starting the third peer review of round 2. This ArXiv RAG project combines BM25 and dense retrieval with RRF, then applies cross-encoder reranking. It also separates dlt/DuckDB ingestion, Postgres feedback logging, and Grafana monitoring into distinct components. #llmzoomcamp
August 20, 2026 at 11:45 AM
19/ Starting the third peer review of round 2. This ArXiv RAG project combines BM25 and dense retrieval with RRF, then applies cross-encoder reranking. It also separates dlt/DuckDB ingestion, Postgres feedback logging, and Grafana monitoring into distinct components. #llmzoomcamp
18/ One of the best parts of peer reviews is testing out different local environments. It’s a great reminder of how valuable clean setup guides and isolated environments are for smooth collaboration and evaluation. #llmzoomcamp
August 20, 2026 at 10:51 AM
18/ One of the best parts of peer reviews is testing out different local environments. It’s a great reminder of how valuable clean setup guides and isolated environments are for smooth collaboration and evaluation. #llmzoomcamp
17/ Starting the second peer review of round 2. This project implements a local-first RAG architecture using Ollama and pgvector for weather data analysis. The use of a two-pass RAG pattern for structured screening prior to a deep LLM audit is an interesting approach to data extraction. #llmzoomcamp
August 20, 2026 at 7:57 AM
17/ Starting the second peer review of round 2. This project implements a local-first RAG architecture using Ollama and pgvector for weather data analysis. The use of a two-pass RAG pattern for structured screening prior to a deep LLM audit is an interesting approach to data extraction. #llmzoomcamp
16/ A notable pattern from this review was handling aggregate queries by pre-computing summary documents for the RAG context. They also used strict prompt grounding for their LLM judge to prevent knowledge cutoff bias. Practical approaches to common issues. #llmzoomcamp
August 20, 2026 at 7:39 AM
16/ A notable pattern from this review was handling aggregate queries by pre-computing summary documents for the RAG context. They also used strict prompt grounding for their LLM judge to prevent knowledge cutoff bias. Practical approaches to common issues. #llmzoomcamp