Seeking Independent Reviewers: Wheat Rust AI Cross-Dataset Generalisation
Hello Hugging Face community!
I’m Devanandha VS, an independent AI researcher and MSc Artificial Intelligence graduate from Ulster University, UK.
I’m seeking independent researchers, computer-vision practitioners and open-source developers interested in evaluating an agricultural AI research project.
## Project Overview
**Wheat Rust AI Detection** is an open-source deep-learning project using a MobileNetV2-based classifier to identify three wheat leaf categories:
* Healthy
* Brown Rust
* Yellow Rust
The classifier achieved **98.78% accuracy (728/737 correct predictions)** on its internal validation dataset.
However, subsequent evaluations using independent datasets revealed substantial generalisation weaknesses.
Rather than presenting the initial accuracy as proof of real-world reliability, I investigated and publicly documented the model’s limitations, unsuccessful experiments and Grad-CAM explainability findings.
The project is an experimental research demonstration, not a production-ready agricultural diagnostic system.
## What I’m Looking For
I welcome independent contributions in four areas:
**1. Cross-Dataset Generalisation**
Evaluate the model on independent wheat leaf datasets and investigate differences in performance across data sources.
**2. Reproducibility**
Attempt to reproduce the documented experiments, including preprocessing, evaluation methodology and dataset configurations.
**3. Explainable AI**
Review the Grad-CAM analysis and assess the usefulness and limitations of the generated explanations.
**4. Model Reliability**
Explore improved evaluation methods, relevant baselines, calibration techniques and strategies for addressing dataset shift.
## How to Participate
I’ve opened a dedicated GitHub issue for independent evaluation and collaboration:
**Independent Reproduction and External Validation:**
Open issue
devanandha/wheat-rust-ai-detection
#### Call for independent reproduction and external validation — Wheat Rust AI
opened 12:44PM - 05 Nov 26 UTC
devanandha
I'm inviting independent researchers, machine-learning practitioners, plant-path…ology specialists, and open-source developers to **evaluate the Wheat Rust AI Detection project critically**. ### Background The project's MobileNetV2 classifier achieved **98.78% accuracy (728/737)** on its **internal validation set**, but later experiments using independent datasets revealed **substantial generalisation limitations**. This project is a research demonstration, **not a production-ready agricultural diagnostic tool**. I welcome positive and negative findings alike. ### Areas where external contributions would be useful 1. **Reproducibility:** Attempt to reproduce documented evaluation results; report environment, software versions, dataset provenance, splits, preprocessing and any discrepancies. 2. **External validation:** Evaluate on suitably licensed independent wheat-leaf image sets, with transparent class mapping and attention to data leakage. 3. **Dataset shift:** Examine possible reasons for performance differences across sources, such as acquisition conditions, label definitions or class imbalance. 4. **Explainability:** Critically assess Grad-CAM visualisations and interpretation, including potential limitations. 5. **Improved methods:** Propose or test baselines, uncertainty estimates, calibration methods and robustness evaluations. ### Where to contribute - **GitHub repository:** https://github.com/devanandha/wheat-rust-ai-detection - **Archived project:** https://doi.org/10.5281/zenodo.22726591 - **Interactive research demonstration:** https://devanandha-wheat-rust-ai.streamlit.app/ Please share independent observations in a comment, open a dedicated issue for a specific result, or submit a pull request for reproducible improvements. Include sufficient methodological detail for others to review your findings. Respect third-party dataset licences and privacy; avoid uploading data that cannot be redistributed. **There is no expectation of a favourable review, endorsement or citation.** Critical feedback, unsuccessful reproduction attempts and alternative explanations are equally welcome. The goal is to improve transparency, reproducibility and reliability in agricultural computer vision.
**Project Repository:**
Repository
devanandha/wheat-rust-ai-detection
### GitHub - devanandha/wheat-rust-ai-detection: AI-powered wheat rust classification using...
AI-powered wheat rust classification using MobileNetV2, with reproducible evaluation and a Streamlit demonstration.
Contributors are welcome to report findings, suggest improvements, submit pull requests or discuss potential methodological limitations.
Positive feedback, critical assessments and unsuccessful reproduction attempts are equally welcome.
## A Question for the Community
When evaluating agricultural image classifiers across independent datasets, what factors would you investigate first?
* Differences in image acquisition and environmental conditions?
* Class definitions and annotation quality?
* Data preprocessing and dataset composition?
* Domain shift, class imbalance or model calibration?
I’d appreciate hearing your experience and welcome collaboration on a more rigorous evaluation.
Thank you for your time and consideration!
**Devanandha VS**
MSc Artificial Intelligence, Ulster University
Independent AI Researcher