PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks
Manyi Wang, Junjielong Xu, Pinjia He
#arXiv #cs.SE #cs.AI
Manyi Wang, Junjielong Xu, Pinjia He
#arXiv #cs.SE #cs.AI
PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks
SWE-bench-like benchmarks are widely used for evaluating LLM's issue resolution capability. They typically follow a common construction pipeline: each PR (Pull Request) is paired with its linked issue by extracting issue references from the PR description; the issue description is used as the probl…
arxiv.org
July 31, 2026 at 9:07 PM
Manyi Wang, Junjielong Xu, Pinjia He: PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks https://arxiv.org/abs/2607.28587 https://arxiv.org/pdf/2607.28587 https://arxiv.org/html/2607.28587
July 31, 2026 at 6:45 AM
Manyi Wang, Junjielong Xu, Pinjia He: PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks https://arxiv.org/abs/2607.28587 https://arxiv.org/pdf/2607.28587 https://arxiv.org/html/2607.28587
Changyue Li, Jiaming He, Youliang Yuan, Jialin Wu, Boxi Yu, Zhicong Huang, Pinjia He: TRACE: Trajectory-Based Safety Patch Learning for LLM Post-Training Realignment https://arxiv.org/abs/2607.16242 https://arxiv.org/pdf/2607.16242 https://arxiv.org/html/2607.16242
July 21, 2026 at 6:42 AM
Changyue Li, Jiaming He, Youliang Yuan, Jialin Wu, Boxi Yu, Zhicong Huang, Pinjia He: TRACE: Trajectory-Based Safety Patch Learning for LLM Post-Training Realignment https://arxiv.org/abs/2607.16242 https://arxiv.org/pdf/2607.16242 https://arxiv.org/html/2607.16242
Piaopiao Jin, Qi Wang, Guokang Sun, Ziwen Cai, Pinjia He, Yangwei You
Dual-Actor Fine-Tuning of VLA Models: A Talk-and-Tweak Human-in-the-Loop Approach
https://arxiv.org/abs/2509.13774
Dual-Actor Fine-Tuning of VLA Models: A Talk-and-Tweak Human-in-the-Loop Approach
https://arxiv.org/abs/2509.13774
September 18, 2025 at 7:15 AM
Piaopiao Jin, Qi Wang, Guokang Sun, Ziwen Cai, Pinjia He, Yangwei You
Dual-Actor Fine-Tuning of VLA Models: A Talk-and-Tweak Human-in-the-Loop Approach
https://arxiv.org/abs/2509.13774
Dual-Actor Fine-Tuning of VLA Models: A Talk-and-Tweak Human-in-the-Loop Approach
https://arxiv.org/abs/2509.13774
Changyue Li, Jiaying Li, Youliang Yuan, Jiaming He, Zhicong Huang, Pinjia He: On the Feasibility of Hijacking MLLMs' Decision Chain via One Perturbation https://arxiv.org/abs/2511.20002 https://arxiv.org/pdf/2511.20002 https://arxiv.org/html/2511.20002
November 26, 2025 at 6:31 AM
Changyue Li, Jiaying Li, Youliang Yuan, Jiaming He, Zhicong Huang, Pinjia He: On the Feasibility of Hijacking MLLMs' Decision Chain via One Perturbation https://arxiv.org/abs/2511.20002 https://arxiv.org/pdf/2511.20002 https://arxiv.org/html/2511.20002
A & B == B & A: Triggering logical reasoning failures in large language models. ~ Yuxuan Wan, Wenxuan Wang, Yiliu Yang, Youliang Yuan, Jen-tse Huang, Pinjia He, Wenxiang Jiao, Michael R. Lyu. bit.ly/3tzWrIN #LLMs #Reasoning
A & B == B & A: Triggering Logical Reasoning Failures in...
Recent advancements in large language models (LLMs) have propelled Artificial Intelligence (AI) to new heights, enabling breakthroughs in various tasks such as writing assistance, code generation,...
bit.ly
January 2, 2024 at 10:16 AM
A & B == B & A: Triggering logical reasoning failures in large language models. ~ Yuxuan Wan, Wenxuan Wang, Yiliu Yang, Youliang Yuan, Jen-tse Huang, Pinjia He, Wenxiang Jiao, Michael R. Lyu. bit.ly/3tzWrIN #LLMs #Reasoning
Sihang Zhao, Shoucong Carol Xiong, Bo Pang, Xiaoying Tang, Pinjia He: Let AI Read First: Enhancing Reading Abilities for Individuals with Dyslexia through Artificial Intelligence https://arxiv.org/abs/2504.00941 https://arxiv.org/pdf/2504.00941 https://arxiv.org/html/2504.00941
April 2, 2025 at 5:58 AM
Sihang Zhao, Shoucong Carol Xiong, Bo Pang, Xiaoying Tang, Pinjia He: Let AI Read First: Enhancing Reading Abilities for Individuals with Dyslexia through Artificial Intelligence https://arxiv.org/abs/2504.00941 https://arxiv.org/pdf/2504.00941 https://arxiv.org/html/2504.00941
Piaopiao Jin, Qi Wang, Guokang Sun, Ziwen Cai, Pinjia He, Yangwei You: Dual-Actor Fine-Tuning of VLA Models: A Talk-and-Tweak Human-in-the-Loop Approach https://arxiv.org/abs/2509.13774 https://arxiv.org/pdf/2509.13774 https://arxiv.org/html/2509.13774
September 18, 2025 at 6:35 AM
Piaopiao Jin, Qi Wang, Guokang Sun, Ziwen Cai, Pinjia He, Yangwei You: Dual-Actor Fine-Tuning of VLA Models: A Talk-and-Tweak Human-in-the-Loop Approach https://arxiv.org/abs/2509.13774 https://arxiv.org/pdf/2509.13774 https://arxiv.org/html/2509.13774
Yifan Yang, Aoyang FANG, Songhan Zhang, Pinjia He: Gleaner: A Semantically-Rich and Efficient Online Sampler for Microservice Diagnostics https://arxiv.org/abs/2604.16810 https://arxiv.org/pdf/2604.16810 https://arxiv.org/html/2604.16810
April 21, 2026 at 6:45 AM
Yifan Yang, Aoyang FANG, Songhan Zhang, Pinjia He: Gleaner: A Semantically-Rich and Efficient Online Sampler for Microservice Diagnostics https://arxiv.org/abs/2604.16810 https://arxiv.org/pdf/2604.16810 https://arxiv.org/html/2604.16810
Boyin Tan, Haoning Deng, Junyuan Zhang, Junjielong Xu, Pinjia He, Youcheng Sun: SWE-Manager: Selecting and Synthesizing Golden Proposals Before Coding https://arxiv.org/abs/2601.22956 https://arxiv.org/pdf/2601.22956 https://arxiv.org/html/2601.22956
February 2, 2026 at 6:35 AM
Boyin Tan, Haoning Deng, Junyuan Zhang, Junjielong Xu, Pinjia He, Youcheng Sun: SWE-Manager: Selecting and Synthesizing Golden Proposals Before Coding https://arxiv.org/abs/2601.22956 https://arxiv.org/pdf/2601.22956 https://arxiv.org/html/2601.22956
Jianbo Yu, Yixuan Li, Hai Xu, Kang Xu, Junjielong Xu, Zhijing Li, Pinjia He, Wanyuan Wang: MicLog: Towards Accurate and Efficient LLM-based Log Parsing via Progressive Meta In-Context Learning https://arxiv.org/abs/2601.07005 https://arxiv.org/pdf/2601.07005 https://arxiv.org/html/2601.07005
January 13, 2026 at 6:35 AM
Jianbo Yu, Yixuan Li, Hai Xu, Kang Xu, Junjielong Xu, Zhijing Li, Pinjia He, Wanyuan Wang: MicLog: Towards Accurate and Efficient LLM-based Log Parsing via Progressive Meta In-Context Learning https://arxiv.org/abs/2601.07005 https://arxiv.org/pdf/2601.07005 https://arxiv.org/html/2601.07005
Zhiqing Zhong, Jiaming Huang, Pinjia He: BackportBench: A Multilingual Benchmark for Automated Backporting of Patches https://arxiv.org/abs/2512.01396 https://arxiv.org/pdf/2512.01396 https://arxiv.org/html/2512.01396
December 2, 2025 at 6:35 AM
Zhiqing Zhong, Jiaming Huang, Pinjia He: BackportBench: A Multilingual Benchmark for Automated Backporting of Patches https://arxiv.org/abs/2512.01396 https://arxiv.org/pdf/2512.01396 https://arxiv.org/html/2512.01396
Songhan Zhang, Aoyang Fang, Yifan Yang, Ruiyi Cheng, Xiaoying Tang, Pinjia He: DynaCausal: Dynamic Causality-Aware Root Cause Analysis for Distributed Microservices https://arxiv.org/abs/2510.22613 https://arxiv.org/pdf/2510.22613 https://arxiv.org/html/2510.22613
October 28, 2025 at 6:35 AM
Songhan Zhang, Aoyang Fang, Yifan Yang, Ruiyi Cheng, Xiaoying Tang, Pinjia He: DynaCausal: Dynamic Causality-Aware Root Cause Analysis for Distributed Microservices https://arxiv.org/abs/2510.22613 https://arxiv.org/pdf/2510.22613 https://arxiv.org/html/2510.22613
Aoyang Fang, Haowen Yang, Haoze Dong, Qisheng Lu, Junjielong Xu, Pinjia He: A Goal-Driven Survey on Root Cause Analysis https://arxiv.org/abs/2510.19593 https://arxiv.org/pdf/2510.19593 https://arxiv.org/html/2510.19593
October 23, 2025 at 6:35 AM
Aoyang Fang, Haowen Yang, Haoze Dong, Qisheng Lu, Junjielong Xu, Pinjia He: A Goal-Driven Survey on Root Cause Analysis https://arxiv.org/abs/2510.19593 https://arxiv.org/pdf/2510.19593 https://arxiv.org/html/2510.19593
Zhouruixing Zhu, Zhihan Jiang, Tianyi Yang, Pinjia He: UniSage: A Unified and Post-Analysis-Aware Sampling for Microservices https://arxiv.org/abs/2509.26336 https://arxiv.org/pdf/2509.26336 https://arxiv.org/html/2509.26336
October 1, 2025 at 6:35 AM
Zhouruixing Zhu, Zhihan Jiang, Tianyi Yang, Pinjia He: UniSage: A Unified and Post-Analysis-Aware Sampling for Microservices https://arxiv.org/abs/2509.26336 https://arxiv.org/pdf/2509.26336 https://arxiv.org/html/2509.26336
Junjielong Xu, Boyin Tan, Xiaoyuan Liu, Chao Peng, Pengfei Gao, Pinjia He
Scalable Supervising Software Agents with Patch Reasoner
https://arxiv.org/abs/2510.22775
Scalable Supervising Software Agents with Patch Reasoner
https://arxiv.org/abs/2510.22775
October 28, 2025 at 9:29 AM
Junjielong Xu, Boyin Tan, Xiaoyuan Liu, Chao Peng, Pengfei Gao, Pinjia He
Scalable Supervising Software Agents with Patch Reasoner
https://arxiv.org/abs/2510.22775
Scalable Supervising Software Agents with Patch Reasoner
https://arxiv.org/abs/2510.22775
Youliang Yuan, Qiuyang Mang, Jingbang Chen, Hong Wan, Xiaoyuan Liu, Junjielong Xu, Jen-tse Huang, Wenxuan Wang, Wenxiang Jiao, Pinjia He
Curing Miracle Steps in LLM Mathematical Reasoning with Rubric Rewards
https://arxiv.org/abs/2510.07774
Curing Miracle Steps in LLM Mathematical Reasoning with Rubric Rewards
https://arxiv.org/abs/2510.07774
October 10, 2025 at 8:42 AM
Youliang Yuan, Qiuyang Mang, Jingbang Chen, Hong Wan, Xiaoyuan Liu, Junjielong Xu, Jen-tse Huang, Wenxuan Wang, Wenxiang Jiao, Pinjia He
Curing Miracle Steps in LLM Mathematical Reasoning with Rubric Rewards
https://arxiv.org/abs/2510.07774
Curing Miracle Steps in LLM Mathematical Reasoning with Rubric Rewards
https://arxiv.org/abs/2510.07774
Youliang Yuan, Wenxiang Jiao, Yuejin Xie, Chihao Shen, Menghan Tian, Wenxuan Wang, Jen-tse Huang, Pinjia He
Towards Evaluating Proactive Risk Awareness of Multimodal Language Models
https://arxiv.org/abs/2505.17455
Towards Evaluating Proactive Risk Awareness of Multimodal Language Models
https://arxiv.org/abs/2505.17455
May 26, 2025 at 9:07 AM
Youliang Yuan, Wenxiang Jiao, Yuejin Xie, Chihao Shen, Menghan Tian, Wenxuan Wang, Jen-tse Huang, Pinjia He
Towards Evaluating Proactive Risk Awareness of Multimodal Language Models
https://arxiv.org/abs/2505.17455
Towards Evaluating Proactive Risk Awareness of Multimodal Language Models
https://arxiv.org/abs/2505.17455
Wenxuan Wang, Xiaoyuan Liu, Kuiyi Gao, Jen-tse Huang, Youliang Yuan, Pinjia He, Shuai Wang, Zhaopeng Tu
Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
https://arxiv.org/abs/2502.11184
Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
https://arxiv.org/abs/2502.11184
February 18, 2025 at 1:17 PM
Wenxuan Wang, Xiaoyuan Liu, Kuiyi Gao, Jen-tse Huang, Youliang Yuan, Pinjia He, Shuai Wang, Zhaopeng Tu
Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
https://arxiv.org/abs/2502.11184
Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
https://arxiv.org/abs/2502.11184
Sihang Zhao, Youliang Yuan, Xiaoying Tang, Pinjia He
Difficult Task Yes but Simple Task No: Unveiling the Laziness in Multimodal LLMs
https://arxiv.org/abs/2410.11437
Difficult Task Yes but Simple Task No: Unveiling the Laziness in Multimodal LLMs
https://arxiv.org/abs/2410.11437
October 16, 2024 at 11:02 AM
Sihang Zhao, Youliang Yuan, Xiaoying Tang, Pinjia He
Difficult Task Yes but Simple Task No: Unveiling the Laziness in Multimodal LLMs
https://arxiv.org/abs/2410.11437
Difficult Task Yes but Simple Task No: Unveiling the Laziness in Multimodal LLMs
https://arxiv.org/abs/2410.11437
Xiaoyuan Liu, Wenxuan Wang, Youliang Yuan, Jen-tse Huang, Qiuzhi Liu, Pinjia He, Zhaopeng Tu
Insight Over Sight? Exploring the Vision-Knowledge Conflicts in Multimodal LLMs
https://arxiv.org/abs/2410.08145
Insight Over Sight? Exploring the Vision-Knowledge Conflicts in Multimodal LLMs
https://arxiv.org/abs/2410.08145
October 11, 2024 at 10:32 AM
Xiaoyuan Liu, Wenxuan Wang, Youliang Yuan, Jen-tse Huang, Qiuzhi Liu, Pinjia He, Zhaopeng Tu
Insight Over Sight? Exploring the Vision-Knowledge Conflicts in Multimodal LLMs
https://arxiv.org/abs/2410.08145
Insight Over Sight? Exploring the Vision-Knowledge Conflicts in Multimodal LLMs
https://arxiv.org/abs/2410.08145
Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Jiahao Xu, Tian Liang, Pinjia He, Zhaopeng Tu
Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training
https://arxiv.org/abs/2407.09121
Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training
https://arxiv.org/abs/2407.09121
July 15, 2024 at 6:01 AM
Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Jiahao Xu, Tian Liang, Pinjia He, Zhaopeng Tu
Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training
https://arxiv.org/abs/2407.09121
Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training
https://arxiv.org/abs/2407.09121