**Abstract:** This paper presents a novel approach to stochastic policy optimization (SPO) for highly dexterous robotic manipulation within cluttered environments. Existing reinforcement learning (RL) methods often struggle with the high dimensionality and complex dynamics inherent in such scenarios. Our method, Adaptive Curriculum of Stochastic Policy Exploration (ACSPE), addresses this challenge by dynamically tailoring the training curriculum based on the robot’s performance and environment configuration.