Reconciling Information Theory and Probability in Chaotic Dynamics: A Dual-Perspective Analysis of Stochastic Systems