Interpreting Deep Q-Networks: A Rule-Based Comparison with First-Order Logic in Wumpus World

Abstract

Deep reinforcement learning models such as Deep Q-Networks (DQNs) have achieved great performance in both simple and complex environments, but their decision-making process remains largely opaque. This work addresses the interpretability challenge by proposing a~method of extracting and comparing symbolic rules from a trained DQN and logic-based agents. The method is showcased in the popular Wumpus World domain. Rules extraction from the agents is done via training decision trees to mimic the agent's behavior. Comparison is done using Jaccard similarity and simple structural metrics. Results show that despite similar performance, DQNs and logic agents rely on partially overlapping but structurally largely distinct decision rules. This highlights the feasibility of translating subsymbolic policies into interpretable rules and reveals meaningful structural differences between learned and symbolic strategies.