In this paper, a novel approach termed Dynamic Self-Generated Fuzzy Q-Learning (DSGFQL) for automatically generating Fuzzy Neural Networks (FNNs) is presented. The structure and premises of FNNs are to be generated through the reward evaluation and unsupervised approaches while the consequents are trained via a Fuzzy Q-Learning (FQL) approach. The proposed DSGFQL methodology can automatically create, delete and adjust fuzzy neurons without either any priori knowledge or supervised learning. Structure self-identification and automatic parameter estimation are achieved. Fuzzy neurons can be created or deleted dynamically and the membership functions of those fuzzy neurons can be adjusted according to the reward evaluations. Simulation studies on an obstacle avoidance task by a mobile robot show that the proposed DSGFQL algorithm is superior to other existing methodologies.