Beyond the Map: Learning to Navigate Unseen Urban Dynamics Using Diffusion-Guided Deep Reinforcement Learning

Monu Nagar, Debasis Das,
Indian Institute of Technology Jodhpur, India
Code Paper Supplementary file Poster
Image 1

Sequential observations at different intervals show minimal changes in subsequent scenes and nearly identical actions between closely spaced frames. This indicates that minor variations in observations do not significantly influence vehicle behavior. Such stability is crucial, as it provides a robust foundation for developing a generalized solution capable of effectively addressing environmental uncertainties.

Abstract

Vision-based motion planning is a crucial task in Autonomous Driving (AD). Recent advancements in urban AD show that integrating Imitation Learning (IL) with Deep Reinforcement Learning (DRL) improves decision-making to be more like humans. However, IL methods depend on expert demonstrations to learn the optimal policy. The main drawback of this approach is the assumption that expert demonstrations are always optimal, which isn’t always true in real-world settings. This creates challenges in adapting to diverse weather conditions and dynamic traffic scenarios, often resulting in higher collision rates and increased risks to pedestrian safety. To address these challenges, we propose a Diffusion-Guided Deep Reinforcement Learning (DGDRL) framework that integrates a diffusion model with a Soft Actor-Critic DRL method to effectively mitigate environmental uncertainties and enable self-learning beyond the training maps for new tasks. This framework follows a novel modified partially observable Markov decision process (mPOMDP) to choose optimal action from original and diffusion-generated observations, ensuring that the policy behavior remains consistent with the current action. We use the CARLA NoCrash benchmark to train and evaluate the proposed framework. The method is validated in diverse urban environments (e.g., empty, regular, and dense) across multiple towns. Additionally, we compare our model against state-of-the-art techniques to ensure robustness and generalizability to new environments.

Environment settings

First Image
Second Image

Visualizing the testing environment across all towns (Town01 to Town05) under test weather conditions..

Testing Results (Weather: W02, Town: Town1 to Town5)

Town 1: Dense Traffic different routes

Town 2: Dense Traffic different routes

Town 3: Dense Traffic different routes

Town 4: Dense Traffic different routes

Town 5: Dense Traffic different routes

Evaluating the generalization capability of the proposed DGRL method across different towns under dense traffic and test weather conditions to assess its effectiveness in varying town structures.

Analysis of the model’s generalization capability to ensure safety measures during unexpected events in dense traffic setup under test weather conditions

Results captured during testing across different towns.

Generalization to unseen tasks: Despite being trained in Town01 (two-lane setup), the model autonomously learns and successfully executes smooth lane-switching tasks on a four-lane road in a completely new town setup without any collisions. This underscores the model’s capability to learn and master tasks beyond its training scope through self-learning.

Success Case

Success Case

Failure Case

Image 3

Comparative performance analysis between the proposed model and the most recent state-of-the-art method (RLFOLD). (a) Average rewards during evaluation (up to 106 steps), (b) Critic loss, and (c) Policy loss during training (up to 500K steps) versus agent steps.

Image 1 Image 2 Image 3

BibTeX

@inproceedings{ijcai2025p973,
  title     = {Beyond the Map: Learning to Navigate Unseen Urban Dynamics Using Diffusion-Guided Deep Reinforcement Learning},
  author    = {Nagar, Monu and Das, Debasis},
  booktitle = {Proceedings of the Thirty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-25}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {James Kwok},
  pages     = {8750--8758},
  year      = {2025},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2025/973},
  url       = {https://doi.org/10.24963/ijcai.2025/973},
}