Learning to control traffic
Deep reinforcement learning in SUMO
Jun – Nov 2024
Framing traffic signal timing as a Markov decision process and studying a DQN-based controller in simulation.
ROLE
Project lead
APPROACH
Reinforcement learning · DQN · SUMO
Research question
Traffic signal timing is a sequential decision problem: each phase choice changes the traffic state seen by the next decision. This project explores reinforcement learning as an approach to adapting signal control in a simulated road environment.
My contribution
- Led the project and formulated traffic signal control as a Markov decision process.
- Designed a DQN-based signal-timing approach and worked on SUMO simulation experiments.
- Developed a Vue-based interface for presenting the traffic-control workflow and visualizations.
Technical approach
- Encode lane density from vehicle counts and lane capacity, and define discrete actions over available green phases. This translates traffic measurements into the observation and action spaces used by the controller.
- Connect the DQN learning loop to SUMO with separate environment, signal-control, network, and replay modules. Handle yellow-light transitions and phase durations so proposed actions respect the simulator’s signal timing.
- In the inspected implementation, construct a reward from whether the selected action serves the largest grouped queue. Record stopped-vehicle counts and phase histories to inspect how this heuristic shapes control behavior.
Results & outcomes
- Built a modular workflow connecting traffic-state extraction, DQN decisions, signal execution, and CSV-based analysis. The engineering focus is making the learning loop operate under phase-transition and timing constraints.
- The implementation makes the control process inspectable through lane-density observations, queue-based feedback, and phase histories, connecting algorithm design with observable simulator behavior.
