The algorithms are coming for physical reality
There’s a new paper out this week from a team of researchers out of UC Berkeley and elsewhere titled “Reinforcement Learning Based Oscillation Dampening.”
Agents taught to reduce the peaks of troughs of waves through positive and negative feedback sounds awfully technical and academic. But the real world application they built and tested their system against is very relatable: the traffic jams that seemingly emerge at random on highways, costing every driver involved time and money even though there’s no apparent blockage of the actual roadway.
These bubbles come from late braking, late acceleration, and lane changes by impatient and distracted human drivers compounding into extreme delays.
So these researchers decided to explore what could be done to stop that compounding if you had autonomous vehicles amongst the traffic. Maybe, if the cars could always be patient and leave a bit more room in front of them than a human defending their ego by keeping people from merging into their space, there would be less hitting the brake at the last second, less stomping on the gas, and less disruptive merging to get into a lane that looks like it’s decided to go faster.
These autonomous vehicles had constraints: they couldn’t communicate. They also weren’t covered in Lidar like a Waymo car or even lots of cameras fed into neural net like a Tesla. Instead, their much simpler model would be running on each individual vehicle independently, working from that specific vehicle’s current speed and gap to the car in front of it.
Despite these constraints, after collecting real-world traffic data and training agents acting on these models in simulated driving environments, they were able to deploy 100 vehicles running the software into real traffic and demonstrated stabilized speeds between stop-and-go waves by reducing the variability in accelerations.
Here’s the big takeaway from this research: robots acting in our real world will modify our behavior and experiences well before they become the dominant way we do things.
It’s easy to imagine highways without traffic jams if every car on the road is driven by an impersonal neural net translating pixels into control commands. But what we might not anticipate is that just a few intelligent systems added to the mix can radically change how humans act. Some cars leave some more gaps by driving a little slower and everyone gets to save fuel/battery charge and provide a smoother ride for their passengers.
We’re aware of and reckoning with these kinds of influences in the digital landscape in the form of feeds and LLM-powered bots. Now they’re coming to the real world.