Which Traffic Simulation Software is Best? (Phase 2)

Aerial view of a two lane roundabout with trucks and cars entering and exiting the legs
Bryant Ficek and I did some testing of the accuracy of different roundabout modeling software packages at the end of last year (check it out here). We were trying to answer the questions:
  • How do you calibrate to something that doesn't exist?
  • How do you take an existing all way stop sign control intersection and calibrate to increased traffic forecasts with traffic signal or roundabout control at an intersection?

Our procedure was to collect existing volume, queuing and delay data at an intersection and then model the intersection in different software packages to see what results we got based on the existing turning movement counts. We recently expanded our study to include stop sign controlled intersections and traffic signal controlled intersections. We gave a presentation to NCITE as well as the Oregon ITE simulation group. Below is the screencast from the Oregon webinar and here's a pdf of our slides Which Traffic Simulation Software is Best-Updated. The webinar is a little long at 75 minutes, but we had a fun question answer session at the end. You can look at the data in the later slides of the pdf to get a flavor of our results. In a nutshell - we didn't have great results. Highway Capacity Software, Rodel, Synchro, SimTraffic, and Vissim all fail to consistently replicate real world conditions. (and sorry to all of the other modeling software programs out there, we focused on the ones that are heavily used in Minnesota). We believe much more data needs to be collected and then iterations need to be done to figure out if we can improve the models by regionally calibrating them. For example, maybe in Minnesota Vissim with a Peak Hour Factor of 0.94 and a Saturation Flow Rate of 2,180 passenger vehicles per lane per hour will get us the most realistic results. With the current state of the practice, I think the big takeaway is realizing software is just a tool and that our industry still relies a lot on engineering judgment. We need to better understand the shortcomings of our software tools in order to still provide sound engineering. We can't just go on blind faith that the software provides "answers" to sub-second level accuracy. We'd love to put all of our raw data into a freely open shareware platform so researchers and software companies could use it to improve our tools. Does anyone know of such a repository?