Predicting Decision-Making in the Future: Human Versus Machine

Hoe Sung Ryu, Uijong Ju, Christian Wallraven

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Deep neural networks (DNNs) have become remarkably successful in data prediction, and have even been used to predict future actions based on limited input. This raises the question: do these systems actually “understand” the event similar to humans? Here, we address this issue using videos taken from an accident situation in a driving simulation. In this situation, drivers had to choose between crashing into a suddenly-appeared obstacle or steering their car off a previously indicated cliff. We compared how well humans and a DNN predicted this decision as a function of time before the event. The DNN outperformed humans for early time-points, but had an equal performance for later time-points. Interestingly, spatio-temporal image manipulations and Grad-CAM visualizations uncovered some expected behavior, but also highlighted potential differences in temporal processing for the DNN.

Original languageEnglish
Title of host publicationPattern Recognition - 6th Asian Conference, ACPR 2021, Revised Selected Papers
EditorsChristian Wallraven, Qingshan Liu, Hajime Nagahara
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages15
ISBN (Print)9783031024436
Publication statusPublished - 2022
Event6th Asian Conference on Pattern Recognition, ACPR 2021 - Virtual, Online
Duration: 9 Nov 202112 Nov 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13189 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference6th Asian Conference on Pattern Recognition, ACPR 2021
CityVirtual, Online


  • Decision-making
  • Deep learning
  • Humans versus machines
  • Video analysis
  • Video prediction


Dive into the research topics of 'Predicting Decision-Making in the Future: Human Versus Machine'. Together they form a unique fingerprint.

Cite this