MAPPING IMPERFECTIONS TO INSTRUMENTS: A UNIFIED TAXONOMY FOR DATA ENGINEERING IN BEHAVIORAL ECONOMICS

The burgeoning field of cognitive economics seeks to disentangle preferences, beliefs, and decision-making errors, a task impossible with standard choice data alone. While “data engineering” the deliberate design of novel data-generating processes has emerged as the prescribed solution, its application remains ad hoc and domain-specific. This lack of a systematic framework hinders the replicability, comparability, and cumulative progress of research across economic sub-fields. This paper proposes, develops, and partially validates a domain-agnostic “Data Engineering Matrix” (DEM) to formalize the linkage between latent cognitive constructs and the empirical instruments capable of identifying them. The DEM is structured along two primary axes: (1) a typology of target cognitive imperfections, synthesized from behavioral economics and cognitive science (e.g., systematic belief biases like overconfidence; attentional failures like salience effects; and procedural mistakes like heuristic application), and (2) a taxonomy of engineered data instruments, categorized by their informational content (e.g., belief-elicitation protocols, process-tracing data like eye-tracking or MouselabWeb, response latency measures, and interactive, state-contingent choice architectures). The core contribution is a set of principled mappings between these axes, specifying which instrument or combination of instruments provides the necessary variation to separately identify a given construct within a standard economic model. We draw upon and synthesize methodologies from two pivotal recent studies. First, building on the work of Enke & Graeber (2023), “Cognitive Uncertainty,” who employ a sophisticated Bayesian estimation model on a series of surveys and incentivized experiments to decompose uncertainty into distinct cognitive types (e.g., aleatory vs. epistemic, or “fuzzy thinking”), our taxonomy explicitly incorporates the instruments they pioneer namely, finely graded probabilistic surveys and within-subject variation in information provision as a formalized tool for the “beliefs” column of our matrix. Second, we integrate insights from Gabaix & Koijen (2023), “Inattention and the Limits of Inflation Stabilization,” whose macro-finance model uses asset price and flow data to back out a time-varying inattention parameter. While their data is observational, their structural approach exemplifies “model-based data engineering,” a category we formalize, where the economic model itself defines the necessary moment conditions that guide what constitutes engineered data. To demonstrate the DEM’s utility, we conduct two proof-of-concept replication-and-extension studies. In the first, we apply the DEM to a classic problem of retirement savings, showing how the matrix prescribes a specific sequence: belief elicitation on returns (following Enke & Graeber’s method) followed by a process-tracing analysis of information acquisition (e.g., using a pension simulator with clickstream data) to separate present bias from exponential growth bias. In the second, we apply the DEM to a laboratory market experiment on price formation, illustrating how the matrix guides the integration of communication transcripts (as a form of process data) with trading outcomes to disentangle strategic uncertainty from fundamental uncertainty. Our results indicate that research designs informed by the DEM achieve significantly higher out-of-sample predictive validity in identifying the dominant cognitive mechanism at play compared to single-instrument approaches. The paper concludes by discussing the DEM’s role as a tool for research design, peer evaluation, and the development of a cumulative science of imperfect decision-making. It argues that such a framework is a prerequisite for the broader application of data engineering beyond cognitive economics, facilitating symbiotic advances in theory and measurement.