Towards representing human behavior and decision making in Earth system models – an overview of techniques and approaches

Abstract

Today, humans have a critical impact on the Earth system and vice versa, which can generate complex feedback processes between social and ecological dynamics. Integrating human behavior into formal Earth system models (ESMs), however, requires crucial modeling assumptions about actors and their goals, behavioral options, and decision rules, as well as modeling decisions regarding human social interactions and the aggregation of individuals' behavior. Here, we review existing modeling approaches and techniques from various disciplines and schools of thought dealing with human behavior at different levels of decision making. We demonstrate modelers' often vast degrees of freedom but also seek to make modelers aware of the often crucial consequences of seemingly innocent modeling assumptions.

After discussing which socioeconomic units are potentially important for ESMs, we compare models of individual decision making that correspond to alternative behavioral theories and that make diverse modeling assumptions about individuals' preferences, beliefs, decision rules, and foresight. We review approaches to model social interaction, covering game theoretic frameworks, models of social influence, and network models. Finally, we discuss approaches to studying how the behavior of individuals, groups, and organizations can aggregate to complex collective phenomena, discussing agent-based, statistical, and representative-agent modeling and economic macro-dynamics. We illustrate the main ingredients of modeling techniques with examples from land-use dynamics as one of the main drivers of environmental change bridging local to global scales.

Introduction

Even though Earth system models (ESMs) are used to study human impacts on the complex interdependencies between various compartments of the Earth, humans are not represented explicitly in these models. ESMs usually consider human influence in terms of scenarios for comparison of the impacts of alternative narratives about the future development of key socioeconomic characteristics. For instance, the IPCC process uses integrated assessment models to compute plausible future emission pathways from energy and land use for different scenarios of climate mitigation. These projections determine the radiative forcing used as external input in ESMs to study its natural impacts . The latter can, however, have socioeconomic consequences that may be fed back into the scenario process. However, the complex interplay of the dynamics of the natural Earth system and the social, cultural, and economic responses to them are not captured.

The concept of the Anthropocene epoch implies that humans have become a dominant geological force interfering with biophysical Earth system processes . However, a changing environment also alters human behavior . For example, climate change will affect land use and energy consumption. Likewise, perceived environmental risks modify consumption and mobility patterns. Therefore, with increasing human impact on the Earth system, feedbacks between shifts in the biophysical Earth system and human responses will gain importance . provide a classification of these feedbacks in this Special Issue.

Studying feedback loops between human behavior and the Earth system, projecting its consequences, and developing interventions to manage the human impact on the Earth system requires a suitable dynamic representation of human behavior and decision making. In fact, even a very accurate statistical description of human behavior may be insufficient for several reasons. First, in a closed loop, humans constantly respond to changes in the Earth system, facing novel environmental conditions and decision problems. Hence, their response cannot be predicted with a statistical model. Second, for a correct assessment of different policy options (e.g., command and control policy vs. market-based solutions), a sound theoretical and empirical account of the principles underlying decision making in the relevant context is needed because they guide the development of intervention programs, such as incentives schemes, social institutions, and nudges . A statistical model could mislead decision makers that want to design policy interventions to induce changes in human behavior.

Incorporating human behavior in ESMs is challenging. In contrast to physical laws that traditional ESMs can use as a basis, there is no single theory of human behavior that can be taken as a general law . The understanding of human behavior is limited by its determinants often being contingent and socially formed by norms and institutions. This allows for a view on social systems as socially constructed realities, which is in stark contrast to the positivist epistemology of one objective reality prevalent in the natural sciences. In fact, past attempts to develop grand theories have been criticized for being too remote from reality and, as a consequence, hard if not impossible to test empirically . Accordingly, many social scientists favor a so-called “middle-range approach”, trying to tailor theoretical models to specific contexts rather than developing overarching general theories. This acknowledges, for instance, that individuals act in some contexts egoistically and based on rational calculus, while in other contexts they may act altruistically and according to simple heuristics. The principles that determine human decisions depend on, for example, whether the decision maker has faced the decision problem before, the complexity of the decision, the amount of time and information available to the individual, and whether the decision affects others or is framed in a specific social situation. Likewise, different actor types might apply different decision principles. Furthermore, the decision determinants of agents can be affected by others through social interactions or aggregate outcomes of collective processes.

Here, we give an overview of existing approaches to model human behavior and decision making to provide readers with a toolbox of model ingredients. Rather than promoting one theory and dismissing another, we list decisions that modelers face when modeling humans, point to important modeling options, and discuss methodological principles that help in developing the best model for a given purpose.

We define decision making as the cognitive process of deliberately choosing between alternative actions, which may involve analytic and intuitive modes of thinking. Actions are intentional and subjectively meaningful activities of an agent. Behavior, in contrast, is a broader concept that also includes unconscious and automatic activities, such as habits and reflexes. The outcome of a decision is therefore a certain type of behavior, which might be explained by a decision-making theory.

In ESMs, only those human decisions and behaviors that have a considerable impact on the Earth system are relevant, i.e., primarily behavior towards the environment of a large number of individuals or decisions amplified through the social position of the decision maker or technology. Therefore, this paper also covers techniques to model interactions between agents and to aggregate behavior and interactions to a macrolevel. On the microlevel, relevant decisions include the reproduction, consumption, and production of energy- and material-intensive products, place of living, and land use. These decisions lead to aggregate and long-term dynamics of populations, production and consumption patterns, and migration.

There are diverse social science theories explaining human behavior and decision making in environmental and ecological contexts, for example in environmental economics, sociology, and psychology. In this paper, we focus on mathematical and computational models of human decision making and behavior. Here, we understand the terms “modeling approach” and “modeling technique” as a class of mathematical or computational structures that can be interpreted as a simplified representation of physical objects and actors or collections thereof, events and processes, causal relations, or information flows. Modeling approaches draw on theories of human behavior that make – often contested – assumptions about the structure of decision processes. Furthermore, modeling approaches can have different purposes: the objective of descriptive models is to explore empirical questions (e.g., which components and processes can explain the system's dynamics), while normative models aim at answering ethical questions (e.g., which policy we should choose to reach a certain goal).

Recent reviews focus on existing modeling approaches and theories that are applied in the context of environmental management and change. For example, assess existing modeling approaches and identify challenges for improving these models in order to better understand Anthropocene dynamics. , , and focus on cognitive and behavioral theories in ecological contexts, providing an overview for developers of agent-based, land-use, and social–ecological models. and review different micro- and macro-approaches with applications to agro-ecology and the economics of climate change, respectively. The present paper complements this literature by reviewing modeling approaches of (1) individual agent behavior, (2) agent interactions, and (3) the aggregation of individual behaviors with the aim of supporting the integration of human decision making and behavior into Earth system models. The combination of these three different categories is crucial to describe human behavior at scales relevant for Earth system dynamics. Furthermore, this review highlights the strengths and limitations of different approaches by connecting the modeling techniques and their underlying assumptions about human behavior and discusses criteria to guide modeling choices.

Our survey of techniques has a bias towards economic modeling techniques for two simple reasons. First, economics is the social science discipline that has the longest and strongest tradition in the formal modeling of human decision making. Second, economics focuses on the study of production and consumption as well as the allocation of scarce resources. In most industrialized countries today, a major part of human interactions with the environment is mediated through markets, which are central in economic analyses. This review aims to go beyond the often narrow framing of economic approaches while at the same time not ignoring important economic insights. For instance, consumption and production decisions not only follow purely economic calculations, but are also deeply influenced, for instance, by behavioral patterns, traditions, and social norms .

Because we discuss different approaches to model decision making and behavior from various disciplinary or sub-disciplinary scientific fields, there are considerable differences in terminology that make a harmonized presentation of the material challenging. For example, the same terms are used to describe quite separate varieties of an approach in different fields, and different terms from separate fields may refer to very similar approaches. We adopt a terminology that aims for a better interdisciplinary understanding and point out different understandings of contested terms where we are aware of them.

This paper works with land-use change as a guiding and illustrative example. Land-cover change and land use make up the second-largest source of greenhouse gases – besides the burning of fossil fuels – and thus contribute strongly to climate change. Behavioral responses related to land use will play a crucial role for successful mitigation and adaptation to projected climatic changes, thereby challenging modelers to represent decision making in models of land-use change . The complexity of land-use change provides various examples of how collective and individual decision making interacts with the environment across spatial scales and organizational levels. Land-use models consider environmental conditions as important factors in decision-making processes, giving rise to feedbacks between environmental and socioeconomic dynamics . However, this paper does not provide an exhaustive overview of existing land-use models. For this purpose, the reader is referred to the various reviews in the literature e.g.,.

Overview of modeling categories, corresponding modeling approaches, and techniques discussed in this paper and important considerations for model choice and assumptions about human behavior and decision making.

[Figure omitted. See PDF]

The remainder of the paper is organized as follows. In Sect. , we give an overview of different levels of description of social systems and the socioeconomic units or agents associated with them. Sections – form the main part of the paper, presenting different modeling techniques and their underlying assumptions about human decision making and behavior. First, Sect.  introduces approaches to model individual decisions and behavior from rational choice to learning theories. Many of these techniques can be used to also model higher-level social entities. Second, Sect.  puts the focus on techniques for modeling interactions between agents. Strategic interactions and social influence are significant determinants of individual decisions and therefore important for long-term changes in collective behavior, i.e., the group outcome of mutually dependent individual decisions. Third, Sect.  reviews different aggregation techniques that allow for a description of human activities at the level of social collectives or systems. These approaches make use of simplifications to scale up theories about individual decision making. Figure  summarizes these main parts of the paper, the corresponding modeling approaches, and important considerations for model selection, which we discuss in detail in Sect. . The discussion also reflects on important distinctions between models of natural and social systems that are crucial to consider when including human behavior into ESMs. The paper concludes with remarks on the remaining challenges for this endeavor.

The challenge: modeling decision making and behavior across different levels of organization

The decision making and behavior of humans can be described and analyzed at different levels of social systems. While decisions are made and behavior is performed by individual humans, it is often useful to not represent individual humans in a model but to treat social collectives, such as households, neighborhoods, cities, political and economic organizations, and states, as decision makers or agents.

Figure  shows a hierarchy of socioeconomic units, i.e., the groups, organizations, and structures of individuals that play a crucial role in human interactions with the Earth system. We consider a broad scheme of levels ranging from the microlevel across intermediate levels to the global level. This hierarchy of socioeconomic units is not only distinguishable by level of complexity but also by the different spatial scales involved. However, there is no one-to-one correspondence. For instance, some individuals have impacts at the global level, while many transnational organizations operate at specific local levels. Especially in the context of human–environment interactions in ESMs, scaling and spatial extent are therefore important issues . Furthermore, we note that the strict separation between a microlevel and macrolevel may result in treating very different phenomena alike. For instance, many economic models describe both small businesses and transnational corporations as actors on the microlevel and model their decision processes with the same set of assumptions, even though they operate very differently.

One major challenge for modeling humans in the Earth system is therefore to bridge the diverse levels between individuals and the global scale, thereby integrating different levels of social organization and spatial and temporal scales.

The relation between individual agents and social collectives and structures has been the subject of considerable debate in the social sciences. In the social scientific tradition of methodological individualism

We note that there are different accounts of methodological individualism, and it often remains unclear to what extent structural and interactionist elements can be part of an explanation see.

, the analysis aims to explain social macro-phenomena, for example phenomena at the level of groups, organizations, or societies, with theories of individual behavior. This approach deviates from structuralist traditions, which claim that collective phenomena are of their own kind and thus cannot be traced back to the behavior of individuals . Positions between these two extremes emphasize the interdependency of individual agents and social structure, which is understood as an emerging phenomenon that stabilizes particular behaviors . While it very much depends on the purpose of the given modeling exercise whether the model should represent individuals or collectives, we mainly focus here on the research tradition that acknowledges the fact that complex and unexpected collective phenomena can arise from the interplay of individual behavior.

In Table , we provide an overview of socioeconomic units at different levels that are potentially important for Earth system modeling. We list common theories, frameworks and assumptions made about decision making and behavior for these socioeconomic units and link them to scientific fields that focus on them.

Overview of particular levels of description of socioeconomic units, associated scientific fields and communities, and some common approaches and assumptions about decisions and behavior. The list gives a broad overview but is far from being exhaustive.

Level	Socioeconomic units	Fields/communities	Common approaches and theories	Common assumptions about decision making
Micro	Individual humans	Psychology, neuroscience, sociology, economics, anthropology	Rational choice, bounded rationality, heuristics, learning theory, cognitive architectures	[All assumptions presented in this column]
	Households, families, small businesses	Economics, anthropology	Rational choice, heuristics, social influence	Maximization of consumption, leisure, profits
Intermediate	Communities (villages, neighborhoods), cities	Sociology, anthropology, urban studies	Social influence, networks	Transmission and evolution of cultural traits and traditions
	Political parties, NGOs, lobby organizations, educational institutions	Political science, sociology	Strategic decision making, public/social choice, social influence and evolutionary interactions	Agents form coalitions and cooperate to achieve goals, influenced by beliefs and opinions of others
	Governments	Political science, operations research	Strategic decision making, cost–benefit and welfare analysis, multi-criteria decision making,	Agents choose for the common good
	Nation states, societies	Economics, political science, sociology	welfare maximization, social choice	Majority vote
Global	Multinational firms, trade networks	Economics, management science	Rational choice	Maximization of profits or shareholder value
	Intergovernmental organizations	Political science (international relations)	Strategic decision making, cost–benefit analysis	Coalition formation

Socioeconomic units and their corresponding level and scales.

[Figure omitted. See PDF]

At the microlevel, models consider individuals, households, families, and small businesses. For instance, individuals can make decisions as policy makers, investors, business managers, consumers, or resource users. At this level, decisions about lifestyle, consumption, individual natural resource use, migration, and reproduction are particularly relevant in the environmental context. Individual decisions have to be made by a large number of individuals or have to be reinforced by organizations, institutions, or technology to become relevant at the level of the Earth system. Individuals' participation in collective decision processes, such as voting, may also have consequences for the environment at a global level.

At various intermediate levels, communities and organizations like firms, political parties, labor unions, educational institutions, and nongovernmental and lobby organizations play a crucial role in shaping economic and political decisions and therefore have a huge impact on aggregate behavior. Governments at different levels representing different territories, from cities to nation states, enact laws that strongly frame the economic and social activities of their citizens. Important decisions for the Earth system context include environmental regulations and standards, the production and distribution of commodities and assets, trade, the extraction and use of natural resources, and the development and building of physical infrastructures.

At the global level, multinational companies and intergovernmental organizations negotiate decisions. This level has considerable impacts on policy and business decisions even though it is remote from the daily life of most individuals. Often this level provides framing for activities on lower organizational levels and thus strongly influences the problem statements and perceived solutions, for instance regarding environmental issues. Decisions important for the Earth system at this level include international climate and trade agreements, the decisions of internationally operating corporations and financial institutions, and the adoption of global frameworks like the UN Sustainable Development Goals .

An overarching question that has triggered considerable debate between different disciplines is the allocation of agency at different levels of description. Even if individuals can decide between numerous options, the perception of options and decisions between them are shaped by social context and institutional embedding. Institutions

The notion of institution is used in the literature with slightly different meanings: (1) formal and informal rules that shape behavior, (2) informal social order, i.e., regular patterns of behavior, and (3) organizations. Here, we adopt an understanding of institutions as formal (e.g., law, property rights) or informal rules (e.g., norms, religion). However, formal rules often manifest in social, political, and economic organizations and informal rules may be shaped by them.

and organizations can display their own dynamics and lead to outcomes unintended by the individuals. On the other hand, social movements can initiate disruptive changes in institutional development. The attribution and perception of agency for a specific problem is therefore important for the choice of a suitable level of model description. The following section starts our discussion of different modeling techniques at the level of individual decision making and behavior.

Modeling individual behavior and decision making

In a nutshell, models of individual decision making and behavior differ with regard to their assumptions about three crucial determinants of human choices: goals, restrictions, and decision rules . First, the models assume that individuals have motives, goals, or preferences. That is, agents rank goods or outcomes in terms of their desirability and seek to realize highly ranked outcomes. A prominent but debated assumption of many models is that preferences or goals are assumed to be stable over time. Stable preferences are included to prevent researchers from developing trivial explanations, as a theory that models a given change in behavior only based on changed preferences does not have explanatory power. However, empirical research shows that preferences can change even in relatively short time frames . Changing individuals' goals or preferences is an important mechanism to affect their behavior, for example through policies, making flexible preferences particularly interesting for Earth system modelers.

Second, decision models make assumptions about restrictions and opportunities that constrain or help agents pursue their goals. For instance, each behavioral option comes with certain costs (e.g., money and time), and decision makers form more or less accurate beliefs about these costs and how likely they are to occur depending on the information available to the agent.

Third, models assume that agents apply some decision rule that translates their preferences and restrictions into a choice. Although decision rules differ very much in their complexity, they can be categorized into three types. First, there are decision rules that are forward looking. Rational choice theory, for instance, assumes that individuals list all positive and negative future consequences of a decision and choose the optimal option. Alternatively, backward-looking approaches, such as classical reinforcement learning, assume that actors remember the satisfaction experienced when they chose a given behavior in the past and tend to choose a behavior with a high satisfaction again. Finally, there are sideward-looking decision rules, which assume that actors adopt the behavior of others, for instance because they imitate successful others . Theories assume different degrees of the context dependency of rules and make different implicit assumptions about the underlying cognitive capabilities of agents.

In the remainder of this section, we describe in more detail three important approaches to individual decision making and point out typical assumptions about motives, restrictions, and decision rules.

Optimal decisions and utility theory in rational choice models

Rational choice theory, a standard model in many social sciences (especially in economics) that is widely studied in mathematics, assumes that decision making is goal oriented: rational agents have preferences and choose the strategy with the expected outcome that is most preferred, given some external constraints and potentially based on their beliefs represented by subjective probability distributions; see the beliefs, preferences, and constraints model in. It can either be used to represent actual behavior or serve as a normative benchmark for other theories of behavior.

How to judge the “rationality” of individual decisions is subject to ongoing debates. distinguishes between strong rationality (“homo economicus”), assuming purely self-interested agents with unlimited cognitive capacities knowing all possible actions and probabilities of consequences, and weak rationality that makes less strong assumptions. distinguishes between standard and nonstandard assumptions regarding preferences, beliefs, and decision-making rules. Before discussing nonoptimal decision making in Sect. , we review here common assumptions on preferences and beliefs.

Usually, agents are assumed to be mainly self-interested, having fixed preferences regarding their personal consequences in possible futures and being indifferent to how a decision was made and to consequences for others. Exceptions are procedural and other-regarding preferences .

Preferences can be modeled as binary preference relations, $x{P}_{i}y$, denoting that individual $i$ prefers situation or outcome $x$ to $y$. Most authors assume that $P_i$ is complete (for every pair $(x,y)$ either $x{P}_{i}y$ or $y{P}_{i}x$) and transitive (if $x{P}_{i}y$ and $y{P}_{i}z$ then $x{P}_{i}z$), which allows for the representation of the preferences with a utility function $u_i$ .

$u_i\left(x\right)>u_i\left(y\right)$ implies $x{P}_{i}y$, where $u_i$ is only defined up to positive linear (affine) transformations.

Some authors also allow for incomplete or cyclic preferences . In the land-use context, $i$ could be a farmer, $x$ might denote growing some traditional crops generating a moderate profit, and $y$ growing hybrid seeds for more profit but making $i$ dependent on the seed supplier. If $i$ considers independence valuable enough to make up for the lower profit, $x{P}_{i}y$ would denote $i$'s preference of $x$ over $y$.

In decision making under uncertainty, agents have to choose between different risky prospects modeled as probability distributions $p\left(x\right)$ over outcomes $x$. In expected utility theory, $p$ is preferred to $p^{\prime }$ if and only if ${\sum }_{x}p\left(x\right)u_i\left(x\right)>{\sum }_x{p}^{\prime }\left(x\right)u_i\left(x\right)$. Empirical research shows that only a minority of people evaluate uncertainty in this risk-neutral way . Prospect theory therefore models agents that overestimate small probabilities and evaluate outcomes relative to a reference point, which leads to risk-averse or risk-seeking behavior regarding losses or gains, respectively. . A conceptual example from the land-use context illustrates decision making under risk. A farmer $i$ might face the choice of whether to stick to her current crop $x$ or switch to a new crop $y$. She may think that with 20 % probability the switch will result in a 50 % reduction in her profits, while with 80 % probability the profits would double. If her utility is proportional to the profits and she evaluates this uncertain prospect as described by expected utility theory, her gain from switching to $y$ would be positive. If, however, she is averse to losses and thus conforms to prospect theory, she might evaluate the switch as negative and prefer to stick to $x$.

If several time points $t$ are involved in a decision, agents are typically assumed to discount future consequences by using utility weights that decay in time and reflect the agent's time preferences. Discounted utility quantifies the present desirability of some utility obtained in the future. Most authors use exponentially decaying weights of the form $e^{-rt}$ with a discounting rate $r>\mathrm{0}$ because this makes the evaluation independent of its time point. However, empirical studies suggest that people often use slower decaying weights (e.g., hyperbolic discounting), especially in the presence of uncertainty , although this might lead to time-inconsistent choices that appear suboptimal at a later time. A farmer $i$ may compare different crops not only by next year's expected profit $u_i(x,\mathrm{1})$ but, due to the various crops' different effects on future soil quality, also by future years' profits $u_i(x,t)$ for $t>\mathrm{1}$. Crop $y$ might promise higher yields than $x$ in the short run but lower ones in the long run due to faster soil depletion. If $i$ is “patient”, having small $r$, she might prefer $y{P}_{i}x$ even though $u_i(x,\mathrm{1})>u_i(y,\mathrm{1})$.

Preferences can be aggregated not only in time but also across several interrelated issues or consequences. For example, consumer theory models preferences over consumption bundles by combining the utility derived from consuming different products into a total consumption utility and simply adding up these utilities or combining them in some nonlinear way with imperfect substitutability of goods (Leontief, Cobb–Douglas, or CES utility functions). A farmers' utility from leisure time and crop yield $y\left(l\right)$ depending on working time $l$ might, for example, be combined using the Cobb–Douglas utility function $u_i=y^{\mathit{\alpha }}(\mathrm{12}-l{)}^{\mathrm{1}-\mathit{\alpha }}$ for some elasticity $\mathit{\alpha }\in (\mathrm{0},\mathrm{1})$.

Complex optimization problems arising from rational choice theory can be solved by mathematical programming, calculus of variations, and similar methods see, e.g.,. Optimal decisions under constraints are not only discussed as a description of human behavior, but are also often taken as the normative benchmark for comparison with other nonoptimal approaches that we discuss in Sect. .

Regarding decision modeling in ESMs, rational choice theory is useful when agents have clear goals and possess enough information and cognitive resources to assess the optimality of strategies. For instance, individuals' decisions regarding long-term investments or the decisions of organizations, such as firms or governments, in competitive situations can often be assumed to follow a rational choice model reasonably well. It can also be useful when actors make repeated similar decisions and can learn optimal strategies from fast feedback, making them behave “as if” they were rational.

Bounded rationality and heuristic decision making

Empirical research on human decision making finds that individual behavior depends on the framing and context of the decision . Human decision making is characterized by deviations from the normative standards of the rational choice model, so-called cognitive biases, challenging the assumption that rational choice theory serves not only as a normative benchmark, but also as a descriptive model of individual decision making. Biases can be the result of time-limited information processing , heuristic decision making , or emotional influences e.g., wishful thinking,. Bounded rationality theory assumes that human decision making is constrained by the cognitive capabilities of the agents in addition to the constraints imposed by the environment and the available information about it . In the economic literature, non-transitive preferences, time-inconsistent discounting, and deviations from expected utility that we already introduced in the previous subsection are also often considered as boundedly rational . Boundedly rational agents can be considered as satisficers that try to find a satisfying action in a situation given their available information and cognitive capabilities .

Constraints on information processing imply that agents do not integrate all the available information to compute the utility of every possible option in complex decision situations and choose an action with maximal utility. Instead, agents use heuristics to judge the available information and choose actions that lead to the more preferred outcome over less preferred ones. define heuristics in decision making as a “strategy that ignores part of the information, with the goal of making decisions more quickly, frugally, and/or accurately than more complex methods.” It is argued that instead of an all-purpose tool, the mind carries an “adaptive toolbox” of different heuristic decision schemes applicable in particular environments .

In general, heuristic rules are formalized either as decision trees or flowcharts and consist of three building blocks: one for information search, one for stopping the information search, and one to derive a decision from the information found. They evaluate a number of pieces of information – so-called cues – to either categorize a certain object or to choose between several options. Many heuristics evaluate these cues in a certain order and make a decision as soon as a cue value allows for classification or discriminates between options.

This is illustrated by means of the take-the-best heuristic: pieces of information (cues) are compared between alternatives according to a prescribed order, which is crucial for the decision process. At each step in the cue order, some information is searched for and evaluated. If the information does not allow for discrimination between the options, the process moves on to the next cue. This repeats as the process moves down the cue order until a cue is reached for which the differentiation between options is possible and the option with the higher cue value is chosen. Another notable example is the satisficing heuristic that evaluates information sequentially and chooses the first option satisfying certain criteria. Heuristics, especially cue orders, can be interpreted as encoding norms and preferences in individual decision making as they prioritize features of different options over others and hierarchically structure the evaluation of available information. An overview and explanation of numerous other decision heuristics can be found in the recent review paper by .

question the usefulness of rational choice theory as the normative benchmark because it is not designed for so-called “large worlds” where information relevant for the decision process is either unknown or has to be estimated from small samples. Instead, they want to relieve heuristic decision making of its stigma of cognitive laziness, bias, and irrationality. With their account of ecological rationality, they suggest that heuristics can also serve as a normative choice model providing context-specific rules for normative questions. This is motivated by the observation that in many real-world situations, especially when high uncertainties are involved, some decision heuristics perform equally good or even better than more elaborated decision strategies .

So far, heuristics have been used to describe decisions, for instance in consumer choice , voter behavior , and organizational behavior . However, fast and frugal decision heuristics are not yet commonly applied in dynamic modeling of human–nature interactions. One exception is the description of farmer and pastoralist behavior in a study of the origins of conflict in East Africa . However, as the following example shows, similar decision trees have been used to model decision making in agent-based simulations of land-use change. The model by describes colonist household decisions in the Amazon rainforest. Each household is a potential farmer who first checks whether a subsistence requirement is met. If this is not the case, the household farms annual crops. If the subsistence requirement is met, the household eventually plants perennials or breeds livestock depending on the soil quality. The model shows how heuristic decision trees can be used to simplify complex decision processes and represent them in an intelligible way. However, the example also shows the many degrees of freedom in the construction of heuristics, pointing at the difficulty to obtain these structures from empirical research.

Heuristics are a promising tool for including individual human decision making into ESMs because they can capture crucial choices in a computationally efficient way. In order to describe the long-term evolution of preferences, norms, and values relevant for human interactions with the Earth system, heuristics could also be used to model meta-decisions of preference or value adoption. Recent findings suggest that cue orders can spread via social learning and social influence analogously to norm and opinion spreading in social networks (see Sects.  and ), which could be a promising approach to model social change. However, in contrast to fully rational decision making, it can be very challenging to aggregate heuristic decision making analytically to higher organizational levels. Therefore, approaches like agent-based modeling are suitable to explore the aggregate outcomes of many agents with such decision rules (see Sect. ).

Learning theory

The approaches discussed in the previous two subsections mainly took the perspective of a forward-looking agent. Rational or boundedly rational actors optimize future payoffs based on information or beliefs about how their behavior affects future payoffs, while the procedures to optimize may be more or less bounded. However, these techniques do not specify how the information is acquired and how the beliefs are formed. Computational learning theory focuses on behavior from a backward-looking perspective: an agent learned in the past that a certain action gives a reward that feels good or is satisfying and is therefore more likely to repeat this behavior. It can describe the adaptivity of agent behavior to a changing environment and is particularly suited for modeling behavior under limited information. To model the learning of agents, unsupervised learning techniques are mostly used because they do not require training with an external correction.

Reinforcement learning is such a technique that models how an agent maps environmental conditions to desirable actions in a way that optimizes a stream of rewards (and/or punishments). The obtained reward depends on the state of the environment and the chosen action, but may also be influenced by chosen actions and environmental conditions in the past. According to , reinforcement learning differs from forward-looking behavioral models regarding three key aspects. (1) Because agents explore the likely consequences and learn from outcomes that actually occurred rather than those which are intended to occur but may only be obtained with a certain probability, reinforcement learning does not need to assume that the consequences are intended. (2) Decisions are guided by rewards or punishments that lead to approach or avoidance rather than by static utilities. (3) Learning is characterized by stepwise melioration and models the dynamic search for an optimum rather than assuming that the optimal strategy can be determined right away.

The learning process is modeled via a learning algorithm (e.g., Q-learning, SARSA learning, actor-critic learning) based on iteratively evaluating the current value of the environmental state utilizing a temporal difference error of expected value and experience value . Artificial neural network algorithms can explore very high dimensional state and action spaces. Genetic algorithms, which are inspired by evolutionary mechanisms such as mutation and selection, are also applied to learning problems. The learning algorithm has to balance a trade-off between the exploration of actions with unknown consequences and the exploitation of current knowledge. In order to not exploit only the currently learned strategy, many algorithms use randomness to induce deviations from already learned behavior.

The environment in reinforcement learning problems is often modeled with Markovian transition probabilities. The special case of a single agent is called a Markov decision process . In each of the discrete states of the environment the agent can choose from a set of possible actions. The choice then influences the transition probabilities to the next state and the reward. As an illustration, consider a farmer adapting her planting and irrigation practices to new climatic conditions. The environment could be modeled by a Markov process with different states of soil fertility and moisture, in which transitions between states reflect the influence of stochastic weather events. Without the possibility to acquire knowledge through other channels, she would explore different possible actions and evaluate how they change the yield (her reward). Eventually, through a trial-and-error process, her yield would increase on average.

A common approach to model the acquisition of subjective probabilities associated with the consequences of actions is Bayesian learning, which has also been applied to reinforcement learning problems . Starting with some prior probability (e.g., from some high-entropy “uninformative” distribution) $P\left(h_i\right)$ that some hypothesis $h_i$ about the relation of actions and outcomes is true, new information or evidence $P\left(E\right)$ is used to update the subjective probability with the posterior $P\left(E\right|h_i)$ calculated with Bayes' theorem: $P\left(h_i\right|E)=P(E\left|h_i\right)P\left(h_i\right)/P\left(E\right)$ . The most probable hypothesis can then be chosen to determine further action.

By combining various approaches to model the acquisition of beliefs through learning, the formation of preferences and different decision rules discussed in the previous sections with further insights from psychology and neuroscience has led to the development of very diverse and detailed behavioral theories which are often formalized in cognitive architectures . These approaches can be used to describe human behavior in computational models, but are too complex and diverse to discuss them here in detail.

Learning and related theories that emphasize the adaptability of human behavior might be important building blocks to model the long-term evolution of human interactions with the Earth system from an individual perspective. On the other hand, they can capture short-term responses to drastically changing natural environments that are relevant, for instance, in the context of tipping elements in the Earth system.

Table  summarizes the approaches that focus on individual human behavior. Besides the forward- and backward-looking behavior that we introduced in this section, agents may exhibit sideways-looking behavior: agents can copy the behavior of successful others, thereby contributing to a social learning process. For this kind of behavior, interactions between different agents are crucial. This will be the focus of the next section.

Summary table for individual behavior and decision making.

Theories	Key considerations	Strengths	Limitations
Optimal decisions in rational choice: individuals make the decision that maximizes their expected utility given economic, social, and environmental constraints.	What are the agent's preferences? What information (and beliefs) do they have?	Highly researched theory with strong theoretical foundation and many applications	Individuals assumed to have strong capabilities for information processing and perfect self-control
Bounded rationality and heuristic decision making: individuals have biases and heuristic decision rules that help them navigate complex environments effectively.	Which cue order is used to gather and evaluate information? When do agents stop gathering more information and decide?	Simple decision processes that capture observed biases in decision making	Suitable decision rules highly context dependent
Learning: agents explore possible actions through repeated learning from past experience.	How do agents interact with their environment? What is the trade-off between exploitation of knowledge and exploration of new options?	Captures information and belief acquisition processes	High degree of randomness in behavioral changes

Modeling interactions between agents

In the previous section, we discussed modeling approaches that focus on the choices of individuals that are confronted with a decision in a specified situation. In contrast, this section reviews techniques to model how actors interact with each other and influence or respond to each other's decisions. Interactions at the system level that are also aggregation mechanisms (e.g., voting procedures and markets) will be discussed in Sect. .

The section starts with a review of strategic interactions as modeled in classical game theory and dynamic interactions in evolutionary approaches. Then, we address models of social influence that are used to study opinion and preference formation or the transmission of cultural traits, i.e., culturally significant behaviors. Finally, we discuss how interaction structures can be modeled as dynamic networks.

Strategic interactions between rational agents: classical game theory

Game theory focuses on decision problems of “strategic interdependence”, in which the utility that a decision maker (called the player) gets depends not only on her own decision, but also on the choices of others. These are often situations of conflict or cooperation. Players choose an action (behavioral option, control) based on a strategy, i.e., a rule specifying which action to take in a given situation. Classical game theory explores how rational actors identify strategies, usually assuming the rationality of other players. However, rational players can also base their choices on beliefs about others players' decisions, which can lead to an infinite regress of mutual beliefs about each other's decisions.

Formally, a game is described by what game theorists call a game form or mechanism. The game form specifies the actions $a_i\left(t\right)$ that agents can choose at well-defined time points $t$ from an action set $A_i\left(t\right)$ that may vary over time, having to respect all kinds of situation-dependent rules. The game form may furthermore allow for communication with the other agent(s) (signaling) or binding agreements (commitment power). Simple social situations are formalized in so-called normal-form games represented by a payoff matrix specifying the individual utilities

Note that despite the term “payoff matrix”, these utilities are unexplained attributes of the agents and need not have a relation to monetary quantities.

for all possible action combinations, while more complex situations are modeled as a stepwise movement through the nodes of a decision tree or game tree .

Classical game theory assumes that players form consistent beliefs about each other's unobservable strategies, in particular that the other's behavior results from an optimal strategy. However, multiplayer interaction and optimization often leads to recursive relationships between beliefs and strategies, which makes solving complex classical games often very difficult. Many problems have several solutions, called equilibria (not to be confused with the steady-state meaning of the word), and call for sophisticated nonlinear fixed-point solvers . Only in special cases, for example in which players have complete information and moves are not simultaneous but alternating, game-theoretic equilibria can easily be predicted by simple solution concepts such as backwards induction . In other cases, one can identify strategies and belief combinations consistent with the following two assumptions. First, each player eventually chooses a strategy that is optimal given her beliefs about all other players' strategies (rational behavior). Second, each player's eventual beliefs about other players' strategies are correct (rational expectations). The solutions are called Nash equilibria. However, many games have multiple Nash equilibria, and the question of which equilibrium will be selected arises.

Therefore, game theorists try to narrow down the likely strategy combinations by assuming additional forms of consistency and rationality , such as consistency over time (sequential and subgame perfect equilibria), stability against small deviations stable equilibria,, or small random mistakes trembling hand perfect equilibria,. After a plausible strategic equilibrium has been identified, it can be used in a simulation of the actual behavior resulting from these strategies over time, possibly including noise and mistakes.

As an example from the land-use context, consider two farmers living on the same road. They get their irrigation water from the same stream. A dispute over the use of water emerges. Both may react to the actions of the other in several turns. The upstream farmer located at the end of the road may increase or decrease her water use and/or pay compensation for using too much water to the other. The downstream farmer at the entrance of the road may demand compensation or block the road and thereby cut the access of the upstream farmer to other supplies. A complex game tree encodes which actions are feasible at which moment and what are the consequences on players' utilities. If it is possible to specify the information and options available to the players at each time point, then a classical game theoretical analysis allows for the determination of the rational equilibrium strategies that the farmers would follow.

Classical game theory is widely applied to interactions in market settings in economics (see also Sect. ), but increasingly also in the social and political sciences to political and voting behavior in public and social choice theory see, e.g.,and Sect. . For example, public choice theory studies strategic interactions between groups of politicians, bureaucrats, and voters with potentially completely different preferences and action sets.

While many simple models of strategic interactions between rational and selfish agents will predict only low levels of cooperation, more complex models can well explain how bilateral and multilateral cooperation, consensus, and stable social structure emerges . This has been shown in contexts such as multiplayer public goods problems and international climate policy e.g.,.

To model relevant decision processes in the Earth system, classical game-theoretic analysis could be used for describing strategic interactions between agents that could be assumed as highly rational and well informed, i.e., international negotiations of climate agreements between governments, bargaining between social partners, or monopolistic competition between firms. Similarly, international negotiations and their interactions with domestic policy can also be framed as two-level or multilevel games as in some models of political science, e.g.,. Furthermore, social choice theory could be used to simulate simple voting procedures that (to a certain extent) determine the goals of regional or national governments.

Interactions with dynamic strategies: evolutionary approaches and learning in game theory

In game-theoretic settings, complex individual behavioral rules are typically modeled as strategies specifying an action for each node in the game tree. Consider as an example the repeated version of the prisoners' dilemma in which each of two players can either “cooperate” or “defect” in each period . A typical complex strategy in this game could involve reciprocity (defect temporarily after a defection of your opponent), forgiveness (every so often not reciprocate), and making up (do not defect again after being punished by a defection of your opponent after your own defection).

Many or even most nodes of a game tree will not be visited in the eventual realization of the game, and strategies may involve the deliberate randomization of actions. Therefore, strategies, unlike actual behavior, are principally unobservable, and assumptions about them are hard to validate. For this and other reasons, several kinds of additional assumptions are often made that constrain the set of strategies further that a player can choose, e.g., assuming only very short memory or low farsightedness (myopic behavior) and disallowing randomization, or allowing only strategies of a specific formal structure such as heuristics (see Sect. ).

The water conflict example from Sect.  bears some similarity to the repeated prisoners' dilemma in that the farmers' possible actions can be interpreted as either defective (using too much water, blocking the road) or cooperative (not doing any of this, compensating for past defections). Assuming different levels of farsightedness may thus lead to radically different actions because myopic players would much more likely get trapped in a cycle of alternating defections than farsighted players. The latter would recognize some degree of forgiveness because that maximizes long-term payoff and would thus desist from defection with some probability. In any case, both farmers' choices can be modeled as depending on what they believe the other will likely do or how she will react to the last action.

Evolutionary approaches in game theory study the interaction of different strategies and analyze which strategies prevail on a population level as a result of selection mechanisms. Thus, in contrast to classical game theory, evolutionary approaches focus on the dynamics of strategy selection in populations. The agent's strategies may be hardwired, acquired, or adapted by learning . Although many evolutionary techniques in game theory are used in biology to study biological evolution (variation through mutation, selection by fitness, and reproduction with inheritance), evolutionary game theory can be used to study all kinds of strategy changes in game-theoretic settings, for instance cultural evolution (transmission of memes), social learning through the imitation of successful strategies, or the emergence of cooperation .

In an evolutionary game, a population of agents is divided into factions with different strategies. They interact in a formal game (given by a payoff matrix or game tree, see Sect. ), in which their strategy results in a fitness (or payoff). The factions change according to some replicator rules that depend on the acquired fitness. This can be modeled using different techniques. Simple evolutionary games in well-mixed large populations can be described with replicator equations. The dynamics describing the relative change in the factions with a particular strategy is proportional to the deviation of the fitness of this faction from the average fitness .

Alternatively, the behavior resulting from evolutionary interactions is often easy to simulate numerically as a discrete-time dynamical system even for large numbers of players if the individual action sets are finite or low-dimensional and only certain simple types of strategies are considered. This type of agent-based model (see Sect. ) simply implements features such as mutation or experimentation and replication via strategy transfer (e.g., imitation and inheritance) at the microlevel. Combined with network approaches (see Sect. ), the influence of interaction structure can also be studied . Strategies can be characterized as evolutionary stable if a population with this strategy cannot be invaded by another, initially rare strategy. If a strategy is furthermore stable for finite populations or noisy dynamics, it is called stochastically stable.

In our water conflict example, the farmers could use a heuristic strategy (see Sect. ) that determines how much water they extract given the actions of the other. The evolution of the strategies could either be modeled with a learning algorithm, repeating the game again and again. Alternatively, to determine feasible strategies in an evolutionary setting, a meta-model could consider an ensemble of similar villages consisting of two farmers. The strategies of the farmers would then be the result of either an imitation process between the villages or of an evolutionary process, assuming that less successful villages die out over time.

Evolutionary approaches to game theory are a promising framework to better understand the prevalence of certain human behaviors regarding interaction with the Earth system. This is especially interesting regarding the modeling of long-term cultural evolution and changes in individuals' goals, beliefs, and decision strategies or the transmission of endogenous preferences .

Modeling social influence

Human behavior and its determinants (beliefs, goals, and preferences) are strongly shaped by social influence, which can result from various cognitive processes. Individuals may be convinced by persuasive arguments , aim to be similar to esteemed others , be unsure about what is the best behavior in a given situation , or perceive social pressure to conform with others .

Models of social influence allow for the study of the outcomes of repeated influence in social networks and have been used to explain the formation of consensus, the development of monoculture, the emergence of clustered opinion distributions, and the emergence of opinion polarization, for instance. Models of social influence are very general and can be applied to any setting in which individuals exert some form of influence on each other. However, seemingly innocent differences in the formal implementation of social influence can have decisive effects on the model outcomes, as the following list of important modeling decisions documents.

A first question is how social influence changes individual attributes. For example, a farmer deciding when to till his field might either choose the date that most of his neighbors think is best, take the average of the proposed dates, or even try to counter coordinate with disliked farmers. Classical models incorporate influence as averaging, which implies that interacting individuals always grow more similar over time . Averaging is an accepted and empirically supported model of influence resulting, for instance, from the social pressure that an actor exerts on someone else . Models assume different forms of averaging. Rather than following the arithmetic average of all opinions, actors might only consider the majority view . In other models, social influence can lead to polarization . For instance, in models of argument communication, actor opinions can turn more extreme when the interaction partners provide them with new arguments that support their own opinion .

Second, modelers need to decide whether there is just one or multiple dimensions of influence. For instance, it is often argued that political opinions are multidimensional and cannot be captured by the one-dimensional left–right spectrum. Explaining the dynamics of opinion polarization and clustering is often more difficult when multiple dimensions are taken into account . Additionally, model predictions often depend on whether the influence dimension is a discrete or a continuous variable. Models of individuals' decisions about certain policies often model the decisions as binary choices . However, binary scales fail to capture the fact that many opinions vary on a continuous scale and that differences between individuals can therefore also increase in a single dimension . Therefore, models that describe opinion polarization usually treat opinions as continuous attributes.

A third critical question is how the interaction process is modeled. In models of opinion dynamics, for example, influence is bidirectional in that an actor who exerts influence on someone else can also be influenced by the other . In diffusion models, in contrast, the effective influence is directed. For instance, information can spread only from informed to uninformed individuals, but not the other way around. Furthermore, actors may be influenced dyadically or multilaterally. Model outcomes often depend on whether the influence that a group exerts on an actor is modeled as a sequence of events involving dyads of actors or as a single opinion update in which the actor considers all contacts' influences at once . In models that assume binary influence dimensions, for instance, dyadic influence implies that an agent copies a trait from her interaction partner. When influence is multilateral, agents aggregate the influence exerted by multiple interaction partners (using, e.g., the mode of the neighbors' opinions), which can imply that agents with rare traits are not considered even though they would have an influence in the case of dyadic influence events. For example, a farmer seeking advice on whether to adopt a new technology can either consult his friends one after another or all together, likely leading to different outcomes if they have different opinions on the matter.

Fourth, agents may slightly deviate from the influence of their contacts. The exact type of these deviations affects model outcomes and can introduce a source of diversity into models of social influence . For instance, some models of continuous opinion dynamics include deviations as Gaussian noise, i.e., random values drawn from a normal distribution. In such a model, opinions in homogeneous subgroups will fluctuate randomly and subgroups with similar opinions can merge that would have remained split in a model without deviations . In other contexts, deviations are better modeled by uniformly distributed noise, assuming that big deviations are as likely as small ones. This can help to explain, for instance, the emergence and stability of subgroups with different opinions that do not emerge in settings with Gaussian noise

Gaussian noise needs to be very strong to generate enough diversity for the emergence of subgroups with different opinions. However, when noise is strong, subgroups will not be stable.

.

Finally, the effects of social influence depend on the structure of the network that determines who influences whom. Complex dynamics can arise when this interaction network is dynamic and depends on the attributes of the agents, as we discuss in the following section.

Models of social influence are a promising approach to explore how social transitions interact with the Earth system, for example transitions of norms regarding admissible resource use and emissions, lifestyle changes, and adoption of new technology. They can be used to explore the conditions under which social learning enables groups of agents to adopt sustainable management practices.

Modeling the interaction structure: (adaptive) network approaches

In most of the models discussed in the previous section, the social network is formally modeled as a graph (the mathematical notion for a network): a collection of nodes that are connected by links. In this mathematical framework, nodes (vertices) represent agents and links (edges) indicate interaction, communication, or a social relationship. Agents can only interact and thus influence each other if they are connected by a link in the underlying network.

Classical social influence models study the dynamics of influence on static networks, assuming that agents are always affected by the same subset of interaction partners e.g.,. These networks can be undirected or directed, possibly restricting the direction of influence, but their structure does not change over time. Furthermore, the topology of the network, i.e., the arrangement of links, can be more or less random or regular, clustered, and hierarchical. In social influence models on static networks, connected populations will usually reach consensus in the long run.

Especially when modeling social processes over longer timescales, it is reasonable to assume that the social network is dynamic, i.e., that its structure evolves over time. This time evolution can be independent of the dynamics on the network and encoded in a temporal network . However, for many social processes, the structure of the social network and the dynamics on the network (e.g., social influence) interact. Adaptive network models make the removal of existing and the formation of new links between agents dependent on attributes of the agents by building on the insight that the social structure influences the behavior, opinions, or beliefs of individual actors, which in turn drives changes in social structure .

Local update rules for the social network structure and the agent behavior can be chosen very flexibly. Changes in agent behaviors may be governed by rules such as random or boundedly rational imitation of the behavior of network neighbors (see above). Update rules for the network structure are often based on the insight that agents tend to be influenced by similar others and ignore those who hold too-distant views . Many models assume that agents with similar characteristics tend to form new links between each other (homophily) while breaking links with agents having diverging characteristics . In adaptive network models, homophily in combination with social influence generates a positive feedback loop: influence increases similarity, which leads to more influence and so on. Such models can explain, for instance, the emergence and stability of multiple internally homogeneous but mutually different subgroups. Other applications of coevolutionary network models allow us to understand the presence of social tipping points in opinion formation , epidemic spreading , the emergence of cooperation in social dilemmas , and the interdependence of coalition formation with social networks . Such adaptive network models exhibit complex and nonlinear dynamics such as phase transitions , multi-stability , oscillations in both agent states and network structure , and structural changes in network properties .

While adaptive networks have so far mostly been applied to networks of agents representing individuals, the framework can in principle be used to model coevolutionary dynamics on various levels of social interaction as introduced in Table . For instance, global complex network structures such as financial risk networks between banks, trade networks between countries, transportation networks between cities and other communication, organizational, and infrastructure networks can be modeled . Furthermore, approaches such as multi-layer and hierarchical networks or networks of networks allow for the modeling of the interactions between different levels of a system .

As an illustration, consider a community of agents each harvesting a renewable resource, for example wood from a forest. The agents interact on a social network, imitating the harvesting effort of neighbors that harvest more and may drop links to neighbors that use another effort. The interaction of the resource dynamics with the network dynamics either leads to a convergence of harvest efforts or a segregation of the community into groups with higher or lower effort depending on the model parameters .

In the context of long timescales in the Earth system, the time evolution of social structures that determine interactions with the environment are particularly important. Adaptive networks offer a promising approach to modeling the structural change of the internal connectivity of a complex system . For example, this could be applied to explore mechanisms behind transitions between centralized and decentralized infrastructure and organizational networks.

Table  summarizes the different modeling approaches that focus on agent interactions in human decision making and behavior. These interactions occur between two or several agents. For including the effect of these interactions into ESMs, their aggregate effects need to be taken into account as well. Therefore, we introduce in the next section approaches that allow us to aggregate individual behavior and local interactions and to study the resulting macrolevel dynamics.

Summary table for agent interactions.

Approaches and frameworks	Key considerations	Strengths	Limitations
Classical game theory: strategic interactions between rational agents	What is the game structure (options, possible outcomes, timing, information flow) and what are the players' preferences?	Elegant solutions for low-complexity problems	Difficult to solve for complex games, agents cannot change the rules of the game
Evolutionary game theory: competition and selection between hardwired strategies	What competition and selection mechanisms are there?	Can explain how dominant strategies come about	Agent strategies are modeled as hardwired (no conscious strategy change)
Social influence: agents influence each other's beliefs, preferences, or behaviors	How do influence mechanisms change agent attributes? Is the influence multilateral, dyadic, or directed? How large are deviations?	Allows for the modeling of social learning, preference formation, and herding behavior	Local dynamics are often stylized
Network theory: changing social interaction structures	Is the social network static or adaptive? How much randomness and hierarchy is in the structure? How do agents form new links?	Mathematical formalization to model coevolution of social structure with agent attributes	Micro-interactions mostly dyadic and schematic

Aggregating behavior and decision making and modeling dynamics at the system level

So far, we focused on theories and modeling techniques that describe the decision processes and behavior of single actors, their interactions, and the interaction structure. This section builds on the previously discussed approaches and highlights different aggregation methods for the behavior of an ensemble or group of agents. This is an important step if models shall describe system-level outcomes or collective decision making and behavior in the context of Earth system modeling. Aggregation techniques link modeling assumptions at one level (often called the microlevel) to a higher level (the macrolevel). They enable the analysis of macrolevel outcomes and help to transfer models from one scale to another. In general, this could link all levels introduced in Sect. .

In this section, we describe different approaches that are used to make this connection. Analytical approaches generally represent groups of individual agents through some macrolevel or average characteristic, often using simplifying assumptions regarding the range of individual agents' characteristics. Simulation approaches describe individual behavior and interactions and then compute the resulting aggregate macroscopic dynamics.

The question of how to aggregate micro-processes to macro-phenomena is not specific to modeling human decision making and behavior. The aggregation of individual behavior and the resulting description of collective action, such as collective motion, is also an ongoing challenge in the natural sciences . Specific assumptions about individual behavior and agent interactions have consequences for the degree of complexity of the macrolevel description. For instance, if agent goals and means do not interact, the properties of single agents can often be added up. If, on the contrary, agents influence each other's goals or interact via the environment, complex aggregate dynamics can arise.

The following sections discuss different aggregation techniques, their underlying assumptions, and how these reflect specific aggregation mechanisms. They are summarized in Table .

Summary table for aggregation and system-level descriptions.

Approaches and frameworks	Key considerations	Strengths	Limitations
Social utility and welfare: aggregate individual utility, possibly taking inequalities into account	How is inequality evaluated? How is welfare compared between societies and generations?	Basis for cost–benefit analysis, a widely applied decision model for policy evaluation	Assumes that individual utility can be compared on a common scale
Aggregation via markets: representative agents in economic models	What goals or preferences do representative agents have? How efficient do market mechanisms allocate on which spatial and temporal scales? What market imperfections are there?	Well-developed formalism that makes the connection between microeconomics and macroeconomics analytically traceable	Assumes that aggregated agent properties are similar to individual ones to derive economic equilibrium, coordination effort between agents neglected
Social planner and economic policy in integrated assessment models: model ways to internalize environmental externalities	Which economic policy instruments internalize environmental externalities best? What are plausible scenarios for policy implementation? How do agents react to changes in policy?	Allows for the determination of optimal paths for reaching societal goals	Models focus on production and investment in the economy
Distributions and moments: model heterogeneous agent attributes via statistical properties of distributions	Which heterogeneities are most important for the macro-outcome?	Systematic way to analytically treat heterogeneities	Only applicable for rather simple behaviors and interactions
Agent-based models: simulate agent behavior and interactions explicitly to study emergent macro-dynamics computationally	What kind of agent types are important? How do they make decisions? How do the agents interact with each other and the environment?	Very flexible framework regarding assumptions about decision rules and interactions	Models often with many unknown parameters, difficult to analyze mathematically
Dynamics at the system level	Which crucial parameters in the model can be influenced by decision makers?	Allows for the exploration of possible dynamical properties of the system based on macro-mechanisms	No explicit micro-foundation

Aggregation of preferences: social welfare and voting

Rational choice approaches can also be used to model decision making by agents on higher levels from Table , for example firms or countries. The “preferences” of such groups of individuals are often represented by using as the optimization target a social welfare function, which aggregates the members' utility functions either additively (“utilitarian” welfare) or in some nonlinear way to represent inequality aversion e.g., the Gini–Sen, Atkinson–Theil–Foster, or egalitarian welfare functions;. To do so, a common scale of utility must be assumed. For example, individual utility in many economic models equals the logarithm of the total monetary value of the individual's consumption. Social welfare functions are indeed used to find optimal policy, for example in cost–benefit analysis . Consider a village of farmers growing crops that need different amounts of water so that water management policies affect farmer incomes. The effects of a water policy could then be evaluated using the average, minimal, or average logarithmic income of farmers as a measure of social welfare. The policy option maximizing the chosen indicator should be implemented.

However, it is highly debated whether the utilities of different individuals can really be compared and substituted in the sense that a drop in collective welfare resulting from an actor's decrease in utility can be compensated for by increasing the utility of another actor. Defining suitable group preferences is especially hard when group composition or size changes over time as in intergenerational models . Also, in complex organizations, real decisions might be nonoptimal for the group and more explicit models of actual decision procedures may be needed. Models in subfields of game theory (bargaining, voting, or social choice theory) explore the outcomes of formal protocols that are designed to aggregate the group member's heterogeneous preferences. Under different voting or bargaining protocols, subgroups may dominate the decision or the group may be able to reach a compromise . In the above example, the farmers may not agree on a social welfare measure that a policy should optimize but instead on a formal protocol that would allow them to determine a policy for water usage that is acceptable for all.

Aggregation via markets: economic models and representative agents

A major part of the relevant interaction of contemporary societies with the Earth system is related to the organization of production and consumption on markets. Markets not only mediate between the spheres of production and consumption, but they can also be seen as a mechanism to aggregate agents' decisions and behavior. Economic theory explores how goods and services are allocated and distributed among the various activities (sectors of production) and agents (firms, households, governments) in an economy. Goods and services may be consumed or can be the input factors to economic production. Input factors for production are usually labor and physical capital but can also include financial capital, land, energy, natural resources, and intermediate goods. In markets, the coordination between the demand and supply of goods is mediated through prices that are assumed to reflect information about the scarcity and production costs of goods. Economics compares different kinds of market settings (e.g., auctions, stock exchanges, international trade) with respect to different criteria such as allocative efficiency.

Building on rational choice theory for modeling the decisions of individual agents, microeconomic models in the tradition of neoclassical economics analyze the conditions for an equilibrium between supply and demand on single markets (partial equilibrium theory) and between all markets (general equilibrium theory). The behavior of households and firms is usually modeled as utility maximization under budget constraints and profit maximization under technological constraints in production, respectively. A central assumption is that an economy is characterized by decreasing marginal utility and diminishing returns: the additional individual utility derived from the consumption of one additional unit of some good is declining. Similarly, the additional production derived from an additional unit of a single input factor is declining with its absolute amount when holding other input factors fixed. Accordingly, the output of the production process is described as a production function, which is concave in its input factor arguments.

Assuming that there is perfect competition between producers, resources and goods are allocated in a Pareto efficient way so that no further redistribution is possible that benefits somebody without making somebody else worse off . It has been shown that this leads to the emergence of an equilibrium price for each good as the market is cleared and supply meets demand . The idea of this market equilibrium can be understood by the associated prices. The rational market participants trade goods as long as there is somebody who is willing to offer some good at a lower price than somebody else is willing to pay for it. However, in markets dominated by a few or very heterogeneous agents, perfect competition cannot be assumed, and price wars, hoarding, and cartel formation can occur. Such situations can be described in models of oligopoly, bargaining, or monopolistic competition but are sometimes difficult to integrate into macroeconomic frameworks.

Macroeconomic models build on this microeconomic theory by modeling the decision making of firms and households with the representative agent approach. A representative agent stands for an ensemble of agents or an average agent of a population. An underlying assumption is that heterogeneities and local interactions cancel out for large numbers of agents. While representative firms model the supply of different sectors, the demand is determined by one or several representative households. Representative firms and households are assumed to act as if there were perfect competition and they had no market power, i.e., that they optimize their production or consumption taking the prices of goods and production factors as given. The prices of production factors are assumed to equal the value of what they are able to produce additionally by using one additional unit, i.e., their marginal product. In simple macroeconomic models, representative agents interact on perfect markets for all production factors and goods. The solution of the associated optimization problem (with constraints given by a system of nonlinear algebraic equations) specifies the quantity and allocation of input factors, their prices (wages and interest rates), and the production and allocation of consumer goods. A change in one constraint can therefore lead to adjustments in all sectors and new equilibrium prices. For example, in an economy with only two sectors, industry and agriculture, modeled by two representative firms and a representative household, increases in agricultural productivity may lead to the reallocation of labor into the industrial sector and changes in wages.

In reality, prices can undergo rapid fluctuations, which challenges the validity of equilibrium assumptions at least in the short run. Furthermore, production factors may not be fully employed as general equilibrium considerations suggest. Other deviations from efficient equilibria are discussed as market imperfections such as transaction costs, asymmetries in available information, and noncompetitive market structures. Dynamic stochastic general equilibrium (DSGE) models account for the consumption and investment decisions of economic agents under uncertainty and explore the consequences of stochastic shocks on public information or technology for macroeconomic indicators. Many modern DSGE models also incorporate short-term market frictions such as barriers to nominal price adjustments (“sticky” prices) or other market imperfections . However, these models still build on the key concept of general equilibrium because they assume that the state of the economy is always near such an equilibrium and market clearance is fast.

Economic growth models are used to study the long-term dynamics of production and consumption and are therefore an important approach for Earth system modeling. In simple growth models, a homogeneous product is produced per time according to an aggregate production function. A part of the output can be saved as new capital, while the remaining output is consumed. The evolution of the capital stock is given by a differential equation taking into account investments and capital depreciation. In the standard neoclassical growth model, the savings are endogenously determined by the inter-temporal optimization of a representative household and equal investments. The household maximizes an exponentially discounted utility stream (compare Sect. ), which is a function of consumption . The central decision of the representative household is how much of the produced output it saves to increase production in the future and therefore cannot consume and enjoy directly. Such inter-temporal optimization problems can be solved either computationally by discretization in time or analytically by applying techniques from optimal control theory

Optimal control theory deals with finding an optimal choice for some control variables (often called policy) of a dynamical system that optimizes a certain objective function using, for example, variational calculus .

. Besides population growth, the only long-term drivers of growth in the standard neoclassical model are exogenously modeled increases in productivity through technological change. In contrast, so-called endogenous growth models exhibit long-run growth and endogenously account for increases in productivity, for example through innovation, human capital, or knowledge accumulation .

The use of representative agents in macroeconomic models has implications that stem from the implicit assumption that the representative agent has the same properties as an individual of the underlying group . First, the approach neglects the fact that single agents in the represented group have to coordinate themselves, leaving out problems that arise due to incomplete and asymmetric information. Second, a group of individual maximizers does not necessarily imply collective maximization, challenging the equivalence of the equilibrium outcome. Finally, the representative agent approach may neglect emergent phenomena from heterogeneous micro-interactions .

In spite of the deficiencies of the representative agent approach, its application to markets allows for the aggregation of behavior in simple and analytically tractable forms. Modelers who wish to describe economic dynamics at an aggregate level can rely on a well-developed theory that describes many economic phenomena in a good approximation. In the following section, we will discuss how this approach is used to analyze the impacts of economic activities on the environment.

Modeling of decisions in integrated assessment models: social planner and economic policy

Integrated assessment models (IAMs) comprise a large modeling family that combine economic with environmental dynamics. However, the majority of currently used IAMs draws on ideas from environmental economics. Using the concept of environmental externality, they evaluate the extraction of exhaustible resources, environmental pollution, and overexploitation of ecosystems economically. Externalities are benefits from or damages to the environment that are not reflected in prices and affect other agents in the economy see, e.g.,. These models therefore help to assess economic policies that tackle environmental problems.

State-of-the-art global IAMs combine macroeconomic representations of sectors like the energy and land systems with models of the biophysical bases and environmental impacts of these sectors. For example, CO${}_{\mathrm{2}}$ emitted from burning fossil fuels is linked to economic production by carbon intensities and energy efficiencies in different production technologies. IAMs often model technological change endogenously, for example with investments in R&D or learning by doing (i.e., decreasing costs with increasing utilization of a technology). Because of the possibility to induce technological change, the models capture the path dependencies of investment decisions. Many IAMs take the perspective of a social planner who makes decisions on behalf of society by optimizing a social welfare function (see Sect. ). It is assumed that the social optimum equals the perfect market outcome with economic regulations that internalize all external effects (e.g., emission trading schemes).

This argument is based on the second fundamental theorem of welfare economics; see, for example, 63–70.

IAMs are mostly computational general or partial equilibrium models describing market clearing between all sectors or using exogenous projections of macroeconomic variables (see Sect. ). They also differ with respect to inter-temporal allocation. While inter-temporal optimization models use discounted social welfare functions to allocate investments and consumption optimally over time, recursive dynamic models solve an equilibrium for every time step . Furthermore, IAMs are either designed for (1) determining the optimal environmental outcomes of a policy by making a complete welfare analysis between different policy options or (2) evaluating different paths to reach a political target with respect to their cost effectiveness . In the context of climate change, for example, many IAMs have emission targets as constraints in their optimization procedure and determine the best way to reach them .

For the analysis of global land use, IAMs combine geographical and economic modeling frameworks . These models are used, for example, to investigate the competition between different land uses and trade-offs between agricultural expansion and intensification. With the optimization, land uses are instantaneously and globally allocated and only constrained by environmental factors such as soil quality, water availability, and climate and protection policies.

IAMs differ from ESMs not only regarding their modeling technique (mostly optimization) but also regarding their purpose: they help policy advisors to assess normative paths that the economy could take to reach environmental policy goals. While the decision about the policy is exogenous to the model, the investment decisions within and between sectors are modeled as a reaction to the political constraints. However, most IAMs do not account for possible changes on the demand side, for example through changes in consumer preferences for green products. A better cooperation between the IAM and ESM communities, as called for by in this Special Issue, is certainly desirable because some of the problems that arise when including human decision making into ESMs have already been dealt with in IAMs. However, when considering the coupling of IAMs and ESMs with different methods , modelers have to keep in mind not only technical compatibility (e.g., regarding the treatment of time in inter-temporal optimization models) but also the possibly conflicting modeling purposes.

Modeling agent heterogeneity via distributions and moments

As discussed in Sect. , the representative agent approach can hardly capture heterogeneity in human behavior and interaction. In this section we describe analytical techniques that allow for the representation of at least some forms of agent heterogeneity.

An ensemble of similar agents can be modeled via statistical distributions if the agents are heterogeneous regarding only some quantitative characteristics, for example parameters in utility functions or endowments such as income and wealth. In simple models, techniques from statistical physics and theoretical ecology can be used to derive a macro-description from micro-decision processes and interactions. For instance, the distribution of agent properties representing an ensemble of agents can be described via a small number of statistics such as mean, variance, and other moments or cumulants. The dynamics in the form of the difference or differential equations of such statistical parameters can be derived by different kinds of approximations. A common technique is moment closure that expresses the dynamics of lower moments in terms of higher-order moments. At some order, the approximation is made by neglecting all higher-order moments or approximating them by using functions of lower-order ones see, e.g.,.

To aggregate simple interactions between single nodes in network models, similar techniques can be used to describe with differential equations how the occurrence of simple subgraphs (motifs) changes with the dynamics on and of the network. In network theory, these approaches are also called moment closure, although the closure refers here to neglecting more complicated subgraphs e.g.,. For example, the simple pair approximation only considers different subgraphs consisting of two vertices (agents) and one link. To abstract from the finite-size effects of fluctuations at the microlevel in stochastic modeling approaches and arrive at deterministic equations, analytical calculations often take the limit of the agent number going to infinity in statistical physics called the thermodynamic limit;.

Techniques based on moment closure and network approximations are used to aggregate the dynamics of processes like opinion formation on networks. This might be especially useful in reducing computational complexity when modeling social processes at intermediate levels of aggregation and could allow for the investigation of the interplay of mesoscale social processes with the natural dynamics of the Earth system.

Aggregation in agent-based models

Agent-based modeling is a computational approach to modeling the emergence of macrolevel or system-level outcomes from microlevel interactions between individual, autonomous agents and between agents and their social and/or biophysical environments . In agent-based models (ABMs), human behavior is not aggregated to the system level a priori, nor is it assumed that individual behavioral diversity can be represented by a single representative agent as in many macroeconomic models (see Sect. ). Instead, the behavior of heterogeneous agents or groups of agents is explicitly simulated to study the resulting aggregate outcomes. As each action of an individual agent is interdependent, i.e., it depends on the decisions or actions of other agents within structures such as networks or space, local interactions can give rise to complex, emergent patterns of aggregate behavior at the macrolevel . ABMs allow for the exploration of such nonlinear behavior in order to understand possible future developments of the system or assess possible unexpected outcomes of disturbances or policy interventions. Agent-based modeling is widely used to study complex systems in computational social science , land-use science , political science , computational economics , social–ecological systems research , and ecology , among others.

Note that in some scientific communities, this class of modeling approaches is also known as multi-agent simulation MAS; or individual-based modeling .

Agents in ABMs can be individuals, households, firms, or other collective actors, as well as other entities or groups thereof, such as fish, fish populations, or plant functional types. Agents are assumed to be diverse and heterogeneous; i.e., they can belong to different types and can vary within one type, respectively. Agent types can be characterized by different attributes and decision-making models (e.g., large and commercial versus small and traditional farms). Heterogeneity within a type is often represented through quantitative differences in the values of these attributes (e.g., regarding market access, social, or financial capital). The decision making and behavior of the agents can be modeled with any of the approaches introduced in Sect.  or can be based on data or observations that are formalized in equations, decision trees, or other formal rules. In empirical ABMs, agents are often classified into empirically based agent types, which are characterized by attributes and decision heuristics derived from empirical data obtained through interviews or surveys . Increasingly, social science theories of human behavior beyond the rational actor are being used in ABMs to represent more realistic human decision making. However, many challenges remain to translate these theories for usage in ABMs .

Probabilistic and stochastic processes are often used to capture uncertainty in and the impact of random events on human decision making and assess the consequences for macrolevel outcomes. For example, random events at the local level, such as a random encounter between two agents that results in a strategy change of one agent or a system-level environmental variation, can give rise to nonlinear macro-dynamics such as a sudden shift into a different system state .

In addition to the behavior of the agents, ABMs of human–environment systems incorporate the dynamics of the biophysical environment resulting from natural processes and human actions insofar as it is relevant for the agents' behavior and to understand feedbacks between human behavior and environmental processes. For example, in an ABM by , a number of cattle ranchers can move their livestock between grassland patches in a landscape. Overgrazing in one year decreases feed availability in the following year because of the underlying biomass regrowth dynamics. Agents decide how many cattle to graze on a particular land patch based on their individual goals or needs, information on the state of the grassland, beliefs about the future, and interactions with other ranchers. The model can reveal the interplay and success of different land-use strategies on common land and assess their vulnerability to shocks such as droughts. Most ABMs in the context of land-use science have so far been developed for local or regional study areas, taking into account local specificities and fitting behavioral patterns to data acquired in the field . They are often combined with cellular automaton models that describe the dynamics and state of the physical land system e.g.,. In these ABMs, the spatial embedding of agents usually plays an important role .

Because ABMs can integrate a diversity of individual decision making, heterogeneity of actors, and interactions between agents constrained by social networks or space and social and environmental processes, they are particularly suitable to study feedbacks between human action and biophysical processes. In the context of ESM these may include human adaptive responses to environmental change, such as the effects of climate change on agriculture and water availability, to policies such as bioenergy production or the global consequences of shifts in diets in particular regions. Agent-based modeling is also a useful tool to unravel the causal mechanisms underlying system-level phenomena and thus enhance the understanding of key human–environment interactions that may give rise to observed Earth system dynamics. However, because of their potentially high complexity and dimensionality in state and parameter space, ABMs are often difficult to analyze and may require high computational capacities and sophisticated model analysis techniques to understand their dynamics beyond single trajectories.

Agent-based approaches can be applied without modeling each individual agent explicitly. It suffices to model a representative statistical sample of agents that depicts the important heterogeneities of the underlying population. To capture major types of human behavior, a recent proposal involves agent functional types based on a theoretically derived typology of agent attributes, interactions, and roles . This proposal is explored for modeling the adaptation of land-use practices to climate change impacts . Agent functional types represent a typology that is theoretically constructed instead of data driven, which is common in empirically based ABMs. Agent-based approaches are promising for Earth system modeling because they allow modelers to address questions of interactions across levels, for instance how global patterns of land use emerge from interdependent regional and local land-use decisions, which are in turn constrained by the emerging global patterns. Furthermore, they would allow for the integration of uncertainty, agent heterogeneity, and the aggregation of detailed technological and environmental changes .

Dynamics at the system level: system dynamics, stock-flow consistent, and input–output models

This final subsection discusses modeling approaches without explicit micro-foundations. Decisions in such models are not modeled explicitly with one of the options discussed in Sect.  but, as policy decisions in integrated assessment models, through the construction of different scenarios for the evolution of crucial exogenous parameters in the model.

Global system dynamics models describe the economy, population, and crucial parts of the Earth system and their dynamic interactions at the level of aggregate dynamic variables, usually modeling the dynamics as ordinary differential equations or difference equations to project future developments. The equations are often built on stylized facts about the dynamics of the underlying subsystems and are linked by functions with typically many parameters. Modelers employ system dynamics models to develop scenarios based on different sets of model parameters and assess the system stability and transient dynamics. In comparison to equilibrium approaches, system dynamics models capture the inertia of socioeconomic systems at the cost of a higher dimensional parameter space. This can lead to more complex dynamics like oscillations or overshooting. System dynamics models can be very detailed, like the World3 model commissioned by the Club of Rome for their famous report “Limits to Growth” , the GUMBO model , or the International Futures model . Subsystems of such models comprise the human population (sometimes disaggregated between regions and age groups), the agricultural and industrial sector, and the state of the environment (pollution and resource availability). Simpler models describe the dynamics of only a few aggregated variables at the global level or confined to a region .

Other system-level approaches to macroeconomic modeling emphasize self-reinforcing processes in the economy and point at positive feedback mechanisms, resulting in multi-stability or even instability (e.g., increasing returns to scale in production and self-amplification of expectations during economic bubbles). For example, post-Keynesian economists use stock-flow consistent models to track the complete monetary flows in an economy in which low aggregate demand can lead to underutilization of production factors and the state plays an active role to stabilize the economy. In these models, a social accounting matrix provides a detailed framework of transactions (e.g., monetary flows) between households, firms, and the government, which hold stocks of assets and commodities .

Input–output models track flows to much more detail between different industries or sectors of production . Each industry or production process is modeled by a “Leontief” production function, which is characterized by fixed proportions of input factors that depend on the available technology. For example, an input–output model can describe which input factors, such as land, fertilizer, machinery, irrigation water, and labor, are required for satisfying the demand of an agricultural commodity with a mix of production techniques. The model would consider that some of these inputs have to be produced themselves using other types of inputs. Outputs also include unwanted side products, such as manure in cattle production. Such models are used, for instance, to explore how changes in demand would lead to higher-order effects along the supply chain. Regional input–output models also account for spatial heterogeneity and are used, for example, to evaluate the possible impacts of extreme climate events on the global supply chain .

While the approaches discussed above focus on the monetary dimension of capital and goods, models from ecological economics track material flows or integrate material with financial accounting. For example, input–output modeling has been extended to analyze industrial metabolism, i.e., material and energy flows and their environmental impacts in modern economies . Regionalized versions of such models can, for instance, be used to estimate the environmental footprint that industrialized countries have in other regions . In the emerging field of ecological macroeconomics seefor a detailed review of modeling approaches, stock-flow consistent and input–output models have been combined into one framework for tracking financial and material flows . Other ecological models use the flow–fund approach by or combine it with stock-flow consistent modeling approaches . While the flow concept refers to a stock per time, a fund is the potentiality of a system to provide a service. The important difference lies in the observation that a stock can be depleted or accumulated in one time step, while a fund can provide its service only once per time step. This distinction reflects physical constraints on the production process that have important consequences for modeling the social metabolism. and in this Special Issue provide an extreme view on the dynamics of social metabolism based only on thermodynamic considerations without taking human decision making or agency into account.

In order to make approaches that only consider the system level useful for modeling the impact of humans on the Earth system, they could be combined with approaches that model the development of new production technologies and how the deployment of new technologies is affected by decisions at different levels (consumers, firms, and governments). Even if this integration with decision models proves difficult, the approaches discussed in this section can help link social and environmental dynamics in new ways, providing an important methodology to include humans into ESMs.

Discussion

In the previous three sections, we showed that there is a diversity of approaches to model individual human decision making and behavior, to describe interactions between agents, and to aggregate these processes. The discussion of strengths and limitations of the modeling approaches showed possible underlying assumptions and connections to theories of human behavior. While some modeling techniques are compatible with many theories of human behavior or decision making and can thus be used with a variety of assumptions, other techniques significantly constrain possible assumptions.

For many relevant questions in global environmental change research, a dynamical representation of humans in ESMs may not be necessary. If behavioral patterns are not expected to change over the relevant timescales or feedbacks between natural and social dynamics are sufficiently weak, modelers can simply use conventional scenario approaches.

However, if behavioral patterns are expected to change over time and give rise to strong feedbacks with the environment, then an explicit representation of human decision making will provide new insights into the joint dynamics. In this case, modelers have to carefully choose which assumptions about human behavior and decision making are plausible for their specific modeling purpose. Modeling choices require a constant interplay between model development and the research questions that drive it.

Because there is no general theory of human decision making and behavior, especially not for social collectives, we cannot provide a specific recipe for including humans into ESMs. In Table , we summarize the approaches we discussed in this paper and collect important questions to guide the choice of appropriate model assumptions and approaches. To find the right assumptions for a specific context, modelers can further build on and consult existing social scientific research, even though ambiguities due to a fragmentation of the literature between opposing schools of thought and difficulties in generalizing single case studies from their local or cultural specificities can make some of the research difficult to access. In case of doubt, modelers can team up with social scientists to conduct empirical research in the specific context needed to select the appropriate approach. The selection of a modeling technique compatible with the chosen assumptions also has to consider its limitations for meaningfully answerable research questions and the analyses that it can provide. In the following, we discuss some important considerations regarding individual decision making, interactions, and aggregation.

Concerning individual agents, we identified three important determinants in decision models: motives, restrictions, and decision rules. Modelers need to take the many factors into account that influence which assumptions about each of these three determinants are applicable in a given context. For instance, modelers can make different assumptions about whether agents only consider financial incentives or also take into account other criteria, such as a desire for fair outcome distributions , depending on whether a situation is more or less competitive or cooperative. Research shows that the relevance of motives and goals can vary over time and that surprisingly subtle cues can change their importance . Likewise, the choice of a plausible decision rule depends on the studied context. For instance, a decision rule that requires complex computations may be relatively plausible in contexts in which agents make decisions with important consequences and in which they have the information and time needed to compare alternatives. When stakes are low and time to decide is limited, however, more simple decision rules are certainly more plausible. Cognitively demanding decision rules are also more plausible when decision makers are collectives, such as companies and governments. Sometimes, it may even be reasonable to assume that agents use combinations of different decision models .

Important criteria for choosing an appropriate model of agent interactions are the type and setting of interactions, the assumptions that agents make about each other, the influence they may exert on each other, and the structure of interactions. For example, interactions in competitive environments will only lead to cooperation if this is individually beneficial. In such environments, agents may assume that the others form their strategies rationally. In less competitive settings in which social norms and traditions play a crucial role, however, behavior may not be strategically chosen but rather adaptively, for example by imitating other agents. This might also be important on timescales at which cultural evolution happens. Furthermore, social settings might favor interactions in which agents primarily exchange opinions or share beliefs and influence each other's decisions in this way.

Crucial criteria for the choice of an appropriate aggregation technique for behavior and interactions are the properties of relevant economic and political institutions (e.g., market mechanisms or voting procedures), decision criteria for collective agents, heterogeneity of modeled agents, availability of data to evaluate the model, and relevant time and spatial scales of macro-descriptions. Depending on the specific research questions, modelers have to choose the aggregation method that fits the real-world systems of interest and describes their aggregation mechanisms and aggregate behavior reasonably. Whether the aggregate behavior of many agents is better represented by a representative agent as in macroeconomic models, a distribution of agent characteristics, or many diverse individuals as in ABMs depends on the importance of agent heterogeneity and interaction structures such as networks or spatial embeddedness. The choice of an aggregation technique then determines which characteristics and processes of the system are modeled explicitly and which assumptions influence the form of the model only implicitly.

If the local structure of interaction matters, this would require a gridded or networked approach; otherwise a mean field approximation is justified. Similar choices have to be made in classical ESMs. For example, the interaction of ocean and atmosphere temperature near the surface on a spatial grid could be modeled either by only taking interactions between neighboring grid points into account or by coupling the ocean temperature to the atmospheric mean field. Analogously, the interactions between groups of two types of agents may be modeled explicitly on a social network. However, it might also suffice to only consider interactions between two agents representing the mean of each group. The question of whether the interaction structure matters often cannot be answered a priori but may be the result of a comparison between an approximation and an explicit simulation.

For the choice of an appropriate aggregation technique, modelers also have to decide on the level of detail to describe the system and whether the modeling of individuals or intermediate levels of the system is necessary or an aggregate description suffices. This choice depends on the expected importance of interactions and heterogeneity in an assumed set of agents. As an example from classical Earth system modeling, consider vegetation models in which modelers choose between the simulation of representative plant functional types or ensembles of individual adaptive plants depending on whether they consider the interaction and heterogeneity important for the macro-dynamics. Analogously, a model of social dynamics may use a representative agent approach or model heterogeneous agents explicitly in an agent-based model depending on the research question. The choice between a detailed and aggregated description depends strongly on the model purpose. For example, if the goal is to predict the future development of a system, a system-level description could suffice, while a more detailed model (e.g., ABM) would be needed for understanding the mechanisms that explain these outcomes in terms of the underlying heterogeneous responses of individuals. Likewise, for a normative model aiming to identify the action that maximizes social welfare, an intermediate level of detail could suffice, taking only specific agent heterogeneities into account.

Collection of questions that may guide the choice of modeling approaches and assumptions.

Category	Important modeling questions
Modeling individual decision making and behavior	What goals do agents pursue? What constraints do they have? What decision rules do agents use? How do agents acquire information and beliefs about their environment?
Modeling interactions between agents	Do agents interact in a competitive environment, or are interactions primarily governed by social norms? What do agents assume about each other's rationality? Do agents choose actions strategically or adaptively? How are agents influenced by others regarding their beliefs and norms? What structure do the interactions have, and how does the structure evolve?
Aggregating behavior and modeling dynamics at the system level	Are decisions aggregated through political institutions (e.g., voting procedures) or markets? According to what criteria do policy makers decide, and what controls do they have? Is the heterogeneity of agent characteristics and interactions important? Which macrolevel measures are dynamic and which can be assumed to be fixed?

In general, the evaluation of timescales can help in many of the abovementioned modeling choices to decide whether the social processes and properties of socioeconomic units should be represented as evolving over time, can be fixed, or need not be modeled explicitly at all for a macrolevel description of the system. For example, CO${}_{\mathrm{2}}$ concentration in global circulation models can be assumed to be well mixed for the atmosphere, while assuming this for the ocean with its slow convection would considerably distort results on politically relevant timescales . Similarly, general equilibrium models can provide a good description if the convergence of prices happens on fast timescales and market imperfections are negligible. Dynamical system models, on the contrary, may be more appropriate to describe systems with a high inertia that operate far from equilibrium due to continuous changes in system parameters and slow convergence. A decisive question is therefore if the timescales of processes in the system allow for a separation of scales. For instance, this is possible if the micro-interactions are some orders of magnitude faster than changes in system parameters or boundary conditions. Similar considerations apply for spatial scales.

As we have shown in the examples above, there are many similarities regarding the choice of modeling techniques and assumptions in ESMs and models of socioeconomic systems. However, fundamental differences between the modeled systems pose a big challenge for an informed choice of modeling techniques. ESMs can often build on physical laws describing micro-interactions that can be tested and scrutinized. Of course this can result in very complex macroscopic system behavior with high uncertainties, but models including human behavior have to draw on a variety of accounts of basic motivations in human decision making. These motivations may change over time while societies evolve and humans change their actions because of new available knowledge.

This can lead to a crucial feedback between the real world and models. Agents (e.g., policy makers) may decide differently when they take the information provided by model projections into account. Therefore, modeling choices regarding human behavior might change this behavior. This aspect of human reflexivity makes models of human societies fundamentally different from natural science models and is closely linked to the important difference in social modeling between normative and descriptive model purposes. For example, models that optimize social welfare usually reflect the goal that a government should pursue and therefore have a normative purpose. However, if this model is used to guide policy making while taking into account the actual and perceived controls of policy makers and considers the effect of compromises between different interest groups, it could also describe its behavior. This example shows the often intricate interconnections between normative and descriptive assumptions in decision modeling that modelers should be aware of.

This is further complicated by the observation that the same assumption may be understood in one model as a descriptive (positive) statement, whereas in another model it may be meant as a prescriptive (normative) one. For example, in a model of agricultural markets, the assumption that big commercial farms maximize their profits might be a reasonable descriptive approximation. In contrast, in a model that asks how smallholder farms could survive under competitive market conditions, the same assumption gets a strong normative content.

Another difficulty is that model choices are often not only based on the most plausible assumptions about human decision making but are also strongly influenced by considerations about the assumption's mathematical convenience. Choosing assumptions for technical reasons, for example mathematical simplicity and tractability, may be problematic because it remains unexplained how they are related to the real world. Because not all assumptions can be easily implemented in formal models, a trade-off often has to be found between the plausibility and technical practicality of the assumptions.

Most of the global models reviewed here that describe human interactions with the Earth system are based on economic assumptions about the behavior of humans and societies. They are often only linked in a one-way fashion to the biogeophysical part of the Earth system. Including closed feedback loops between social and environmental dynamics into ESMs is still a big challenge. To advance this endeavor, more work is needed to synthesize modeling approaches that can represent various aspects of human behavior in the context of global modeling, even if the need for generalizations and the formalization of human behavior is sometimes met with skepticism or rejection by social scientists who emphasize the context dependence and idiosyncrasy of human behavior. Of course, models that use simple theories of human decision making and behavior to describe human–environment interactions in the global context cannot claim to capture all real-world social interactions. If models considered the heterogeneity of agents in all relevant aspects, they would have to be much more complex than all models that have been developed to date. However, in many real-life settings, even simple conceptual models of social mechanisms are good descriptions of the key features of the dynamics at work, as we have highlighted throughout this review. Including such formal descriptions of idealized social mechanisms can therefore be a good starting point for understanding feedbacks in the Earth system and their qualitative consequences, which have so far not been considered explicitly in global models.

Summary and conclusion

In this review, we discussed common modeling techniques and theories that could be potentially used to include human decision making and the resulting feedbacks with environmental dynamics into Earth system models (ESMs). Although we could only discuss the basic aspects of the presented modeling techniques, it is apparent that modelers who want to include humans into ESMs are confronted with crucial choices of which assumptions to make about human behavior and which appropriate techniques to use.

As Table  summarizes, we discussed techniques and modeling assumptions in three different categories. First, individual decision modeling focuses on decision processes and the resulting behavior of single agents and therefore has to make assumptions about the determinants of choices between behavioral options. Second, models of interactions between agents capture how decisions depend upon each other and how agents influence each other regarding different decision criteria. Third, modeling techniques that aggregate agent behavior and interactions to a system-level description are crucial for modeling human behavior at scales relevant for the Earth system and require ingredients from the first and second categories. To include human decision making into ESMs, techniques and assumptions from these three categories have to be combined. Finally, we discussed important questions regarding the choice of modeling approaches and their interrelation with assumptions about human behavior and decision making, for example regarding the level of description and the relevant timescales but also the difficulties that can arise due to human reflexivity and the amalgamation of normative and descriptive assumptions in models.

Most formal models that describe human behavior in global environmental contexts are based on economic approaches. This is not surprising because many human interactions with the environment are driven by economic forces, and economics has a stronger focus on formal models than other social sciences. However, we think that it is necessary to advance research that builds on insights from other social sciences and applies social modeling and simulation in the context of global environmental change. One important aim of such research would be to provide a theoretical basis for including processes of social evolution and institutional development into ESMs. If we want to explore the possible futures of the Earth, we need to get a better understanding of how the long-term dynamics of the Earth system are shaped by these cultural and social processes.

A new generation of ESMs can build on various approaches, some of which we reviewed here, to include human decision making and behavior explicitly into Earth system dynamics. However, ambitious endeavors like this have to take into account that the modeling of human behavior and social processes is a contested topic, and the assumptions and corresponding modeling techniques need to be chosen carefully with an awareness of their strengths and limitations for the specific modeling purpose.

No data sets were used in this article.

The authors declare that they have no conflict of interest.

This article is part of the special issue “Social dynamics and planetary boundaries in Earth system modelling”. It is not associated with a conference.

Acknowledgements

The idea for this paper emerged during discussions at the LOOPS workshop in 2014. We thank all the participants for their contributions. Furthermore, we thank the referees and the editor for helpful comments that improved the paper. We thank Rainer Hegselmann, Silvana Tiedemann, Tim Kittel, Wolfram Barfuss, Nicola Botta, and Felix Kersting for their comments on previous versions of the paper. Finn Müller-Hansen acknowledges funding by the DFG (IRTG 1740/TRP 2011/50151-0). Maja Schlüter acknowledges funding by the European Research Council under the European Union Seventh Framework Programme (FP/2007-2013)/ERC grant agreement no. 283950, SES-LINK, and a core grant to the Stockholm Resilience Centre by Mistra. Jonathan F. Donges thanks the Stordalen Foundation (via the Planetary Boundary Research Network PB.net), the EarthDoc network, and the Leibniz Association (project DOMINOES) for financial support. Edited by: Sarah Cornell Reviewed by: Andrew Jarvis and one anonymous referee