Harnessing Computational Simulations to Design and Engineer Policy

By Maxime Stauffer

‍I was recently invited to speak on a panel on Democratic and Public Policy Challenges, at the Artificial Intelligence Governance Forum in June 2020.

In my intervention, I articulated two avenues where computational simulations offer exciting potential: 1) designing policies and 2) engineer policy-making processes. I summarise my key points in this blog post and would be delighted to have follow-up discussions about them.

What are computational simulations?

Computational simulations are models of reality that offer explanations and predictions by capturing the adaptive and nonlinear nature of complex phenomena.They are particularly useful and relevant for policy because policy-making processes and the systems in which policies are implemented satisfy the hallmarks of a complex system. They consist of multiple moving parts (such as individuals and their environment) with often idiosyncratic characteristics whose interaction leads to the emergence of system-level properties (such as levels of inequality, welfare, etc.).

This process of emergence is characterised by non-linearity. Small changes in the micro-level dynamics (say a shift in opinion) might result in drastic changes at the macro-level, and vice-versa. This means that the system behaviour often is different from the sum of its parts.This complex system view of reality has been burgeoning since the 1980s, and the policy community has become increasingly sympathetic towards it because it adequately portrays the arduous reality of policy-making.

The ultimate challenge is to deliver empirical insights on how to navigate such systems. This task is difficult because it requires a delicate balancing act between simplification and embracing complexity. I would argue that scientists often fall into one extreme or the other, resulting in simplistic models that do not account for realistic dynamics or textbooks where authors attempt to capture complexity through lengthy descriptions.Computational simulations enable progress on understanding complex systems by pairing formalism and dynamism. Simulations are run as computer software that formalises the interaction between different system variables (e.g. a network structure, how individuals learn from one another, etc.) and, through iterations, generates system-level dynamics. Simulations can either be empirically validated when data is available or serve as a laboratory to explore theoretical scenarios. As such, simulations remedy three scientific challenges:

They allow reconciling micro-level dynamics with macro-level dynamics (e.g. the relationship between individual opinions and policy outcomes)

They enable the analysis of nonlinear dynamics (e.g. how small/large changes in individual opinion result in large/small changes in policy outcomes)

They provide a tool to test hypotheses in a safe manner (e.g. how different implementation plans lead to different outcomes)I recommend this short visual and interactive introduction to complexity science to read more on the topic.

How can computational simulations be useful for policy?

1. Simulations allow policy-testing safely and robustly.

It is difficult to predict the effects of a policy. Simulations provide policy-makers with the equivalent of a flight pilot simulator. For instance, when French President Macron introduced an environmental tax in 2018, it would have been useful to use models that account for other effects than economic and environmental benefits and losses. Current macroeconomic models do not account for externalities such as the emergence of ‘yellow vests’ shortly after the tax was implemented. The predominance of neoclassical economic models in economic policy prevents the use of other models, such as computational simulations, that offer a way to explore the interrelationships between many more variables. Given the uncertain effects of a policy, it is essential to test policies iteratively, in a safe manner, and identify failure modes before implementation.

On top of uncertainty, policy-making is characterised by slow feedback loops. It is thus challenging to know the effects of a policy until well into the future. The use of simulations can speed up such feedback loops by running as many simulations as necessary before implementation and evaluation.Policy actors do run scenarios already, either individually or collectively. They do so all the time. Yet, the human brain does not allow us to both be explicit about all possible assumptions and compute nonlinear dynamics. Computational simulations remedy these limitations by providing a transparent, formal and consistent way to analyse complex phenomena.Last but not least, more and more of today’s policy-making focuses on building resilience to future risks. The problem is that future risks often have not happened yet. We thus have little data about them and limited policy experience in dealing with them.

Computational simulations enable explorations of different futures and identify positive and negative scenarios. Simulations are particularly useful because they allow quantitative analyses where data is scarce. They enable us to define the debate and reduce ambiguity.‍

In light of the above, my recommendation is that policy-makers work more actively with scientists from the computational sciences in order to develop simulations that are tailored to their needs which can, in turn, strengthen their decision-making in the face of complexity.

2. Simulations allow us to better understand policy-making processes.

The study of policy-making can benefit greatly from computational simulations, which can, in turn, facilitate the design of policy-making processes. Yet, to date, the analysis of policy-making dynamics is primarily driven by public policy theories, qualitative micro-level case-studies and quantitative macro-level analyses.

As I mentioned previously, computational simulations reconcile micro and macro. They can thus explore how configurations of policy networks and the adaptation of political behaviours lead to the emergence of agendas and public policies.Since their creation, political institutions, from local governments to international organisations, have gone through plenty of reforms. Those reforms, or system reconfigurations, shape how policy actors interact and define the rules by which policies are designed, adopted and implemented. It is often ill-understood whether institutional reforms will improve policy-making processes and lead to more beneficial outcomes before one implements them. This difficulty is not surprising, given the complexity of the system and the multiplicity of interpretations of ‘better outcomes’.

I would like to suggest that it is possible to use computational simulations to better understand the impact of reforms before their implementations. For example, how does changing voting procedures influence actor incentives? How does altering the access to negotiations shape policy networks, knowledge exchanges and consensus formation?

Beyond exploring reforms, computational simulations can also shed light on the drivers of policy-making as it happens. For instance, the usual instrument that most policy actors, such as NGOs, use to influence policy consists of providing information (e.g. new evidence, reports, policy briefs, etc.) to decision-makers. Yet, it is unclear whether more (even if more accurate) information leads to better outcomes if we assume that decision-makers can only consume a small subset of all the information they are bombarded with. Perhaps, the crux lies in how groups and individuals process information, that is, how they select, use and turn information into decisions.

How can we understand those dynamics to improve policy research and, perhaps, shift the focus from information provision to equipping policy-makers with tools to process information better?

‍I believe that computational simulations of collective decision-making are particularly relevant to analyse such dynamics which we can barely observe at a system level, but seem vital to envision improvements of our policy-making processes.

If both ideas above are actually promising, how do we get there?

Applying computational simulations to policy-testing and policy-making processes rely upon a number of assumptions, including the idea that policy systems are fundamentally complex, the added-value of quantitative analyses, and the need to acknowledge uncertainty and foster experimental learning.

In our interactions with international organizations from the Geneva ecosystem, we at the GSPI have received validating feedback that confirms the interests in using computational simulations to support decision-making. In particular, it has been emphasised that coupling computational simulations with in-person simulations offers great promise to improve collective decision-making in the face of uncertainty and high-stake scenarios.

The GSPI is interested in fostering collaborations between policy actors and scientists from the computational sciences. As to in-person simulations, the GSPI will launch one in 2021 on building resilience to future pandemics. Stay tuned!