RA1. Veridical AI
Veridicity is the quality of being true, honest, and accurate. However, achieving this in the context of AI is challenging. AI must make predictions that emerge as correct in future data, AI’s outputs must be robust when retrained with the judicious adjustment of hypotheses, data, models, and approximations, and there must be capacity to generalise. Yet, current systems often fail to meet these requirements. Decisions about data (e.g.,imputations, projections, resolving data conflicts, ownership, and data privacy) are often undocumented and poorly justified. Users are sometimes unaware of the AI's intended purpose, which loss functions were used in training, and to what degree veridical principles have been followed.
Drawing on a recent proposal for veridical data science, based on predictability, computability, and stability, we introduce a new and interdisciplinary framework of veridical AI. We define this as workflows and documentations that aims to provide responsible, reliable, reproducible, and transparent results across the AI system life cycle, with a governance ecosystem that incentivises this practice. In filling out this framework, we will explore particularly three directions. First, using philosophical theories of truth together with conceptual engineering and critical data studies, we will refine concepts surrounding veridical AI, so that they are applicable and explainable across diverse research and practice communities. Second, specific aspects of AI (model architecture, distribution-shift, complexity, gradient optimisation) will be foregrounded in targeting predictability, computability, and stability. For veridical AI at scale, we will propose new methods to automate workflows and documentations. We will analyse chain-of-thought faithfulness in LLMs and their representation in the latent space (RA5). Third, we will explore which governance mechanisms (from designer liability to user information to veridical penalties) would encourage veridical AI practices.
In moving forward this agenda, we will study the benefits and challenges of veridical AI. A key affordance is the generation of more reproducible and generalizable results. We will investigate the potential role of veridical AI in promoting scientific discoveries, such as biomarker identification in personalised cancer treatment, and examining its potential in attribution and uncertainty quantification (see RA2, RA3). As to challenges, we will study the tensions between veridical AI and privacy protection. For example, blurring personal data in training (differential privacy) challenges veridicity principles and risks unfairness. We will advance new approaches in discrimination and privacy law that permit greater use of personal data to address veridicity under certain conditions and develop new ML methods that implement this approach.
RA2. Responsible uncertainty quantification
Uncertainty is usually characterised as a problem, especially for trustworthiness and responsibility ascription. Consequentialist risk-based frameworks that rely on probabilities to calculate degrees of moral acceptability and legal/factual likelihood call for reliable and qualified uncertainty quantification (UQ). However, most ML methods do not provide quantified uncertainty. Accurate and interpretable AI requires responsible UQ that addresses aleatoric, epistemic, ontological uncertainties as well as compliance with legal and ethical requirements. TRUST will (i) develop methods to quantify and reduce uncertainty responsibly; (ii) investigate approaches to qualify UQ; and (iii) study the consequences of UQ at scale; and how UQ is socially constructed.
A key challenge for any UQ methodology is the capacity to perform UQ in new domains and under distribution shift. To approach this, we will start with Bayesian model-agnostic meta learning to create models that adapt quickly UQ across tasks. Additionally, we will examine the robustness of conformal prediction and how to correct its uncertainty estimates when there are invariances across tasks or predictable trends. When the aim is to quantify and reduce uncertainty, one can incorporate knowledge into models, leading to physics, biology-, judicial-, etc-informed ML. Yet, when knowledge cannot be easily represented as a mathematical differentiable constraint (RA5), this approach is challenging. We will focus on these hard cases and explore two approaches. First, we will deploy ontologies in logic to represent domain knowledge, and develop new methods for logic-aware UQ and show their applicability. Second, we will investigate the potential of Gaussian processes, which are probabilistic nonparametric methods used as Bayesian priors to incorporate domain knowledge, leading to analytic posteriors that are ideally posed for UQ. As they relate to deep learning through asymptotic theory, we will explore their use in high dimensions, by developing new approximations and hyperparameter optimisation methods, to leverage UQ for deep neural networks.
To qualify UQ, we aim to decompose uncertainty and identify its sources (data components, learning steps, priors approximations). For this purpose, we will develop new Shapley-types scores for dependent factors, where these sources are seen as collaborating players. In this way, we single out system components needing improvement, but also aid in investigating causal relevance and responsibility. The methods for attribution suggested in RA3 will be also tested for UQ.
In some scenarios, there is a strong need to simultaneously quantify, reduce, and qualify uncertainty. One area is high-stakes adjudication of legal claims, such as asylum applications. Adjudicators must navigate high levels of uncertainty due to the inherent (and necessary) vagueness in legal provisions, contradictory legal regimes, and the difficulty of assessing the credibility of persecution claims. Any use of AI predicting asylum decisions at scale – to assist decision-makers or systemic understanding – must be grounded in responsible UQ. To examine combining UQ aims and approaches, we will build a veridical ontology of Norwegian asylum law (RA1, 5), train a deep neural network on 160203 asylum decisions and 4 million surrounding documents, and develop a digital twin for simulations (RA10). We will then equip this model with different measures of UQ, distinguish sources of uncertainty, and examine the consequences of using UQ measures for simulated outcomes on both generative agents and real decision-makers in the immigration directorate and courts.
Whereas reduction of uncertainty can be an aim, uncertainty also offers degrees of freedom. Uncertainty may undermine transparency and explainability, but it may also be preferable under certain conditions. Given the dilemmas faced by citizens, organisations, governments when dealing with (un)quantified uncertainties, we will examine in the asylum situation above and the longitudinal student experiments (RA8) when uncertainty is preferable, because it eventually leads to better qualified decisions. Finally, drawing on science and technology studies (STS) and critical forensic studies, we will investigate ethnographically how UQ is ‘constructed’ by TRUST researchers and partners, and how users (workers, judges, police) handle, qualify, and judge uncertainty. This can foster new, context-specific measures of uncertainty and novel forms of UQ.
RA3. Attribution
Attribution identifies specific parts of an input, knowledge, model or learning algorithm that are most influential in the AI's decisions, predictions and UQ. This can elucidate better how an AI system functions and facilitate algorithm improvement and the identification of biases and moral/legal liability. A attribution thus plays a major role in trustworthy AI, especially XAI, and we will work on new methods and social impacts.
Regarding methods, we propose to use tools from game theory to investigate the origin of biases in an AI systems, including Large Language Models (LLMs), which stem from specific parts of the training data, aspects of the objective function, or dependence between outcomes and protected characteristics (e.g. gender, race) that are mediated by other features. For time series, we will develop models that capture high-order interactions and temporal dynamics, and a multi-scale attribution method that identifies patterns in the data at different scales that influence forecast uncertainty. Our approach will integrate domain-specific knowledge and adaptive explanations to address the limitations of existing static methods and provide deeper, context-aware insights for real-world time series applications. Furthermore, TRUST will train a new foundational meta model that can estimate attributions, by masking in training in turn components in simulated and real data and posing counterfactual questions. Finally, as multiple techniques can produce different attributions, we will develop a Bayesian Mallows model to estimate consensus attribution.
As to consequences, we will study how the uncertainty of attribution affects ascription of moral agency and societal demands for explanation. We explore the ethical and epistemic status of explanations generated by attribution techniques and develop new philosophical tools to balance better epistemic virtues against deontological/consequentialist values, especially in light of bias theory (psychology, social epistemology), social group theory, and theories of justice. We will examine how existing governance models could address legal requirements for precision (e.g., responsibility in tort law, EU AIAct) through partial legalisation of scores, AI-generated counterfactuals, and greater reliance on forms of liability and insurance. To improve human understandability, we will test XAI models grounded in social explanation theory. This includes a conversational explanation approach, based on fine-tuned LLMs, allowing a user to clarify attribution through questions and answers; and a theory of mind-based approach based on enhancing attention around explanations using representational patterns in neural models of the brain.
RA4. Causal AI
Estimating the causal effects of interventions on a system, as opposed to detecting associations, is often critical for making the right decision. Current research on the intersection between AI and causal inference can be divided into: (i) using AI to learn and estimate the causal relations between variables in a system; (ii) leveraging ideas and principles from causality for building better predictive AI models, improving accuracy, explainability, fairness, generalisability and possibly reason. TRUST will work on both. We start with conceptual engineering to draw on multi-disciplinary understandings of causality, to establish how causation is and should be interpreted in AI, and inform understanding of responsibility, explanation, and agency.
For causal learning, a fundamental aspect is judging actual causation, i.e. causal dependence between actual individual events. In ethics and law, to attribute blame often requires showing that an action caused harm. Fair judgements thus go beyond the interventional level of causal inference, requiring fine-grained and counterfactual reasoning. With this purpose, we will develop a new inferential method and theory of actual causation, incorporating relevant constraints, including those in logical form, and assumptions known to be necessary for inferring actual causation between events, exploiting attribution (RA3). The aim is to develop an approach that satisfies legal causality-based demands for fairness, as opposed to purely associative fairness, especially in the context of proxy discrimination, which can be unlawful. In a similar vein, we will examine when a right to an explanation for an AI-based decision (e.g., AI Act, GDPR) implies a right to a causal explanation; and develop new methods to quantify the effect of a cause to improve compensation calculation.
To leverage causality, we will develop causal multitask neural networks, which exploit invariance to protect AI systems from using forbidden features. Causal relations are stable across domains, meaning that if AI can capture causal features it can generalise better. We will examine the presence of representations of causal relations in latent space, for example as directed acyclic graph, and if these influence generalisability across new domains/tasks. In LLMs, we can do this by analysing the latent representation of sentences explaining causal relations and reasoning, and by experimentally tracking flows in latent space when LLMs are given such tasks. Finally, we will study relations between statistical sufficiency and these causal effects representations, as they can lead to more efficient and sustainable model architectures.
RA5. Representation in latent space and alignment
Embeddings in latent space are a key reason for AI’s success. They represent input data (text, images, time series, etc), in a vector space. These embeddings learn meanings and relationships between the represented units by constructing geometric structures in latent space. Their success arises because these representations are invariant across similar tasks and domains, becoming the building blocks of foundation models. Alignment for trustworthiness calls for new notions of learning, novel optimisation algorithms and theoretical guarantees of generalisation, and we will focus on ways to understand and control representations in latent space.
Alignment requires principles, rules, or laws that reflect normative values (RA14). To represent these, we will develop ontologies in formal languages; and explore formal approaches including simple decision trees, knowledge graphs, and knowledge basis in advanced knowledge representation languages such as Description Logics and Datalog, with a probabilistic dimension. These ontologies will be incorporated in ML models, resulting in ethics/law-informed predictions and reasoning, in a Bayesian manner or with (relaxed) differential constraints. Some deductions may be ambiguous, requiring the method to leave conclusions open (RA8, 13). We propose an original ‘hub and spoke’ approach, where universal constraints always hold (hub), while specific ones (spokes) apply to certain applications. Dilemmas arise when spokes conflict. We also study the inverse problem: determining which spokes an AI agent is trained to align with (RA3). To detect faked alignment, as in LLMs “dishonesty”, we will try new forms of statistical anomaly detection.
The sufficient dimension and complexity of the latent space will be studied theoretically. We investigate the minimal dimension of the latent space that allows for alignment of guaranteed quality and propose a version of the information bottleneck where alignment is part of the constraint. Theoretically and experimentally, we will study how alignment do generalise under distribution shift.
Our hypothesis is that useful latent space is a much lower-dimensional manifold, with a complex structure. We will try to describe the ‘useful’ manifolds geometrically. The aim is to build directly this manifold without reducing learning ability and generalisation. This is also important for addressing the inverse problem: given the geometry of the ‘useful’ latent space, can we identify the rules to which a specific AI has been aligned (RA3). This could be explored by training a deep neural network to output a description of its latent space given knowledge represented in a specific way, such as in form of a graph or logical theory (forward mode), and then using it in reverse mode to estimate the graph from samples in latent space.
RA6. Foundation models
Foundation models are complex and deep neural networks trained via self-supervised learning on vast amounts of diverse data across different domains. Designed to be adaptable for a wide range of related tasks (e.g., language translation, text generation, new materials, coding), specific model architectures using transformers enable very informative embeddings, which successfully transfer across tasks and applications. Fine-tuning with delimited datasets for specific tasks and prompt engineering reduce the need for building AI models from scratch. Given the challenges of competing with large AI companies in developing new large architectures,TRUST will focus on theoretical and conceptual parts of foundation models: (i) parsimonious parameterisations, (ii) learning with small data, (iii) temporal and spatial predictions; (iv) liability and ethical implications of fine-tuned models; and (v) the advent of world models.
Drawing on new approaches to reduce cache memory of transformers, we will develop parsimonious parameterisations by using Bayesian priors to borrow strengths across tasks used in training or predicted to be relevant in operations. As to data dimension, we take a resource-constrained setting to build smart and mindful models by informing them with domain knowledge-derived priors36, designing calibrated training data (including synthetic data sampled to span the training space), and reducing data redundancy by recognising this during training. Various forms of information bottleneck have been proposed to build deep models that efficiently summarise the input data, while controlling complexity. While we investigate the role of statistical sufficiency further, we will propose instead adequacy as the correct property.
TRUST will work on foundation models for time series and spatial-temporal data to learn efficient embeddings of complex dependencies across domains and under irregular sampling. A systematic performance assessment of state-of-the-art time series foundation models against statistical and ML ones will clarify how temporal scale and domain affect need for data. We will develop methods that allow detailed benchmarking of across-domain representational learning, including whether relations learned for different domains rely mostly on shared or orthogonal representations at specific layers of the model; and develop spatio-temporal Graph Neural Networks and Bayesian hierarchical models to share information across space and time.
The emerging practice of repeated fine-tuning foundational models produces multiple AI ‘products’ raising significant legal and ethical dilemmas. How can alignment and interpretability efforts undertaken in earlier trainings be maintained during transfers and over time? We will investigate this experimentally in: macroeconomic predictions, where on-line data calls for repeated training; an AI-based small claims tribunal which must give reasons based on updated precedents; and repeated fine-tuning for anomaly detection.
Foundational models can pose a threat to individual autonomy, as the utility functions employed in their training may not align with the user's values, desires, and legal rights. Fine-tuning enables the introduction of additional rewards, and we will explore the extent to which the original utility can be overridden. We also envision situations where users may want to frequently realign their 'private' AI assistant to better suit their needs. Therefore, we develop new, efficient algorithms that can be easily fine-tuned on limited computing resources, such as a smartphone and study liability consequences and ethical aspects.
Finally, world models are representations of an environment that help agents predict the future evolution of their surroundings and estimate missing information. The next years will show how well generic world models can generalise, perform reasoning40, and make decisions under uncertainty and in the presence of constraints. We will assess the philosophical, socio-legal, and technological dilemmas raised by these foundation models and explore predictive programming through automatic code preparation for future needs.
RA7. Generative AI
Generative AI (Gen-AI) produces text, data, images, video, motion, materials and more, when properly instructed and trained. Probabilistic ML is in the core of Gen-AI, and computational efficiency on dedicated hardware has been a game changer. LLMs portend a major impact in all aspects of society, yet much of the development of Gen-AI is performed by a few private companies, posing a major threat to the control, transparency, and sustainability of AI. TRUST will work on key issues in Gen-AI that do not require computational resources that are unavailable for an academic centre.
Smaller models trained on smaller data are more controllable, which is important when a predictable distribution of errors is essential. We will try to substitute larger Gen-AI models with smaller ones, by exploiting quantisation, or by generating synthetic data especially designed to sample systematically the quantiles needed in training. Synthetic data must be sufficiently veridical, extend the data distribution in realistic and unexplored directions, and meet legal/ethical demands. One idea is to define and estimate data axes in embedded space, which we will follow towards the tail of the distribution and beyond, differently from current isotropic diffusion models. An alternative idea is to improve finetuning of Gen-AIs. First, we will try to develop new mathematically grounded methods for finetuning that modify LLMs in a more controlled way, compared to low rank adaptation, for better alignment. Second, instead of relying on the commonly used Bradley-Terry model for reward modelling, we will propose the powerful tools of Bayesian preference learning, to synthesise better the preference consensus of human or algorithmic assessors.
Gen-AI raises deep issues concerning transparency and human agency. The public should know when data have been generated by or used in AI. TRUST will work on watermarking of domain-specialised Gen-AI generated text, which will not affect its content and form but permit detection of its origin. We will try to estimate the probability that a text is LLM-generated, without knowing which potential model authored it, using a wide collection of tests (checking for watermarks and invisible characters, exploiting Bayesian predictions, and other classifiers), an ensemble model that exploits individual strengths and dependencies between tests, and multiple testing theory. The method will be validated on digital evidence in courtroom scenarios in light of new EU AI Act requirements. We will also explore watermarking of generated tabular data, seeking to avoid damaging the statistical content of the data, by relying on truncated vine copula.
Generating motion is important for AI driven robots. Gen-AI is particularly needed for soft robotics, as traditional robots are made from hard materials, requiring complex sensors to generate movements that handle soft objects without damaging them. We will advance ML methods that merge evolutionary algorithms and physical models for soft materials to develop a new non-contact robot (utilising vibration induced lubrication), allowing detailed control over adhesion and locomotion without the need for direct physical contact.
Finally, we will examine questions of whether ‘human&AI’ productions can be understood as ‘co- authored’ with the complex and emergent interaction of human and synthetic voices, artworks, and materials. In the emerging ethical and legal approaches to these questions, TRUST will study how the balance might be best struck between democratising creativity and avoiding inaccurate/malicious representations and harms, especially in the fields of learning, infrastructure, and armed conflict. With the advent of protein folding AI capabilities and synthetic DNA printing, the possibility of life itself as an AI-generated artefact is plausible. We will examine the bioethical significance and risks raised by these possibilities.
RA8. Emergent behaviours
Emergent, self-organising behaviours in complex non-linear systems refer to spontaneous, often unexpected properties that occur from the interactions between simpler components of a system, rather than their individual operation.46 As the scale of AI systems expands, with increasing model architectures, cooperating stochastic processes, data, agents, networks, infrastructure, and human-AI interaction, the likelihood of such emergence increases. For mobilising creativity with Gen-AI such emergence is desirable; but less so when hallucinations happen, and transparency or control are needed. TRUST will study emergent AI behaviours by (i) proposing explanatory theories, (ii) developing procedures to identify them; and (iii) studying dual emergent behaviours in human-AI interaction.
For the first research line, we will merge the theory of interacting particles in statistical mechanics with transformer architectures, investigating the role of asymmetry in learning and emergence resorting to concentration and information theory. These explanatory theories will be exploited to develop protective mechanisms against undesirable emerging behaviours. For instance, to avoid hallucinations in Gen-AIs, we will develop a mathematical theory of ‘I don’t know’-functions, which distinguish what is correct and what AI cannot answer, and test it in learning.
In order to detect emergent behaviours, we will develop new identification measures. One interesting form of emergent behaviour that TRUST will work on is self-organising cooperation between independent AI systems operating autonomously in technology-driven markets. Such emergent ‘algorithmic collusion’ can lead to higher prices and crowd-out smaller actors, and, if undertaken intentionally by humans, would violate competition law. To identify emergent collusion, TRUST will design reinforcement learning models that realistically simulate market behaviour under competitive algorithmic pricing, incorporating uncertainty in external market forcings. We will develop and apply new Turing-type tests for identifying emergent behaviour, deploy game theory to develop ML strategies that reduce this risk (e.g., whistle blowing agents), and propose new legal doctrines that use simulation evidence and governance models that anticipate algorithmic collusion (RA13).
Interaction between humans and AI can generate dual emergent behaviour in artificial and human cognition. AI-driven cognitive augmentation integrates technological artefacts into human cognitive processes, enhancing reasoning, sensemaking, and creativity. This can improve learning, productivity, health, and decision-making. However, the subsequent impact of AI on human cognition, and its feedback on retrained AI systems, remains unclear. TRUST will study the consequences of this circularity by modelling human-AI interactions using neuroscientific, computational, and visual methods, which will also offer new insights into human cognition and the subjective dimension of trust. This will include a longitudinal experiment on student learning interactions with Gen-AI (RA7); a cross-sectional study on how students process hallucinatory material; and an ethnographic study on engagement with AI recommendations by students/user partner staff.
RA9. Secure AI
Trustworthy AI requires development systems that are robust against a variety of threats, ranging from data integrity issues to adversarial manipulations,51 while ensuring that they are resource-efficient and maintain high performance. TRUST will implement adversarial training, develop defence mechanisms such as anomaly detection, improve robustness to distribution shifts and system heterogeneity, design interpretable models, employ post-hoc explanations, and build benchmark data sets. In so doing, focus on security by design, to foreground the need for foundational level of security rather than ex post patching of vulnerabilities.
To improve the state-of-the-art, we will focus on three areas. First, we will develop blockchain-based mechanisms to ensure data integrity and employ advanced cryptographic methods to create tamper-proof, decentralised frameworks for distributed AI systems. Second, we will propose new methods for online real- time, high-dimensional anomaly detection and robustness to distribution shifts that surpass traditional methods, enabling the identification of nuanced threat patterns and actionable prevention/mitigation strategies. This approach will also identify the type of anomaly and we will work on the prediction of anomalies in time, opening for new prevention strategies (RA12). Finally, we will develop hybrid models. A common way to represent security requirements and policies is by means of constraints expressed in logic-based languages, but incorporating such constraints into ML is challenging. We will develop new hybrid learning architectures that will leverage the advantages of both paradigms, building upon Graph Neural Network architectures.
Security and sustainability/safety can be sometimes in conflict. We will explore resource-efficient adversarial defence methods, minimising privacy compromise and computational overhead while maintaining the necessary security of the AI model. Moreover, the rising deployment of AI systems in conflict areas is raising fundamental questions over their safety and blurring ascription of agency and responsibility. Drawing on STS, we will study how such AI shapes, alters, and transforms how conflicts are perceived (by sensors/vision systems), memorised (with data and other mnemonic devices) and understood (via AI-based knowledge production/digital twins). This raises broader questions over the consequences of the growing amalgam of security paradigms and AI technology.
RA10. Digital twins and hybrid modelling
A digital twin is a set of virtual information constructs that mimics the structure, context, and behaviour of a natural, engineered, or social system, is dynamically updated with data from its physical twin, has a predictive capability, and informs decisions that realise value. Digital twins often include physical, biological and mechanistic models as well as real-time data from their physical twins and are able to provide an overview of the complex operations, draw on causal properties in the models, and explore what-if scenarios and predict outcomes. Current digital twins struggle to account for complex non-linearities and interactions and the power and scale of AI-driven digital twins are prompting governance concerns. TRUST will work on (i) self-adaptive twins for automatic composition, (ii) complementing twins with other AI components; and the (iii) democratic governance and philosophical implications of digital twins.
To advance self-adaptive twins, we will explore how queries from a stakeholder (the user of the digital twin, a human or a machine) in a custom, domain-specific digital twin interaction language, can form the basis for experiments in a self-adaptive digital twin. Such a twin can configure automatically and compose models in its model ecosystem, enriched by logically formalised domain knowledge about its own predictive abilities. To this aim, we organise the digital twin following a MAPE-K (Monitor-Analyse-Plan-Execute over a shared Knowledge) architecture, where AI models enhance the MAPE components as well as a knowledge base. The twin will provide justified answers to queries by making assumptions explicit, and explaining how different models have been interacting, clarifying sources of authority, etc.
As to AI components, we will examine how to increase the flexibility and data representation fidelity of twins, while preserving their ability in what-if scenario exploration with causal interpretation. We will explore systematically the behaviour of various types of hybrid models (e.g. by training a general AI model on the residuals (errors) of a mechanistic model) versus pure mechanistic models, so as to better understand the trade-offs between increased data representation fidelity and interpretational/causal properties.
The large investment in digital twins raises many governance and conceptual questions. We will explore potential anticipatory governance mechanisms (RA13) for trustworthy twins. The focus will be on data control and ownership, assurance, participatory governance mechanisms in AI digital twin engineering, accountability, and penalty mechanisms for deviant behaviour; as well as AI-based methods to monitor the behaviour of digital twins and identify adversarial gaming. Moreover, digital twins raise philosophical questions about the nature of representation. What elements of a system must be replicated, and how accurately is a twin constituted? We will explore conceptions of identity and individuality in relation to AI-powered twins, and how they cast new light on classic problems such as mind vs. body dualism, the ship of Theseus, and brain-in-a-vat.
RA11. AI engineering
Trustworthiness of AI does not only depend on the model itself, but on the total software system that implements it, including its operation. This requires the regulation of the whole AI system, including pipelines, datasets, and the combination of AI components with other software artefacts. The lifetime of AI systems and their ease of use, generate a host of prototypes that are validated to work in the short term, but cannot be sustained in the long run, creating a very fragile and untrustworthy AI infrastructure. Since developing effective AI systems require large investments, it is crucial to be able to trust, reuse, combine and modify AI models across application domains, deployments geographies, societies, and over time. This is challenging since AI models are typically trained and assessed according to a very specific context, and a few technology companies control how AI systems are rolled out as infrastructure.
TRUST will seek to transition traditional software engineering research towards adaptive AI-focused software engineering, including with new uses of continuous integration practices, low code or no code integration languages, and efficient integration platform as a service. We will propose modelling frameworks that support the modularisation of AI models, so that components can be updated and combined according to needs. Another challenge is testing of AI modules and systems that are largely black boxes, making their customisation and integration into IT systems, and their governance, particularly difficult. Assurance and governance practices traditionally rely on engineers testing the behaviour of systems within well understood bounds. This does not apply to AI systems with emergent behaviours. To mitigate this challenge, we will propose new models of quality assurance for AI systems, possibly based on our theory for emerging behaviours (RA8), statistical adaptive stress testing, sensitivity analysis, and explainability practices with an anticipatory governance model (RA13); and examine how emerging ISO/EU standards accommodate or encourage them.
RA12. Disruptive AI-driven digitalisation
AI is slated to be a disruptive technology, substantially modifying or transforming politics, markets, institutions, societal values, and consumer behaviour.55 This disruption may be positive contributing to human flourishing, but also have negative aspects, creating new ‘winners and losers’. Developing trustworthy AI at scale requires attention to these direct and emergent effects and potential solutions, and TRUST will thus research AI-driven digital disruption in organisations, democracy, economic crime, and the geopolitics of AI.
In organisations, AI’s content- and process-based qualities may challenge long-standing delineations between different forms of knowledge (factual/technical vs judgments/value-based) and expertise (technical vs domain-centric vs private consultants). We will investigate how AI engenders, supports, re-composes, and unsettles knowledge production and expertise deployment by studying some user partners with a qualitative practice-oriented research design. We will also research which forms of circumspection and control best help govern this disruption and whether TRUST’s approaches can minimise the negative disruptive effects. Equally, we will study ‘hype’ narratives accompanying AI-driven disruption and draw on the ‘genetic revolution’ history to study how organisations can/should navigate utopian (and dystopian) AI messages. For democracy, these tensions and risks are equally present. AI-driven dis/misinformation can undermine democratic representation and corrode political trust. We will begin by studying two AI-disrupted elections in the ‘2024 super election year’ to understand how AI is impacting the information landscape, enables covert influence, and undermines cybersecurity; and the surrounding 'hype' narratives of fear and optimism for AI in elections.
The global architecture of digitalisation creates a new space for disruptive AI. AI-generated deviance, and AI-driven fraud has exploded (e.g., FraudGPT, deepfakes). TRUST will investigate these new risks by developing/testing anomaly detection to identify fraud attempts, pilot an AI-simulated stress test on their vulnerability, revisit the current regulatory division of risks between financial institutions and victims for ID theft, and examine the ethical implications of new proposals to use biometric data to combat AI-based fraud. Finally, AI is the subject of a new form of geopolitics, as states race to dominate the technology, cement their regulation as the global model, and conquer the global commons (high seas, deep seabed, outer space) with the AI’s affordances.56 TRUST will study the actions of the key actors (US, China, EU, multinationals) and examine how small/middle powers, including Norway and the Global South, can strengthen their agency as a responsible actor to better ensure fairness/sustainability but also avoid exclusion from the benefits of AI.
RA13. Reforming AI governance
Given the pace of AI’s development, its emergent characteristics, and the uncertainties inherent in the large temporal distance between its deployment and ultimate effects, it is a perfect embodiment of the Collingridge dilemma – by the time the uncertainty is resolved, it may be too late to respond. TRUST will examine how AI governance can be trustworthy through analysing existing AI governance and its alternatives.
The governance ecosystem for AI is deeply complex and fast changing with multiple national and international regimes of rules and responsible actors. There are AI-specific regimes, such as the AI Act, the COE AI convention, ISO standards, and over 100 soft law initiatives (e.g., Hiroshima, Bletchley Park); generic regimes that regulate aspects of AI, such as those on privacy, intellectual property, liability, human rights, competition; and domain-specific regimes which constrain or encourage AI, such as in the finance, justice, maritime, energy, and health sectors. Combining legal interpretation, social scientific description, data science, and complexity and game theory, TRUST will map in a dynamic manner this governance complexity, and seek to identify ‘policy hotspots’ of overlapping, clashing, and absent regulation that may require greater attention. This will be matched with a fine-grained focus on interpretation issues in the AI Act, liability law, and data privacy. This will include the ‘lost in translation’ challenge,58 exploring with conceptual and legal engineering how concepts like accuracy, robustness, explainability, security can be best understood when placed in law.
Using empirical qualitative methods and ethical baselines we will analyse the quality of AI governance by developing a framework for evaluating the extent to which governance within and across domains meets markers of good democratic governance such as: sufficient control/transparency; stakeholder participation (drawing on earlier Nordic models of stakeholder participation in IT); feedback and accountability mechanisms; respect for sustainability and human rights; and governance capacity and future preparedness. This framework will be applied on four user partners and the EU’s AI Office. In addition, we will analyse the growing privatisation of AI governance (self-regulation, certification, contract law and automated regulation), and how new AI architectures driven by multinational corporations (RA10) are reshaping AI governance.
Drawing this together, TRUST will examine which models of governance can best manage the risks and benefits of AI while securing rule of law and democratic values. TRUST will explore especially the model of anticipatory governance. It requires the active use of foresight techniques to understand the evolving nature of the policy challenge; experimental engagement in identifying/piloting governance solutions; and the choice of legal/policy instruments that can produce catalytic change when and where it is needed (e.g., penalty defaults for poor progress, trigger/sunset clauses, ‘mindshift’ campaigns, early governance adopters, green limits). Emerging governance regimes such as the AI Act contain elements of this approach but tend to be a bricolage of existing regimes. We will create processes for foresight of AI/governance trajectories, simulate/initiate governance experiments with user partners, and explore which reform instruments could be catalytic, culminating in proposals for amending national laws and institutions, the AI Act, and ISO standards.
RA14 AI for flourishing
AI can contribute to flourishing for human beings. Flourishing means to live a good life, including the exercise of intellectual capabilities, education, creativity, meaningful relationships, participation in fair, democratic societies, and responsible interaction with nature. TRUST explores AI for flourishing, seeking to conceptualise the term in ways that help optimise the potential of AI, and develop an ethical approach that goes beyond a risk-avoidance paradigm that dominates AI ethics (RA3,7).
While AI heralds a new democratisation as citizens can better access knowledge in ways previously confined to elites and developed countries, this may be resisted by those in power.61 Moreover, AI may lead to a de-learning of existing knowledge and skills and greater dependence on private and public platforms. TRUST will research specifically how AI can democratise knowledge. We will use knowledge-informed ML to improve cognitive modelling of learning processes (RA8) and test which types of adaptive AI technologies optimise learning and minimise residual effects with a student- based trial.
Governance of large and opaque systems could be improved substantially through AI. It carries the potential to analyse legal standards, reduce human bias and costs, extend legal aid, identify citizens’ views/needs, and embed governance by design in AI systems through ‘AI as law’. This requires though complex and bespoke AI models. TRUST will research how AI can be used to help monitor large-scale atrocity crimes and illegal transnational fishing (RA9) and develop efficient and low-cost small claims processes for citizens and integrate legal regulation within spatial-temporal AI models (RA6).
AI for flourishing must take account of the need for a green transition. AI may strengthen societal resilience and disaster preparedness, efficiency, and distribution measures for renewable energy systems, and spread of green know-how. TRUST will work to understand the drivers, hindrances, and political challenges in the realm of AI at scale for the purpose of advancing the green transition. This will include uncovering minimal neural networks that are sufficient for efficient training and inference, using statistics and information theory. We will establish key performance indicators (KPIs) and metrics for green AI; and understanding trade-offs between green AI and AI for green flourishing. In developing KPIs, we will distinguish between organisations without significant analytical capabilities or data access, while imposing more stringent sets of indicators for developers, regulators, and green auditors. Finally, we explore, design and model new AI taxation regimes for the creation of planet-friendly development and narrow the global AI divide.