Guidelines for the Use of Artificial Intelligence in Financial Institutions (FINMA)

Many banks, insurance companies and other Swiss financial institutions are currently running projects on the use of artificial intelligence – some of them for years, others are new. While the Federal Council is still considering where and how AI should be specifically regulated, FINMA has already defined its expectations in the form of four guiding principles for AI. But what do these guiding principles mean? We provide an answer in this part 22 of our AI blog series.

First of all, despite the current hype about AI, the use of artificial intelligence in the Swiss financial industry is not new. For example, applications trained with machine learning have been used for many years, for example in the detection of money laundering or fraud, in investment decisions or for forecasts in the insurance industry. Applications based on generative AI are particularly new. They are used in financial institutions on the one hand via tools such as “ChatGPT”, but also installed in applications for specific applications. Examples we see with our clients are the minutes and evaluation of meetings, the extraction of content from documents, the analysis of call center conversations, the summarization of news or the automated analysis and drafting of contracts.

“Supervisory expectations” and controls

Of course, these activities have not escaped FINMA’s notice. In November 2023, it already commented on the challenges of AI in its “Risk Monitor 2023” and formulated its supervisory expectations of Swiss financial institutions. It is to be expected that AI will also bring about various changes in the financial market, she writes in it. In its “Annual Report 2023” a little later, it states that the autonomy and complexity of AI systems pose various risks. She cited the danger that AI-generated results cannot be understood or explained by humans, that errors or unequal treatment creep in unnoticed, or that responsibilities are unclear. It has already been conducting on-site inspections and supervisory discussions on this topic with financial institutions since the end of 2023 and intends to continue to do so.

Financial institutions that use AI must therefore expect to be questioned and monitored by FINMA and to explain how they record, limit and monitor AI-specific risks. They will have to show how they implement FINMA’s “supervisory expectations”. These consist of the following four guiding principles:

1. Clear roles and responsibilities as well as risk management processes must be defined and implemented. Responsibility for decisions cannot be delegated to AI or third parties. All parties involved must have sufficient know-how in the field of AI.
2. When developing, adapting and applying AI, it must be ensured that the results are sufficiently accurate, robust and reliable. In doing so, both the data and the models and the results must be critically questioned.
3. The explainability of the results of an application as well as the transparency of its use must be ensured depending on the recipient, relevance and process integration.
4. Unjustifiable unequal treatment is to be avoided.

While some of these expectations define the familiar, other elements of these guiding principles raise questions about how they are meant and can be implemented. We will deal with this in the following. The explanations come from our understanding of the matter, our experiences from concrete cases and conversations. FINMA has not yet publicly elaborated on its guidelines on the Risk Monitor 2023. However, it regularly holds discussions with individual financial institutions, in which the topic is further explored. It should be noted that although the term “supervisory expectations” chosen by FINMA itself may suggest a formal nature, the guidelines do not come from a supervisory announcement or a circular and are therefore less binding. So an institute can deal with the issue differently. However, we believe it is important that financial institutions deal with the issue in such a way that they have a sensible plan for dealing with the risks associated with AI. Then they can answer any critical questions from FINMA even without adopting FINMA’s guiding principles.

Guiding principle No. 1: Governance and accountability

This guiding principle consists of three different elements that do not necessarily belong closely together. First of all, it is stated what a clean governance of compliance and risks in a company always requires: The tasks, competencies and responsibilities (so-called AKV) must be defined, i.e. it must also be clear with regard to the use of AI which departments (preferably people, not committees) take care of which aspects, how they have to do this and who is responsible for the respective success or achievement of goals (e.g. compliance compliance). Since AI is ultimately only one of several possible forms of computer science (AI legally means a system in which not only humans have programmed how an output is created from input, but where this is sometimes only done on the basis of training, see box), the AKV for AI can often be mapped within the framework of the already existing directives, processes and other governance measures, or already are. However, because AI has a strong interdisciplinary element and requires new perspectives, we see many companies that regulate the AKV and other requirements for the use of AI separately, e.g. in a directive for AI. Such a separate AI instruction also offers communicative advantages.

In the present case, FINMA’s statement that “[t]he responsibility for decisions … cannot be delegated to AI or third parties”. At first glance, this sounds like a matter of course: A financial institution remains responsible for its management even if it outsources functions to third parties. This has always been the case in outsourcing, and it must be all the more true when using a machine operated by the financial institution itself. The fact that FINMA nevertheless mentions this has to do with the fear that the people responsible for decisions in the institutions could in fact still relieve themselves of responsibility if decisions are made by the AI. In addition to the risk of poor quality (see Principle No. 2) and incomprehensible behavior (see Principle No. 3), she sees this danger above all where AI errors go unnoticed, processes become so complex that it is no longer clear who is responsible for what, or where there is simply a lack of know-how because the systems have become too complex. The peculiarity of AI systems, the element of “autonomy” (no longer every decision will be pre-programmed, but “only” trained), can also contribute to these risks.

FINMA is concerned with fully automated decisions for which no one feels responsible, as well as those made by people based on AI results, whose errors they do not recognize, whether due to negligence or incompetence. This also has an indirect impact on supervision by FINMA, for example because it can no longer effectively control how a financial institution has arrived at certain decisions because no one can be found to answer its questions in this regard. It does not want the “blame” for AI errors to be shifted, nor does it want the institution to no longer have the necessary know-how to either make important decisions without AI or to be able to comprehend and, if necessary, override. A financial institution should therefore not rely on a tool or technology in relevant areas that it does not understand itself. Although FINMA explicitly mentions generative AI (i.e. “ChatGPT”) as an example in the Risk Monitor, the problem concerns just as much deterministic or predictive AI (as it has been used for some time, for example, in AML checks, for example), if not even more so in fact, because it is used in more important applications where errors can have a correspondingly higher impact (such as the use of AI for risk management or the fight against money laundering).

However, it is also interesting to note what FINMA does not mention in its Guiding Principle No. 1 and thus apparently does not expect e contrario: that decisions are only made by people. In other words, it is possible to have decisions made by AI. This means that AI may be used for or to support decisions, as long as someone in the bank who is allowed to make such decisions actually controls the use of AI and also “sticks their head out” for them – end-to-end, not just for individual components. So if AI is used to make investment decisions, someone from the area where such decisions are usually made must remain responsible. This person should know that if there are problems, they cannot hide behind IT or the supplier of the AI because it had a defect.

FINMA’s position is fundamentally correct: there will be more and more areas in which it is almost appropriate to let computers make decisions, also based on pattern recognition, because human decisions can have relevant disadvantages. Under certain circumstances, transactions can be checked much better by a machine for signs of money laundering, sanctions violations or fraud and, if necessary, stopped. We may therefore be able to meet the goal much better here with machine decisions, at least in a first step. It doesn’t matter that mistakes happen, because they also happen to people. The decisive factor is that an appropriate quality is delivered overall. We can compare this to the approval of medicines: they also have side effects, but if the profit is sufficient and the ratio is right, we allow them. In practice, the main problem is that we sometimes lack some experience with regard to certain forms of AI, such as the use of large language models, or we are sometimes deceived by the pretended quality of the output. Finally, this topic also includes the third element of Guiding Principle No. 1, namely that all participants must have sufficient know-how in the field of AI. In practice, we see some deficits here. Although many companies have now introduced generative AI and tools such as “ChatGPT” with their employees, most of these activities focus on the legally compliant use of such tools in the personal sphere. A “prompting” workshop may well provide a good insight into the possibilities, limits and certain risks of generative AI. The “fear-mongering” lectures on the subject of AI, which are always popular in the early days of a hype, can also raise awareness (but we have always refrained from doing so). However, FINMA’s expectations in terms of know-how rightly go well beyond this. On the one hand, this involves know-how about how AI actually works, i.e. which AI methods can be used for which applications and how, what technical and organizational measures exist to address AI-specific risks, etc.

On the other hand, it is important to understand the AI-specific risks that the use of AI poses for the institution as a whole, namely financial, operational, reputative and legal. It is not enough for employees to know where “ChatGPT” could be a problem. It is also not enough for an institute’s AI experts to know what they are doing. The top management of the institute must also understand what and what risks the institute is taking when using AI. The members of the Executive Board and the Board of Directors must also be able to describe what the AI-specific risks are for the institution and how it deals with AI, and they must have thought about what they are willing to accept before corresponding projects and initiatives are approved. This in turn means that they must have a basic understanding of what AI actually is, where the weaknesses and strengths lie, and what approaches there are for dealing with it. It’s about understanding what AI really brings to the financial institution, and what it doesn’t, and in which applications it is used.

Of course, all this does not have to go into every detail at this level and it can and will be prepared by experts, but in our opinion there must be a basic understanding of the AI-specific risks from the point of view of the institution as a whole, and this is also how we understand FINMA. Governance bodies need to have a good sense of the issue, without scaremongering and without exaggerating the power of AI. In our experience, this is still often lacking, which sometimes has to do with the fact that – apart from the aforementioned “ChatGPT” workshops and “fear-mongering” lectures – AI risk training at C-level and for VR tends to be neglected in our experience; this has led to the fact that we are now also active in the field of management training, although this is not in our typical area of activity.

The need to deal with AI-specific risks explains the last element of Guiding Principle No. 1, the definition and implementation of risk management processes. This is nothing new either. One challenge for many institutions is the expansion of their existing risk maps and catalogues to include AI-specific risks. However, this is only a matter of time. We have also developed tools for this purpose, which are available free of charge as open source and are already used by a number of companies, including banks and insurance companies.

This also includes a proposal on how AI projects can be classified according to their risk exposure for a company, because not every AI application is equally risky.

Incidentally, FINMA also advocates the risk-based approach in its guiding principles. It is primarily about applications with correspondingly high risks: What happens if the AI makes a gross mistake or does not work? How many customers would be affected and how? What would be the financial implications? What impact would this have on compliance with legal requirements?

Guiding principle No. 2: Robustness and reliability

On the one hand, this requires appropriate quality management with regard to AI-based components and systems and, on the other hand, implicitly formulates the expectation that AI systems will only be used autonomously if they are sufficiently reliable and this can ultimately be proven.

FINMA is particularly concerned that AI systems are being used that deliver incorrect results or results that are otherwise unacceptable because they have been trained with incorrect or non-representative data, or because the circumstances change in such a way that their output no longer fits, for example because the model has not been updated (so-called concept drift)). In fact, in practice, the convincingly designed outputs of generative AI in particular can lead to overestimating the quality of their content. So there can be no errors or bias of the output, but also a bias with regard to the assumption of the correctness of the AI – in other words, it becomes too trusted, even if a human controls the result (so-called AI overreliance). For example, a large language model can easily be asked how likely a previously generated answer is to correspond to the facts. It will provide an answer. However, as empirical research shows, these answers are extremely unreliable, because although the model works with probabilities when it generates text, it cannot incorporate these probabilities into its own text. However, these things are not visible in the answer – it seems convincing and self-critical.

In addition to the risk of inadequate training data and changing circumstances, institutes should also be aware of the limitations of the techniques they use. This also applies to predictive AI, such as that which can be used to predict financial parameters. Thus, each machine learning method has its advantages, disadvantages and areas of application that must be known. For example, a certain methodology for forecasting certain values can work reliably in its normal ranges, but fail with more extreme inputs or certain combinations – without this being visible in the output. Anyone who uses them must be aware of these limits and be able to deal with them.

FINMA also sees AI systems as a new gateway for cyber attacks that must be counteracted. She has every reason for this expectation, especially generative AI systems allow new forms of attacks.

This often poses a double challenge for financial institutions: first, they must ensure that information security is ensured in the classic way; many of the new AI applications are being used in the cloud, with which some financial institutions (and their providers) are still gaining experience. Secondly, the new forms of attack require new defensive measures. For example, an EDR or firewall is of no use against an attack via prompt injection, in which a correspondingly formulated command to a chatbot that is freely accessible to all causes it to override its security protocols.

Therefore, if a financial institution wants to implement a chatbot for customer requests, it must make sure to do so on a carefully selected model, test it extensively with success (also for misuse) and monitor it continuously after launch (e.g. to detect if the model needs to be “recalibrated” or if misuse occurs). If AI is used to detect problematic transactions, KPIs of the model must be defined (e.g. accuracy, precision, recall) and measured regularly so that corrective measures can be initiated in time. Is a financial institution able to detect outliers or major errors in the output of its AI?

Guiding principle No. 3: Transparency and explainability

Guiding principle no. 3 combines two topics that have only a limited connection with each other. What both have in common, however, is that they are about classic AI buzzwords: Demanding them sounds plausible at first glance, but on closer inspection it becomes clear that they do not necessarily lead any further in the matter.

First of all, FINMA requires the use of AI to be transparent. This is rightly not to be implemented according to the watering can principle, but in line with the risk and addressee, as FINMA also notes. She is primarily concerned with transparency towards customers, less with other market participants or employees. The aim is to enable customers to assess the risks that affect them when using AI, because they (can) know where and how AI is used in terms of it.

There is nothing wrong with that in itself. The reality, however, is that transparency in the field of AI is usually either a mere fulfilment of duty or an alibi exercise for a clear conscience, but in both cases it has only a limited positive effect. We have all had this experience in connection with data protection declarations: companies today provide much more comprehensive information about their data processing than they did five or ten years ago (because they have to), and yet the data subjects do not really know what happens to their data, even if they are interested in it and read the privacy statements, which hardly anyone does. In addition, they regularly feel powerless: they learn what is happening with their data, but they can’t really do anything about it.

This is hardly any different in the field of AI: FINMA, for example, expects that if an AI answers a client’s questions directly, suggests personal investment decisions or monitors his payment behaviour in order to combat fraud, he should know that it is doing so and on what basis. He will therefore know, for example, that his payments are scanned for suspicious patterns. But he will not understand how this works in detail, and it will not be reasonably possible to convey it to him. He will be able to assess his risks no more and no less than if no AI were used for this, and he will not be able to meaningfully align his behavior accordingly. If he knows that the investment decision comes from an AI instead of a human, this may be subjectively important information for him, but a reliable statement about the quality of the information, for example, will not enable him to do so – if guiding principle no. 2 is followed, it should not make any difference.

The transparency requirement will thus lead to financial institutions being specifically informed when customers are dealing with trained instead of programmed algorithms. However, this will not change their use. However, FINMA does not go as far as the Federal Fiscal Court. Data protection and information officer, who demands transparency about every use of AI. FINMA obviously only expects such a situation where the client is directly confronted with it in a way that is relevant to it. If the AI helps the consultant to better formulate or translate his emails, there is no need to be informed about this. If a bot answers the request without human intervention, however, this should be clear. We have explained in detail how information can be provided and what transparency is required in the field of AI,

Pro memoria: Art. 17 FinSA requires good faith in the processing of client orders, which also includes transparency. Art. 8 FinSA, on the other hand, requires clients to receive certain information about financial services (such as investment advice). According to Art. 7 para. 1 FinSO, this includes information on the nature of the financial service, its characteristics and modes of operation. This may result in a concrete obligation to provide information about AI. However, not every use of AI is special; AI is already being used in many places today without anyone coming up with the idea of demanding transparency in this regard.

FINMA’s second topic, the explainability of the results of AI, should also be treated with a certain amount of caution and not taken literally. First of all, it is obvious to demand that the results of an AI that are used for a decision should be explainable. Nobody wants to trust a black box. However, with some of the advanced methods of AI, we are simply not able to explain why a specific result has been achieved exactly the way it is. We are able to build such systems, we understand their mode of operation in principle, and we are increasingly learning about their behavior through empirical research. However, we do not really understand large language models, for example, in all depths. With other machine learning methods, we could in principle understand every decision down to the last detail, but this can involve so many decision-making steps or calculations that this is no longer practical. That’s why the explainability of the results is wishful thinking in the truest sense of the word, at least in certain AI processes. This does not change FINMA’s demand. It is demanded in many places, but this does not mean that it can be implemented as conceived.

We must therefore interpret and adjust the requirement accordingly in order to be able to implement it sensibly. First of all, FINMA is of course aware of the problem itself. She knows that due to the large number of parameters and the complexity of AI models, the influence of the individual parameters on the result cannot be understood by us humans, at least not today. However, she sees the risk that without an understanding of how an AI arrives at a certain result, a decision by a financial institution based on it can also no longer be explained. If an institution’s decisions can no longer be explained, they cannot be meaningfully reviewed – and the audit firms and FINMA can no longer fulfil their supervisory obligations. This fear is also the subject of headnote no. 1. Not mentioned by FINMA, but just as relevant, is the fact that the institution itself can no longer really control any AI-supported decision-making if it does not know why decisions are made the way they are – by the oracle of AI, so to speak.

The explainability of the results of an (AI) application is thus intended to ensure that decisions based on AI remain comprehensible and thus verifiable. Understood in this sense, explainability does not mean that a financial institution must understand why and how an AI has arrived at a certain or every single result. It is sufficient if the result can be understood and confirmed in some way, even if the explanation is made by way of an alternative justification. It is therefore essential to validate the result. It’s about the question of whether the result makes logical sense. We humans don’t work any differently: when we recognize a certain type of object, we can’t explain why we immediately know what it’s about. But we can deduce afterwards why the object is what was spontaneously clear to us – even if we do this in a different way than our brain instinctively did.

To ensure this, a financial institution can, for example, determine what determines the output of an AI, i.e. which aspects in the input are the drivers in relation to the output. Especially in the case of predictive AI, a so-called sensitivity analysis can be carried out to determine how sensitively the results of a model react to changes in the input variables. This makes it possible to understand which variables have the greatest influence on the model and how uncertainties in the input data influence the predictions. In this way, the robustness and reliability of a model can be assessed and the most important influencing factors can be identified. This not only allows models to be optimized, but also better understood.

So if AI is used to combat fraud and it blocks a customer’s card transaction it is monitoring as suspicious, the question arises as to whether the bank understands which patterns were responsible for checking whether the blocking was justified and can understand the decision. If this is not easily possible, the bank can alternatively show that the circumstances would have justified a block even if considered separately. This does not necessarily have to be the case with every block, because a certain tolerance must naturally be given for fuzzy ratings, but the AI should come to a result that can be justified sufficiently often. To find out, for example, the sensitivity of the AI model to very high and very low transaction amounts as a driver of suspicious activity could be tested, as well as with regard to other parameters. By systematically varying extreme cases, it could be determined how the model reacts – and whether this is comprehensible.

In the case of generative AI, for example, references in the output can contribute to explainability if the AI uses information from certain databases (so-called retrieval augmented generation). It can thus be understood where a certain answer comes from, even if the user may not be able to find out why the AI has chosen this particular content. But he will be able to classify the answer.

An AI result is therefore “explainable” if it can be validated or justified independently of the AI. In this context, it has even been suggested that the principle of explainability should be fulfilled if the result of one AI can be validated by another AI trained on a different basis.

Guiding principle No. 4: Equal treatment

Unjustifiable unequal treatment due to the use of AI is to be avoided. This requirement goes a little less far than it may seem at first glance. To this end, we must be aware that Swiss law does not recognise a general prohibition of discrimination in the private sphere. This is only available in certain areas (e.g. at the workplace). In addition, unequal treatment at financial institutions, for example in the granting of loans, is normal. There is no entitlement to credit, and no entitlement to the same conditions. The principle of equal treatment only applies in narrow constellations, such as when processing client orders (Art. 17 FinSA).

The use of AI (for the time being) should not change this. The guiding principle is formulated accordingly softly: The avoidance of unequal treatment by AI is intended to ensure that the lack of balance of an AI system does not lead to one-sided results and thus unintentionally discriminate against groups of people. So if customers with high potential are to be selected by an AI on the basis of their data in order to make them special offers, carefully curated training material, tests and other measures must be used to ensure that, for example, women or certain nationalities are not automatically rated worse because one-sided data sets have been used for machine learning.

The financial institution may use AI to favor or disadvantage groups of people, but it must do so on the basis of a conscious decision. This is not allowed to be left to AI or chance. AI should therefore not make subjective decisions in financial institutions and should not discriminate on its own initiative or inability.

Implementation in financial institutions

There is no transitional period for the implementation of these guidelines, which is already evident from the fact that they are not binding. Nevertheless, they express FINMA’s expectations and specify general principles for the use of AI, as FINMA believes regulated financial institutions must comply with even without the guidelines it has pre-formulated (even if it is not clear everywhere what the legal basis is).

In concrete terms, this means that institutions must take the usual governance measures, such as issuing appropriate directives, defining AKV and implementing compliance processes.

They should also think about how FINMA’s Guiding Principles (or their own Guiding Principles) can be operationalised in the context of the implementation and review of specific AI projects. The challenge lies not in the principle, but in the concrete operational implementation. Projects with AI will already exist at many financial institutions, i.e. there is a need for action. However, in our experience, only a few institutions have taken a closer look at the concrete implementation of these supervisory expectations, and in our experience, some institutions do not yet have a real overview of all relevant AI activities.

As a first step, we therefore recommend that in addition to appropriate instructions, we primarily carry out a “Map & Track” of the relevant applications, i.e. a survey of the applications in which AI is used in a relevant way. In a second step, these applications should be classified according to a risk scale tailored to the respective institution and finally assessed in a risk-oriented manner.

In addition, FINMA’s “supervisory expectations” will not remain the same: by the end of the year (or the beginning of 2025), the Federal Council will set out in an overview where it sees a need for further adaptation of Swiss law on AI. Furthermore, financial institutions should prepare for the application of the EU AI Act. This has an extraterritorial effect, especially in cases where the output of an AI is also used in the EU as intended. In doing so, FINMA expects Swiss financial institutions to comply with the AI Act insofar as it intends to be applied to AI projects by these institutions.

By Vischer, Switzerland, a Transatlantic Law International Affiliated Firm.

For further information or for any assistance please contact switzerland@transatlanticlaw.com

Disclaimer: Transatlantic Law International Limited is a UK registered limited liability company providing international business and legal solutions through its own resources and the expertise of over 105 affiliated independent law firms in over 95 countries worldwide. This article is for background information only and provided in the context of the applicable law when published and does not constitute legal advice and cannot be relied on as such for any matter. Legal advice may be provided subject to the retention of Transatlantic Law International Limited’s services and its governing terms and conditions of service. Transatlantic Law International Limited, based at 84 Brook Street, London W1K 5EH, United Kingdom, is registered with Companies House, Reg Nr. 361484, with its registered address at 83 Cambridge Street, London SW1V 4PS, United Kingdom.