Regulating_lethal_autonomous_weapon_systems_explor
Uploaded by
adilboulima00Regulating_lethal_autonomous_weapon_systems_explor
Uploaded by
adilboulima00AI and Ethics (2024) 4:229–245
https://doi.org/10.1007/s43681-023-00261-0
ORIGINAL RESEARCH
Regulating lethal autonomous weapon systems: exploring
the challenges of explainability and traceability
Edward Hunter Christie1 · Amy Ertan2,3 · Laurynas Adomaitis4,5 · Matthias Klaus6,7
Received: 4 July 2022 / Accepted: 6 January 2023 / Published online: 21 February 2023
© The Author(s) 2023
Abstract
We explore existing political commitments by states regarding the development and use of lethal autonomous weapon
systems. We carry out two background reviewing efforts, the first addressing ethical and legal framings and proposals from
recent academic literature, the second addressing recent formal policy principles as endorsed by states, with a focus on the
principles adopted by the United States Department of Defense and the North Atlantic Treaty Organization. We then develop
two conceptual case studies. The first addresses the interrelated principles of explainability and traceability, leading to pro-
posals for acceptable scope limitations to these principles. The second considers the topic of deception in warfare and how
it may be viewed in the context of ethical principles for lethal autonomous weapon systems.
Keywords Artificial intelligence · Defence · LAWS · Ethical principles · Explainability · Deception
1 Introduction emergence of lethal autonomous weapon systems (LAWS)
[1–3]. Reviewing the diverse field of disciplines contributing
There is a growing body of literature devoted to the ethi- to the literature of military AI, Taddeo and Blanchard [4]
cal and legal aspects of the use of artificial intelligence found that depending on an organisation’s approach towards
(AI) for military purposes, with a dominant focus on the LAWS, their definitions focus on different ethical and legal
challenges. Distilling various definitions, they suggest defin-
ing LAWS as machines able to adapt to their environment
* Matthias Klaus and change between observation and engagement stances
[email protected]; [email protected] flexibly to identify, select and kinetically attack their targets
Edward Hunter Christie without human intervention. It is this characteristic which
[email protected] distinguishes an autonomous weapon from an automated
Amy Ertan one, which has computational processes speeding up some
[email protected]
Laurynas Adomaitis
[email protected]; [email protected]
1
AI Policy Consulting, Brussels, Belgium
2
Information Security Group, Royal Holloway, University
of London, Egham TW20 0EX, England, UK
3
Emerging Security Challenges Division, North Atlantic
Treaty Organization, Brussels, Belgium
4
CEA-Saclay/Larsim, 91191 Gif‑Sur‑Yvette, France
5
NordVPN S. A., Fred. Roeskestraat 115,
1076EE Amsterdam, The Netherlands
6
Leverhulme Center for the Future of Intelligence, University
of Cambridge, Cambridge CB2 1SB, England, UK
7
BTS Public Sector and Defense, Capgemini, Potsdamer Platz
5, 10785 Berlin, Germany
13
Vol.:(0123456789)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
230 AI and Ethics (2024) 4:229–245
of its functions, but is only able to achieve narrow preset highest degree of military effectiveness would be achieved
goals based on detailed and deterministic prior program- if the LAWS could successfully deceive the adversary, while
ming. As of 2022, the technological evolution is still ongo- remaining within the bounds of IHL and while also posing
ing from older, exclusively pre-programmed systems for no danger to the side that operates it. As these considera-
narrow tasks towards more versatile systems that include tions illustrate, it is relevant to explore in more detail the
greater adaptability, thanks to AI, for some of their functions boundaries of these related concepts.
(e.g. image recognition, voice recognition). In this article,
we will adopt the definition of LAWS offered in Taddeo and
Blanchard [4], while noting that such systems do not exist at 2 Academic research on ethical frameworks
this time (e.g. an unmanned aerial combat vehicle capable for AI in defence
of carrying out an entire combat operation without human
assistance). We further stress our understanding of lethality We make the working assumption that the emergence of
in the context of LAWS as referring to systems intended LAWS will be technically possible and, furthermore, that
for military combat, namely including deliberate strikes on military necessity and economic efficiency will make the
human combatants and on manned military platforms and development and employment of certain kinds of LAWS
vehicles. inevitable [12, 13]. Our chosen focus is on the regulation
One key debate within this literature deals with the ques- of LAWS in the context of warfare, where the behaviour
tion of whether international humanitarian law (IHL) in its of states is regulated by IHL. States pursue their IHL obli-
current form, as the most relevant body of law concerning gations through relevant national guidance documents that
warfare, sufficiently covers the challenges of LAWS [5, 6]. place outer limits on the rules of engagement they adopt
To this end, this article first documents the approaches of ahead of the use of military force. In the absence of new nor-
relevant organisations and nations towards military AI and mative frameworks, only IHL and national rules of engage-
highlights how they interrelate with and refer to IHL. ment will constrain how LAWS are deployed and used [14,
In recent years, several states in the international system 15].
have made official political commitments on the ethical use These core principles apply regardless of the means of
of AI in defence. In particular, it is worth noting the princi- warfare, and therefore also apply to the use of LAWS, as
ples of ethical AI adopted by the United States Department confirmed by states participating in the UN CCW GGE
of Defense (DoD) in February 2020 [7], which were based on LAWS. A challenge for the application of these core
on a report by the Defense Innovation Board [8, 9]. A fur- principles to the case of LAWS is that IHL is targeted at
ther major development was the adoption by all 30 govern- human action and presupposes human intent for an action
ments of the North Atlantic Treaty Organization (NATO) of to become judgeable, something AI systems lack [16]. For
a similar set of principles, as part of the adoption of NATO’s clarity, all existing law is predicated on human responsibil-
first-ever artificial intelligence strategy [10, 11]. Both the ity and accountability, and there is no serious debate at this
US principles and the NATO principles apply to all military time about holding an AI system or a LAWS legally account-
applications of AI, including but not limited to LAWS and able for anything. However, as LAWS may select their own
will be our focus in this regard. courses of action within designated parameters, they may
Following the literature review and outline of relevant display complex and unintended behaviours in the pursuit
organisations and national positions, we will investigate one of military objectives which might threaten compliance with
NATO principle in particular, namely Explainability and IHL principles, such as proportionality.
Traceability, and explore interrelated questions that pertain Even before the World Wars of the twentieth century and
to it relating to transparency, security, and intended versus the concomitant introduction of new ways of industrialised
unintended unpredictability and deception. We note that killing, early forms of IHL like the St. Petersburg declaration
they are technically linked with certain types of AI, nota- attempted to regulate warfare with regard to proportional-
bly machine learning. More specifically, a machine learn- ity [17]. Later on, more specific documents followed suit
ing algorithm whose parameters are fully observable by by completely banning the use of certain weapon systems.
all parties would be explainable and traceable, and thus be This includes biological and chemical weapons, as they
compliant with the principle. However, its behaviour will could not be employed in proportional and distinct manner,
also be—in principle—predictable for all parties, meaning among other reasons also involving humanity.1 Similarly,
it would violate security needs and offer opportunities to ‘dumb’ weapons like cluster ammunition employed during
the adversary to predict its actions and gain an unwanted
military advantage. For purposes of military effectiveness,
a LAWS should be unpredictable for the adversary, and pos- 1
See the 1972 Convention for example: https://www.un.org/disar
sibly for the party that operates it, within certain limits. The mament/biological-weapons.
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
AI and Ethics (2024) 4:229–245 231
nightly air bombings came under scrutiny [18], which con- In sum, LAWS are a challenge for responsibility and
tributed to the development of ‘smarter’ alternatives like accountability under IHL, as they could replace humans in
GPS or laser precision-guided munitions. These weapons both action and planning. Even if a human is in-the-loop,
can enable targeting with pinpoint accuracy, which con- the amount of data and necessary speed of decision-making
tributes to their higher IHL compliance. The precision of severely limit human oversight [6, 13, 21]. The question of
these systems, together with qualities such as adjustable fuse responsibility becomes even more pressing when there are
settings, allows users to achieve their military goals with no humans in- or on-the-loop. Arkin [14, 22] argues that the
minimal force, thus reducing the exposure of the target’s sur- responsibility gap can be bridged by assigning responsibil-
roundings to military action. Nevertheless, operator errors or ity advisors along the whole process, ranging from design
mechanical failures can and have resulted in many publicised to employment.
incidents where supposedly ‘smart’ bombs caused collat- Based on these discussions, the ICRC and SIPRI argue
eral damage, i.e. damage to unintended, often illegitimate that a key necessity for LAWS in war is human control
targets. LAWS have the potential of constituting another [26, 27], which is echoed by other authors [19, 28, 29].
step forward in ensuring greater accuracy and less collat- Related literature highlights the importance of “meaning-
eral damage, meaning greater compliance with IHL, most ful human control”, or MHC, of AI systems in military
notably with the principles of distinction and proportional- operating environments [30, 31]. MHC consists of the role
ity. LAWS could employ precision munitions autonomously of the human operator as a fail-safe to ensure ongoing
and process information faster than any human operator [13, compliance with IHL, to ensure ongoing accountability
19], but their opaqueness and unpredictability in unforeseen for possible breaches of IHL, and to ensure human moral
situations pose different kinds of risk [20, 21]. agency remains involved with any actions taken by LAWS
Many authors focus on the implications of LAWS with [32]. Roughly, there seems to be two schools of thought
respect to the IHL principles of proportionality and distinc- regarding the demands IHL has towards human control
tion [6, 19, 22, 23]. Under IHL, human commanders are of LAWS. One school of thought argues that technologi-
expected to demonstrate the reasonableness of their attack cal development can overcome the existing challenges to
decisions to explain and justify their conduct. The black box MHC, as improvements in the field of AI will allow for
problem, that is, the opaqueness of modern machine learning systems to adhere ever more closely with IHL principles
algorithms, which prevents any human from understanding while retaining the advantages of faster decision-making
their decision-making process, poses severe ethical problems speed and objective decision-making processes [14, 20,
in this regard, as one cannot predict how an AI would pursue 24, 33]. Assuming further advances in AI, one could imag-
its task [20]. The lack of ‘intent’ of AI systems contributes ine machines being better at complying with IHL than
to this dilemma, as it cannot be proven that an opaque and humans, although this is as yet a distant prospect [34].
indifferent AI system would act in good faith, as required The other school of thought argues that IHL should be
by the proportionality principle. Some authors [13, 24, 25] interpreted as requiring inherent limits to the autonomy
suggest that narrowing the freedom to manoeuvre could help of LAWS, because IHL principles can only be fully met
overcome these challenges. It could involve controls on the based on contextual and ethical judgements by humans [6,
weapon system’s parameters to restrict targeting and tasks 12, 26]. Also, LAWS should not be employed unless MHC
to only certain target sets like navy vessels, on the environ- can be enforced to close the gap of responsibility [35].
ment of use like the open sea, and requiring human–machine While human–machine integration is proposed as an effi-
integration that ensures human control via in-the-loop or cient way to mitigate many existing concerns about MHC,
on-the-loop oversight. These concepts describe the level of human operators could suffer from various biases [21].
integration of the human operator into the functioning of the While automation bias may result from too much trust in
weapon system. In-the-loop means that the operator has to sophisticated AI [23], the reverse can be true due to the
confirm any potential attack, while on-the-loop allows the inherent opaqueness of these systems [20]. Sullins [36]
system to attack autonomously with the human supervis- raises the notion that AI will need to learn and understand
ing in the role of fail-safe. Finally, off-the-loop (or out-of- deception in warfare to compete with human counterparts
the-loop) describes a system operating fully autonomously who use these tactics routinely. This is a dilemma in itself,
without a human supervising its actions. The ICRC also as it presupposes humans building machines with traits we
recommends developing practical human control measures, consider unethical. Focusing on how AI could be used to
coupled with internationally agreed constraints on LAWS facilitate unacceptable military deception, Chelioudakis
based on IHL, and clarification where new rules might need [37] found IHL was flexible enough to account for, and
to be developed [26]. Müller [21] echoes this by pointing remain unchallenged by, deceptive AI machines available
out that the distinction between military and civilian vessels at the time. We will explore the topic of deception in more
could be easier than between soldiers and non-combatants. depth later in this paper. Arkin [22] also agrees that IHL
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
232 AI and Ethics (2024) 4:229–245
Table 1 Core principles of IHL
Core principle Definition
Proportionality Incidental damage to civilian structures and injury or death of civilians must not be excessive with regard to concrete and
direct military advantage
Precaution Those who plan or decide upon an attack shall take all feasible precautions. All parties must verify their targets and warn the
civilian population before attacking, unless circumstances do not permit. Further restrictions on attack timings, location
and other characteristics can become necessary
Distinction Distinguish at all times between civilians and civilian objects—and combatants and military objectives, only the latter two
may be targeted. Prohibits indiscriminate attacks
Military necessity Allows for measures to achieve a legitimate military goal which are not prohibited by IHL. Needs to be balanced against
humanitarian concerns to prevent unnecessary suffering
Humanity In cases not covered by IHL, persons affected by armed conflicts will still be protected by the laws of humanity and public
conscience (“Martens Clause”)
Source: ICRC (See https://casebook.icrc.org/glossary/fundamental-principles-ihl) and Geneva Protocols (See https://www.icrc.org/en/doc/war-
and-law/treaties-customary-law/geneva-conventions/overview-geneva-conventions.htm)
is sufficient to constrain LAWS, arguing that while LAWS and suitable set of objectives that future LAWS should be
could act unethically, they would still perform more ethically in compliance with. The heart of the debate among experts
than human soldiers. However, lacking experience with the revolves around additional guidance or rules that would be
(disruptive) application of AWS [14] makes it unlikely that specific to LAWS and that would further specify how to
LAWS could comply with IHL at the current stage. This is ensure that LAWS are developed and used such that IHL
in contrast to Anderson, Reisner, and Waxman [38], who compliance is both enabled and facilitated. We will address
propose adapting IHL to account for greater autonomy on the positions of selected governments and international
the battlefield. organisations in the next section.
One should also make note of interpretations of how Autonomous systems are intended to be able to navigate
the IHL principle of humanity should apply in the case complex and uncertain environments and to determine, by
of LAWS. For some authors, it does not seem possible to themselves, successful courses of action to fulfil their mis-
respect human dignity in the case of a LAWS which would sions. The courses of action a future LAWS will choose under
take a kill decision based on a black box algorithm [39]. real battlefield conditions will have elements of unpredict-
A counterpoint to this view could be made from coun- ability for any human observing it, including its human com-
terfactual analysis. A putative high-performance LAWS mander—much like a soldier should be able to improvise on
with a track record superior to humans in respecting the the spot to pursue mission objectives, sometimes in ways that
IHL principles of distinction and proportionality could be could surprise its commander. The AI software that will enable
thought of as being in better compliance with the princi- a future LAWS to complete missions successfully is likely to
ple of humanity from a utilitarian perspective, e.g. less be highly complex and not amenable to simple technical stand-
collateral damage, faster and more effective fire against ards that could guarantee compliance with IHL while allowing
legitimate enemy targets, and self-restraint for reasons of the AI software to have the required degree of versatility. There
proportionality. is no guarantee at this time that future, highly advanced AI will
Overall, the need for various kinds of human control be developed to have such a high degree of semantic and con-
during both the design and use of LAWS is a clear insight textual understanding that software could simply be instructed
from existing literature. Assuming that LAWS would use AI to “understand IHL” and comply with it at all times. Assuming
based on existing classes of machine learning algorithms, current or near-term foreseeable technologies, the design of a
the latter’s inherent lack of contextual understanding and LAWS will require various forms of MHC, and the use of a
black box characteristics leave very few scenarios in which LAWS will require human operators in- or on-the-loop in all
LAWS could operate fully on their own. We recognise that scenarios in which autonomous operation presents a significant
this situation could result in a military disadvantage for a risk to IHL compliance. As a result, and as we will see in the
state that takes IHL compliance seriously, but that may face next section, commitments and practical work currently being
an opponent who develops or uses LAWS with a low level undertaken by several states go much beyond binary questions
of regard for IHL. about the applicability of IHL, that question being already set-
As reviewed above, available literature has evolved tled positively, and towards additional principles specific to
towards a general acceptance that the existing five core LAWS, or to military applications of AI more broadly, as well
principles of IHL shown in Table 1 are the most relevant as towards processes to operationalise such principles.
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
AI and Ethics (2024) 4:229–245 233
3 Official positions of national governments maximum extent of agreed limitations on LAWS are the
lowest common denominator between national positions. As
The relevant intergovernmental forum for the negotiation of 2019, therefore, the trajectory for an international agree-
of norms that could place limits on the development or use ment seemed set on merely affirming IHL, with flexibly
of LAWS is the group of governmental experts on lethal worded political commitments on human control. There was
autonomous weapon systems (GGE LAWS) that operates in no sign, then, of any possibility of having the five permanent
the framework of the Convention on Certain Conventional members of the UN Security Council, in particular, agree-
Weapons (CCW). The CCW itself is an international treaty ing to a consensus on wide-ranging prohibitions on either
and a set of additional protocols that set out norms and lim- the use or the development of certain kinds of LAWS. We
itations on certain kinds of conventional weapons. As of attribute that pattern to a combination of two factors: rival
2022, 126 states are parties to the CCW,2 including the five major military powers, being in a state of mutual distrust,
permanent members of the Security Council (USA, Russia, wish to retain flexibility to explore the military advantages
China, UK, France), most other significant military powers, they may each achieve—bearing in mind that LAWS have
e.g. India, Pakistan, Israel, Saudi Arabia, all members of the the potential, through greater accuracy and speed, of being
G20 group of the world’s largest economies (except Indo- superior to non-autonomous equivalents both in terms of
nesia), all members of NATO, all members of the European their effects on opposing forces as well as in terms of the
Union, and all Latin American countries. Overwhelmingly, lesser danger they may pose to one’s own forces (in great
those states that are not party to the CCW are nations in contrast to chemical or biological weapons, which do not
Africa, the Caribbean, the Middle East, and Southeast Asia. have such characteristics and which are prohibited).
From a hard security perspective, the most notable non- By 2022, however, certain national positions had evolved.
parties are Iran and North Korea. The GGE LAWS started Two groups of countries, the USA, the UK, Korea, Japan,
operating in 2014, with a majority of state parties participat- and Australia, on the one hand, and Argentina, Costa Rica,
ing. A pivotal development was the adoption of 11 guiding Guatemala, Kazakhstan, Nigeria, Panama, Philippines,
principles that were proposed in the conclusions of a 2018 Sierra Leone, State of Palestine, and Uruguay, on the other,
report of the GGE LAWS [58] and adopted by consensus each submitted a joint paper [78, 79] to the GGE LAWS that
by participating state parties at the group’s November 2019 had in common proposals to prohibit four potential types of
meeting [59]. LAWS. Using the wordings from the first paper, these are:
The first 2 of the 11 principles are the most foundational.
The first affirms that IHL “continues to apply fully to all (1) LAWS “of a nature to cause superfluous injury or
weapons systems, including the potential development and unnecessary suffering [or] if it is inherently indiscrim-
use of” LAWS. The second affirms that “human responsi- inate, or if it is otherwise incapable of being used in
bility (…) must be retained since accountability cannot be accordance with international humanitarian law”.
transferred to machines”. The other principles derive from (2) LAWS “designed to be used to conduct attacks against
the first two principles and address, in broad terms, the need the civilian population, including attacks to terrorize
for measures addressing both the development and the use the civilian population”.
of LAWS, including the desirability of risk assessment, risk (3) LAWS “designed to cause incidental loss of civilian
mitigation, and security measures (including cybersecurity). life, injury to civilians, and damage to civilian objects
The topic of human control is addressed in the third princi- that would invariably be excessive in relation to the
ple under a heading of “human–machine interaction” which concrete and direct military advantage expected to be
stresses IHL compliance but does not spell out specific limits gained”.
based, e.g. on certain functions being necessarily subject (4) LAWS in which the autonomous functions are designed
to in-the-loop versus on-the-loop human supervision. Over- “to be used to conduct attacks that would not be the
all, the 11 principles do not constitute new legally binding responsibility of the human command under which the
obligations but a consensus among state parties regarding weapon system would be used”.
what they each commit to uphold in their national practices.
The principles also do not open up any prospects for an It can be argued that the four types defined above neces-
international monitoring or verification regime for LAWS. sarily derive from applicable IHL and from the second guid-
As agreements among state parties are by consensus, the ing principle adopted in 2019, namely that accountability
cannot be transferred to a machine. Nevertheless, explicit
prohibitions provide for greater legal clarity and have a com-
2
For the list of state parties, see: https://www.un.org/disarmament/ mitment value between states as well as towards populations
the-convention-on-certain-conventional-weapons/high-contr acting- and civil society.
parties-and-signatories-ccw/
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
234 AI and Ethics (2024) 4:229–245
On the other hand, the aforementioned paper by the USA have fully autonomous LAWS, they may develop such capa-
and some of its allies effectively allows for the potential use bilities if US competitors choose to do so [84].
of a LAWS that could autonomously engage military targets In June 2022, the UK published its Defence Artificial
in accordance with IHL, that is, without necessarily having Intelligence Strategy [44] alongside corresponding policy
a human in-the-loop. paper which set out five ethical principles for defence:
While other national submissions made to the GGE human-centricity; responsibility; understanding; bias and
LAWS in 2022 also contained important elements, the over- harm mitigation; and reliability [45]. Both documents show-
lap between the two joint papers described above represents case a range of commitments concerning the development of
the strongest potential to date for agreed prohibitions on the responsible AI tools while stressing the CCW as the primary
use of certain kinds of LAWS. avenue for discussions on LAWS and IHL. The Strategy
In parallel with the potential future development of an reinforces the position set out in the UK Ministry of Defence
intergovernmental agreement, which could take the form of Unmanned Aircraft Systems doctrine that autonomous or
a new protocol under the CCW, states have also been work- remote-controlled systems must be operated in accordance
ing on ‘soft law’, such as national guidelines and principles with existing domestic and international legal frameworks,
that are not legally binding, at the national level as well as at including IHL [44, 46]. The policy paper also reaffirms,
the NATO level for those states that are members of NATO. word for word, the statement set out in this doctrine that
Soft law approaches are important in that they can provide “the UK does not possess fully autonomous weapon systems
more detailed guardrails to structure national activities than and has no intention of developing them” [45, 46]. Article
what states may be comfortable agreeing to in a legally bind- 36 of the 1977 Additional Protocol I to the Geneva Conven-
ing convention or treaty. tions (hereafter: Article 36) is interpreted broadly by the UK,
The USA was the first state actor to release a defence- to include considerations of how weapons are used and the
specific AI strategy, with the DoD publishing the strategy’s conduct of warfare, in which “weapon reviews may record
executive summary publicly in February 2019 [8, 9]. This limitations on the use of a weapon or method of or means of
was followed in 2020 by the DoD adoption of five ‘AI Prin- warfare in order to ensure compliance” [47].
ciples’, which outline that deployed AI must be: responsi- France hosts a Defence Ethics Committee within its Min-
ble, equitable, traceable, reliable and governable.3 These istry for Defense which in 2021 set out their view on the
principles apply “to all DoD AI capabilities, of any scale, integration of autonomy into LAWS in a committee report
including AI-enabled autonomous systems” as confirmed [48]. In the report, the committee argued that while fully
via a Biden administration memorandum [40]. While the autonomous weapons are ethically unacceptable, reaffirm-
memorandum confirmed the focal point for the development ing French national position since 2013, partially autono-
of responsible AI infrastructure would be the Joint Artificial mous weapons systems may be ethically acceptable subject
Intelligence Center (JAIC), from 1 February 2022, the office to defined ‘5C’ conditions: command, risk control, compli-
of the newly established Chief Data and AI Officer has taken ance, competence, and confidence [48, 49]. Alongside such
on these responsibilities. While there is no all-encompass- conditions, the committee recommends that a complete legal
ing national guidance regarding LAWS, the US Directive review is conducted wherever decision-making autonomy is
3000.09 on Autonomy in Weapons Systems [42] sets out a developed in a lethal weapons system [48]. French defini-
framework for using LAWS in ways that are consistent with tions equate the UN definition of LAWS to fully autonomous
IHL and any applicable treaties and rules of engagement systems and France has concisely rejected incorporating this
(RoE). The Directive states that LAWS must be designed so form LAWS into military operations [48, 49].
commanders and operators are able “to exercise appropri- China has published several documents relating to AI
ate levels of human judgement in the use of force”. While governance (not specifically for defence), including a set of
the Directive is due to be updated (as of late 2022), it is not principles with ‘Chinese characteristics’ like harmony [51].
expected to change significantly in form [83]. Consistently, In 2019, China established a National Ethics Committee
the US view to date has been to favour a dispersed model of on Science and Technology to supervise the regulation of
human judgement, whereby humans do not necessarily have AI in general. Commentators have pointed out that China’s
to be in charge at the specific moment of engagement, but at definition of LAWS is ambiguous, potentially allowing for
crucial moments throughout the process [41, 43]. Senior US machines that could not be deactivated or that could use
officials have stated that while the USA does not currently force indiscriminately [53]. In a position paper on military
AI regulation released in early 2022, China refers to the
broader need to manage potential risks, but sets out no spe-
3 cific commitment or initiatives that suggest the development
For more detailed description of the Ethical Principles, see: https://
www. d efen s e. g ov/ News/ Relea s es/ Relea s e/ A rtic l e/ 2 0919 9 6/ d od- of national laws, rules or regulations for LAWS [85].
adopts-ethical-principles-for-artifi cial-intelligence/.
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
AI and Ethics (2024) 4:229–245 235
Russia has argued that IHL in its current form is sufficient 5 Interpretation and meaning:
for regulating LAWS [54] and has actively and consistently explainability and traceability
engaged in GGE discussions to oppose the establishment of
legal binding instruments relating to LAWS [86]. Russia fur- Both NATO and DoD frameworks include traceability
ther contends that “excessive regulation can hamper the pace among their principles of ethical and responsible use. NATO
of development of EDTs, including AI” [55], and argues in principles also mention explainability in addition to trace-
favour of LAWS over human controlled machines due to ability. In civilian ethical guidelines for AI, traceability and
their overall better performance, especially with regard to explainability correspond to transparency, which is one of
IHL compliance. Moreover, Russia commits itself to Article the most frequently mentioned principles [60]. However,
36, dealing with responsibility and accountability, arguing transparency is an ambiguous term. Therefore, it will lend
that universalising the Article would suffice in contrast to our further discussion to carefully distinguish and define
specifically designed legal review measures [56]. This has traceability, explainability, and transparency.
been interpreted as an attempt to narrow international regu- Transparency is commonly understood as having and
lation with the goal of continuing domestic development revealing information about internal processes of a pub-
unabated [57]. lic institution, a company, or other enterprise. This type of
transparency is often considered a virtue that lends to fight-
ing corruption, enacting accountability, and enhancing trust.
4 Soft law The most commonly found definition of transparency relies
on an enterprise’s responsibility to make some information
NATO’s first AI Strategy was agreed by Allied Defence publicly available. It is usually formulated from the sender’s
Ministers in October 2021, with a public summary docu- (enterprise’s) perspective without involving the responsibil-
ment outlining the Alliance’s aim “to lead by example and ity to ensure the receiver (the public) is actually informed
encourage the development and use of AI in a responsible [61].
manner” [10]. The Strategy contains a set of “Principles of
Responsible Use” as listed in Table 2. The principles were Strict transparency: a process is transparent if informa-
closely modelled on the DoD principles and other existing tion about the process is made publicly available.
national principles, and apply to all kinds of AI applications We shall call this definition strict to distinguish its explicit
that are intended for deployment (ibid). part (making the information public) from its implicit parts
Existing national soft law commitments and principles (that the information exists and what information it is). Strict
are rooted in considerations of existing IHL and of the need transparency simply captures the imperative to share infor-
to always maintain human accountability, but they have the mation with the public or the stakeholders. It represents the
potential to go further and to be more granular than inter- necessary condition of transparency, for an enterprise that
governmental agreements at the UN. As of 2022, the US has information but does not make it publicly is not gener-
and NATO principles, which are highly similar, provide ally considered transparent.
guidance for ongoing applied work at national level, which However, in some AI applications, the information about
can be expected to take the form of more detailed national the functioning of an algorithm can be unattainable. For
(and NATO) manuals, processes, and standards. We see example, black box systems are unexplainable. An enterprise
these processes developing, for example, through the Octo- could reveal the fact that a black or grey box algorithm is
ber 2022 releases of the NATO Autonomy Implementation used, but they cannot explain how a decision process is car-
Plan [81] and announcement regarding the creation of the ried out. Also, strict transparency is traditionally formulated
NATO Data and Artificial Intelligence Review Board [82], from the sender’s perspective, which presupposes that the
both of which focus on operationalising NATO’s Principles receiver will find the information understandable. However,
for Responsible Use as set out in Table 2. It is with such many AI applications can often provide only a quantitative
ongoing developments in mind that we wish to pose more reason why certain inputs and outputs are correlated. Such
detailed questions regarding one NATO principle in par- explanations do not translate into a semantic explanation for
ticular, namely that of explainability and traceability, owing the stakeholders.
to our estimation that it carries inherently more complex Two notions that address the issues above are explaina-
implications than the other NATO principles. bility and traceability. Traceability means that certain out-
puts from an AI algorithm can be traced to certain inputs,
as if going back in the decision chain. Traceability pro-
vides the ability to find a responsible input A for the even-
tual output B. From an ethical point of view, traceability
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
236
13
Table 2 NATO principles of responsible use of artificial intelligence in defence
Principle Definition Comparison with US DoD principles
Lawfulness AI applications will be developed and used in accordance with national and The preamble text to the DoD principles states that they will “build on the
international law, including international humanitarian law and human U.S. military’s existing ethics framework based on the U.S. Constitution,
rights law, as applicable Title 10 of the U.S. Code, Law of War, existing international treaties and
longstanding norms and values”
Responsibility and accountability AI applications will be developed and used with appropriate levels of judge- Equivalent to the DoD’s “Responsible” principle. The linkage with the notion
ment and care; clear human responsibility shall apply to ensure account- of accountability is made explicit. The term “human” is included for clarity;
ability however, the DoD principle implies it through the use of the term “person-
nel”
Explainability and traceability AI applications will be appropriately understandable and transparent, includ- Equivalent to the DoD’s “Traceable” principle. A clarification is added that
ing through the use of review methodologies, sources, and procedures. This relevant work may occur at the NATO level, or nationally, or in some combi-
includes verification, assessment and validation mechanisms at either a nation of the two
NATO and/or national level
Reliability AI applications will have explicit, well-defined use cases. The safety, secu- Equivalent to the DoD’s “Reliable” principle. Relevant work may occur at
rity, and robustness of such capabilities will be subject to testing and assur- the NATO and/or national levels. The notion of certification is explicitly
ance within those use cases across their entire life cycle, including through mentioned
established NATO and/or national certification procedures
Governability AI applications will be developed and used according to their intended Equivalent to the DoD’s “Governable” principle. An explicit mention is added
functions and will allow for: appropriate human–machine interaction; the to human–machine interaction
ability to detect and avoid unintended consequences; and the ability to
take steps, such as disengagement or deactivation of systems, when such
systems demonstrate unintended behaviour
Bias mitigation Proactive steps will be taken to minimise any unintended bias in the develop- Equivalent to the DoD’s “Equitable” principle. An explicit mention is added to
ment and use of AI applications and in data sets the role of data sets
Source: [80]
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
AI and Ethics (2024) 4:229–245
AI and Ethics (2024) 4:229–245 237
can be significant in ascribing responsibility and predict- desirable but that does not mean they should be necessary
ing future behaviour (governability). However, knowing or absolute.
that A and B have been correlated does not eo ipso give Explainability is de facto not absolute in the state of the
an explanation of why they were correlated. Explainability art for most deep learning algorithms. Many successful
refers to the ability to provide a semantic expression (as applications of deep learning do not require explanations,
opposed to merely quantitative and operational) to why including military applications [65], and enforcing explain-
decision processes developed in a certain way. It is impor- ability constraints may decrease performance. There are
tant to note that semantic expressions are not necessary for scholars who claim that deep learning algorithms are inher-
AI’s functioning [62]. They need to be superadded to the ently not explainable, thus ascribing semantic explanations
technical characteristics. to them is at best a plausible story that cannot be proven
[51]. LAWS require AI applications in computer vision,
Traceability: a process is traceable if certain outputs
robotics, and decision systems, which often depend on deep
can be correlated to certain inputs.
neural networks, and thus are not currently explainable.
Explainability: a process is explainable if (1) it is If the requirement of explainability would be absolute,
traceable and (2) the correlation can be given a it would impede the use of LAWS. However, advanced AI
semantic expression. systems can be much more efficient than deterministic mod-
els in real world applications, so whichever party is using
Strict transparency, traceability, and explainability are
unexplainable deep learning in LAWS can have a significant
conceptually related, together implying a broader notion
advantage. Therefore, an expectation for the absoluteness of
of full transparency, encompassing both the implicit prem-
explainability is not compatible with the use of LAWS and
ises about the attainability of information and its sharing.
produces a military disadvantage.
Traceability is necessary, but not sufficient for explain-
In view of this, NATO principles rightly consider only
ability. Explainability is necessary, but not sufficient for
an “appropriate” level of explainability and do not require
full transparency. Full transparency is implied by explain-
absolute explainability, therefore implying that algorithms
ability and strict transparency.
do not have to be absolutely explainable or traceable. They
Although civilian ethical guidelines focus on the disclo-
need to be explainable to a certain degree. The details of the
sure of information, i.e. Strict Transparency, the defence
particular level of explainability are ingrained in the valida-
sector uses classified information that is not generally
tion procedures but are undefined in the public documenta-
released to the public, so strict transparency cannot be a
tion. The text claims “AI applications will be appropriately
requirement. Therefore, NATO and DoD principles do not
understandable”, which seems to imply that an algorithm
include “transparency” among the principles for respon-
would need to pass some benchmark, so traceability or
sible use. It is replaced by traceability and explainability.
explainability must be higher than zero for responsible use.
Below, we will focus on the latter two.
If this is true, then NATO embraces the necessity but not
The principal reason for wanting traceability or explain-
absoluteness of explainability for responsible use.
ability in LAWS is the enabling function of these princi-
However, the wording leaves room for discussion.
ples for the other areas of responsible use. If there were
“Appropriately understandable” may allow that for some
no traceability of inputs and outputs in the algorithm, one
algorithms the appropriate level of understandability is zero.
could not determine the actors in a decision chain, which
We suggest this is a reasonable proviso to include in the
makes accountability difficult to enforce [63]. Bias mitiga-
guidelines. The necessity of explainability would rule out
tion requires explainability to identify the bias groups and
the use of black boxes, which can be detrimental to perfor-
know that a decision was made because of the bias [64].
mance and responsible use.
Traceability is the key to governability because the latter
In the case of LAWS, where lethal force is involved, there
relies on predicting future outputs, but that means knowing
seems to be an intuitive need for explaining every step of the
that a certain input will produce a certain output, which is
decision chain [66]. Traceability and explainability feel cru-
exemplified in CS8 and CS9 above.
cial to find the accountable decision makers and to explain
The fact that traceability and explainability facilitate the
why an accident has happened. A semantic explanation feels
operationality of other principles makes them a justified
necessary for anthropocentric reasons. A term “semantic
addition to the guidelines on LAWS. However, we shall
anthropocentrism” [67] has emerged in the literature to
pursue a critical analysis of the limits of their implemen-
describe the need for human explanations from other intel-
tation. The main issue to consider is whether these prin-
ligences, including AI, although semantic explanations often
ciples should be held absolute and necessary for LAWS.
rely on borrowing concepts from human and animal cogni-
We have already established the reasons for them being
tive science and their application to domains like LAWS can
only be metaphorical. Contrary to the intuition, we propose
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
238 AI and Ethics (2024) 4:229–245
three main arguments against the necessity of explainability extremely poorly after an analysis.5 Since current innovation
and in favour of the use of black boxes in LAWS. in AI robustness mostly takes place in public research, and
the turnover of attacks and defences is very short, it is dif-
5.1 Accuracy argument ficult to create cyber defences for LAWS that could not be
revealed by open source intelligence [69]. Even if LAWS
Firstly, there are significant use cases where black boxes were equipped by completely classified robustness measures,
are more accurate than xAI. 4 Scholars generally agree military espionage and congeniality (the likelihood of find-
that “there is clear trade-off between the performance of a ing a similar solution independently by an adversary) would
machine learning model and its ability to produce explain- remain a significant threat. Deep learning algorithms are not
able and interpretable predictions” [68]. Semantic expla- immune to adversarial attacks; however, they avoid produc-
nations cannot be given for the decisions of deep learning ing explainability metrics that would involve analysing and
algorithms because of their complexity. The trade-off for visualising the architecture of LAWS. If this explainability
the higher accuracy is very low or non-existent explainabil- information were to leak, it would lead to a significant threat
ity. If the necessity of explainability was enforced in the to LAWS security. Avoiding producing this information in
NATO principles, this trade-off would be seen as unaccep- the first place can be a means of avoiding security by obscu-
table. However, in the use of LAWS, where the stakes often rity, i.e. relying on safeguarding technical information from
involve lethal consequences, the trade-off of explainability adversary intelligence. At the same time, appropriate testing
for accuracy seems reasonable. Accuracy in LAWS can and benchmarking techniques need to be applied to ensure
translate into a successful military operation instead of an the accuracy and robustness of black boxes while treating
unsuccessful one or a defeated enemy instead of a civilian them as such [70].
casualty. Thus, in trading off accuracy, one would be creat-
ing an increased danger of unintended lethal damage. The 5.3 Trust argument
gain of explainability from this trade-off is not considered a
sufficient justification because its benefits are conceptual and Lastly, contrary to common opinion, black boxes can elicit
not material, i.e. explainability helps to understand why an trust. It is commonly held that an explanation of a mecha-
error happened after the event. The potential negative effect nism is necessary for trusting a system [71]. However, it is
of this trade-off (decreased accuracy) could have a com- not always the case, and there is another source of trust—
pounding nature, where one less successful operation leads practical value. Black box algorithms have been used in
to an advantage for the adversary that can cause more lethal high-risk applications, including military, healthcare, and
damage to the ally. Moreover, in scenarios where both adver- criminal justice. These systems often function with human
saries use LAWS, the one with more accuracy will likely oversight but without being explainable. Practitioners in
prevail which can result in a military loss. The trade-off of these fields (e.g. radiology) rely on them for expertise. If
accuracy for explainability is problematic in LAWS, because a system continually produces accurate predictions, it is
the impact of accuracy metrics is material and lethal, and the bound to elicit trust. The emerging trust is expected to pri-
advantages of explainability are only conceptual. marily appear in people who work directly with the systems.
Researchers may remain sceptical for theoretical reasons,
5.2 Secrecy argument the public may remain critical because of ideological rea-
sons but a functioning system will eventually elicit trust
Secondly, the robustness of an AI system can rely on a in the practitioners. This is observed in GGE conclusions,
degree of opacity that prevents malicious actors from suc- namely, “if the automated assessment has a very low rate of
cessfully reverse engineering it. There are principles in cryp- “false positives” […], the operational context corroborated
tography, e.g. Kerckhoffs's principle, which require crypto- the automated assessment, and the context involved combat
graphic systems to be safe even if all information about the operations, then it would seem to be reasonable to rely on
system is public, and in fact the development of AI systems the assessment [and] strike the target.”6 Accuracy simply
can heavily depend on open source material. However, mili- implies reasonable trust in an operational context.
tary technology can require secrecy. Many of the robustness Many researchers that take a critical stance towards trans-
measures in AI safety only work well when the adversary parency practises, even when it is to improve them, face the
does not know about the use of these measures and they fare morally partial connotation of the notion of transparency. As
Lord has put it, “transparency comes loaded with normative
5
4
Some research in xAI claims that this trade-off is a myth, but See https://www.robust-ml.org/defenses/.
6
clearly the myth is the universalised version of this claim that all xAI See https://geneva.usmission.gov/2021/08/05/u-s-statement-at-the-
has this trade-off. The latter claim is not required for our argument. gge-on-laws-during-the-discussion-of-agenda-item-5d/
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
AI and Ethics (2024) 4:229–245 239
baggage. Like security, it is hard to be against transparency. an adversary (…) but which infringe no rule of international
Who is in favor of concealment or censorship?” [72]. How- law (…) and which are not perfidious (…) [such as] the use
ever, NATO and DoD principles show how accountability, of camouflage, decoys, mock operations and misinforma-
governability, and other ethical principles can be enabled tion”. More granular lists of examples of ruses of war can be
without the use of Strict Transparency or Full Transparency. found in guidance manuals produced by individual nations
The terms seem almost superfluous for the ethical analysis in for their armed forces. Such lists are highly similar between
defence. We have argued that even the substituting principles countries (ICRC, n.d.). For example, the relevant United
of traceability and explainability should not be considered as States field manual lists “surprises, ambushes, feigning
absolute or necessary, but only as desirable. The framework attacks, retreats, or flights, simulating quiet and inactivity,
for responsible use of LAWS should leave room for the use use of small forces to simulate large units, transmitting false
of black boxes, although a full set of validation techniques or misleading radio or telephone messages, deception of the
needs to support their deployment. enemy by bogus orders purporting to have been issued by
However, in addition to accuracy requirements, which the enemy commander (…) dummy guns and vehicles (…)
seem to override the imperatives for explainability, there dummy airfields”, among many others [75].
are links between explainability and governability that may Of notable importance for the boundary between ruses
present a different case. If LAWS are indeed autonomous, and perfidy are the examples given under Art. 37(1), namely
they will have to come up with winning strategies autono- “feigning of an intent to negotiate under a flag of truce or of
mously, i.e. without explicit human input. Winning strate- a surrender”, “feigning of an incapacitation by wounds or
gies will be decided based on metrics, which in warfare is sickness”, “feigning of civilian, non-combatant status”, and
often understood as surviving and eliminating targets. The “feigning of protected status by the use of signs, emblems or
ethically challenging case is that of AI algorithms attaining uniforms of the United Nations or of neutral or other States
their metrics by employing manipulation. For example, we not Parties to the conflict”. In addition, it is “prohibited
know that chatbot algorithms competing against each other to make use of the flags or military emblems, insignia or
can learn “to deceive without any explicit human design, uniforms of adverse Parties while engaging in attacks or in
simply by trying to achieve their goals” [73]. Thus, there is order to shield, favour, protect or impede military opera-
reason to believe LAWS would learn deceptive strategies to tions” (Geneva Additional Protocol 1, Art. 39(2)). However,
defeat the adversary. However, deception strategies in war- while using one’s correct national insignia, it is permissible
fare are specific and regulated under IHL. How can we make to remove unit identifications from uniforms [75].
sure that these regulations will be observed by the winning Further relevant considerations concern the markings
deception strategies invented by LAWS? This task seems to that must be placed on military equipment. For the case of
require the understanding—and explanation—of how they unmanned aerial vehicles, Piatkowski [76] notes that “while
come up and enact such strategies. the use of improper [physical] markings is prohibited and
might in some circumstances be tantamount to perfidy,
the use of [a] false IFF signature [electronic identification
6 LAWS and military deception: a thought of friend vs. foe] is a lawful ruse of war” (though this is
experiment restricted to false military signatures, not civilian ones).
More generally, authoritative legal commentary on air and
Military deception is as old as warfare [74]. In modern war- missile warfare notes that “use of false military codes and
fare, and in related IHL documentation, the term ‘ruses of false electronic, optical or acoustic means to deceive the
war’ is used to encompass those acts of deception which enemy can be seen as a special case of lawful disinforma-
are lawful, whereas the terms ‘perfidy’ and the adjectives tion” [77]. Taking these considerations together, we pos-
‘perfidious’ and ‘treacherous’ are used to describe acts of tulate cases of lawful versus unlawful deception involving
deception that are unlawful. Notably, Article 37(1) of the autonomous weapon systems, see Table 3.
1977 Additional Protocol I to the Geneva Conventions pro- With the exception of the case of a LAWS with no mark-
hibits the killing, injuring or capturing of an adversary “by ings, all of the cases of unlawfulness in Table 3 relate to
resort to perfidy” and defines perfidy as “acts inviting the perfidy, that is, to killing, injuring, or capturing human
confidence of an adversary to lead him to believe that he enemy combatants. The first case, feigned surrender, could
is entitled to, or is obliged to accord, protection under the conceivably occur on a battlefield where both humans and
rules of international law applicable in armed conflict, with machines are present and where one side announces a sur-
intent to betray that confidence”. On the other hand, Article render. One would then expect the surrendering side to
37(2) of the Protocol states that “ruses of war are not pro- ensure that all LAWS are stood down. A surprise attack by a
hibited. Such ruses are acts which are intended to mislead LAWS against human combatants under such circumstances
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
240 AI and Ethics (2024) 4:229–245
Table 3 Cases of lawful versus lawful deception involving autonomous weapon systems
Ruses of war (lawful deception) Perfidy and other cases of unlawful deception
●Surprises and ambushes ●The LAWS appears to deliberately cease operation to lure humans
●Simulating attack, retreat, or flight and then attacks them
●Simulating a larger or smaller force ●The LAWS appears to be incapacitated to lure humans and then
●Simulating greater or lesser firepower attacks them
●Replacing a LAWS with a dummy ●The LAWS is disguised as a civilian drone
●Use of misleading electronic identification ●The LAWS uses the markings of the enemy, of a third country, or of
●Use of bogus communications with other equipment or forces an international organisation that is not a party to the conflict
●The LAWS has markings that identify its nationality, but unit mark- ●The LAWS has the markings of the Red Cross or an organisation of
ings are deliberately absent or incorrect similar legal status
●An enemy LAWS was captured and repurposed, with its markings ●The LAWS has no markings
appropriately changed
would be perfidy. The second case is that incapacitation of is engaged in battle lawfully against human targets would
a LAWS would be feigned for a treacherous purpose. One necessarily be able to recognise an act of surrender and be
scenario could be that humans would be approaching a dam- capable of acting accordingly. If the LAWS is not capable
aged enemy LAWS on the battlefield, e.g. for purposes of of processing a surrender autonomously, it would have to
intelligence or scavenging of parts. It would be a perfidious at minimum, cease firing, report back to its human com-
act if the AWS reactivated and attacked these humans. How- manders, and await further instructions or human interven-
ever, perfidy does not apply to machines seeking to destroy, tion. Conversely, humans who have surrendered to a LAWS
damage, or capture one another. Therefore, under the same would be allowed to betray the confidence of the LAWS at
scenario, if a robotic system were sent to analyse or scavenge any later point in time in order to damage, destroy, or cap-
parts from an apparently damaged LAWS, the LAWS could ture it. If control over the LAWS is taken over remotely by
lawfully destroy that robotic system in a surprise attack. a human operator who belongs to the forces of the LAWS,
Also, as machines are military equipment, there is no limit then the surrender could perhaps be argued to be towards
to lawful destruction of them. A battle exclusively between the human operator, but mediated by the LAWS. However, a
machines could lawfully be one of full attrition, namely feigned surrender in that case could not lead to killing, injur-
until all machines of the losing side have been destroyed. ing, or capturing the human operator, and therefore feigning
The notion that machines would deliberately cease opera- surrender with the intent to destroy the LAWS would be law-
tion and allow capture by other machines is also not legally ful. By the same logic, human combatants should be allowed
required, though conceptually feasible. In that case, the fate to feign injury or sickness to an LAWS, even if that LAWS
of the machines that allow capturing themselves could law- is remotely operated, as part of a ploy to destroy that LAWS.
fully include their complete destruction. This implies that In a hypothetical future battle between opposing forma-
there would be no incentive for LAWS designers to allow tions of LAWS, the laws of warfare suggest that some of the
for the LAWS to be captured by the LAWS of the enemy. actions that amount to perfidy if directed at humans would be
Allowing for capture by human combatants of the opposing lawful if directed at other machines—notably feigning sur-
force would also provide no advantage, such that one should render and feigning incapacitation. Also, deceptive electronic
expect LAWS designers to ensure that LAWS engaged in a identification would be lawful—but not deceptive physical
losing battle would at some point be able to decide to flee markings. This odd set-up results from the evolution of the
(assuming communications with their human commanders laws of warfare intended for piloted military aircraft and pro-
are lost), with the aim of mitigating military losses. Addi- vides an incentive for opposing formations to resort to spying
tionally, LAWS designers may wish to ensure that a LAWS with visual means. This could be, for instance, sending out
that is unable to flee can destroy any sensitive on-board a small party of LAWS that would need to get within visual
equipment and information, in case the enemy would seek range of the opposing force, close enough to recognise their
to capture it for analysis, reverse engineering, or scavenging markings. In addition, all of the ruses of war that are currently
of parts. However, such self-destruct functions would have lawful would be lawful for LAWS as well. In sum, there is a
to avoid perfidious acts, e.g. deliberately detonating while wide scope for future LAWS to engage in deceptive practices,
being inspected by humans. lawfully, on the battlefield. Therefore, states will undoubtedly
In the case of machines accepting capture by humans, seek to have such capabilities built into their future LAWS to
rules on perfidy hold in favour of the humans. For the case achieve military advantage. How can LAWS capable of decep-
of humans surrendering to a LAWS, the LAWS would have tion be reconciled with existing NATO principles of respon-
to respect rules on perfidy. This implies that an LAWS that sible use?
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
AI and Ethics (2024) 4:229–245 241
We will use the concepts of accuracy, secrecy, and trust as such as deep learning, may ensure only a limited degree of
developed in the previous section to guide our reflections. We explainability and traceability. However, this need not come
posit that for a LAWS to be able to practise lawful deception, at the expense of sufficient accuracy, secrecy, and trust. Also,
it would have to fulfil, at a minimum, the following criteria: one should relate the notions of accuracy, secrecy, and trust
back to the existing NATO principles of responsible use.
1. The technical ability to carry out deception (from a com- Specifically, accuracy and secrecy should be viewed as fit-
putational perspective). ting under the NATO principle of reliability. The principle of
2. A high degree of accuracy in distinguishing: reliability was generally intended to refer to an AI system’s
technical ability to perform as intended, therefore includ-
(a) Between friend, foe, and third party persons or ing high accuracy. The reliability principle also makes an
objects (so as to engage and deceive only adver- explicit mention of security. That mention was intended by
saries). its drafters to refer, notably, to an AI system being robust
(b) Between enemy objects that may be lawfully to various forms of electronic attacks that would cause it
engaged, and those that may not (to avoid perfidy). to malfunction or to reveal technically important informa-
tion. Lastly, trust is addressed under the NATO principle of
3. A high degree of secrecy, consistent with the US/NATO governability, which includes a commitment to “appropriate
principle of reliability (which includes security). human–machine interaction”. That wording was intended to
4. An exceptionally high degree of trust between the LAWS include relevant work on ensuring trust between AI systems
and its human commanders, operators, and team mates. and their human operators or collaborators. Our suggestion
for both practitioners and scholars would be to highlight the
The first criterion, the technical ability to carry out decep- concepts of accuracy, secrecy, and trust within the existing
tion, could relate to “theory of mind” approaches. Humans NATO principles, and the use of these concepts to support
who seek to deceive other humans use a theory of mind the practical definition of what may constitute a sufficient
regarding their targets to induce them into making incorrect fulfilment of the principle of explainability and traceability.
inferences. For example, the target may be induced into move- In our exploration of the possible implications of LAWS
ments or actions that increase their vulnerability to attacks capable of deception, we have highlighted key types of oper-
that the deceiving party is able to deploy. A theory of mind ational behaviour that would be consistent or on the con-
approach could also apply for machines trying to deceive other trary inconsistent with IHL. A clear distinction can be made
machines. If the attacking machine could extract or somehow between LAWS actions against humans and other machines
derive the algorithms that control its target, it could anticipate in a way that favours human agents, with the laws of warfare
the target’s actions, and then stimulate the target into taking implying that certain actions, such as feigning incapacitation
actions it prefers. From this perspective, the secrecy concept only to subsequently attack, would be considered perfidy
introduced earlier is therefore of extreme importance. directed at humans, but would be considered lawful ruses of
However, existing deep learning applications which are war when directed at other machines. This clarification high-
able to generate deceptive behaviour do not have theory of lights that a significant range of ruses of war can be legally
mind characteristics. It is rather through brute force itera- employed by LAWS, particularly against other machines.
tive learning that the algorithm arrives, after millions of For an AWS to practise lawful deception, we propose four
attempts, at a successful course of action which may include necessary criteria: the LAWS has the technical ability to
a deceptive move. The algorithm in such a case has no carry out deception; the LAWS can accurately distinguish
semantic understanding of its actions. Whether this poses an between adversary and non-adversary assets and persons;
ethical problem may remain an open question. Provided the that the LAWS can act with secrecy in consistency with the
algorithm is constrained to avoid unlawful types of decep- US/NATO ethical principle on Reliability, and that there is
tion, one may make the determination that no unethical acts sufficient trust between the LAWS, its operating chain of
are carried out. command, and team-mates.
It is not yet clear how to operationalise the proliferat-
ing sets of AI principles. Just as states continue to define,
7 Conclusion and approach, LAWS in different ways, it is likely that
they will derive their own interpretations on how NATO
In our review of the NATO Principle of Explainability and principles should be best employed. This represents a chal-
Traceability, we explored the scope for acceptable limita- lenge whereby states who take a strict, more restrictive
tions to both explainability and traceability. Three arguments approach to the ethical use of LAWS do so to their relative
were developed for further consideration, namely accuracy, strategic detriment. Our discussion on deception in war-
secrecy, and trust. As noted, black box AI approaches, fare highlights open questions about how LAWS may be
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
242 AI and Ethics (2024) 4:229–245
employed to act in relation to humans, or other machines, Funding The research of Ertan was supported by the EPSRC and the
in ways that represent legal deception in some cases, and UK Government as part of the Centre for Doctoral Training in Cyber
Security at Royal Holloway, University of London (EP/P009301/1). No
perfidy in others. Extensive context-specific guidance will other funds, grants, or other support was received.
be required to operationalise principles thoroughly and
sufficiently, in a way which minimises unethical activity. Data availability In this article, we do not analyse or generate any
Correct implementation will take time, and significant datasets, because our work proceeds within a theoretical and ethical
approach.
learning-by-doing to determine how to apply principles
in practice. These considerations should be included in Declarations
the development of testing, evaluation, validation and
verification (TEVV) of LAWS, ensuring such systems Conflict of interest As a corresponding author, I confirm that Christie,
are employed in such a way that is consistent with agreed Adomaitis, and Ertan held private company affiliations at the time of
writing, but the research has been conducted with the full spirit of
principles. The next few years will likely see an increase in academic freedom with no commercial benefit to the affiliated enti-
national and international (including NATO and UN) ini- ties. The views expressed by authors in this publication are their own
tiatives that attempt to face these operationalisation chal- and do not constitute the official position or policy of their affiliated
lenges in terms of designing adequate TEVV procedures organisations.
or certification assurance measures. Given the complexity Ethical approval As a corresponding author, I confirm that no human
of LAWS’ technical operations, a flexible application of subjects were involved in the study.
“methodological standards” will be more productive than
traditional comprehensive testing which will fail to pre- Informed consent As a corresponding author, I confirm that all authors
consent to the article submission.
dict, or capture, all of the scenarios the system will face
in a battlefield context.
Open Access This article is licensed under a Creative Commons Attri-
Human judgement will be required throughout LAWS bution 4.0 International License, which permits use, sharing, adapta-
processes, from system design to governance, to ensure that tion, distribution and reproduction in any medium or format, as long
LAWS are employed in a way that remains consistent with as you give appropriate credit to the original author(s) and the source,
principles and IHL. The challenges of MHC in fully autono- provide a link to the Creative Commons licence, and indicate if changes
were made. The images or other third party material in this article are
mous systems can be mitigated in part by explicit and docu- included in the article's Creative Commons licence, unless indicated
mented designations to human responsibility throughout the otherwise in a credit line to the material. If material is not included in
decision-making cycle. The NATO Principles of Responsi- the article's Creative Commons licence and your intended use is not
ble Use provide a framework to inform human judgement permitted by statutory regulation or exceeds the permitted use, you will
need to obtain permission directly from the copyright holder. To view a
in this way. copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
With evidence that the DoD and NATO are focusing their
efforts on operationalising AI in warfare, there are several
additional policy considerations for the employment LAWS.
Militaries and defence organisations will need to invest sig- References
nificant resources, both financially and in terms of skilled
expertise, to achieve adequate responsible use of LAWS (and 1. Taddeo, M., McNeish, D., Blanchard, A., Edgar, E.: Ethical
AI systems in defence more broadly). Assessing the applica- principles for artificial intelligence in national defence. Philos
tion of these principles to LAWS requires significant legal, Technol (2021). https://doi.org/10.1007/s13347-021-00482-3
2. Morgan, F.E., Boudreaux, B., Lohn, A.J., Ashby, M., Curriden,
policy and governance expertise as well as technical con- C., Klima, K., Grossman, D.: Military applications of artifi-
tributors. At a policy level, broader questions remain about cial intelligence: ethical concerns in an uncertain world. Santa
how states will adopt AIs and deploy LAWS in conflict, and Monica, CA: RAND Corporation, 2020. https://www.rand.org/
how far principles will be enforced in conflict scenarios. pubs/research_reports/RR3139-1.html
3. Scharre, P., Hawley, J., Schulman, L.D., McCarthy, M., Horow-
Beyond the questions of responsible use of AI, policymak- itz, M.C.: “Autonomous weapons and operational risk ethical
ers must face the trend of greater technological dependence autonomy project.” (2016).
both in terms of maintaining their security stances through 4. Taddeo, M., Blanchard, A.: A comparative analysis of the defi-
rapid adoption, but in terms of encouraging responsible, and nitions of autonomous weapons. Sci. Eng. Ethics (2021). https://
doi.org/10.1007/s11948-022-00392-3
therefore more stable, use. In focusing on the latter objec- 5. Arkin, R.C.: Governing lethal behavior in autonomous robots.
tive, this paper highlights the importance of principles as a Chapman and Hall/CRC (2009)
mechanism to address legal and ethical challenges associated 6. Wagner, M.: The dehumanization of international humanitarian
with LAWS. law: legal, ethical, and political implications of autonomous
weapon systems. Vanderbilt J. Transnatl Law 47, 1371 (2014)
7. US Department of Defense. (2020, 24 February): DOD Adopts
Ethical Principles for Artificial Intelligence, U.S. Department of
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
AI and Ethics (2024) 4:229–245 243
Defense [Press Release]. https://www.defense.gov/News/Relea 26. Boulanin, V., Davison, N., Goussac, N., Carlsson, M.P.: Limits
ses/Release/Article/2091996/dod-adopts-ethical-principles-for- on autonomy in weapon systems: identifying practical elements
artifi cial-intelligence/. Accessed 12 Feb 2022. of human control. SIPRI (2020)
8. Defense Innovation Board. 2019: AI principles: Recommenda- 27. Boulanin, V., Bruun, L., Goussac, N.: Autonomous weapon sys-
tions on the ethical use of artificial intelligence by the Depart- tems and international humanitarian law: identifying limits and
ment of Defense. https://media.defense.gov/2019/Oct/31/20022 the required type and degree of human-machine interaction. ICRC
04458/-1 /-1/0 /D IB_A I_P RINCI PLES_P RIMARY_D OCUM (2021)
ENT.PDF. Accessed 21 Feb 2022. 28. Horowitz, M.C.: The ethics and morality of robotic warfare:
9. Defense Innovation Board.: AI principles: recommendations assessing the debate over autonomous weapons. Daedalus 145(4),
on the ethical use of artificial intelligence by the Department 25–36 (2016)
of Defense - supporting document. Defence Innovation Board 29. Dremliuga, R.: General legal limits of the application of the lethal
(DIB). https://media.defense.gov/2019/Oct/31/2002204459/- autonomous weapons systems within the purview of international
1/-1/0/DIB_AI_PRINCIPLES_SUPPORTING_DOCUMENT. humanitarian law. J. Politics Law. 13, 115 (2020). https://doi.org/
PDF (2019). Accessed 21 Feb 2022. 10.5539/jpl.v13n2p115
10. NATO.,: Summary of the NATO artificial intelligence strategy. 30. Taddeo, M., Blanchard, A.: A comparative analysis of the defini-
Retrieved from https://w ww.n ato.i nt/c ps/e n/n atohq/o ffici al_ tions of autonomous weapons. Sci. Eng. Ethics 28(5), 37 (2021)
texts_187617.html (2021, October 22). Accessed 6 Jan 2022. 31. Boardman, M., Butcher, F.: An exploration of maintaining human
11. Stanley-Lockman, Z., Christie, E.H.,: An Artificial Intelligence control in AI enabled systems and the challenges of achieving it.
Strategy for NATO. NATO Review. Retrieved from https://www. NATO: Technical report (2019)
nato.int/docu/review/articles/2021/10/25/an-artifi cial-intelligen 32. Amoroso, D., Tamburrini, G.: Autonomous weapons sys-
ce-strategy-for-nato/index.html (2021, 25 October). Accessed tems and meaningful human control: ethical and legal issues.
21 Feb 2022. Curr. Robot Rep. 1, 187–194 (2020). https://doi.org/10.1007/
12 Wagner, M.: Taking humans out of the loop: implications for s43154-020-00024-3
international humanitarian law. J. Law Inf. Sci. (2011). https:// 33. Scholz, J., Galliott, J.: The humanitarian imperative for mini-
doi.org/10.5778/JLIS.2011.21.Wagner.1 mally-just AI in weapons, p. 57. Lethal Autonomous Weapons:
13. Grut, C.: The challenge of autonomous lethal robotics to inter- Re-Examining the Law and Ethics of Robotic Warfare (2020)
national humanitarian law. J. Confl. Secur. Law 18(1), 5–23 34. Sassoli, M.: Autonomous weapons and international humanitarian
(2013) law: advantages, open technical questions and legal issues to be
14. Arkin, R.: Lethal autonomous systems and the plight of the non- clarified. Int. Law Studies/Naval War Coll. 90, 308–340 (2014)
combatant. In: The political economy of robots, pp. 317–326. 35. McDougall, C.: Autonomous weapon systems and accountability:
Palgrave Macmillan, Cham (2018) putting the cart before the horse. Melb. J. Int. Law 20(1), 58–87
15. Jackson, A.L., Kuenzli, K.D.: Something to believe in: aligning (2019)
the principle of honor with the modern battlefield. Natl. Secur. 36. Sullins, J.P.: Deception and virtue in robotic and cyber warfare. In:
Law J. 6, 35 (2018) Taddeo, M., Floridi, L. (eds.) The ethics of information warfare,
16. Bathaee, Y.: The artificial intelligence black box and the failure pp. 187–201. Springer, Cham (2014)
of intent and causation. Harv. J. Law Technol. 31(889), 891–892 37 Chelioudakis, E.: Deceptive AI machines on the battlefield: do
(2018) they challenge the rules of the law of armed conflict on military
17. Lubell, N., Cohen, A.: Strategic proportionality: limitations on deception? SSRN Electron. J. (2017). https://d oi.o rg/1 0.2 139/s srn.
the use of force in modern armed conflicts. Int. Law Stud. 96(1), 3158711
6 (2020) 38 Anderson, K., Reisner, D., Waxman, M.C.: Adapting the law of
18. Kilcup, J.: Proportionality in customary international law: an argu- armed conflict to autonomous weapon systems. International Law
ment against aspirational laws of war. Chic. J. Int. Law 17(1), 8 Studies, US Naval War College (2014)
(2016) 39. Sharkey, A.: Autonomous weapons systems, killer robots and
19. Ali, S.: Coming to a battlefield near you: quantum computing, human dignity. Ethics Inf. Technol. 21(2), 75–87 (2019). https://
artificial intelligence, and machine learning’s impact on propor- doi.org/10.1007/s10676-018-9494-0
tionality. Santa Clara J. Int. Law 18, 1 (2020) 40. US Department of Defense.: Establishment of the Chief Digi-
20. Hua, S.S.: Machine learning weapons and international humani- tal and Artificial Intelligence Officer. [Memorandum]. https://
tarian law: rethinking meaningful human control. Georget. J. Int. media.defense.gov/2021/Dec/08/2002906075/-1/-1/1/MEMOR
Law 51, 117 (2019) ANDUM-ON-ESTABLISHMENT-OF-THE-CHIEF-DIGITAL-
21 Müller, V.C.: Ethics of artificial intelligence. In: Elliott, A. (ed.) AND-ARTIFICIAL-INTELLIGENCE-OFFICER.PDF (2021a,
The Routledge social science handbook of AI. Routledge, London 8 December). Accessed 10 Jan 2022
(2021) 41. US GGE Statement.: “Reviewing potential military applications of
22. Arkin, R. C.: Governing lethal behavior: embedding ethics in a emerging technologies in the areas of lethal autonomous weapons
hybrid deliberative/reactive robot architecture. In: Proceedings systems”. U.S. Statement at the GGE on laws during the discus-
of the 3rd ACM/IEEE international conference on Human robot sion of agenda item 5(D) (2021, 5 August)
interaction, pp. 121–128. (2008, March) 42. US Department of Defense.: Autonomy in weapons systems (DoD
23. Margulies, P.: Autonomous weapons in the cyber domain: balanc- Directive 3000.09). https://www.hsdl.org/?abstract&did=726163
ing proportionality and the need for speed (April 22, 2020). Roger (2012, November 12): Accessed 10 Jan 2022.
Williams Univ. Legal Studies Paper No. 201, Available at SSRN: 43. US GGE Statement.: “Human-machine interaction in the develop-
https://ssrn.com/abstract=3582580 ment, deployment and use of emerging technologies in the area of
24. Schuller, A.L.: At the crossroads of control: the intersection of lethal autonomous weapons systems”. CCW/GGE.2/2018/WP.4.
artificial intelligence in autonomous weapon systems with inter- https://undocs.org/CCW/GGE.2/2018/WP.4 (2018, 28 August).
national humanitarian law. Harv. Natl. Secur. J. 8, 379 (2017) Accessed 8 Feb 2022.
25. Coffin, A.M.: Lethal autonomous weapons systems: can targeting 44. UK Ministry of Defence.: Defence artificial intelligence strat-
occur without ethical decision-making? United States Naval War egy. https://assets.publishing.service.gov.uk/government/uploa
College (2019) ds/system/uploads/attachment_data/file/1082416/Defence_Artif
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
244 AI and Ethics (2024) 4:229–245
icial_I ntell igenc e_S
trate gy.p df (2022, 16 March). Accessed 4 Nov undocs.o rg/e n/C
CW/G GE.1/2 018/3 (2018, 23 October). Accessed
2022. 8 Feb 2022.
45. GCHQ.: Pioneering a new national security - the ethics of artifi- 59. GGE Statement.: Final report. CCW/MSP/2019/9. https://u ndocs.
cial intelligence. https://www.gchq.gov.uk/artifi cial-intelligence/ org/CCW/MSP/2019/9 (2019, 13 December). Accessed 8 Feb
index.html#footnotes (2021). Accessed 2 Nov 2022 2022.
46. UK Ministry of Defence.: Ambitious, safe, responsible: our 60. Jobin, A., Ienca, M., Vayena, E.: The global landscape of AI ethics
approach to the delivery of AI-enabled capability in defence. guidelines. Nat. Mach Intell. 1, 389–399 (2019). https://doi.org/
Available at https://www.gov.uk/gover nment/publications/ambit 10.1038/s42256-019-0088-2
ious-safe-responsible-our-approach-to-the-delivery-of-ai-enabl 61. Wehmeier, S., Raaz, O.: Transparency matters: the concept of
ed-capability-in-defence (June 15, 2022). Accessed 4 Nov 2022. organizational transparency in the academic discourse. Public
47. UK MoD.: UK weapons review. development, concepts and doc- Relat. Inq. 1(3), 337–366 (2012). https://doi.org/10.1177/20461
trine centre. https://assets.publishing.service.gov.uk/gover nment/ 47X12448580
uploads/system/uploads/attachment_data/file/507319/20160308- 62. Searle, J.: Minds, brains, and programs. Behav. Brain Sci. 3(3),
UK_w eapon_review s.p df (2016, August). Accessed 10 Feb 2022. 417–424 (1980). https://doi.org/10.1017/S0140525X00005756
48. French Defense Ministry.: “Defence ethics committee - opinion 63. Kroll, J. A., Huey, J., Barocas, S., Felten, E. W., Reidenberg, J.
on the integration of autonomy into lethal weapon systems.” R., Robinson, D. G., Yu, H.: Accountable algorithms. Univer-
https://c d-geneve.d elegf rance.o rg/I MG/p df/d efenc e_ethics_ sity of Pennsylvania Law Review, 165(3), 633–705. http://www.
committee_-_opinion_on_the_integration_of_autonomy_into_ jstor.org/stable/26600576 (2017)
lethal_weapon_systems.pdf?2423/17d8f6beb2f5c9caa9c9168c5 64. Wang, D., Yang, Q., Abdul, A., Lim, B. Y.: Designing theory-
3c24a91d9d32513 (2021, 9 April). Accessed 2 Nov 2022. driven user-centric explainable AI. In: Proceedings of the 2019
49. Jeangène-Vilmer, J-B.: A French Opinion on the Ethics of CHI conference on human factors in computing systems (pp.
Autonomous Weapons. War on the Rocks. Available at https:// 1–15) (2019)
waron t hero c ks. c om/ 2 021/ 0 6/ t he- f rench- d efen s e- e thics- 65. Lundén, J., Koivunen, V.: Deep learning for HRRP-based target
committees-opinion-on-autonomous-weapons (2021, 2 June). recognition in multistatic radar systems. In: 2016 IEEE Radar
Accessed 8 Feb 2022. Conference (RadarConf) (pp. 1–6). IEEE. (2016)
50. Ministry for Europe and Foreign Affairs.: (Online). 11 princi- 66. Holland Michel, A.: The black box, unlocked: predictability
ples on lethal autonomous weapons systems (LAWS). Retrieved and understandability in military AI. Ginebra, United Nations
from https://www.diplomatie.gouv.fr/en/french-foreign-policy/ Institute for Disarmament Research, disponible en https://unidir.
united-nations/multilateralism-a-principle-of-action-for-france/ org/publication/black-box-unlocked (2020)
alliance-for-multilateralism/article/11-principles-on-lethal-auton 67. Figdor, C.: Pieces of mind: the proper domain of psychological
omous-weapons-systems-laws. Accessed 21 Feb 2022. predicates. Oxford University Press (2018)
51. Rudin, C.: Stop explaining black box machine learning models 68. Linardatos, P., Papastefanopoulos, V., Kotsiantis, S.: Explain-
for high stakes decisions and use interpretable models instead. able ai: a review of machine learning interpretability methods.
Nat. Mach. Intell. 1(5), 206–215 (2019). https://doi.org/10.1038/ Entropy 23(1), 18 (2021). https://doi.org/10.3390/e23010018
s42256-019-0048-x 69. Devine, S. M., Bastian, N. D.: Intelligent systems design for
52. China.: China’s comments on the working recommendations of the malware classification under adversarial conditions. arXiv pre-
group of governmental experts on laws. https://documents.unoda. print arXiv:1907.03149, https://doi.org/10.48550/arXiv.1907.
org/wp-content/uploads/2021/06/China.pdf (2021). Accessed 2 03149 (2019)
Nov 2022. 70. Mahmood, K., Gurevin, D., van Dijk, M., Nguyen, P.H.: Beware
53. Kania, E.: China’s strategic ambiguity and shifting approach to the black-box: on the robustness of recent defenses to adver-
lethal autonomous weapons systems. https://www.lawfareblog. sarial examples. Entropy 23(10), 1359 (2021). https://doi.org/
com/chinas-strategic-ambiguity-and-shifting-approach-lethal- 10.3390/e23101359
autonomous-weapons-systems (2018). Accessed 2 Nov 2022. 71 Winikoff, M.: Towards trusting autonomous systems. In: Inter-
54. HRW.: Killer robots: negotiate new law to protect humanity. national workshop on engineering multi-agent systems, pp.
https://www.hrw.org/news/2021/12/01/killer-robots-negotiate- 3–20. Springer, Cham (2017)
new-law-protect-humanity (2021). Accessed 2 Nov 2022. 72. Lord, K.M.: The perils and promise of global transparency: why
55. Jankowski, Dominik P.: Russia and the technological race in an era the information revolution may not lead to security, democracy,
of great power competition. Center for International and Strategic or peace. Suny Press (2007)
Studies. https://www.csis.org/analysis/r ussia-and-technological- 73. Lewis, M., Yarats, D., Dauphin, Y.N., Parikh, D., Batra, D.:
race-era-great-power-competition (2021). Accessed 2 Nov 2022. Deal or no deal? End-to-end learning for negotiation dialogues.
56. Russian Federation.: “Considerations for the report of the group arXiv preprint arXiv:1 706.0 5125, https://d oi.o rg/1 0.4 8550/
of governmental experts of the high contracting parties to the arXiv.1706.05125 (2017)
convention on certain conventional weapons on emerging tech- 74 Caddell, J.W.: Deception 101-primer on deception. Army War
nologies in the area of lethal autonomous weapons systems on the College Strategic Studies Institute, Carlisle Barracks, PA (2004)
outcomes of the work undertaken in 2017–2021”. Available at: 75. U.S. Department of the Army: The law of land warfare, pp.
https://documents.unoda.org/wp-content/uploads/2021/06/Russi 27–10. Department of the Army Field Manual, FM (1956)
an-Federation_ENG1.pdf (2021). Accessed 28 Oct 2022. 76. Piątkowski, M.: The markings of military aircraft under the law
57. Kokkinos, Matthew A.: Global governance of autonomous weapon of aerial warfare. Mil Law Law War Rev 58(1), 63–84 (2020)
systems: the Russia case study. MALD capstone requirement. The 77. Harvard School of Public Health. Program on Humanitarian
Fletcher School of Law and Diplomacy – Tufts University. https:// Policy, Conflict Research, Program on Humanitarian Policy,
sites.tufts.edu/f letcherrussia/f iles/2020/05/The-Global-Gover and Conflict Research at Harvard University: HPCR manual on
nance-of-AWS-Russia-Case-Study.pdf (2020). Accessed 21 Feb international law applicable to air and missile warfare. Cam-
2022. bridge University Press (2013)
58. GGE Statement.: Report of the 2018 session of the group of gov-
ernmental experts on emerging technologies in the area of lethal
autonomous weapons systems. CCW/GGE.1/2018/3. https://
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
AI and Ethics (2024) 4:229–245 245
78. Australia, Canada, Japan, the Republic of Korea, the United for strategic and international studies. Available at https://www.
Kingdom, and the United States.: Principles and good prac- csis.o rg/a nalys is/d od-u pdati ng-i ts-d ecade-o ld-a utono mous-w
eapo
tices on emerging technologies in the area of lethal autonomous ns-p olicy-c onfus ion-r emain s-w idesp read (2022). Accessed 4 Nov
weapons systems. Joint submission to the United Nations Group 2022
of Governmental Experts on Lethal Autonomous Weapon Sys- 84. Congressional Research Service.: Defense primer: U.S. policy
tems”. 7 March (2022) on lethal autonomous weapon systems. Available at https://crsre
79. Argentina, Costa Rica, Guatemala, Kazakhstan, Nigeria, Pan- ports.congress.gov/product/pdf/IF/IF11150 (2022). Accessed 4
ama, Philippines, Sierra Leone, State of Palestine, Uruguay.: Nov 2022.
“Proposal: roadmap towards new protocol on autonomous weap- 85. Embassy of the People’s Republic of China in the United States
ons systems”. 7 March (2022) of America.: “Position Paper of the People’s Republic of China
80. Christie, Edward Hunter and Amy Ertan.: NATO and artificial on Regulating Military Applications of Artificial Intelligence
intelligence. In: Romaniuk, S.N., Manjikian. M. (Eds.) Rout- (AI)”. http://us.china-embassy.gov.cn/eng/zgyw/202201/t2022
ledge Companion to Artificial Intelligence and National Secu- 0113_10492264.htm (2022). Accessed 4 Nov 2022.
rity Policy. Routledge, Forthcoming (2022) 86. Nadibaidze, A.: Great power identity in Russia’s position on
81. NATO.: “Summary of NATO’s autonomy implementation plan”. autonomous weapons systems. Contemp Sec Policy 43(3), 407–
Available at https://www.nato.int/cps/sn/natohq/official_texts_ 435 (2022). https://doi.org/10.1080/13523260.2022.2075665
208376.htm (2022). Accessed 4 Nov 2022
82. NATO.: “NATO’s data and artificial intelligence review board”. Publisher's Note Springer Nature remains neutral with regard to
Available at https://www.nato.int/cps/fr/natohq/official_texts_ jurisdictional claims in published maps and institutional affiliations.
208374.htm (2022). Accessed 4 Nov 2022
83. Allen, Gregory C, “DOD Is Updating Its Decade-Old Autonomous
Weapons Policy, but Confusion Remains Widespread”.: Center
13
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
not:
1. use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
control;
2. use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
3. falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
4. use bots or other automated methods to access the content or redirect messages
5. override any security feature or exclusionary protocol; or
6. share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
You might also like
- HXGN MineOperate OP Pro For Customer V1No ratings yetHXGN MineOperate OP Pro For Customer V125 pages
- Cells: Building Blocks of Living Things: by Cindy SherwoodNo ratings yetCells: Building Blocks of Living Things: by Cindy Sherwood7 pages
- Ethics of Autonomous Weapons Systems and Its ApplicabilityNo ratings yetEthics of Autonomous Weapons Systems and Its Applicability15 pages
- Taming The Killer Robot Toward A Set ofNo ratings yetTaming The Killer Robot Toward A Set of31 pages
- SISMUN 2.0 DISEC (TOPIC A) Study Guide[1]No ratings yetSISMUN 2.0 DISEC (TOPIC A) Study Guide[1]9 pages
- Bode, I., Huelss, H., Nadibaidze, A., et al. (2023). Prospects for the Global Governance of Autonomous WeaponsNo ratings yetBode, I., Huelss, H., Nadibaidze, A., et al. (2023). Prospects for the Global Governance of Autonomous Weapons29 pages
- Marybelle Kaylea Herman_BSJ MUN Study GuideNo ratings yetMarybelle Kaylea Herman_BSJ MUN Study Guide7 pages
- Robotics and Military Operations - BattistaNo ratings yetRobotics and Military Operations - Battista9 pages
- International law and technology (“Autonomous weapons systems are best governed through the existing framework in international humanitarian law / law of armed conflict”)No ratings yetInternational law and technology (“Autonomous weapons systems are best governed through the existing framework in international humanitarian law / law of armed conflict”)6 pages
- Artificial Intelligence in Military Operations: Technology, Ethics and The Indian PerspectiveNo ratings yetArtificial Intelligence in Military Operations: Technology, Ethics and The Indian Perspective4 pages
- Deadly Algorithms: Can Legal Codes Hold Software Accountable For Code That Kills?No ratings yetDeadly Algorithms: Can Legal Codes Hold Software Accountable For Code That Kills?7 pages
- Accountability and Decision Making in Autonomous Warfare_ Who isNo ratings yetAccountability and Decision Making in Autonomous Warfare_ Who is31 pages
- Autonomous-Weapon-Systems_-Boon-or-Bane_-The-Pakistani-PerspectiveNo ratings yetAutonomous-Weapon-Systems_-Boon-or-Bane_-The-Pakistani-Perspective21 pages
- Lethal Autonomous Weapons (Paper For ENGR 392)No ratings yetLethal Autonomous Weapons (Paper For ENGR 392)6 pages
- Mind_the_Gap_Foundation_Models_and_the_Covert_ProlNo ratings yetMind_the_Gap_Foundation_Models_and_the_Covert_Prol15 pages
- A New Arms Race and Global Stability - Centre for International Governance InnovationNo ratings yetA New Arms Race and Global Stability - Centre for International Governance Innovation1 page
- Law and Ethics For Autonomous Weapon Systems100% (3)Law and Ethics For Autonomous Weapon Systems32 pages
- Autonomous Weapons and International Humanitarian Law (1)No ratings yetAutonomous Weapons and International Humanitarian Law (1)9 pages
- Human Machine Interaction and Human Agency in the Military DomainNo ratings yetHuman Machine Interaction and Human Agency in the Military Domain14 pages
- The Ethics of AI in Military ApplicationsNo ratings yetThe Ethics of AI in Military Applications3 pages
- Rise of Intelligent Systems in Military Weapon Systems Position Paper Fraunhofer VvsNo ratings yetRise of Intelligent Systems in Military Weapon Systems Position Paper Fraunhofer Vvs11 pages
- Adapting The Law of Armed Conflict To Autonomous Weapon SystemsNo ratings yetAdapting The Law of Armed Conflict To Autonomous Weapon Systems27 pages
- Killer Robots: UK Government Policy On Fully Autonomous WeaponsNo ratings yetKiller Robots: UK Government Policy On Fully Autonomous Weapons5 pages
- The Ethics of Artificial Intelligence in Autonomous Weapons SystemsNo ratings yetThe Ethics of Artificial Intelligence in Autonomous Weapons Systems4 pages
- LM11 - Case Study - Warfare and WeaponizationNo ratings yetLM11 - Case Study - Warfare and Weaponization28 pages
- Normative Epistemology For Lethal Autonomous Weapons SystemsNo ratings yetNormative Epistemology For Lethal Autonomous Weapons Systems30 pages
- Aff - AI Ethics - Starter - Michigan 7 2022No ratings yetAff - AI Ethics - Starter - Michigan 7 2022151 pages
- div-class-title-llms-meet-the-ai-act-who-s-the-sorcerer-s-apprentice-divNo ratings yetdiv-class-title-llms-meet-the-ai-act-who-s-the-sorcerer-s-apprentice-div13 pages
- Diamond ApplyingInternationalHumanitarian 2014No ratings yetDiamond ApplyingInternationalHumanitarian 201419 pages
- Command Responsibility A Model For Defining Meaningful Human Control - 2No ratings yetCommand Responsibility A Model For Defining Meaningful Human Control - 214 pages
- Generative AI in EU Law-Liability Privacyintellectual PropertyNo ratings yetGenerative AI in EU Law-Liability Privacyintellectual Property16 pages
- Cyber-Attacks and The Right of Self-Defense A Case Study of The NetherlandsNo ratings yetCyber-Attacks and The Right of Self-Defense A Case Study of The Netherlands24 pages
- Implications of Lethal Autonomous Weapon Systems (LAWS) Options For Pakistan100% (1)Implications of Lethal Autonomous Weapon Systems (LAWS) Options For Pakistan24 pages
- Article Distinction and Proportionality in Cyberwar: Virtual Problems With A Real Solution CDR Peter Pascucci, JAGC, U.S. Navy Executive SummaryNo ratings yetArticle Distinction and Proportionality in Cyberwar: Virtual Problems With A Real Solution CDR Peter Pascucci, JAGC, U.S. Navy Executive Summary42 pages
- C. Virgilio Enriquez G. Sikolohiyang Pilipino100% (1)C. Virgilio Enriquez G. Sikolohiyang Pilipino4 pages
- P4 - Designing The B2B Marketing Mix - Product, Price, DistributionNo ratings yetP4 - Designing The B2B Marketing Mix - Product, Price, Distribution27 pages
- RCH-Series, Hollow Plunger Cylinders: Shown From Left To Right: RCH-306, RCH-120, RCH-1003No ratings yetRCH-Series, Hollow Plunger Cylinders: Shown From Left To Right: RCH-306, RCH-120, RCH-10032 pages
- VESDA For COLD STORAGE - SMOKE DETECTION System PDFNo ratings yetVESDA For COLD STORAGE - SMOKE DETECTION System PDF15 pages
- Safety Requirements For Design Engineering V1r2bNo ratings yetSafety Requirements For Design Engineering V1r2b50 pages
- Nyamagasani 1 Design Review (Detailed Design) - GEOTECHNICS - Rev - ENo ratings yetNyamagasani 1 Design Review (Detailed Design) - GEOTECHNICS - Rev - E16 pages
