Māori algorithmic sovereignty: ideas, principles, and use.

Paul T. Brown; Daniel Wilson; Kiri West; Kirita-Rose Escott; Kiya Basabas; Ben Ritchie; Danielle Lucas; Ivy Taia; Natalie Kusabs; Te Taka Keegan

Research Papers

Māori algorithmic sovereignty: ideas, principles, and use.

Authors

Paul T. Brown
Daniel Wilson
Kiri West
Kirita-Rose Escott
Kiya Basabas
Ben Ritchie
Danielle Lucas
Ivy Taia
Natalie Kusabs
Te Taka Keegan

Abstract

Due to the emergence of data-driven technologies in Aotearoa New Zealand that use Māori data, there is a need for values-based frameworks to guide thinking around balancing the tension between the opportunities these create, and the inherent risks that these technologies can impose. Algorithms can be framed as a particular use of data, therefore data frameworks that currently exist could be extended to include algorithms. Māori data sovereignty principles are well-known and are used by researchers and government agencies to guide the culturally appropriate use of Māori data. Extending these principles to fit the context of algorithms, and re-working the underlying sub-principles to address issues related to responsible algorithms from a Māori perspective leads to the Māori algorithmic sovereignty principles. We define this idea, present the updated principles and sub-principles, and highlight how these can be used to decolonise algorithms currently in use, and argue that these ideas could potentially be used to developed Indigenised algorithms.

Keywords:

Year: 2024

Volume 23

Page/Article: 15

DOI: 10.5334/dsj-2024-015

Submitted on Oct 9, 2023

Accepted on Mar 7, 2024

Published on Apr 2, 2024

Peer Reviewed

CC Attribution 4.0

1. Introduction

Artificial Intelligence (AI) technologies that involve data and algorithms are becoming more ubiquitous in decision-making processes (). These technologies are sold as solutions to biased human decision-making, and as a step towards wealthy societies, prosperity, and progress (; ). The positive intentions regarding the development and use of these technologies are usually genuine, however the outcomes may not be optimal from an equity perspective. A more critical look at the impact of these technologies shows that they have been used to generate wealth for large corporations (), reduce labour costs and protect capital (; ). The impact of these uses has perpetuated and amplified historical injustices, particularly racial (; ; ; ; ), gendered (; ), and economic injustice (; ). Added to this is the significant energy costs associated with building and maintaining these systems which are detrimental to the environment (; ). This creates additional burdens for marginalised communities that are least likely to realise the benefits, and most likely to be impacted by the harms of these technologies ().

Indigenous voices are generally ignored in the process of building algorithms, from conception through to implementation and maintenance over its lifecycle. However, there is little to no hesitation in applying algorithms to these populations, sometimes for benevolent intentions, but often for exploitation and commercial profits (; ; ). Yet, Indigenous peoples have important and valuable perspectives to offer in this space. For example, the ideas of Indigenous data sovereignty (IDSov: ; ; ) have challenged the notion of data sovereignty by suggesting that data should be subject to the laws and governance structures of nations, including Indigenous nations, of those who the data is about, not just subject to the laws of the nation-state where the data lies (). This perspective offers the idea and a suite of methods for correcting power imbalances for Indigenous nations in a world where billions upon billions of bytes of data are collected, stored, bought, and sold amongst large corporations and governments for their own purposes. Currently, there is a growing list of examples bridging the gap between IDSov and data technologies especially in the genomic data space, for example see Boscarino et al. () and Mackey et al. ().

The Māori data sovereignty (MDSov) principles is a good example of IDSov principles, specifically for the culturally appropriate governance and use of Māori data (), where Māori are the Indigenous people of Aotearoa New Zealand (hereafter referred to as Aotearoa or NZ where appropriate). From these principles, several frameworks have been developed for specific uses, including Te Mana o te Raraunga, for secondary use of Māori data in big data ecosystems (), and Ngā Tikanga Paihere (), a framework for culturally appropriate use of Māori data in the Integrated Data Infrastructure managed by Statistics NZ (). Recently developed works include the Māori Data Governance Model (), and the Māori Data Sovereignty and Privacy Framework (), both of which build upon the fundamental concepts and principles introduced in MDSov and apply them to the issues of data governance and data privacy respectively. In this paper, we introduce the idea of Māori algorithmic sovereignty (MASov), where algorithms can be understood as a particular use of data. We define a set of corresponding principles by extending the principles of MDSov to include appropriate use of algorithms that utilise Māori data, or that are applied to Māori individuals, communities, or environments that Māori have rights and/or interests in. The MASov principles are the starting point for the development of Māori tikanga (cultural)-based methods, frameworks, guidelines, or standards that can be used to assess existing algorithms that are applied to Māori, work towards decolonising existing algorithms, or developing indigenised algorithms so that they may produce fairer outcomes for Māori.

The structure of this paper is as follows; we define important terms in Section 2, including what we mean by algorithms, and define terms associated with algorithms. Section 3 introduces the idea of MASov and the corresponding principles and sub-principles that underpin MASov. We provide the reader with some historical context to justify the reasons why these principles are used and how they relate to the principles of responsible algorithms. Section 4 gives an example of using MASov principles to generate a framework to help assess an algorithm that produces biased outputs. We give our concluding remarks in Section 5 and discuss the transformational changes that may be possible for Māori as ideas such as MASov become more developed and used in practice.

2. Definitions and Terms

The term ‘algorithm’ is broad and ambiguous and can mean different things under different contexts and perspectives. This is also true when defining some of the socio-technical terms often associated with algorithms, such as ‘bias’, ‘fairness’, and ‘transparency’. This section provides the reader with some clarity about the terms we are using. We first distinguish between what we mean by computational algorithms and algorithmic systems, the latter of which we are interested in. Synonyms typically used for algorithmic systems include AI, AI systems, AI technologies, algorithms, automated decision-making processes, or models. We also provide some definitions regarding bias in relation to algorithms, definitions relating to Māori data, and definitions of terms typically used regarding responsible algorithm development, deployment, and use.

Algorithms

Dourish () describes an algorithm from a computational perspective as ‘an abstract, formalised description of a computational procedure’. The ‘computational algorithm’ defined by Dourish, in practice, takes in a set of inputs as chosen by a user, and generates a set of outputs that is interpreted by a user (see Figure 1). This definition is too narrow for what we want to investigate. Not only do we want to investigate the inputs and outputs of the algorithm, but also the human decision-making that drives the process. In addition to this, we wish to elucidate the ways in which algorithms are shaped by society and vice versa. As such, we recognise a distinction here between computational algorithms as described by Dourish, and something more general, which we call an algorithmic system:

Algorithmic System: An iterative decision-making process that is driven by humans, data, and computational algorithms.

Figure 1

A generic algorithmic system broken down into several foundational components.

Therefore, when we talk about investigating algorithms, our intention is to investigate algorithmic systems as defined above. This is far broader than just looking solely at the computational algorithm. The scope of any analysis of algorithmic systems includes who is involved in all aspects of the development (from funders to designers and implementers), the motivations that drive the algorithm’s existence, decisions regarding algorithm design, the inputs used, the outputs generated, the key decisions and policy that are made from the outputs, and the wider process including how the system is managed, monitored and maintained over time. For convenience, we will refer to algorithmic systems as ‘algorithms’ and will distinguish between algorithms and computational algorithms to keep the terminology clear.

Figure 1 displays the structure of a generic algorithm, which is characterized by several different components. The yellow components are what we typically think of computational algorithms – a machine that takes inputs and generates outputs. However, there is a significant amount of human decision-making (blue components) that goes into this process, from the conceptualization of the algorithm through to implementation. Motives set the tone for the development, as this directly influences the aspirations the algorithm is constructed to achieve. The design component refers to how the algorithm is constructed to answer the motives, what data and variables are required for inputs, and how the data is to be collected. These inputs are fed into the algorithm component, which are the set of computational algorithms and specified model(s) that turn inputs into outputs. Once outputs are generated, there are interpretations and analyses of the outputs where new knowledge is gained before decisions about how the new knowledge will be used. The process component represents the decisions that are made throughout the lifecycle of the algorithm. An algorithm typically requires funding, and needs to be managed, maintained, monitored, and tested throughout its lifecycle. Note that the diagram in Figure 1 visually implies that the development of an algorithm is linear. However, we acknowledge in practice that this is most likely never the case! The dashed-dotted lines represent the iterative nature of the system, and the arrows represent the dependency structure of the system (e.g., the design depends on the motives that drive the algorithm development).

Algorithmic Bias

In practice, algorithms are used to assist as a decision support, or decision-making tool, and provide economic value to the organisations that successfully implement them. However, there are ethical risks that can have detrimental impacts to the organisation () and society at large (; ). One of the largest concerns is that the algorithms can replicate and perpetuate systemic biases (the inherent tendency of a process or system to favour certain outcomes) that exist or are inherent within society. This phenomenon is what is referred to as algorithmic bias, which occurs when the outputs of an algorithm benefit or disadvantage certain individuals or groups over others without justification or reason for such unequal impacts ().

Systemic biases that algorithms perpetuate mainly impact marginalised populations, leading to the reinforcement of historical and current injustices, such as racial, gendered, and economic injustices. For the purposes of our analysis, the type of bias we are interested in is ‘colonising bias’. We define colonising bias as follows:

Colonising Bias: Prejudice or injustices against an indigenous group due to the effects of colonisation, that results in negative outcomes for that group.

Colonising bias can be thought of as a particular type of racial bias. In the context of this article, we refer to colonising bias within algorithms, or colonising bias that occurs as a result of the outcomes of an algorithm nested colonising bias, and specifically focus on Māori as the indigenous people of Aotearoa, whose culture, traditions, and lives have been negatively affected by colonisation. We frame an algorithm as a use of data, thus if an algorithm uses Māori data, there are certain considerations that should be involved, and MDSov principles must be applied. Māori data is defined as digital or digitisable information that is about, or generated from Māori people, Māori language, Māori culture, or resources and environments that Māori have rights and interests in (). There are six principles generally associated with MDSov, Rangatiratanga (Authority), Whakapapa (Relationships), Whanaungatanga (Obligations), Kotahitanga (Collective Benefits), Manaakitanga (Reciprocity), and Kaitiakitanga (Guardianship). We expand more on what these ideas mean in the context of data and algorithms in the upcoming section, but interested readers can see Appendix 1 in Kukutai and Taylor () for further clarification.

As algorithms have become more prominent, private companies, research institutions, and government agencies around the world are racing to develop frameworks that contain principles that constitute the responsible development and use of algorithms. A scoping review surveyed 84 different documents containing ethical principles and guidelines to map the landscape of existing principles and to determine if a global convergence of certain principles was visible (). The study found that the responsible algorithm principles of transparency, fairness, non-maleficence, responsibility, and privacy were cited by many of the documents as the principles most associated and important for responsible algorithms. Other principles highlighted were beneficence, freedom and autonomy, trust, sustainability, dignity, and solidarity. Whilst these principles do not necessarily have a strict definition in the context of algorithms, we have provided some general guidance on what these principles mean in Appendix 1, along with related terms.

3. Māori Algorithmic Sovereignty

Māori algorithmic sovereignty is a particular case of IDSov and is the idea that Māori data should be subject to the laws and governance structures of Māori. Since algorithms depend on data, we take the perspective that MASov should be defined similarly. Māori algorithmic sovereignty is the idea that algorithms that use Māori data, or that are applied to Māori individuals or collectives (including groups or organisations), or environments that Māori have rights and interests in, are subject to laws and governance structures of Māori. As a broad and high-level idea, applying MASov gives Māori a way to meaningfully participate in all aspects of the development, deployment, and use of algorithms, protect Māori data and the information that stem from the outputs of algorithms, and partner with non-Māori to ensure use of algorithms uphold Māori rights, interests, and values.

Expanding on the idea of MASov, we specify a set of high-level principles and sub-principles that lay the foundations for what MASov is and what it might look like in practice. To understand how these principles are formulated, we provide some historical context before presenting the principles and sub-principles.

Historical Context

Māori are the original inhabitants of Aotearoa and settled the lands centuries prior to the arrival of the first Europeans. In 1835, He Whakaputanga o te Rangatiratanga o Nu Tereni (the Declaration of Independence of the United Tribes of New Zealand) was signed as a formal deal between Māori and the British crown whereby mana (authority) and sovereign power of New Zealand rested with Māori, and foreign subjects could not make laws. The founding document of modern Aotearoa, Te Tiriti o Waitangi/The Treaty of Waitangi signed in 1840 between the Rangatira (tribal chiefs) of Aotearoa and representatives of the British Crown, established the foundations for the formation of a partnership between Māori and the Crown in Aotearoa. Generally speaking, the details of Te Tiriti are as follows; Article 1 establishes the authority of the Crown over its subjects through Kāwanatanga (loosely translated to mean governance). Article 2 gives recognition to the already established authority of Rangatira and grants continuation of their right of control over their taonga (treasured possessions, including objects, lands, and environment). Article 3 ensures individual citizenship rights and equality under the law. Both Articles 1 and 2 refer to the governing of two distinct populations and establish a framework for co-governance, whereas Article 3 speaks to individual citizen rights.

Since its signing, the importance and status of Te Tiriti has oscillated, but its role for Māori has always remained important (). What has not oscillated are past injustices, including treaty breaches, that Māori have been subject to historically and continue to this day (). This has caused a large amount of distrust that Māori have toward Western colonial systems. Prominent Māori lawyer and scholar, Moana Jackson, notes that there is a tendency to historicise colonisation and its consequences as if it isn’t a living, breathing reality (). The systems established in colonisation are still functioning today and continue to contribute to our contemporary struggles (). Algorithms act as an extension of existing colonial infrastructure and as such, we see the extension of Indigenous distrust in these systems. Relatedly, there is a history of Indigenous communities, including Māori, being impacted negatively by quantitative research, which is generally imposed onto these communities, and have been used to reinforce negative stereotypes and reproduce deficit narratives (; ; ).

In the absence of a formal constitution, the Treaty of Waitangi Act 1975 was an important step in recognising the legal relevance of the Treaty of Waitangi and Te Tiriti as foundational documents of Aotearoa. The Act also led to the establishment of the Waitangi Tribunal, which allowed a legal pathway for Māori to redress historical grievances and injustices, such as land confiscation. In 1988, the NZ Royal Commission on Social Policy examined Te Tiriti, and the Treaty of Waitangi, and through their analysis identified the principles of (1) Partnership, (2) Protection, and (3) Participation. These principles imply that the crown has an obligation to recognise and empower Māori self-determination aspirations, and to protect Māori interests.

Currently, government agencies within Aotearoa are moving towards developing policies, legislation, and frameworks to fulfil their Tiriti obligations. For example, Manatū Hauora (NZ Ministry of Health) identified the principles of Tino Rangatiratanga (Māori self-determination), Equity, Active Protection, Options, and Partnership in a framework for delivering their services (). The Data and Statistics Act 2022, whose purpose is to ensure high quality, impartial, and objective official statistics, acknowledges the Crown’s responsibility to give effect to Te Tiriti and requires ‘the Statistician’ to engage with Māori communities when collecting data and acknowledging Māori interests in data (see Sections 14(b) and 14(c)). This Act repealed the Statistics Act 1975, which made no mention or acknowledgement of Te Tiriti.

Te Tiriti Principles and MDSov

The principles outlining MDSov are fundamental concepts within Te Ao Māori (the Māori world), correspond to the principles of Te Tiriti (Partnership, Protection, and Participation), and are applicable to all Māori data. The establishment of the Kāwanatanga within the existing and established authority of Rangatira in Article 1 describes a partnership and a duty of care that each group has towards the other. The principles of Manaakitanga and Whakapapa speak to the principle of Partnership. Manaakitanga speaks to issues of respect for Māori data, and that free, prior, and informed consent underpins the collection, use and dissemination of Māori data. Whakapapa speaks to the acknowledgement of the genealogy of Māori data, the importance of data disaggregation, and decision-making around the use of Māori data to minimise future harms.

Article 2 of Te Tiriti acknowledges the existing authority of the Rangatira and their property rights over their taonga, speaking directly to the Tiriti principle of Protection. Both Dewes () and Hudson et al. () argue that Māori data are observations of the world around them, and is a source of information about Māori people, Māori language, and Māori environments, which therefore constitutes a taonga. Recently, Waitangi Tribunal inquiries and reports such as WAI 262 () and WAI 2522 () formally recognised that all data have the potential to be taonga, reinforcing the assertion that the crown has a responsibility to protect Māori rights to data as affirmed by Te Tiriti. The Rangatiratanga principle speaks to Māori having control over their data and how it is used, the right to physically store Māori data in Aotearoa and the right to use Māori data in ways that empowers self-determination and furthers Māori aspirations. The Kaitiakitanga principle acknowledges that Māori have rights and obligations over their data, including the obligation to be responsible stewards over Māori data, that appropriate Māori ethical principles underpin the protection processes, and that Māori should decide what data is tapu (restricted) or noa (open).

The Te Tiriti principle of Participation relates to the equal individual rights for all citizens of Aotearoa and is enshrined in Article 3. The MDSov principles of Whanaungatanga and Kotahitanga stem from the Participation principle. Whanaungatanga speaks to the balancing of individual and collective rights, benefits and risks, and the accountabilities of individuals and organisations that are responsible for the generation, management, access, etc. of Māori data, to the individuals, communities, and organisations of who the data derives from. Kotahitanga speaks to the idea that Māori should be able to derive individual and collective benefits from Māori data ecosystems, that capacity building and development of a Māori data workforce is needed, and that connections of other indigenous people with Māori must be encouraged.

The MASov Principles

The MDSov principles are Tiriti-centred and fit for the purpose of guiding appropriate governance and use of Māori data. Since algorithms are a specific use of data, we use the same six principles to underpin MASov. Many of the sub-principles remain, but the details of each are contextualised to the use of algorithms – including the use of Māori data, the computational algorithm, and the generated outputs of the system. The authors, who are a mix of data practitioners, Indigenous researchers, ethicists, and public policy analysts, held several meetings where the sub-principles were formulated. Preliminary results were presented to several different scholars for feedback and were also presented at two Indigenous conferences. Workshops were held to garner feedback, which was then used to change and refine principles and sub-principles.

Below are the MASov principles and sub-principles. Note that when referring to algorithms below, we mean all algorithms built that either (1) involve Māori data, (2) used to make decisions about Māori, (3) used to make decisions about environments that Māori have rights and interests in, or any combination of all three.

Rangatiratanga/Authority

Control – Māori have the right to control the development, and use of an algorithm, including (but not limited to) motives, design, choice of inputs, interpretation of outputs, maintenance, management, and deployment.
Jurisdiction – Decisions about the physical and virtual storage of the inputs and computational algorithms used, and the outputs generated from the algorithms shall enhance control for current and future generations. Whenever possible, the inputs and the outputs of the algorithms shall be stored in Aotearoa New Zealand.
Self Determination – Māori have the right to participate in the development and use of algorithms in a way that empowers sustainable self-determination and effective self-governance.

Whakapapa/Relationships

Transparency – Transparency in all aspects of the algorithm, including (but not limited to) who is involved, motivations, data and data provenance, outputs, management, maintenance, and deployment, should be clear prior to the application of the algorithm to ensure explainability.
Data Relationship – The use of Māori data throughout the algorithm process should be clear, and uphold the principles set out in MDSov.
Sustainability – It must be shown that the data and outputs used and generated from algorithms must provide long-term sustainable benefits to Māori, including environmental sustainability.

Whanaungatanga/Obligations

Balancing Rights – Individuals’ rights, risks, and benefits in relation to the algorithms need to be balanced with the collectives they may be a part of.
Redress – Māori have the right to challenge the output or outcome of an algorithm if applied to them, and mechanisms for redress must be established in the process of algorithm development.
Accountability – Individuals and institutions that are responsible for the development of the algorithms are accountable to the Māori individuals and communities that the algorithm affects.

Kotahitanga/Collective Benefits

Benefit – Algorithms must be designed in ways that enable Māori to derive both individual and collective benefits, and to minimize harms.
Capacity Building – Individuals and institutions developing and using algorithms must include Māori in all parts of the process for meaningful partnership and to build capability for both Māori and non-Māori.
Solidarity – Māori must be supported to connect with other Indigenous groups for the purposes of sharing knowledge, ideas, and strategies regarding the development and use of algorithms. Where appropriate, Māori should also be supported to work with other groups that face discrimination from algorithms.

Manaakitanga/Reciprocity

Respect – The use of algorithms shall uphold the mana (respect) and dignity of Māori individuals and communities.
Privacy – Individual and collective privacy must be considered during the processes of data collection, storage, data re-use, and the dissemination of the outputs of the algorithm.
Consent – Any Māori community that an algorithm is applied to must give free, prior, and informed consent, for both the development and use of the system. This includes consents for data, outputs, and elements of the system that Māori control.

Kaitiakitanga/Guardianship

Protection – Inputs used in the algorithms and the resulting outputs must be treated in such a way that enables and reinforces the capacity of Māori to exercise kaitiakitanga over all components of the algorithm, including the inputs, outputs, and computational algorithms.
Ethics – Tikanga, kawa (protocols), and mātauranga (knowledge) shall underpin the protection, access, and use of the algorithms.
Restrictions – Māori shall decide how the inputs and outputs of the algorithms shall be considered tapu (restricted) or noa (accessible).

More on the MASov Principles

All MDSov sub-principles have been extended to include data, computational algorithms, and outputs, where necessary. There are issues with algorithms that are unique, and so we have changed and developed new sub-principles to address these issues. We note these changes below:

Context (Whakapapa) has changed to Transparency, which acknowledges data genealogy and provenance, and extends this to all aspects of the algorithm to ensure that the algorithm is explainable.
Data Disaggregation (Whakapapa) has changed to Data Relationship to ensure the way Māori data is used within an algorithm is tracked and that Māori data within an algorithm adheres to MDSov principles.
Future Use (Whakapapa) has changed to Sustainability, to acknowledge that Māori have an interest in protecting their environments, and to ensure that the long-term benefits of an algorithm are sustainable.
Redress (Whanaungatanga) was added to provide a mechanism for Māori to challenge the output or outcome of an algorithm that was applied to them. The right of redress is a Tiriti principle and is a widely discussed issue in responsible algorithms.
Connect (Kotahitanga) has changed to Solidarity, to highlight that unity with all communities facing discrimination by algorithms (including Indigenous communities) is important and can benefit all.
Privacy (Manaakitanga) has been added to ensure privacy of individuals and collectives is upheld and maintained through the algorithm process.

The set of principles underpinning responsible algorithm development, deployment, and use, as stated in Section 2 and Appendix 1, are noble and important principles to adhere to. The MASov principles and sub-principles touch on all the responsible algorithm principles. Moreover, they expand on issues specifically pertaining to Māori values such as the idea of stewardship of data and outputs (as opposed to ownership), highlight the importance of Māori culture, protocols, and knowledge when exercising stewardship over Māori data, the right of redress, and expand more on the issues of free, informed, and prior consent. Table 1 indicates which responsible algorithm principles correspond with the MASov principles.

Table 1

The responsible algorithm principles alignment with MASov Principles.


MASov PRINCIPLE	RESPONSIBLE ALGORITHM PRINCIPLES

Rangatiratanga	Fairness and Justice, responsibility, beneficience, freedom, trust, dignity

Whakapapa	Transparency, responsibility, non-maleficience, beneficience, sustainability

Whanaungatanga	Transparency, fairness and justice, responsibility, trust, solidarity

Kotahitanga	Fairness and justice, non-maleficience, beneficience, dignity, solidarity

Manaakitanga	Responsibility, privacy, trust, dignity

Kaitiakitanga	Transparency, fairness and justice, responsibility, privacy, trust, sustainability, solidarity

4. Using MASov and Principles

Government operationalised algorithms have been used in Aotearoa for some time and for many different applications (). The potential for algorithms and technologies causing harm through algorithmic bias has been recognised by the government. Several ethical frameworks for algorithm use have been developed along with the Algorithm Charter for Aotearoa NZ (; ). Several issues have been raised with the Algorithm Charter from a Māori perspective. The charter mentions that it is a commitment by government agencies to reflect the principles of Te Tiriti o Waitangi. The charter notes that, under its commitment to partnership, it will embed a Māori perspective in the development and use of algorithms, yet nothing about the right for Māori to be active participants in this process, or provisions for the protection of Māori and Māori taonga. It also mentions the charters inability to ‘…fully consider important considerations, such as Māori data sovereignty, as these are complex and require separate consideration’, though, as West et al. () points out, MDSov should be integral to algorithm development.

There is a need to ensure that algorithms used or developed in Aotearoa are equitable and work for Māori. West et al. () highlights the need for (a) the creation of a Māori values framework and tikanga guidelines to support automated decision-making design, development, use, and maintenance, (b) robust equity assessment protocols for algorithms, and (c) meaningful Māori participation in institutional algorithm self-assessment processes. The MASov principles defined in Section 3 are a solution to point (a) and can be used to develop frameworks to evaluate a particular algorithm and its potential for generating biased outputs, acting as a solution to point (b). The MASov principles may or may not increase meaningful Māori participation. However, it will provide non-Māori algorithm developers and users with some perspective of how algorithms should be applied to Māori communities.

In this section we put the MASov principles to use. We outline a strategy to investigate an algorithm that shows evidence of nested colonising bias through its outputs. We generate a framework for the assessment of an algorithm, where the MASov principles are used to generate in-depth questions that can investigate where, why, and how colonising biases may creep into the algorithm. We also put forward the idea of using MASov principles in the construction of Indigenised algorithms, where the algorithmic system and its processes are founded on the ideas of tikanga Māori.

Strategy

In Section 2, we define a generic structure of an algorithmic system. We acknowledge that algorithms come in various forms, and our generic structure we defined may not encompass the important component that may be present in a particular algorithm. However, we will assume for the purposes of this exercise that this is the structure of an algorithm we wish to analyse. The values in Section 3 give us a guide as to the types of questions we may wish to ask at each component to analyse where, why, and how colonising bias may creep into the system. We could provide a fixed set of questions as a framework, but the idea of developing a more dynamic and living framework has several advantages. First, the problem with a generic structure and fixed set of questions is that it is not adaptive enough for the wide world of algorithmic constructions. Our previous work using a more fixed framework for analysis worked well for some algorithms but was significantly harder for others. Secondly, though it used MDSov principles in the structuring of the questions, it was not immediately apparent in some cases how these principles applied to algorithms. Having a dynamic framework coupled with well-defined Māori tikanga principles applied specifically for algorithms allow for a greater chance of detecting how and where nested colonising bias creeps into the system.

The full investigation of nested colonising bias requires a Kaupapa Māori mixed methods approach (). Given we have a concerning algorithm that utilises Māori data, or is applied to Māori individuals or environments, the first requirement is to quantifiably assess whether the algorithm produces biased outcomes and determine the scale of the bias. If bias exists, we can qualitatively, and in a Māori way, assess the algorithm using the MASov principles to assess where, why, and how bias has crept into the system. If solutions can be found to tighten up the algorithm, another quantitative analysis to assess new outputs can be performed to quantify any existing biases and scale. The methods of the quantification of algorithmic bias are outside the scope of this paper, but for readers interested in case studies, see for example Bartley et al. (), Grother et al. (), and Yesiler et al. ().

The general strategy can be summarised in three steps: given we have an algorithmic system that produces biased outcomes for Māori:

Structure: define the key components of the algorithmic system and whether it contains nested colonising bias.
Framework: use the MASov principles to generate a framework to critically evaluate the components of the algorithm, providing insights to fix the system of its nested colonising biases.
Post-Analysis: perform an analysis where a comparison of bias in the fixed system and the old system is performed.

An example of generated table of questions in Appendix 2 is an example of a framework to assess each foundational component within an algorithm. The questions generated stem directly from the MASov principles, allowing Māori values to guide the deconstruction of the algorithm. In the tables, we have given a description of the component, prior information that may be needed, and a set of questions. The questions are structured under the headers of MASov principles. In practice, it may be more useful to ask questions pertaining to the sub-principles for a lower-level and detailed analysis.

Indigenised Algorithms

Algorithms already deployed are not embedded with Māori tikanga values and perspectives, and minimal consideration for Māori is usually given. Application of MASov principles for deconstructing and fixing existing algorithms are an incremental step towards change, but frustrating in the sense that the process of embedding tikanga values within algorithms should occur from the very start. What is required is transformational change in this space, where Indigenous values are embedded from the very start of development and continue throughout, from motives to decisions, and throughout the lifecycle of the algorithm. Figure 2 displays the idea of an indigenous algorithm, where an algorithm is housed in a structure founded on tikanga principles. The MASov principles are the tikanga values that could lay the foundation for the indigenous values, and whilst we recommend that they be used to retrofit existing algorithms, we would prefer that they be used as the foundational principles for the development of new, transformative, Indigenised algorithms.

Figure 2

An algorithm housed in a structure where tikanga values are at the foundation of the system. Using the MASov principles as the Tikanga values that is the foundation of the algorithmic system ensures that the system is Indigenised – that Māori perspectives underpin the systems, and that Māori needs are at the forefront of outcomes.

5. Discussion

With algorithms becoming ever more ubiquitous, there is a growing need to ensure that these technologies are not used to perpetuate biases and cause harm to marginalised communities, including indigenous peoples. Algorithms currently deployed in Aotearoa have the potential of harming Māori, hence it is important that Māori can actively and meaningfully participate in the deployment, development and use of algorithms that are applied to them. In this paper we have presented the idea of MASov and defined its core principles. The principles are an extension of MDSov principles and are based on Te Ao Māori values and Te Tiriti principles. We also introduce the strategy of producing frameworks using these principles to investigate existing algorithms to understand, if evidence of colonising bias exists, how, where, and why it creeps into the system. Solutions may then be found to fix these issues, such that the algorithm can produce fairer results.

This work has focussed on data and algorithmic sovereignty with respect to Māori as the Indigenous peoples of Aotearoa. However, we believe the high-level idea of algorithmic sovereignty can be applied to other Indigenous nations. Although there are similar aspirations that Indigenous nations have been fighting for, such as sovereignty and self-determination, the principles that underpin algorithmic sovereignty will change depending on the Indigenous nation that is defining it. If the idea of algorithmic sovereignty adds value to an Indigenous nation, we encourage the uptake of the idea, and for the nation to define their own principles and values that underlie the idea.

To minimise the risks data and algorithms impose on Māori communities, it is vitally important that those working with Māori data, and those involved with developing algorithmic systems that can impact Māori individuals and communities, apply MASov principles and values to their work. We encourage building meaningful partnerships with Māori to ensure active participation that allows Māori communities to protect themselves as individuals and collectives, and to protect their taonga with their brand of kaitiakitanga. Increasing capacity in these technical fields is also important, and we encourage organisations, government agencies, schools, and universities of Aotearoa to develop and fund the training of Māori computer scientists and data practitioners (scientists, engineers, etc.) who can help build the next generation of Māori algorithmic systems.

Appendix 1 Principles for Responsible AI/Algorithms

Below are the principles of responsible algorithms as described in Jobin et al. ().


PRINCIPLE	MEANING (IN THE CONTEXT OF ALGORITHMS)	RELATED PRINCIPLES

Transparency	Algorithms should be open, explainable, and explicit with regards to their purpose, development, use and maintenance, Efforts should be made to increase the explanability of how it works, interpretability of outputs, and understandability of the system.	Explainability, explicability, understandability, interpretability, communication, disclosure, non-opaque, showing.

Fairness and Justice	Outputs of algorithmic systems should be free of algorithmic bias, or at the very least, tested for bias and disclose findings, Purpose of the outputs must allign with the ideas of fairness, equity and inclusion, and should be explanable enough so that the outputs can be challenged.	Consistency, inclusion, equality, equity, non-biased, non-discriminatory, diversity, plurality, accessibility, reversibility, remedy, redress, challenge, access, distribution.

Non-Maleficence	Algorithms should be safe and secure, and should not purposely cause foreseeable or unintentional harm (discrimination, physical harm, violation of privacy etc.).	Security, safety, harm minimisation, protection, precaution, prevention, integrity, non-subversion.

Responsibility	Use of algorithms should be done with integrity, and the allocation of responsibility, obligations, and legal liability should be clear in all parts of the process. A focus on the underlying sources of harm is necessary, as is the focus on diversity, inclusion, and participation of all relevant groups.	Responsible, accountability, liability, obligations, acting with integrity, participation.

Privacy	Algorithms require data, and privacy (typically presented in relation to data protection and security) is a value to uphold, and as a protected right!	Personal or Private information, anonymisation

Beneficience	Algorithms should be used in such a way that promotes human wellbeing and flourishing, peace, happiness, creation of socio-economic opportunites and prosperity.	Benefits, well-being, flourishing peace, social good, common good

Freedom	Use of algorithms should promote freedoms, empowerment, autonomy, and self-determination through democratic means, There must be freedoms to withdraw consent, and individuals must be free from manipulation, surveillence, or technological experimentation.	Autonomy, consent, choice, self-determination, liberty, empowerment

Trust	Trust in Algorithms refers to the algorithm having a noble purpose, used by trustworthy individuals and organisations, built on good design principles, and should aspire to gain the trust of stakeholders.	Trust, Purpose

Sustainability	Development and deployment of algorithms should consider the protection of the environment, improving Earth’s ecosystem and biodiversity, contribution to fair and equitable societies, and the promotion of peace.	Environment (nature), energy, resources (energy)

Dignity	Algorithms should uphold human rights, It should not diminish or destroy, but preserve or increase human dignity.	Human dignity

Solidarity	Algorithms have a large implication on the labour market, so benefits of the algorithm should uphold strong safety nets, wealth generated should be redistributed to those whose labour has been taken through automation.	Solidarity, social security (welfare), cohesion, redistribution of benefits

Appendix 2 Generated Framework Using MASov Principles

Below are generated questions using the MASov principles, for the algorithm components of process, motives, inputs, and outputs.


Process	The Process component looks at the algorithm in its entirety. Important information to extract here are the people/organisations involved in the creation, development, maintenance, ownership, and funding. It is also important to understand the intent of the algorithm, how the data is being protected, and how the system will be maintained throughout its use. Questions to investigate the algorithm come straight from the MASov principles, which will provide a high-level look at the system.

Example Questions	Rangatiratanga: What controls do Māori have in all stages of the development of the algorithm? Whakapapa: Has this algorithmic system been used previously on Māori or other indigenous communities? For what purpose? Whanaungatanga: How are individuals and institutions involved in the development and use of the system accountable to Māori? Kotahitanga: What strategies for capacity building are there to ensure technical literacy in the Māori communities to which the algorithm applies? Manaakitanga: Have the necessary Māori individuals and communities given free, informed, and prior consent for their data to be used in the algorithm? Kaitiakitanga: Do Māori ethics underpin the protection, access, and use of the algorithm?

Motives	The Motives component looks at understanding what problems the algorithm is solving (or goals it is trying to achieve) and asks questions involving what the problem/goals are, and who is involved when defining the problems/goals, and where/if Māori consultation has been sought. This helps understand if an algorithmic system is the correct tool for solving the problem/achieving the goal. Preliminary steps before analysis would be to understand the underlying motivations for the algorithm, and whose motivations are driving the process, what the system is trying to achieve, who is involved in the process, and if Māori in any way have been involved in defining the motivations and in what capacity?

Example Questions	Rangatiratanga: Do the motivations/purpose of the algorithm further Māori collective aspirations? Whakapapa: Are the motivations underlying the use of the algorithm clear in providing future benefit to Māori Whanaungatanga: Which individuals and institutions have defined the motivations of the algorithm, and what are their obligations to Māori? Kotahitanga: What harms and benefits do the motivations provide for Māori? Manaakitanga: Do the motivations uphold and maintain dignity for Māori individuals and communities? Kaitiakitanga: Do Māori have the right to change the motivations if tikanga values are not involved in the construction of the motivations?

Inputs	The Inputs component looks specifically at what data, variables, and other inputs (such as weights, priors, model specifications) have been chosen to be used in the computational algorithm. The inputs all depend on the decisions made and the data collected throughout the process so far, so it is important to ask questions regarding the consistency of the process thus far as we approach this pivotal step of the process. Prior to analysis, it is important to understand the technical components of the process, including understanding what variables have been defined for the eventual model, what data has been used and whether it is sufficient, details surrounding the model, and the algorithms that could be considered useful to run with the data available.

Example Questions	Rangatiratanga: What controls do Māori have to determine what inputs are tapū (closed) or noa (open)? Whakapapa: Do the inputs used align with the (Māori) motivations of the algorithm? Whanaungatanga: Who (individuals/institutions) is responsible for the protection of inputs (Māori data)? Kotahitanga: What are the strategies to build technical capacity and knowledge for Māori issues surrounding the protection of inputs? Manaakitanga: Have appropriate Māori communities given consent for the application of their data being used in the algorithm? Kaitiakitanga: Do the inputs have the necessary protocols in place for protection and security?

Outputs	Once inputs are chosen, they are plugged into the computational algorithms, and outputs are generated. The Outputs component looks at how the outputs are interpreted and communicated to decision-makers. Other important aspects involve access to outputs, benefit-sharing, capacity building, and if/how Māori are involved with the interpretation of outputs. Important things to understand is who is analysing and interpreting the outputs, the quality of the outputs, and who decides if outputs are correct or relevant to the motivations of the algorithm. Since outputs are newly generated knowledge created from inputs, it is important that all outputs that are about Māori are treated with the same care and respect as Māori data.

Example Questions	Rangatiratanga: Do the outputs and the analysis and interpretation of the outputs contribute to Māori self-determination and aspirations? Whakapapa: Are outputs about Māori consistent with the inputs (specifically Māori data)? Whanaungatanga: What are the obligations of the individuals and institutions that generate new outputs, to the Māori individuals and communities that the outputs describe? Kotahitanga: Have the outputs been interpreted from the correct Māori lens? Manaakitanga: Do the findings of the outputs, and the analysis and interpretation of the outputs uphold Māori dignity? Kaitiakitanga: Are the outputs about Māori treated the same as Māori data with respect to controls and protections?

Acknowledgements

The authors would like to thank Maui Hudson for his feedback, and the feedback from the participants of the 1st and 2nd Māori AI Wānanga and Workshops, both held at the University of Waikato on the 30–31st August 2022, and the 15th September 2023 respectively.

Funding Information

This work was funded by MBIE Tikanga in Technology (TinT), project no. 108303. Paul Brown’s research was also funded by an MBIE Science Whitinga Fellowship, project no. 109120.

Competing Interests

The authors have no competing interests to declare.

References

Angwin, J, Larson, J, Mattu, S and Kirchner, L. 2016. Machine Bias. Propublica, 23 May. Available at http://propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.
Bartley, N, Abeliuk, A, Ferrera, E and Lerman, K. 2021. Auditing algorithmic bias on Twitter. In: Websci ’21: Proceedings of the 13th ACM Web Science Conference, Stuttgart, Germany on 21–25 June 2021, pp. 65–73. DOI: https://doi.org/10.1145/3447535.3462491
Beller, J. 2018. The message is murder: Substrates of computational capital. London: Pluto Press. DOI: https://doi.org/10.2307/j.ctt1x07z9t
Bender, EM, Gebru, T, McMillan-Major, A and Shmitchell, S. 2021. On the dangers of stochastic parrots: Can language models be too big? In: FAccT ‘21: Proceedings of the 2021 ACM Conference on Fairness, Accountability and Transparency, Rio de Janeiro, Brazil on 3–10 March 2021, pp. 610–623. DOI: https://doi.org/10.1145/3442188.3445922
Benjamin, R. 2019. Race after technology: Abolitionist tools for the New Jim Code. London: Polity. DOI: https://doi.org/10.1093/sf/soz162
Berardi, F. 2009. The soul at work. Cambridge: MIT Press.
Boscarino, N, Cartwright, RA, Fox, K and Tsosie, KS. 2022. Federated learning and Indigenous genomic data sovereignty. Nature Machine Intelligence, 4(11): 909–911. DOI: https://doi.org/10.1038/s42256-022-00551-y
Brynjolfsson, E and McAfee, A. 2014. The second machine age: Work, progress, and prosperity in a time of brilliant technologies. New York: WW Norton.
Buolamwini, J and Gebru, T. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research, 81: 77–91.
Carroll, SR, Garba, I, Figueroa-Rodriguez, OL, Holbrook, J, Lovett, R, Materechera, S, Parsons, M, Raseroka, K, Rodriguez-Lonebear, D, Rowe, R, Sara, R, Walker, JD, Anderson, J and Hudson, M. 2020. The care principles for indigenous data governance. Data Science Journal, 19(4): 1–12. DOI: https://doi.org/10.5334/dsj-2020-043
Checketts, L. 2022. Artificial intelligence and the marginalisation of the poor. Journal of Moral Theology, 11(1): 87–111. DOI: https://doi.org/10.55476/001c.34125
Dewes, TK. 2017. He taonga ranei tēnei mea te raraunga – How is data a taonga? Report submitted for MAOR591 Directed Study. Hamilton, NZ: University of Waikato.
Dourish, P. 2016. Algorithms and their others: Algorithmic culture in context. Big Data and Society, 3(2): 1–11. DOI: https://doi.org/10.1177/2053951716665128
Dressel, J and Farid, H. 2018. The accuracy, fairness, and limits of predicting recidivism. Science Advances, 4(1): 1–5. DOI: https://doi.org/10.1126/sciadv.aao5580
Grother, P, Ngan, M and Hanaoka, K. 2019. Face recognition vendor test (FRVT) Part 3: Demographic effects. National Institute of Standards and Technology. DOI: https://doi.org/10.6028/NIST.IR.8280
Henderson, P, Hu, J, Romoff, J, Brunskill, E, Jurafsky, D and Pineau, J. 2020. Towards the systematic reporting of the energy and carbon footprints of machine learning. Journal of Machine Learning Research, 21(248): 1–43.
Hudson, ML, Anderson, T, Dewes, TK, Temara, P, Whaanga, H and Roa, T. 2017. He Matapihi ki te Mana Raraunga: Conceptualising Big Data through a Māori lens. In: Whaanga, H, Keegan, TTAG and Apperley, M (eds.), He Whare Hangarau Māori – Language, culture & technology. pp. 64–73. Hamilton, New Zealand: Te Pua Wānanga ki te Ao / Faculty of Māori and Indigenous Studies, the University of Waikato.
Hudson, ML and Russel, K. 2009. The Treaty of Waitangi and research ethics in Aotearoa. Bioethical Inquiry, 6(1): 61–68. DOI: https://doi.org/10.1007/s11673-008-9127-0
Huws, U. 2014. Labor in the Growing Digital Economy: The cybertariat comes of age. New York: NYU Press.
Jackson, M. 2019. In the end “the hope of decolonisation”. In: McKinley, A and Smith, LT (eds.), Handbook of Indigenous Education. Springer Nature. pp. 101–110. DOI: https://doi.org/10.1007/978-981-10-3899-0_59
Jobin, A, Ienca, M and Vayena, E. 2019. The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1(9): 389–399. DOI: https://doi.org/10.1038/s42256-019-0088-2
Kordzadeh, N and Ghasemaghaei, M. 2022. Algorithmic bias: Review, synthesis, and future research directions. European Journal of Information Systems, 31(3): 388–409. DOI: https://doi.org/10.1080/0960085X.2021.1927212
Kukutai, T and Taylor, J. (eds.) 2016a. Indigenous data sovereignty: Toward an agenda, Vol. 38. Canberra, AU: ANU Press. http://www.jstor.org/stable/j.ctt1q1crgf. DOI: https://doi.org/10.22459/CAEPR38.11.2016
Kukutai, T and Taylor, J. 2016b. Data sovereignty for indigenous peoples: Current practice and future needs. In: Kukutai, T and Taylor, J (eds.), Indigenous Data Sovereignty: Toward and Agenda, Vol. 38. Canberra, AU: ANU Press. DOI: https://doi.org/10.22459/CAEPR38.11.2016.01
Kukutai, T, Cassim, S, Clark, V, Jones, N, Mika, J, Morar, R, Muru-Lanning, M, Pouwhare, R, Teague, V, Tuffery Huria, L, Watts, D and Sterling, R. 2023a. Māori data sovereignty and privacy. Tikanga in Technology discussion paper. Hamilton: Te Ngira Institute for Population Research.
Kukutai, T, Campbell-Kamariera, K, Mead, A, Mikaere, K, Moses, C, Whitehead, J and Cormack, D. 2023b. Māori data Governance model. Te Kāhui Raraunga.
Lambrecht, A and Tucker, C. 2019. Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads. Management Science, 65(7): 2966–2981. DOI: https://doi.org/10.1287/mnsc.2018.3093
Lamdan, S. 2022. Data Cartels: The Companies that Control and Monopolize Our Information. Stanford, CA: Stanford University Press. DOI: https://doi.org/10.1515/9781503633728
Mackey, TK, Calac, AJ, Chenna Keshava, BS, Yracheta, J, Tsosie, KS and Fox, K. 2022. Establishing a blockchain-enabled Indigenous data sovereignty framework for genomic data. Cell, 185(15): 2626–2631. PMID: 35868267. DOI: https://doi.org/10.1016/j.cell.2022.06.030
Martel, R, Shepherd, M and Goodyear-Smith, F. 2022. He awa whiria – A braided river: An indigenous Māori approach to mixed methods research. Journal of Mixed Methods Research, 16(1): 17–33. DOI: https://doi.org/10.1177/1558689820984028
Martin, K. 2019. Designing ethical algorithms. MIS Quarterly Executive, 18(5): 2. DOI: https://doi.org/10.17705/2msqe.00012
Meijas, UA and Couldry, N. 2019. The costs of connection: How data is colonizing human life and appropriating it for capitalism. Stanford, CA: Stanford University Press. DOI: https://doi.org/10.1515/9781503609754
Milne, BJ, Atkinson, J, Blakely, T, Day, H, Douwes, J, Gibb, S, Nicholson, M, Shackleton, N, Sporle, A, and Teng, A. 2019. Data resource profile: The New Zealand integrated data infrastructure (IDI). International Journal of Epidemiology, 48(3): 677–677e. DOI: https://doi.org/10.1093/ije/dyz014
Moewaka Barnes, H and McCreanor, T. 2019. Colonisation, Hauora, and whenua in Aotearoa. Journal of the Royal Society of New Zealand, 49(1): 19–33. DOI: https://doi.org/10.1080/03036758.2019.1668439
Munn, L. 2017. I am a driver-partner. Work, Organisation, Labour and Globalisation, 11(2): 7–20. DOI: https://doi.org/10.13169/workorgalaboglob.11.2.0007
Munn, L. 2023. The five tests: designing and evaluating AI according to indigenous Māori principles. AI & Society. DOI: https://doi.org/10.1007/s00146-023-01636-x
O’Neil, C. 2016. Weapons of math destruction: How big data increases inequality and threatens democracy. Broadway Books.
Olhede, SC and Wolfe, PJ. 2018. The growing ubiquity of algorithms in society: Implications, impacts and innovations. Philosophical Transactions of the Royal Society A, 376(1): 20170364. DOI: https://doi.org/10.1098/rsta.2017.0364
Pool, I. 2016. Colonialism’s and postcolonialism’s fellow traveller: The collection, use and misuse of data on indigenous people. In: Kukutai, T and Taylor, J (eds.), Indigenous Data Sovereignty: Toward an Agenda. ANU Press. pp 57–78. DOI: https://doi.org/10.22459/CAEPR38.11.2016.04
Rainie, SC, Kukutai, T, Walter, M, Figueroa-Rodriguez, OL, Walker, J and Axelsson, P. 2019. Issues in open data: Indigenous data sovereignty. In: Davies, T, Walker, S, Rubinstein, M and Perini, F (eds.), The State of Open Data: Histories and Horizons. Cape Town and Ottawa: African Minds and International Development Research Centre. pp. 300–319.
Smith, LT. 2012. Decolonising Methodologies: Research and Indigenous Peoples. 2nd Ed. Zed Books Ltd.
Someh, I, Davern, M, Breidbach, CF and Shanks, G. 2019. Ethical issues in big data analytics: A stakeholder perspective. Communications of the Association of Information Systems, 44(1): 34. DOI: https://doi.org/10.17705/1CAIS.04434
Stats, NZ. 2018. Algorithm Assessment Report. Available from https://data.govt.nz/use-data/analyse-data/government-algorithm-transparency.
Stats, NZ. 2020a. Ngā Tikanga Paihere: a framework guiding ethical and culturally appropriate data use. Available from https://data.govt.nz/toolkit/data-ethics/nga-tikanga-paihere/.
Stats, NZ. 2020b. Algorithm Charter for Aotearoa New Zealand. Available from https://data.govt.nz/assets/data-ethics/algorithm/Algorithm-Charter-2020_Final-English-1.pdf.
Strubell, E, Ganesh, A and MacCallum, A. 2019. Energy and policy considerations for deep learning In: NLP. Proceedings of the 57th Annual Meeting of the Association of Computational Linguistics, pp. 3645–3650. DOI: https://doi.org/10.18653/v1/P19-1355
Taylor Fry. 2021. Algorithm Charter for Aotearoa New Zealand: Year 1 review. Available at https://www.data.govt.nz/assets/data-ethics/algorithm/Algorithm-Charter-Year-1-Review-FINAL.pdf.
Te Mana Raraunga. 2016. Te Mana Raraunga Māori Data Sovereignty Network Charter. Available at https://static1.squarespace.com/static/58e9b10f9de4bb8d1fb5ebbc/t/5bda208b4ae237cd89ee16e9/1541021836126/TMR+Ma%CC%84ori+Data+Sovereignty+Principles+Oct+2018.pdf.
Waitangi Tribunal. 2011. Ko Aotearoa Tēnei (WAI 262). Wellington, NZ: Waitangi Tribunal. Available at https://forms.justice.govt.nz/search/WT/reports/reportSummary.html.
Waitangi Tribunal. 2016. Report on the Trans-Pacific Partnership Agreement. Wellington, NZ: Waitangi Tribunal.
Waitangi Tribunal. 2019. Hauora – Report on Stage One of the Health Services and Outcomes Kaupapa Inquiry (WAI 2575). Wellington, NZ: Waitangi Tribunal. pp 163–164.
Walter, M and Anderson, C. 2013. Indigenous Statistics: A Quantitative Research Methodology. Left Coast Press, California.
Walter, M and Kukutai, T. 2018. Artificial intelligence and Indigenous data sovereignty. Input paper for the Horizon Scanning Project, “The Effective and Ethical Development of Artificial Intelligence: An Opportunity to Improve Our Wellbeing”. Australian Council of Learned Academies. Available at www.acola.org.
Walter, M and Suina, M. 2019 Indigenous data, indigenous methodologies and indigenous data sovereignty. International Journal of Social Research Methodology, 22(3): 233–243. DOI: https://doi.org/10.1080/13645579.2018.1531228
Waziyatawin and Yellow Bird, M. 2005. For Indigenous Eyes Only: A Decolonization Handbook 1. New Mexico, AZ: SAR Press.
West, K, Wilson, D, Thompson, A and Hudson, M. 2020. Māori perspectives on trust and automated decision-making. Report for the Digital Council Aotearoa New Zealand.
Will, P, Krpan, D and Lordan, G. 2023. People versus machines: introducing the HIRE framework. Artificial Intelligence Review, 56(1): 1071–1100. DOI: https://doi.org/10.1007/s10462-022-10193-6
Yesiler, F, Marius, M, Serra, J and Gomez, E. 2022. Addressing algorithmic biases for musical version identification. In: WSDM ’22: Proceedings of the 15th ACM International Conference on Web Search and Data Mining, pp. 1284–1290. DOI: https://doi.org/10.1145/3488560.3498397