The Use of Large Language Models in National Security: Balancing Innovation with Ethical Responsibility

On Large Language Models in National Security Applications

Caballero, William N., and Phillip R. Jenkins. "On large language models in national security applications." arXiv preprint arXiv:2407.03453 (2024). 

Link to article: https://arxiv.org/abs/2407.03453

Integrating large language models (LLMs) into national security applications has sparked intense debate among stakeholders, including government agencies, technologists, and librarians. While LLMs like GPT-4 hold the potential to transform intelligence and defense operations through efficient data processing and rapid decision support, they also bring significant ethical and operational challenges. For librarians, who have a deep commitment to privacy, information ethics, and public trust, LLM use in such high-stakes areas raises several concerns. This essay examines the advantages and risks of LLMs in national security, addressing the technology's ability to enhance operations and the ethical and practical objections from information professionals.

The Transformative Potential of LLMs in National Security

LLMs have demonstrated exceptional capabilities in processing and analyzing vast amounts of unstructured data, making them attractive tools in the national security domain. Their ability to quickly summarize documents, detect patterns, and provide insights aligns well with the information-heavy demands of national defense and intelligence operations. Agencies like the U.S. Department of Defense (DoD) are experimenting with LLMs to streamline labor-intensive tasks, such as summarizing intelligence reports, automating administrative duties, and facilitating wargaming simulations. These applications not only promise to reduce human workload and accelerate decision-making but also hold the potential to significantly enhance operational readiness, ushering in a new era of national security.

For example, the U.S. Air Force has integrated LLMs to automate report generation and streamline data analysis in flight testing. By automating repetitive tasks, LLMs allow analysts and decision-makers to allocate their expertise toward more strategic functions. In addition, the technology's integration with machine learning and statistical forecasting tools allows for more comprehensive threat assessments and predictive modeling, supporting the military's goal of maintaining a competitive edge in a rapidly evolving geopolitical landscape.

However, while LLMs provide clear advantages, their deployment in national security introduces a complex set of ethical, operational, and practical challenges that must be addressed. These concerns are paramount for librarians, as they touch on fundamental principles of privacy, transparency, and information accuracy.

Privacy and Data Protection: A Core Librarian Concern

Privacy is a cornerstone of librarianship, and LLM deployment in national security settings raises pressing questions about data protection and user confidentiality. LLMs require vast datasets to train and operate effectively, often including sensitive or personal information. When applied to national security, LLMs may access classified or confidential data, raising the stakes for data protection. The potential for unauthorized access to such information could lead to severe privacy violations and misuse, infringing on individuals' rights and compromising national security. This potential misuse underscores the urgent need for strict ethical guidelines in using LLMs.

The DoD has acknowledged these risks and has taken steps to address them by experimenting with "sandbox" environments to test LLM applications under controlled conditions. Task Force Lima, for instance, has established protocols to examine low-risk LLM applications, focusing on ethical and secure uses of the technology. However, librarians may still question whether such safeguards are sufficient, given the potential for data breaches or adversarial attacks. If LLMs in national security are not carefully protected, they could become targets for cyber threats, posing risks to individual privacy and broader public safety.

Accuracy and Reliability: The Problem of Hallucinations

LLMs, while highly advanced, are prone to generating "hallucinations"—plausible yet incorrect or misleading responses. These hallucinations are essentially the result of the model's predictive nature, which may generate responses that are not factually accurate but are plausible based on the input data. In national security, where precise information is essential for sound decision-making, the risk of hallucinations is especially problematic. If LLMs produce incorrect summaries or recommendations, they could misinform military commanders, leading to flawed strategies with potentially grave consequences. For librarians, this issue is critical because public trust hinges on the accuracy and reliability of information. In a library setting, inaccurate information affects user trust; in national security, it can impact lives.

Proponents argue that these hallucinations can be managed with human oversight and proper model tuning. However, librarians might counter that even with oversight, errors in LLM outputs may be more complicated to detect due to the sheer volume of information they process. In such scenarios, the potential for unnoticed inaccuracies remains a serious concern, cautioning against over-reliance on LLMs. Furthermore, the challenge of verifying LLM outputs—given their black-box nature—complicates the ability of human reviewers to catch and correct errors in real-time.

Transparency and Explainability: Addressing the Black Box

Transparency is central to librarianship, which values open access and traceability of information. LLMs, however, are often "black boxes"—complex systems that make decisions in ways that are not easily understandable or interpretable. This lack of transparency concerns librarians committed to helping users understand and critically assess information sources. In national security applications, the lack of explainability could lead to unchecked reliance on LLM outputs, making it difficult to determine the validity of their recommendations or understand their reasoning.

Supporters of LLMs argue that explainability tools, like SHAP values or model interpretability techniques, can offer insights into how LLMs make confident decisions. However, librarians might contend that these tools are only sometimes sufficient to guarantee full transparency, especially in high-stakes applications like national security. Without a clear understanding of how LLMs arrive at specific conclusions, the technology remains opaque, potentially leading decision-makers to trust outputs without fully understanding their accuracy or biases.

Bias and Fairness: Preventing Systemic Discrimination

Librarians are dedicated to providing unbiased and equitable information access, but LLMs often reflect biases inherent in their training data. Such biases could affect intelligence assessments, operational decisions, or risk evaluations in national security. For instance, if an LLM is trained on biased historical data, it might generate outputs that unfairly prioritize specific demographics or reinforce stereotypes in threat analyses. The potential for systemic discrimination is significant in scenarios where bias could influence policy decisions. The consequences of such discrimination could be severe, potentially leading to unfair treatment of certain groups or the reinforcement of harmful stereotypes, undermining national security operations' credibility and effectiveness.

Efforts to mitigate LLM bias include refining training datasets, using diverse sources, and incorporating bias-detection algorithms. Proponents argue that these techniques can effectively minimize harmful bias. Yet, librarians may remain skeptical, pointing out that no method is foolproof and that biases in training data can still manifest in subtle, hard-to-detect ways. Ensuring fair and unbiased outputs from LLMs is thus an ongoing challenge, particularly in national security settings where biases may have far-reaching implications. This ongoing nature of the challenge underscores the need for continuous vigilance and improvement in LLM applications to ensure fairness and equity.

Information Ethics and Intellectual Freedom: The Potential for Surveillance and Censorship

Librarianship is grounded in intellectual freedom and open access to information. Using LLMs in national security could conflict with these principles, mainly if they are applied to surveillance, censorship, or information control. For example, LLMs could monitor communications, analyze public sentiment, or track individuals' online activities, raising ethical questions about privacy and freedom of expression. Librarians advocating unrestricted access to information may view such uses as infringing on fundamental rights and freedoms.

In response, national security advocates might argue that surveillance is necessary to protect public safety and prevent threats. However, librarians might counter that such applications should be narrowly defined and carefully regulated to avoid misuse. Without clear ethical guidelines and oversight, the risk of LLMs being used to infringe upon intellectual freedom remains a point of concern.

The Changing Role of Human Information Professionals

As LLMs become more capable of automating tasks traditionally performed by human information professionals, librarians might question the impact of their roles and the value placed on human expertise. LLMs can already perform data summarization, information retrieval, and analysis tasks, potentially reducing the need for human input. In national security, where efficiency and speed are prioritized, the role of human librarians and analysts might shift, potentially undervaluing the ethical insights and critical thinking skills they bring to information work.

Supporters of LLMs may argue that rather than replacing humans, these models will augment human capabilities, allowing librarians and analysts to focus on more strategic responsibilities. However, librarians might remain wary of a future where automated systems increasingly assume roles that require ethical judgment and human empathy—qualities that are difficult to encode into AI models. As LLMs become more entrenched in information tasks, the importance of preserving human expertise in libraries and national security becomes even more evident.

Conclusion: Balancing Innovation with Ethical Responsibility

Applying LLMs in national security represents a dual-edged sword, with transformative potential on one side and ethical challenges on the other. While LLMs can enhance operational efficiency and support decision-making, they also raise significant concerns about privacy, accuracy, transparency, bias, intellectual freedom, and the evolving role of human professionals. For librarians, these concerns are about the immediate risks and the broader implications of relying on automated systems in areas that affect public safety and individual rights.

Balancing the benefits of LLMs with ethical responsibilities will require a collaborative effort across fields. National security professionals, technologists, and librarians alike must work together to develop guidelines, implement safeguards, and advocate for transparent, accountable use of LLMs. By approaching LLM integration with caution and a solid ethical framework, it may be possible to leverage these tools to enhance national security in ways that align with the values of privacy, fairness, and public trust that librarians uphold.

Real World Data Governance: How Generative AI and LLMs Shape Data Governance

The webinar focuses on the evolving role of generative AI (Artificial Intelligence) and large language models (LLMs) in shaping data governance practices. 

Introduction and Background

The speaker discusses the increasing significance of AI, specifically generative AI and LLMs, in data governance. While numerous organizations are still adopting these technologies, they rapidly reshape data governance management. Data governance encompasses the execution and enforcement of authority over data management and usage, while generative AI and LLMs introduce new capabilities to automate, enhance, and transform these traditional processes.

Context and Historical Milestone:  

AI, incredibly generative AI, gained significant attention in late 2022 with the release of tools like ChatGPT, which revolutionized natural language processing. Although these technologies are still considered cutting-edge for data governance, their potential is immense. The presenter emphasizes how AI will significantly alter the future of data governance in terms of compliance and automation, instilling a sense of optimism about the transformative power of these technologies.

Core Definitions and Technologies

To establish a foundation, the presenter defines critical terms:

Artificial Intelligence (AI): Artificial Intelligence (AI)  encompasses systems capable of performing tasks that typically require human intelligence, such as problem-solving, natural language processing, and learning from experience.

  

Generative AI: Generative AI  is a subset of AI focused on creating new content (e.g., text, images, or videos) based on examples it has been trained on. Unlike traditional AI, which focuses on specific tasks, generative AI can generate new material based on learned data patterns.

  

Large Language Models (LLMs): AI models trained on vast datasets to generate humanlike text responses. LLMs use deep learning techniques commonly used in ChatGPT and Google's Bard to provide responses or generate content.

Potential Uses of Generative AI and LLMs in Data Governance

The presenter identifies several ways these technologies can potentially shape data governance practices:

  

Streamlining Policy Creation: Generative AI can create dynamic data governance policies based on existing templates or frameworks, saving time and ensuring consistency across policy documents.

  

Compliance Monitoring and Automation: AI can monitor compliance with regulations by analyzing data and tracking policy adherence, enabling real-time compliance checks.

Data Quality Enhancement: AI can proactively detect anomalies in data, monitor data quality, and offer suggestions or automate the correction of data discrepancies. This potential of AI to enhance data quality can reassure the audience about the reliability of their data, instilling a sense of confidence in the data governance process.

Data Stewardship Customization: Generative AI can help customize and evolve data stewardship roles, aligning them more closely with organizational needs.

Privacy and Security Improvement: AI can enhance data privacy and security by analyzing and securing sensitive data. It can also ensure proper controls and protections are implemented according to organizational standards.

Automating Key Data Governance Tasks

AI and LLMs can automate several aspects of data governance, providing efficiency and improving accuracy in previously manual processes:

Data Classification: AI can classify vast amounts of data by applying rules based on learned patterns, automating what would otherwise be a manual task. This capability is handy for large organizations managing extensive data assets.

Documentation Generation: AI can create consistent and comprehensive documentation for data governance processes, improve metadata management, and help maintain records for auditing and compliance purposes.

Policy Enforcement and Adaptation: AI can translate written policies into actionable rules and help enforce them across data systems. It can also adapt policies as regulatory environments change, ensuring organizations remain compliant.

Data Stewardship Task Automation: AI can automate routine data stewardship tasks, supporting decision-making and consistently applying data standards. This automation can relieve data stewards from repetitive tasks, allowing them to focus on high-level strategic activities, reduce manual work, and increase efficiency.

Challenges and Considerations for Implementing AI in Data Governance

The presenter outlines critical issues:

Data Privacy and Security: While AI can enhance data security, it raises concerns about how sensitive data is handled, especially when integrated into LLMs. Strong encryption and anonymization techniques are necessary to protect data.

Bias and Fairness: AI models can unintentionally propagate biases in the data they are trained on. 

Ensuring fairness and minimizing bias is critical, and organizations need to audit and cleanse data before feeding it into AI systems.

Integration with Existing Systems: Integrating AI tools with existing data governance systems requires developing APIs and ensuring that AI is compatible with the organization's current infrastructure. This integration can be a slow, gradual process.

Scalability and Cost: AI implementation can be costly, especially for organizations seeking to build custom LLMs. Scalability and maintenance costs are critical in deciding whether to adopt off-the-shelf tools or invest in building proprietary models.

Strategies for Integrating AI into Data Governance Frameworks

To effectively leverage AI in data governance, organizations should develop a strategy that integrates AI tools into their existing governance frameworks. The presenter suggests:

AIEnabled Policy Management: Use AI to automate policy creation and ensure consistent application of data governance policies across the organization.

Regulatory Compliance Monitoring: AI tools can continuously monitor changing regulations and adapt organizational policies to meet new requirements.

Enhancing Data Quality with AI: AI can automate data quality management by detecting anomalies and enforcing data standards. This leads to more accurate and reliable data within the organization.

Automating Data Stewardship: AI can identify repetitive tasks, streamline them, and allocate resources more efficiently, ensuring that stewards focus on higher-level strategic activities.

RealWorld Case Studies

The webinar presents several examples of how AI is being used in practice:

Data Classification Automation: A financial services company uses AI to automatically classify and label data assets, speeding up the process and improving accuracy.

  

Regulatory Compliance: A healthcare organization uses AI tools to continuously monitor compliance with evolving international regulations, reducing the risk of non-compliance.

Data Quality Management: A health sciences organization applied AI to automate data quality checks, improving data reliability while freeing human resources for more strategic activities.

Concluding Remarks

Let's Talk About Data and AI Webinar: Global Framing Session from the Datasphere Initiative

Let's Talk About Data and AI Webinar: Global Framing Session

Key Concepts Summarized:

Responsible AI: AI development and governance should prioritize human rights and democracy and actively involve all stakeholders, ensuring inclusivity at every step of the process.

Data Governance: Proper governance is essential for AI systems to function ethically and inclusively, with a particular focus on data from diverse sources.

Global Index for Responsible AI: This tool plays a crucial role in measuring and promoting responsible AI practices globally. By focusing on human rights, sustainability, and gender equality, it instills optimism about the future of AI governance.

Challenges of Implementation: It's essential to be aware that moving beyond principles to practical application, especially in underresourced regions, is challenging. This underscores the need for collective effort in implementing responsible AI.

Inclusivity and Data Colonialism: Ensuring AI systems reflect diverse populations and do not perpetuate historical patterns of exploitation.

Introduction to Responsible AI

• The  AI framework ensures that AI technologies are developed, used, and governed in a manner that respects human rights and reinforces democratic values.
• The discussion highlights the impact of artificial intelligence (AI) on various aspects of our lives, both positively (by spurring innovation and enhancing healthcare access) and negatively (by enabling mass surveillance and eroding civil liberties).
• This dual nature underscores the central challenge of responsible AI.

Data Governance and AI

The panelists discuss the crucial role of data as the foundation of AI systems and how the quality, quantity, and governance of data have a direct impact on AI outcomes. They argue that data governance frameworks need to be specifically designed for AI, with a focus on:

• Inclusive democratic principles are being integrated into data practices.
• Ethical considerations regarding data sovereignty, particularly concerning marginalized or underrepresented communities.

Global Index for Responsible AI

The core concept discussed is the Global Index for Responsible AI, which seeks to:

• Provide benchmarks to measure how well different countries perform in AI governance.
• Ensure that AI use aligns with human rights, sustainability, and gender equality.
• Track progress over time with a focus on the global South.

The Index aims to provide measurable indicators to understand how various regions are advancing responsible AI practices. The categories include human rights, responsible AI governance, national capacities, and enabling environments. This global initiative considers individual and collective rights to assess a nation's ability to implement accountable AI practices.

Challenges in AI Implementation

Another key concept is the challenge of implementation. While there are many principles for AI ethics, such as the UNESCO AI principles and OECD guidelines, implementation still needs to be discovered.

The speakers argue that:

• There must be more connection between AI principles and practical implementation in many regions, particularly developing economies.
• Implementation is complex due to data access inequalities, lack of internet connectivity, and other infrastructural barriers.
• Furthermore, bias in AI models exacerbates existing societal inequalities, especially when training data fails to represent marginalized groups.

Inclusivity in AI and Data Governance

The speakers repeatedly emphasize the importance of diversity in data sets and warn of the dangers of unrepresentative data in AI systems. They stress how data colonialism—the extraction of data from marginalized communities—can perpetuate inequalities. They strongly advocate that AI systems need to account for diverse populations to avoid perpetuating structural inequalities, making the audience feel the necessity of inclusivity in AI systems.

Inclusive and Ethical AI for Academic Libraries

The webinar focuses on how academic libraries can ethically and inclusively adopt and integrate artificial intelligence (AI). It brings together experts to share insights on the potential and challenges of AI in library services, notably how AI can support diversity, equity, and inclusion (DEI) in higher education. The discussion also covers the broader implications of AI technologies in academic settings, including governance, accessibility, ethics, and employment impacts.

Defining Inclusive AI

Inclusive AI emphasizes developing AI systems designed to be fair, transparent, and representative of diverse groups. It is not enough for AI to be efficient; it must be created consciously to eliminate biases, especially those that reinforce historical inequities. AI systems should serve all users, including historically marginalized and underrepresented groups.

In academic libraries, inclusive AI would ensure that all students, faculty, and staff—regardless of race, gender, socioeconomic status, or ability—can access and benefit from AI-driven tools and resources. Libraries are increasingly integrating AI into their systems, and these tools must reflect the values of inclusivity.

The Role of Academic Libraries in Ethical AI

Academic libraries have a unique opportunity to lead the ethical use of AI in higher education. The presenters stressed that libraries must not just adopt AI for modernization but should focus on using AI to support ethical research and education. Libraries are historically seen as places of equitable access to information, and this mission should guide their approach to AI.

However, a key challenge lies in avoiding ethical paralysis—an overemphasis on potential harm that stifles innovation. The presenters encourage libraries to actively shape AI use by applying ethical frameworks while embracing AI’s potential to expand access and services. This means that while it's essential to be mindful of the potential ethical issues, it's equally important not to let these concerns hinder the adoption and innovation of AI in libraries.

The role of libraries extends beyond mere AI adoption. 

Libraries can champion ethical AI by Developing AI Governance Structures. Creating internal committees or teams to oversee AI development and implementation ensures that moral principles are embedded in library AI systems.

Educating the Community: Libraries should inform students and faculty about AI, not only using these tools but also their limitations and the biases they may reflect.

Ethical Auditing: Libraries can lead in auditing AI systems to check for bias, discrimination, and inequities that may arise in the data these systems use or the results they generate.

Libraries as Centers for AI Education and Skill Development

Libraries are ideal institutions for promoting AI literacy. They provide a safe and secure environment for students, faculty, and staff to learn AI tools. Presenters have pointed out that many individuals still lack confidence or skills in using AI technologies, and libraries can bridge this gap by offering training programs. This is particularly important in helping individuals understand how AI systems work and their applications in academic research. However, grasping AI's ethical implications is equally crucial, as this understanding empowers us to use AI responsibly. 

Libraries can play a crucial role in educating their communities about AI by using these tools and understanding their limitations and the biases they may reflect. AI Labs and Resources: By introducing specific AI tools such as Bard for natural language processing and ChatGPT for conversational AI, libraries provide controlled environments where students can learn to use these technologies safely and responsibly, instilling confidence in their abilities.

Upskilling Library Staff

Staff training in AI literacy is essential for libraries and other organizations to ensure employees can effectively work with AI technologies and support users in navigating AI-driven systems. Training should cover several key areas:

Understanding AI Functionality: Staff should learn how AI systems operate, including machine learning, natural language processing, and data analysis techniques. This knowledge allows them to interact with AI tools confidently, making troubleshooting issues or answering user questions easier.

Ethical Considerations: AI systems often involve ethical issues such as data privacy, bias, transparency, and the impact of AI on employment. Training should emphasize these concerns, recognizing the staff's role in responsibly guiding users through these issues. By understanding these ethical challenges, staff can ensure AI technologies are used to promote fairness and inclusivity, making them an integral part of the process.

AI as a Collaborative Tool: Rather than viewing AI as a threat to their jobs, staff should be taught how AI can complement their work, automate repetitive tasks, and allow them to focus on more complex, value-added services. For instance, AI can assist in tasks like resource curation, chatbots for customer service, or data management, while human staff can focus on user engagement and decision-making. This can lead to significant cost savings and efficiency improvements for the library.

Practical Applications: Staff training should also include practical applications of AI systems, such as using AI-driven cataloging systems or chatbots and assisting users in navigating AI-enabled services like personalized recommendations or automated research assistance. This practical knowledge will make staff feel more prepared and competent.

Addressing Bias in AI Systems

One of the major concerns discussed was the inherent bias in many AI systems. Large language models and other AI technologies often draw from existing data sources, which may reflect societal biases, particularly those rooted in colonial, Eurocentric, or otherwise exclusionary perspectives. As a result, AI systems can unintentionally perpetuate the same biases in their training data.

This is where libraries can play a significant role by Ensuring Diverse Data Sources. When training AI models, the data must come from diverse, inclusive sources representing various cultures, languages, and perspectives. This commitment to inclusivity in AI training should make the audience feel integral to creating a fair and representative AI system. Critical Use of AI Outputs: Users of AI tools in academic libraries should be encouraged to critically evaluate the results generated by AI, recognizing the possibility of biased outputs.

The presenters emphasized that libraries must educate their communities on how to interpret AI outputs and make decisions about the credibility and relevance of information, especially when using generative AI in research and learning.

AI and Accessibility

The integration of AI also brings new opportunities for improving library accessibility. AI tools such as text-to-speech, automatic transcription, and machine translation can significantly enhance access for students with disabilities or language barriers. This potential of AI to break down accessibility barriers should inspire optimism about the future of library services. However, the presenters cautioned that AI systems must be designed with accessibility in mind from the outset. Many current AI models still need to be improved in understanding diverse languages and dialects, which can be a significant limitation for inclusive access.

AI Governance and Policy in Libraries

Another key topic was the need for robust governance structures within academic libraries to manage AI technologies. The presenters suggested that libraries implement AI governance frameworks that address questions like: How do we ensure AI is aligned with our DEI goals?
How do we regularly audit AI tools for bias or inequity?
What processes are in place for user feedback on AI tools?

The Impact of AI on Library Jobs

There was also discussion about the fear that AI might replace library jobs. However, it's important to note that AI can automate specific tasks, such as cataloging, answering basic reference queries, or analyzing large datasets, freeing up library staff from repetitive tasks. This can allow them to focus on more complex, human-centered services such as personalized research assistance, instructional design, and DEI initiatives. While some routine tasks may be automated, the presenters argued that AI should be seen as an enhancement to human labor, not a replacement.

AI can free library staff from repetitive tasks, allowing them to focus on more complex, human-centered services such as personalized research assistance, instructional design, and DEI initiatives. To mitigate the fear of job displacement, the presenters suggested libraries provide ongoing training and reskilling opportunities so staff can effectively collaborate with AI tools.

Engaging Our Power Beyond Algorithmic Bias: Reframing and Resisting AI Empire

Introduction and Context

The presentation begins by addressing the influence of algorithmic bias in AI systems and introduces the term AI Empire. This framework is a critical lens through which the presenter examines how the ideologies of capitalism, colonialism, racism, and heteropatriarchy fuel AI's development and perpetuate social inequalities. 

These interlocking systems of oppression, deeply embedded in the technology sector, manifest in the automation of social control and essentialism (reducing individuals to predefined characteristics), serving capitalist ends such as profit and societal domination.

The AI Empire is an understatement as a technological advancement. It is better understood as a socio-technical system in which AI and society shape each other in culture. AI, in particular, plays a pivotal role in influencing societal structures and behaviors. It's time to move beyond the notion of AI as a neutral tool and acknowledge its active role in maintaining power structures.

AI in Libraries: Resistance and Implication

The presenter highlights that libraries and librarians, historically positioned as gatekeepers of knowledge, are often framed as reactionary to technological advancements. The dominant narrative suggests that librarians must constantly defend their relevance, especially in the face of innovations like the internet and, more recently, AI tools like ChatGPT.

This approach reflects a broader challenge. Libraries were initially designed to uphold the social and moral order rather than to encourage or enable radical change. This historical context creates friction in the current technological landscape, where innovation is rapid, and libraries struggle to keep up while maintaining ethical and inclusive practices.

A crucial concept here is classification—a fundamental task of libraries. 

While designed to organize knowledge, classification systems (such as cataloging and collection development) have often upheld systems of inequality. Despite efforts to become more inclusive, Libraries continue to engage in gatekeeping practices by defining who has access to certain types of knowledge or spaces. In this sense, libraries are not immune to the broader AI Empire, as they, too, are implicated in sustaining systems of control through their organizational structures.
AI Empire and Socio-Technical Systems

The presenter introduces AI Empire to challenge the common perception of AI as a tool that can be improved by correcting biases or refining algorithms. Instead, AI must be considered part of a socio-technical system, where society and technology are co-constitutive—that is, they shape each other. 

For libraries, this means recognizing that AI technologies cannot be fully understood or critiqued without addressing the broader systems (capitalism, patriarchy, colonialism) that influence their design and deployment. This understanding equips librarians with a more comprehensive approach to AI.

To better illustrate this point, the presenter discusses the history of libraries as institutions that have historically upheld oppressive systems. For example, libraries once played a role in segregation (e.g., segregated spaces or discriminatory access policies) and continue to perpetuate social inequality through policies that restrict access based on literacy or socioeconomic status.
Resistance Strategies: Beyond Algorithmic Literacy

Algorithmic literacy, which emphasizes understanding how AI works and identifying biases in AI systems, is an essential but insufficient step in addressing the root issues. The presenter argues that focusing solely on the technological artifact (the AI tool itself) overlooks the structural biases intentionally embedded within these systems to maintain inequality. In this regard, AI does not merely reflect society's biases—it is built to reinforce them.

To honestly resist the AI Empire, librarians, and academic institutions have the potential to move beyond the "bad apples" or "biased algorithms" narrative and engage with the foundational ideologies—capitalism, colonialism, patriarchy, and white supremacy—that underpin AI's development. This potential for change requires a broader critical consciousness about the systems in which AI operates and the power it wields over society.

Frameworks for Reframing AI: Critical Concepts

The presentation references two key frameworks that deepen our understanding of how AI and its socio-technical systems function: AI Empire: As already defined, this framework conceptualizes AI as a tool of social control deeply embedded in the same power structures that dominate society—capitalism, colonialism, racism, and patriarchy. Rather than seeing AI as a neutral or objective tool, this framework calls attention to how AI perpetuates existing systems of inequality.

The presenter delves into the ethical costs of AI development, including its impact on marginalized communities and environmental degradation. AI technology's huge language models are built on vast amounts of data extracted from existing systems of knowledge production, most of which have been shaped by Western, white, male perspectives. This creates a feedback loop where the elite's biases and assumptions are amplified and perpetuated through AI, with the effects disproportionately felt by the global majority and marginalized communities.

Moreover, the environmental cost of AI systems, such as the massive computational power required for machine learning models, must be addressed. The extraction of resources for server farms and the energy needed to run them are significant contributors to environmental destruction, further exacerbating global inequalities.

Practical Steps: Resisting AI Empire in Libraries

The presentation concludes with practical advice for librarians and information professionals on resisting the AI Empire: Encouraging critical reflection, the presenter urges librarians to question their practices and the systems they uphold. This involves critically examining the role of technology in reinforcing existing power structures and considering how they can resist these forces in their day-to-day work. Fostering Critical Consciousness: Librarians should strive to cultivate critical consciousness among their communities, helping users understand how to use AI tools and how they are implicated in broader power systems.

Collective Imagination: The presenter emphasizes the importance of collective imagination—envisioning and working towards new systems of knowledge that challenge the status quo. This could involve designing alternative classification systems, promoting open access to information, and actively working to decolonize library practices.

Strategizing and Organizing: The presenter calls for organizing within the library profession to build collective power. This involves solidarity with marginalized groups and actively resisting technologies and systems reinforcing inequality.