In the world of data; power and ethics walk a tightrope; one misstep can turn a tool for enlightenment into a weapon of exploitation. The future isn’t about collecting more data, it’s about collecting the right data, ethically.
Ajay Pundhir
Introduction
The recent backlash over Facebook’s exploitation of users’ data is a stark reminder of the importance of ethical practice in data science. When users’ data is shared or used without their explicit consent, we can’t be surprised at public outcry or a renewed focus on ethics in data handling.
But what is “data ethics,” and how do companies ensure that they don’t find themselves in hot water like Facebook?
“Data Ethics,” at its core, refers to the moral principles that guide the collection, processing, storage, and sharing of data. It’s about making conscientious decisions that respect privacy, confidentiality, consent, and transparency.
This article aims to provide a comprehensive overview of today’s moral landscape in data science, dissecting the current state of data ethics, ethical considerations, and strategies for implementing ethical guidelines. We will also explore the future implications of data ethics and offer practical recommendations to help corporations navigate their challenges.
Importance of Data Ethics
We generate an overwhelming amount of data each day. According to the International Data Corporation (IDC), the global data sphere is expected to grow to 175 zettabytes by 2025, up from 33 zettabytes in 2018.
This data, if harnessed properly, can bring about remarkable innovation and societal benefits. The question, then, is where does all of this data come from, and are we using it properly?
We live in a data-driven product and service era. Anyone who wants to stay connected with loved ones, learn or shop online, or even utilize modern tools to make their professional life easier generates data.
Along the way, users consent to agreements full of legalese they rarely read so they’re usually in the dark about what data they generate. Most people feel they have no choice about how their data is used and collected anyway, so they simply hope that companies and governments will use it ethically.
New businesses and services are constantly popping up to make something beneficial out of all of this data. However, many companies begin creation and monetization without considering data ethics first. Recent trends with generative AI have amplified the harmful effects of neglecting data ethics. For example, applications like ChatGPT rely heavily on large datasets for training, raising concerns about the source, quality, and ethical implications of the data used. Generative AI models have inadvertently perpetuated biases present in the training data, resulting in discriminatory or harmful outputs. This has been evidenced in numerous cases, including the controversy around OpenAI’s GPT-3 and its biased language generation (Bender et al., 2021).
The issue of consent has become critical when considering the data used to train these models, as many use public data scraped from the internet. The question of whether the individuals who generated this data have given their informed consent for their data to be used in such a manner is one we’re still grappling with. (Rieke et al., 2020).
In some cases, generative AI models that use public data are being used for potentially hostile applications, like creating deep fakes and synthetic media. This presents a significant threat to privacy and security. As noted by Chesney and Citron (2018), deepfakes can be used for malicious purposes, such as fraud, identity theft, and spreading disinformation, posing a severe challenge to data ethics and significant potential harm to individuals.
It’s no surprise then that a recent study by the Pew Research Center revealed that 79% of Americans are concerned about how their data is being used by companies and 64% are concerned about how the government is using their data. Also, 81% of the public thinks that they have no control over their data and how it is being used. This indicates a growing public awareness and concern about data privacy and security, reinforcing the need for robust data ethics.
High-Profile Examples
People aren’t necessarily wrong to be concerned about how their data is collected and used, given the near-constant string of bombshell news stories about personal data misuse like the recent high-profile examples below. Both corporations and users are learning about potential pitfalls and baked-in biases as we go, but that begs the question, is it possible to predict these issues ahead of time by establishing rigorous data ethics?
Gender Bias of AI Algorithms
In 2019, Apple and Goldman Sachs were accused of gender bias in the way their credit card, the Apple Card, determined credit limits. Some users reported that the AI algorithm was giving women lower credit limits than men, despite the women having similar or even better financial profiles. This highlighted an ethical issue related to algorithmic bias and fairness in AI systems, as the AI algorithm seemed to discriminate based on gender despite the absence of explicit gender information in the training data.
The AI system might have inferred gender indirectly from other variables or patterns in the data, underscoring the importance of transparency and ethical oversight in AI-driven decision-making processes. Data scientists, regulators, and companies need to ensure that their AI algorithms are not only accurate but also fair and unbiased. This case was an important reminder that businesses must consider building fairness, justice, and respect for human rights in their AI algorithms.
Predictive Policing and Racial Bias
Predictive policing, an application of data science in law enforcement, has the potential to have extremely positive impacts on safety by using historical data to predict potential crime hotspots. However, it has become a contentious issue due to its tendency to exacerbate racial biases. In 2016, ProPublica, an independent investigative journalism organization, analyzed the COMPAS recidivism risk score algorithm used in several US states. The algorithm uses historical data to predict which criminals are likely to re-offend. ProPublica’s analysis revealed that the algorithm was biased against African Americans, wrongly labeling them as future criminals at nearly twice the rate of white defendants.
Because the algorithm relied on historical data, it captured the systemic biases already present in the criminal justice system. The algorithm, in essence, was ‘learning’ and perpetuating these biases, leading to unfair outcomes.
This case highlighted the importance of rigorous testing for algorithm bias, transparency in how predictive models are developed, and ongoing monitoring to ensure they do not perpetuate systemic injustices. It also shows that without proper ethical oversight, data science can inadvertently reinforce societal inequalities rather than help to eliminate them.
Google DeepMind Controversy
In 2016, Google’s AI subsidiary, DeepMind, incited a major controversy around data privacy and data ethics. DeepMind obtained medical data from 1.6 million patients from a London hospital trust to develop its healthcare app, Streams. The app was designed to assist in identifying patients at risk of acute kidney injury. However, DeepMind and the Trust failed to inform patients about this usage explicitly, triggering intense criticism.
The incident exemplifies the ethical and privacy concerns associated with the use of personal data in AI development. The key issue here was the lack of transparency and consent. Even though the data was intended to be used for the benefit of patients, the lack of explicit patient consent violated data norms.
It also highlights the necessity for clear and informed consent, transparency about data usage, and stringent data protection measures in AI applications, particularly in healthcare settings. It serves as a stark reminder for organizations to maintain strict data ethics. No matter how honorable their intentions are, they must ensure they still respect privacy rights while in pursuit of technological advancement.
Ethical Considerations in Data Science
Before adopting a strategy to solve the ethical challenges corporations continue to face with AI, it’s important to understand the different ethical principles at play.
Privacy and Confidentiality
Privacy and confidentiality are the primary ethical considerations in data science. Data scientists and businesses should be asking what data they’re collecting and how it is being stored, used, and protected.
The General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) provide frameworks for protecting personal information. Implementing privacy-by-design principles, conducting privacy impact assessments, and engaging data protection officers are effective measures to ensure privacy and confidentiality.
Consent
Consent requires getting permission from individuals before collecting, processing, or sharing their data. Data scientists and companies need to be asking whether they have obtained explicit consent, whether the data subjects understand how their data will be used, and whether they have the right to withdraw their consent.
The GDPR provides a framework for obtaining and managing consent.
Transparency and Interpretability
Transparency and interpretability involve explaining how algorithms make decisions. Data scientists and corporations should ask how decisions are made, whether the decision-making process is transparent, and whether the results are interpretable.
Explainable AI (XAI) provides ways to implement transparency and interpretability of AI algorithms. Implementing model interpretability tools and providing clear explanations and visualizations can enhance transparency and interpretability.
Bias and Fairness
Bias and fairness require companies to ensure their algorithms do not discriminate or create unfair outcomes. Data scientists and companies should ask whether their data is representative, whether their algorithms are unbiased, and whether the outcomes are fair.
The Fairness, Accountability, and Transparency in Machine Learning (FATML) provides a framework for addressing bias and fairness. Implementing bias detection and fairness improvement tools and conducting regular audits is an essential step toward combating bias and ensuring fairness.
Accountability and Responsibility
Accountability and responsibility involve holding entities accountable for the outcomes of their algorithms. Data scientists and businesses should ask who is responsible for the outcomes, whether there are mechanisms in place to hold them accountable, and whether they have the capacity to remedy any harm caused.
The Algorithmic Accountability Act in the US provides a framework for algorithmic accountability. Implementing robust governance structures and setting up redressal mechanisms can ensure accountability and responsibility.
Future Implications and Recommendations
As we move towards Artificial General Intelligence (AGI), the complexity of AI systems and the volume of data they process is increasing exponentially. With AGI, the ability of AI to understand and replicate human-like cognition would further increase the chances of potential misuse of personal and sensitive data, leading to serious ethical and privacy issues. Both Governments and Corporates need to join hands to overcome this challenge.
In fact, this dilemma is already recognized by governments, businesses, and research institutions. For example, the Federal Government has developed a Data Ethics Framework. This framework, developed by the General Services Administration (GSA), is designed to guide federal employees in making ethical data management and usage decisions. It is updated every 24 months, making it a dynamic guide that can adapt to the rapidly evolving data landscape.
The corporate world, too, is placing more emphasis on ethical data management. Corporate authorities are urging companies to ensure that their data practices preserve security, protect customer information, offer clear benefits to consumers, and align with the company’s promises. At the most basic level, companies want to avoid the public scrutiny that tech giants like Google and Facebook have faced for data privacy issues. Rightfully so, these organizations now have a greater focus on ethical data management and have implemented measures such as enhanced data security, transparency in data handling, and giving users more control over their data.
Strategies to Navigate These Potential Challenges
Though it seems that everyone is talking about the importance of rigorous protections for users’ data, many organizations don’t have ethical frameworks in place, either because they don’t know where to start or because they assume that another department has taken charge of it. However, assuming that someone else has ethics covered puts the company at huge risk.
To ensure that a corporation’s collection and handling of users’ data is ethical, we suggest starting with the following steps:
Adherence to Global Standards
The adoption of standards like UNESCO’s Recommendation on the Ethics of Artificial Intelligence or General Data Protection Regulation (GDPR) can provide a framework for ethical AI development. These frameworks provide guidelines on transparency, justice and fairness, non-discrimination and non-bias, privacy and data protection, and accountability.
Enhanced Transparency and Interpretability
AI developers need to focus on creating more transparent and interpretable models where the decision-making process can be understood and explained to all stakeholders. Doing so would certainly minimize the chances of biased or discriminatory AI applications.
Accountability and Responsibility
Organizations need to ensure that they are accountable for the decisions made by their AI systems. This involves establishing responsibility mechanisms, such as in-house and third-party audits or ethics committees. Organizations should create a framework to identify the risks of collecting, processing, and retaining data. These frameworks can vary as per the industry’s need but at the bare minimum can make organizations accountable for ethical practices.
Organization-wide awareness
There can not be a better remedy than preparing people at all levels in the organization to understand the importance of Data Ethics and be responsible. Anyone in the organization who toches the data, a developer, analyst, operations, or marketing should follow the protocol. Leadership needs to create a culture by educating employees and empowering them to raise questions to the relevant committee.
Conclusion
While we are making strides towards AGI, it’s crucial to acknowledge and address the ethical implications of AI and data science. Continued discourse and collaboration between AI developers, ethicists, and policymakers are needed to establish robust frameworks and regulations. This will not only ensure AI’s responsible use but also help build public trust and acceptance.
