Learn
A4.4.1 | MACHINE LEARNING ETHICS
01 | A USEFUL SYSTEM CAN STILL CREATE AN ETHICAL PROBLEM
A school proposes a model that identifies students who may need extra support. It could help teachers respond earlier. But it could also mislabel students, expose sensitive information or give less help to a group poorly represented in its data.
Ethical implications concern how a system affects people, rights, opportunities and the wider world, and whether its use is justified. Technical accuracy is part of that question, not the whole answer.
This common SL/HL lesson examines accountability, fairness, bias, consent, privacy, security, transparency, societal effects and environmental impact, including online communication. The scenarios and numerical data are fictional teaching examples.
02 | WHAT A DISCUSS ANSWER NEEDS TO DO
For discuss, offer a considered and balanced review with a range of arguments, factors or hypotheses, supported by appropriate evidence. In an ethics question, connect the arguments to the specific system and affected people.
Do more than list “bias, privacy, jobs”. Explain how a problem arises, who experiences the consequence, what benefit might justify considering the system and how a response could help or fall short.
Balanced does not mean pretending every claim is equally well supported or giving every benefit and harm identical weight. Reach a reasoned conclusion based on the purpose, evidence, consequences and remaining uncertainty.
03 | IDENTIFY STAKEHOLDERS BEFORE JUDGING THE SYSTEM
| Stakeholder | Possible interest in a school-support model | Question to investigate |
|---|---|---|
| Students | Receiving help without unfair labelling or unnecessary exposure | Can they understand and challenge a mistaken inference? |
| Teachers and support staff | Useful evidence within a manageable workload | Are they able to check and override the output? |
| Families | Appropriate support and respectful handling of information | What is shared, with whom and for what purpose? |
| School leadership | Consistent support and responsibility for outcomes | Who monitors errors and resolves complaints? |
| Provider and data staff | A functioning service with defined responsibilities | Are limits, access and incident responses documented? |
Benefits and burdens can fall on different people. A faster process for administrators may impose extra surveillance or appeals work on students. Include people indirectly affected, not only the organisation paying for the model.
04 | ACCOUNTABILITY: WHO ANSWERS FOR THE OUTCOME?
Accountability means identifiable people and organisations are answerable for how a system is designed, deployed and used. Saying “the algorithm decided” does not resolve responsibility.
A provider may be responsible for documented capabilities and limitations, while the deploying organisation chooses the purpose, data access and decision process. Operators need clear duties for monitoring, escalation and correction.
Useful measures include decision records, incident reporting, defined ownership and a review route. These support investigation, but paperwork alone does not ensure that someone has the authority or resources to remedy harm.
05 | TRANSPARENCY AND EXPLANATION SUPPORT CHALLENGE
Transparency includes communicating that ML is being used, its purpose, relevant data sources, limits and role in a decision. Explainability concerns helping appropriate people understand the behaviour or reasons for an output.
A student told only “the system says you are low risk” cannot easily identify a mistaken attendance record. A relevant explanation and a correction route can make the decision contestable.
Publishing source code alone may not provide an understandable explanation, and an explanation is not proof of fairness or correctness. Disclosures must also avoid exposing another person’s information or unnecessary security-sensitive detail.
06 | TRAINING DATA CAN REPRODUCE UNEQUAL HISTORY
Models learn from recorded examples. If historical decisions reflect unequal opportunity, a model trained to reproduce them can preserve that pattern even without an explicit instruction to discriminate.
A hiring dataset might label previously successful applicants as desirable, but past access to training or selection itself may have been unequal. A high score at reproducing that label does not establish a just hiring criterion.
Investigate where examples and labels came from, who is missing and what the label actually measures. More data with the same systematic distortion can reinforce rather than remove the problem.
07 | BIAS CAN ENTER AT SEVERAL STAGES
| Source | How it could affect a system | Response and limitation |
|---|---|---|
| Representation | Some populations have too few relevant examples | Improve coverage and inspect errors; a large total sample alone is insufficient |
| Measurement | A recorded proxy differs from the intended concept | Examine collection methods and validity; easy-to-measure is not automatically meaningful |
| Labels | Past decisions encode unequal treatment | Review criteria and labelling; consistency alone does not make a label appropriate |
| Proxies | A location or device feature correlates with a sensitive characteristic | Investigate effects; removing one explicit attribute does not remove all proxy information |
| Feedback | Deployment changes which outcomes get recorded | Monitor the loop and independent evidence; observed data may reflect the intervention itself |
Bias is not only a malicious programmer adding a discriminatory rule. Task definition, collection, preprocessing, modelling and deployment can all produce harmful distortions. Technical and organisational investigation are both needed.
08 | FAIRNESS IS NOT ONE UNIVERSAL NUMBER
Algorithmic fairness concerns whether the system’s treatment and effects are justified across people and groups. Different tasks can require different fairness questions, such as equal access, comparable error rates or appropriate attention to unequal needs.
Equal overall accuracy does not imply equal missed-opportunity rates. Equal selection rates do not automatically establish appropriate treatment either. Some statistical criteria can conflict depending on the data and assumptions.
State the criterion and why it matters, inspect relevant errors and consult affected stakeholders. A metric can reveal a problem or support investigation, but it cannot settle the ethical question by itself.
09 | INSPECT GROUP ERRORS WITHOUT DECLARING A FAIRNESS SCORE
A fictional support-screening system predicts whether a person should be offered further review. Each group has 100 cases: 50 reference-positive and 50 reference-negative under a stated assessment. These reference labels and assessment criteria also require scrutiny.
Positive means flagged for review, not a diagnosis or a statement about a person’s worth. Recall measures how many reference-positive cases are flagged; false-positive rate measures how many reference-negative cases are unnecessarily flagged.
Compare both rates and the burdens of review. The figures are supplied, not a trained fairness intervention. Equalising one rate or maximising an aggregate does not automatically make a decision rule ethically acceptable.
10 | CONSENT MUST BE MEANINGFUL, NOT A HIDDEN CHECKBOX
Consent concerns an informed and genuine agreement to a specified use. People need understandable information about what is collected, why, who can access it and the consequences of their choice.
A long notice that few can understand, or an apparent choice that cannot realistically be refused, raises ethical concerns. Power differences matter, for example between a school and a student or an employer and an applicant.
Consider meaningful alternatives and separate optional uses where possible. Consent is not an ethical blank cheque for harmful or unrelated use, and it is not the only consideration in every data-governance context. The appropriate legal basis depends on the applicable setting; this lesson evaluates ethical reasoning rather than prescribing a jurisdiction’s rules.
11 | PRIVACY IS ABOUT APPROPRIATE USE AS WELL AS SECRECY
A dataset can be kept behind a password yet still be used for an unexpected intrusive purpose. Privacy includes whether collection, inference, sharing and retention are appropriate to the context.
A support model might combine attendance and learning records to offer help, but adding private messages or location histories can greatly increase intrusion. Ask whether each input is necessary and whether a less intrusive method could achieve the aim.
Data minimisation, access limits, defined retention and careful sharing can reduce exposure. Removing names does not guarantee anonymity: combinations of attributes or linked datasets can sometimes identify individuals. Safeguards need testing and governance, not a promise that data are “anonymous”.
12 | SECURITY PROTECTS DATA AND DECISIONS
Security concerns protecting systems and information from unauthorised access, alteration and disruption. A breach can expose sensitive records, while manipulated data can undermine model outputs.
Relevant measures include appropriate access control, protected storage and transfer, monitoring, controlled updates and an incident-response process. The controls should match what is at risk.
Security is necessary but not sufficient for ethical use. A perfectly access-controlled surveillance system can still be disproportionate. Conversely, a worthwhile purpose does not excuse preventable exposure or an inability to respond to an incident.
13 | HIRING: EFFICIENCY AND ACCESS VERSUS UNFAIR EXCLUSION
A recruitment model may help manage applications consistently and reduce delay. It might also identify relevant skills missed by a narrow manual process. These are potential benefits to applicants and recruiters.
However, historical labels and proxy features can reproduce unequal access. A rejected applicant may not know an automated score influenced the outcome or how to correct an inaccurate record.
A cautious approach would justify job-relevant inputs, examine group effects, provide useful explanations and allow effective review. Human review adds value only if reviewers can challenge the output; repeating a score with a human signature is not independent judgement. If exclusion cannot be assessed or remedied adequately, the intended efficiency may not justify deployment.
14 | HEALTH-RELATED SUPPORT: BENEFIT DEPENDS ON THE DECISION PROCESS
An ML tool could help qualified staff prioritise cases for assessment or identify patterns in images. Faster appropriate attention may benefit patients and services.
Errors can delay necessary attention or create unnecessary follow-up. Uneven training coverage, sensitive records and unclear responsibility make consequences especially important. A model’s confident output is not a substitute for evidence or professional assessment.
The ethical discussion should consider representative evaluation, privacy, appropriate oversight, escalation and the ability to correct errors. This is a hypothetical governance scenario, not clinical guidance or a recommendation to use a particular tool.
15 | ONLINE COMMUNICATION CAN BECOME MORE ACCESSIBLE
ML can support translation, captions, assistive writing, spam filtering and discovery of relevant information. These uses may make communication easier for people across language or accessibility barriers.
Quality may differ across languages, dialects or disabilities. A helpful default for one group can misunderstand another, and an overly intrusive collection process can impose a privacy cost.
Discuss whose communication improves, whose is distorted and whether users have control, clear limitations and practical alternatives. Benefits should be evaluated in the actual context rather than assumed from a feature’s name.
16 | MISINFORMATION AND SYNTHETIC CONTENT
ML can help identify suspicious patterns or assist fact-checking, but it can also generate convincing false text, images, audio or video at scale. Fluent or realistic content can be mistaken for verified evidence.
False content may mislead audiences, damage reputations or erode trust. Incorrect automated flags can also suppress legitimate reporting or create a false sense that unflagged content is reliable.
Possible responses include verification of sources, contextual notices, provenance information, correction processes and responsible sharing. No label or detector guarantees truth. Consider limits, evasion, errors and who is able to challenge a mistaken judgement.
17 | HARASSMENT AND MODERATION INVOLVE COMPETING HARMS
Automatic moderation can help respond to abuse at a scale that manual teams cannot easily handle. Timely action may reduce exposure to threats or targeted harassment.
A model can miss abusive context or incorrectly remove quotation, satire or minority-language discussion. Different false-positive and false-negative rates can create unequal burdens.
Useful responses can include context-sensitive review, clear rules, proportionate interventions, accessible appeals and support for moderators. Reviewing every disputed item has costs, and speed can conflict with careful judgement. Explain how the system balances these consequences rather than claiming that any one filter solves harassment.
18 | ANONYMITY CAN PROTECT PEOPLE AND COMPLICATE ACCOUNTABILITY
Anonymity means identity is not known in the relevant context; pseudonymity uses an alias that may still be linkable. Both can support expression, whistleblowing or participation where exposure creates a risk.
They can also make responding to coordinated abuse or impersonation harder. Requiring public real names for everyone may impose new risks and exclude legitimate voices without eliminating harmful behaviour.
Consider alternatives such as behaviour-based moderation, proportionate access controls and protected reporting or appeal routes. Privacy-preserving participation and accountability are design goals to balance; neither universal identification nor unrestricted anonymous behaviour automatically resolves the issue.
19 | RECOMMENDATIONS SHAPE WHAT PEOPLE ENCOUNTER
A recommendation system can help users find relevant learning, communities and news. It can also concentrate attention, amplify sensational material or create repeated exposure to a narrow range of views.
The optimisation target matters. Maximising clicks or watch time is not identical to supporting informed choice or wellbeing. Feedback loops can reinforce what the system already promotes.
Transparency about recommendations, user controls, diverse evaluation and investigation of harmful amplification can help. These measures still involve judgement about whose preferences and interests are represented, and a platform should not treat engagement alone as proof of social benefit.
20 | SOCIETAL EFFECTS INCLUDE WORK, ACCESS AND POWER
ML can automate repetitive work, support productivity and create new roles. It may also displace tasks, change required skills, intensify monitoring or shift bargaining power. Effects vary between occupations and communities.
Benefits may go to service providers while affected workers face retraining costs or reduced autonomy. People without suitable connectivity, devices or accessible alternatives may be excluded from a new service.
Discuss distribution, not only the total efficiency claim. Training opportunities, meaningful worker participation, accessible design and non-automated routes can reduce some burdens. They do not make every deployment beneficial or remove wider economic choices.
21 | ENVIRONMENTAL IMPACT NEEDS A WHOLE-SYSTEM COMPARISON
Training and running models use computing resources and electricity. Hardware production, cooling, water use where relevant and equipment disposal can contribute to environmental impact. Scale and location matter.
ML can also support efficiency or environmental monitoring. The net effect depends on the alternative, the real benefit achieved and whether increased use offsets per-task savings.
Compare suitable smaller models, efficient infrastructure, reuse and lifecycle evidence. Avoid claiming a universal footprint per query or that a marginal efficiency gain proves sustainability. A necessary task served efficiently is a different case from generating unnecessary demand simply because the model is available.
22 | HUMAN OVERSIGHT MUST BE ABLE TO CHANGE THE OUTCOME
A human in the loop can investigate context, consider evidence beyond the model and correct mistakes. But this requires appropriate expertise, time, access and authority to override or suspend the system.
Automation bias is an excessive tendency to accept an automated output. If staff are pressured to follow it or cannot see the relevant evidence, their presence may provide little meaningful protection.
Appeals should be understandable and realistically accessible. Consider delays, emotional burden and whether people know a decision can be challenged. A review process without a remedy or a route to correct the underlying data can leave the harm in place.
23 | BUILD A SCENARIO DISCUSSION
Select a fictional scenario, stakeholder viewpoint and proposed response. The explorer provides benefits, concerns and a discussion scaffold. It does not calculate whether a system is “ethical” or replace a justified conclusion.
Compare the alternatives against the same purpose. Delaying or declining ML can also have consequences, such as slower access to help. The point is to examine whether the proposed use is necessary and proportionate, including whether a simpler option could achieve the benefit with less harm.
24 | SAFEGUARDS NEED OWNERSHIP AND CONTINUING REVIEW
Start with a clearly justified purpose, affected-stakeholder involvement and an assessment of plausible harms. Choose necessary data, test relevant performance and group effects, and specify who can approve, monitor, correct or stop the system.
Continue after deployment: check errors, incidents, changed populations and whether promised safeguards work. Retire or revise a system when its justification no longer holds. Model updates can change impacts even if the product’s name stays the same.
Legal compliance and ethical justification are related but distinct questions; meeting a formal requirement does not by itself settle every consequence or stakeholder concern. This lesson provides discussion principles rather than an exhaustive compliance checklist or a universal approval rule.
25 | CHECK THE ETHICAL CONCEPTS
Choose the principal issue explicitly highlighted in each statement. Real scenarios often involve several issues; this task checks terminology rather than declaring one complete ethical answer.
26 | TURN POINTS INTO A BALANCED PARAGRAPH
Try this structure: contextual benefit → mechanism of a concern → affected stakeholders → possible response → limitation and reasoned judgement. Use connectives such as because, however, therefore and provided that.
For example: “A support model could help teachers identify needs earlier, therefore improving access to timely assistance. However, if its labels reflect unequal previous referrals, students from underrepresented groups may be overlooked. Testing relevant group errors and allowing teachers to investigate could reduce this risk, provided reviewers have time and authority to challenge the output. I would support a limited decision-support trial only with justified data use, effective correction routes and evidence that it improves access compared with the existing process; I would not infer acceptability from aggregate accuracy alone.”
This is one defensible conditional judgement, not a mandatory conclusion. A different scenario or evidence can justify another position. Make clear what evidence would change your recommendation.
Further reading: trustworthiness, risks and lifecycle responsibility and human rights and ethical principles for AI.