Protections for Vulnerable Populations and Beneficence in Research

Recently there was a controversy in the news about an AI Teddy Bear that, among other things, would tell small children where to find knives (see Sale of this AI toy suspended over dangerous messages to kids - YouTube). Though much was made in the mainstream press about exposing children to potentially harmful information (rightfully so), the full report from U.S. PIRG also describes concerns about what was happening with the sensitive data, such as children’s facial features and voices, being recorded by these devices (see the full report here). Years ago, when I was conducting cognitive labs (see here for a detailed description of cognitive labs) to develop questions for a survey that would be administered to children, we had to provide clear, detailed information to both an Institutional Review Board and to the parents that were providing consent, about what recordings would be used for, how the data would be stored, and how the data would be disposed of. The Children’s Online Privacy Protection Act (COPPA) requires digital creators of material for children to provide parents with similar information in order to obtain parental consent, though the U.S. PIRG report indicates it is unclear whether these companies are in compliance.

For research and data collection purposes, children are considered a vulnerable population. Vulnerable populations are those lacking the ability of self-determination due to circumstances such as age, illness, or a loss of liberty (see The Belmont Report). For example, incarcerated individuals would be considered a vulnerable group. As with other data collected by AI, though this might not be research in the traditional sense, data are being collected and will be analyzed, and thus there should be special protections for vulnerable populations.

In addition to protections for vulnerable populations, human subjects’ protections call for studies to ensure that the benefits of the research outweigh the risks, and that participants are adequately informed of the risks when they participate. This is the concept from the Belmont Report called beneficence. The Belmont Report describes beneficence as an “obligation” with two general rules: 1) do no harm and 2) maximize benefits and minimize harms (see The Belmont Report). In a practical sense, the benefits for participation may be substantial and direct. Ill people in clinical trials might be cured by the drug being tested, or insights from a study might be given back to the population being studied in order to improve conditions (as happens with much evaluation work). Research may also contribute to generalized knowledge of the world. Under the principle of informed consent (as discussed in an earlier article), individuals need to be provided with details of the potential risks and benefits of participating in a data collection activity in order to make an informed decision.

Returning to the earlier topic of AI toys, we should ask who is benefitting from AI toys collecting data on children. Are parents being adequately informed about the risks and benefits? For that matter, are any of us? If our data are being used to improve AI, who benefits from that? Do the benefits outweigh the risks? This is just one example, but the principles are universal for all data collections. As you put together your data collections, with or without AI, consider the following tips:

• Design with participant well-being in mind

Structure your study to minimize risks and maximize potential benefits for participants and society.

• Conduct a thorough risk–benefit analysis

Explicitly weigh possible harms (physical, psychological, social) against anticipated benefits before finalizing your design.

• Use the least risky methods possible

Choose procedures that achieve your research goals while reducing exposure to unnecessary risks.

• Ensure informed consent is truly informed

Clearly communicate risks, benefits, and alternatives so participants can make autonomous, well-considered decisions.

• Implement ongoing monitoring

Regularly assess participant safety during the study, with protocols for responding to adverse events.

• Prioritize protecting vulnerable populations

Take extra precautions when working with groups who may be at higher risk of harm or exploitation.

• Provide direct or indirect benefits

Consider ways participants may benefit—such as access to resources, knowledge, or improved services—even if the study’s primary benefit is societal.

• Maintain confidentiality and privacy

Protect sensitive data to prevent harm from breaches of trust or misuse of information.

• Seek independent ethical review

Submit your study to an Institutional Review Board (IRB) or ethics committee to ensure beneficence is upheld.

• Plan for dissemination of results

Share findings responsibly so that respondents are protected. For example, anonymize results in your report.