ChatGPT is underpinned by a large language model that requires massive amounts of data to function and improve. The more data the model is trained on, the better it gets at detecting patterns, anticipating what will come next and generating plausible text.
Uri Gal notes the following privacy concerns in The Conversation:
- None of us were asked whether OpenAI could use our data. This is a clear violation of privacy, especially when data are sensitive and can be used to identify us, our family members, or our location.
- Even when data are publicly available their use can breach what we call contextual integrity. This is a fundamental principle in legal discussions of privacy. It requires that individuals’ information is not revealed outside of the context in which it was originally produced.
- OpenAI offers no procedures for individuals to check whether the company stores their personal information, or to request it be deleted. This is a guaranteed right in accordance with the European General Data Protection Regulation (GDPR) – although it’s still under debate whether ChatGPT is compliant with GDPR requirements.
- This “right to be forgotten” is particularly important in cases where the information is inaccurate or misleading, which seems to be a regular occurrencewith ChatGPT.
- Moreover, the scraped data ChatGPT was trained on can be proprietary or copyrighted.
When we use AI tools, including detection tools, we are feeding data into these systems. It is important that we understand our obligations and risks.
When an assignment is submitted to Turnitin, the student’s work is saved as part of Turnitin’s database of more than 1 billion student papers. This raises privacy concerns that include:
- Students’ inability to remove their work from the database
- The indefinite length of time that papers are stored
- Access to the content of the papers, especially personal data or sensitive content, including potential security breaches of the server
AI detection tools, including Turnitin, should not be used without students’ knowledge and consent. While Turnitin is a college-approved tool, using it without students’ consent poses a copyright risk (Strawczynski, 2004). Other AI detection tools have not undergone privacy and risk assessments by our institution and present potential data privacy and copyright risks.
For more information, see our Guidelines for Using Turnitin.
