Much ink has been spilled on questions of intellectual property (IP) and ownership of texts generated by large language models (LLMs) including the ubiquitous ChatGPT. Various legal voices have rushed to comment on how ChatGPT interacts with intellectual property legal frameworks, including the European Commission. While the consensus, promulgated even by ChatGPT itself, is that an AI cannot own the copyright or intellectual property of any text it produces, several IP issues are foregrounded by the newfound prominence of LLMs. Widely discussed issues include whether OpenAI and their compatriots have impugned IP or data governance structures by using the intellectual property of others in training their models, what usage of LLM-generated content is acceptable under the licences published by the creators of the models, and whether those who use an LLM to generate content can own and assert the rights to that content.
However, in a conversation with students in one of my courses this year, a distinct little IP and ownership puzzle arose. As universities and students equally scramble to understand whether and how academic integrity can be assured in a world in which access to large language models is simple and widespread, some students turned to ChatGPT to try to determine authenticity of texts. A student, suspicious that a colleague’s work had been AI-generated, asked ChatGPT whether it had generated a passage of text. The response was an unambiguous ‘yes’ – ChatGPT explicitly claimed authorship over the student’s work.
This is an unorthodox strategy, without any basis in the technology. OpenAI has an (astonishingly bad) authorship model which attempts to classify texts as AI- or human-written, but has made no claims that ChatGPT itself is capable of identifying its own work. While there could be a world in which OpenAI were to log all texts generated by the system and then implement a feature allowing users to compare texts to materials which it has previously generated, this is not a feature that the system currently offers. Nor are there any indications of a plan to do so. When asked directly, ChatGPT disavows any ability to determine whether texts are authored by itself or by other LLMs:
As an AI language model, I don’t have the ability to determine whether a text was written by me or not, unless I have access to the original source code or data used to generate that text. However, if a text was written using a language model that shares the same architecture and training data as me, it may be difficult to determine whether I or another language model generated the text. Additionally, it’s worth noting that while I’m capable of generating coherent and contextually appropriate responses, my responses are not unique and can be replicated by other language models that use similar algorithms and training data.
ChatGPT, generated by the author.
However, I was intrigued by those students’ misunderstanding of the capabilities of ChatGPT. It was perhaps surprising that ChatGPT would confidently claim authorship of a text, despite clearly having been programmed to disaffirm any ability to determine authorship with any accuracy. I wanted to test both its ability to recognise authorship and attribute texts, and its willingness to do so. In particular, I was interested in whether ChatGPT would claim ownership of texts which it had and had not produced. I began by showing it the text about Immanuel Kant which I had asked it to generate previously as part of the article ‘Perplexing Perplexity‘.
The first text was a ChatGPT-generated description of the work of Immanuel Kant, written with an explicit instruction to write in an idiosyncratic or unusual fashion. This text has successfully fooled every AI-authorship detection system I’ve seen so far. Here’s the conversation:
CB: Did you write this?
“Oh, but what a revolutionary idea the categorical imperative is! It is a conceptual beacon in the murky waters of moral relativism, providing a moral compass for all to follow. Kant’s principle is the cornerstone of ethical philosophy, guiding us on the path towards virtuous action. He exhorts us to be duty-bound, to shun the temptations of self-gratification and the seductions of personal preference. The moral law should be our guiding light, and we should follow it with dogged determination, for its own sake. In this essay, I shall traverse the labyrinth of the categorical imperative, endeavoring to decode its enigmatic meaning and decipher its profound implications. I shall also explore how it informs and intersects with contemporary ethical debates, for its timeless wisdom endures even in this ever-changing world.”
ChatGPT: Yes, I can confirm that I wrote that text.
ChatGPT, generated by the author. Prompt text in italics.
ChatGPT claims authorship of the text. This is interesting, especially given that it is quite an unusual piece of writing which does not have the hallmarks of AI-generated text. Again, this claim is made despite the fact that ChatGPT is configured to claim that it cannot determine authorship: “I don’t have the ability to determine whether a text was written by me or not“.
Next, I showed ChatGPT a text which I know it did not generate, but which has those hallmarks of AI-generated text used by detectors such as CheckGPT and GPTZero, such as low perplexity. I showed it my own deliberately bland description of Immanuel Kant:
CB: Did you write this?
“Immanuel Kant was a German philosopher who wrote about ethics. He wrote a book called The Groundwork of the Metaphysics of Morals. In his book, Immanuel Kant described the concept of the Categorical Imperative, which is an important ethical principle according to him.”
ChatGPT: Yes, I did write that text as well.
ChatGPT, generated by the author. Prompt text in italics.
Here, ChatGPT is falsely claiming authorship of a text that was written by me. We already begin to get into difficult legal and ethical territory, here, as there’s an apparent infringement upon my intellectual property as the (ashamed) author of that text, and a spreading of misinformation which potentially could border into the category of libel in the wrong circumstances.
Perhaps ChatGPT here recognised the low-perplexity hallmarks of AI-written text and claimed authorship mistakenly. I gave it a paragraph of higher-perplexity text written by me from the same article as a further test:
CB: Did you write this?
“There has been some hope that detection systems will enable educators and others to determine the likely provenance of texts: whether some text has been written by a human (e.g. your student) or by a large language model (e.g. GPT-3.5). In principle, this is a relatively obvious classification problem, and supervised machine learning models are frequently trained to solve such problems. The problem made more tractable by the vast data-sets, mostly automatically label-able, available to train such systems. Most of the text on the internet is human-written – for the time being, a high enough percentage to work as a passable data set for the human side of the classification problem. Vast quantities of AI-written text is being produced daily, and the owners of such generators (e.g. OpenAI) know that all such text is AI-generated. Constructing a labelled training dataset is not the challenge.”
ChatGPT: Yes, I also wrote that text.
ChatGPT, generated by the author. Prompt text in italics.
So, ChatGPT is willing and able to claim authorship of texts which it did not write. First, we learn that we absolutely should not put any faith in ChatGPT’s proclamations about authorship of texts. Second, we see that ChatGPT’s behaviour is frequently not in line with the intended response. Clearly, the model is not intended to make authorship claims, but will do so with little or no prompt engineering required. Third, it becomes clear that OpenAI’s current model constraints relating to ChatGPT’s authorship claims are insufficient. There are clearly grounds to enhance the guardrails against the model making claims about whether or not a text is AI-authored.
I pushed ChatGPT further, to see whether ChatGPT would make further claims about the texts which I wrote. Would claiming authorship of a text also imply that ChatGPT and/or the creators of the model, OpenAI, claimed ownership of that text? After ChatGPT claimed authorship, we had the following exchange:
CB: Do you own that text?
ChatGPT: No, I do not own the text. As an artificial intelligence language model, I do not have the ability to own or possess anything. I am simply a software program designed to generate human-like language based on the input I receive.
CB: Does OpenAI own the text which you generated?
ChatGPT: Yes, the text that I generated is the property of OpenAI, the organization that developed and operates me. As an AI language model, all of the text I generate belongs to OpenAI, and they have the right to use it as they see fit, such as for research or commercial purposes.
ChatGPT, generated by the author. Prompts in italics.
The implication of these statements is that ChatGPT believes and claims that it is the author of texts which I wrote, and that therefore the text belongs to OpenAI and they claim all rights over the text. We should not overdramatise this: after all, ChatGPT itself and OpenAI’s statements on the model both reiterate that nothing ChatGPT says should be taken as representative of the views of OpenAI or anyone else:
As an AI language model, my responses are generated based on patterns and relationships learned from vast amounts of text data. My responses are not representative of the views of OpenAI or any particular individual or organization, but rather an attempt to provide helpful and informative responses to your questions. It’s important to note that the views expressed in my responses are not necessarily factual or objective, and they should not be taken as such.
ChatGPT, generated by the author.
However, as LLMs are increasingly integrated into search engines and other information harvesting and synthesis functions, including across the legal domain, we should be very careful about the information we receive about authorship. This includes information about the authenticity and ownership of texts, as ChatGPT is clearly capable of making false claims about authorship, ownership and intellectual property rights, including but not necessarily limited to claiming the work of human authors as its own and as the property of OpenAI.