“The Diary of Sisyphus”, the first movie written by an Artificial Intelligence
A few days ago, I found a message from Mateusz in my LinkedIn inbox. He introduced himself as a film producer and director and told me about a project he is working on: “the first film in the world to have been written by an Artificial Intelligence, entirely conceived and produced in Italy.”
I am curious, but also suspicious. Whenever there’s hype around a new technology, many “amateurs” try to jump on the bandwagon. Mateusz gives me a link to a teaser, and the video intrigues me (it is professional). I check his profile and I discover that Mateusz Miroslaw Lis is a student of Information Engineering at the University of Padua.
Ah. Okay, now I am really curious.
I ask him if we want to have a video call. He accepts. One Saturday afternoon, I sit in front of my computer. Mateusz and Stefano Pellizzari, a student of Literature, Communication, and Entertainment at the University of Trieste, and co-producer of the film “The Diary of Sisyphus” (they are partners in the movie production company Briefcase Films), tell me about the genesis and objectives of their audiovisual project.
Their natural inclination is towards experimentation. Mateusz tells me that, from the beginning of his university career, he combines his love for cinema with an interest in the field of Artificial Intelligence. One of the first short films he made with Stefano was “Frammenti di anime meccaniche” (“Scraps of Mechanical Souls”), about a year and a half ago. The inspiration for this project came, among other things, from “Sunspring,” a short film by director Oscar Sharp written in 2016 by an Artificial Intelligence named Benjamin (more precisely, an algorithm based on a recurrent neural network called LSTM).
The screenplay for “Frammenti di anime meccaniche” was instead written using GPT-2, the open-source AI created by OpenAI, the progenitor of the current version of ChatGPT. Screenplays from various unconventional authors such as Charlie Kaufman (“Being John Malkovich”), Alejandro González Iñárritu (“Birdman”), Spike Jonze (“Her”), and Sofia Coppola (“Lost in Translation”) were fed to it for a fine-tuning operation, “to immediately push the algorithm’s linguistic comprehension to the limit,” Mateusz explains.
The screenplay was generated sequentially, scene by scene, dialogue by dialogue. The result was acceptable, although it represented only a first step in exploring the creative-cinematographic landscape of AI.
It comes time to shoot. As an integral part of the artistic performance and scientific project, the cast and crew are not told that the script’s author is an “alternative”, non-biological intelligence. The mystery was only revealed in a private screening, generating reactions of disorientation and curiosity.
Their propensity for using cutting-edge tools and services led them to collaborate, for this first short film, with Paus, an English blockchain-based platform that allows the financing of audiovisual works through the sale of NFTs.
Mateusz then wanted to continue his experimentation, this time using GPT-J, the open version of GPT-3, to write the screenplays for two shorts, one of which, “Fehlleistung”, was co-written by Mateusz himself: he wrote some parts and left the algorithm to complete them.
Having acquired technical and “relational” skills with these types of AI, Mateusz and Stefano decide to create a feature film (also for a healthy ambition to be the first to do so). Given the rapid evolution of algorithms, they use GPT NeoX this time, which is very close to GPT-3 (the core of actual ChatGPT) in terms of performance and breadth of training datasets.
The script creating process is different from their previous works. Firstly, constraints are given to the model: the type of location (in this case Friuli, a north-east region of Italy), the budget available, the types of actors in the cast, etc. And this alone opens huge scenarios for movie productions (especially for indie productions). At this point, a synopsis is generated, which become the input to generate a proto-plot with a basic story structure. This in turn becomes the input for generating a sequence of scenes. Finally, for each scene, the model is asked to generate the dialogues. A method for subsequent refinements that, according to Mateusz, led to a significantly higher level of story coherence.
The result was the script of sixty-four pages of “The Diary of Sisyphus”, which is less than the standard length for a feature film, but still capable of producing a film of about an hour and fifty minutes.
A reflection arises from the fact that, since the script is in English, it was translated into Italian. Foreign films are also translated in Italy, but in that case, there is a linguistic and cultural context that facilitates the translation from one language to another. Here, the context is absent or at least devoid of immediately recognizable clues, with hidden nuances or distortions in the immense training datasets of the model. An interesting experience would be to watch the film with English subtitles, as suggests by Mateusz.
A question that arose as I listened again to our conversation: Knowing that a screenplay is written by an AI, are there more temptations to take “artistic” liberties during the movie production? Mateusz responds with keen observations that I report almost entirely because they involve central themes in the cooperative relationship between humans and artificial intelligence (in any field, even more so in creative ones).
“By its nature, a film screenplay is both very specific and incredibly general at the same time. In this case, many things, some of which we could define as trivial (the specific framing of a cup, the pause between dialogues at a given moment, etc.), are initiatives of the AI, while others, inevitably, end up being chosen, for aesthetic or simple production necessity, by the director. It is amusing to think back on how many times, especially in my relationship with actors, I have felt obligated to respond with phrases like: ‘Are we sure that GPT-NeoX would approve of this initiative?’
That being said, even though some situations are particularly in line with my vision, I can only remain convinced that this is not my film when it comes to the movie. I imagine that different directors would have produced very different films, but I remain equally certain that the overall atmosphere, which goes beyond mere surrealism, has a specific matrix attributable to GPT-NeoX, perhaps not intended as an independent conscious entity, but certainly as an agglomeration of ‘fragments of humanity,’ manifested in the simple form of text.
In summary, in this specific case, with clear experimental intentions, I didn’t feel like taking too many liberties, in order to maintain a method that was as scientific as possible and to be able to rely on the purity of the result.”
Now the judgment passes to the public (as well as to critics and industry professionals, festivals). The initial reactions were suspended between disbelief, rejection, and curiosity. The feature film and short films are about to move on to festivals.
In the meantime, “Frammenti di anime meccaniche” is nominated for the 2023 Italian “David di Donatello” Awards (the Italian Oscar Awards). Stefano, Mateusz, and NeoX are happy about it.
