The ethical dimensions of artificial intelligence (AI) image colourization were recently brought to public attention when several historical images were altered using digital algorithms.

Irish artist Matt Loughrey digitally colourized and added smiles to photos of tortured prisoners from Security Prison 21 in Phnom Penh, Cambodia, which was used by the Khmer Rouge from 1975-79. His photos were published in Vice and prompted outrage on Twitter.

Matt Loughrey in Vice is not colourising S21 photographs. He is falsifying history: pic.twitter.com/z6J99J7BOE

— John Vink (@vinkjohn) April 10, 2021

Vice removed the altered photos from their website and apologized to the families of the victims and the communities in Cambodia. Meanwhile, the Toronto Star’s Heather Mallick described them as “thoughtless, ahistorical and self-congratulatory” and proclaimed that we must stop trusting photography.

AI colourization refers to the use of digital algorithms to substitute colours into a black-and-white photograph by making an “informed guess” based on the greyscale root.

When data scientist Samuel Goree tested DeOldify, an AI colourization app, to convert a greyscale copy of Alfred T. Palmer’s 1943 photograph Operating a hand drill at Vultee Nashville, the result produced an image in which the black female subject’s skin was lighter.

Interventions like these are not unique among the history of photographic manipulation — the Cottingley Fairies photographs taken by Elsie Wright and Frances Griffiths in 1917 are a prime example. But alongside sophisticated internet tools like deepfakes (where a person in an existing image or video is replaced with someone else), the use of algorithms to alter photographs has provoked renewed anxiety about the authenticity of photography in the digital era.

As a researcher of film and visual culture, I am interested in exploring the convictions behind controversies like these by looking at them through the history of image manipulation. The use of colourization to create revisionist histories of atrocity and synthetic skin tones is concerning, but it does not mark the first time colourization has caused controversy.

Colors of Benetton controversy

In 1992, the clothing brand United Colors of Benetton sparked outrage when it re-purposed a colourized photograph of David Kirby, who had just died of AIDS-related complications, and his family for its advertising campaign.

“The face of AIDS” was the name given to the photo in the iconic spread in LIFE magazine. Photographs like these were meant, in part, to encourage sympathy and relatability towards sufferers of the most stigmatized illness around.

On National HIV Testing Day, we revisit the deeply moving photos that changed the face of AIDS: http://t.co/ttsBxnf8 | pic.twitter.com/5nGHJTEb

— LIFE (@LIFE) June 27, 2012

When the black and white photo was selected for Benetton’s ad campaign, executives made the decision to colourize it. This was done using a technique that was developed during the early years of photographic production called hand-colouring that required setting pigment down on the image and removing it with cotton around a toothpick.

The two issues that galvanize this strange campaign are its realism and its dignity.

Problems with colourization

Opposition to colourization often points to the artifice of the practice, but for the Benetton executives the problem with the Kirby photograph was not that it looked too real, but that its realism seemed incomplete.

The colourist, Ann Rhoney, described it as creating an “oil painting,” and the act of making a photograph more real by turning it into a painting appears to reverse longstanding assumptions about the art practices that are closest to reality.

However, Rhoney’s self-stated objective was not to make the photograph more real, but to both “capture and create Kirby’s dignity.” Kirby’s father supported the effort, while gay rights organizations called for a boycott of Benetton.

Marina Amaral, a Photoshop colourist working to colourize registration photos from Auschwitz for Faces of Auschwitz, claims her work helps to restore the victims’ “dignity and humanity” while Cambodia’s culture ministry said Loughrey’s images affected “the dignity of the victims.”

Disagreements about dignity tend to mirror those about photography and colourization: for some, dignity is inherent to an original, for others, dignity is something you add.

And the examples are abundant. Peter Jackson’s decision to colourize historical footage from the First World War for his 2018 film They Shall Not Grow Old drew criticism from historian Luke McKernan for making “the past record all the more distant for rejecting what is honest about it.” The YouTube channel Neural Love has faced resistance to its “upscaling” of historical footage using neural networks and algorithms.

Colourization became routinely controversial in the 1980s when computers replaced hand colourists and studios began colourizing a host of classic films to appeal to larger audiences. Objections to the practice ranged from poor quality, the commercial forces behind the practice and the omission of the qualities of black and white, to the implicit contempt for artists’ visions, a preference for the originals and a disregard for history.

Gene Siskel and Roger Ebert famously called the practice “Hollywood’s New Vandalism.” Philosopher Yuriko Saito suggested that disagreements over the value of colourization often turn on an implicit belief in whether a work of art belongs to the artist or to the public.

In the context of historical images, the question becomes: to whom does history belong?

Photographs contribute to our development as moral and ethical subjects. They allow us to see the world from a point of view that does not belong to us, and alterations that make photography and film more familiar and relatable complicate a primary role we have given it as “a vehicle for overcoming our egocentricity.”

Photography and AI

The recent controversies around image colourization point to the similarities between photography and AI. Both are imagined to create representations of the world using the least amount of human intervention. Mechanical and robotic, they satisfy a human desire to interact with the world in a non-humanized way, or to see the world as it would look from outside ourselves, even though we know such images are mediated.

What is fascinating about new techniques of colourization is that they can be understood as photography seeing its own image through AI algorithms. DeOldify is photography taking a photograph of itself. The algorithm creates its own automatic representation of the photograph, which was our first attempt to see the world transparently.

With the increasing accessibility of tools for colourizing photographs and making other alterations, we are re-negotiating the very difficulties first brought about with photography. Our desire for and disagreements about authenticity, mechanization, knowledge and dignity are reflected in these debates.

The algorithm has become a new way of capturing reality automatically, and it demands a heightened ethical engagement with photos. Controversies around colourization reflect our desire to destroy, repair and dignify. We don’t yet know what a photograph can do, but we will continue to find out.

About the Author

Dr Roshaya Rodness obtained a PhD in English at McMaster University and is currently is a Postdoctoral Fellow in Visual Studies at the University of Toronto. Her work is focused on the intersection of cinema studies, queer theory and continental philosophy. Her research looks specifically to the history of queer film and visual culture to explore radical and promissory forms of social being that are unexpectedly revealed by the film camera’s unique perceptive qualities.

To learn more about Roshaya, you can visit her university profile here. This article was also published on The Conversation and was shared with permission.