Vision and Automation in a post-Covid World (Journal of Visual Culture & HaFI, 42)

This is the forty-second instalment of a collaborative effort by the Journal of Visual Culture and the Harun Farocki Institut, initiated by the COVID-19 crisis. The call sent to JVC’s editorial board, and a wide selection of previous contributors and members of its extended communities, described the task as follows: “There is a lot of spontaneous, ad hoc opinion-making and premature commentary around, as to be expected. However, the ethics and politics of artistic and theoretical practice to be pursued in this situation should oblige us to stay cautious and to intervene with care in the discussion. As one of JVC’s editors, Brooke Belisle, explains: ‘We are not looking for sensationalism, but rather, moments of reflection that: make connections between what’s happening now and the larger intellectual contexts that our readership shares; offer small ways to be reflective and to draw on tools we have and things we know instead of just feeling numb and overwhelmed; help serve as intellectual community for one another while we are isolated; support the work of being thoughtful and trying to find/make meaning…which is always a collective endeavour, even if we are forced to be apart.'” TH


Image from Common Objects in Context (COCO) image dataset used to train computer vision systems


Vision and Automation in a post-Covid World

By Joel McKim


The Guardian headline reading “Robots to be used in UK care homes to help reduce loneliness” is accompanied by a photo of a gleeful senior citizen interacting with a wheeled semi-humanoid robot named Pepper. The article describes a research trial studying the use of “culturally competent” robots to assist in caring for the elderly in the UK and Japan and although the project actually began several years ago, it’s only now, post-onset of Covid, that the story has found journalistic attention. The image of AI-powered robots offering a possible solution to the financial difficulties, health concerns and staff burnout facing the care industry, is one seemingly designed to elicit equal parts anxiety and condemnation.

But the role automation will play in our post-Covid world is likely to be less spectacular, yet more disruptive than such contentious visions of robotic health facilities. And while we may balk at AI replacing the forms of care and intimacy we view as innately human, there are already indications that the current crisis is making many of us increasingly amenable to other forms of more mundane automation. The move towards cashier-less retailers and automated warehouses that perhaps caused some unease in the past, may now appear as a sensible and even ethical solution. A debate about mass unemployment and increased corporate profits is quickly reframed into one about responsible health decisions and employee safety. As the human-computer interaction researcher Richard Pak asserts in a New York Times article on this very subject, “Pre-pandemic, people might have thought we were automating too much. This event is going to push people to think what more should be automated.”[i] For those with much to gain from greater levels of work-place automation, this would seem an opportunity to seize upon a moment of increased public receptivity.

If we’re to trust the investment reports, business magazines and policy think tank whitepapers on the topic, post-pandemic interest in automation is already surging. To take one circulating, but not-entirely disinterested example, the Honeywell Intelligrated Automation Investment Study found that since the onset of Covid over half of US businesses are increasingly open to investment in automation with e-commerce; grocery, food and beverage; and logistics companies chief among them. But on the topic of labour impact, these sources tend to offer either platitudes – “The vulnerable will be the most vulnerable” (Brookings Institute)[ii], or hollow reassurances – “People empowered by automation will bring us out of this crisis” (Forbes).[iii]

A flurry of more sustained academic considerations of the potential impact of AI and automation have emerged over recent years, all cautionary but with various levels of optimism or pessimism regarding how humans will fare in the automated world to come – books like Erik Brynjolfsson and Andrew McAfee’s The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies and Carl Benedikt Frey’s The Technology Trap: Capital, Labor, and Power in the Age of Automation. Frey’s 2013 research with Michael Osborne at the Oxford Martin School, published as “The Future of Employment: How Susceptible Are Jobs to Computerization,”[iv] was an early and influential warning of just how cataclysmically disruptive processes of automation may be for the current labour force. Analyzing O∗NET, an online detailed description of occupations developed for the US Department of Labor, Frey and Osborne determined that 47 percent of US employment falls in the high-risk category of being automated within the next decade or two. Frey has also acknowledged the potential of the Covid-crisis to acerbate an already troubling situation, writing in the Financial Times, “automation anxiety looks set to witness a revival – and with good reason. Coronavirus is likely to accelerate automation.”[v]

Why does this admittedly disheartening discussion of automation and work precarity belong in a series reflecting on Covid visual culture? Recent advances in computer vision technology (enabled by machine learning and deep neural networks) are one of the primary factors driving forms of automation that importantly span both the manufacturing and service economy, from driverless vehicles, to assembly line robotics, to checkout free shops. One indication of this development, image recognition error rates for neural networks have fallen precipitously in recent years, with computer vision algorithms now able to classify images in large datasets like ImageNet with as much as 98% accuracy.[vi] Our powers of perception – once, like caring, a seemingly innately human capacity resistant to machine replication – are apparently no longer the insurmountable barrier to automation we might have believed. Much of this technology has moved beyond mere speculation and into a prototyping phase, and of course some forms of workplace automation, like warehouse robotics, are already with us. The drone delivery systems and autonomous shipping vehicles, envisioned by companies like Amazon are still unproven, but the underlying computer vision systems they rely upon are developing quickly and a Covid investment push may well bring forward their timetable of deployment. In an economy increasingly reliant on now vulnerable service sector jobs, like transportation and retail, this could have a devastating impact.

These concerns regarding the relationship between human and computational vison were of course central to Harun Farocki’s own prescient investigations of machine vision and operational images, from military tracking systems to computer simulations. Today, artists like Heather Dewey-Hagborg and Adam Harvey, are producing work that compels us to think critically about our contemporary technologies of artificial perception, exposing the technical and social infrastructures that underpin facial recognition systems, for example. These artists help shed light on an already unfolding regime of vision and automation, one potentially accelerated in our post-Covid world.

Joel McKim is Senior Lecturer in Digital Media and Culture and the Director of the Vasari Research Centre for Art and Technology at Birkbeck, University of London. He was recently a visiting fellow at the V&A Museum researching “A Pre-history of Machine Vision.”
September 17th, 2020 — Rosa Mercedes / 02

A  word on “post-truth” by postcolonial and photography scholar Zahid R. Chauhary (from his 2020 essay “The Politics of Exposure: Truth after Post-Facts”):So perhaps it is not simply that truth acts (such as whistleblowing) expose what we already know, but that the place of knowledge in an atmosphere of fetishistic disavowal lends such disavowal a libidinal frisson. In cynical reasoning, truth actually matters a great deal because acting in spite of it is what endows the action with its distinctive fetishistic pleasure.”

October 26th, 2021

Lauren Berlant, the brilliant theorist of “cruel optimism” and related issues, died of a rare form of cancer on June 28. The following, devastatingly optimistic quote is from a 2016 essay on the commons as “infrastructures for troubling times,” part of a book that they worked on with the typically double-edged title On the Inconvenience of Other People: “What remains for our pedagogy of unlearning is to build affective infrastructures that admit the work of desire as the work of an aspirational ambivalence. What remains is the potential we have to common infrastructures that absorb the blows of our aggressive need for the world to accommodate us and our resistance to adaptation and that, at the same time, hold out the prospect of a world worth attaching to that’s something other than an old hope’s bitter echo. A failed episode is not evidence that the project was in error. By definition, the common forms of life are always going through a phase, as infrastructures will.”


Some basics from the Strike MoMA site: “Campaigns, actions, and letters chip away at the regime’s facade from the outside. Inside, every time workers organize, defy the boss, care for a coworker, disrespect secrecy, or enact other forms of subversion, cracks are created in the core. Cracking and chipping, chipping and cracking. As the walls that artificially separate the museum from the world collapse, we reorient away from the institution and come together to make plans. Let us strike in all the ways possible to exit from the terms of the museum so we can set our own.”


via Hyperallergic on the environmental impact of blockchain referring to recent NFT (non-fungible token) art sales: “This is not the first time the art world has come under scrutiny for being on the wrong side of the climate conversation. Artists and activists have protested everything from the carbon footprint of physical art fairs to the fossil fuel money funding major museums. But some say the energy consumption of cryptocurrencies is particularly egregious, and research shows it’s relatively easily quantifiable. A study by Cambridge University, for instance, estimates that bitcoin uses more electricity per year than the entire nation of Argentina. (Ethereum mining consumes a quarter to half of what Bitcoin mining does, but one transaction uses more power than an average US household in a day, according to the Institute of Electrical and Electronics Engineers.)”


Nicholas Mirzoeff on “Artificial vision, white space and racial surveillance capitalism”: “Based as it is on ‘epidermalization’ (the assertion of absolute difference based on relative differences in skin color), AI’s racial surveillance deploys an all-too-familiar racialized way of seeing operating at plan-etary scale. It is the plantation future we are now living in. All such operations take place in and via the new imagined white space of technology known as the cloud. In reality, a very material arrangement of servers and cables, the cloud is both an engine of high-return low-employment capitalism and one of the prime drivers of carbon emissions.”


Sara Ahmed on the performativity of disgust (from The Cultural Politics of Emotion, 2004): “To name something as disgusting is to transfer the stickiness of the word ‘disgust’ to an object, which henceforth becomes generated as the very thing that is spoken. The relationship between the stickiness of the sign and the stickiness of the object is crucial to the performativity of disgust as well as the apparent resistance of disgust reactions to ‘newness’ in terms of the generation of different kinds of objects. The object that is generated as a disgusting (bad) object through the speech act comes to stick. It becomes sticky and acquires a fetish quality, which then engenders its own effects.”

November 7th, 2020
moreless news