Materiality & Embodiment

Abstraction and material concretisation in computing

Debates about the character of computer science, about what kind of discipline it is (applied mathematics, engineering, Gestaltungswissenschaft, or even a social science) and what topics, methods and tools are employed, also touch upon core questions on abstraction and concrete materializations. This has to do with the fact that unlike mechanical machines, computers are semiotic machines. The duality of the algorithmic sign, the performative dance between abstraction and concretization, shapes how digital systems are conceptualized, designed, and embedded in sociotechnical contexts. Noteworthy, Konrad Zuse wrote in the patent application of the Z3, the first fully functional electromechanical computer completed in 1941, that this machine is “mathematics made flesh” (”fleischgewordene Mathematik” [62, p. 100]).

Despite the obvious link between hardware and computational processes, computer science is often framed as immaterial.¹ Yet, of course, abstractions rely on the physical world. Hardware constraints, energy consumption and limits in raw materials impact how algorithmic systems can be build and perform. These aspects also situate computing in broader sociocultural and geopolitical contexts. In addition, there is no immaterial software (some even claim, there is no software), an aspect more easily comprehensible through the history of computing. However, critical theorists argue that abstraction obscures material aspects of computing [63], and, in particular, the material impact computing technology has, such as waste, energy and raw material consumption.

Furthermore, the history of computing highlights computation as an embodied process that is closely interwoven with stories of in- and exclusion. In a field that also designs and constructs, bodily engagement and material aspects are evident through tangible interaction [64], coding practices, artefact building etc. Frieder Nake describes computer science as “the scientific discipline that deals with the machinization of cognitive labor” (”Informatik ist die Wissenschaft der Maschinisierung von Kopfarbeit” [59]) which is reflected in works on embodied cognition in the field [65].

With the pervasiveness of digital technology, products and services, computing plays a powerful role in world-making. Translating abstractions into functional systems shows how these become effective as “material-semiotic nodes” [34, p. 595] mediating power and values. AI models, for example, may materialize designer’s assumptions, discriminatory data and algorithms in a way that encodes and amplifies bias into deployable systems [66]. How, where and by whom AI systems are then used has concrete material effects.

Reimagining abstraction and materiality through the carrier bag

The carrier bag theory of computing allows us to bring questions of materiality and embodiment to the fore and to reflect on the performative dance between abstraction and concretization of computer science more broadly. In particular, we could play with different scenarios around abstraction and its relation to materiality through the bag metaphor.

With the carrier bag we can reimagine abstraction as a practice of curation rather than a process of distillation. Abstraction as curation would entail remaining attentive where something is abstracted from, with what tools and towards what ends [67]. Like gathering diverse objects into a bag, computational systems organize material realities (data, hardware, user interactions²) into relational structures. Here, a couple of examples come to mind. For one, we could rethink databases as carrier bags, for example, emphasizing relational databases that aggregate heterogenous data types into interconnected tables [68]. That would mirror the bag’s capacity to hold multiple objects/types without having a strict hierarchical structure. Also, we could use the bag to rethink neural networks in machine learning. Machine learning models, in a way, collect patterns from training data, much like a bag collects objects, but their abstractions remain contingent on the materiality of input sources and hardware constraints.

In line with new materialist Informatics, we could reconceptualise materiality as such as generative. Obviously, a carrier bag’s physical properties shape what can be gathered. We could use this to think about how material realities ground and limit abstraction. The case of raw materials, including the geopolitical implications and social consequences (exploitative labour practices etc.), comes to mind posing questions of social, political and environmental sustainability at the core of abstraction.

Additionally, a carrier bag approach with its focus on materiality foregrounds computing itself as design material [69]. What kind of material properties of computing are pertinent for design? How might their agency be accounted for in design? What are the materialities (physical, but also semiotic) that are active in computing technologies? This attention to computing as design material corresponds with emerging research on data as design material [70] and AI as design material [71].

Furthermore, the carrier bag could serve to reframe abstraction as a ‘leaky practice’. Depending on the material qualities of the bag it could be porous, semi-permeable or like a fish-net – allowing for objects to enter, exit, spill, leak through. This could help us view the above described problems of bias in algorithmic systems as leakage, reflecting on what we lose. Regarding computing infrastructures, we could consider server farms’ materiality obscured by the cloud metaphor as ecological spillage.

Summing up, the carrier bag theory of computing disrupts the myth of disembodied abstraction, revealing computation as a material-semiotic practice that gathers, transforms, and redistributes both signs/symbols and substances/matters. Through this, abstraction processes in computing should always be reframed as grounded in the embodied realities they seek to represent. Last but not least, computer science is thus linked to questions of ethics, responsibility and accountability at its core, and not at the edges of the discipline or only when it comes to application domains.

¹ Following this, David Berry [2011] frames bringing software back into visibility as one of the central challenges of software studies. Hiding behind the cloud-methaphor - and the metaphoric character of software itself [36], the materiality of computing technologies is usually hidden or disregarded.

² Before this organisation even takes place, a process of classification has happened in which lived realities, relationships, and life stories have been made readable for computers in data structures.