Invited talk at Bentley University on September 15, 2023. I talk about my recent paper co-authored with Marta Stelmaszak on trace data and how knowledge workers actively perform such data.

1. September 15, 2023
Bentley University
The Performative Production of Trace
Data in Knowledge Work
Aleksi Aaltonen (a paper co-authored with Marta Stelmaszak)
aleksi@temple.edu
@aleksi_aaltonen

2. My research:
Data and data-based innovation and organizing
‘Artifactual perspective’:
Digital data as human-made resource or material

3. Today’s paper
Aaltonen, A., & Stelmaszak, M. (forthcoming). The
performative production of trace data in knowledge work.
Information Systems Research.

4. Assumptions About Trace Data in IS
Naturally occurring ‘by-products’ of information systems use, whose collection
does not inconvenience the employee (Howison et al. 2011, Hüllman 2019,
Leonardi 2021)
A major opportunity to understand employee behavior in research and
managerial practice (Andersen et al. 2016, Berente et al. 2019, Pentland et al. 2020,
2021, Leonardi 2021, Bailey et al. 2022)
Strictly behavioral–capture traces of users’ actions but not what those actions
mean for the actors (Grover et al. 2020, Pentland et al. 2020)
For a summary of trace data definitions in IS, see Online Appendix A of our paper.

5. Performativity of Data
The broader literature challenges the depiction of trace data as naturally occurring
facts that are found rather than actively produced: “trace data are performative.
Not only records of performance, they also contribute to the constitution of the
reality that they trace” (Østerlund et al. 2020, p. 3).
Trace data could seemingly (paradoxically) provide both greater behavioral visibility
and less transparency of work practices if knowledge workers start to shape the
data to their own interests.

6. RQ: How do knowledge workers produce
trace data?

7. Performativity Lens
Organizational reality is produced and reproduced in work practices (Scott and Orlikowski 2008,
Cecez-Kecmanovic et al. 2014, Orlikowski and Scott 2014).
As knowledge workers harness trace data for their work practices, they also become aware that
trace data will be generated from those practices and that such data may become part of
subsequent iterations of their own practices or those of their managers.
Reactions to data collection have been described as gaming (Stein et al. 2015, Marjanovic and
Cecez-Kecmanovic 2017, Faraj et al. 2018), impression management (Brivot and Gendron 2011,
Leonardi 2014), and transfiguration work (Cunha and Carugati 2018).

8. In-depth Case Study
Knowledge work as the context: The nature of knowledge work has made it traditionally difficult to
monitor and control knowledge workers because most direct or behavioral measures have lacked
specificity, reliability, or efficacy with respect to knowledge work and its aims (Alvesson 2001,
Rennstam 2012).
Research site: A large business school in the UK that implemented a learning management and
analytics system to monitor teaching-related practices (knowledge work par excellence).
Data collection: 32 semi-structured interviews using snowballing sampling, 700 pages of technology
strategy committee meeting minutes, other documents and system usage observations.
Data analysis: Analyzed work practices before and after the implementation of the system
collecting and displaying trace data (cf. Cunha and Carugati 2018).

9. Six types of changes to work practices
Table 2. Changes to work practices
Type of change Description Change affects
Curating data-based
impressions (I)
The pervasive collection of trace data motivates knowledge workers to constantly anticipate how their own and
their students’ actions are inscribed and may subsequently be represented from the data.
The actions of:
• Focal knowledge worker
Encouraging students
to engage with data-
producing activities (II)
Knowledge workers anticipate how their students’ actions may figure in future decisions and, consequently,
encourage the students to engage in data-producing activities, even by representing the data back to the students
to motivate actions that produce further data.
The actions of:
• Students
Restructuring teaching
materials for data
production (III)
Knowledge workers perceive a managerial need to compare courses and teaching practices, which requires
breaking down and restructuring teaching materials onto a common template that enables the inscription of
granular and comparable data across courses.
Artifacts used by:
• Focal knowledge worker
• Students
Designing new learning
activities that produce
data (IV)
Knowledge workers design new learning activities with a view to producing trace data from an expanded scope of
activities, which is anticipated to help obtain funding for more teaching positions by showing the breadth of
student activity from the data.
The actions of:
• Focal knowledge worker
• Students
Shifting pedagogical
approaches to produce
more data (V)
Knowledge workers perceive an overall shift toward online pedagogy, which expands the opportunities to inscribe
teaching and learning as trace data and calls for new ways to teach and to justify pedagogical choices.
The actions of:
• Knowledge workers
• Students (by implication)
Supporting investment
in data-producing
teaching positions (VI)
Knowledge workers perceive that the data make it easier to justify hiring for teaching-related positions and
support teams, which results in more teaching and learning activities being inscribed as trace data.
The actions of:
• Other knowledge workers

10. First- and second-order prefiguration
Two patterns of changes to work practices driven by anticipated or ‘envisioned’
uses of the data in the future.
First-order prefiguration: changes to work practices that shape data from worker’s
own actions.
Second-order prefiguration: changes to work practices that shape data from
others’ actions.
These are not just one-off reactions to the initial ‘shock’ of becoming aware of trace
data in the system…

11. Inscription
Work
practice Data
Representation
First-order prefiguration:
(I) Curating data-based impressions
(IV) Designing new learning activities
(V) Shifting pedagogical approaches
Using trace data representations to motivate further data
production
Anticipatory changes
by a knowledge worker
Second-order prefiguration:
(II) Encouraging data-producing activities
(IV) Designing new learning activities
(V) Shifting pedagogical approaches
(VI) Supporting investment in data-producing positions
Changes to artifacts inscribing data:
(III) Restructuring teaching materials
Elements of Configuration work

12. Contributions
1. We show that under the conditions of trace data i) awareness and ii) interest,
trace data may not occur naturally à several implications to research and
methods based on trace data
2. We show how trace data are performed in and through work practices that are
jointly enacted by different parties à different actors have different agencies in
these performances
3. We show that collecting trace data to monitor knowledge workers motivates
them to engage in configuration work aimed at transforming the work itself to
fit continuous data-based monitoring à the data may produce a more detailed
picture of work being done, but they also make people work differently

13. Managerial Implications
A digital work environment will almost unavoidably inscribe some trace data.
Organizations may find it useful to draw up a trace data policy that specifies the purposes for which
trace data may and may not be used. This could:
1. Curb some unnecessary configuration work.
2. Be used to embrace the performativity of trace data by drawing attention to organizationally
valuable behavior.
Finally, managers may need to become involved in the design of trace data and not to leave it for
systems developers, to make the data truly valuable organizational resources.

14. Thank you!

15. Appendix

16. What’s New About Prefiguration?
Focus on practices not on individual behavior.
Gaming typically happens against immediate or almost immediate behavioral feedback from the
system, such as when Uber drivers log off to trigger surge pricing (Rosenblat and Stark 2016, Faraj et
al. 2018, Möhlmann et al. 2020) à the expected benefits of prefiguration are envisioned: that is,
projected to the future.
Impression management and transfiguration work entail shaping the impressions that the work
entails, whereas prefiguration must directly target the work itself to change the production of trace
data.

17. Table 1. Empirical evidence
Data source The amount and type of data Collection period
Semi-structured
interviews
32 interviews with 29 informants, totaling 1,528 minutes with an average length of 49 minutes
(shortest 24 minutes, longest 85 minutes)
• 15 interviews (3 shared roles) with administrative and professional services employees;
responsibilities include operations and program management at both undergraduate and
postgraduate levels, as well as teaching and learning support roles and administrative
roles (quotes identified as “Administrator”)
• 8 interviews (1 shared role) with teaching staff; responsibilities include teaching and
teaching-related administrative duties (quotes identified as “Teacher”)
• 5 interviews (2 shared roles) with technical staff; responsibilities include developing the
learning management and analytics system and supporting program management teams
(quotes identified as “Developer”)
• 4 interviews with academic staff; responsibilities include academic research and some
teaching and teaching-related administrative duties (quotes identified as “Academic”)
June to September
2017
Learning management
and analytics system
• Observation notes, 24 hours of observation in total
• Screenshots of trace data and analytics-related features, a total of 25 documents and 5
additional documents supplied by interviewees
March to January
2018
Committee minutes • 30 sets of minutes from the meetings of the Technology Strategy Committee held between
2013 and 2016, a total of 700 pages
January 2018