What's the relationship between user experience and design professionals (UX&D), and (academic) researchers in human-computer interaction (HCI)? This webpage is all about providing resources to answer that question.

There has been intermittent interest in this question from both HCI researchers and UX&D professionals (I use this acronym to hint at the broad gamut of people working in the UX, interaction design, information architecture, and broader digital design space). This page attempts to document instances of research, discussions, findings and people who have investigated this topic (or something related to it) in the past and / or currently. This list covers:

• Research studies on this topic
• Personnel investigating it
• Personnel practicing it

As such, it is part annotated bibliography and part resource list. For each I include my own notes (which may be quite selective, of course!). The list is currently not very ordered or organised.

If you have something to add, suggest or critique, please contact me as I would like to build the page into something that is ever-more comprehensive and usable, offering a range of perspectives and approaches. If I have written about your work and have summarised it poorly I would welcome contributions in this regard. Finally, it is worth noting that the list will probably naturally have an academic bias, but this is not intentional and I would be keen to hear about views from the 'other side of the fence'.

Contents

1. Annotated resource list (i.e., research on HCI-UX connections)
2. A note on terminology (UX? HCI? 'practitioner'? etc.)
3. Key UX&D literature (for professionals)
4. Extended bibliography (academic research literature on the HCI-UX connections topic)

Annotated Resource List

Amodeus-1 / Amodeus-2

The Amodeus project (actually two projects, Amodeus-1 and Amodeus-2) was a large-scale European (ESPRIT scheme) project that lasted some seven years, running from the late 1980s through to the mid 1990s. The final report (from 1995) describes the project's outputs in detail.

A significant portion of the project was dedicated to uncovering "routes for transferring basic research in Human-Computer Interaction (HCI) to interface designers". It should be noted that the project commenced at a critical point in the design of user interfaces, as many systems were shifting from text-based interfaces to fully graphical user interfaces. These changes also happen against a backdrop of ever-increasing applications of computer systems for all manner of tasks with ever-more diverse sets of users. These factors presented an explosion in complexity for the design of these new interfaces. It makes sense that this project would address matters of research-practice "transfer" (as it conceived of it). It is worth noting also that at this point, HCI was very much a young field, and far smaller community of researchers that it currently is (as of 2015).

The key outputs of the project in terms of HCI research and design practice connections seemed to focus around design rationales work—with a 'design rationale' being a set of documentation articulating why certain decisions have been made during a design process. This work included things like Design Space Analysis, and the Questions Options Criteria notation (see Bellotti et al. below). The QOC notation in particular is "semi-formal" structured interrogation method for design problems. To quote from Bellotti (1993):

"A QOC design space analysis captures argumentation about multiple possible solutions to design problems. The space is structured by Questions expressing design issues. Each Question is linked to multiple Options which are alternative solutions to the same problem. Options are positively or negatively assessed by each of a set of Criteria. An Option can also spawn consequent Questions which assume that Option is part of the design context of further argumentation"

A particularly enlightening paper is by Victoria Bellotti (1993) on "Integrating theoreticians' and practitioners' perspectives with design rationales". In the paper theoreticians ("cognitive modellers") are pitted against practitioners ("software designers") when solving a design problem. For instance, formalist theoreticians create evaluations like 'all mouse actions are non-deterministic'; designers indicated that they thought they could not benefit from this. QOC was then used as an evaluative tool to compare practitioners and theoreticians solutions. The paper essentially argues that theoreticians' results can shape / support designers' work, and provide richer design solutions. Further it indicated that designers have confirmation biases about their own solutions, and overall argues that designers could have used theoretical insight.

Elizabeth Goodman

Goodman has conducted ethnographic studies of interaction designers at work in their design practices / studios. Her ethnography largely focuses on the ways in which the designers conduct prototyping (e.g., with wireframes) and—crucially—how those prototypes get presented to clients.

Notes

• Goodman's work generally advocates the idea of empirically examining design practices and their "lived complexity" rather than assuming that HCI research is a natural fit for practitioner's work.
• Broadly speaking, this work examines the use of representations in interaction design work (e.g., wireframes, paper prototypes, specification documents, website maps, etc.).
• A key concept in this work is the 'performative' nature of prototyping interaction designs ('performative' as in theatre, acting, etc.).
• The work articulates how considerable practical physical presentational skills are mobilised to animate and 'fill in' the necessarily partial representations of interactivity that is possible in prototype designs.
• Goodman uses the term 'handwaving' to describe a range of largely unrecognised (at least, formally speaking) practices that are vital to 'working around' the immateriality of interactivity in design objects.
• One paper by Goodman, Stolterman and Wakkary ("Understanding Interaction Design Practices") provides a nice overview of the 'state of play' of HCI understandings of design practitioners. They argue that HCI has tended to "rationalize away" the need to examine the link between HCI and relevant design practices like UX&D.
• Goodman has also worked with Stolterman (see below).

NSF Project on Research into Interaction Design Practice at University of Indiana

This NSF-funded research project has been engaging in study of interaction design practice as a topic for some years (2012(?) to present).

• The project PIs are Erik Stolterman and Martin Seigel. Colin Gray has done work in this project which is of most interest to this list of resources.
• A broad goal of the project is to examine "what design methods [interaction design] practitioners actually use, why they use them, and how they understand methods".
• A key output for the project is methodological in that they seek to improve method development to better fit practitioner needs.
• This is one of the few significant projects looking at studying design practices in the technology space.

Notes

Below is a summary of relevant publications from this research work. I have restricted my focus to the work on investigations of designer's work practices.

• Some of this research has analysed the proportion and type of literature published at HCI venues (CHI in particular) that seems intended for practitioner take-up. They find that HCI research targeted at or pitched towards practitioners tended to make key assumptions about the nature of that practice (e.g., overgeneralising results rather than considering specific situations, not taking into account the collaborative nature of design practices in IxD and UX, and assuming very linear design practice processes).
• A theme of this research is about looking at how design practitioners develop competence in their work and thus explores matters of education, reflection on practice, etc.
• This research has explored self-reports (interviews, surveys, etc.) of UX professional's developing competences in their practice. They found that the development of (and therefore self-perception of) UX practitioner competence was enmeshed with the culture / rationale of the organisation they worked for. UX practitioners need to develop strong "visual competencies" and key "related manual skills" so as to transcend tool-reliance.
• The research has also explored UX&D practitioner perspectives more broadly via interview. Below are some key aspects of this.
  • Perhaps predictably they find that practitioners tend to ad-hoc methods to suit the situation rather than following more formal approaches, they also struggle to connect with academic work, and are strongly oriented to how the design methods they use might be more or less successful in working with stakeholders.
  • Regarding methods, the 'idea' of the method was more important than the method per se. Further, methods were (often radically) appropriated as part of their use by UX and IxD practitioners, i.e., they were used in ways not necessarily expected / intended by their progenitors.
  • In summary, then, "Design methods (including tools, activities, and theories, among others) from academic sources are co-opted by designers in a highly pragmatic sense where the generative work of the designer takes precedence over order or guidance imposed by any one academic method."

Erin Friess

Friess has been performing empirical studies of UX&D with particular focus on the ways in which personas (a popular tool) feature in the design process. Although she has published at HCI-related venues, her domain seems to be (technical) communication-research focussed. She has also published findings from her research in the Journal of Usability Studies, which is a UXPA-organised journal.

Notes

• Friess's studies involve discourse analysis of usability testing as it happens, and also design meetings where scenarios and personas are invoked.
• Some of this work examines the translation of usability testing into formal reports (usability findings). Here, Friess's empirical work detects differences between the formal reports and the proceedings of the actual usability testing site itself (e.g., via think aloud protocols or similar), although she states that "about 84% of the findings in the oral reports had some referent in usability testing". Firstly, she finds that some usability findings don't have a basis in the usability testing, while others do; secondly, that some findings are emphasised over others in potential conflict with their prominence within usability testing itself (a "confirmation bias", perhaps partly driven by the expectations of a client)—equally, the opposite may also be the case, i.e., that certain findings may be omitted for various reasons.
• Other work examines the use of personas, a common technique in user-centred design processes, particularly UX and IxD. Friess finds that direct / explicit persona use was actually quite limited in design meetings ("3% of the conversational turns"): few of the developed personas were routinely employed, and persona use was generally limited to specific aspects of the design process, such as a cognitive walkthrough of the design in question. In other situations, such as convincing others of a particular course of action, personas were not used to formulate arguments for or against this or that design decision. (This research is an interesting contrast to examinations of how 'the user' is deployed in strategic ways during design meetings.)

Susan Dray

Dray maintains (as is a rare case, it seems) both practitioner and academic careers. Like other 'boundary hoppers' listed here she has noted the gaps between academic research and practice, and has written on the topic.

Notes

• Like Instone's description of the research-practice bridge (see below), Dray highlights the different 'games' (my term, not hers) that are at play for academic researchers and those doing UX / IxD work, which she characterises by contrasting the academic's "publish or perish" problem with the practitioner's "produce or perish" problem. Thus, the product development process may have a "tendency to suppress skepticism and critical thinking" often due to time pressures and the challenges of cross-disciplinarity.
• Dray argues for greater rigour in practice, which she sees as emerging from increased engagements between HCI academics and practitioners (which also benefit academic work)—which could be delivered via "engaged scholars". This is seen as way to ensure that UX work remains on a strong conceptual base, guarding it against encroachments by other practices such as marketing.
• Thus, the key value academic work can bring to practice is what Dray terms "critical thinking"—which is a willingness to ask deeper questions (e.g., "Whether the data is qualitative or quantitative, was it analyzed rigorously or merely impressionistically and taken at face value?").
• Dray recently accepted a Lifetime Practice Award at CHI 2015. The slides for her talk can be found here. The talk makes various points similar to those noted above.

Gitte Lindgaard

Lindgaard has practitioner and academic credentials. She has begun conducting interviews with UX practitioners, exploring the methods they use (e.g., usability evaluations, expert reviews / heuristic evaluation) as a way of investigating the gap between HCI research and practitioner's work.

UX Research-Practice Interaction ACM SIG

This is a Special Interest Group that has been set up to encourage links between (academic) research and (industry) practice worlds within HCI, and the CHI community specifically. There are various members involved (e.g., Clare Hooper, Elizabeth Buie), but Keith Instone has the broadest range of resources on his website. There have been several events, reaching back to 2010, for which I provide a rough timeline:

• A workshop and special interest group at CHI in 2010.
• A panel at the Internet User Experience conference in 2010, called "Research-Practice Interaction".
• A workshop at UPA conference in 2011 (now UXPA).
• An "idea market" at UXPA conference in 2013.

Notes

Instone's notes to his talk at the Connecting Dots Conference in 2014 are perhaps the most clear summary of the above series of meetings, workshops, panels etc. Drawing out a few key points from this:

• The bridge / link between academic research (HCI) and industry practice (UX, IxD) is described in terms of a series of challenges around communication, knowledge and education.
  • Communication between the two sides is problematised by things like a lack of "shared language", differences in the "speed-of-operation" (e.g., academics tend to be slow), and translational problems involved in mapping "research answers" to "practical questions".
  • Regarding the knowledge bridging challenge, Instone identifies the need for shared knowledge bases, that academic knowledge is quite hard to organise / reformat into practically-oriented resources.
  • Finally, there is the educational challenge of differences between what is taught as HCI syllabus versus what practical concerns there are for practitioners at work.
• It seems that one of the key aims of this work is in organising a series of activities that at least starts relevant people talking about the topic (and therefore raising awareness, etc.). This seems to chime as a relevant motivation for other efforts listed in this page.

Don Norman

Norman is one of the few people who seems to figure strongly both within the academic HCI world and that of UX / IxD. Within HCI Norman is a continuous contributor, but is perhaps best known for his '7 stage model' of action that describes the user's cognitive processes and their effect on the world—and the attendant 'gulfs' of execution and evaluation. He also is well known for redescribing Gibson's notion of affordances. His work seems similarly known to practitioners. This status makes him uniquely relevant for discussions of the links between academic HCI and UX / IxD.

Notes

• Norman has argued that the gap between HCI and practitioners (he does not really mention what kind of practitioners in this article) is a wide one and—while it is there for reasons—could be lessened.
• Norman argues for a new role, that of the "translational developer": "Translational developers are needed who can mine the insights of researchers and hone them into practical, reliable and useful results. Similarly translational developers must help translate the problems and concerns of practice into the clear, need-based statements that can drive researchers to develop new insights. Neither direction of translation is easy."

Rolf Molich

Molich is an important figure particularly for UX researchers, in that he did foundational work on developing usability heuristics (often called 'expert review') for inspection of interface designs, and also has run various 'comparative usability evaluations' (CUE) that evaluate the evaluations produced by practitioners when inspecting a particular design (such as a website). Like Norman, Molich is a 'boundary hopper', who engages with UX&D communities as well as academic HCI.

Notes

The CUE studies are examinations of practitioner's work within usability testing and as such bring an academic approach to this key practitioner technique. Molich provides a helpful list of four key issues the CUE studies have demonstrated. I can't sum them up any more clearly than he does, so I provide a quotation here in full:

1. "The number of usability problems in a typical website is often so large that you can't hope to find more than a fraction of the problems in an ordinary usability test."
2. "There's no measurable difference in the quality of the results produced by usability tests and expert reviews."
3. "Six—or even 15—test participants are nowhere near enough to find 80% of the usability problems. Six test participants will, however, provide sufficient information to drive a useful iterative development process."
4. "Even professional usability evaluators make many mistakes in usability test task construction, problem reporting, and recommendations."

Microsoft "Things we've learnt about" magazines

Larger companies such as Google or Microsoft have dedicated research divisions which often engage in high quality academic research. Often it is feasible for them to embed members of their research groups into product teams. However, it seems that the links between research and practice can be problematic for them too.

As a case in point, Microsoft Research has been producing public magazines that summarise their research in a format that is intended for the "busy people who work for Microsoft in the US, building products". This is an attempt to reformat some of their research division's findings in ways that demonstrate a strong link between research and product development at Microsoft's company as well as providing a glossy, accessible presentation of content from academic papers published out of work conducted at MSR. These magazines are freely downloadable.

Torkil Clemmensen

Alongside a range of co-authors, Torkil Clemmensen has been investigating the perspectives of usability and user experience professionals for some time. I've yet to summarise his work in this area, however there is a set of references below that provide links to his work.

Kath Straub

Straub has run a "Research in Practice" tutorial at the UXPA conference for a number of years.

• She has made "translating" research into practice a key selling point for her consultancy work. As such she is addressing this topic from within the UX community, and seems notable as one of the few explicitly engaging with practitioners as consumers of research work done in HCI and related fields.
• There may me more examples of those on the practitioner side offering this kind of translation work as a service, but I am not aware of it.

The Interaction Design Foundation (http://www.interaction-design.org)

This organisation seeks to offer materials and resources for UX practitioners

Me (Stuart Reeves—Academic researcher, University of Nottingham, UK)

This page is more about what others are doing at the moment, but you can find out more about my work in the area by looking at the Fellowship section of my website.

A note on terminology

In this page I refer to:

• Human-Computer Interaction (HCI) research. HCI is a problematic term: it can have a very different meaning for practitioners than for academic HCI researchers. Thus, for some practitioners it may seem an old fashioned and narrow research area from the 1980s that is superseded by the network of terminologies like User Experience, Interaction Design, Service Design, Information Architecture, etc. Whereas for HCI academics, HCI is becoming so broad and encompassing as to provoke yearly discussions about what is 'in' scope and what is 'out'.
• Characterisation of 'interaction design' (IxD) and 'user experience design' (UX). From my own (limited) understanding, it is better perhaps to conceptualise interaction design and user experience design as two practices that a single industry professional may engage in (either, or perhaps both). Equally, this characterisation, and the way I discuss UX / IxD tends to exclude other related practices, such as information architecture (IA). This is partly due to my own lack of knowledge, but also the fragmented nature of terminology around practitioners' naming choices over what it is they do. For now, I'll leave it as a vague, but knowing, 'gloss' instead that hopefully communicates the right thing.
• The 'CHI community'. This is a designation for the largest manifestation of the HCI research community: the CHI series of international conferences. While HCI is not CHI and vice versa, CHI represents probably the largest public forum for discussions of HCI research.
• 'Academic' research. I am aware that UX professionals in particular themselves conduct extensive research, however I make a distinction between this activity (which is often called 'user research') and the professional academic research activities of HCI (which are done for different purposes and different 'clients'). Thus, 'academic' research may be conducted by industry professionals (for example, Microsoft Research or Autodesk or Google) as much as academics in universities.

It is worth foregrounding the problems of some combinations of terms we face when dealing with the literature in this area. There are several that crop up to muddy the waters: 'research', 'theory', 'academic', 'practitioner', 'practice'. At the moment I am tempted to organise this work in the following three categories (although currently the list is not organised in this way):

1. Academics studying their own research practices. This is primarily a reflective activity performed by academics looking at their own work practices. Confusingly, here the term 'practitioner' refers to academic researchers as practitioners. This includes two subcategories:
   • Studies of research methods (e.g., usability testing) as performed in academic environments (e.g., labs).
   • Examinations of the link between theory and practice within academic HCI research.
2. Academics studying (non-academic) practitioners of UX, IxD, etc. This involves examinations of the work practices of practitioners in some way, primarily consisting of examinations of their methods, techniques, etc. This work takes place in practitioner-oriented contexts such as industry design studios, and involves interactions with practitioners in some way (e.g., via interview, observation, learning to be a practitioner, etc.).
3. (UX / IxD) Practitioners studying practitioners. This category involves people who are primarily practitioners examining and reflecting on the various methods, techniques, etc. that they use.

Orienting literature

I've collected together a range of literature that it seems can do a good job of orienting someone to the concerns, methods, approaches, concepts, challenges, etc. of UX&D professionals. As well as seeking out literature myself I've had help from Stavros Garzonis, Elsa Bartley and Rebecca Gill.

Related research literature

There is quite a lot of related work that I have not fully examined yet. However, I have created a list of further work that I intend to look at, so I include this as a 'placeholder' for expansion of the above resource list.