Brahms is a multiagent simulation tool for modeling the activities of groups in different locations and the physical environment consisting of objects and documents, including especially computer systems. A Brahms model of work practice reveals circumstantial, interactional influences on how work actually gets done, especially how people involve each other in their work. In particular, a model of practice reveals how people accomplish a collaboration through multiple and alternative means of communication, such as meetings, computer tools, and written documents. Choices of what and how to communicate are dependent upon social beliefs and behaviors--what people know about each other's activities, intentions, and capabilities and their understanding of the norms of the group. As a result, Brahms models can help human-computer system designers to understand how tasks and information actually flow between people and machines, what work is required to synchronize individual contributions, and how tools hinder or help this process. In particular, workflow diagrams generated by Brahms are the emergent product of local interactions between agents and representational artifacts, not pre-ordained, end-to-end paths built in by a modeler.
We developed Brahms as a tool to support the design of work by illuminating how formal flow descriptions relate to the social systems of work; we accomplish this by incorporating multiple views--relating people, information, systems, and geography--in one tool. Applications of Brahms could also include system requirements analysis, instruction, implementing software agents, and a workbench for relating cognitive and social theories of human behavior.
This introduction provides an overview of our objectives, theoretical stance, and contribution to human-computer system design. Subsequent sections in this paper describe the relation of Brahms to situated cognition and workflow modeling, the methodology, provide examples, present results, and analyze broader implications.
The notion of "practice" is central to work systems design, which has its roots in the design of socio-technical systems, a method developed in the 1950s by Eric Trist and Fred Emery (1959, 1960). Socio-technical systems design sought to analyze the relationship of the social system and the technical system, such as manufacturing machinery, and then design a "socio-technical system" that leveraged the advantages of each. Work design (Ehn, 1989; Greenbaum and Kyng, 1991; Pasmore, 1994; Weisbord, 1987 (see Chapter 16)) extends this tradition by focusing on both the formal features of work (explicit, intentional) and the informal features of work (as it is actually carried out "in practice," analyzed with the use of ethnographic techniques).
The aim of analyzing both the formal and the informal work practices is two-fold: to understand what it takes to actually accomplish a business function in order to use those insights in design, and to ensure that new designs of work can be effectively implemented. Socio-technical and work systems design aim at producing both a productive workplace and a positive work environment, which fosters human development by providing dignity, meaning and community. By contrast, business process reengineering focuses on the structuring of key business process, eliminating duplication and unnecessary steps, formulating processes and procedures and using information technology extensively to improve work processing. Consequently, business process reengineering focuses more exclusively on tasks, does not take into account the informal nature of work, and does not hold as important the dignity of work or human development. In short, work design includes a focus on practice, while business process design exclusively focuses on process (Sachs, 1993).1
The methods of business process reengineering (BPR) and work system design differ considerably. BPR tends to be conducted by external consultants who analyze the flow of work products in terms of business functions, and not how product get from one place to another. Work design is more typically carried out by people who actually do the work (both workers and managers), and emphasizes not only what flows, but how and why work products manage to get from one place to another. For example, managers, office workers, crafts people, and researcher-facilitators collaborate to understand an existing work setting (such as new order processing for a telecommunications company) in order to develop a comprehensive design for the business organization, work process, computer tools, documents, facilities (such as seating arrangements), training, performance metrics and incentives, etc.
The two approaches differ in their view of how information technology should be used. In BPR, models of technical problem-solving, for example, tend to result in business process designs in which information technology is seen as an opportunity to "do the work" (e.g., to automate as much of the work as possible). In work systems design, information technology is seen in terms of augmenting and supporting human work practices. These outlooks have profoundly different consequences for the design of software systems.
We emphasize that in work design the design process attempts to treat everyday work as the result of a combination of interacting conceptual and physical influences and that practices will develop over time through learning and worker invention. Because work systems are not simply "implemented," but develop and grow through the learning of communities, workers and designers (such as software engineers) must collaborate in the design process (Greenbaum and Kyng, 1991). The differences in these two approaches reflect theoretical underpinnings about the nature of knowledge and learning (what does "knowledge of work" mean when an individual is part of the work, or not part of it all at? See discussion below on Situated Cognition).
Finding ways to effectively "see" process and practice at work has been a key challenge for us. We have developed Brahms because we think it provides a step forward in visualizing, analyzing, and thinking about the multiple dimensions of work that we think informs design. By understanding the distinction between process and practice, we realize that organizations function at many levels and in many ways simultaneously. Both process and practice exist, but they are not the same thing. It is therefore unrealistic to assume that one could design a process and expect the practice--the actual doing of the work--to follow flawlessly. At the same time, one should not expect to design practice and assume it will produce the process that the business needs. Brahms, as a tool to build integrated models, offers a way of seeing both process and practice.
A key finding of our work is that a representation of work process, such as work flow diagrams, can be derived from the result of a simulation of practice. Brahms has been developed as a tool to facilitate the representation and visualization of both process and practice. While it focuses primarily on practice, the simulated model generates a work flow which can be analyzed and discussed, and which is integrated with actual practice. This kind of model can leverage an integrated understanding of work flow by making visible both practice and process, and can close the distance between business process models and realistic implementation, which takes place in practice.
Models to support an integrated socio-technical view of work design
A model of practice is especially useful for revealing informal aspects of work. Conventionally, tools for modeling work produce either detailed descriptions of reasoning, as in cognitive task models (Newell, 1990), or descriptions of work product flows between job functions, as in business process models (Tyo, 1995). These modeling techniques cannot be used to easily or directly represent informal interactions that have a direct effect on the quality of work: how collaborative troubleshooting occurs, how learning occurs on the job, how people work on multiple orders at once, when people engage each other in each other's work, how people use communication media in practice (telephone, e-mail, face-to-face conversation, etc.), and so on. Such factors are circumstantial and conventional and cannot be strictly specified in advance (or controlled by management). Incorporating circumstantial factors in a model of work leads us to consider what people are actually doing, how these practices came about, and whether or not advantageous interactions are emerging. In contrast, models of technical problem solving (as in most cognitive models and expert systems) or formal policies and organization charts ("what should be") are of limited value for communicating to workers and managers alike what informal interactions are occurring and how computer systems, workplace layout, management attitudes, etc. facilitate or hinder such interactions.
More generally, with respect to the overall objectives of human-computer system design, Brahms provides a tool for engaging workers during the software design process, which can be a key aspect of work systems design. In contrast to business reengineering's use of information technology, work system design aims to understand how the computer system and the human system can be most productively integrated. As a model of how work actually occurs, Brahms helps designers understand the context in which computer tools are used (Weickert, 1995). For example, a model of practice reveals how information that is entered into a computer database is first acquired by reading a faxed form, by talking to the person in the next cubicle, or by looking up instructions in a manual. Thus, we approach human-computer interaction from a comprehensive, systemic perspective that seeks to relate how people interact with each other, kinds of representations in the environment, and physical layout of materials (Greenbaum and Kyng, 1991). As we will show, this comprehensive approach provides insights for understanding the human and social factors of software engineering, both in terms of content for representing work processes and in terms of methodology for facilitating the design process. Specifically, by providing a language for representing activities, Brahms improves upon empirical studies and user models that focus only on psychological processes. That is, Brahms models show not only what reasoning occurs, but how the information being used came to be available--based on where the reasoning is occurring, what other tools are being used, who is participating in the problem solving situation, and how (because of other circumstantial physical and social factors) they came to participate.
A cognitive model would of course emphasize that the knowledge of a person affects the quality of the work produced. But a task model of work does not explain how particular people became involved in solving particular problems. Work systems design emphasizes the crucial process by which people reconfigure their organization and tools to bring resources to bear on a given situation. Specifically, who is involved in a situation assessment will determine how the situation is framed (is it a craft problem? a software problem? a management problem? a problem with policy?) and what problem solving methods are applied. In this respect, modeling practice addresses the issues that are raised by the theme of "resource-bounded reasoning" in artificial intelligence research (Zilberstein, 1996) by broadening from issues of how reasoning is managed to how the social-interactional environment determines which agents are involved in reasoning at all about a given problem.
In summary, our approach to practice, work systems design, and model-building in Brahms brings together psychological, social, and physical environmental factors in a coherent manner. As a multi-agent simulation program, Brahms relates traditional engineering approaches to the study of people (e.g., task models) and knowledge-based approaches for representing processes qualitatively. Specifically, agents' behaviors and attention are modeled in Brahms using a rule-based, subsumption architecture (Brooks, 1991; Nakashima, Noda, and Hana, 1996). Behaviors are organized into and inherited from groups to which agents belong; groups include not only technical functions (such as "splicer"), but where people work ("1 World Trade Center people"), their temporary roles ("turf coordinator"), their background ("new hire from outside the company"), and the tools they use ("CIMAP database users"). Most importantly, we model located behaviors of people in time and view the rule-like constructs in the model as descriptions of what people do, not what they know per se.
Because Brahms' design is a combination of process modeling and situated cognition ideas, additional preliminary discussion is helpful to understand what we seek to include in Brahms models (hence why other tools are inadequate) and how our methods and objectives relate to the field of artificial intelligence more broadly. The following section uses an example to relate Brahms relates to situated cognition theories.
Obviously, a great deal more could be and has been said about these
topics (e.g., see Brown, Collins, and Duguid, 1988; Resnick, Levine, and
Teasley, 1991; Agre 1997; Clark, 1997); the point of this overview is that
situated cognition is not just a claim about "the context-dependent nature
of symbolic descriptions of human knowledge" (Menzies, in press). Instead,
criticisms of the "symbolic" approach to building an artificial intelligence
(e.g., see Steels and Brooks, 1995) are based on a host of more fundamental
claims about:
b) the coordination of speaking (verbal descriptions) with other modes of conception (e.g., gesture, imagery, verbal, auditory)
c) the transactional relation of descriptive models (any statements, maps, knowledge bases, policies, design, recipes, rules) to future human activity (i.e., why descriptions must be adapted to new situations),
d) the improvisatory, structural aspect of memory (i.e., reconstructed, self-organizing processes, not stored programs that are merely reapplied in new circumstances),
e) the interactive, behavioral aspect of performance (behavior is interactive means that action changes the person and the environment; action is not merely an input/output function),
f) the social-activity aspect of conception (i.e., the content of knowledge is not just technical, but organized by identity and the choreography of participation; intention is not concerned only with technical goals or tasks, but becoming a member of a community of practice; and these are broader, more pervasive influences on behavior than technical problem solving viewed narrowly as reasoning).
In short, a situated cognition perspective suggested that we improve business process models by representing task performance within the context of social-interactional behaviors. In this way, we are drawn to consider how psychological, social, and physical factors interact to affect what is normally taken for granted by problem solving models, namely how people make observations (gaining new information), how people prioritize their tasks while juggling multiple responsibilities, and how they decide whether and by what means to communicate information. Simply put, a straight-forward knowledge-based approach to modeling an office would suggest that people are literally following their job function descriptions, and hence doing the same thing at 9 AM as at 4:45 PM, that they are either knowledge clones or in need of training, that they are never working on other people's problems, that they do not answer a phone on someone else's desk, etc.
Models can be built at different levels to describe different phenomena. But the constructs in a modeling language often reflect assumptions about what a model should include. Specifically, the original paradigm of expert systems (Hayes-Roth, Waterman, and Lenat, 1983) assumes that knowledge is exclusively technical, objective, and used for reasoning. In the original formulation, all human action is viewed as being problem solving in an unspecified environment. At issue is not so much the modeling apparatus (though indeed more is needed to model human attention and physical interaction), but the concepts used for describing human behavior. Specifically, expert systems did not represent the conceptual and physical context in which reasoning occurs, which we call work practice.
The analysis of work practice originated in the social sciences, particularly
anthropology and the ecological approach to sociology called situated
action (Mills, 1940). The application to work systems design is perhaps
most evident in socio-technical systems approaches in the 1950s (Emery
and Trist, 1960). We were specifically most influenced by the research
on situated action collected in Design at Work (Greenbaum and Kyng,
1991). Simply put, the situated action perspective claims that the quality
of work (including the quality of software design) depends on circumstantial,
physical interaction with materials and who participates:
A critique of this diagram from the perspective of situated action would inquire why order processing occurs this way and how it might be improved. Perhaps surprisingly, the figure leaves out what a problem solving (cognitive task) model would typically focus upon. For example, what information does the engineer read from the order form and what deductions are required in order to assign the circuit loop? This particular model leaves out how orders are planned and assigned, multitasking (the fact that a rep or engineer works on several jobs at once before completing them) and how people interrupt and resume their work (e.g., use of notes and stacks). A cognitive model of the same business process might consider some of these factors, but would leave out how people come to be synchronized in a phone conversation, how an engineer might help a representative do his job, and broader considerations of how a representative actually spends her day. In particular, because interpreting and executing orders can be problematic in unexpected ways, people need to improvise in ways that work system designers might not have anticipated:
To analyze the example more precisely, consider what the various branches
and joins mean:
3. NYNEX merged with Bell Atlantic in 1997, after
most of the work reported here was completed.
At this point in our conversation with our collaborator, we generalized the apparent social interaction and deduced that the switching electronic technicians (SETs) and their supervisor, the "turf coordinator," have the same practice in the central office. But it turned out that the SETs find out about new orders and their daily assignments after the coffee meeting by examining the schedule previously posted on-line by the turf coordinator. Why doesn't this group have the same practice as the technicians out in the field? Why do they prefer to use an indirect approach involving a computer system? In fact, the turf coordinator is another SET on temporary assignment as a supervisor. One SET would not ordinarily tell another what to do. Thus, a proper "social" way of coordinating their work is to make and communicate assignments indirectly, using a computer system.
The example illustrates how understanding practice, including whether and when people will use computer tools, requires relating multiple perspectives. Adopting an idealized, optimizing view, one might have suggested that the coffee meeting was a distraction and that all groups should handle assignments through an on-line system. After all, an on-line system would be available from many locations, provides a written record, is easily changed, etc. But this would ignore the other aspects of planning and communication about past and future orders that occurs in the coffee meeting. Eliminating the coffee meeting would eliminate how information is informally shared and people are learning about other jobs, methods, and each other. On the other hand, one must not adopt an idealized view about social interactions either. Concluding that all planning should occur during face-to-face interaction, such as at a coffee meeting, would ignore how a computer tool can provide a resource for handling a key issue of social identity, namely that in some groups it is not permissible for peers to tell each other what to do.
Thus, we have found in this simple coffee meeting an example of how two groups accomplish the same "task" of order assignment in strikingly different ways, based on the constraints and opportunities afforded by tools and social relations. The example illustrates how the turf coordinator's identity with respect to the SET community of practice makes a computer solution more tenable, although it may have other disadvantages over a face-to-face discussion. How these groups accomplish their work cannot be easily predicted either from a technical or a social perspective alone. The organization, processes, and tools cannot be effectively changed without a systemic view of diverse, interacting social, physical, and psychological influences. The example illustrates how redesigning a "human-computer system" is facilitated by relating these different views.
In view of these considerations, we deliberately chose to include multiple, complementary views in Brahms, rather than the single-dimensional perspective of the work flow model (as shown in Figure 1). Specifically, people performing the same task may do so in different ways based on the group to which they belong, not just for historical reasons, but because of their interpersonal relations to each other. Logically speaking, there is no reason why SETs can't tell each other what to do. But human actions are not just technical, functional performances, but indirectly, even when we might prefer otherwise, are conceived as constructing and maintaining an identity relative to other people. The social effects of actions necessarily constrain choices people make about how to accomplish their formal job functions.
A Brahms model does not necessarily describe the intricate details of reasoning or calculation, but instead models the social-physical context in which reasoning occurs, especially how observations are made based on what activities the person is already doing (such that a problem situation may be detected). A Brahms model does not describe only what people are supposed to accomplish (e.g., functional transformations of materials). Instead, such diagrams are derived from the results of the interactions that occur; that is workflows are emergent during the simulation.
To be sure, any qualitative model will specify conditions and consequences in a rule-like way. We still build into Brahms simplified descriptions, which are idealizations. But first, they are descriptions of spatially and temporally located behavior, and second they are smaller pieces, such as the many ways in which information and work product paths may be formed. That is, the model provides for circumstantial interactions to occur and to have an effect. In this sense, because the flows are pieced together dynamically as the model runs, the flows are emergent. The modeler is still of course concerned with overall completeness and connectivity of the model, but the relevant factors must be specified in terms of the informal, circumstantial influences by which a practice develops. Due to interactions between agents, physical objects, and locations, the paths followed and the quality of results will be contingent on what resources are actually available, who actually participates, etc.
As an example, consider how phone conversations are modeled in different simulation tools. In Sparks, one can leave out entirely that a phone call is even made, focusing on the information conveyed. The problem of synchronization (having two people at the same time in a conversation) is handled stocastically, by modeling the time required for information transfer as a statistical function (e.g., 80% of the time a conversation occurs within an hour, but 20% of the time it will take a full day). In another multiagent simulation such as Virtual Design Team (Levitt, Jin, Oralkan., Kunz, and Christiansen, 1995), a phone conversation is modeled as a message arriving in an in-box, which disappears if it isn't read within a minute. How the caller knows that the message was not received is not considered. In Brahms, phone calls are modeled in considerably more detail including phone numbers, busy signals, hearing a phone ringing, deciding that the conversation is over, etc. Thus, whether the conversation occurs is not built in as a statistic, but is the emergent result of where the agents and phones are located, what people are doing when a phone rings, etc. In contrast, a model such as ITHINK5 is also dynamic, but generally does not represent individual agents, phones, locations, etc. Rather, ITHINK represents aggregate behavior of a group, representing not particular actions at particular moments by particular agents, but the cumulative influences of an organization, leading to the total number of orders processed, the total number of backlogged orders, etc.
Finally, in designing Brahms, we concluded that we should be flexible in how we create and use models. For example, one can begin modeling by pretending that one agent does all the work (a useful heuristic for identifying what tasks are essential), or model multiple groups with one agent each, or place all the agents in one location or move them all apart, etc. We view model building as inherently an experimental endeavor, aimed at gaining insights, and only secondarily, if at all, in developing something "complete" or making accurate predictions. Our predominant interest is to create models in order, first, to engage workers in work systems design conversations and, second, as a tool for raising good questions (often from the obvious deficiencies of the model, as in the example of the coffee meeting). On the other hand, models may be evaluated in terms of statistics of flows generated during a simulation run (e.g., number of orders processed/day/person or group, backlogs, number and kinds of errors generated/detected/resolved, touch-time/job). But the presentational value of the computer model--like a theatrical play--is our top-most concern, using Brahms as a way of representing complex human-computer systems in a way that helps people reflect on their practices and how to improve them.
5. Trademark, High Performance Systems.
The behaviors we model in Brahms include reasoning, but more generally include ways of coordinating with other jobs/tasks, stepping outside literal responsibility to offer assistance, and a host of general activities such as "reading my e-mail," "following the service technician to the customer's site in my truck," and "having coffee with my supervisor." A model of activities includes traditional components of cognitive modeling (specifically, the vocabulary of "belief" and "inference"), Traditional problem-solving models are embedded in this larger simulation of activities, not replaced. Similarly, as stated, business process workflow diagrams are derived from a Brahms simulation. They are not replaced by Brahms, but rather generated in a different way.
Some researchers view every model that includes inferential processes as being a mechanism of how the brain works or how cognition occurs in the individual. But aside from coarsely mimicking behaviors of people, nothing in Brahms requires such a commitment. Brahms is not designed to replace people or replicate their intelligence, per se, no more than a play in a book replaces people and their experiences. A Brahms model, like a play, is just a kind of map, not the territory.
From the perspective of work systems design, Brahms models should be evocative and have a basis in reality. But to be useful, a Brahms model need not replicate or predict any particular group's behavior. Keeping in mind that the audience of workers often consists of people without advanced university degrees, a dramatical presentation is a way of making complex theories about human interaction accessible. A model may have rhetorical effect, bringing about an emotional response -- "No, it's not like that! We do work doing the coffee meeting, we don't just sit around and talk," or maybe, "Yes, that's exactly what is going on here at Pearl Street--can we show this to people at the World Trade Center so they can see what things are like over here?" On the other hand, one might devote sufficient effort to portions of the model so it will have enough veracity to allow testing hypotheses about various redesigns and making relative numerical comparisons (e.g., time and cost for processing orders).
In summary, Brahms's architecture is partly a reconception of the meaning and use of existing modeling techniques. In this respect, a Brahms model could be viewed as a group of interacting knowledge-based systems in a simulated environment. But new architectural features were required for modeling how people conceive of activity and how attention and inference are contextually scoped by parallel-ongoing activities (described below).
These systems have the following characteristics:
In general, the notion of "social" in simulations of organizations is quite impoverished. For example, one tool models social behaviors in terms of "decreasing information-processing capability" (e.g., emotional responses) and deceit (Carley, Park, and Prietula, 1993, p. 3). Malone and Crowston (1991) define coordination as the "act of working together" but apply the conventional management perspective. Their models do not capture practice, but instead descriptively abstract coordination in terms of bidding and communicating interdependencies. Indeed, they view coordination as "the additional information processing performed when multiple, connected actors pursue goals that a single actor pursuing the same goals would not perform." That is, coordination is the overhead required when you can't do everything yourself!
The next subsection summarizes the specification that distinguishes Brahms from other multiagent simulations.
Workframes are organized hierarchically into activities (e.g., Morning Planning with STs is an activity with one workframe, shown in Figure 4). Actions in a workframe may be simple (just indicating a name, duration, and priority) or composite (another activity). Figure 4 indicates that during the Morning Planning activity, the Field Supervisor engages in face-to-face conversation with service technicians. For each service tech and order that needs to be discussed, the Field Supervisor will tell the service tech that he or she is assigned to an order.
Simple actions also include movement to another location. Consequences
and actions are ordered and interleaved. Detectables may be indicated as
"impasses" that interrupt the workframe or as "end conditions" that end
the workframe or its encompassing activity. The detectable in Figure 3
simply observes facts in the environment.
| WORKFRAME: Morning Coffee Meeting
VARIABLES:
PRE-CONDITIONS: Agent knows: The hour of the clock is >= 8
SIMPLE ACTION: Notice who's in the room
(5 minutes)
COMPOSITE ACTION: Morning Planning with STs |
| WORKFRAME: Face to Face Conversation with
Service Technician about Assignment
VARIABLES:
PRE-CONDITIONS:
SIMPLE ACTION: Talk (5 minutes)
CONSEQUENCES:
|
Workframes are inherited by agents from all groups to which they belong; groups may belong to other groups (Figure 5). Priorities allow workframes to interrupt each other or carry out specific aspects of a more general protocol. For example, workframes at the "all groups" (top) level specify how to use a telephone and have face-to-face conversations; these have intermediate priority. Workframes that trigger conversations are most specific and have the lowest priority. Workframes that specify what to say during certain kinds of conversations have the highest priority. By this simple scheme, it is possible for one agent to initiate a conversation and for the responder to "remember" something he wanted to tell the first agent when he called; thus a give and take may ensue.
Thoughtframes model agent reasoning about implications of beliefs, leading to changes in what they do next (thus a distinction is draw between "action rules" and "thinking rules") Thoughtframes take no time.
Changes to beliefs may occur by virtue of: broadcast (e.g., speaking outloud), transfer from agent (telling or asking), transfer from object (e.g., reading a database or a fax), detectables, and consequences.
Activities are spatially-dependent:
Facts are an eagle-eye view-from-nowhere--the outsider's view of the simulation, for example, the state of telephones, location of agents, etc. Detectables specify what facts an agent might detect during the action of a workframe. Beliefs are propositions agents believe about objects (state of the world) or other agents.
A communication may involve asking or telling. A communication may be from an agent or object to a specific agent or object, a group of agents, a class of objects, or may be broadcast. For example, a factframe for the fax object broadcasts to every agent within proximity that a fax has arrived. An agent can only communicate what he or she believes.
Brahms currently models geography in a rudimentary way, consisting of regions, buildings, and their connections. Duration of movement is simply proportional to distance; for convenience movement between non-connected locations takes no time. We believe that our objective of making models accessible will only be realized when graphics are incorporated of the caliber one commonly finds in games on personal computers.
In general, descriptions of activities are associated with groups. In
practice, there may only be one member of a group in a given workplace
(e.g., one "customer representative" at the customer's site) or roles may
be highly differentiated (e.g., the role of the "turf coordinator"). Depending
on the purpose for building the model, models may represent:
The front-end model was begun in 1996 and was intended to help managers and software engineers understand why orders rejected by an on-line system were generated and resolved. Workers collaborating with us found the modeling process to be valuable. Specifically, the focus on what people actually did when they processed orders revealed that the program's rejects, called "errors" heretofore, were not necessarily human mistakes, but just orders that the software could not process. Our systemic approach led backwards from this downstream processing (in another part of Manhattan) back to the BNA Center and to its peer at World Trade Center. The actual causes of computer system problems were found to be not just typos, but primarily an inability to specify certain kinds of jobs using existing forms. Assumptions built into software also ignored pragmatic issues, such as the need to start order processing before getting customer credit approval. We also showed that the error rate dropped not because of "fewer mistakes," but because the work group shifted to a fully manual process that worked around the limitations of the order-processing that was supposed to partially automate circuit design. This analysis raised each group's awareness of the other's work and gave the software engineer responsible for the downstream system a better appreciation of the difficulties the BNA Center encountered and appropriately handled.
Finally, in the back-end turf coordinator model we had treated all members of a work group as being clones, such that all SETs behave identically. This choice followed from the social science preference not to view people as individuals, but to focus on trends and commonalities. In building this second model we questioned this simplification and asked how the individuals in a group differed from one another. Our analysis of BNA engineers revealed a kind of knowledge variability that was unexpected--people were of course not clones, but differences were not errors, either. We found alternative methods being used for the same task (such as verifying that a given circuit was available for assignment); we conjecture that such differences are a vital source for learning. Possibly the lack of consideration of individual differences heretofore by the social scientists led them to emphasize cross-functional learning (one group learning to do another's tasks), rather than learning among people with similar responsibilities. Furthermore, the idea of legitimate knowledge variability contrasts with the typical corporate view that all variability in job performance is non-optimal or based on misconceptions or lack of knowledge (hence, one goal of corporate training is standardization). Instead, knowledge variability may be a source of robustness, allowing adaptability when the environment changes (like variability in a biological population).
Activity-based modeling in Brahms led us to ask new kinds of questions
about the BNA Center:
To understand the advantages of activity-based modeling for developing
a software agent application, such as an "intelligent agent," consider
the example shown in Figure 6.
Given information about the location of different service technicians and knowing that the turf coordinator had last read the order database that morning and wouldn't review it again until the following morning, a program might offer advice such as, "TC Allen, ST Aronson just completed the job down on Wall Street and is now available; perhaps you want to have her go over to Broadway to handle the Teleport job?" More broadly, activity-based modeling provides a new way of modeling users (such as the turf coordinator), which includes not only what tasks they do and the information they use, plus some of the deductions they might make, but also where and when they do such reasoning, where they might be found at a particular moment, who might know where they are located, what interests them at a particular time of day, etc. Thus a Brahms user model would combine cognitive and social-interactional considerations. Similarly, such a model would be potentially more useful for instruction than a typical knowledge-based model because it would help a student understand the practices by which different tools are related, who is typically available for providing help, what kind of assistance may be sought, and so on. Finally, one could use Brahms for implementing a software agent itself, locating the agent in the modeled social-interactional context, making explicit what external resources are available to it, how it should behave when participating in different groups, what it should do at different times of the day, and so on. In summary, activity-based modeling provides a way to inform computer systems of everyday practice of the people they are serving, and thus, in a very limited way, integrate them into human communities.
The demands of modeling work practice almost turn inside out the conventional view of knowledge engineering. Context is not just something in the environment ("data"), but partly conceptual and partly about other people. A social system is not just an organization, but a choreography of interaction, a set of practices for doing things in certain places at certain times. Knowledge is not just technical, but is about the group--social knowledge. What people know and do is organized around their roles as social actors; they are not plug-compatible problem solvers, but people with different interests and different ways of working together. A kind of task might be accomplished pragmatically in many ways, without the variability being a matter of misconceptions or missing knowledge. Expertise includes knowing what other people know, how to get help, who is trustworthy and who is diplomatic, and how to team a patient, careful worker with an imaginative explorer.
In modeling work practice, standard AI issues of scheduling, planning, and information processing are not omitted, but are made problematic: How does a supervisor remember what everyone is doing during the day? How do members of a team at different locations coordinate their work day? How does informal, circumstantial encounters (such as conversations in a hallway) help align the expectations and understanding of the group about group's capabilities, how busy they are, and what they are becoming? To say that such issues are ignored in expert systems is an understatement. Indeed, social knowledge and interaction are ignored in most software engineering tools for designing computer systems. But how can we design computer tools if we don't know what people need? That's the ultimate value of modeling work practice.
This much said, we hasten to point out that Brahms' models of practice
are exceedingly limited. One value of developing Brahms has been to highlight
for us the concerns that are typically raised in work system design that
are not easily modeled in the current Brahms language. In particular, Brahms
models do not represent the following:
Similarly, we believe that cognitive methods for modeling could be embedded in Brahms, with a resulting model that is more comprehensive and insightful about how learning actually occurs. This is of particular interest for analyzing learning on the job (Clancey, 1993).
Other aspects omitted involve historical, cumulative effects, which are considered by a variety of disciplines ranging from sociology to human factors. On the one hand, the sociological theories would benefit from being forced to relate to contingent psychological and physical factors; the human factors theories would need to relate to a social analysis of norms and values.
Modeling reconceptualization and juggling of activities requires better neuropsychological theories of memory, attention, and perception. The use of the subsumption architecture in Brahms for modeling conceptual hierarchies (of activities) may be usefully extended by combination with a selectionist or connectionist model of multiple constraint satisfaction (e.g., Mitchell, 1993).
Finally, broader considerations such as life away from work may be directly incorporated by extending the models we have developed to include families, weekend activities, and so on. This would be especially useful if one wanted to analyze and represent what workers learn about each other through informal activities outside of work. As for all of the considerations listed here and those already included in Brahms models, the relevance of any level of detail depends on the kind of work system being designed and pragmatics of the design process.
A work systems design effort presupposes what engineers call a "design stance." That is, rather than simply studying or describing a workplace, which has been the conventional approach in anthropology, we intend to use Brahms to change how work gets done. This means that we must have theories of how different kinds of designs produce different results, otherwise we would not be able to suggest alternative designs or reason about trade-offs. Of course, the very idea of participatory design means that facilitators (e.g., anthropologists and model builders) need not (and could not) develop an optimal design and deliver it to clients. Instead, Brahms is conceived as a tool by which researcher-facilitators in collaboration with workers and management will bring about change incrementally and iteratively. Nevertheless, using Brahms as a design tool presupposes that facilitators have some understanding of how interactions between people, technology, and organizations relate to measurable changes in work quality. In contrast with traditional ethnographic descriptions of cultures, observations and models of work practice are fundamentally evaluative.
For example, to show the benefits of alternative work system designs,
we might model how beliefs people have about members of their local work
group (e.g., who is doing what with what results) change because of the
design of work processes and the workplace layout. For instance, co-location
(working in the same office) facilitates face-to-face conversations, in
which beliefs about the group are communicated (e.g., service technicians
during a coffee meeting might learn about how a given technique produced
an expensive failure at a customer site; Orr, 1995). In contrast to communications
via on-line systems, face-to-face conversations tend to articulate beliefs
about skills and experience, not just beliefs about the status of different
orders. We can show these relations causally:
(e.g., assigning jobs, getting help, dividing up the work, teaming people)
A simple way of stating our incipient understanding of work systems design is given by this causal relation:
A given design (configuration of people, technology, and organization) causes certain everyday interactions in activities (both positive and negative, including antagonistic relationships and customer-focused conversations). These interactions are manifest in certain properties of the workflow (e.g., bottlenecks, early detection of errors, multiple error-prone hand-offs). Finally, these workflow properties are manifest in the business results management uses to assess productivity (e.g., average number of days to process an order).
As an example, consider the logic behind part of the redesign of the
back-end order provisioning process (Figure 7).
Work System Design: co-location, turf coordinator, turf teams
|
The design incorporated elements such as "co-location" and a turf coordinator. This combination of people, technology, and organization changed the interactions occurring during work activities to improve communication and promote collaborative planning. These changes in turn affected the workflow: Bridging front and back ends of order processing and reduced hand-offs released work early, provided a single dispatch to the customer, made scheduling of jobs more efficient, etc. This in turn lowered the error rate and shortened the time required to turn up the circuit.
This example illustrates that reasoning about designs requires adopting different perspectives and arguing about how they are causally related. The advantage of the ethnographic perspective is that it introduces the level of "activity interactions," complementing the systems analysis perspective on workflow and business results. A whole range of social conception is thus introduced in terms of location of people, conversations, informal relationships, etc. that better explains how responsibility and attention arise and are sustained in a group. (Indeed, the advantage is to understand motivation and concern as conceived and developed with respect to a group.)
To summarize, the key perspectives that we relate when comparing and
promoting change in work practice are:
In general, ethnographic observation is required to identify situations that illustrate abstract design guidelines. For example, another change brought about in the back-end redesign is that pressure and rancor were replaced by "shared end-to-end responsibility." How are "pressure" and "rancor" manifest in activities? In the workflow? How is "shared end-to-end responsibility" manifest in activities and the workflow? To make such abstract, but central concepts explicit, anecdotes are converted into cases, whose preconditions and consequences are modeled, just as in traditional knowledge engineering. In effect, expressing such conventional social concepts in a formal model brings to social science a change in practice that complements the change in business process modeling in adopting the social perspective.
In summary, a design stance requires a vocabulary for describing practice (e.g., functional hand-off, alerting, discussing context, lack of contact, developed relationships, rancor, cryptic notes, responsibility) and a theory of how changes in jobs and technology will transform one work practice into another. This theory is necessarily quite detailed, for it must explain how the practice will develop (e.g., how do we get from "lack of contact between people" to "developed relationships"?) and how this changed practice will affect workflow and hence business results (e.g., how do "developed relationships" improve the quality of the work?). This means that we must organize ethnographic characterizations to portray shifts from "dysfunctional" to "desired" practice and make explicit the causal relations to the analysis provided by other system designers who are focusing on workflow, quality, and productivity. In practice, one might develop an initial model, knowing only that certain scenarios and behaviors are important, but without having an explicit theory of how they came about, what effects they are having on quality, and how to change them. Indeed, we usually build models to get such an understanding.
To conclude, Brahms is a tool for bringing a mixture of observational methods (ethnography) and insights about how work occurs to practical application in work systems design. As described, there are many other implications, ranging from instruction to the development of software agents. We believe the most important technical result from our work is the discovery that a workflow diagram can be generated from the local interactions between human agents, media (e.g., fax machine, computer terminal, phone) and representational artifacts (e.g., faxes, databases, voice mail). By building in fewer assumptions about information and workflow, essentially modeling how connections and reinterpretations occur in practice, Brahms models are potentially more useful for human-computer systems design than task and business process models alone. In particular, the idea of knowledge variability illustrates how synthesizing social and cognitive theories may be productive. Finally, the clarified distinction between tasks and activities provides a framework that changes how we view cognitive models, and suggests a broader foundation for studying and formalizing expertise.
This particular module was chosen to test an electronic medical record (EMR) system for a large HMO. An ethnographic pre-study of several months was followed by a week of videotaping routine work in the outpatient setting. Our overall interest was in promoting participatory design of the EMR, plus understanding the difficulties and benefits of transitioning from a paper chart to an EMR.
We had available the following information for constructing this model:
The most important representational artifact at this work site is the chart. There are also two computer systems to be modeled, one for patient scheduling and other for patient data. The chart is modeled in terms of sections and what information is written in each section (e.g., Current progress note, new orders, lab and pathology results, back page). Other forms used by the caregivers and patients include: Pathology lab order, labels, and consent forms. Details about content and use of forms and computer systems are included in the model only as they arise in the description of activities. Other representational artifacts include messages from the appointment center in the clinic, a prescription refill (such as a fax from a pharmacy), phone messages from patients, and notes from physicians to nurses to call patients. Each of these is represented insofar as it is read, written, or hand-carried during a patient visit.
Early on, one describes the "geography" of activities, including the
following locations:
The primary activity of the module is the patient visit. When not engaged in this activity, participants are engaged in the activity of "being in the module" and what they do may be omitted.
In scoping the model, we decided to distinguish the roles of the nurses and the capabilities of the providers (MD vs. physician assistant). We also wanted to model the mentoring activity between a physician (Dr. A) and a PA (Mr. D) (which occurs as a scheduled meeting).
The primary actors during the patient visit are: the patient, the registration clerk, the nurse (assigned to a provider for the day), the provider (MD or PA), and assisting nurse.
The main activities of the patient visit are, in order of occurrence
(included optional activities are indicated by brackets):
The flow of people is as follows:
Similarly, we are aware of different ways nurses work together. For example, one nurse (RN or LVN) works with two providers on Tuesdays and Thursdays. Two or three other nurses (possibly including a TCA) work with three or four providers the other days of the week. The TCA prefers to work with Dr. A (the physician-in-charge), etc. Such interactional patterns illustrate part of what is meant by practice, as opposed to policy or procedures. We describe what people do together, how they interact with each other and objects in the environment.
In modeling the clinic, we focus on the nurses' first rule, "Keep the rooms filled." This orientations gives us the nurses' point of view, so we can expect what they might notice and helps us detect gaps in our model. This is an example of a choreography constraint, which describes what the group is trying to accomplish in their routine interactions. Accordingly, if a provider is absent on a given day, his exam rooms will be used for placing the third patient of another provider.
As an example, here is a sketch of the nurse's activity of coordinating patient visits, which occurs as a subactivity of "being a nurse in the module."8 This sketch would be filled in by additional workframes that specify what the nurse does while waiting, etc. Or we could decide that because this model focuses on the patient, we will omit such details (corresponding to actors who leave the stage and return later).
COORDINATING PATIENT VISITS
THE PRE-EXAM PATIENT INTERVIEW
=> In the waiting room
BRING TOGETHER PATIENT AND PROVIDER
=> In the examination room ($X)
Wait for the patient to return
Brahms exists as a prototype developed in G2 (trademark Gensym Corporation) on a SUN workstation; a graphic interface implemented in Visual Basic displays simulation outcomes on a PC. Current work includes comparative studies of tools and exploratory use on client projects. The name "Brahms" stands for "Business Redesign Agent-based Holistic Modeling System," but it applies to any human activities.
The model of the health care module was developed with the cooperation and financial support of Southern California Kaiser Permanente. Judith Gregory provided the ethnographic data. Charlotte Linde (IRL) and Jean Gilbert (KP) were project managers. Support for the development of Brahms has been provided by NYNEX Science and Technology, Inc.
Brooks, R.A. (1991). Intelligence without representation. Artificial Intelligence. 47, pp. 139-159: Elsevier Science Publishers, B.V.
Brown, J. S., Collins, A., and Duguid, P. (1988). Situated cognition and the culture of learning. IRL Report No. 88-0008. Shorter version appears in Educational Researcher, 18(1), February, 1989.
Burstein, M.H., Ferguson, W., and Abrett, G. (1993). A Simulation Development Tool for Evaluating Coordination Strategies in Organizations. AAAI Workshop on AI & Theories of Groups & Organizations: Conceptual & Empirical Research,. pp. 24-28.
Carley, D. Park, and M. Prietula. (1993). Agent honesty, Cooperation and Benevolence in an Artificial Organization. AAAI Workshop on AI & Theories of Groups & Organizations: Conceptual & Empirical Research. pp. 1-7.
Clancey, W. J. (1992). Model construction operators. Artificial Intelligence, 53(1): 1-124.
Clancey, W. J. (1993). A situated cognition perspective on learning on demand in Proceedings of the Fifteenth Annual Conference of the Cognitive Science Society. pp. 181-3. Boulder: Lawrence Erlbaum Associates.
Clancey, W. J. (1997a). The conceptual nature of knowledge, situations, and activity. In P. Feltovich, K. Ford and R. Hoffman (eds.), Expertise in Context, pps. 247-291. Cambridge, MA: The MIT Press.
Clancey, W.J. (1997b). Situated Cognition; On Human Knowledge and Computer Representations. Cambridge, UK: Cambridge university Press.
Clark, A. (1997). Being There: Putting Brain, Body, and World Together Again. Cambridge: The MIT Press.
Cohen, P. R., Greenberg, M. L., Hart, D. M., & Howe, A. E. (1989). Trial by fire: Understanding the design requirements for agents in complex environments. AI Magazine 10(3), 34-48.
Corcoran, E. (1992). Building networks: New York Telephone rethinks how to regain lost customers. Scientific American, November, pp. 119-120.
Davenport, T.H. (1995). The fad that forgot people. Fast Company. November.
Dourish, P. Holmes, J., MacLean, A., Marqvardsen, P., Zbyslaw, A. (1996). Freeflow: Mediating between representation and action in workflow systems. Proceedings of Computer Supported Collaborative Work, pp. 190-8. Cambridge MA: ACM Press.
Ehn, P. (1989). Work-oriented design of computer artifacts. (2nd Edition). Hillsdale, NJ. Lawrence Erlbaum Associates.
Emery, F. E. (1959). The Emergence of a New Paradign of Work. Canberra, Australia: Australian Naional University, Centre for Continuing Education.
Emery, F. E., Trist, E.L.(1960). "Socio-Technical Systems." In C.W. Churchman and others (eds.), Management Sciences, Models and Techniques. London: Pergamon.
Fafchamps, D. (1991). Ethnographic workflow analysis: specifications for design. In Human Aspects in Computing, Design, and Use of Interactive Systems and Work with Terminals. Proceedings of the Fourth International Conference on Human-Computer Interaction (Amsterdam, 1991), H.-J.Bullinger, Ed., Elsevier, pp. 709-15.
Gilbert, N. and Doran, J. (1993). Simulating Societies: The computer simulation of social phenomena. UCL Press.
Greenbaum, J., Kyng, M. (Eds). (1991). Design at work: Cooperative design of computer systems. Hillsdale, NJ. Lawrence Erlbaum Associates.
Gustavsson, R. (1993). Societies of Computation: A framework. AAAI '93 Workshop on AI & Theories of Groups & Organizations: Conceptual & Empirical Research. pp. 96-102.
Hayes-Roth, B., Brownston, L., and Sincoff, E. (1995). Directed improvisation by computer characters. Proceedings of the International Joint Conference on Artificial Intelligence.
Hayes-Roth, F., Waterman, D., and Lenat, D. (eds.). (1983). Building Expert Systems. New York: Addison-Wesley.
Hutchins, E. (1995). Cognition in the Wild. Cambridge: MIT Press.
Joosten, S.M.M. and S. Brinkkemper. (1996). Fundamental Concepts for Workflow Automation in Practice. In S. Wrycza and J. Zupancic (Eds.), Proceedings of the Fifth International Conference Information Systems Development (ISD'96): Methods and Tools, Theory and Practice. pp. 311-322, ISBN 83-86230-18-5. Gdansk, Poland.
Levitt, R. E., Jin, Y., Oralkan, G.A., Kunz, J.C., and Christiansen, T.R. (1995). Computational enterprise models: Toward analysis tools for designing organizations. CIFE Working Paper, Stanford University, Department of Civil Engineering. February.
Malone, T.W. and Crowston, K. (1991). Toward an interdisciplinary theory of coordination. Technical Report CCS TR# 120, Center for Coordination Science, MIT.
Menzies, T., (in press). Is knowledge maintenance an adequate response to the challenge of situated cognition for symbolic knowledge-bases systems? IJHCS, Special issue on Situated Cognition. pps. ???
Mills, C. W. (1940). Situated actions and vocabularies of motive. American Sociological Review, 5: 904-913.
Mitchell, M. (1993). Analogy-Making as Perception: A Computer Model. Cambridge: Bradford Books.
Nakashima, H., Noda, I., and Handa, K. (1996). Organic programming language GAEA for multi-agents. Proceedings of the Second International Conference on Multi-Agent Systems. December 10-13, 1996, Kyoto, Japan. pps 236-243. Menlo Park: AAAI Press.
Nagel, T. (1986). The View from Nowhere. New York: Oxford University Press.
Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press.
Nonaka, I., H. Takeuchi. (1995). The Knowledge-Creating Company. New York: Oxford University Press.
Orr, J. E. (1995). Ethnography and organizational learning: In pursuit of learning at work. In S. Bagnara, C. Zuccermaglio and S. Stucky (eds.), Organizational Learning and Technological Change, pp. 47-60. Berlin: Springer.
Pasmore, W.A. (1994) Creating Strategic Change: Designing the Fliexible, High Performing Organization. New York: Wiley.
Resnick, L. B., Levine, J. M., and Teasley, S. D. (editors). (1991). Perspectives on Socially Shared Cognition. Washington, D.C.: American Psychological Association.
Sachs, P. (1993) "Shadows in the Soup: Conceptions of Work and the Nature of Evidence" in Newsletter of the Laboratory of Comparative Human Cognition, Fall ISSUE
Sachs, P. (1995). Transforming Work: Collaboration, Learning, and Design. Communications of the ACM, Vol. 38, No.9, pp.36-44.
Salgame, R. R., Becker, S.G., and Yu, D. H. (19??). SPARKS: A knowledge-based process modeling and simulation system. Proceedings??? Pages??? Publisher???
Schön, D.A. (1983). The Reflective Practitioner; How Professionals Think in Action. Basic Books, Inc.
Steels, L. and Brooks, R. (Eds.). (1995). The "Artificial Life" Route to "Artificial Intelligence": Building Situated Embodied Agents. Hillsdale, NJ: Lawrence Erlbaum Associates.
Tambe, M, Johnson, W.L., Jones, R.M., Laird, J.E., Rosenbloom, P.S., and Schwamb, K. (1995). Intelligent Agents for Interactive Simulation Environments. AI Magazine, 16(1):15-39,Spring.
Tokoro, M. (Ed.) (1996). Proceedings of the Second International Conference on Multi-Agent Systems. December 10-13, 1996, Kyoto, Japan. Menlo Park: AAAI Press.
Tyo, J. (1995). Simulation modeling tools. Information Week, 60-67. July 10, 1995.
Weickert, K., (1995). Design under Uncertainty: Engaging the Context of Use in the Design of Expert Systems. Dissertation in Information and Computer Science, University of California, Irvine.
Weisbord, M.R. (1987).Productive Work Places: Organizing, and managing for dignity, meaning, and community. Jossey-Bass Inc, Publishers,CA.
Wenger, E. (1997) Communities of practice; Learning, meaning, and identity. New York: Cambridge University Press.
Whalen, J. and Vinkhuyzen, E. (forthcoming). Expert systems in (inter)action: Diagnosing document machine problems over the telephone. In Christian Heath, John Hindmarsh, and Paul Luff (eds.), Workplace Studies: Recovering Work Practice and Informing Systems Design. New York: Cambridge University Press.
Wynn, E. (1991). "Taking practice seriously." In J. Greenbaum and M. Kyng (eds.), Design at work: Cooperative design of computer systems, pp. 45-64. Hillsdale, NJ: Lawrence Erlbaum Associates.
Zilberstein, S. (1996). Resource-bounded reasoning in intelligent systems. ACM Computing Surveys, Vol. 28, No. 4 (Dec. 1996), Pages 15-16.
Zuboff, S. 1987. In the Age of the Smart Machine: The Future of Work and Power . New York: Basic Books.