Observation
What is it?
Also referred to as Job Shadowing
and Following people around
, at first glance, Observation seems like the easiest business analysis technique for eliciting requirements that there is. You just go out and watch someone work, and write down what they do, right? You can do that, but you may not be leveraging the technique as best you can.
The first decision you have to make is, what type of observation do you want to undertake? Of course, at this point you are probably thinking how many ways are there to sit and watch someone do something?
Perhaps surprisingly, there at least five that I have come across. The first three below come from the BABOK Guide[1], with two others from the Business Analysis Techniques
" book.[2]
Methods of Observation
Active
With the active observation method, the BA asks questions of the person they are observing if the person does something the BA does not understand. Or if the BA is trying to understand why a person did something in a particular way. Even if by asking the question, the BA is breaking the participants work flow or otherwise causing a disruption of their normal work patterns.
Passive
With the passive approach, the BA simply observes and takes notes. They do not ask questions while the participants is conducting the work being observed and generally try to be as unobtrusive as possible. Only after the BA has viewed the entire work process one or more times will they ask questions of the person being observed to make sure they understand the process or why certain actions were taken.
Apprenticing
The IIBA also identifies a third option, which is that the BA becomes an apprentice
in the job and asks an experienced user to train them.
Protocol Analysis
The book Business Analysis Techniques: 72 Essential Tools for Success [2] also recommends an observation technique known as protocol analysis. In protocol analysis the person being observed essentially provides a running monologue of what they are doing and why they are doing it as they go about their job. This is also in the BABOK guide, but not given a specific name.
STROBE
The Business Analysis Techniques book also discusses the STructured Observation of the Business Environment (STROBE) observation technique. With the STROBE technique the BA is not trying to document a business process, but is rather engaging in a form of observation in order to gather specific metrics such as how many calls a worker takes, how many calls are converted to orders, how often specific applications or screens are used, how often a worker has to turn to a specific resource to find information, and other similar specific metrics.
Why do it?
So why do Observation at all? Observation is invaluable if you need to understand the way a current business process functions, if you are looking to change or improve a current business process, or if you are trying to learn specifically how users actually engage with a piece of software or other job aid. You can just ask someone who does the job that you need to understand, but while most workers can tell you what they consciously know, they can't tell what they unconsciously know. Observation allows you to watch someone at work in their normal environment and see what they actually do, not what they can consciously think to tell you.
How do I do it?
The observation process generally involves three steps. They are:
Preparation
Determine who you are going to observe. You generally want to observe more than one person in order to see a wider range of activities. Think about:
- Ensuring you observe all of the different roles involved with the domain you are investigating.
- Observe people with different skill levels and job experience. The new hire may do things differently, and have different needs, than the experienced worker.
- Determine when you are going to observe the participants. Different times of day, week, month, quarter, and even year can all involve different work in some cases. So try to identify any particularities of the time periods you have available, especially if different stakeholders work may change on varying schedules.
- Determine how much time you have to dedicate to observation and allocate it as best you can to ensure the widest array of information, but the greatest depth of coverage possible.
- Review any existing documentation on the processes you will be observing ahead of time. Both to familiarize yourself with what in theory should be happening and to help put what you will be observing in context. This helps improve your understanding of what you are observing, and reduces the amount of
needless
questions (from the perspective of the person being observed) you may ask.
- Prepare questions ahead of time to help elicit information you know you are trying to gather.
- If you are going to try and gather specific statistical data (for example; the number of calls received, the number of orders entered, etc.), prepare a worksheet that will let you easily gather this information with the minimal amount of effort or disturbance.
Observation
First, conduct the observation session(s) you have planned and keep in mind the following.
- Be sure to inform the person you are observing that their work is not being questioned in any way, but rather that what is learned will go into trying to improve their work processes. [3]
- If it's true, you may want to tell them that you are observing them because of how well they do their job and that they were suggested as expert who could provide demonstrate the widest range of work and expertise. This not only makes them feel more comfortable with the observation process, but encourages them to share more information.
- Inform them that the observation can be stopped at any time if it is interfering with their work. [3]
Second, if the environment and work allows, consider recording the observation session.
- This works especially well for Protocol Analysis.
- In the case of software usage, don't forget that some software (SnagIt at least) will capture the users screen and let them dictate into a headset (for at least a period of time). [See Tip #1 for a related idea]
- Having a recording that you can refer back to lets you focus less on taking notes, and more on what is actually being done and why. [See Tip #1 for a related idea]
Afterwards
- Analyze your notes, looking for gaps or questions that were not answered
- Follow-up with the persons you observed to both summarize what you observed and to fill in any gaps.
- Assemble the analyzed information of the process and consider validating it with not just the workers you observed but others involved in the same process. They may identify missing information that you were not able to observe.
What Should the Results be?
The results of your observation sessions should be complete (hopefully) documentation of the process(es) being observed, with problem steps identified, and possible improvements already determined. Or, if you are undertaking observation for statistical purposes (the STROBE method), then you should end the observation process with the statistical information you set out to gather.
Advantages
- Observation allows the BA to capture information on a process the way it is actually done, not the way it may be officially documented or the way participants consciously think they do it. This can capture information communication or references, and identify how the "real" process may differ from the documented process (if there is documentation).
- Observation is especially invaluable for finding ways to improve existing software, as you can see how users actually interact with an application and see where improvements can possibly be made.
- Observation is useful when stakeholders find it hard to explain exactly what they do. [4]
Disadvantages
- Only existing work flows can be observed.
- Can be VERY time consuming if the BA wants to be thorough. Otherwise the BA risks missing information that is known to participants that were not observed, or that is not executed by the participants observed. This is especially true of infrequent "exceptions" or variations to the base process.
- You can never be sure that you observed all of the process, or learned everything you needed to learn. Unless you have a VERY large amount of time to do observation, and can ensure you observe EVERYONE doing the work involved, it is likely that you will miss some information. And even if you have that much time and observe that many people, it is still likely that you missed the importance of something that was done. Or that a rare occurrence (an "exception" process) simply did not occur while you were observing. The means that while observation can provide valuable data, you must always consider you knowledge to be "full of holes".
- There is a strong risk that the very act of being observed will cause the participant to change the way they behave or conduct their work.
- The process can be disruptive not only to the person being observed, but possibly to others around them.
- The process may not be feasible if much of the work being observed is highly mental or not easily visible.
- The output can influenced by the observers bias or even by conscious choice of the person being observed (who can potentially tailor the actions they take or how they do their job in order to suggest specific issues or actions to the BA).
- Statistical values generated from observation may be influenced by the Hawthorne Effect in which those being observed modify their behavior as a result of being observed. This can make the process seem more efficient than it normally is for example, because those being observed work harder or skip non-obvious steps in order to appear to be working harder.
Tips
- For observation involving computer usage, it is now possible to do at least partial observation without having to be with the user. Microsoft Lync 2010 (and probably other applications) will let the user share their desktop while conversing with you. And with Lync at least, it is possible to record this session as well (if your corporate configuration allows). While there is nothing as good as actually being with the user, if you work in a globally distributed firm this provides an option for those times when the budget just doesn't cover flying you off to another work site.
- If you are going to do observation, try to make sure you observe both the newest member of the user group and the most senior. Unless the most senior is the person who trained the newest, in which case observe the next most senior who did not train the newest user.
- When observing many users, compile notes at regular intervals to identify commonalities and differences between users. Review findings with the entire shadowed group to ensure that the final details represent the entire group, not selected individuals. [3]
References
- BABOK Guide, section 9.18
- Business Analysis Techniques: 72 Essential Tools for Success; by James Cadle, Debra Paul, and Paul Turner. 2010. ISBN: 978-1-906124-23-6
- Article: Elicitation Techniques. By Christina L. Tenerowicz. Part of the Cornell University BA Toolbox.
- Blog: Using the Observation Technique for Requirements Elicitation. By Abidemi Stephanie Famuyide. On the Business Analysis Learning's blog.
Related Resources