A User Centered Design Approach to Customer Service Applications
The goal of the paper is describes the work in progress of how the PayPal User Interface(UI) design group develops a user centered design process for customer service agent applications. This process also is aimed at gathering customer feedback from the agents via users and plugging that back into the organization for the purpose of improving the products. The paper provides details on our plan and some lessons learned that could be generally applied to other similar systems. Additionally, we outline our challenges and issues that surround this real world project.
Ethnography, Responsive Environments, User-Centered Design / Human-Centered Design, User Experience, User Interface Design, User Research, User Studies.
PayPal employs a User-Centered Design (UCD) process  for creating and developing usable products. However, no matter how well the customer application is designed users may need to contact a customer service (CS) agent. These resulting contacts (email or phone) must be timely, efficient and leave the user feeling satisfied. The applications and tools that the CS agents use to manage these touch points and contacts are not currently being designed with a UCD methodology in mind.
Our first goal was to improve quality and usability of agent applications which would then improve efficiency of agents. In order to do this we needed to understand the context of use. Our second goal was to establish a process which we can gather feedback from agents on emerging designs. Lastly, we want to create a system for capturing and bringing user issues via agents into the organization for the purpose of improving products and the customer’s experience. Ultimately, we want bring the UCD approach to each interaction between the agent and the customer to improve all of PayPal’s products.
- Team members
- Customer service agents
- Customer service supervisors
- Customer service managers
- UED director
- UI manager
- UI designer
- Visual designer
- Web dev
Dates and duration
We began the planning the effort in Q2 of 2005. We wrote problem statements, goals, and solutions which were vetted with the UED director. Solutions were prioritized and phased to span over multiple quarters. Overall, this effort is considered a long term goal for our organization.
Context of project
PayPal is growing strong and is now up to 71.6 million account holders. These customers pumped $6.2 million through the company in the first quarter of the year. The service now boasts more accounts than Bank of America and Discover.
PayPal handles these payments through its headquarters and operations in San Jose. In addition, the San Jose location creates, designs, deploys, and maintains services for its users. However, there are some operations conducted outside of San Jose which directly interact with customers. One of these is the customer service team which is located in the Midwest and is ready to answer customer questions and issues nearly 24 hours a day.
The first challenge of this effort was that we did not understand how the CS operation was organized. Understanding how the CS operation is organized was necessary in order to determine who to talk to and comprehend what the various groups accomplish. Since the CS operation is large and in a different location it was difficult to gather this information. It was almost impossible to know exactly who to observe and at what precise time. Once we arrived, we did find that people were very responsive and supplied lots of information and direction.
The next challenge was that we had a limited number of resources available for observations. We were unable to take the entire team to the location because of time and budget constraints. In the end, we were able to minimize this issue by leveraging resources from other departments.
Since the agents deal with real customers and real cases all day long we had to be careful not to significantly impact their work load or quality. We were profoundly respectful of their time and how they spent it with us. We did not take for granted that we were in their environment. We understood that they have important work to do.
Before we could conduct one single user study or test any UI designs we needed to figure out what we were going to do and how we were going to phase our approach. We had to phase our approach as it was impossible to get everything accomplished in one quarter. In addition, after drafting a plan we needed to vet it with product and UI management.
The first phase consisted mostly of research. In order to understand the agent environment and the agents themselves we felt important to witness first hand how they work. For us this meant traveling to their location in the Midwest and spending time observing them work, asking them questions, and “job shadowing” them.
We wrote data collection surveys and observation worksheets to aid in our information gathering process. The data collection was focused on collecting demographic information about the agents as well as their environment.
Some of information that we collected included:
- Position title and organization
- Length of time with the company and organization
- Level of education
- Technical ability
- System information (Monitors size, Screen resolution, CPU, etc.)
- Applications used
As momentum built around a site visit other groups became interested in what we were doing. We were excited to have such interest and decided to include people from visual design and web dev. This had several positive effects for our site visit. First, it helped generate “buy-in” from around the organization. By involving people beyond the ui design group we felt that this would help others to understand our methodologies and processes and those individuals would become ambassadors for our continued efforts. Secondly, we felt that the individuals that participated in the site visit could provide unique insights and observations. Lastly, these people added a very practical aspect to our site visit – they helped us overcome our resource constraints for observations.
As a result we needed to get folks trained at a high level on how to observe and collect information. The UI director and I held a training session to get folks familiar with the process. A template was created to facilitate and guide the process of data collection for the observations.
Some basic observation tips included were:
- Take pictures of the agent in their environment
- Explain the purpose of our visit
- Set the subject at ease by reassuring them there are no right or wrong answers
- Due to little or no room use paper and pen
- While hand writing notes using symbols can help save time
· Use quotes to capture what person said.
· Circle or eye for observation of what the person did but did not say or realize.
· Start or key mark for new findings.
Future analysis of the data that we collected and observation notes taken during our site visit will be used to create user profiles / personas. These personas will be used to guide decisions about product features, navigation, and interaction design.
While the observations were fruitful, in retrospect it was difficult to learn the agent’s areas of expertise while simultaneously observing how they perform their work. However, due to their location and lack of understanding of their organization we did not have a opportunity or the ability to learn this information before we departed San Jose. Future visits will be easier and more productive as we now have learned what they do and what information they review.
During our visit we discovered an important factor about CS agents. They are measured against very specific performance criteria.
Some of these performance criteria are:
- Hold times
- Handling times
- Cases handled
- Time spent on emails
These performance criteria impact the agent workflow and their processes. Modifications to interfaces have to be created in a way that does not adversely impact their workflow and performance.
We realized that it was also important to understand the reporting and outputs from the CS group back to the organization. The information that is reported by CS agents provides insight into problems and issues that users have with the site. This data is not being fully leveraged by the product team today. In the future this information should feedback directly into those who create and deploy products. This is really a long term goal for our organization which requires processes to be created and put in place.
Communication and relationships are critical to our success. It is important that we establish and maintain relationships with key members of the CS organization. These will allow us to better understand their point of view, design better applications, increase agent performance, increase customer satisfaction, increase agent satisfaction, and improve ease of use. Additionally, establishing regular channels and frequency of communication will build increase trust and pave the way for the ability to capture feedback on emerging designs.
As we continue to discover and understand the CS environment we have begun to diagram that system. There are many benefits to diagramming the system.
Some of these benefits are:
- Helps get everyone on the same page
- Defines common vocabulary
- Puts the system on “paper” and out of the heads of people
- Facilitates open discussions between people involved
- An opportunity to portray participants and their responsibilities
In the course of our site visits we observed many white boards around the operations which contained: organization names, mission statements, workflows, metrics. We captured these with our digital camera and have been reproducing them and hanging them in our area. We feel that these constant reminders of the CS areas, work flows and impact they make help us make better UI decisions.
There have been several positive results from our visit. One of them is that we have been engaged in more intelligent and more productive conversations. In a recent project, we were able to provide feedback on how the proposed changes would affect the agents and their workflow. This level of conversation is a direct result of our site visit and our new found understanding of the context of use.
We recently participated in a brainstorm exercise with agent supervisors and managers to discuss strategies to reduce claims. This brainstorm goes a long way in building relationships and trust.
These are a couple of the initial “wins” we have had from the site visit. I expect many more in the coming weeks and months. It is very exciting and rewarding to see the benefits of our effort.
We have many activities planned for the quarter(s) ahead. Some of these activities include:
- Gathering feedback on a system diagram
- Creating personas that represent agents and supervisors to guide product development and UI design
- Conducting task analysis
- Conducting additional site observations
- Establishing process for conducting usability tests with agents on some existing features and new features
- Evangelizing our successes to gather more support and buy-in from our team and from management.
 Norman, Donald A. The Design of Everyday Things Basic Books; 1st Basic edition (September, 2002).
 Cooper, Alan,.The Inmates Are Running the Asylum : Why High Tech Products Drive Us Crazy and How To Restore The Sanity, Sams; 1st edition (April 6, 1999)
I would like to thank Corey Bernardo and Laura Ward for their support and feedback. I’d also like to thank all of the individuals who participated in our research.