TOOLS
CONTRIBUTIONS
With the rapid development of technology, AI has been integrated into our lives in many ways. Accurate machine learning and NLP make them smart. But is this trend only good for humans? How can we trust AI with our data? In this capstone group project with two IBM mentors, we aim to address AI trust issues, specifically chatbot trust. We chose Dyson's official website chatbot for a redesign and ultimately came up with a conversational AI chatbot with a focus on privacy and other fields to build more trust.
I'm super excited to share that our design solution has won first place in the MDEI program capstone project competition! I really appreciate all the hard work of my team members and support from instructor Tobias Thielen as well as IBM mentors Gord Davison and Will Ragan.
Artificial intelligence systems are rapidly starting to dominate our app and tech services. But it's important to remember that the AI is only as intelligent, fair and ethical as the data it is fed and the values of its creators.
Designers and developers of AI systems have the responsibility of understanding the impact of the systems they are building and consider that ethics of their work.
Only through transparency and ethics considerations can designers create the trust needed for users to adopt and use the systems and applications they build.
This is an intensive 14-day project aim at providing solutions to AI trustworthiness and ethics. The project focuses on solving the business prompt below with IBM mentors:
How might we help our users build trust in an AI-infused tool or application?
By analyzing the ethical problems of AI technologies and investigating various chatbots in the market, we determined our project scope to be redesigning the chatbot of Dyson's website to build user trust and AI ethics.
We started by looking into the various types of AI technologies and decided to narrow our scope to chatbots as research shows that there is a strong need of reliable chatbot services in the market.
80%
of customer interactions can be resolved by well-designed bots
60%
of customers want easier access to self-serve solutions for customer service
50%
of enterprise will spend more on bots than traditional mobile app dev by 2021
We initially identified general ethical issues for chatbots, the main issues include
We then narrowed our scope down to e-commerce chatbots and identified problems that prevent user trust in e-commerce chatbots.
Finally, we looked at the various chatbots on the market and defined their general problems. We found Dyson's chatbot to be the most problematic as the figure shown below, so we decided to go with it.
Have to click through a long list of choices
System defaults to connecting users to a human advisor
Conversation context lost when reaching a human
Lack of easily accessible privacy policy
These pain points pointed to a singular problem which was that the chatbot isn't utilizing conversational AI.
A conversational AI can better help with trust issues by supporting:
Conversational AI refers to AI tools that users can talk to. They utilize data, machine learning and natural language processing to imitate human interactions, recognizing speech and text inputs.
We categorized the issues into IBM's five areas of ethical focus: accountability, value alignment, explainability, fairness, and user data rights. The areas provide an intentional framework for establishing an ethical foundation for building and using AI systems.
AI designers and developers are responsible for considering AI design, development, decision processes, and outcomes
AI should be designed to align with the norms and values of your user group in mind
AI should be designed for humans to easily perceive, detect, and understand its decision process
AI must be designed to minimize bias and promote inclusive representation
AI must be designed to protect user data and preserve the user's power over access and uses
Let's meet our user, Julie, a sales director who values quality and efficiency. She wants to buy a vacuum cleaner, and Dyson is her first choice, but with so many options, it is difficult for her to know which one is best for her.
We then mapped out the user flow for purchasing with the chatbot, which helped us figure out how the chatbot would perform to meet user expectations. We initially put the entire checkout process in a chatbot.
We then designd a low-fidelity prototype with five flows, including product recommendation, checkout with an existing account, checkout with a new account, privacy policy, and end-chat survey.
"Username and password is like a scam."
"Perhaps not asking the user to type in personal information in the chat box."
"I would say it's better to send me the link to the website and I can do the check out by myself."
Action: Move the entire checkout flow to Dyson's external secure checkout link instead of having it in the chat
"Better to provide privacy-related options ahead or highlight those options."
"I can't close the 'cookies settings' on the top."
Action: Add a popup at the beginning to allow changes to user data collection settings and inform user how to access the privacy policy during chatbot use
"I wasn't able to get in touch with a human throughout the experience."
Action: Color the "live agent" button yellow and add a text in addition to the icon
"I think 80% of customers won't purchase a product using a chatbot."
"'Make a purchase' makes me feel uneasy, because it feels like they want me to input credit card and payment info in the chat."
Action: Reword "Make a purchase" in the chatbot menu to "Help me choose a product"
"I think you could also add some description to 'Anything we can do better.'"
Action: Add description to 'Anything we can do better' in the end-chat survey
"Honestly, I don't know my answer of this question."
Action: Add "Skip Questions" options to handle cases where chatbots ask users a question that may not apply to them or want to leave the question unanswered
"I don't know what 'How to make a purchase' means."
"I don't know how to checkout after receving recommendation."
Action: Reword 'How to make a purchase' to 'I'd like to make a purchase' and add a shopping cart icon on the bottom right corner of the screen
"The chatbot connect me to a live agent without any explanation."
Action: Add a "Troubleshoot or Maintain My Dyson" flow and apply sentiment analysis by enabling the chatbot to analyze the user's sentiment and hand over the user to a live agent with proper explanation as appropriate
We designed a gender-neutral chatbot character named 'Dyson Bot'. The design is inspired from the image of Sir James Dyson, a billionaire entrepreneur who founded Dyson Ltd.
The final high-fidelity prototype consisted of five flows: product recommendation, checkout a vacuum cleaner, troubleshoot a vacuum cleaner, privacy policy, and end-chat survey.
There are eight design highlights, each of which helps address trust issues.
Ensure user data rights: Users can access data collection settings through the pop-up window at the beginning, "Privacy Policy" in the menu, entering "Privacy Policy" in the chat, or the end-chat survey.
Keep transparency and fairness: Users are informed about their data collection and usage when entering data (address, payment info, etc.). There is also a summary of collected data in the end-chat survey.
Minimize bias and constantly train AI: There is a feedback option that allows user to evaluate whether the bot's response is helpful or not.
Quick and smart reply: There are multiple user response templates throughout the flow, and if applicable, users can easily click on the response in the template.
Offer explainability: Chatbot will ask users a series of questions to recommend a product. When a recommendation is given, the bot will also explain reasoning behind the recommendation.
Empathize better with users: Sentiment analysis is applied in the troubleshooting flow. Chatbot can analyze users' sentiment to decide whether to connect them to a human advisor.
Ensure user data rights, fairness, transparency, and minimize bias: There is a survey at the end to ask users for feedback on improvement opportunities and biases. This section also summarizes the data collected.
Give personality and empathize better with users: A gender-neutral chatbot character with name and image is developed so that users can feel more personality while chatting with the bot.
