How to Introduce AI to Your Team
To introduce AI to your team, name the tasks the tool is for and show it working on examples of the team's own work. Say what it got wrong and how to check its output. Say what information people can put into it and what you expect them to do. Promise only what you've seen it do.
Your organization has approved an AI tool, and you're the one introducing it to your team. Some people on the team have tried AI tools on their own, and someone has already asked whether this one will do their job.
What Should You Do before Introducing AI Tools to Your Team?
Choose the tasks you'll name in the introduction. For each one, try the tool on several real examples of the team's work and compare what it gives back with the work the team did. Write down what it did well, what it got wrong, how you found each mistake, and how long checking took.
Find out your organization's rules for the tool. Ask whoever approved it, or your IT or security team, these questions.
What information can go into it, and what has to stay out, such as client details, personal data or confidential documents?
Does someone have to review the output before it's used, and for which kinds of work?
Does any work need to show that AI was used, such as documents that go to clients?
If the people on the team report to someone else, agree with their manager which tasks the tool is for and what the manager will expect.
How Do You Announce AI Tools to Employees?
Introduce the tool in a team meeting where you show it working on the examples you tried, and send the same information in writing that day. Cover these points.
Which tasks the tool is for, and which tasks stay with people.
What it did well and what it got wrong on your examples.
How to check its output on each task.
What information can go into it and what has to stay out.
That you expect people to use it on those tasks and to mention when they have.
What you expect each person to do in the first weeks, such as using it on one named task, and from what date.
Who to ask when it gives a wrong or unusable answer.
When you'll check in with the team.
Say why the organization brought in the tool, in the words of the person who decided. Brief managers before the meeting and send the next update on the date you gave, as you would when announcing any change to your team.
Why Do Expectations Matter When You Introduce an AI Tool?
Michael Ginzberg measured what users expected of new information systems before the systems were put in place, and followed the projects afterward (Ginzberg, 1981, Management Science). His results suggested that users with realistic expectations before the system arrived were more satisfied with it and used it more than users with unrealistic ones.
Anol Bhattacherjee surveyed online banking users about whether they intended to keep using the service (Bhattacherjee, 2001, MIS Quarterly). Users whose use of the service had confirmed what they expected, and who found it useful, were more satisfied. Satisfaction and seeing the service as useful went with intending to keep using it.
Time to learn the tool, help when it gets something wrong, and what managers expect also affect whether teams use AI tools you paid for.
How Do You Set Expectations for an AI Tool without Overpromising?
Rafal Kocielnik, Saleema Amershi and Paul Bennett tested ways to set people's expectations of an imperfect AI assistant before they used it (Kocielnik, Amershi and Bennett, 2019, CHI Conference on Human Factors in Computing Systems). The assistant looked for meeting requests in email, and the expectations were set on its screens, such as an accuracy display. The researchers built two equally accurate versions. One wrongly flagged some emails as meeting requests, and the other missed some real requests. The accuracy display brought people's expectations closer to the assistant's real accuracy. In the version that missed requests, the screens that set expectations raised how willing people were to accept it after using it. In the other version, they made no clear difference. People rated the version that wrongly flagged emails as more accurate and were more willing to accept it. The authors linked the difference to how easily users could recover from each kind of mistake. The participants came from a crowdsourcing platform, and the task was low-stakes.
The study didn't test what a person tells a team about a tool. When you introduce one, describe it by what you saw on the team's work, such as "On the examples I tried, it drafted the weekly client summary in a few minutes, and some drafts had the wrong meeting date." For each task, say which kind of mistake the tool made and what it takes to catch and fix it. Leave out claims you haven't checked, such as how much time it saves or how easy it is to learn. Say which tasks you haven't tried it on yet.
How Do You Show Your Team What the AI Tool Gets Wrong?
The US National Institute of Standards and Technology (NIST) lists confabulation, its term for false content that the tool presents confidently, among the risks of generative AI (National Institute of Standards and Technology, 2024, NIST AI 600-1). It notes the problem is especially relevant to open-ended prompts that ask for long answers and to work that needs a lot of context or expertise. The output can include made-up citations or reasoning that seems to support a wrong answer. People may believe false content and act on it, often because the tool states it confidently.
NIST also notes that people without detailed knowledge of how these tools work may avoid them and miss their benefits. Over time, people may also rely on them too much, or rate their output higher than work from other sources without good reason.
In the introduction, show the team an output from your trial that looked right and was wrong, and how you found the mistake. For each task, agree what to check before the output is used, such as names, dates, figures, quotes and sources, and what to check them against. Agree who checks each output before it goes to anyone outside the team.
What Should You Do in the First Weeks after Introducing the AI Tool?
Check in with the team on the dates you gave. Ask each person which tasks they used the tool on, what they expected it to do, and what happened. Where it did worse on a task than you told them, tell the team what you found and change what you've said about that task. Where it did well on a task you didn't name, try it on more examples of that task before you add it to the list.
To measure AI adoption on your team, check use on each task you named in the introduction. When the rules for the tool or the list of tasks change, send the update in writing to everyone who got the introduction.
Where Do You Start?
This week, write down the tasks you'll name, what the tool did on your examples of each, and your organization's rules for it. Ask the person who approved the tool to check the rules, and book the meeting with your team.
Frequently Asked Questions
Should You Tell Your Team the AI Tool Makes Mistakes?
Yes, with the mistakes you saw on their tasks and how to catch them. In Ginzberg's study, users with realistic expectations before a new system arrived were more satisfied with it and used it more. In Kocielnik and colleagues' study, screens that set expectations before people used an AI assistant raised acceptance in one of the two versions tested.
How Do You Introduce AI to Employees without Causing a Panic?
Answer questions about jobs during the introduction. When you talk to employees who fear AI will take their jobs, tell them what's been decided about roles, what hasn't, who decides, and when they'll hear more. Promise only what you control, such as the scope and dates of your rollout. Name the tasks the tool is for and the tasks that stay with people.
What If Leadership's Announcement Promised Too Much?
Describe the tool to your team by what it did on their work. Before the introduction, show your results to the person who made the announcement, name the tasks where the tool did badly, and tell them you'll describe it to the team by those results.
What Information Can Employees Put into an AI Tool?
It depends on your organization's rules. NIST lists data privacy among the risks of generative AI and describes it as leakage and unauthorized use or disclosure of personal or sensitive information. Give the team the rules with examples from their own work, such as client details or staff records.
What If Some People Won't Use the AI Tool after the Introduction?
Ask each of them why before you decide how to respond. The reasons employees resist AI tools can include worry about their job, the time it takes to learn, or a bad result on their own work. Answer the reason the person gives. If the tool did badly on one of their tasks, take that task off the list until you've tried more examples.
If you're running a rollout and want a plan for each stage, The DANCE System gives you 29 tools across five phases, from defining the change to keeping it in place after launch.
Related Blog Posts:
Why Isn't My Team Using the AI Tools We Paid For?
How to Announce a Change to Your Team
How to Talk to Employees Who Fear AI Will Take Their Jobs
References
Ginzberg, M. J. (1981). Early diagnosis of MIS implementation failure: Promising results and unanswered questions. Management Science, 27(4), 459-478. https://doi.org/10.1287/mnsc.27.4.459
Bhattacherjee, A. (2001). Understanding information systems continuance: An expectation-confirmation model. MIS Quarterly, 25(3), 351-370. https://doi.org/10.2307/3250921
Kocielnik, R., Amershi, S., & Bennett, P. N. (2019). Will you accept an imperfect AI? Exploring designs for adjusting end-user expectations of AI systems. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1-14). Association for Computing Machinery. https://doi.org/10.1145/3290605.3300641
National Institute of Standards and Technology. (2024). Artificial intelligence risk management framework: Generative artificial intelligence profile (NIST AI 600-1). US Department of Commerce. https://doi.org/10.6028/NIST.AI.600-1
