AI Policy 101: a Beginner's Framework
How to make a case for AI experimentation & testing in learning & development
Hey folks!
As the world transitions from a period of discussion about AI to a period of experimentation with AI, a question I’ve been asked a lot in recent weeks is: How do I make a case to enable me to experiment with AI? And what sorts of experiments should I run?
As Learning & Development (L&D) professionals, making a case for AI experimentation requires a structured approach to demonstrate its potential benefits.
Based on my work with a number of companies which are experimenting with AI in L&D, here’s a seven step framework to help you justify and implement AI-driven initiatives within your organisation.
Let’s go!
Step 1: Communicate the Need for AI in L&D
The first question you will be asked is, why? Make a clear case of how AI can impact both the efficiency and effectiveness of L&D initiatives:
Efficiency: Enhancing Organisational Processes
AI can significantly improve L&D processes by automating routine tasks, personalising learning experiences, and providing real-time analytics. Highlight these areas to show potential improvements:
Automation of Administrative Tasks: AI can handle scheduling, tracking progress, and reporting, freeing up the L&D team to focus on strategic activities. For example, tools like chatbots can answer common questions, reducing the workload on support staff.
Content Creation and Curation: AI can rapidly create and update training materials, saving time and reducing costs associated with content development. Generative AI can produce high-quality content tailored to specific learning objectives, ensuring that materials remain relevant and engaging over time.
Resource Optimisation: AI helps to optimise resource allocation by analysing usage patterns and learner needs. This ensures that resources are deployed where they are most effective, minimising waste and maximising impact.
Smart Training Design: Using Predictive Analytics, AI can predict which employees are likely to need specific training programs, allowing organisations to allocate resources more effectively and prevent skills gaps before they impact performance.
Effectiveness: Improving Training Quality and Impact
Emphasise how AI can enhance the effectiveness of L&D initiatives:
Personalised Learning Pathways: AI can create individualised learning paths based on each learner's performance, preferences, and needs. This ensures that learners receive the most relevant content at the right time, improving engagement and retention.
Adaptive Learning Systems: Adaptive AI systems can adjust the difficulty level of training modules in real-time based on the learner’s progress. This constant adjustment keeps learners challenged and supported, leading to better skill acquisition and performance.
Real-Time Feedback and Assessment: AI-powered tools can provide immediate “always on” feedback on assignments and assessments, allowing learners to understand their mistakes and correct them quickly. This accelerates the learning process and enhances retention.
Skill Gap Analysis and Development: AI can analyse employee performance data to identify skill gaps and recommend targeted training programs. This ensures that employees develop the necessary skills to excel in their roles, directly contributing to improved organisational performance.
Step 2: Gather Supporting Data
To bolster your answer to the “why?” question, use robust data to show the big picture potential of AI in the workplace.
Research Studies
Present data from reputable studies to support your case. Some helpful go-to resources sources include:
BCG: Boston Consulting Group produced research which shows that, in the right context, AI can increase workplace efficiency and effectiveness by ~40%.
McKinsey & Company: have created a series of reports on the impact of AI n productivity, including a recent report which states that AI will increase productivity and raise global profits by $2.6 to $4.4 trillion over the next 15 years.
Academic Research: a number of academic studies have revealed interesting data about the impact of AI on high value workplace skills like creativity and problem solving.
Existing AI Experimentation in L&D
Show examples of how organisations already using AI to improve L&D, for example:
The use of AI in content production: existing experimentation shows that AI has made it possible to create more, more relevant and tailored content faster than ever, enhancing efficiency and effectiveness within L&D departments.
The use of AI for “always on” staff support & feedback: existing experimentation shows that AI has made it possible for some L&D teams to automate the process of assessing and giving feedback on employee performance in the flow of work.
The use of AI for training impact evaluation: some organisations are using AI to track the impact of training on employee behaviour and performance by running semantic analyses on staff interactions, e.g. conversations with clients.
Step 3: Propose Specific Hypotheses & Pilot Projects
Get concrete and practical by proposing specific hypotheses and pilot projects. Continue to answer the “why?” and “why now?” questions by selecting hypotheses which map directly to your organisation’s KPIs, vision and mission. For example:
KPI 1 - Improve Return on Investment (RoI) in L&D by 20%
Pilot & Hypothesis - Automation of Content Creation: By implementing AI we can generate high quality and more differentiated content at a lower cost and increased velocity. This will result in at least 20% improvement in Return on Investment (RoI) for L&D compared to our current baseline.
KPI 2 - Lower Overall Organisational Costs by 15%
Pilot & Hypothesis - Automation of Admin: By using AI to automate routine administrative tasks such as email writing, scheduling, note taking and survey writing, we will reduce the current overhead costs associated with manual administrative processes by at least 15%.
Step 4: Talk About Technology, Security & Risk
The second big question you’ll be be asked will almost certainly be, “what about data risks and security?”.
Technology
Be specific about the the AI tools and platforms you will use use, e.g.
OpenAI Enterprise: For creating and updating training materials, quizzes, and instructional videos.
Grammarly: For writing reports, emails, and other communications.
Otter.ai: For note-taking and survey generation.
Security & Data Compliance
Communicate how each of the tools complies with your org’s data security and privacy regulations, e.g.
OpenAI provides comprehensive security and compliance measures, including:
Data Security: State-of-the-art encryption and access controls to protect data.
Privacy: Compliance with GDPR and other international data protection regulations.
Ethical AI Use: Transparency and accountability in AI applications, promoting ethical use of AI technologies.
For more detailed information, visit OpenAI’s data and compliance pages.
Grammarly complies with applicable regulations including the EU’s General Data Protection Regulation (GDPR), the California Consumer Privacy Act (CCPA), and the Health Insurance Portability and Accountability Act (HIPAA), among other frameworks that govern Grammarly’s privacy obligations.
For more details information, visit Grammarly’s privacy pages.
OtterAI utilizes best practices to protect customer data and works and employs independent experts to verify its security measures. It has achieved SOC 2 Type 2 report against stringent standards and complies with both GDPR and California Consumer Privacy Act (CCPA).
For more details information, visit OtterAI’s privacy & security pages.
Step 5: Develop Guidelines & Principles
It’s likely that your organisation won’t yet have developed AI Guidelines & Principles. If this is the case, take the lead by suggesting a set of foundational AI principles to guide your project. For example:
Foundational AI Principles
Transparency: Ensure all AI decisions and processes are transparent and understandable to stakeholders. This builds trust and ensures that all participants can clearly see how decisions are made, fostering confidence in the AI tools used.
Equity: Guarantee equitable learning opportunities and avoid perpetuating biases through AI. This principle ensures that all learners have fair access to training resources and that AI does not reinforce existing inequalities.
Accuracy: Maintain high standards for the accuracy of AI-generated content and recommendations. Accurate AI outputs are essential to provide reliable and effective learning experiences that truly enhance learner performance.
Privacy: Safeguard learner data with stringent data security measures. Protecting privacy is crucial to maintain the confidentiality and integrity of learner information, ensuring compliance with data protection regulations and building learner trust.
Step 6: Identify Stakeholders
The other questions that you’ll be asked relate to who and how. Be specific about who will be involved. Keep the team as small as is viable to minimise both real and perceived risk:
L&D Team Members
Role: Primary users of AI tools, integrating AI into training programs.
Support: Provide training and support to ensure effective use of AI tools.
IT & Data Security Team
Role: Ensure secure implementation and integration of AI tools.
Support: Collaborate with AI vendors and stakeholders to address security concerns.
Senior Leadership
Role: Oversee pilot projects, provide strategic direction, and allocate resources.
Support: Regular updates and presentations to keep leadership informed.
Step 7: Write a Compelling Conclusion & Call to Action
Write a compelling conclusion which underlines the strategic business benefits of the experimentation and the careful management of AI-associated risks.
Generate FOMO and reassurance by reminding your audience that others in the industry are already experimenting and feeling the strategic benefits of AI.
To keep the process moving, it can help to include some questions for the leadership team. For example:
Resource Allocation: What budget can be allocated to AI experimentation in L&D? Are there specific financial constraints or preferred vendors?
Timeline: What is the desired timeline for initiating and completing the AI experimentation phase? Are there critical deadlines we need to meet?
Metrics of Success: What specific metrics or KPIs will the executive team prioritise to measure the success of AI integration in L&D?
Risk Tolerance: What is the executive team's risk tolerance regarding data privacy and potential biases in AI recommendations? How should these risks be reported and managed?
Stakeholder Involvement: Who from the executive team would likely be the primary point of contact for this initiative? Are there specific stakeholders or departments that should be involved in the planning and implementation phases?
That’s all folks! I really hope this is helpful. Give me a shout out on LinkedIn to let me know.
Happy innovating,
Phil 👋
PS: If you’re already an early adopter or want to become one and meet others like you, apply for a place on my AI Learning Design Bootcamp. Here’s a review from a student from my June cohort.