AI readiness assessment (what you need, and how to measure it)

Let’s not sugarcoat it. A lot of people are panicking. “If we don’t adopt AI, we'll become irrelevant. If we try to do too much with AI, it can open us to unknown risks and destroy our business.” Measuring your AI readiness is a good way to find the middle path. 

5 things you need for successful AI adoption 

Before we get into the specific questions you can ask to assess AI readiness, it might help to define what we’re trying to measure. These are the five pillars that will support your AI implementation. 

1 – Business Strategy 

AI adoption in itself is not a strategy. If you’re a widget manufacturing company, your goal is still to manufacture the best widgets possible. If your AI adoption plans don’t serve that goal, you’re not ready. 

2 – Data Strategy 

Are you going to be using your own data? If so, is it well structured, or is it a blob? Feeding the LLM something organized is much better than pointing it at your SharePoint and wishing it luck. If you’re using the internet’s data, or third-party data, then you also have to worry about how relevant, accurate, or diluted it is. The data you’ve accumulated is part of your competitive advantage, so it’s best to find a way to use it. 

3 – AI Experience and Expertise 

A lot of companies are hiring “AI Teams” to guide their implementations. We’ve seen this with DevOps before. Building an ivory tower team is a bad way to implement something in productive way that yields the outcomes you need. If you find a way to get everyone involved by infusing their work with AI, you don’t limit your organization by only allowing one team to grow and have great ideas. 

4 – Organizational Management and Culture 

Does your existing org structure and hierarchy support rapid implementation and decentralized decision making? If a good idea has to travel up and then back down five levels of bureaucracy before someone can run with it, you’ll have a hard time keeping up.  

The way management rolls out the AI adoption has a big impact on whether the organization will be excited by or afraid of AI. If leaders create AI-based OKRs or professional development goals that serve the greater company goals, people will be more excited about the productivity gains than they are afraid of being replaced.  

Think about when past leadership decisions have emboldened people versus crushed them—if you’re coming off of a year with three force reductions when you announce AI adoption, people will read between the lines whether you want them to or not., 

5 – AI Governance and Compliance 

If you’re creating agents, you have to be mindful of which systems you give them access to. In regulated industries like finance and healthcare, it’s often illegal to send certain data off-prem for processing.  

The trick is to stay compliant and secure without being so conservative you stagnate. Do you have an equation for balancing the risk of rapid AI adoption with the risk of getting left behind by your competitors who will innovate more boldly? 

Questions to check your AI readiness 

So, how do you know if you have all five of those things in order? There’s much to consider for a successful AI adoption, more than we could put in a blog post. Still, these 11 questions will help you think about getting value from artificial intelligence without taking on too much risk or burning cash. 

1. Have you aligned your AI adoption goals with greater business goals? 
2. Have you thoroughly thought out use cases for AI, including the goals of those use cases? Are you chasing efficiency, productivity, new capabilities, or replacement? 
3. Do you know which data you’ll need for your AI use cases, including the data you already have and additional data you’ll need? 
4. Do you have protocols in place for security, compliance, and governance? 
5. Do you know which AI models or tools are best suited to your specific needs? 
6. Do you have a structure that allows company-wide experimentation with AI, or are you limiting exposure to one isolated team of “experts”? 
7. Can you clearly communicate how AI will affect the future of the busines, and why this is important? 
8. Do you have resources and avenues in place to help people upskill and start experimenting on their own (rather than a training course people will click through)? 
9. Do you have controls in place to track the accuracy of AI’s outputs and flag hallucinations? 
10. How will you ensure visibility into what your AI agents are doing and keep your processes auditable if needed? 
11. Do employees have clarity around what they can experiment with AI safely, and where the boundaries of acceptable use are? 

One way to gauge your AI readiness is to score yourself on a 1-10 scale for each of these questions and average the answers. Most importantly, you’ve started the conversation that will help you develop a roadmap toward a smooth AI implementation. 

Building the AI adoption roadmap 

Building the roadmap to a successful AI implementation depends heavily on your specific needs, but it normally boils down to three steps: 

Explore  

As you do for other products and projects, identify the key stakeholders across the organization who will need to be satisfied for this initiative to succeed. Sit with them to learn more about their needs, gather insights, and document your findings. Instantiate metrics and take watermarks so you can measure change, negative or positive, throughout the course of your implementation 

Analyze 

First, analyze your own organization, specifically the gaps between where you are and where you’d like to be. For example, identify AI use cases by thinking about tasks that take a lot of time and wouldn’t be risky for AI to do. One of the key differentiators in our approach is that we assess cultural readiness alongside technical readiness. AI essentially becomes a member of the team, so people have to be ready to work with it. 

Next, analyze AI to see how it can fill gaps and help you hit your goals. Examine trends, study relevant case studies, and get a feel for the technical aspects. In doing so, you’ll find matches to your own opportunities for improvement. 

Analyze the constraints. This includes any regulatory and legal boundaries that could prevent you from using AI everywhere you otherwise might. 

Prioritize 

Consider the effort, cost, feasibility, impact, and risk involved with each of your opportunities to implement AI. Ideally, use a defined scoring framework to make the decisions as objective as possible (we like the Prioritizr Jira plugin for this). This helps you make sense of the backlog, create the roadmap, and develop an actionable implementation plan. 

Next steps 

If you’ve been nodding along this whole time, maybe the way forward is already obvious to you. If you’re scratching your head instead, it’s ok to admit you’re not an AI implementation expert. Few are. This only became pressing recently. 

We hope you’ll lean on the team at Sketch for guidance throughout your AI journey, starting with the assessment and going all the way through implementation. Our AI consultants aren’t the type who hand you a PowerPoint deck and wish you good luck. We’ll help you put it into practice, just like we did to help an HR tech company launch an agentic AI product in only three sprints. 

The questions in this article should help you get started. For an AI readiness assessment with a more formal readout (and suggestions for next steps), get in touch.