The Biggest AI Challenges Businesses Face in 2026

Back in 2015, every company was trying to figure out how they could use data science to improve their business outcomes. Now there is a similar discussion around AI.

Every business always wants a magic bullet. Something that will give them a huge edge against their competitors. AI seems like that, as did data science. But like data science, many companies quickly find out you can’t simply purchase OpenAI or Claude and suddenly see impact. In fact, there are many challenges your team will face as you look to set up AI tooling. Some aren’t that different from the challenges data teams faced a decade ago, and some are new.

In this article, we’ll discuss the challenges your team will face as you start trying to deliver business impact with AI and data.

Your Data Is Not Centralized

Tell me if this sounds familiar. You need to answer a business question. You might need to segment customers or analyze patient claims data from multiple hospitals. The problem is that before you even get started, you need to pull data from several different sources to even get started.

One of those sources is an SFTP, another is an API, three are TSVs, two are CSVs, and another is JSON. Not to mention, in some cases you need to actually union those data sets together even though their columns don’t match. You have to do that every time you want to develop your analysis. Your manager would love you answering more questions from the data, but it already takes so long for you to process all that data. So you don’t

This is often where companies realize they need to centralize their data. They might call the thing they centralize everything into a data warehouse or lakehouse or just a generic data platform. The purpose is usually the same.

To have a centralized location where data lives that also has a consistent method for getting data into that location. We often refer to these methods as data pipelines, ETLs, etc.

This was important before AI, and arguably becomes even more important with AI. If you want an agent to answer questions about your customers, operations, claims, finances, or products, it needs a reliable way to access that information. You don’t want every new AI use case to start with someone figuring out how to download eight files, reconcile their schemas, and glue them together again.

AI might change how we interact with data, but it doesn’t remove the need to actually get the data into a usable state first. 

AI Costs 

I’ve now worked on several data projects where companies were looking to integrate AI. Here is how many of the initial calls go.

“So we’ve tried OpenAI, that got expensive, we tried Claude, same thing….” From there, they might reference a few other tools. Now, it’s not that these solutions don’t work. But it’s very easy to start spending thousands of dollars per month per developer. Many teams are completely outsourcing their thinking to AI.

With that comes even more reliance and spend on AI. This isn’t entirely new either. I’ve previously broken down the same cost-management problem on the data side in How To Reduce Your Data Team’s Costs. Of course, there are benefits to using AI like any tool, but here are a few tips we’d recommend you consider. 

Use deterministic systems where AI isn’t needed

In my mind, this actually refers to several aspects of deterministic systems. First, there are tools and approaches we have, such as APIs and SQL, that can provide a clear contract for input and output. Where, sure, you could just ask an AI the same question over and over again and let it blow through credits trying to figure out eight different ways to get the same solution, or you could provide a SQL layer that already has the correct answer in terms of what the code itself should be.

Use smaller models by default

Most tasks don’t require the most expensive model. In fact, at Codestrap, we lean heavily on smaller models with less reasoning. Models provided proper guidelines and guardrails don’t need to spend a lot of time looping through your codebase. If all you’re trying to do is generate repetitive SQL or create boilerplate code, then you don’t need the newest model to do that.

Reduce the context you send to the model

One of the easiest ways to waste money is sending an entire codebase, database schema, or document repository into every request. Retrieval should identify the small amount of context actually relevant to the task. Better context selection can reduce cost while also improving accuracy because the model has less irrelevant information to reason through.

Give agents tools instead of unlimited autonomy

Allowing agents to loop freely and try a dozen different approaches will quickly burn tokens. Instead, give agents a set of tools that mix deterministic and non-deterministic outputs(where needed). Sometimes you need the flexibility of an LLM to perform a task; other times, all you’re trying to do is pull specific revenue numbers or create a specific chart. I 

Reduce context size

Just because an agent can read your entire codebase, database schema, Slack history, or documentation repository doesn’t mean it should. In most cases, only a small portion of that information is actually relevant to the task. If someone asks for revenue by customer, the model probably doesn’t need access to every table in your warehouse. Build retrieval into the process so the model gets the handful of tables, files, or documents it actually needs. You’ll reduce token usage and often get a better answer because there is less irrelevant information for the model to reason through.

Stop repeatedly sending the same context

Another easy way to burn through tokens is sending the same instructions, schemas, documentation, and examples with every request. A lot of agent systems do this without anyone really noticing. If your system prompt is several thousand tokens and every request includes the same database definitions and coding guidelines, you’re paying to process that information over and over again. Use prompt caching where it’s available, and think about what information actually needs to be resent versus what can remain relatively static. 

Limit output tokens

More output isn’t necessarily better output. If you ask an agent to explain every decision it makes, generate multiple alternatives, and return several thousand tokens when all you really needed was a SQL query or a short answer, you’re paying for a lot of unnecessary work. Give the model a clear expected output. If you need a query, ask for the query. If you need three fields extracted from a document, return those three fields. Your agents don’t need to write an essay every time they complete a task.

The goal shouldn’t be to minimize AI usage at all costs. It should be to use AI where it provides leverage and rely on cheaper, more predictable approaches everywhere else.

Vendor Sprawl Happens Fast

There are so many types of sprawl that occur inside of companies. Ontology sprawl, dbt sprawl, dashboard sprawl, etc., etc. These problems have existed for a very long time.

But this isn’t just an issue that impacts individual contributors. Companies deal with these issues as well, specifically with vendor sprawl. These companies will often have multiple solutions for every problem.

Need a data platform?

Don’t worry, your company likely has Snowflake, Databricks, Palantir, and BigQuery.

Need a dashboard?

Well, they also have Tableau, Power BI, Omni, and Metabase.

And guess what, we are seeing the same thing with LLMs and harnesses. Companies seem to be picking every tool. OpenAI, Claude, Gemini, Cursor, Copilot, on top of their own internal agents.

Between the actual management overhead of all these solutions and the contract minimums, there are millions of dollars wasted.

This problem hasn’t shifted. It’s why many of our projects are migrations and consolidations. There is a huge benefit in consolidating your solutions. It helps create a single source of truth that many teams can rely on, reduces costs to the business, and creates less noise internally.

If Snowflake and Databricks are both there, what is each one responsible for?

If you’re paying for Claude, OpenAI, Gemini, Cursor, and Copilot, what does each one do that the others don’t?

If there isn’t a clear answer, you probably don’t have a technology strategy. You really are just buying things the same way some people buy books. You enjoy the idea of what your team could do with said technology or solution. But actually doing it is a very different thing.

AI is going to make this problem worse because the barrier to adopting another tool is so low. The companies that manage this well won’t necessarily be the ones with the most AI tools. They’ll be the ones that are clear about which tools they use, why they use them, and which ones they don’t need.

Access and Permissions Get Harder

One of the challenges that will likely remain for a while, unless companies get very comfortable with AI becoming the gatekeeper, is handling permission control for documents, systems, and data.

For example, let’s say you have an analyst on the HR team. Should they be able to access payroll day one? What about immigration data? Recruiting data? And every other possible slice of data?

Should all those various slices be approved for the same request? Or are they all different?

As much as companies want an easy button here, this is a hard problem. Are there ways you can make it simpler? Sure. But as stated above, until you’re willing to let AI make decisions about who should have access to what, someone still needs to define and maintain those rules.

An analyst might only have access to certain tables, but what happens when an agent can search across documents, databases, Slack, and internal tools? AI makes existing permission problems much more consequential.

Have you ever tried using an LLM to write code. They love trying to read env files if you don’t tell them to stop. They need help to understand where their permissions stop. That way they don’t accidentally expose information to someone who shouldn’t see it. Whether that be via taking your data in for their training data or providing access to dashboard info that the end-user shouldn’t see.  As agents become connected to more systems, identity and permissions are going to become an increasingly important part of AI infrastructure.

Measuring ROI Is Harder Than Buying the Tool

“We gave everyone Copilot” isn’t an outcome. Are engineers shipping faster? Are support tickets resolved faster? Are analysts answering more questions? Is quality improving? Companies need to measure impact at the workflow level. 

Agents Need Guardrails

Once AI starts taking actions instead of just answering questions, retries, spending limits, approvals, tool permissions, and stopping conditions become important. Otherwise, an agent can make a cheap task extremely expensive – or do something you didn’t intend.

Data accessibility matters too

An agent can only be useful if it can actually reach the systems and information needed to answer the question.

Maybe you think this is obvious. But this is one of the reasons companies with messy data infrastructure will struggle to get as much value from AI as they expect. If customer information is spread across five systems, definitions differ between teams, and important context lives inside random spreadsheets and Slack threads, an agent inherits all of those problems.

Final Thoughts

Every new technology provides a whole host of new opportunities for businesses to find value. 

But it’s very easy for costs to sky rocket. We’ve seen it in the era of pay per use and cloud solutions.

And even before that with companies signing far too many technology contracts. No one wants to be left behind so they have to buy all the solutions.

It makes sense to a degree, but if you’re not careful, you start focusing too much on the how and not enough on the why.

If you’re reading this. You’re likely a business. You provide a service or a product. The goal is to do that, make profit and create happy customers.

If AI can help, great. If it can’t then don’t use it.

Also! Don’t forget to check the articles below.

ETLs vs ELTs: Why are ELTs Disrupting the Data Market? – Data Engineering Consulting

NetSuite to Snowflake Integration: Ultimate Guide to 2 Effective Methods

Bridging the Gap: A Data Leader’s Guide To Helping Your Data Team Create Next-Level Analysis

The Data Engineer’s Guide to ETL Alternatives

What Is Snowflake – Breaking Down What Snowflake Is, How Snowflake Credits Work And More

Explaining Data Lakes, Data Lake Houses, Table Formats and Catalogs

How to cut exact scoring moments from Euro 2024 videos with SQL

How To Modernize Your Data Strategy And Infrastructure For 2025