The case for embedded AI in government

AI is reshaping how work gets done in institutions, both public and private. However, the impact is uneven—consumer AI chat interfaces like ChatGPT, Copilot, Claude, and Gemini are fundamentally mismatched to the realities of government work.

That doesn’t mean government agencies aren’t turning to AI. They cannot hire their way to capacity, so they’re looking to technology to lighten the load. More than half of local governments report difficulty filling positions, a problem especially potent in larger metros. San Francisco’s local government, for example, has more than 4,700 open positions. Since 2020, state government employment has increased, but much of that is a bounce-back from the pandemic, not true growth needed to deliver services with the efficacy governments want.

But drop-in chatbots can’t make a significant impact on operations because data within government agencies—and even within individual departments in agencies—is exceedingly siloed. State and local governments are managing budget constraints. IT teams are stretched thin. It’s no surprise, then, that consumer AI tools don’t meet their promise in government institutions. They fundamentally lack all the information they need to be effective in a public service context.

THE TOOL GAP

Commercial software tools, including AI chatbots, are built for private companies with hierarchies, contracts, and linear processes. Government work is inherently different. Public institutions are cross-organizational, the work is omnidirectional, and success necessitates constant collaboration across agencies, nonprofits, and community partners.

Emergency management departments, for example, must identify points of contact within local police departments, fire departments, sheriff’s offices, emergency medical services, utility companies, FEMA regional offices, and community emergency response teams to effectively deliver their service—coordinating disaster response and recovery operations.

The sum of these problems—from data siloes to budget constraints to technological roadblocks—is that consumer AI struggles in government institutions because these tools lack the necessary context. Try asking ChatGPT “Provide our current reimbursement process” or “Create a survey to gather feedback on the latest heat season coordination.” These are tasks that require explicit, non-public knowledge. To execute such tasks, AI needs not only access to data that lives across disparate systems, but also governance and rules specifying which definition is best suited for each purpose. And even if you get a good answer, traditional AI is not built to help users know what to do next.

THE PATH FORWARD

Embedding AI is the path forward to deploy AI in government, effectively overcoming data silos to tackle inter-organizational work. Embeddedness refers to AI that lives directly within the platforms where work happens, within coordination, memory, and decision-making systems—not inside a chatbot.

Government is uniquely suited to benefit from AI that’s truly embedded because it has the perfect raw material to make AI maximally useful: conversations, decisions, workflows, institutional memory, and loads of information. In other words, government has context. Unfortunately, most AI chatbots today assume their users work inside of a single organization, and therefore only need context from one organization’s systems.

For government, that’s not the reality.

A homelessness coordinator in Essex County or an election administrator in New York isn’t just working “at” one organization: they’re juggling relationships, meetings, decisions, and shared knowledge across constantly shifting agencies, nonprofits, contractors, and community partners. The hardest parts are aligning, coordinating, remembering, onboarding, and maintaining shared understanding across people who come and go.

Because government work is inherently lateral, true embedded AI has information not only across one organization’s systems but also across its partners’ systems, too. Taking the idea of embeddedness one step further, it’s vital for AI to seemingly “disappear” into public servants’ workflows. If AI is one more tool, one more thing government employees must be trained on, then AI is notembedded in the way it needs to be.

Public servants don’t want or need another new widget. They need technology that helps them do what they do faster, better, and more accurately. That is embedded AI.

Consider this example: It’s common for certain AI tools to send a summary and action items after a meeting. But embedded AI goes further. It would ask the user if they wanted a follow up email to socialize action items. It could draft the follow-up email, too. Then, after a couple of days, the embedded AI would surface engagement data, showing that someone assigned an action item hasn’t read the follow-up email, and the AI would ask the user if they want a follow-up email, and then draft the note.

When a public servant is searching for information, the next step is usually to read it to send something to somebody or take an action. AI that supports that next step without explicit prompting or forcing the user to switch tools is the kind of technology that can help public servants better deliver services more quickly. This is the shift that accelerates action.

THE END GOAL

It’s vital we do not lose sight of the goal. State and local governments are where the rubber meets the road, turning rules into real services—from handling daily operations to helping low-income people get nutrition assistance, providing veterans services, supporting people experiencing homelessness, and everything in between.

Yes, chatbots and similar tools can help one person be more effective. But when AI is embedded, benefits are magnified; people and entire programs are faster and more effective. That means the people who rely upon municipal government services are better served.

And that is the ultimate promise of technology, like embedded AI efficiently deployed in government—better communities for all.

Madeleine Smith is the CEO and cofounder of Civic Roundtable.

View the full article