Cities everywhere are trying to find better ways to answer resident questions, reduce staff overload, and make policies easier to navigate. Chatbots powered by AI hold a lot of promise—but there’s a real decision to make: do you build one in-house or buy something that’s ready to go? 

Let’s break it down, honestly.

‍

Building In-House: Full Control, Full Responsibility

Some cities have the right setup to build their own solution. Maybe you’ve got a tech-forward innovation team, or your IT department wants full control over data and design. That’s great, and there are clear upsides.

‍

👍Pros:

• Tailored experience: Full control over user flow, branding, and integrations.
  
  
• Custom training: You can define exactly what documents, policies, and data sources to use.
  
  
• Internal capacity building: Great learning opportunity for in-house IT team or innovative city staff.
  
  
• Lower platform costs: You will still need to pay for credits from the LLM / Custom ChatGPT you use, but these will likely be lower than the cost of an off the shelf AI Chatbot Tool (if that’s all you need).
  
  

👎Cons:

• Time & complexity: Building a chatbot isn’t just about writing code, it involves training large language models, designing intuitive UX, and maintaining source integrity.
  
  
• Risk of hallucination or misinformation: Even NYC faced backlash when its bot gave incorrect legal advice to residents and business owners.
  
  
• Upkeep costs: Ongoing hosting, content updates, security patches, and staff training are required.
  
  

Tips from our team if you build:

The most important aspect of your chatbot is accurate answers. As seen with the NYC City chatbot, providing inaccurate responses, or hallucinations, can not only steer residents in the wrong direction, but can open up your city to legal issues. Our team mitigates this risk by using a Retrieval-Augmented Generation (RAG) to ground answers in your own policies. If you are building your own bot, be sure to specify that it must only give answers based on the documents you give it to train on, and ideally every response has a source link or reference.

Also:

• Set up a feedback loop with department staff so your bot reflects how work actually gets done (even if it’s a simple feedback form)
  
  
• Add guardrails for when the bot doesn’t know an answer (What response does it give? How can it direct users in the right direction to find it on their own?)
  
  
• Assign clear ownership for maintaining and documenting your AI tool (model updates, data upkeep, billing tokens, etc.)
  
  

Buying a Proven Tool: Less Risk, Less Control

Many cities don’t have the bandwidth to build from scratch. That doesn’t mean they can’t lead on innovation. A well-designed AI tool built specifically for local governments can get you live faster, with fewer surprises.

Tools like Ordinal Connect offer plug-and-play AI powered by your city’s documents and policies. They’re often designed specifically for small to mid-sized cities without large tech teams.

👍 Pros:

• Fast setup: Ordinal Connect can be live in days without IT lift.
  
  
• Reliable answers: Cites the exact reference from your city code or policy.
  
  
• Dedicated expert upkeep: You get an entire team’s dedicated time to upkeep and improve the chatbot your city is using (so you get AI engineers without the cost).
  
  
• Public + internal use: Staff and residents share the same trusted source of truth.
  
  

👎 Cons:

• Less UI flexibility: You’re using their framework (although most offer light customization).
  
  
• Recurring cost: Subscription-based pricing, although often cheaper than FTE time.

What to look for in a good vendor:

• Answers grounded in your own city documents, codes, and plans
• Transparent, verifiable responses so staff and residents can trust the tool
• Ongoing support and updates
  
  

‍

The Hybrid Option

Some cities build internal tools but still license external chat tech. Atlanta’s ATL311 bot was built on a commercial AI platform (Zammo AI), others are using a custom ChatGPT (which is risky unless done well) – but still required significant staff time and seems to carry branding from the third-party provider.

‍

Quick Decision Guide

Bottom Line

We’ve worked with cities that tried both routes. Some built great tools in-house. Others started that way, hit delays, and switched to something pre-built to serve their residents faster.

You don’t have to make this a forever decision. What matters most is giving residents and your staff clear answers and freeing up your team’s time to focus on what only your people can do.

If you’re not sure where to start, we’re happy to show you what’s possible and how other cities have handled this exact choice. Just ask!

‍