I've been using AI tools seriously for the past couple of years: mostly ChatGPT and Claude for writing, research, and content work, and Claude and GitHub Copilot for programming. Most recently, I gave OpenCode a shot at implementing a larger set of features for an internal application. That range of use has given me a clearer picture of what these tools can actually do, where they can be genuinely helpful, and where they fall short.
What AI Tools Are Actually Doing
The first thing to understand is that these LLM tools are not magic, and they're definitely not sentient. They're just exceptionally good at pattern matching. These models have been fed a huge amount of content, and they're just predicting, letter by letter, what the next most likely letter is based on what they've seen before. That training makes them very good at generating plausible-sounding output that looks and feels like the real thing, and they've been improving rapidly over the past couple of years. The tools have gotten better at not hallucinating - just making good-sounding stuff up - but they still do it sometimes, and they sound just as confident about that as they do about correct information.
That confidence is a big part of the problem. You need to know enough about the subject to evaluate what you're getting, because the tools will give you something that looks right even when it's wrong. That means these tools work best when you're already reasonably competent in the area and are using them to move faster, not when you're hoping they'll do work you couldn't do yourself.
Where They're Genuinely Useful
For writing and content work, I've found them to be the most useful for getting off a blank page. Using an AI tool to take your one-sentence idea and help you outline and produce a first draft of something gives you a reasonable place to start from. It's a helpful brainstorming tool, too, generating options you can work off of. The AI draft is rarely publishable as-is, but it's a useful skeleton to build from. I've found myself having to take less and less of a red pen to the AI's output as the tools have improved, but I still expect to have to do a lot of editing and rewriting to get it up to my standards. I also find them incredibly helpful on the other end of the process, running almost finished stuff through a final pass to clean things up, fix up awkward bits, and flag anything that needs some additional clarification.
For search & research work, AI tools can sometimes be faster than reading documentation or relying on traditional search tools. "How do I configure X in Y software" questions can generally get answered more quickly through a conversation with your AI tool of choice than by navigating support docs — with the significant caveat that the instructions can be outdated (especially for less common or cutting-edge stuff) or just plain wrong. Again, treat the output as a starting point to experiment with, not the definitive answer.
They're also pretty useful for summarizing and reformatting-type work. Paste your meeting notes in and ask for a summary with action items, or take a long manifesto-y essay and ask it to produce multiple social media posts from what you've covered.
My Experience with AI-Assisted Programming
How useful AI has been in my programming & IT work has varied considerably, depending on exactly what I was asking the tool to do.
As a smarter autocomplete, GitHub Copilot is genuinely pretty good. I've been using it in my editor for the better part of a year now, and it's consistently pretty helpful. It fills in boilerplate, completes patterns you've established, and suggests the next logical chunk of code. It's easy to accept what's helpful or to ignore what isn't — and it takes a lot of the grunt work out of writing code. It does occasionally kind of get in the way, suggesting huge chunks of code that are going in the completely wrong direction, but that doesn't happen too often.
As a scaffold tool, asking Claude or Copilot to generate a working starting point and editing from there is useful in the same way it is with actual English writing, except it understands your naming conventions, existing style, and the work you've already done even better. You still write most of the code, but you get a jumpstart on the structure and the details, so you're just not writing the first version of it from nothing.
I recently tried OpenCode, an agentic coding tool, paired with my GitHub Copilot subscription, to build the reporting end of a new website auditing tool I've been working on. I had the data collection pieces in place, but I still needed a flexible, well-designed mechanism for presenting the stuff to clients. That's a non-trivial chunk of work, and it made for a good test case for seeing how far I could push the AI.
It worked — but not the way the vibe-coding gurus make it look. It was very much like working with a capable, book-learning heavy but experience-light junior developer. OpenCode handled figuring out what to do reasonably well, and it produced something that roughly did what I had asked. The friction was in refining the details, especially getting the interface to look how I wanted.
Part of the problem was that I didn't fully know what I wanted until I saw something I definitively didn't want. The AI's first pass was useful in that it gave me something, but getting from "ooh, don't like that" to "maybe this?" to "this one" was often faster to just do by hand. I'd take over, get the relevant piece into a state I was happy with, and then hand it back to OpenCode to continue from that cleaner starting point. That back-and-forth turned out to be the natural rhythm of the whole project.
The AI can take on meaningful chunks of work, but it needs supervised direction. The architectural and design decisions still require someone with judgment, and you have to be willing to step in and do things directly when the results aren't coming together like you want. It's genuinely faster than doing everything yourself in the same way a good junior developer makes a team faster. But "faster" is not the same as "autonomous," and without someone competent enough to review and redirect the work, things can go off the rails pretty quick. Like at one point, it dropped and recreated the application's database attempting to debug a problem... not exactly the kind of thing you want an AI to do without supervision.
Where the Promises Fall Apart
The pitch you'll hear most often is that AI can handle your operations: automate your communications, manage your workflows, answer member questions, and surface insights from your data, largely hands-off. That's not an entirely accurate picture of where these tools are today. They can assist specific, well-defined tasks when a human is supervising the output. But they can't replace the person doing the work.
Custom AI chatbots are a particularly common disappointment. We've seen many a smaller organization deploy them on their websites expecting them to handle common questions, provide information about services, or reduce the load on staff. But in practice, they still tend to hallucinate, especially when asked off-the-wall sorts of questions, and give out confidently wrong information.
AI writing also doesn't always hold up well where voice matters. AI-produced content is pretty recognizable. For writing that needs to sound authentically like your organization, the AI draft is, again, a good skeleton to start with at best. The more your audience knows you, the more obvious a fully AI-generated piece reads.
Data and Privacy
What you put into these tools is worth thinking about before your organization starts relying on them, too. Using ChatGPT to help draft a grant proposal is different from pasting in your full donor list to help generate a report. The specifics vary significantly by tool and account tier: some providers use submitted content to train their models by default, some don't, and some offer enterprise tiers with stronger data safeguards. It's worth reading the privacy policy before you get yourself in the habit of just pasting everything and anything into your AI tool of choice, especially if your organization is handling proprietary information, sensitive client or member data, or confidential financial details.
Further Philosophical Implications
The truth is I'd much rather have a junior developer who can learn and grow over time than offloading all my work to an AI tool. My bigger concern is that if we hand too many of our lower-end development tasks over to AI instead of keeping those tasks in the hands of people who are learning, we risk losing a lot of opportunities for people to learn and grow into the more complex habits of thought that come with experience and mentorship - the stuff that you really need to keep our AI tools on track. That's a big part of why I'm all in on bringing high school & college students into working on real projects with us, and why I think it's a much better use of AI tools to help people learn than to replace them.
How to Actually Start
The most valuable way to get started with AI tools is to pick one specific, semi-repetitive task that you do regularly. Use Copilot as a code autocomplete tool, ask Claude to help you outline and draft a newsletter article, or use an AI tool to summarize your meeting notes and build a real sense of what these tools can and can't do. Then evaluate honestly: did it save time? Did the stuff it produced meet your quality standards? Did it actually free you up to do more of the work you want to do, or did it just add a layer of supervision and correction that ended up taking more time than it saved?
I've settled into a pretty good rhythm of using AI tools to help me get started on things and to clean up the end of the process, but I still do the bulk of the work myself, and I still have to be ready to step in and take over when the AI's output isn't quite right. If you go into it with the expectation that these tools will do the work for you, there's a good chance you'll end up disappointed. But if you go into it with the expectation that these tools can help you get started and polish up the final product, and that you still need to be involved in doing the work, you might find that they can be a pretty helpful part of your workflow.
Figuring out where AI tools actually fit in your workflows — and where they don't — is something we help organizations think through. Here's more about how we work.