Eric J Ma's Website

If you're still on Twitter, you've probably seen the wave of ChatGPT3 tweets. There are rave reviews and there's also a lot of skepticism. I was skeptical, at first, but I decided to give these AI coding tools a try. I happen to be in the privileged position of being an open-source maintainer, so I have access to GitHub Copilot for free. (It costs $10/month otherwise.)

In VSCode, which is my daily driver text editor, I installed the Copilot extension, turned it on for Markdown, Python, and Plain Text files, and started typing away. You get suggestions as you type, and you can accept them by pressing Tab. My conclusion? I'm not blown away, but neither am I disappointed. Rather, I'm starting to realize the subtle value in using these tools, and who would be the best target audience for them.

The value of AI coding tools

Firstly, I think it's important to note that these tools are not meant to replace human coders and writers. Rather, their best use is in augmenting human capabilities. Moreover, I'd argue that the best use of these tools is in the hands of people who have already accumulated sufficient experience. In other words, skilled practitioners.

By skilled practitioners, I mean people who possess the ability of verification. They should be able to verify the output of AI tools for correctness. Without this ability, AI coding assistants may give us incorrect code, or in the case of writing prose, factually incorrect statements.

Yet at the same time, I think there is value to the incorrectness. I saw one LinkedIn post by Cassie Kozyrkov that eloquently expressed where that value lies:

But here's the catch - ChatGPT's responses only touch reality at a tangent. While they may sound convincing, they are ultimately fictional creations of the GAN.

This might sound like a drawback, but it actually makes ChatGPT incredibly useful. Because it isn't tied to the constraints of reality, ChatGPT can engage in completely imaginary conversations and provide creative, out-of-the-box responses.

For example, you could ask ChatGPT what it would do if it could fly, and it might respond with something like "I would soar through the skies like a majestic eagle, feeling the wind beneath my wings and the freedom of flight." This type of response would be impossible for a human to come up with, but it's perfectly within the realm of possibility for ChatGPT.

So, why use ChatGPT if its responses only touch reality at a tangent? Because sometimes, it's exactly this type of creative, imaginative thinking that we need to solve complex problems and generate new ideas. ChatGPT allows us to explore possibilities that are beyond the constraints of our everyday reality, and that can be incredibly powerful.

The internet community has a wonderful property: state something incorrectly, and someone will correct you. Outside of bad-faith actors, it is usually a skilled practitioner will be the one who will do so. They can do so because they possess the relevant background knowledge and analytical and research skills to evaluate incorrect statements and produce a correction. This skill will go a long way in establishing trust with work products that end up in the hands of other humans. Likewise, with reality-tangential (or even incorrect) responses, it is the skilled practitioner who will be able to verify the correctness of the response. AI coding skills won't take away the need for skilled practitioners; rather, I predict that it is skilled practitioners who will find their skills most supercharged, and it will be the skilled practitioners who will become the necessarily trusted partners in a world of automated generation of incorrectness.

The corollary? Never before has it been more important to become a skilled practitioner and to build trust with other skilled practitioners.

Will AI take our jobs? I don't think so. But I do think that AI will augment our skills, and make us more productive. For anyone willing to put in the time and effort to become a skilled practitioner, I can only see these developments in a positive light!