This is a devlog! I work on a bunch of random software projects on my own time and the devlogs are a way to keep track of and talk about what I’ve done, because the one thing the world needs is MORE CONTENT.

Devlog: opencode-energy-logger

I’ve been using AI (LLM agents) to code for a couple months now. I got started with them at my previous job, and was probably the last to start. Everyone was diving head first into it’s use, and at some point I gave up my resistance and figured I should at least be aware of what it actually does, rather than take second hand accounts at face value.face value.

While I’m generally an AI (and big tech) skeptic, it’s pretty obvious from my 4-5 months of using it that AI agents with frontier models (like Opus 5.5 or GLM 5.3) can code quicker and with more accuracy than I can. While they can make mistakes they tend to make less of the stupid kind humans make, aka the kind that can cost you hours just to realize there was a typo somewhere. And in the case they do make a mistake, they can correct themselves very quickly.

But more significantly, frontier models far better at reasoning about complex systems than I am. A human can’t possibly fit an entire complex system in their head and reason about it methodically, exhaustively looking for bugs and vulnerabilities (I’m assuming anyway). In contrast, an AI can do this, and do it rather quickly. Which isn’t to say that these issues that AI’s are finding would simply never be found by humans before AI. Some of them would still have been found and fixed, but over the course of weeks or months as testers and users sent reports in, rather than before a release. So AI here isn’t just speeding up how quickly a developer can spit out some code, but speeding up how quickly a project or feature can get to stable. This effectively means users get a better quality product right away, rather than down the line.

These benefits, however, don’t make AI use mandatory for development (in my opinion at least). In fact, if this was the end of the story, it really wouldn’t be a big deal to do everything slower, by hand. Users will still get a great product eventually. The thing that changes this, however, is the reality that adversaries and threat actors also have access to AI.

These folkle can use AI to find vulnerabilities in your imperfect, hand written product and have no interest in reporting them. Instead they will exploit them to your users’ detriment. Knowing this and releasing software that I know could be exploited by AI is massively irresponsible.

Which means if I want to creating software that other people will use, I have to use AI to better ensure my users are safe when using it.

But of course AI is compute intensive and climate change is now an inevitability that will only get worse with unrestrained energy usage. Using AI without limits or introspection (ie, so-called tokenmaxxing), then, would also be massively irresponsible. So how do I use AI to ensure my code has as few bugs and vulnerabilities as possible, while also ensuring my AI use has as little impact on the planet and environment as possible?

I think there have been a few orgs out there who’ve been thinking about “responsible AI use”. Well, only one that I know of (Debian), but I’m sure there are more. But I have not actually seen anyone define what that means, which one would think is an important thing to do. For my own definition, I thought about what I actually value in the software development process and what I couldn’t care less about.

What I don’t care about:

  • automating every single part of the process: putting every part of the SDLC into an automated AI job is definitely not something I want to do. Apart from it being an incredible waste of energy, it also removes human accountability for what is created. I feel responsible for what I deliver and part of that means knowing exactly what it is I’m giving to my users.

  • coding in natural language: the new, so-called “spec driven development” approach to using AI is also not something I am interested. In this workflow, developers create a high level specification that is fed into an AI agent, either manually or in some automated manner, and the code changes that are output are just included into a release (assuming the AI controlled CI passes). Again, the lack of knowledge about what I’m responsible for is not attractive to me here.

What I do care about:

  • being fully transparent in my development process and how AI use fits into it
  • being fully transparent in how much energy was consumed by my AI use
  • minimizing energy usage while maintaining a baseline level of quality
  • including human oversight over every part of the development process

This is what I concluded responsible or conscientious use of AI during development is for me, and how I want to work going forward: providing users, or whoever else is interested, with as much insight as possible into my use of AI and providing myself with a way to analyze and reduce my AI usage.

My solution to providing this transparency and analytical capability is the project I want to talk about in this post: opencode-energy-logger.

This is an opencode plugin that logs every prompt made during a session along with the energy expended to fulfill that prompt to files that are committed to a git repo.

The logs, stored in a .energy/ directory, provide anyone that’s interested insight into how AI was used, not just for the entire project, but for each individual change, each feature, each bug fix, etc. And it provides the data necessary for me to understand how I can reduce that AI usage (along with a small viewer app that uses duckdb and perspective.js that lets someone explore the data).

Granted, this doesn’t necessarily tell the whole story. It’s certainly possible to use AI for a project, without it being directly involved in the coding process. For example, I could create a small development only tool with AI that I then use while developing. That AI use would just disappear with no record. But having something is better than having nothing, especially given how contentious an issue AI is at the moment, and the logs this plugin will generate will tell a large part of the story.

Ok, so the first part of this devlog is done, describing what I made and why I made it. What follows is some reflection on my use of AI while making this plugin.

Reflection: How exactly was AI used to make opencode-energy-logger?

My current development process is:

  • human prompt for a change
  • AI agent researches and generates code
  • human code review of the generated code
  • iterate until there are no issues found:
    • AI code review
    • human decision on issues that need fixes
    • AI changing the code according to the fix instructions
    • human review of these changes
  • human review of entire change

Once no new issues are found, the change is complete and can be merged with a high level of confidence in its correctness and quality.

Note that this process only works with frontier models released this year. Earlier models are not nearly as accurate enough in their code generation and analysis to be trusted (imo).

One thing to clarify, I said above that the only reason I think AI is necessary is because of folkle who will use AI to find vulnerabilities. If this is true, then I could conceivably just use it to find bugs, and code everything else by hand. But I don’t, so what gives?

From my previous experience, coding by hand is just error prone. I get a couple things wrong, usually small, stupid things, but my brain doesn’t think in code so it’s inevitable. An AI review will catch these issues, but having to do multiple AI reviews rather than just letting the agent write the code once ends up using more resources.

Reflection: Was using an AI agent useful and how so?

I can say with certainty, just as I would of my previous AI use, that using AI here was very helpful. It greatly sped the process of creation up, both in terms of researching technologies that I wasn’t familiar with and in avoiding bugs.

Specifically, it helped with the following:

  • determining how an opencode plugin is structured
  • what functionality is available for an opencode plugin (in v1 vs v2 of the plugin API)
  • how Neuralwatt provides its non-standard energy data in API responses
  • how to setup and glue duckdb together with perspective.js
  • how to write a next.js app (next.js is a framework I hadn’t used before, which I intentionally picked to gauge how well I could learn about a framework with AI assisted coding)

For all of these things the AI was able to find answers on its own and generate working code based on those answers. And for all of these things, save one, I was able to learn by looking at the generated code. And finally, for all of these things, save one, that knowledge is something I would not consider to be useful for me to have and retain.

As an example, take the first item about an opencode plugin’s structure. This information is unlikely to be useful to me in the future unless I end up creating another plugin. However, if I do end up working on another opencode plugin, it would pretty easy to regain that knowledge.

The one thing that in this list that would actually have been valuable to know is one part of Neuralwatt’s API. Neuralwatt delivers energy data as a non-standard property in HTTP responses and as comments in streaming responses. Streaming responses use Server Side Events, which is part of the HTML standard, and is something I was not aware existed before working on this.

I also did not actually look into it when reviewing the generated code and only looked into it while writing this post. So if I hadn’t written this post I would not have ever learned about it. Whereas, had I coded this by hand, I would have been forced to learn about it.

Additionally not looking into this makes my own code review less reliable. If I don’t know how a technology used works I can’t reliably sign off on the generated code.

Reflection: What changes to my process will I make going forward?

The mistake I describe above, about not looking into a technology used by the AI generated code, is one I need to fix. I need to be more thorough in reviewing code that is generated, even if what’s generated is most likely without issue.

Using AI makes it a lot easier to be lazy and NOT care about how a thing works, as long as it looks like it works. That ignorance creates a fundamental disconnect in what I as the creator intend and what is actually committed. Which is unacceptable.

I’ll be modifying my review process a bit: instead of just looking over the code I’ll also be writing down every question that comes up while reviewing and forcing myself to answer before moving on. I’ll probably build this into the utility app I use to review changes to a git working tree.