Should we be writing software for humans or models?

It's Monday morning, you had a great weekend and you're set for work. You open a repository, find a variable called retention and wonder what on earth it means. Days? Hours? A string like 15d? You run git blame to find the culprit and it is you, three years ago, with a commit message that says "fix". (Guilty. I write a lot of those.)

I have written a lot of Ansible roles, Terraform modules and Go packages over the years, and I would love to say I remember why each of them works the way it does. As good as my memory is, I don't. At best there is a README I wrote by hand on the day I finished, which has been quietly drifting away from the code ever since.

For a long time this was a tolerable problem. The only people reading the code were us, and we could always walk over to the person who wrote it (or, more often, ask them on Slack). That is no longer the case. These days most of the automation we write starts with a model doing the typing, and the model reads our shared libraries constantly. It doesn't skim, it doesn't get bored and it can't ask me what I meant. It takes whatever the interface says and runs with it.

That raises an awkward question. Should we now be writing software for models rather than for humans?

What did we find when we audited our own repositories?

We recently raised an internal ticket to standardise the interfaces of our Ansible roles and Terraform modules so that a model could use them properly. Before changing anything we audited two of our public repositories, the digitalis-ansible-collection and the AxonOps Terraform provider, to see how far off we were.

The results were humbling, in the way audits usually are.

• The Prometheus role had 13 options with no type at all, including prometheus_storage_retention. You can guess how that one goes.
• Plenty of variables used in defaults/ were never declared in the argument spec, so nothing validated them.
• Many descriptions were under 40 characters. They told you the variable existed, which you already knew because you were looking at it.
• The README variable tables were written by hand. Nobody could say with confidence that they matched the code.
• The Terraform provider, thanks to terraform-docs, was in better shape, but not by much.

This is what happens to every codebase written by people under time pressure and read mostly by the people who wrote it. The difference now is that the new reader is much less forgiving.

How does a model read code differently from a human?

A human reading a README skims, fills the gaps with experience, and asks a colleague or reads the source if something looks odd. A model does something closer to the opposite. It reads everything you give it, takes it at face value and fills the gaps with whatever is most statistically plausible. If your description says "Retention." it will pick a value that looks like a retention, and there is a reasonable chance it will be an integer when your template expects 15d.

That is not the model being stupid. It is doing exactly what a new starter would do with the same information, only faster and with more confidence. The interface is the contract, and a vague contract gets you a vague result.

So the useful question is not "humans or models". It is "what does the reader need that we never bothered to write down?"

Look at that table for a minute. There is no row where the model needs something a human would not also benefit from. The model just needs it more urgently, because it can't fall back on memory or on asking around.

What does writing for a model look like in practice?

It is the same discipline we have always preached, but enforced.

Ansible: a typed argument spec for every variable

Every variable lives in meta/argument_specs.yml with a type, a default or a required flag, choices where the set of values is closed, and a description that says when to use it, what the units are and what happens when you change it:

argument_specs:
  main:
    options:
      prometheus_storage_retention:
        type: str
        default: "15d"
        description:
          - How long Prometheus keeps samples on local disk.
          - Uses Prometheus duration syntax such as 15d, 12w or 1y. A bare number is rejected.
          - Lowering it deletes older blocks at the next compaction. Changing it restarts the service.

The nice bit is that Ansible validates this at runtime. If the model (or I) pass an integer, the play fails before it touches a server, with a message that says exactly what was wrong.

Terraform: precise types, validation and nullability

Terraform modules get the same treatment: a description on every variable and output, precise types instead of any or map(any), and validation, sensitive and nullable where they make sense:

variable "backup_retention_days" {
  description = "Days to keep automated snapshots. 0 disables backups. Changing it does not remove snapshots already taken."
  type        = number
  default     = 7
  nullable    = false

  validation {
    condition     = var.backup_retention_days >= 0 && var.backup_retention_days <= 35
    error_message = "backup_retention_days must be between 0 and 35."
  }
}

Go providers: test the schema itself

For a Go provider the equivalent is the schema, so we check it with a plain Go test that walks every resource and data source:

// allAttributes() and requiresReplace() are small helpers, not shown here.
func TestSchemaDescriptions(t *testing.T) {
	for name, attr := range allAttributes(t) {
		d := strings.ToLower(attr.GetDescription())
		switch {
		case d == "":
			t.Errorf("%s: missing description", name)
		case d == strings.ReplaceAll(name, "_", " "):
			t.Errorf("%s: description only repeats the name", name)
		case requiresReplace(attr) && !strings.Contains(d, "replace"):
			t.Errorf("%s: forces replacement but the description does not say so", name)
		}
	}
}

Any shared library: types and validation at the boundary

The same idea applies to any shared library, not just infrastructure code. In a Go package that means typed option structs instead of a map[string]interface{}, and godoc that states the units, the defaults and what happens on error. In Python it means type hints on every public function, docstrings that say whether timeout is seconds or milliseconds, and validation at the boundary (pydantic or plain checks), so a bad argument fails loudly on the first call rather than quietly three layers down.

On top of that, README tables are generated from the spec with aar-doc and terraform-docs (or tfplugindocs for providers) rather than typed by hand. Each repository gets an examples/ directory with two or three realistic scenarios rather than one snippet per resource, and a short CLAUDE.md with the conventions, the test commands and the things you must never do. We wrote more about how we use CLAUDE.md in AI-assisted development in an earlier post.

Which lessons from the review were not obvious?

A few things came out of the review that I didn't expect.

Don't enforce description length. It is tempting to add a CI rule that says every description must be at least 80 characters. Don't. You will get 80 characters of padding, written by a model that has been told to make the linter happy. Check that a description exists, that it is not just the variable name, and that it mentions replacement when the attribute forces it. Leave quality to code review.

Two sources of truth will drift. In Ansible, the default in argument_specs.yml is there for documentation. The Ansible docs are clear that the value actually used always comes from the role's defaults/main.yml. Give it a few months and the two will disagree, and the model will believe the wrong one. A small script in CI that compares them catches more real problems than any rule about prose.

Being precise can break things. Adding a type to a variable that never had one is a breaking change in disguise. If you declare prometheus_storage_retention as an int because that is what you assumed, every existing inventory that sets it to 15d stops working. Derive the types from how the templates actually use them, and run the existing scenarios before merging. The same goes for renaming or retyping provider attributes, which breaks people's state.

So, humans or models?

My answer is models, though not for the reason I had in mind when I opened the ticket. There is nothing new on that list. Types, units, a couple of examples, a line saying "this restarts the service". We have been telling each other to do this for twenty years and mostly we didn't, because writing documentation is boring and nobody ever got a pat on the back for it.

What has changed is who does the writing. Most of the new variable descriptions in these repositories were written by the model itself. I point it at a role, it reads the templates and the handlers, works out that prometheus_storage_retention is a duration string and that changing it bounces the service, and drafts the spec entry. I review it, fix the odd thing it got wrong and move on. When someone adds a new variable next month the same thing happens again, and if they forget, CI won't let the PR through. Keeping the docs up to date used to be the chore we left for "after the release". Now it is a few minutes of review.

So the model writes the documentation, reads the documentation and uses it to write more code, and I'm mostly there to mark its homework. I can live with that, as long as in three years' time I can open one of these repositories and finally know what retention means. If you are weighing up the other half of this question, our post on choosing a language when the AI writes the code covers it.

Note to future me: yes, you wrote this too.