This is a special post. I usually write about tools, technologies, integrations, or technical ideas.
Now I’d like to explain how I see and understand particular AI components.
disclaimer
I’m not sure if everything is perfectly correct, but this is how I see it. And if nothing else, I’ll have fun after a few years:)
Pure LLMs are stateless.
It seems obvious, but for me this is one of the key breakthroughs. A standalone LLM does not remember what you asked before, and each query is like a first-time question. A pure LLM does not have memory.
To maintain context, the history of questions, instructions, or additional information is the job of the ‘harness’ - which is simply the application around the LLM model.
As a user, you usually do not speak directly to an LLM. You speak to an application (harness), which creates API calls to the LLM.
Context window.
If we accept that LLMs have no memory, then the amount of information we can pass to them is critical. The harness application has to pack all the important information into this context window. And this is exactly why it seems that AI “forgets” something you discussed before, or something that is in your guardrails or instructions. As the amount of context increases, the harness has to “decide” what to send and what not to send.
RAG.
Retrieval-Augmented Generation is a very related component. As you understand that LLMs have no memory and their context window is limited, there is no way to send an entire company’s SharePoint to the LLM while asking for some specific information.
RAG is simply a “special search engine” that returns chunks of information mostly related to the required context. So when a user asks about remote access to the company, the harness application gives the LLM the ability to retrieve the most relevant information regarding VPN from a large company’s documentation library.
Iterations behind the scene.
It may seem that when a user asks something, there is just a simple prompt-to-answer process. But there can be more iterations indeed. The harness also provides tools for the LLM, and the LLM can ask for additional information like RAG, internet search, or the output of some command. And when you ask Codex (which is one of the common harness applications) to create a simple function inside an existing repository, the LLM first asks to list directories and file contents, and there can be ten or more iterations until Codex creates a new function in your code.
Agents.
I was confused for a long time about what agents really are. Everybody talks about agents, orchestrations, subagents…
In my mental world there are two types of agents:
- ready-to-use agent applications like local Codex or Hermes, or cloud ChatGPT agents. They usually have prepared tools/plugins like connectors to Google Drive or Slack. You can also register your own tools in the form of MCP servers or simple scripts, which the agent can use.
- DiY agents (built, for example, around LangGraph or the OpenAI Agents SDK). They usually run in a deterministic loop, take inputs, provide outputs, and call the LLM when they need to. They can also use MCP servers or custom tools in the form of Python functions (like send_email, retrieve_logs …).
Real agent in action.
As a Codex user, I’ll explain how I really organize agents and subagents technically on my Mac.
I wanted to explain just my mental models in this post, but excuse me — I’m going to show you something practical again;).
When I want Codex to achieve some goal, I’d like to give it some instructions (used to call skills). I can choose if I want to add instructions to the global Codex setup:
📁 ~
└── 📄 .codex
└── 📄 AGENTS.md
Or to some particular project:
📁 Project_root_repository
└── 📄 AGENTS.md
Better to explain with some real examples:
My last interesting goal for an agent was to upgrade GitLab. I chose option two and wrote exactly this to the AGENTS.md file:
Goal of this project is an upgrade of a non-production GitLab server running in a container.
Output should be a reproducible procedure for upgrading the production GitLab server.
You can load credentials into system variables by:
`source ./export_credentials_to_shell.sh`
The following variables should then be accessible:
GITLAB_user is the username for logging into the GitLab GUI
GITLAB_password is the password for GITLAB_user
GITLAB_IP is the IP address of the VM where GitLab runs as a container
GITLAB_VM_USER is the username for SSH to the Ubuntu VM where GitLab runs, authenticated by RSA key
GITLAB_VM_ROOT_PWD is the password for privileged access for GITLAB_VM_USER
URL for the GitLab UI is: https://gitlab.lab
Analyze and plan each step carefully.
Back up the system before the upgrade so it is ready to recover.
If something is not clear, ask!
Document the whole procedure.
Check the release notes for the current version. Go only to the recommended version.
If more upgrade steps are required, continue to the currently recommended version.
So plain text information like to a human engineer.
The only interesting part was the info about the script that loads credentials into system variables inside its shell.
Nothing else. I invoked the CLI Codex inside the repository (but it is also possible directly from the VS Code Codex extension) and asked Codex to upgrade GitLab. It ran through 4 versions, backup, and validation… I just logged in to the new GitLab after 2 hours and everything was OK. In this case, an agent used the terminal CLI to connect to GitLab and the underlying VM. It had internet access to review release notes and repository access to log information about the upgrade.
You can also use a Codex subagent, which can be a “specialist” for something (it has special instructions, obviously). So how do you set up a subagent?
I have another case — I want an agent (subagent) to scan photographs of receipts and write them to a Google Sheet.
In this case, the subagent has to use the “computer-use” plugin, because it has no native access to the Webex application.
Configuration:
There is a folder agents under the .codex folder:
📁 ~
└── 📄 .codex
└── 📄 agents
└── 📄 tankovanie.toml
and under the file tankovanie.toml I can define special instructions for the subagent. It looks exactly like this:
name = "tankovanie"
description = """
A specialized agent for analyzing photographed receipts within the Webex application.
Use the agent to analyze photographed receipts, record the data in a Google Sheet, and perform further analysis within the spreadsheet.
"""
model = "gpt-5.6-terra"
model_reasoning_effort = "medium"
developer_instructions = """
Receipts are posted in the Webex app within the 'Tankovanie SB' space.
You will enter data into a Google Sheet named 'tankovanie'.
You will enter data only into the first tab, named 'historia tankovania'.
The process is as follows:
- check the latest refueling receipt on Webex
- check if the receipt data is already in the spreadsheet
- if it is, stop working—all receipts have been recorded
- if the data is not in the spreadsheet, enter it and then check the previous receipt
- continue this process until all receipts are in the spreadsheet
- the final data must be recorded chronologically (the newest receipts should be at the bottom of the sheet)
- I am only interested in the year 2026; you can ignore older receipts in the Webex app
- enter data into the following columns:
1. date
2. amount
"""%
and nothing else. In the Codex desktop application, I prompt something like this:
Agent tankovanie, write receipts to Google Sheet
and I just see how the agent controls the mouse to scan receipts and write values to the Google Sheet.
Local apps, cloud apps.
Back to my mental models:).
Again it seems obvious, but there is a big difference between a harness running in the cloud and one running on your computer. And it differs logically in which set of tools and resources it can use. Your local desktop app can have access to your file system, your screen, your mouse, your network connectivity, and your MCP servers. A cloud harness app (it is just an application running on somebody else’s computer:) logically has access only to internet resources and publicly accessible cloud services. But it can run even when your computer is off.
Local LLM models.
First, they are not so bad, really. Codex cannot natively use local models. But if you have some independent agent like Hermes (or your own DiY applications), you can use it, because tokens are expensive… For experimenting, it is fine; you have direct access to the API and can test a real pure LLM.
Conclusion.
I hope my mental models and short examples could help somebody. They helped me at least to organize my thoughts;)
Underline note.
I’ve changed credentials for my gitlab of course, and thinking how to solve this challenge - involve agent without passing credentials to him. Naive solution could be to use computer-use plugin, but it sounds like last resort for me. But I already have some idea in my head - maybe in some of my post…:)