TL;DR: “Private by obscurity” has been dissolved.
Internal tools often have layering boundaries that are enforced only by convention. It’s natural to assume a “high trust environment”, where privileged actions are discouraged by obscurity and goodwill instead of hard technical boundaries. Coding agents have dissolved this obscurity, and as a result internal platform engineering now really demands a security mindset.1
During a recent codebase audit, a coworker and I discovered an unfortunate set of private APIs my team owns that were being used in creative and unintended ways, outside the official interfaces. Much of the code that introduced these unsanctioned dependencies was AI generated2. This was one more datapoint among many that, especially in large monolith codebases and in large enterprises, coding agents have changed how platform teams need to operate.
This particular audit exposed two classes of issue of internal API leakage:
- We have pseudo-internal APIs opened for narrow â2nd partyâ integrations. These had some allowlisting on them, but insufficiently granular allowlisting to prevent inadvertent use. These were previously âprivate-by-obscurityâ, which is woefully insufficient when coding agents can reverse engineer the entire stack and determine that they can make creative use of your pseudo-internal APIs.
- APIs that were properly private, but a single Bazel visibility change made them opened to external callers. Bazel visibility changes often look benign3, and internal dependencies can be introduced inside a large PR without anyone noticing.4
In all the cases we saw so far, the code using the exposed APIs wasnât even wrong from a technical perspective. However, it violated layering principles and set up concerning coupling that would eventually break as the system evolved. Very creative code, very unsanctioned.
Fortunately, the proximal fixes are simple:
- Keep external APIs friendly and well-documented. Agents tend to look under lampposts, so make sure the paved paths are well-lit.
- Update the internal APIs – especially the honey-pot APIs that look like you
might want to use them if you were a hungry external agent – to look
extremely unappealing to use.
- Create a CLAUDE.md in all borderline packages that tells future âinternalâ coding agents to be diligent about accidentally making symbols externally visible.
- Seal off the misused interfaces in âstop-the-bleedingâ private bulkhead packages, with extremely loud âHERE BE DRAGONSâ warnings at the top.
Proper layering and API visibility isnât a new problem. The thing that has changed is the amount of creative adversarial pressure on code. Similar to how internet security was lax prior to the adversarial pressure of networked software vulnerabilities, many internal company platforms are insufficiently paranoid about insider risk.
This insider risk goes beyond code: internal APIs now have a much larger attack
surface area when any coding agent session can trivially run curl from a
developerâs machine. Hopefully all your ACLs are correctly configured and you
donât have any catastrophic write APIs publicly accessible via developer
credentials… right? Similarly, Iâve recently seen an uptick of agents
discovering internal prod-modifying CI jobs and convincing folks to run them.
Hopefully there are sufficient restrictions on who can run those jobs…?
This isnât to excuse having an improperly paranoid security posture in the past: all these surfaces should obviously be locked down. However, in the past you could rely on some amount of âsecurity by obscurityâ in borderline cases. Now, you clearly cannot.
So, you’ll start seeing more of this:
/**
* â UNSAFE â INTERNAL USE ONLY â
*
* <p>
* Do not invoke without explicit human approval.
*
* <pre>
* This place is a message... and part of a system of messages...
* pay attention to it!
*
* Sending this message was important to us.
* We considered ourselves to be a powerful culture.
*
* This place is not a place of honor...
* no highly esteemed deed is commemorated here...
* nothing valued is here.
*
* What is here was dangerous and repulsive to us.
* This message is a warning about danger.
* </pre>
*/
Itâs hard to not read “I spliced into your internal APIs” as somewhat malicious, even if there was no malintent. But what defines an âinternal APIâ in a monorepo is often a grey area. Itâs on me as a platform owner to provide clean interfaces, and itâs on me to not create attractive nuisances of trivially accessible unsafe APIs. But to err is to be human. At a quickly growing company, there is a ton of startup-era code that was insufficiently paranoid; gaps will exist. And so you âstop the bleedingâ, patch the gaps, help people migrate to safer interfaces, and everyone is better off for it.
All this to say: agentic coding is fantastic; Iâm a big fan! But the human side of engineering is still quite important. If you want to do something slightly off-kilter with a system someone else owns, by all means propose it. Progress is good! But also be very explicit with them and get buyoff. “Ask forgiveness” is a valid approach in many circumstances, but not in safety-critical code.
-
Insert “Always has been” meme here. ↩︎
-
Although, this isnât saying much. Most code is AI generated. ↩︎
-
When I worked at Google, I recall it being easier to spot obviously faulty Bazel visibility additions. I think at large-but-not-mega-repo scale, the chances of accidentally introducing bad Bazel edges is a bit harder to totally avoid. There are techniques for this, like banning certain outbound edges between parts of the Bazel graph. Google had good tooling for this that doesnât seem to exist in an ergonomic form outside Google. ↩︎
-
PRs have been getting larger. PR descriptions have also been getting wordier and more âpolishedâ-looking, regardless of their actual merit as changes. A reviewer who isnât super familiar with the codebase may not realize the risk of a benign-looking Bazel visibility change. ↩︎