Safeguarding LLM-Assisted Dev at Guardsquare

This post is not meant to tell you how to use large language models (LLMs) or to claim we've found the right approach. As a cybersecurity company working with particularly sensitive IP, our decision to use LLMs for development was never just about productivity. The broader enthusiasm around LLMs was not itself a reason for us to adopt them quickly. For some time, our position was that the risks outweighed the productivity gains, and incidents involving AI agents elsewhere in the industry reinforced that assessment.

The balance of risks to productivity gains gradually shifted as LLMs became more capable. The gains eventually became significant enough for us to justify investing in ways to use them safely.

I am sharing why we made the choice to invest in the use of LLMs and what we put in place as a result. The post is split into two parts:

1. Technical risks and safeguards, addressed in three layers:
   • Reducing the sensitive surface by separating sensitive code wherever possible and accounting for copies in Git and elsewhere on the filesystem.
   • Isolating agent execution as much as possible using development containers or host-level sandboxing.
   • Controlling model access and outbound data through an LLM gateway and guardrail service.
2. Organizational concerns, which includes the effects of LLM-assisted development on technical foundations, critical judgment, and review burdens, together with the policies, processes, and guidance we use to address these issues.

Technical risks

We identified two critical technical risks:

1. An LLM agent could go rogue and harm the developer's infrastructure or data. This has already been demonstrated by others through deleted production databases and wiped local drives, showing why agents should only receive the access they strictly need.
2. Sensitive intellectual property (IP) or personally identifiable information (PII) could be shared with an untrusted LLM provider.

For additional context, the LLM client is the application running on the developer’s machine, while the LLM provider is the external service processing its requests. The following section focuses on the second risk in detail.

IP and PII disclosure

A critical risk is the leakage of IP and PII - an issue that affects most companies to varying degrees. The question is a bit more difficult. First, what is the risk? For Guardsquare, the main risk is exposing sensitive code that would weaken our protections. Any data sent to an external LLM provider creates a potential exposure risk, whether through provider-side training practices or a security breach.

For a cybersecurity company, this risk may be even more concerning than a rogue agent causing an outage. Here again, you don't have to look far to find examples of sensitive data disclosures.

• March 2023: Samsung. Engineers submitted proprietary semiconductor code and confidential meeting notes to ChatGPT, leaking corporate IP to an external service.
• January-February 2026: Microsoft 365 Copilot. A bug allowed Copilot to process and summarize emails marked as confidential, bypassing an organization’s data-loss-prevention policies that protect sensitive communications.
• March 2026: Meta. An internal AI agent published incorrect engineering advice without its user's approval. Another employee followed the published recommendations, resulting in the exposure of privileged company and user data to unauthorized engineers for approximately two hours.
• July 2026: Grok Build. The coding assistant software was silently uploading all complete Git repositories to SpaceXAI-controlled storage, potentially including proprietary code, full commit histories, and committed credentials (regardless of whether the "Improve the model" option was enabled or not).

With all this in mind, we implemented a number of safeguards to prevent any sensitive IP or PII from leaving our control.

Technical safeguards

There's no perfect solution to these problems, except by banning LLM usage alltogether. After careful analysis, we decided on multiple layers of control to limit both the risks of a leak and the potential impacts.

1. Reduce the sensitive surface

The first step was to isolate certain parts of our codebase containing sensitive IP in separate repositories where LLM use would be prohibited entirely. To enforce this limitation and allow safe usage of LLMs in the remaining repositories (which may still contain sensitive IP), we apply an additional layer of protection.

2. Isolate agent execution

After isolating as much sensitive code as possible, two problems remain:

• The agent may explore outside the repository and access sensitive IP elsewhere on the developer's machine.
• Sensitive IP that could not be separated may remain inside the repository.

We mitigate both of these risks by sandboxing the LLM client and its subprocesses. Some clients include their own sandboxes, but their default policies are not strict enough for our needs. Others use a second LLM to evaluate commands before executing them. Needless to say, if we don't trust the first model, we don't trust its evaluator enough to be satisfied with that either, no matter what the benchmark says. We therefore wrap the client in a sandbox that we control.

We also explored and adopted two alternative implementations of this layer: container-based isolation and host-level, kernel-enforced policy sandboxing.

In both cases, restricting the working-tree path is only part of the solution. The same source code may also be available through Git objects, module caches, or package-manager stores. Those copies must either be removed or covered by the same boundary.

Container-based isolation

Containers provide an isolation boundary around the LLM client, giving us control over the files, tools, and system resources exposed to it. We currently use this approach in some of our development environments.

We define the container environment with Nix, including the LLM clients, their configuration, and the tooling required by the repository. We then use Dev Containers to build and start the image, mount the workspace, and apply the required runtime parameters.

Figure 1: The repository is mounted into the isolated development container.

Only the host resources explicitly mounted into the container are visible to the agent, preventing it from freely exploring the rest of the developer's machine.

Access inside the repository can also be restricted. Although Dev Containers normally mount the complete workspace, additional mounts can obscure selected paths while leaving the original files untouched on the host. This follows the standard behavior of mounting over existing container data.

This approach introduces several sources of friction. Most can be mitigated, but each requires additional configuration or changes to the development workflow:

• A Linux container cannot reuse host-native binaries built for a different operating system, architecture, or ABI. This commonly affects native executables and add-ons provided by Node modules when the host's node_modules directory is visible inside the container.
• Some plugin and MCP authentication flows may not work out of the box when their callback server listens inside the container while the browser follows the redirect on the host. Depending on the flow, this may be addressed through port forwarding or a configurable callback address.
• Build directories produced by tools such as CMake and Ninja may contain absolute paths or platform-specific state. Reusing the same intermediate artifacts on the host and inside the container can therefore fail, and separate build directories may be needed.

These constraints led us to explore a second approach that preserves the host development environment.

Host-level policy sandboxing

The second solution wraps the LLM client in a sandbox built from native operating-system security primitives. Although the client still runs as the developer's user, the sandbox imposes additional restrictions beyond normal user and file permissions. When the client or one of its subprocesses attempts a protected operation (such as opening a file, connecting to a socket, or controlling another process), the kernegol evaluates the operation against the sandbox policy and denies it when no rule permits it.

On macOS, Safehouse generates a deny-first Seatbelt profile and applies it through sandbox-exec. Seatbelt is implemented on top of XNU's TrustedBSD Mandatory Access Control framework. On Linux, tools such as Bubblewrap can combine namespaces and system-call filtering to restrict filesystem visibility, processes, and networking. Linux can also enforce policies using eBPF through BPF LSM hooks.

Figure 2: The kernel checks protected operations against the sandbox policy.

Safehouse starts from a deny-first policy, allowing the working directory and the system resources needed for common development workflows. This already limits access outside the repository, but the working directory itself is readable and writable by default. The development resources we allow, such as the Go module cache and Nix store, may also contain copies of sensitive source code.

We therefore append stricter rules for protected repositories. These rules deny access to sensitive paths inside the repository while still allowing metadata access where tools need it to traverse the filesystem.

We apply the same principle to copies outside the repository. The relevant paths depend on the repository and its toolchain. For example, some of our policies deny Nix store paths matching /nix/store/.*-source. For the Go module cache, we deny our internal module namespace and its download cache by default, then allowlist only the modules approved for agent use. Metadata remains visible so tools can traverse the cache, while the contents of modules outside the allowlist (including stale cached modules) remain inaccessible.

This approach introduces a different kind of friction: it requires more deliberate policy design. Compared with a container, which primarily limits visibility through the resources mounted into it, Safehouse runs in the developer's existing environment and must explicitly account for sensitive copies in Git objects, caches, and package stores.

Because the client runs directly on the host, it uses the same operating system, ABI, and filesystem layout as the developer's existing toolchain. Containers can also share build directories and other host resources, but only when those resources are mounted and compatible with the container environment. The host-level approach therefore avoids many of the compatibility, port-forwarding, and file-ownership problems introduced by a separate container environment. In return, the policy must describe the permitted access more precisely, and anything explicitly allowed remains accessible.

Shared limitations

There are a few limits that should be noted.

Build and test constraints

Both approaches can limit the files available to the agent and its subprocesses. Metadata access may allow tools to discover and traverse denied paths, but a compiler or test that requires the contents of a sensitive source file cannot run without them.

Whether the agent can build and test the remaining repository depends on the language and project structure. In C++, source files are compiled independently, so a sensitive component can sometimes be compiled separately and exposed only through its public headers and compiled library.

Figure 3: The linker can reuse separately compiled translation units.

When this boundary is available, the agent can compile and link the rest of the project without accessing the implementation sources. A container still requires a library compatible with its operating system, architecture, and ABI, while a host-level sandbox avoids that additional compatibility boundary.

Go has a different constraint. Ordinary Go packages must generally be available as source during the build; Go no longer supports binary-only packages. If an imported package is hidden, the agent cannot complete the build.

Figure 4: The Go compiler needs package source code to produce the final build.

Git

Restricting the working tree does not fully restrict Git. The same files may remain recoverable from Git's object database, even when their paths are obscured.

To prevent recovery through Git, repositories with obscured paths are exposed to the agent without their original .git directory; we initialize a new empty repository instead.

This prevents hidden files from being recovered from the original Git objects, but it also removes the repository's history and original baseline from the agent's environment. The agent can still edit files and create changes, but it cannot use the original repository for normal history inspection or comparison.

These approaches reduce the agent's access to sensitive files, but they do so with different trade-offs. Containers provide a more isolated environment but introduce compatibility and integration problems. Host-level policies preserve the existing development environment but require more precise rules. We use both approaches depending on the repository and its development workflow.

Sandboxing reduces the sensitive data available to the client at its source. We reinforce this protection with a final barrier that controls access to the models and inspects requests before they leave our infrastructure.

3. Control model access and outbound data

The final safeguard layer relies on tools and services for controlling model access and outbound data risks.

LLM gateway

Before discussing the tool we use to prevent sensitive information from leaking, I need to briefly explain our infrastructure.

The first step was to select a limited set of approved LLM providers. We evaluate providers against security requirements (such as certifications, track record, and zero-data-retention policies), as well as practical considerations (such as cost, latency, and access to the latest models). Restricting ourselves to carefully selected providers adds another layer of protection.

We use Aperture by Tailscale as our LLM gateway. It forwards each request to the appropriate provider, limits usage, restricts access, and more. This gives us control, observability, and auditability over how the models are used. But the feature most relevant here comes next.

Guardrail

The gateway can pass each request through a service that we operate ourselves and that has access to our sensitive data. Before the request reaches the LLM provider, this service can allow, modify, or block it. It acts as a final safeguard if earlier protections fail: it can redact sensitive information, block LLM usage entirely for particular repositories, or restrict those repositories to trusted providers.

Figure 5: The guardrail evaluates every prompt against custom rules to determine the sensitivity of the request.

The challenge is achieving high precision with near-perfect recall. A false positive creates temporary friction, while a false negative can expose IP or PII. We therefore prioritize recall: first catching all potentially sensitive information, then reducing false positives as much as possible, without weakening that protection.

Organizational concerns

Using LLMs is not only a security challenge; it is a real shift in how we work. Beyond the debates about whether LLMs will replace us or how many engineers we should hire, three concrete concerns are worth discussing: newcomers building weaker technical foundations, a less critical eye on plausible-sounding output, and a growing review bottleneck.

Weakening technical foundations

A recent study highlights an uncomfortable trade-off: developers using AI while learning an unfamiliar Python library showed weaker conceptual understanding, code-reading ability, and debugging skills, especially when they delegated the task entirely. The learning context is close enough to onboarding in an unfamiliar codebase to influence our approach.

Our principle is simple: newcomers should learn the codebase before an LLM starts writing code for them, though they remain free to use LLMs for other parts of their job. They gain full access to LLM tools only once they can demonstrate that they understand the codebase well enough to review and take responsibility for the resulting changes.

Note that we say "newcomers," not "junior developers." Experienced developers may complete this step more quickly, but they should not skip it.

Non-critical approach to output

While this topic has been discussed at length, the core challenge remains:

LLMs are very good at writing plausible but incorrect code. You can give them more tools and more opportunities to correct themselves, but they can and will hallucinate and make mistakes. It remains our job to identify those mistakes and think critically about their output.

A recent study of AI assistance highlighted a crucial dynamic: AI tends to benefit those who already possess strong domain expertise while hindering those who don't. The differentiator is judgment. Receiving a plausible answer is dangerous if you lack the experience to evaluate it critically.

As with any external source, developers remain accountable for every change they submit. Responsibility cannot be deferred to an LLM: authors should understand, explain and support their contributions. Experiences from other projects illustrate how PRs submitted without sufficient understanding can place a significant burden on reviewers. We want LLMs to help engineers move faster while remaining confident owners of their changes.

This concern also applies when LLMs are used in the review process. They often make different kinds of mistakes than those made by junior developers; their feedback can appear polished and confident even when it is based on an incorrect assumption. LLM-based review must therefore complement, rather than replace, critical human judgement.

Our approach combines concrete process decisions with ongoing monitoring. Beyond the structured onboarding process and additional LLM-based review, we tag every LLM-assisted diff so we can track outcomes and run benchmarks over time. We are also introducing random questioning of authors about their own diffs, to confirm they understand what they submitted rather than merely approve it. Developers who don't follow these guidelines can lose access to LLM tools. Together with signals such as regressions and incidents, these measures help us identify worrying patterns and adjust our approach.

Increased review burden

A related challenge is the potential increase in review burden. If LLMs can produce code faster than we can review it, the review queue quickly becomes the new bottleneck.

There is some evidence that this is already happening. A study looking at open-source projects after the introduction of GitHub Copilot found that projects produced more code, but their pull requests also required more rework. Peripheral contributors produced more, while core contributors spent more time reviewing and made fewer commits themselves. This does not necessarily mean that the code was worse, but it suggests that part of the productivity boost was shifted from producing code to reviewing, reworking, and integrating it.

We have two ways of reducing the review burden: limiting the scope of each PR and creating a review funnel.

1. Limit PR scope

This was already important for human-written code, and it is even more important when using LLMs. When an LLM can generate additional changes almost instantly, it is easy to accept every suggested improvement. This is the trap: you end up with a massive PR trying to solve ten unrelated problems and creating far more review work than the original task required. More than ever, the scope of a PR must be limited as much as possible.

To make this concrete, we limit LLM-assisted diffs to 1,000 lines and require a local LLM review to report no remaining issues before a change reaches human review. This does not replace human review, but should allow it to focus more on intent, architecture, and system-level trade-offs.

2. Review funnel

A review funnel is another established way to reduce the review burden. It predates LLM-assisted development: projects including LLVM and the Linux kernel have long distributed reviews across maintainers and experts familiar with the affected code. Rather than asking the same reviewers to assess every aspect of a change, the work is shared across people with different areas of expertise. For LLM-assisted development, this model can be extended with codebase-specific skills or specialized models focused on different concerns, without replacing human judgment.

We are monitoring how those solutions work in practice through PR size, review time, and the number of review rounds required, as well as feedback from the engineers involved.

Conclusion

Security is our foremost concern when considering LLM usage for development, and that reflects how we approach any new tool or process as a security company. As we've seen, the risks are real when ignored but there are ways to minimize both their likelihood and their impact. The tools became useful enough to justify the work required to use them safely, but not trustworthy enough to use without dedicated safeguards.

On the technical side, Guardsquare’s safeguards focus on preventing agents from taking rogue, destructive actions, as well as on limiting the exposure of sensitive IP and PII to LLM providers through sandboxing, a controlled gateway, and a guardrail service that inspects every prompt before it leaves our infrastructure.

On the organizational side, we're making sure newcomers build solid technical foundations before they delegate work to an LLM. We’re staying critical of LLM output rather than trusting it by default, and we keep our review burdens manageable by limiting PR scope and holding authors accountable for what they submit.

We will continue to expand LLM usage with care and due diligence. Access remains limited to a small group as we assess the balance between productivity gains and the costs of review, rework and regressions, all while exploring additional safeguards.