AI & LLM Integration

From GitLab Duo to Junie: Building an OpenAI-Compatible LLM Adapter

Published on October 1, 2026 by Viliam Simko · 7 min read

AI-assisted software development is increasingly becoming part of the normal developer workflow. Tools such as JetBrains Junie can act as coding agents, while IDEs provide AI-powered code completion, test generation, chat, commit-message generation, and other capabilities.

But there is a problem that is easy to overlook: you cannot always choose which LLM your organization is allowed to use.

In enterprise environments, security and data-governance requirements can be more important than the convenience of using the latest public model.

This article describes a real engineering problem I encountered: I wanted to use GitLab Duo's non-agentic chat as the LLM backend for Junie CLI.

Current status: The adapter is not completely finished yet. The remaining problem is session management, which currently prevents the adapter from being used by Junie CLI. However, the core integration works well enough to demonstrate that the approach is feasible.

When “just use another LLM” is not an option

In a previous customer project, access to AI tools was deliberately restricted.

Source code, prompts, and other potentially sensitive information were not allowed to leave the organization's controlled environment. As a result, simply configuring an external LLM provider was not an option.

This is an important distinction between experimenting with AI on a personal machine and introducing AI into an enterprise development environment.

As developers, we often think about AI tooling in terms of:

Which model is the best?

In an enterprise environment, the first question may instead be:

Which AI services are we actually allowed to use?

In this particular environment, the customer provided access to GitLab Duo's non-agentic chat.

That effectively gave us access to an LLM-based service while keeping the interaction within the customer's approved AI tooling and infrastructure.

I did not need to choose or operate the underlying model. The important part was that GitLab Duo provided the LLM capability that was already available to the development team.

GitLab Duo inside IntelliJ

GitLab Duo also provides an IntelliJ plugin with several useful AI-assisted development capabilities.

It can be used for code-related chat and other AI-assisted development workflows directly inside the IDE. This is useful, particularly because the service is already part of the customer's approved AI tooling.

However, the integration is not as deeply integrated into the IntelliJ development environment as the AI features provided directly by JetBrains.

The experience is therefore somewhat different from using JetBrains' own AI tooling, where AI capabilities are more tightly integrated into the IDE and its existing workflows.

This made the question of interoperability even more interesting.

The goal was not necessarily to replace GitLab Duo or its IntelliJ plugin. Instead, I wanted to find out whether the LLM capability already available through Duo could be reused by another AI development tool — in this case, Junie.

The missing piece: Junie and custom LLM backends

JetBrains IDEs provide several ways to use AI assistance.

Depending on the feature, developers can use AI for things such as:

code completion
generating tests
explaining or transforming code
general LLM chat
generating commit messages
other development tasks

The IDE also provides support for connecting to an OpenAI-compatible API endpoint for several AI features.

That immediately suggested an interesting possibility:

If GitLab Duo could be exposed through an OpenAI-compatible API, could the IDE use GitLab Duo as its backend?

There was another problem, however.

Junie does not currently allow an arbitrary custom LLM backend in the same way.

The Junie IntelliJ plugin supports a defined set of supported models and providers, including different versions of models such as Gemini, Claude, GPT, and Grok.

That means that even though an OpenAI-compatible backend can be useful for other IDE features, it cannot simply be selected as an arbitrary Junie backend inside the IntelliJ plugin.

Junie CLI is different.

The CLI can be configured to use a custom OpenAI-compatible endpoint.

That changed the problem completely.

Instead of trying to make Junie understand GitLab Duo directly, I could build an adapter:

┌──────────────────────┐
│      Junie CLI       │
│                      │
│ OpenAI-compatible    │
│       client         │
└──────────┬───────────┘
           │
           │ OpenAI-compatible API
           ▼
┌──────────────────────┐
│   GitLab Duo Adapter │
│                      │
│ OpenAI API           │
│       ↓              │
│ GitLab GraphQL API   │
└──────────┬───────────┘
           │
           │ GraphQL
           ▼
┌──────────────────────┐
│     GitLab Duo       │
│   Non-Agentic Chat   │
└──────────────────────┘

The adapter would translate requests from Junie's OpenAI-compatible interface into requests understood by GitLab Duo.

Finding out how GitLab Duo actually communicates

The first challenge was discovering how the GitLab Duo IntelliJ plugin communicates with its backend.

I found that the plugin uses GraphQL for its communication.

This was a very useful discovery because it meant that I did not have to guess the API from scratch.

Even more importantly, the relevant GraphQL API was documented and the GraphQL Playground was available.

That made experimentation much easier.

Instead of trying to reproduce the entire IntelliJ plugin, I could inspect and experiment with the individual GraphQL operations.

The workflow became roughly:

GitLab Duo IntelliJ plugin
            │
            ▼
       GraphQL requests
            │
            ▼
      GraphQL Playground
            │
            ▼
     Experiment with queries
            │
            ▼
     Understand request/response
            │
            ▼
       Build adapter

This was the point where the problem changed from:

Can this be done?

to a much more concrete software-engineering task:

What is the smallest compatibility layer required to make the two systems communicate?

Starting with an MVP

I did not try to implement a complete GitLab Duo client immediately.

Instead, I started with the smallest useful adapter that could translate the basic interaction.

The target interface was deliberately simple:

OpenAI-compatible request
        │
        ▼
     Adapter
        │
        ├── extract model
        ├── extract messages
        ├── translate request
        │
        ▼
   GitLab GraphQL
        │
        ▼
    Duo response
        │
        ▼
     Adapter
        │
        ├── translate response
        └── return OpenAI-compatible response
        │
        ▼
OpenAI-compatible client

The important point is that the client does not need to know that the backend is GitLab Duo.

From the client's perspective, it simply talks to an OpenAI-compatible endpoint.

This makes the adapter a compatibility layer rather than another AI agent.

Why an adapter instead of a new agent?

It would have been possible to build a completely separate integration specifically for Junie.

But that would duplicate functionality that already exists.

Junie already knows how to:

maintain the agent workflow
decide which actions to perform
interact with the codebase
execute tools
iterate over tasks
communicate with an LLM

The missing component was simply an LLM endpoint that Junie could talk to.

That is why the adapter approach is interesting.

The architecture becomes:

             Junie
               │
               │ OpenAI API
               ▼
        ┌──────────────┐
        │    Adapter   │
        └──────┬───────┘
               │
               │ GraphQL
               ▼
        ┌──────────────┐
        │ GitLab Duo   │
        └──────┬───────┘
               │
               ▼
        Customer-controlled
             AI service

Junie remains the agent.

GitLab Duo provides the language-model capability.

The adapter simply connects the two.

The interesting part: protocol translation

At first glance, an OpenAI-compatible adapter sounds trivial: POST /v1/chat/completions comes in, and a GraphQL request goes out.

In reality, the difficult part is preserving the semantics of the two APIs.

The adapter needs to deal with things such as:

message roles
conversation history
model selection
request parameters
response structure
errors
streaming, if required
session state

The last item turned out to be particularly important.

The remaining problem: session management

The current MVP is not yet ready to be connected to Junie CLI.

The main missing piece is session management.

A simple stateless request/response mapping is not necessarily sufficient for an agentic client.

Junie can make multiple LLM requests as it works through a task. The backend therefore needs to understand that those requests belong to the same logical conversation or session.

The adapter currently does not handle that lifecycle completely.

Conceptually, the missing layer looks like this:

Junie
  │
  │ request 1
  ▼
Adapter ──────► GitLab Duo session
  │
  │ request 2
  ▼
Adapter ──────► same session
  │
  │ request 3
  ▼
Adapter ──────► same session

Without correct session handling, each request can effectively become an isolated interaction.

For a normal chatbot this can sometimes be hidden by sending the complete conversation history with every request.

For an agentic workflow, however, the lifecycle and identity of the conversation can become much more important.

This is currently the main blocker before I can connect the adapter to Junie CLI.

Why this experiment is interesting

The interesting part of this project is not GitLab Duo itself.

It is the fact that existing enterprise AI infrastructure can potentially be reused by developer agents that were not designed specifically for it.

The customer already had:

an approved AI service
an IDE integration
access to an LLM
an existing authentication and security model

What was missing was compatibility between that service and another developer tool.

Instead of introducing another external LLM provider, the adapter creates a bridge between existing components.

This is a pattern that can be useful well beyond GitLab Duo.

Enterprise AI is often about integration, not model selection

When experimenting with AI as an individual developer, it is tempting to focus on model benchmarks and ask which model is currently the strongest.

Enterprise environments introduce a different set of constraints.

The relevant questions can include:

Where does source code go?
Where are prompts processed?
Who controls the infrastructure?
What data is retained?
Which external services are permitted?
How is access authenticated?
What logging and auditing are available?
Can the service be integrated with existing developer tools?

Sometimes the answer is simply:

We already have an approved LLM service. Make it work with the tools we need.

That was essentially the problem I was solving here.

What remains to be done

The adapter is currently an MVP rather than a finished product.

The main remaining work is:

Implement proper session management.
Verify the complete request lifecycle used by Junie CLI.
Test longer agentic conversations.
Handle errors and edge cases consistently.
Validate streaming behaviour where required.
Test the adapter under realistic workloads.
Document the configuration required by Junie CLI.

Once session management is complete, the interesting test will be to connect the adapter to Junie CLI and see whether Junie can operate as an agent while GitLab Duo provides the underlying LLM capability.

That would give us a rather unusual architecture:

              Junie CLI
            Agent runtime
                  │
                  │
        OpenAI-compatible API
                  │
                  ▼
        GitLab Duo Adapter
                  │
                  │ GraphQL
                  ▼
            GitLab Duo
                  │
                  ▼
       Customer-controlled
             LLM service

The agent and the LLM backend would therefore be separate components.

Conclusion

This project started from a very practical constraint: the customer already had an AI service, but the development tool I wanted to use could not directly consume it.

Rather than introducing another external LLM, I investigated the interfaces that were already available.

Discovering that the GitLab Duo IntelliJ integration uses GraphQL, combined with the availability of documented GraphQL operations and the Playground, made it possible to build an initial adapter relatively quickly.

The resulting MVP demonstrates the basic feasibility of exposing GitLab Duo through an OpenAI-compatible interface.

It is not finished yet. Session management is still missing, and until that part is solved, the adapter cannot be properly integrated with Junie CLI.

Nevertheless, the experiment demonstrates an important idea: AI tooling does not necessarily have to be tied to a particular model provider. With the right compatibility layer, existing enterprise AI services can potentially become backends for developer tools and agents that were not originally designed to use them.

Sometimes the most useful AI engineering work is not building another model integration. It is making the systems you already have work together.