Building Devin-like SWE Agents using Composio and OpenAI

In March 2024, Cognition Labs’ announcement of Devin—the software engineering agent—caught the eye of developers, founders, and investors alike. The idea of automating coding using SWE agents sparked a lot of curiosity. However, it's a closed-source project and isn't publicly available yet.

But what if you could create a software engineering (SWE) agent to resolve GitHub issues in your repository automatically?

This article discusses building software engineering AI agents with Composio and OpenAI SDK in Typescript.

But before that, let’s get a refresher on AI and SWE agents.

What are AI Agents?

AI agents have been generating much buzz lately; everyone has their take on what they are. But what exactly are AI agents?

Simply put, AI agents are systems driven by AI models that can autonomously carry out tasks, interact with their environment, and make decisions based on their programming and the data they process.

An AI model powers the agents. The AI model could be any machine learning model that can accept inputs, make sense of them, and provide outputs. However, most AI agents today are powered by large language models (LLMs).

There are three important components of AI agents.

• Inference Engine: The AI model is responsible for reasoning and decision-making.

• Knowledge Base: The data store that provides context to the AI agents.

• Tools: Enable the agents to interact with external applications and accomplish complex tasks.

Now, let’s understand what makes an AI agent a SWE agent.

What are SWE Agents?

An SWE agent is an AI agent that has the qualities and characteristics of human software engineers, such as

• Long-term planning and execution.

• Efficient in using standard developer tools.

• Fixing codes, writing unit tests and so on.

With current LLMs' excellent reasoning, decision-making, and coding abilities, we can build software engineers that can mimic a few qualities of a human SWE.

Overview of SWE Agent

So, before diving into the coding part, let’s understand what we will build and the prerequisites.

In this article, you will build an SWE agent that can

• Access any given GitHub repository.

• Read, write, update, and delete files as and when needed using Composio’s local tools.

• Orchestrate workflow using OpenAI SDK. However, Composio is framework agnostic, so you can use frameworks like LangChain and LlamaIndex.

• Accepts an issue and Creates a separate branch for changes.

• Runs the program in a sandboxed coding environment like Docker or cloud-hosted environments like E2B, FlyIo, etc.

• Finally, push the fix to the remote repository.

Fig: A schematic diagram of SWE agents built with Composio

Prerequisites for Building SWE Agent

These are requisites for completing this project.

• Composio SDK and API key. To get one, create a user account with Composio and navigate to the Settings tab on the dashboard.

• OpenAI SDK and API key.

• Docker for code sandboxing.

Let’s Get Started

Installing Dependencies

Begin by installing dependencies using your favourite package manager. The recommended method is pnpm, but you can also use npm or yarn.

Defining Environment Variables

You will need a GITHUB_ACCESS_TOKEN, OPENAI_API_KEY, COMPOSIO_API_KEY, GITHUB_USERNAME, and GITHUB_USER_EMAIL to complete the project.

So, create a .env file and add the above variables.

Project Structure

The project is organized as follows:

Here’s a brief description of the files.

• agents/swe.ts: Contains the implementation of the SWE agent.

• app.ts: The main entry point of the application.

• prompts.ts: Defines the prompts used by the agents.

• utils.ts: Utility functions used throughout the project.

To start quickly, clone this repository and install the rest of the dependencies.

swe-js cd swe-js && pnpm i " style="color:#d8dee9ff;display:none" aria-label="Copy">

Now that you have finished with the whole set-up. Let’s code our AI agent.

Defining Prompts and Goals

We begin by defining the prompts and goals for the SWE agent. Explaining each step in detail is crucial, as these definitions significantly influence the agent's performance and execution.

So, create a prompts.ts file if you haven’t done so.

Now, define the role and goal of the agent.

Here, we defined the role as SWE, and the goal is to fix any coding issue and create a patch for the fix using filetool_git_patch. This is a Compsoio Action for the GitHub integration for creating patch files.

Now, define the backstory and a description of the Swe agent.

In the above code block, we have carefully and clearly defined the steps the agent needs to undertake to accomplish the task. This is important to ensure the agent knows what to do when faced with common programming hurdles.

Defining Utility Functions

In this section, we will define two main functions from GitHub and getBranchNameFromIssue, which will be used to extract information regarding an issue.

So, here is what is going on in the above code block.

• readUserInput: This function reads the user inputs from the command line. We only need the GitHub user ID, repository name, and the issue number or description.

• createGithubIssueValidator: This function returns a validator for GitHub issues. It can handle input as a file path, a numeric issue ID, or a plain string description. If the input is a numeric issue ID, it fetches the issue details from GitHub using Composio’s github_issues_get action.

• fromGitHub: This function combines these elements to gather and validate the necessary information about a GitHub repository and an issue.

Now, define the getBranchNameFromIssue to create a branch name from the issue description.

Defining the Swe Agent

This is the most important section, where you will define the Swe agent using the OpenAI assistants and Composio toolsets.

So, first, import the libraries and define the LLM and tools.

In the above code block,

• We created an instance of OpenAI with the API key.

• We also created an instance of OpenAIToolSet with workspaceConfig set to Docker. This is to use Docker to sandbox the coding environment for the Swe agent. You can also use cloud code interpreters like E2B and FlyIo.

Now, we will define the Swe Agent.

Here is what is going on in the above code block.

• Get Tools: Fetches tools from the Composio toolset for filetool, file edit tool, and shelltool. As the name suggests, these will be used to access files, edit files, and use shell for executing commands.

• Trim Tool Descriptions: Limits tool descriptions to a maximum of 1024 characters.

• Update Null Values: Replaces null values in tool configurations with empty arrays.

• Create an Assistant Thread: Initiates an OpenAI assistant thread with predefined prompts.

• Return Statement: Provides the initialized tools, assistant thread, OpenAI instance, and Composio toolset.

Defining the Entry-point to the Application

This is the final section, where we define the application's entry point. Therefore, load the environment variables and import the required modules.

The code block

• Loads environment variables.

• Imports the necessary utility functions.

• Imports the Swe Agent and the agent Goal that we defined earlier.

Now, define the main function.

This is our complete app.ts file, which will be used to execute the agentic workflow.

So, here is what is happening in the above code.

• Initialize SWE Agent: Calls initSWEAgent to get the assistant thread, OpenAI instance, tools, and Composio toolset.

• Fetch Repository and Issue: Fetches repository and issue details from fromGithub.

• Create Assistant: Initializes an OpenAI assistant with instructions, tools, and the language model.

• Send Issue to Assistant: Sends the issue content as a message to the assistant thread.

• Run Assistant and Poll: Runs the assistant and polls for tool call responses. For more information about polling responses, refer to the OpenAI SDK repository.

• Execute Patch Action: Executes filetool_git_patch to generate a patch.

• Handle Patch Response: If a patch is generated, log it, create a branch, commit, and push changes. Wait for 2 seconds before creating a pull request. Create a pull request on GitHub.

• Close Workspace: Closes the Composio toolset workspace.

• Run Main Function: Calls main() to execute the above steps.

Now, run the application using. pnpm start.

This will prompt you to enter the GitHub user ID, repository name, and the issue ID or description of the issue you want to address.

Once completed, it will pull a Composio Docker container from the registry and start working on the issue.

Finally, the patch will be pushed to the remote repository when the workflow is completed. When you open your GitHub repository, you will see a new branch with the proposed fix for the issue. You can compare it with the main branch and create a pull request.

You can find the complete code here on GitHub.

Next Steps

The best thing about SweKit is you can extend the capabilities of the SWE agents using Composio tools and integrations. You can add Slack or Discord to your agent to notify you when the execution is completed. You can also connect Jira or Linear to automatically create and update tasks based on the agent's activities.