CrewAI PR Agent - Automate GitHub PR Review

Introduction

In this article, you will build a GitHub pull request (PR) review agent. Whenever a new PR is raised in a repository, the agent analyzes the diffs and posts a comprehensive review as a comment; a summary is also posted to a configured Slack channel.

What are AI Agents?

But before anything, let’s get familiar with AI agents.

AI agents are systems powered by AI models that can autonomously perform tasks, interact with their environment, and make decisions based on their programming and the data they process.

Let’s Get Started

As with every Python project, first, create a virtual environment.

python -m venv pr-agent
cd pr-agent
source bin/activate

Now, install the required libraries.

pip install composio-core composio-langchain /
langchain-openai/
python-dotenv

A brief description of libraries

1. The composio-core library is the primary resource for accessing and configuring tools and integrations. It also includes a CLI API for easy management of integrations and triggers.
2. The composio-langchain is Composio’s plug-in for LangChain.
3. The langchain-openai library from LangChain allows you to integrate OpenAI models into its framework.
4. The python-dotenv package loads environment variables from a .env file into your Python project’s environment.

Next, Create a .env file and add environment variables for the OpenAI API key.

OPENAI_API_KEY=your API key

Configure the Integrations

Composio allows you to configure SlackBot and GitHub without writing any code for the integration. Composio handles all the user authentication and authorization flows, so you can focus on shipping faster.

You can either do it from the CLI or use the dedicated dashboard.

But before that, log in to Composio from the CLI and update apps by running the following commands.

composio login
composio apps update

Complete the login flow to use Composio CLI API.

From CLI

The Composio core SDK allows you to manage integrations from the terminal.

Execute the following command to configure a Slackbot and a GitHub user account.

composio add slackbot
composio add github

Complete the authentication flow to add both integrations to your Composio account.

From Dashboard

You can also manually add integrations from the dashboard.

First, go to the tools catalogue section, find the integration you want to add, and click on it. In this case, it is the Slackbot.

Now, click on Setup SlackBot integration on the right.

Click on save to add it to your account and connect to the SlackBot app.

Once you finish the integration flow, your live integration will appear in the Integrations section.

Repeat the same process to add the GitHub integration.

Add Triggers for New GitHub PRs

Newt, we set up a trigger for our GitHub integration to fetch event data whenever there is a new pull request.

Again, you can do it from either the CLI or the dashboard.

From CLI, you only need to execute the following command.

composio triggers enable github_pull_request_event

Also, you can add triggers from the dashboard from the GitHub page.

Go to the trigger section and add the triggers you need.

Building the Agentic Workflow

Now that we have finished setting up integrations and triggers let’s hop on to the coding part.

Import Packages and Define Tools

Create a new Python file and paste the following codes.

import os
from dotenv import load_dotenv
from composio_openai import Action, ComposioToolSet
from crewai import Agent, Crew, Process, Task
from langchain_openai import ChatOpenAI

from composio.client.collections import TriggerEventData

load_dotenv()

# Initialize the ComposioToolSet
composio_toolset = ComposioToolSet()

# Initialize the ComposioToolSet
composio_toolset = ComposioToolSet()

# Define the tools
tools = composio_toolset.get_actions(
    actions=[
        Action.GITHUB_GET_CODE_CHANGES_IN_PR,
        Action.GITHUB_PULLS_CREATE_REVIEW_COMMENT,
        Action.SLACKBOT_CHAT_POST_MESSAGE,
    ]
)

Here is what is going on in the code above.

We imported the necessary packages and modules for the project.

Next, we loaded the environment variables using load_dotenv(). We then initialized ComposioToolSet().

After that, we defined the tool variable with several actions, each designed to perform a specific task:

1. Action.GITHUB_GET_CODE_CHANGES_IN_PR: Retrieves code changes in a GitHub pull request.
2. Action.GITHUB_PULLS_CREATE_REVIEW_COMMENT: Adds a review comment to the pull request.
3. Action.SLACKBOT_CHAT_POST_MESSAGE: Sends a message in Slack using the configured SlackBot.

Define System Prompts

Next, we define a system prompt that will be used to ground the model’s response.

channel_id = os.getenv("CHANNEL_ID", "")
if channel_id == "":
    channel_id = input("Enter Channel id:")
code_review_assistant_prompt = (
    """
        You are an experienced code reviewer.
        Your task is to review the provided file diff and give constructive feedback.

        Follow these steps:
        1. Identify if the file contains significant logic changes.
        2. Summarize the changes in the diff in clear and concise English, within 100 words.
        3. Provide actionable suggestions if there are any issues in the code.

        Once you have decided on the changes, for any TODOs, create a Github issue.
        And send the summary of the PR review to """
    + channel_id
    + """ channel on slack. Slack doesn't have markdown and so send a plain text message.
        Also add the comprehensive review to the PR as a comment.
"""
)

In the code above:

1. Received the Slack channel ID from the environment variable or the user.
2. Defined a step-by-step prompt for the agent to follow while executing the tasks.

Note: You can find the Slack channel ID from its URL, which usually starts with “C*”; for example, the channel ID in the URL https://app.slack.com/client/T074SRB4FGS/C073RUF0UQ5 is C074RUF0UQ5.

The Agent will post the summary of the PR in the specified channel.

Define the Agent

Next, define the CrewAI agent.

# Initialize the language model
llm = ChatOpenAI(model="gpt-4o")

# Create CrewAI agent
code_reviewer = Agent(
    role="Code Reviewer",
    goal=system_goal,
    backstory="You are an experienced software engineer with a keen eye for code quality and best practices.",
    verbose=True,
    allow_delegation=False,
    tools=pr_agent_tools,
    llm=llm,
)

In the above code, we defined an OpenAI instance and a CrewAI agent with a role, goal, and backstory, which gives context to the LLMs for solving problem statements. We also provide the agent with the LLM instance and tools.

Create Tasks and a Crew

Now, define a task and the crew to carry out the task.

# Create a task for the agent
def review_code_task(code_to_review):
    return Task(
        description=f"Review the following code changes and provide feedback: {code_to_review}",
        agent=code_reviewer,
    )


# Create the crew
code_review_crew = Crew(
    agents=[code_reviewer], tasks=[], verbose=2, process=Process.sequential
)

print("Assistant is ready")

The task has a description and an assigned agent. The Crew has the agent and task.

Define the Event Listener

The next step is to set up the event listener. This will receive the payloads from the trigger events in Slack.

The payloads contain the required event information, such as PR changes, timestamps, etc.

@listener.callback(filters={"trigger_name": "github_pull_request_event"})
def review_new_pr(event: TriggerEventData) -> None:
    # Using the information from Trigger, execute the agent
    code_to_review = str(event.payload)

    # Create a new task for this specific code review
    task = review_code_task(code_to_review)

    # Add the task to the crew and execute
    code_review_crew.tasks = [task]
    result = code_review_crew.kickoff()

    print("Review result:", result)


print("Listener started!")
print("Create a pr to get the review")
listener.listen()

In the above code block,

The callback function review_new_pr is triggered when a GitHub event matches the github_pull_request_event.

Code changes are extracted from the event payload and passed to the crew, who executes the steps defined to accomplish the task.

Now, run the Python file to set up the event listener.

The agent will act when you make a new PR in the configured GitHub repository. It will retrieve and review the code, post the review in the PR as a comment, and post the summary of the PR review to the configured Slack channel.

Let’s connect!

You can join our Discord community to engage with maintainers and contribute as an open-source developer. Don’t hesitate to visit our GitHub repository to contribute and create issues related to Composio.

CrewAI PR Agent – Automate GitHub PR Review