Testing Limits of Parallel Function Calling with GPT-4o

ELI5 – Function Calling

Function calling lets you describe functions using a format called JSON Schema. Then, a language model (LLM) can intelligently decide which functions to call and what arguments to use for any given task. Imagine it like telling a smart assistant what tools it has and letting it figure out which ones to use and how. This can be super useful in real-world scenarios where you want the LLM to handle tasks that need some level of precision, like calling an API, running a piece of code, or other similar tasks.

Parallel Function Calling

Previously, one of the big limitations with using LLMs was that they could only call a single function at a time, and this often came with high latency. This made it less practical for real-world applications where speed and efficiency are crucial. Parallel function calling changes the game by allowing models to execute more than one function call simultaneously.

Imagine you’re working on a project where you need to summarize notes, add them to Notion, and send an email—all at once. With parallel function calling, the model can handle all these tasks simultaneously, significantly reducing the response time.

Recently, we’ve been experimenting with this feature, and I got curious: just how many parallel function calls can these assistant framework handle?

To find out, I decided to run a series of experiments. My test case? Adding reminders to Google Calendar. I wanted to see the maximum number of parallel function calls model could prompt for at once!

Experiment Steps:

1. Create a new assistant using the GPT-4o model through OpenAI’s platform.
2. Add a System Prompt: “You are a super intelligent assistant.” 😉
3. Define the Function: Use the Calendar Quick Add Function Schema.
4. Task: “Can you add a reminder for 4am?”

Ok it works. Let’s move on we have much to conquer.

Changing task – Can you add a reminder for 4am & 5am 🤯

Works great. So at least 2 parallel calls are being. Phew atleast parallel has some meaning now.

Changing task – Can you add a reminder for 4am & 5am & 6am 🙈

So 3 also worked …. so on… 8 also worked.

My hunch is that this is obviously limited by max output tokens and so I quickly tried the obvious thing, increased the context length used in the task execution and also increased the number of tasks.

Prompt – Can you add a reminder on an hourly basis from 4am on 22nd Jan to 4pm 24th Jan about drinking water and staying hydrated. Also stress the importance of RO water and not tap water. So totally 36 reminders should be created.

This creates 13 function calls at the same time.

To verify that the limiting factor is context length, I am now deleting the extra information and try again.

Prompt – Can you add a reminder on an hourly basis from 4am on 22nd Jan to 4pm 24th Jan about drinking water and staying hydrated. So totally 36 reminders should be created.

This resulted in 36 calls and so it kinda works? 🤔

My other attempts at reproducing more function calls didn’t really work but the context length should be the biggest limit and no other limits currently seem to exist.

Function Call Schema

{
  "name": "googlecalendar_quick_add",
  "description": "Create a new event in a Google Calendar based on a simple text string like 'Appointment at Somewhere on June 3rd 10am-10:25am'\n    You can only give title and timeslot here. No recurring meetings and no attendee can be added here.",
  "parameters": {
    "properties": {
      "calendar_id": {
        "default": "primary",
        "description": "Calendar identifier. To retrieve calendar IDs call the calendarList.list method. If you want to access the primary calendar of the currently logged in user, use the 'primary' keyword.",
        "examples": [
          "user@gmail.com"
        ],
        "title": "Calendar Id",
        "type": "string"
      },
      "text": {
        "default": "",
        "description": "The text describing the event to be created.",
        "examples": [
          "Appointment at Somewhere on June 3rd 10am-10:25am"
        ],
        "title": "Text",
        "type": "string"
      },
      "send_updates": {
        "default": "none",
        "description": "Guests who should receive notifications about the creation of the new event. Acceptable values are: 'all': Notifications are sent to all guests. 'externalOnly': Notifications are sent to non-Google Calendar guests only. 'none': No notifications are sent.",
        "title": "Send Updates",
        "type": "string"
      }
    },
    "title": "QuickAddRequest",
    "type": "object"
  }
}