Skip to main content
At least two running Checkly Agents per Private Location are recommended for redundancy. Agents are stateless and can be scaled horizontally as needed. You can add additional agents to a location at any time and remove them as necessary. If one Checkly Agent fails, the other agents in the same Private Location will continue to run the checks assigned to that location. Even checks that are currently in progress on an agent that fails will be rerun by another agent after a timeout period of 300 seconds. There are two cases where checks assigned to a location could fail to run:
  1. All the agents in the Private Location have failed.
  2. There is insufficient agent capacity in the Private Location.
Checks will stay queued for 6 minutes waiting for an available agent. If there are no agents available or the available agents are so busy that they cannot process all the pending checks in that timeframe, the pending checks older than 6 minutes will be lost. Agents can be scaled up and scaled out. Scaling up means adding additional CPU and memory allocations to existing agent containers. Scaling out means adding additional agent containers.

​
Scale up

Scaling up of individual agents is generally memory-constrained. For reference, a Browser check requires about 1.5GB of RAM and an API check requires about 150MB. Two environment variables control how many checks a single agent runs at the same time:
  • JOB_CONCURRENCY (default 1, max 10) for Browser, Multistep and Playwright checks.
  • AGENT_WORKER_COUNT (default 10, max 50) for API checks and uptime monitors (URL, TCP, DNS, ICMP).
We do not automatically separate browser and API checks on an agent. If you are running Browser checks in a Private Location, the formula (rounded down) is: JOB_CONCURRENCY = Container memory allocation (GB) / 1.5 For example: if your container has 8GB of memory allocated, you should set JOB_CONCURRENCY to 5 (8GB / 1.5GB per check). If your Private Location is only running API checks or uptime monitors, for example on the uptime image, size AGENT_WORKER_COUNT instead: AGENT_WORKER_COUNT = Container memory allocation (GB) / 0.15 For example: if your container has 1GB of memory allocated, you should set AGENT_WORKER_COUNT to 6 (1GB / 0.15GB per API check).

​
Scale out

To determine the number of agents you need in a Private Location you first need to know the number of checks assigned to the location and their frequency. API checks can be scheduled as frequently as every 10 seconds and Browser checks as frequently as every 1 minute. Based on your configuration, you should be able to estimate how many checks will run per minute in your Private Location. API checks have a maximum running time of 30 seconds, Browser checks 4 minutes. This means that in the worst case scenario, a checkly agent can run two API checks per minute per AGENT_WORKER_COUNT slot, or 1 Browser check per 4 minutes per JOB_CONCURRENCY slot. In an average configuration this will be higher and API checks are generally faster than Browser checks. Once you have an idea of how many checks will be running and you know the per-agent concurrency limit based on the agent container memory allocation, you can estimate the number of agents required as follows. Use JOB_CONCURRENCY for Browser checks and AGENT_WORKER_COUNT for API checks and uptime monitors: agents = ((Checks per minute * average check duration in minutes) / JOB_CONCURRENCY) + 1
  • Lower JOB_CONCURRENCY, more agents.
  • More checks per minute, more agents.
  • Longer average check duration in minutes, more agents. I.e. if your estimate is checks run on average for 30 seconds, you plug in 0.5.
  • The + 1 is to ensure redundant capacity in case of rolling upgrades or an agent failure.
Let’s look at an example:
  1. JOB_CONCURRENCY = 4
  2. I expected 35 checks per minute
  3. Checks will take on average 25 seconds. This is ~0.42 minutes.
This means my estimated number of agents is ((35 * 0.42) / 4) + 1 = 4.675 or rounded up 5 agents.