What are Keepers & why they are a critical part of blockchain infrastructure?

This article serves as an introduction into why keepers are needed, why protocols would use keepers, who can be a keeper, what challenges keepers face & where might keeper job evolve in the future

Background

Smart contracts have come a long way since they were first envisioned back in the 1990s by Nick Szabo, further refined in the, now seminal, 2013 Ethereum whitepaper of Vitalik Buterin

Today, by comparison, smart contracts are complex and many have to make use of extensive external off-chain data, such as price feeds — you may have previously heard of oracles? Well, this is the important role they play in blockchain & the evolved smart contract infrastructure (pulling in real world data like the current rate of exchange between USD & EURO, for example)

Smart contracts are limited in their ability to call a function or execute on their own rules, with smart contracts stating logic that goes something like;

If THAT happens, then take THIS action

However, the smart contract alone can’t execute the action without an external transaction to initiate the smart contract logic

It may be helpful to think of an experience you’ve had as an end user of a protocol, whereby the transaction you made triggered the rules of the underlying smart contract. The same is true on the protocol side, where something has to trigger the smart contract rules to kick in

Let’s run this through a quick example and explore how it all works;

In this scenario imagine a protocol that has the ability for token holders to stake their token for a set duration (for example, a year) and in return the token holder will receive rewards that are given as a share of the revenue the protocol generates (i.e. profits) at regular intervals. Well, those rewards have to be distributed to each token holder that has staked according to the rules determined within the smart contract but distribution won’t occur unless something or someone triggers the distribution based on the rules or conditions set-out in the smart contract, once triggered all staked token holders will recieve their rewards at the previously scheduled intervals. Should this not occur at the expected time then the protocol will have a lot of unhappy staked token holders to contend with

Other, similar examples are;

• Harvesting yield on auto-compounding or farming strategies
• Depositing funds into vaults or strategies
• Closing an option upon expiry, or auto-executing options
• Managing leveraged positions to avoid liquidations (debit ratio health)
• Liquidating debt positions

All of these and other similar scenarios (including future use cases not yet anticipated), require something or someone to interact with the smart contract in order to trigger these events and thus ensuring that the action is completed. These external agents driving these actions interacting with smart contracts are called Keepers, and can be thought of as the hidden hands that keep protocols running smoothly

It doesn’t take much imagination to understand how damaging so of these may be if the actions aren’t completed in a timely manner

Who can be a Keeper? Who are Keepers?

Anyone can take on the role of a Keeper and whilst a technical background helps it’s not essential. Keeper tasks can be completed manually but it’s often easier, more effective and manageable to utilize scripts & bots, with the Keeper completing any off-chain logic or computation necessary to supplement and make the call on-chain

Keepers can be either lone-wolf operators working individually or they can be cooperatives that have banded together but are registered and operating as a single Keeper, sharing costs and overheads between all resources contributing to the collective

Keepers can also join keeper networks that seek to attract protocols with recurring tasks that need to be completed matching those tasks with qualified Keepers that are willing to complete the tasks. Keeper networks have the additional advantage of providing a layer of resiliency due to having multiple Keepers participating in the keeper network simultaneously

What options do protocols have?

So, now that we understand how smart contract logic works, and the necessity of Keepers within the infrastructure of a protocol, let’s understand how protocols decide to manage this need. Protocols have three choices for ensuring the smooth running of operations in meeting this need;

1. They can maintain Keeper roles within their own teams, managing the need & cost in-house
2. They seek to outsource the effort to an external group, often via incentivizing external resources directly though an internal Keeper rewards program
3. They can make use of an external keeper network

The 2nd & 3rd options above are quite similar, however have some key differences and benefits. But before we get into that, let’s quickly examine why maintaining Keeper tasks with in-house resourcing isn’t appealing for protocols

Much like off-chain businesses protocols seek to keep costs as low as possible to ensure profitability. Part of their anticipated costs are the development team that helps to build, maintain and further enhance the protocol through the deployment of new features. Keeper tasks can be highly repetitive and high in volume, which can be considered both unrewarding for the team and cause a drag-factor (or distraction) away from focusing upon delivering new features. One clear risk of this approach is that if the keeper tasks are not responded to in a timely fashion, then the protocol could run into troubles, which, in the worst case, can cause loss of revenue — effectively meaning there is a central point of failure within the setup

The second option, outsourcing the task to external teams also comes with a high-cost overhead — whereby the protocol has to fund payments to the external partners. This can often be more costly than maintaining via the in-house development team, however it does allow the core development team to focus on new feature delivery (although some integration efforts are to be expected). The risk highlighted above, with regards to a central point of failure on the timely execution of keeper tasks however remains especially where outsource teams are stretched thin or unmotivated due to low profitability

The third option, allows protocols to leverage an external service that maintains a network of Keepers that will accept jobs & work requested by protocols to the network of keepers. This can serve multiple benefits whilst addressing the risks present in the two options above. The first clear benefit is that keeper networks can lower costs for protocols, as dedicated keepers can maintain many jobs or undertake lots of work concurrently. Whilst on the risk side they remove the single point of failure, as within a good keeper network service there are enough keepers registered and active that if a single keeper does not respond to the initial request, the task can automatically rotate to the next available qualified keeper

What challenges do Keepers face?

Keepers have three main challenges they have to contend with or seek to overcome if undertaking keeper tasks are to prove profitable;

a)Barrier of entry in consideration for Keeper tasks

b) Gas costs & being outbid on gas for a job/task (known as priority gas auctions or PGA)

c) Variability of value in payments received for completing tasks

Let’s look at why each of these can cause issues for Keepers, and what options are available to avoid these challenges;

Barrier of entry in consideration for Keeper tasks — Keeper tasks present an inherent risk for protocols since they do not know whether the keeper is trust worthy or a malicious actor looking to exploit them. Therefore, tasks often have a requirement that Keepers must either prove their qualification and worthiness, such as the number of tasks previous completed, period of time active within a keeper network, or otherwise provide some form of collateral that can be leveraged in the event they do not complete the task in good faith (i.e. the protocol that posted the task can make a claim on the collateral should the task fail or be exploited). Some keeper networks may offer to underwrite the qualification and put up collective collateral for disputes raised by protocols, however they will often offset that by limiting the maximum amount of Keepers that can be registered to the network and/or ask Keepers that wish to register within a keeper network to make a down-payment or deposit to enter the network

Gas costs & being outbid on gas for a job/task — risk manifests where there is more than a single Keeper competing for the same keeper task. In this scenario one Keeper can simply pay more gas than is the going rate in order to secure the task ahead of a competing Keeper. This can prove a zero-sum game where multiple Keepers seek to outbid each other in order to secure the keeper task ahead of their competitors. This is known as a priority gas auction or PGA, and will become problematic when the cost of gas starts to encroach on the profitability of the task (the cost the protocol is willing to pay for the keeper task less the gas fee). Some keeper networks help with this challenge by auto-matching keepers with tasks, based on suitability, availability and requirement

Variability of value in payments received for completing tasks — Keeper tasks are usually rewarded in the native token of the protocol requiring the task, for example a keeper task posted by Curve finance might be rewarded in the native token of Curve, CRV. Since the native tokens have a variable real world value this can prove problematic for Keepers that have the burden of real world costs (cost of living plus tools required to complete keeper tasks). This risk is amplified where keeper tasks are posted by protocols that experience high variability in the value of native tokens and, at worst, can cause a preventative factor in attracting more potential Keepers from seeking a sustainable undertaking up-keeping tasks for protocols. This risk can be partially offset by keeper networks that are willing to compensate Keepers via a token issued by the keeper network, this can remove the risk of multiple variable protocol native tokens (otherwise paid in the different native tokens of each different protocol listing tasks for Keepers to complete). Keeper networks can also offer to pay a retainer + bonus for tasks completed. However, as this is an emergent and currently small industry the risk cannot be offset completely unless a keeper network is willing to make an offer of some/all payment to Keepers via stablecoins

In totality these challenges, combined, can limit the growth of skilled workers willing to act as full-time Keepers if keeper networks are unable to find or accommodate solutions to the risks present for Keepers.

Since it follows that the most successful keeper network will likely be the networks that;

• are able to attract the highest volume of registered Keepers and retain a variety Keepers at different skill-levels
• attract high demand for keeper tasks from multiple / major protocols
• maintain the most effective method of matching Keepers with protocol listed tasks

It will likely be the keeper network that is able to most effectively solve these challenges that see the most success

Should these challenges not be overcome then we may see a substantial gravitation towards centralization of keeper services, and in the worst case, single Keepers establishing dominance within existing keeper networks as the only Keeper registered or actively completing tasks. This would reintroduce the risk of a central point of failure should a centralized keeper not respond to tasks in a timely manner, as earlier detailed

Where could Keeper jobs evolve?

Keepers exist within an industry that is going through its own period of rapid growth and can be considered relatively new even within those terms. As the role continues to evolve we are likely to see the growth of more keeper collectives reaching ever higher states of sophistication, with Keepers undertaking more highly skilled and unique DevOps specialization

This is highly likely where individual lone-wolf Keepers are provided with the opportunities to connect and collaborate, and as Keepers seek to meet the demands of protocols for high-reliability/high-response, meaning there would be an obvious benefit to sharing computational overheads between keeper cooperatives. This would also allow the opportunity for such collectives to task on keeper tasks at all levels of complexity matching tasks with skill-set within their own collective group rather than being reliant on a keeper network to do so on their behalf

Keeper networks have the potential to carve out their own specialization or target markets for attracting relevant protocols and matching them with available Keepers, especially where the challenges highlighted above are overcome through further innovation within the space

Finally, the demand matching aspect of pairing protocols with individuals or teams willing to undertake tasks could potentially be expanded into the sphere of non-technical tasks such as; promotional activity, scouting opportunities, content creation, translation or moderation of forums/groups, etc. Although it should also be noted, that these tasks are and may continue to be managed successfully outside of keeper networks, however there remains an interesting and somewhat correlation of matching need with available skilled & willing workers. Will we one day all be undertaking part-time supplementary on-chain paid work thanks to the innovations driven by Keepers?

See you in the future, where we’ll find out how this all plays out