Chainlink External Initiators
Originally published on Medium by Raymond Mogg
For the ETHDenver hackathon this year, the SDL team utilised an external initiator to interact with their solidity contracts. This successful implementation granted the team first place in the Chainlink bounties for the hackathon event.
Note: external initiators are currently an experimental Chainlink feature. For node operators to use them, an environmental variable must be added to their node configuration:
What are External Initiators
External initiators are a way for an external event to trigger a job run on a Chainlink node. One such example which was used by SDL in the ETHDenver hackathon, was having an IoT sensor trigger a run on the chainlink node when certain conditions (such as the temperature dropping below X degrees) were met.
As one can imagine, this kind of ‘IF X, THEN Y’ functionality opens up endless opportunities for dynamic smart contract applications that bridge the gap between real-world data and on-chain events.
How do they work?
There are a few components involved in creating an external initiator. At a high level, the overall architecture can be visualized as seen in diagram above.
The steps involved for a job run to trigger is as follows:
1. Remote initiator polls / subscribes to a real world trigger (whether this be sensors, an external API, etc)
2. Based on some arbitrary logic (for example checking whether the sensor data polled has dropped below some threshold point), the remote initiator decides if the job run should be triggered or not.
3. The remote initiator notifies the Chainlink Node to run the job.
4. Optional: The Chainlink Node may request some data from the remote initiator as part of the job run, or supply it as part of the initiation request payload.
All requests between the Chainlink Node and remote initiator require an access key and secret key which is setup by the node operator as part of the setup process.
Creating your own external initiator
Setup on node
chainlink initiators create <name> <url (optional)>
This will create an external initiator on the node itself. URL is the URL where the task spec will be posted to from the node. This is necessary so that the remote initiator knows the Job ID to initiate and the criteria (defined in the task spec params) that has been defined to initiate the job.
For example if you are going to run a local initiator service for testing purposes this could be localhost:3002/jobs.
Take note of the values shown here (such as the X-Chainlink-EA-AccessKey and X-Chainlink-EA-Secret) as these will be needed later.
Remote initiator service
The next component to consider when building out the external initiator is the remote initiator service. SDL have created a sample NodeJS server template here, and there is also a very good template service provided by the Fiews team here. The functionality that must be implemented includes:
1. Listen for jobs to be created on the Chainlink node using this external initiator on the URL endpoint defined previously as part of the initiator setup. For our example we will run our initiator service on port 3002 and use the /jobs endpoint to an external event to trigger a job run on a Chainlink node. When this endpoint is called by the Chainlink Node, it will pass in the job ID (as part of the request body) that is utilizing this remote initiator. You can then store this job ID and know that it must be called when the time to trigger jobs comes. Your remote initiator can support many jobs id’s.
2. Poll, subscribe to or create an endpoint for some other service. Your remote initiator service should be listening to some event or have an endpoint that can be called by another service with some parameters, and when some set conditions are met, it should notify the chainlink node that it is time to run the job. To trigger the job run, a POST request must be sent to the endpoint on the node /v2/specs/JOB_ID/runs. Where JOB_ID is replaced by the job you want to trigger. As part of this request you must also pass in the X-Chainlink-EA-AccessKey and X-Chainlink-EA-Secret setup as part of the creation of the external initiator on the node itself. This means that in order to support multiple nodes, you need a secret, a key and at least one job id for every node that is using this service. These keys prevent other parties attempting to call the node and run a job should they know the job id the external initiator service is attempting to use.
3. Optional: Create an endpoint that is run as part of the job run. For our use case, we not only wanted the remote initiator service to tell the node when to run, but also during the run we wanted the node to pull some data from the service (the actual temperature recorded that caused the job to trigger). For this you can simply define another endpoint that will be called as part of the job run from the node.
One issue with this current external initiator example as described above is the centralization. In the example provided only a single node is considered, and it is connecting to a single instance of the external initiator service.
In the future, this needs to be extrapolated out to have multiple nodes running the remote initiator service. Our described remote initiator service already supports this more decentralized infrastructure.
An example of this architecture utilizing a more decentralized setup would be as follows (using a sensor trigger as an example (See above image)
In this setup, each node would be running its own version of the external initiator service, having its own access key and secret to communicate between the service and the node.
The external initiator service would ideally be pointing to multiple tamper proof IoT sensors owned by multiple stakeholders.
In order to be able to support this multiple node strategy, some in contract aggregation of responses may be required.