Run Chainlink “Basic Request Model” locally: Truffle, Ganache, Kubernetes

Link, Operator , Client smart contracts on Ganache. Oracle node on Kubernetes

Introduction

Chainlink documentation and blogs provide lot of detailed tutorials , most of them on public testnets (Rinkeby or Kovan). However, I’ve decided to build my own tutorial — based on Architecture Overview - Basic Request Model which :

• Runs on my local dekstop. This forces me to deploy everything from scratch: Link token , Operator contract, Chain link client contract..Etc. Doing so allows me to understand how all the pieces stick together
• Runs on kubernetes. As I’m thinking of providing node operators on Ethereum testnets, I find containers and containers orchestrators (such as Kubernetes) great candidates for providing resilient architecture. Thus, I would like to get comfortable running a Chainlink oracle node on kubernetes

In this blog, I’m going to share what I’ve built. Feel free to test it out. You can clone the following repo: github

Prerequisites

Tools and frameworks

Although this is a beginner tutorial , it will definitely help to have some previous experience with:

• Solidity
• Familiarity with local Ethereum development. Otherwise you can tryout truffle
• Chainlink (cfr. “What is Chainlink?”)
• Kubernetes. If you have a windows or a mac then you can install docker desktop , which allows you to run a kubernetes server and client on your local machine
• Nodejs and javascript

What is Chainlink?

Due to their deterministic nature, blockchains cannot pull or push data from/to external systems. Although this isolation has several benefits for reliability and security, it becomes useless for most applications in decentralized finance or insurance where smart contracts need external data in order to settle agreements.

Chainlink solves this problem by providing a decentralized network of oracles to connect the blockchain (on-chain) to the outside world (off-chain).

In case you have never worked with chainlink then I strongly recommend the following links:

• What is the Blockchain Oracle Problem ?
• Interview Lex Fridman — Sergey Nazarov: Chainlink, Smart Contracts and Oracles
• Chainlink doc: Running a chainlink Node , Fulfilling requests

What this tutorial contains

In this tutorial, we are going to build a smart contract which requests Ethereum price in USD from this API . The high level view is the following:

A brief explanation of all the building blocks:

• Smart contracts run on a local ETH blockchain (Ganache)
• Developer communicate with the blockchain by running scripts
• Oracle node and postgres run on kubernetes. Oracle nodes use Postgres as database to persist their configuration and state
• An Nginx ingress controller runs on kubernetes. This allows me to directly communicate with Oracle node without using “kubectl port-forward”
• Oracle node listens to events from the blockchain, calls an external API to fetch Etherem price then calls its linked contract to update Ethereum price
• a simple nodejs server is deployed. Its only purpose is to listen to events , format them and log them to the console

Now let’s focus on the Blockchain layer

Remark: dash arrows represent emitted events

Consumer contract

The only solidity code we are going to write is a Consumer contract which is a smart contract that request latest ETH price. Actually I took “TestnetConsumer.sol” from chainlink doc and slightly adapt it so it runs with a newer solidity compiler (v0.8.0)

Consumer contract inherits from “ChainlinkClient” and “ConfirmedOwner” which are both provided by Chainlink. This allows us to benefit from some common chainlink functionalities without reinventing the wheel (e.g.: Lock some calls to the owner of the contract, build request data for oracles..etc)

The method that will be called is “requestEthereumPrice”

As one can see, we build here a HTTP GET request to https://min-api.cryptocompare.com/data/price?fsym=ETH&tsyms=USD .

Important to note that the last argument of “buildChainlinkRequest” is function selector of “fulfillEthereumPrice” . Basically, that’s how we are going to provide the callback method to the Oracle.

“sendChainlinkRequestTo” is implemented by the parent contract “ChainlinkClient”. We can read this method from chainlink github repo

This method calls rawRequest which:

1. Generates a unique request ID (using keccak256)
2. Marks the request as pending
3. Emits a “ChainlinkRequested” event
4. Encodes data and calls “link.transferAndCall” (I will detail this part below)

Remark: Please note that last argument of “rawRequest”. it is a function selector of Oracle contract “oracleRequest” (and which will be discussed below)

As discussed above, Oracle node will call “fulfillEthereumPrice” of “Consumer” contract once it fetches ETH price.

This method emits a “RequestEthereumPriceFulfilled” event and updates the state variable “currentPrice” . Important to note that there is a function modifier “recordChainlinkFulfillment” which is implemented by the parent contract “ChainlinkClient”

This method does 3 things:

1. Validates the Oracle node address
2. Deletes the request from pending requests
3. Emits a “ChainlinkFulfilled” event

Link token contract

Link token implements ERC677 Token standard:

This adds a new function to ERC20 token contracts, transferAndCall which can be called to transfer tokens to a contract and then call the contract with the additional data provided. Once the token is transferred, the token contract calls the receiving contract's function onTokenTransfer(address,uint256,bytes) and triggers an event Transfer(address,address,uint,bytes), following the convention set in ERC223.

Link token implementation can be found here. The Consumer contract calls Operator contract through Link token. The Consumer contract will need some Link Token in order to pay the Operator for its work

Operator contract

This is our oracle contract. The implementation can be found here . As you can notice, Operator inherits from several contracts. If you open the implementation of AuthorizedReceiver , you will find an important method that will be called during before starting the tests “setAuthorizedSenders”

This method allows a contract owner to defined whitelist senders’ addresses. Basically the sender is the oracle node which runs on kubernetes.

Back to Operator contract implementation , you can find the method “oracleRequest” which was discussed above (cfr. rawRequest call)

This method does several controls: For instance, “validateFromLINK” modifier can be found in “LinkTokenReceiver” contract, it checks that the call comes from Link token contract

Once all the modifiers are called, a “OracleRequest” event is emitted. the “Oracle node” on kubernetes listen to this event. When the event fires, it calls the API then calls “fulfillOracleRequest” method

This method :

1. Calls an internal method “_verifyOracleResponse” to validate the response
2. Emits a “OracleResponse” event
3. Checks the gas left by calling a global function “gasleft()”
4. Callsback the consumer contract “callbackAddress.call”

Run the tests

Dependencies

• Tests have been performed with node version 12.22.1 and npm version 7.15.0
• Run in your terminal:

git clone https://github.com/aelmanaa/chainlink-local-kubernetes && cd chainlink-local-kubernetes
npm install

Project structure

• contracts/ : This folder holds solidity smart contracts. As discussed above, there is only one smart contract “Consumer.sol” . Once built, compiled contracts can be found in build/contracts folder
• kubernetes/: This folder holds kubernetes manifests
• migrations/: This folder holds a migration script. This script deploys “LinkToken” , “Oracle” and “Consumer” contracts to the local blockchain
• scripts/: 4 scripts that will be used to communicate with smart contracts
• server/: contains nodejs code for the “event watcher”
• config/ : when migrating contracts into the blockchain, a file “addr.json” containing all contract addresses will be put in here. Those addresses will be used by the “event watcher” in order to monitor the right contracts
• package.json & package-lock.json: use to manage project dependencies
• truffle-config.js: configuration file our our truffle project

Local Ethereum blockchain

First of all, you need to run a local Ethereum blockchain on your computer. In my case, I’m using Ganache running on port 8545.

Migrate contracts

run this truffle command in order to migrate contracts into the blockchain

truffle migrate --f 1 --to 1 --network ganache

Once migrated, config/addr.json will be updated

Create kubernetes namespaces and deploy ingress controller

Here we will :

• create storage and chainlink namespaces . These will be use to deploy respectively postgres and oracle node
• deploy an Nginx ingress controller that will be used to expose chainlink oracle node outside of kubernetes. Note that as I’m using docker-desktop , I took the right manifest from the official installation guide. Please use the right installation guide if you are not using docker-desktop.

kubectl apply -f kubernetes/namespaces.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.47.0/deploy/static/provider/cloud/deploy.yaml

You can check available namespaces using kubectl get namespaces

You can check that Nginx ingress controller have been installed using kubectl get all -n ingress-nginx . Pod is running and exposed through a service available on localhost

Deploy postgres database

Create a folder where potgres data will be persisted. This folder will be then mounted into postgres pod using persistentVolume and persistentVolumeClaim

mkdir -p $HOME/k8smnt/postgres

Create environment variables that will be used to replace placeholders in our yaml files (as I didn’t want to hardcode my own configuration) — Change the passwords . Also, feel free to use your own config if you wish. In your terminal , execute:

POSTGRES_USER=admin
POSTGRES_PASSWORD=<PG_ADMIN_PASS>
POSTGRES_DB=admindb
CHAINLINK_DB_USER=clnode
CHAINLINK_DB_PASSWORD=<CHAINLINK_DB_USER_PASSWORD>
CHAINLINK_DB=chainlink

deploy persistentVolume

cat kubernetes/postgresql/pv.yaml | sed -e "s|__home__|$HOME|g" | kubectl apply -f -

deploy persistentVolumeClaim

kubectl apply -f kubernetes/postgresql/pvc.yaml

deploy secret and configuration

Secret containing admin database username and password

cat kubernetes/postgresql/secret.yaml | sed -e "s|__POSTGRES_USER__|${POSTGRES_USER}|g" | sed -e "s|__POSTGRES_PASSWORD__|${POSTGRES_PASSWORD}|g" | sed -e "s|__POSTGRES_DB__|${POSTGRES_DB}|g" | kubectl apply -f -

Configmap containing a Postgresql initialization PSQL script. The script will create chainlink DB user and its password

kubectl create configmap -n storage postgres-init --from-file=kubernetes/postgresql/init-scripts/ --dry-run=client -o yaml | sed -e "s|__CHAINLINK_USER__|${CHAINLINK_DB_USER}|g" | sed -e "s|__CHAINLINK_PASSWORD__|${CHAINLINK_DB_PASSWORD}|g" | sed -e "s|__CHAINLINK_DB__|${CHAINLINK_DB}|g" | kubectl apply -f -

deploy and expose postgresql as a kubernetes service

kubectl apply -f kubernetes/postgresql/service.yaml
kubectl apply -f kubernetes/postgresql/deployment.yaml

You can check the installation

kubectl get pv
kubectl get pvc -n storage
kubectl get secret -n storage
kubectl get configmap -n storage
kubectl get pod -n storage
kubectl get svc -n storage

Deploy chainlink node

configuration

Here we set :

• Chainlink User email and password. Please change the configuration
• Chainlink wallet password. Please change the configuration
• Chainlink link address. Please put the link token contract address that was returned by the migration script (see above)
• Chainlink ETH URL: this is Ganache endpoint. The URL must be websocket (that’s reason why it starts with ws) . Also, as you can notice, I’m pointing to the host of my pod host.docker.internal . Using directly localhost or 127.0.0.1 will not work as they are local to the pod
• Chainlink ETH Chain ID: every Ethereum network has a unique ID. For instance, the Mainnet has a chainID equal to 1 while Rinkeby has a chainID equal to 4. My local ganache has a chainID equal to 1337

In your terminal , execute:

CHAINLINK_USER_EMAIL=dummy@gmail.com
CHAINLINK_USER_PASSWORD='<CHAINLINK_USER_PASS>'
CHAINLINK_WALLET_PASSWORD='<CHAINLINK_WALLET_PASS>'
CHAINLINK_LINK_ADDRESS='<LINK_CONTRACT_ADDRESS>'
CHAINLINK_ETH_URL='ws://host.docker.internal:8545'
CHAINLINK_ETH_CHAIN_ID='\"1337\"'

now deploy a kubernetes configmap

kubectl create -f kubernetes/basic-request-model/node/config.yaml -n chainlink — dry-run=client -o yaml | sed -e "s|__CHAINLINK_LINK_ADDRESS__|${CHAINLINK_LINK_ADDRESS}|g" | sed -e "s|__CHAINLINK_ETH_CHAIN_ID__|${CHAINLINK_ETH_CHAIN_ID}|g" | kubectl apply -f -

deploy a kubernetes secret

cat kubernetes/basic-request-model/node/secret.yaml | sed -e "s|__DATABASE_URL__|postgresql://${CHAINLINK_DB_USER}:${CHAINLINK_DB_PASSWORD}@postgres.storage:5432/${CHAINLINK_DB}?sslmode=disable|g" | sed -e "s|__USER_EMAIL__|${CHAINLINK_USER_EMAIL}|g" | sed -e "s|__USER_PASSWORD__|${CHAINLINK_USER_PASSWORD}|g" | sed -e "s|__USER_PASSWORD__|${CHAINLINK_USER_PASSWORD}|g" | sed -e "s|__WALLET_PASS__|${CHAINLINK_WALLET_PASSWORD}|g" | sed -e "s|__CHAINLINK_ETH_URL__|${CHAINLINK_ETH_URL}|g" | kubectl apply -n chainlink -f -

run chainlink node

• Run service

kubectl apply -n chainlink -f kubernetes/basic-request-model/node/service.yaml

• Run oracle node

kubectl apply -n chainlink -f kubernetes/basic-request-model/node/deployment.yaml

• Expose oracle node service on Nginx ingress controller

kubectl apply -n chainlink -f kubernetes/basic-request-model/node/ingress.yaml

check installation

You can run the following command and compare with the screenshot below

kubectl get secret -n chainlink
kubectl get configmap -n chainlink
kubectl get pod -n chainlink
kubectl get service -n chainlink
kubectl get ingress -n chainlink

Note that the pod is running and ingress shows that the service is available on http://localhost . Please open it on your browser

Sign in using username and password that were configured above then go to http://localhost/keys

Note the account address . It is the Oracle node account address, you will need it later

Note that I’ve already played around, my node has some ETH. Normally you will see 0 ETH balance but don’t worry we’ll credit your Oracle node account with some ETH later on.

create ETH price job

Remember our Consumer smart contract requests ETH price. Hence the oracle node must have a job to be able to understand how to perform a HTTP(s) GET request.

• Go to http://localhost/jobs/
• Add a new job “EthUint256” as explained here . Copy the specs, please do not forget to replace YOUR_ORACLE_CONTRACT_ADDRESS by the Operator address in config/addr.json (cfr. operatorAddress field)
• Last but not least, copy the jobID

create environment variables

Create now 2 environment variables that will be used next by truffle scripts. Please replace ORACLE_NODE_ADDRESS and JOBID

export ORACLE_NODE_ADDRESS=<ORACLE_NODE_ADDRESS>
export ETH256_JOBID=<JOBID>

Fund Consumer contract

As discussed above, the consumer contract must have some Link tokens in order to contact Link contract. Hence we will need scripts/basic-request-model/fund-contract.js for that .

Run the script

truffle exec scripts/basic-request-model/fund-contract.js --network ganache

Register and fund Oracle node

Remember our Oracle node runs on kubernetes and will have to report the API response to the Operator smart contract. At this stage, there are 2 issues:

• The oracle node is not whitelisted in Operator smart contract
• the oracle node doesn’t have enough ETH in order to call the Operator smart contract. In Ethereum, every transaction costs some gas

We will have to run scripts/basic-request-model/register-node.js . Note below ORACLE_NODE_ADDRESS environment variable (that was provided above) is used in “operator.setAuthorizedSenders”

truffle exec scripts/basic-request-model/register-node.js --network ganache

Back to http://localhost/keys , you should see now that the Oracle node has some ETH

Run event watcher

Now that everything is setup, we are going to run the event watcher in order to confirm that smart contracts emit events as expected ( see above)

Open a new terminal and run

node server/basic-request-model/server.js --network ganache

You can confirm that the server is up by running a simple HTTP request. In another terminal

curl localhost:3000/api

Request Eth price

We will have to run scripts/basic-request-model/req-eth-price.js . Note ETH256_JOBID environment variable (that was provided above) is used in “consumer.requestEthereumPrice” along with the operatorAddress

Now run

truffle exec scripts/basic-request-model/req-eth-price.js --network ganache

Once the script finishes, go back to the event watcher . The expected events are emitted from the different smart contracts

You can also check in the Oracle node UI that the job was triggered and that it run successfully. Go to http://localhost/runs

your most recent run must be marked as completed

Now if you click on the most recent run, you will find the details of the tasks

Now if you click on Jsonparse , you will get find the ETH price that was retrieved from the API call

Retrieve Eth price from smart contract

Now let’s verify that the Consumer contract stored the same value “2352.26 USD/ETH” . We will have to run scripts/basic-request-model/retrieve-eth-price.js .

In your terminal run:

truffle exec scripts/basic-request-model/retrieve-eth-price.js --network ganache

and that’s it , The Consumer smart contract has the most recent ETH price !😀