Reducing ETH Gas by making an Asynchronous Tx with Oraclize

In a previous article (🤑) I was able to reduce an Ethereum transaction costing 95 Million (MM) gas down to 4.1MM by converting arrays to byte strings. This was a big step in the process of building clovers.network but 4.1MM gas was still unacceptable. I was able to reduce it again to 1.5MM by utilizing an Oracle to offload the bulk of the work and save only the result—basically making an asynchronous call on the Ethereum Virtual Machine (EVM).

The transaction in question contains a function that plays a game of Reversi using moves supplied by the user. If the game is valid and hasn’t previously been registered, the user becomes the owner of that board and is able to sell it as a Clover (✤). Furthermore, if the board is symmetrical then the user receives a mining reward in ERC20 ClubToken (︎♣︎) relative to the rarity of the symmetry. While the game is rather simple to program on the EVM the level of complexity is still very expensive. That’s because every step in the process of checking the game is saved along with the result of that game. This is important to prove the method of validation, however there’s another way to prove validation while not having to pay for it: ask an oracle 🔮.

An oracle provides a portal to the world outside of the EVM. If you want to know the current price of Ether in USD, Euro or GBP—ask an oracle. If you want to know the weather 🌤 in Chicago , who won the Cubs 🐻 game or whether your flight ✈️ to ORD is delayed—ask an oracle. You can also do things with an oracle that aren’t possible on the EVM like generate random numbers.

There’s some debate about whether these features belong on the Ethereum Blockchain, since in theory all transactions should be verifiable and repeatable—how can a URL request at a specific moment in time be repeatable? (For more information about that debate and oracles in general look here, here and here.) Luckily for me I wanted an oracle to call a function already on the EVM. That way the method of validation is still verifiable but I don’t have to spend gas recording all the steps producing the result.

He Said She Said…

Oraclize is an oracle that provides a great selection of datasources including Wolfram Alpha, IPFS and any publicly accessible URL. They also offer the ability to query various blockchains for basic info like block number and mining difficulty. I needed to run a non-transactional (constant) function on the Ethereum Blockchain using eth_call but Oraclize doesn’t offer it at this point. Instead I’ll use Infura’s publicly available Ethereum node to make a JSON RPC transaction (same way as Metamask does it). Before I get to building the JSON RPC POST request let’s look at the contract so far:

function claimGame(bytes28 firstMoves, bytes28 lastMoves) {if (isReal(firstMoves, lastMoves)) {saveGame(firstMoves, lastMoves);}}

function isReal(bytes28 p1, bytes28 p2) constant returns(bool) {// play the game and check for completeness and errors...}

function saveGame(bytes28 p1, bytes28 p2) {// finally save the game...}

Here checkGame() is a function with game moves as parameters—in this case the moves are stored in bytes28 format (for a similar technique storing data in bytes check out the arrays to bytes article I mentioned earlier). The first thing checkGame() does is check if the moves play a real game by using isReal(). This is the expensive part of the contract that I’m trying to avoid. However you’ll notice that it is a constant function, meaning it doesn’t change anything on the blockchain. That’s why it would be possible to call it with an oracle, who wouldn’t need to pay any gas to do so. Afterwards the oracle can send the results back to the contract to be saved cheaply.

Adding Oraclize

The first step to utilizing Oraclize is to add their contract to yours. You can download a copy of oraclizeAPI.sol from their github. Add it to the top of your contract and let your contract inherit the functions. In this case I’m calling my contract CheapTrick.

pragma solidity ^0.4.13;

import "./oraclizeAPI.sol";contract CheapTrick is usingOraclize {...}

The next step is triggering the Oracle with the designated URL datasource. The contract has inherited the function oraclize_query which takes a variety of different parameters depending on your needs. We’ll be using the format that takes the first param as the data source, the second param as the URL endpoint and the third parameter as the POST object to be sent along with the request. Alternatively you could add an integer representing the number of seconds to wait before triggering the request, and an explicit amount of gas to be used in the callback.

function claimGame(bytes28 firstMoves, bytes28 lastMoves) {oraclize_query('URL', 'https://infura.io', '{...}');}

The final step is calling the inherited __callback() function to handle the results of the URL datasource query.

function __callback(bytes32 queryId, string results) {if (results == 'true') saveGame(???, ???);}

You’ll notice in this callback you’ve lost the reference to which moves were being played. Oraclize provides a query ID to help with that process. In order to keep track of which callback belongs to which query you can keep track of them with a mapping and a struct like this:

pragma solidity ^0.4.13;

import "./oraclizeAPI.sol";contract CheapTrick is usingOraclize {

struct Moves {bytes28 firstMoves;bytes28 lastMoves;}

mapping (bytes32 => Moves) validIds;

// oraclize_query returns the query ID that is used in the mappingfunction claimGame(bytes28 firstMoves, bytes28 lastMoves) {bytes32 q = oraclize_query('URL', 'https://infura.io', '{...}');validIds[q].firstMoves = firstMoves;validIds[q].lastMoves = lastMoves;}

function __callback(bytes32 q, string result) {if (bytes(result)[65] == 0x31) {saveGame(validIds[q].firstMoves, validIds[q].lastMoves);}}

...

}

In this scenario the query IDs are saved in a mapping of a struct using the query ID as a key. When the callback is triggered the moves can be extracted again using that same query ID.

String Theories

You may have also noticed or been confused by the line if(bytes(result)[65] == 0x31) . This is the real way to perform if (results == ‘true’) which was used falsely earlier. The oraclize_query() hits a JSON RPC endpoint which should in turn call the previously seen isReal() function. This function returns a boolean but since Ethereum works in increments of bytes32 that bool value is returned as bytes32. Instead of returning the string “true” it returns the hexadecimal value of 1.

true

false

This is further complicated by the fact that the result in__callback() is actually a string. So it’s not returning bytes32 but rather bytes32 as represented by a string.

true

In order to detect whether the game is valid or not we need to look at the last value in that string and detect if it is a 1 or a 0. While working with strings in Solidity it’s important to remember that they are stored as byte arrays (bytes[]) of UTF8 characters. According to w3schools.com the UTF8 control characters in our string look like this:

string 0 = decimal 48 = hex 0x30string 1 = decimal 49 = hex 0x31string x = decimal 120 = hex 0x78

In Solidity our string as represented in bytes[] would look something like this:

string = "0x0000000000000000000000000000000000000000000000000000000000000001"

string[] = ["0", "x", "0", "0", ..., "1"];bytes[] = [0x30, 0x78, 0x30, 0x30, ..., 0x31];

We need to check the last element in the array so we use the same snippet from the contract above: bytes(result)[65] == 0x31 (remember the array has a length of 66 due to the 0x preface) and voila we detect whether the result was true or false.

At this point it may be good to point out that Oraclize also offers the ability to upload a snippet of custom code to IPFS with a docker configuration that would allow it to be deployed on an Amazon micro server long enough to be run with the result returned instead of a URL datasource. If this were done the return string could be more efficient than the one we get from the RPC endpoint. However a URL datasource costs the contract owner ~$0.01 per query and the micro server costs ~$0.50 per request. (If you’re still reading this you’ll know by now that I’m always lookin for those deals 🤑)

JSON RPC POST

For the last part it’s important to see what was actually inside of that JSON RPC POST object sent to the Infura endpoint which was earlier represented by the nefarious {…} . This means that we need to craft our transaction manually based on the JSON-RPC specs here and crafting the data object following the Ethereum Contract ABI specs here. Our basic eth_call function follows this format:

// Requestcurl -X POST --data '{"jsonrpc":"2.0","method":"eth_call","params":[{coming soon}],"id":1}'

// Result{"id":1,"jsonrpc": "2.0","result": "0x"}

The params array consists of an object with the contract address and the data being sent, plus the desired block number:

{"jsonrpc": "2.0", "method": "eth_call", "params": [{to:"0xFAK3W4LL374DDR355", data:"...."}, "latest"]}

The data value will be a hexadecimal representation of the desired function name and the parameters being sent along with it. As per the specs, the function name is represented by the first 4 bytes of the hash of the string of the name of the function (including any parameters 😳**)**. In our case using the web3.js utils to help, it would look as follows:

var utils = require('web3-utils')

let functionName = "isReal(bytes28,bytes28)"

functionName = utils.sha3(functionName)//0x6b3bd7986bb57b171ccf6056a91eae803767c4600238e08445ece9b98c39ca21

functionName = utils.hexToBytes(functionName)// [107, 59, 215, 152, 107, 181, 123, 23, 28, 207, 96, 86, 169, 30, 174, 128, 55, 103, 196, 96, 2, 56, 224, 132, 69, 236, 233, 185, 140, 57, 202, 33]

functionName = functionName.slice(0, 4)// [107, 59, 215, 152]

functionName = utils.bytesToHex(functionName)// **0x6b3bd798**

As a result our function name looks like 0x6b3bd798. The next part is making byte representations of the parameters. Since our moves are already in hexadecimal format we just need to adjust them from 28 bytes to 32 bytes by padding them and removing the 0x prefix:

var utils = require('web3-utils')

let firstMoves = "0xd9b7774f9af573c5d69d4996a971f147dfac39f7e9f37785891dfee5"

first32Moves = utils.padRight(first32Moves.slice(2), (32 * 2))// d9b7774f9af573c5d69d4996a971f147dfac39f7e9f37785891dfee500000000

let lastMoves = "0xbd9bb7ed12e559bfcaad69b5f04fa1061438927fc681167470000000"

lastMoves = utils.padRight(lastMoves.slice(2), (32 * 2))// bd9bb7ed12e559bfcaad69b5f04fa1061438927fc68116747000000000000000

Put them all together and we’ve got our data 🎉

{"jsonrpc": "2.0", "method": "eth_call", "params": [{to:"0xFAK3C0N7R4C7W411374DDR355", data:"**6b3bd798**d9b7774f9af573c5d69d4996a971f147dfac39f7e9f37785891dfee500000000bd9bb7ed12e559bfcaad69b5f04fa1061438927fc68116747000000000000000"}, "latest"]}

You can test the results using Oraclize’s great query tester here .This example code doesn’t correspond to a deployed contract so will not actually work. However, if you follow the link you’ll see a working example requesting the current block number. VERY IMPORTANT: Don’t forget to add a space character at the beginning or end of the POST payload. This tells Oraclize that the data is in fact a POST object. I had a lot of trouble with that until someone from the Oraclize team answered my github issue 🙏

In Conclusion

In total we’ve covered:

• What is an oracle
• How to import and use Oraclize’s API contract
• How to keep track of the query and the callback
• How to work with strings in Solidity
• How to craft a JSON RPC eth_call to the Infura endpoint