Solidity is a programming language designed for developing smart contracts that run on the Ethereum Virtual Machine (EVM). Smart contracts, once deployed into a certain address, cannot always be updated. Hence, rigorously testing it is crucial before deploying them to the chosen network. It is important to have in mind that a single mistake could lead to you, or your users, losing all their funds.
Smart contract testing enables the developer to identify bugs and possible vulnerabilities. This task requires a detailed analysis of the code to be done iteratively during the development cycle and also as a final step before deployment. It is the way of verifying that the smart contract behaves exactly as it is supposed to.
There is a wide range of different approaches for smart contract testing, from manual testing to end-to-end setups to automatically test all functionalities. Unit testing is a worthwhile testing approach. Tests are simple to write and enable the developer to create small, focused tests, each of them checking a specific part of the contract. Running unit tests requires the developer to use assertions: single simple and informal sentences which represent a specification for the contract, for example: “only the owner is able to transfer funds”.
Unit testing
Setting up a testing environment
It is clear that using the mainnet of the chosen network where contracts are going to run is very expensive, and even though testnet is free, it is slow if tons of tests are intended to be run after each change done on the code. Hence, local blockchains are the recommended alternative. They run on each developer machine, mine new blocks instantly and do not require getting ether/current network coins for testing.
In this article, the examples are going to be using Hardhat, but unit testing can also be done with Truffle. It is up to each developer to choose the framework to be used. Hardhat vs Truffle article laid out how they differ on certain aspects in order to be able to choose the most suitable for each project of interest.
Writing unit tests
An example of how unit tests should be written and run is going to be shown below.
step 1: setting up a hardhat project
- Choose an empty directory for this project.
- In case hardhat have not been installed yet, run
npm install --save-dev hardhat - Once installation is completed, run
npx hardhatand select “create a basic sample project” on what gets displayed on the terminal.
- Answer yes (y) when asked if you want to install some dependencies.
step 2: Adding the example contract to be tested
Inside /contracts directory, delete the contract that Hardhat includes by default and create a new file called “ContractToBeTested.sol”.
Inside that file copy and paste the following contract. It is important to be clear on the fact that this contract is a completely basic contract only for unit testing exemplifying purposes.
//SPDX-License-Identifier: Unlicense
pragma solidity ^0.8.0;
contract ContractToBeTested {
address payable public owner;
constructor() payable {
owner = payable(msg.sender);
}
modifier onlyOwner() {
require(msg.sender == owner,"caller is not contract owner");
_;
}
function withdraw(uint _amount) public payable {
// users can only withdraw up to a 1 % of contract balance at a time
uint contractBalance = (address(this)).balance;
require(contractBalance > 0, "There are no funds to withdraw");
require(
_amount <= contractBalance / 100,
"exceeds max amount allowed to be withdraw"
);
payable(msg.sender).transfer(_amount);
}
function withdrawAll() public onlyOwner {
owner.transfer(address(this).balance);
}
}Take time to inspect what should be tested.
It can be noticed that some logic in the contract relies on the owner being set correctly on the constructor. Hence, that is something that should be tested.
Moreover, in order to prevent users from draining all contract funds, withdraw method includes an important require statement. Due to the risks it has, it is crucial that this function works as expected.
Finally, the withdrawAll method is supposed to only be called by the owner, so this is another aspect that is necessary to test.
step 3: adding test files
Inside the /test folder, create a ContractToBeTested.test.js file. It is a good practice to have the test directory structured as a mirror of the contracts folder. This way, it makes it easier to identify which contract is tested by each test file.Usually each test file has the tests for a single contract. Hardhat comes with an already pre-written scaffold which is also useful to have as a base. Also, the moloch testing guide has a good set of principles designed for testing solidity smart contracts.
The first step of this stage is to install and import ChaiJS into the test file in order to be able to use expect and assert methods.
run npm install --save-dev chai inside the project directory to install Chai.
Then ethers is imported in order to have access to the library functions via the Hardhat package.
Each time a describe statement is open, it should be thought of as a general function scope that “describes” the test cases enumerated with “it” functions inside it. These “it” functions are each specific unit test target. For example it(“should revert withdrawAll attempt as caller is no owner”) when testing that withdrawAll should only be called by the contract owner.
A contract factory in ether.js is an abstraction used to deploy new instances of the smart contracts. This way, ContractTBT in this test file is a factory for instances of the ContractToBeTested contract. A new instance needs to be created from the contract factory in order to test it. This is what is being awaited on the let contractToTest = await ContractTBT.deploy({ value: ethers.utils.parseUnits("10", "ether"),}) line.
The before hook is added in order to maintain a clean file. It enables the developer to define certain logic that needs to be run before each test; if it weren’t for this hook, those parts of the testing would be repeated in each “it” function.
Code example
Add this content to the test file:
const { expect, assert } = require("chai");
const { ethers } = require("hardhat");
describe("here should be the ContractToBeTested general scope", function () {
let contractTBT, owner, auxiliarAccount;
before("deploy the contract instance first", async function () {
const ContractTBT = await ethers.getContractFactory("ContractToBeTested");
contractTBT = await ContractTBT.deploy({
value: ethers.utils.parseUnits("10", "ether"),
});
await contractTBT.deployed();
[owner,auxiliarAccount] = await ethers.getSigners();
});
it("it should set deployer of the contract as the owner", async function () {
assert.equal(await contractTBT.owner(), owner.address);
});
it("it should revert the withdrawal as it exceeds the limit of 1% of the contract balance", async function () {
let withdrawAmount = ethers.utils.parseUnits("1", "ether");
await expect(contractTBT.withdraw(withdrawAmount)).to.be.rejectedWith("exceeds max amount allowed to be withdraw");
});
it("it should suceed on the withdrawal requested by auxiliarAccount decreasing contract balance an amount equal to the one withdrawn ", async function () {
let withdrawAmount = ethers.utils.parseUnits("0.01", "ether");
//TODO completar este test
let auxiliarAccountBalanceBeforeWithdrawal = await ethers.provider.getBalance(auxiliarAccount.address);
let contractBalanceBeforeWithdrawal = await ethers.provider.getBalance(contractTBT.address);
const txReceipt = await ( await contractTBT.connect(auxiliarAccount).withdraw(withdrawAmount)).wait();
const txFee = txReceipt.cumulativeGasUsed * txReceipt.effectiveGasPrice;
let contractBalanceAfterWithdrawal = await ethers.provider.getBalance(contractTBT.address);
let auxiliarAccountBalanceAfterWithdrawal = await ethers.provider.getBalance(auxiliarAccount.address);
expect(auxiliarAccountBalanceAfterWithdrawal).to.equal(auxiliarAccountBalanceBeforeWithdrawal.add(withdrawAmount).sub(txFee));
expect(contractBalanceAfterWithdrawal).to.equal(contractBalanceBeforeWithdrawal.sub(withdrawAmount));
});
it("it should succeed on the withdrawal of all contract funds, made by the owne, and send them to it", async function () {
let ownerBalanceBeforeWithdrawal = await ethers.provider.getBalance(owner.address);
let contractBalance = await ethers.provider.getBalance(contractTBT.address);
const txReceipt = await (await contractTBT.withdrawAll()).wait();
let ownerBalanceAfterWithdrawal = await ethers.provider.getBalance(owner.address);
//it is important to take into account that the owner payed for the method execution so its balance is going to be
//increased but not an amount equal to contract balance as it has paid the fees
//owner balance should decrease an amount equal to withdrawAll transaction fee
const txFee = txReceipt.cumulativeGasUsed * txReceipt.effectiveGasPrice;
expect(ownerBalanceAfterWithdrawal).to.equal(ownerBalanceBeforeWithdrawal.add(contractBalance).sub(txFee));
});
it("it should revert the withdrawal of all contract funds as caller is not contract owner", async function () {
await expect(contractTBT.connect(auxiliarAccount).withdrawAll()).to.be.rejectedWith("caller is not contract owner");
});
});Finally, running npx hardhat test will execute all tests in the test directory and will give an output on the CLI that shows whether each test passed or failed. In case of failure, it gives details of the reason.
Test coverage
Now that unit testing has been explained and an example was given, “Test coverage” tool is going to be tackled. Test coverage is used to determine how much of the codebase is being tested. In solidity, the Hardhat toolbox includes the solidity coverage plugin.
Prior to running coverage and obtaining a report, the test coverage tool needs to be installed and the plugin, continuing supposing Hardhat is being used, needs to be imported on the hardhat-config.js file.
Then, unit tests must be written as the report is based on them.
Once you consider that the unit tests cover all the important logic and functionality of the contract, run test coverage. This way, once tests end their execution, a report will be displayed on the CLI showing how much of the code was covered by the tests. This report enables the developer to identify which parts of the code were correctly tested and for which parts tests are missing. Then, after making improvements on the unit tests, coverage report can be regenerated until the established threshold is met.
In order to obtain the coverage report run: npx hardhat coverage
For example, for the example contract and unit tests of the previous section, this is the coverage report:
Also, besides showing the report on the CLI, the report is saved to a ./coverage/ folder in the root directory.
Test coverage is an important metric for ensuring quality and reliability of the solidity code. It is a way of letting clear that the project has been tested and that tests covers exactly X percentage of the codebase. It is important to define which is the acceptable minimum of coverage and reach it before deploying contracts to mainnet.
Related Posts:
Posted in Blockchain, Smart Contract, Solidity
