Add forked provider infrastructure and simple TradeModule / ExchangeAdapter tests #64

cgewecke · 2021-04-09T16:00:54Z

Adding some tests that validate logic against forked mainnet was recently recommended by an auditor. These are intended to be supplementary to tests with full mocks (and don't contribute to measured test coverage).

They are useful for:

testing operational edge-cases like sequences of trades
sanity checking the data generated by method signature encoding

PR:

Upgrades Hardhat to latest (2.1.2) (the forking feature receives regular bug-fixes)
Adds a forked provider job in CI (and cleans up the CI script a bit)
Adds forked integration tests for simple trades for ZeroEx, Uniswap, and SushiSwap.

Leaving comments in the diff below about misc. design issues...

~~(Unfortunately this is WIP because I can't get the ZeroEx tests to work as expected...will circle back here.)~~

cgewecke · 2021-04-09T16:03:02Z

.env.default

@@ -1,4 +1,5 @@
 # These are randomly generated hex so that CircleCI will work
+ALCHEMY_TOKEN=fake_alchemy_token


Adding an Alchemy token to the credential requirements. (Have added one to the CI environment variables)

This is necessary because Infura doesn't provide access to archive node data and the forked tests may request it.

cgewecke · 2021-04-09T16:05:36Z

contracts/mocks/ForceFunderMock.sol

+    function fund(address destination) public payable {
+        selfdestruct(payable(address(destination)));
+    }
+}


This contract is a trick to force an ether balance into a non-payable contract using self destruct. This is useful when you want to impersonate (send a message from) a non-payable forked contract and it has to have an eth balance to fund the tx.

cgewecke · 2021-04-09T16:07:39Z

utils/deploys/dependencies.ts

+  DAI_WHALE: "0x47ac0Fb4F2D84898e4D9E7b4DaB3C24507a6D503",
+  WETH_WHALE: "0x94B0A3d511b6EcDb17eBF877278Ab030acb0A878",
+  WBTC_WHALE: "0x2260fac5e5542a773aa44fbcfedf7c193bc2c599",
+


These are really contracts, not whales. They're used as a source of tokens to transfer into the Set system.

cgewecke · 2021-04-09T16:11:20Z

utils/test/accountUtils.ts

+    const funder = await deployer.mocks.deployForceFunderMock();
+    await funder.fund(whale, {value: ether(100)}); // Gas money
+  }
+};


This method initializes the connected signers of the forked token instances so we can call transfer on the token from a hardhat "impersonated", gas funded, mainnet token holder with adequate reserves.

cgewecke · 2021-06-10T17:10:08Z

Am removing the skipped ZeroEx adapter test because fails with an internal hardhat node error:

TransactionExecutionError: Number can only safely store up to 53 bits      
    at HardhatNode.mineBlock (...hardhat-network/provider/node.ts:327:13)

Initially thought this was a hardhat bug or error in my test setup but was running forked provider against a ZeroEx swap endpoint for set.js this week and saw the same error intermittently ... now I think this is likely something about the way 0x interacts with forking and that it may not be possible to get quotes for a block in the past.

The forked block is pinned in these tests for reproducibility.

…dapter tests (#64)

Inject gas setting into ABIs and move tasks from hardhat.config to tasks/ (#50) * Inject gas limit into abi on compilation * Move solc & typechain tasks from hardhat.config to tasks/ Add AaveGovernanceV2Adapter (#14) * add basic structure for SnapshotDelegationModule * add delegate function * add SnapshotGovernanceAdapter * code review changes * add aave v2 governance contracts to aave fixture * add AaveGovernanceV2Adapter * add tests for getting proper calldata * add integration test for aave governance v2 adapter * style fixes * code review changes Add CompoundBravoGovernanceAdapter (#17) * add GovernorBravo to compoundFixture * add CompoundBravoGovernanceAdapter * add CompoundBravoGovernanceAdapter tests and fix new gov deploys/fixture * add integration tests for CompoundBravoGovernanceAdapter * code review changes Fix AaveTokenV2Mintable type imports in utils/fixtures Add batch fetch balances and allowances to protocol viewer Remove aliases from import paths in utils/fixtures Bump package version to 0.0.29 Update build and coverage badges for repo rename Move libraryUtils to test folder Fix image tags in build badges for repo renaming Require fully qualified contract name params for library link util Move libraryUtils back to utils/common Bump new version Fix bug in SingleIndexModule's tradeRemainingWETH test (#65) Add CompoundWrapAdapter (#55) Add CompoundWrapAdapter Co-authored-by: mario <[email protected]> port ctoken oracle (#57) Port CToken Oracle Fix failing CompoundWrap tests (#66) Move getRandomAddress (w/out hardhat) to utils/common (#63) Add GeneralIndexModule contract (#59) Add GeneralIndexModule contract, UniswapV2ExchangeAdapterV2 updates, and BalancerV1ExchangeAdapter Co-authored-by: bweick <[email protected]> Remove UniswapYieldStrategy and tests. (#68) Remove UniswapYieldStrategy and tests Add yearn vault wrapper module and oracle (#61) Add Yearn vaults wrap adapter and oracle for valuing yTokens. Co-authored-by: bweick <[email protected]> Organize integrations folder into integration type. (#70) Updated javadocs for GeneralIndexModule. (#71) Updated javadocs for GeneralIndexModule. Bumped package. (#73) Remove trailing whitespace in GeneralIndexModule.sol Specify gas and blockGasLimit for localhost network Add new Uniswap transfer fee adapter Bump to 0.0.33 Add BalancerV1ExchangeAdapter tests (#72) * Add balancerV1ExchangeAdapter tests * Refactor uniswapV2ExchangeAdapter and add getDataParam test * Re-add deleted functions in deployAdapter.ts GeneralIndexModule internal review and external audit revisions (#75) * Make all revert reason strings < 32bytes * Remove _normalizedTargetMethod getter * Encapsulate _noTokensToSell#canSell logic as internal method * Encapsulate _allTargetsMet#targetUnmet logic in internal method * Validate exchange adapter when setting & store name as bytes32 hash * Consolidate position state and timestamp updates * Validate params for external view methods * Make all external setter method names use prefix `set` * Add RebalanceStarted event * Remove _caller param from onlyAllowedTrader modifier * Add getDefaultPositionRealUnit getter * Encapsulate data aggregation logic in #startRebalance in own method * Prohibit external positions for components in #initialize and #startRebalance * Refactor #createTradeInfo methods * Make weth state variable immutable * Delegate modifier logic to internal helpers in ModuleBase and GeneralIndexModule * Delete trader permissions on module removal / add removeModule tests * Add #validatePairsWithArray methods to AddressArrayUtils library and use in setter methods * Add approximatelyEquals helper in PreciseUnitMath and use in #_targetUnmet * Add AssetTargetsRaised event in RaiseAssetTargets with positionMultiplier arg * Remove trader from permissionInfo.tradersHistory when they are de-authorized * Add getAllowedTraders method and make tradersHistory contain unique elements Added an exchangeData param on the GIM (#76) Added an exchangeData param on the GIM. Allows passing in arbitrary parameter or execution info to an exchange. Made all GIM adapters specific to GIM and not compatible with TradeModule. restructure test/protocol/integration into folders (#79) Final review fixes for GeneralIndexModule (#83) * Add math boundary cases for protocol fee and raise target percentage * Add external position modules check to #_validateTradeParameters * Group internal methods by function and importance * Remove TargetUnitsUpdated event, update RebalanceStarted event * Add explanatory note about fixed / floating limit * 0.0.35 Upgrade hardhat to 2.2.1 (Berlin) / add test:fast yarn commands (#78) * Upgrade hardhat to 2.2.1 (Berlin) / add test:fast yarn commands * Increase Yearn external abi gas estimates by 1 order of magnitude KNC Migration Adapter (#86) * Add Kyber migration adapter contract * Add tests * Add @openzeppelin/[email protected] as dependency * Add external contract KyberNetworkTokenV2 * Updated KyberNetworkTokenV2 abi * Add missing test and remove proxy from contract * Remove external dependency openzeppelin/contracts-upgradeable * Fix variable names and bump package Add AXS migration wrap adapter (#87) * Add Axie Infinity migration wrap adapter contract * Add external TokenSwap contract * Add tests for axie infinity migration wrap adapter * Modify adapter to serve as middleware contract for migration * Fix bugs and add tests * Fixed external TokenSwap contract bytecode * Explain migration process in javadocs * Bump package to 0.0.37 Adding StandardTokenMock#mint Adding CompClaimAdapter feat: initial stab at Uniswap V3 adaptor feat: verify uniswap path for single hops WIP feat: verify path for multiple hops fix: add proper tests, require statements for sellTokenForToken refactor: move decoding tokens from encoded path into reusable function feat: implement sellTokenForEthToUniswapV3 and sellEthForTokenToUniswapV3 fix: address review feedback fix: sellEthForTokenToUniswapV3 impl and address review comments refactor: clean up Uniswap V3 path decoding logic fix: remove incorrect require check for sellEthForTokenToUniswapV3 Update package for public npm (make ethers a non-dev dep) (#92) ZeroExAdapter null recipient fix (#93) 0.0.38 (#94) Add Uniswap V3 Fixture (#95) * add basic Uniswap V3 fixture * add createPool to fixture * add quoter * add getTokenOrder helper * add addLiquidityWide helper function * add external contracts * sort imports * fix imports * improve comments * use exact values in tests * split uniswap contracts into v2/v3 folders * review changes * add weth and wbtc price in initialize function Uniswap V3 Exchange Adapter (#96) * add basic Uniswap V3 fixture * add createPool to fixture * add quoter * add getTokenOrder helper * add addLiquidityWide helper function * add external contracts * sort imports * fix imports * improve comments * use exact values in tests * split uniswap contracts into v2/v3 folders * add UniswapV3ExchangeAdapter * remove stray console log * fix comments * cleanup * add revert on path mismatch * add uniswap v3 tests to trade module integration tests * improve revert messages Add forked provider infrastructure and simple TradeModule / ExchangeAdapter tests (#64) Fix broken imports (#98) * v0.0.39 * fix imports Add uniswap v3 index exchange adapter (#89) * Add external Uniswap V3 contracts * Add UniswapV3 Index Exchange Adapter contract * Modify compiler-version in .solhint.json * Add tests * Fix coverage * Improve javadocs and add getEncodedTradePath() test * Use soldityPack in getEncodedTradePath test * Change compiler version to 0.6.10 * Add suggested changes * Remove getTradeEncodedPath function * Remove @uniswap/v3-periphery as dependency * Add @uniswap/v3-core as dependency * Use exactInputSingle and exactOutputSingle functions instead of exactInput and exactOutput * Move BytesLib to external/contracts/uniswap/v3 * Add Note to set exchange data in GIM * Remove @uniswap/v3-core dependency *Add suggested changes * Add getEncodedFeeData function and add suggested changes * Refactor tests to use UniswapV3 Fixture * Fix uniswap/v3/SwapRotuer ABI * Add integration tests with GIM * Trade DAI for WETH on UniswapV3 during rebalance * Trade WETH for WBTC on UniswapV3 during rebalance * Fix comment fix broken test (#102) Add AMMSplitter (#99) * add _getUniSplit function * add swapping * only rever when total output < min output * use better math for split * add tests * rename contracts and functions to be inline with spec * add multi hop support * add tradeExactOutput * clean up imports * shorten name * add javadocs * add getQuote function * clean up ratio calculation * remove hardhat console * add TradeSplitterExchangeAdapter * fix typo causing tests to break * add TradeSplitterIndexExchangeAdapter * improve coverage * increase coverage * refactor * remove redundant if in getQuote * improve reverts * improve coverage * move TradeSplitter into products * add TradeSplitterExchangeAdapter integration test * split getQuote into two functions for exact input/output * add TradeSplitter GIM integration tests * fix tests * make TradeSplitter adhere to the UniswapV2Router interface * use uniswap v2 adapters for trade splitter * cleanup * adhere to uniswap interface for getting quotes * change name of contracts * improve AMMSplitter tests * refactoring * fix bug with intermediary tokens with less than 18 decimals * refactor * add extra tests * update revert messages * improve comments * add events * use input and output tokens instead of path for events v0.0.41 (#103) change constructor parameters (#104) fix Felix comments (#105) Add Kyber V3 Index Exchange adapter (#101) * Add KyberV3 Index Exchange adapter contract * fix typo * Add Kyber V3 DMM fixture * Remove getPoolWithBestLiquidity() from adapter contract * Add tests for adapter contract * Add integration tests with GIM * Add tests for Kyber V3 DMM fixture * Fix bug * Fix bug in uniswap V3 fixture tests * Validate pool address in exchange adapter * Bump package to 0.0.43 * Move down pool address validation audit fixes (#108) Patch for real -> virtual -> real unit conversion. (#100) Patch for real -> virtual -> real unit conversion. Bump package. (#109) Update UniswapV2IndexExchangeAdapter to support two-hop trades (#110) * update adapter and tests * add integration test * improve comments * review changes bump package.json version number (#111) ClaimModule Audit fixes (#97) * low hanging audit changes * fix tests * audit changes * fix tests * improve tests * clean up contracts * use removeStorage * review changes SingularityNet migrator that burns unmigrated tokens, thus zeroing ou… (#113) SingularityNet migrator that burns unmigrated tokens, thus zeroing out position. Bump package. (#114) Address comments and add integration tests for Sushi wrap adapter

…dapter tests (#64)

cgewecke commented Apr 9, 2021

View reviewed changes

cgewecke force-pushed the chris/forked-provider-tests-infra branch from e5efe4c to 8da1a57 Compare May 5, 2021 21:58

cgewecke added 4 commits June 10, 2021 07:52

Add initial forked provider trade module integration tests

e89f532

Fix circle config

d98c3d3

Add sushi swap trade module integration test

62d264b

Update forked tests for UniswapExchangeAdapter changes

70a5a89

cgewecke force-pushed the chris/forked-provider-tests-infra branch from 8da1a57 to 70a5a89 Compare June 10, 2021 16:59

Remove zeroEx test because of forked provider error

c3d4505

cgewecke marked this pull request as ready for review June 10, 2021 17:11

bweick approved these changes Jun 10, 2021

View reviewed changes

cgewecke merged commit 465a916 into master Jun 10, 2021

cgewecke deleted the chris/forked-provider-tests-infra branch June 10, 2021 22:26

richardliang pushed a commit that referenced this pull request Jul 24, 2021

Add forked provider infrastructure and simple TradeModule / ExchangeA…

55b5bf5

…dapter tests (#64)

richardliang pushed a commit that referenced this pull request Aug 25, 2021

Add forked provider infrastructure and simple TradeModule / ExchangeA…

e835daa

…dapter tests (#64)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add forked provider infrastructure and simple TradeModule / ExchangeAdapter tests #64

Add forked provider infrastructure and simple TradeModule / ExchangeAdapter tests #64

Uh oh!

cgewecke commented Apr 9, 2021 •

edited

Loading

Uh oh!

cgewecke Apr 9, 2021

Uh oh!

cgewecke Apr 9, 2021 •

edited

Loading

Uh oh!

cgewecke Apr 9, 2021

Uh oh!

cgewecke Apr 9, 2021

Uh oh!

cgewecke commented Jun 10, 2021

Uh oh!

Uh oh!

		@@ -1,4 +1,5 @@
		# These are randomly generated hex so that CircleCI will work
		ALCHEMY_TOKEN=fake_alchemy_token

Add forked provider infrastructure and simple TradeModule / ExchangeAdapter tests #64

Add forked provider infrastructure and simple TradeModule / ExchangeAdapter tests #64

Uh oh!

Conversation

cgewecke commented Apr 9, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cgewecke Apr 9, 2021

Choose a reason for hiding this comment

Uh oh!

cgewecke Apr 9, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cgewecke Apr 9, 2021

Choose a reason for hiding this comment

Uh oh!

cgewecke Apr 9, 2021

Choose a reason for hiding this comment

Uh oh!

cgewecke commented Jun 10, 2021

Uh oh!

Uh oh!

cgewecke commented Apr 9, 2021 •

edited

Loading

cgewecke Apr 9, 2021 •

edited

Loading