Crypto: investigations around token pair checking system

Continuing with our Arbitrage management system in this post, I wanted to focus on restoring my PairChecker contract, and this will mostly lead me to the re-implementation of a layer in python to manager the solidity compiler and then some initial experiment on the BSC testnet which is in the end giving me a pretty poor impression, so I won't spend long on that, and at the end of this post, I will eventually decide to move to ganache instead.

  • At the end of our previous article on the topic we could get artibrage setups above the min profit values like the following:
    2022/06/02 11:52:56 [__main__] INFO: Quote token SAFEMOON value: 3.54682e-10 WBNB
    2022/06/02 11:52:56 [__main__] INFO: Quote token DAI value: 0.00327899 WBNB
    2022/06/02 11:52:56 [__main__] INFO: Quote token BANANA value: 0.000508487 WBNB
    2022/06/02 11:55:38 [__main__] INFO: Block 18336166: Found arb setup of 0.002436 WBNB with pairs on WBNB/QUACK#3
    2022/06/02 11:55:48 [__main__] INFO: Block 18336169: Found arb setup of 0.03963 WBNB with pairs on ASS#7/WBNB
    2022/06/02 11:56:20 [__main__] INFO: Block 18336180: Found arb setup of 0.001731 WBNB with pairs on WBNB/GINUX
  • Yet, clearly, we are not going to find arb setup with a value of 0.039 WBNB lol… Typically this will only happen with tokens where we have a large burn/transaction fee, which will in the end make an arbitrage on those pair actually not profitable.
  • ⇒ That's why I created the PairChecker contract to verify which tokens are “really valid” tokens that we may use during an arbitrage.
  • So I think it's now time to restore that contract too to filter our arbitrage setups even further.
  • And here is the method I just added to check a given pair:
        def check_pair(self, pair, qtoken, amount=100000):
            """Check a pair given a pair row."""
            paddr = pair[1]
            t0addr = pair[2]
            t1addr = pair[3]
            qaddr = qtoken.address()
    
            self.check(t0addr == qaddr or t1addr == qaddr, "Invalid quote token %s for pair %s/%s", qaddr, t0addr, t1addr)
    
            # Get the current balance of qtoken in the pair checker:
            b0 = qtoken.get_balance(as_value=False, account=self.pair_checker_address)
            t0 = self.chain.get_token(t0addr)
            t1 = self.chain.get_token(t1addr)
    
            self.check(
                b0 >= amount,
                "Initial balance of %s too low, cannot perform pair check for %s/%s.",
                qtoken.symbol(),
                t0.symbol(),
                t1.symbol(),
            )
    
            # get the exchange:
            dex0 = self.chain.get_exchange(pair[8])
            ifp = int(10000 - dex0.get_swap_fee_points(paddr))
            amount = int(amount)
            stype = 1 if dex0.is_flash_loan_supported() else 0
    
            target = t0 if qaddr == t1addr else t1
    
            opr = self.pair_checker_sc.build_function_call("checkPair", paddr, qaddr, target.address(), amount, ifp, stype)
            try:
                self.chain.get_gas_estimate(opr)
                clresult = TokenClassification.OK
            except ValueError as err:
                logger.error("Pair checker test failed for %s: %s", target.symbol(), str(err))
                clresult = TokenClassification.INVALID
    
            if clresult == TokenClassification.OK:
                # The pair was successfully checked.
                logger.info("PairChecker result for %s (against %s) is: PASSED", target.symbol(), qtoken.symbol())
            else:
                logger.info("PairChecker result for %s (against %s) is: FAILED", target.symbol(), qtoken.symbol())
    
            # target.set_classification(clresult)
    
            return clresult == TokenClassification.OK
  • Now using that method when we find a potential arb setup:
            if best_profit < self.min_profit:
                # Profit too low, so we ignore this arb setup.
                return
    
            # Get the tokens:
            t0 = self.chain.get_token(t0addr)
            t1 = self.chain.get_token(t1addr)
    
            # We should ensure that the tokens are "valid for arbitrage", so we check their classification:
            if (t0.is_class_unknown() or t1.is_class_unknown()) and not self.check_pair(best_p0, qtoken):
                # The target pair is not valid.
                return
    
            if t0.is_class_invalid() or t1.is_class_invalid():
                # Invalid target token.
                return
    
            logger.info(
                "Block %d: Found arb setup of %.4g %s with pairs on %s/%s",
                self.last_block_number,
                best_profit,
                self.native_symbol,
                t0.symbol(),
                t1.symbol(),
            )
  • Except that… this pair checker contract can only be used by its owner 😂… so I need to change the account I use by default in the ArbitrageManager.
  • And this means I need to provide again another private_key into the system hmmm… 🤔: on linux/cygwin systems this is not really a problem because we can store the environment variables in a simple file but with batch files, this is less straightforward…
  • I found this page with a potential solution:
    setlocal
    FOR /F "tokens=*" %%i in ('type Settings.txt') do SET %%i
    java -Dsetting1=%setting1% ...
    endlocal
  • But I still think I need to push this a bit further: I should use a pair of RSA keys to encrypt the data and store that data directly in my config file as usual: then I would only need to store 1 private key as environment variable: should be a good option I think!
  • Let's see how I can do that…
  • Found this page about encryption/decryption in python: https://www.geeksforgeeks.org/how-to-encrypt-and-decrypt-strings-in-python/
  • ⇒ I will build an dedicated component for that: the Encrypter class:
    """Encrypter utility functions"""
    import logging
    import rsa
    
    from nvp.nvp_component import NVPComponent
    from nvp.nvp_context import NVPContext
    import nvp.core.utils as utl
    
    logger = logging.getLogger(__name__)
    
    
    def create_component(ctx: NVPContext):
        """Create an instance of the component"""
        return Encrypter(ctx)
    
    
    class Encrypter(NVPComponent):
        """Encrypter component used to send automatic messages ono Encrypter server"""
    
        def __init__(self, ctx: NVPContext):
            """Script runner constructor"""
            NVPComponent.__init__(self, ctx)
    
            # Get the config for this component:
            self.config = ctx.get_config()["encrypter"]
    
        def generate_keys(self, size):
            """Generate a pair of RSA keys"""
            logger.info("Generating keys of size %d...", size)
            pub_key, priv_key = rsa.newkeys(size)
            # pub_hex = utl.bytes_to_hex(pub_key.save_pkcs1())
            # priv_hex = utl.bytes_to_hex(priv_key.save_pkcs1())
            pub_str = pub_key.save_pkcs1()
            priv_str = priv_key.save_pkcs1()
    
            pub_b64 = utl.bytes_to_b64(pub_str)
            priv_b64 = utl.bytes_to_b64(priv_str)
            pub_2 = utl.b64_to_bytes(pub_b64)
            priv_2 = utl.b64_to_bytes(priv_b64)
            self.check(pub_2 == pub_str, "Mismatch in pub key: %s != %s", pub_2, pub_str)
            self.check(priv_2 == priv_str, "Mismatch in priv key: %s != %s", priv_2, priv_str)
    
            # logger.info("Public key:\n%s", pub_key.save_pkcs1().decode('utf-8'))
    
            logger.info("Public key:\n%s", pub_b64)
            logger.info("Private key:\n%s", priv_b64)
            logger.info("Done.")
    
        def process_command(self, cmd):
            """Check if this component can process the given command"""
    
            if cmd == 'gen-keys':
                size = self.get_param("key_size")
                # logger.info("Should generate key of size %d", size)
                self.generate_keys(size)
                return True
    
            return False
    
    
    if __name__ == "__main__":
        # Create the context:
        context = NVPContext()
    
        # Add our component:
        comp = context.register_component("encrypter", Encrypter(context))
    
        context.define_subparsers("main", {
            'gen-keys': None,
        })
    
        psr = context.get_parser('main.gen-keys')
        psr.add_argument("-s", "--size", dest="key_size", type=int, default=2048,
                         help="Specify the size of the rsa keys.")
    
        comp.run()
    
  • Generating RSA keys on windows with
    cli.bat gen-rsa-keys
  • Now on the storage of that new private key… We cannot store it in an environment variable in windows because it is too long.
  • But maybe we could store it in the registry ? ⇒ No, not a good idea since we don't have clear control on the acces rights.
  • ⇒ Instead we should put that in a file in the user home folder.
  • On cygwin we need to resolve the home to the real windows home.
  • Update: To generate the RSA keys I will now rather use:
    cli.bat encrypter gen-keys
  • And that command above will also write the private key to a file directly and set the correct access rights on that file.
  • Now, time to encrypt/decrypt some content: Let's add a tab on the nervGate app to handle that.
  • OK it works just fine now and I can encrypt/decrypt data directly from the nervgate gui:
{{ blog:2022:0603:encrypt_decrypt.png }}
  • So I encrypted my private keys, and I can now store them in my config file with very high confidence 😎!
  • Note: And the nice thing is that now my CryptoView shortcut is working against thanks to that change ;-)!
  • So now I get my PairChecker running… and I seem to get some failure from it:
    2022/06/03 13:45:23 [__main__] ERROR: Pair checker test failed for BONFIRE#2: execution reverted: transferToken failed.
    2022/06/03 13:45:23 [__main__] INFO: PairChecker result for BONFIRE#2 (against WBNB) is: FAILED
    2022/06/03 13:46:30 [__main__] ERROR: Pair checker test failed for GINUX: execution reverted: transferToken failed.
    2022/06/03 13:46:30 [__main__] INFO: PairChecker result for GINUX (against WBNB) is: FAILED
    2022/06/03 13:47:17 [__main__] ERROR: Pair checker test failed for GINUX: execution reverted: transferToken failed.
    2022/06/03 13:47:17 [__main__] INFO: PairChecker result for GINUX (against WBNB) is: FAILED
    2022/06/03 13:47:56 [__main__] ERROR: Pair checker test failed for QUACK#3: execution reverted: transferToken failed.
    2022/06/03 13:47:56 [__main__] INFO: PairChecker result for QUACK#3 (against WBNB) is: FAILED
    2022/06/03 13:48:14 [__main__] ERROR: Pair checker test failed for QUACK#3: execution reverted: transferToken failed.
    2022/06/03 13:48:14 [__main__] INFO: PairChecker result for QUACK#3 (against WBNB) is: FAILED
    2022/06/03 13:48:19 [__main__] ERROR: Pair checker test failed for QUACK#3: execution reverted: transferToken failed.
    2022/06/03 13:48:19 [__main__] INFO: PairChecker result for QUACK#3 (against WBNB) is: FAILED
  • ⇒ Question is, to what extent are those failure legit ? And could I do anything about these to increase my number of valid arb setups ?
  • So let's get back to some solidity code now, yeeeppeee 🥳!
  • First, here are a few examples with more inputs that might be relevant to us:
    2022/06/03 13:54:16 [__main__] ERROR: Pair checker test failed for PIG: execution reverted: transferToken failed.
    2022/06/03 13:54:16 [__main__] INFO: PairChecker result for PIG (against WBNB) is: FAILED
    2022/06/03 13:54:16 [__main__] INFO: Pair address: 0x51dCAF423FE39F620A13379Cd26821cF8d433308, exchange: PancakeSwap, router: 0x05fF2B0DB69458A0750badebc
    4f9e13aDd608C7F
    2022/06/03 13:54:20 [__main__] ERROR: Pair checker test failed for QUACK#3: execution reverted: transferToken failed.
    2022/06/03 13:54:20 [__main__] INFO: PairChecker result for QUACK#3 (against WBNB) is: FAILED
    2022/06/03 13:54:20 [__main__] INFO: Pair address: 0xa19C85572f0Bf8D32145244d1886E59025aD0d60, exchange: ApeSwap, router: 0xcF0feBd3f17CEf5b47b0cD257aCf6
    025c5BFf3b7
    2022/06/03 13:56:22 [__main__] ERROR: Pair checker test failed for NFTART: execution reverted: transferToken failed.
    2022/06/03 13:56:22 [__main__] INFO: PairChecker result for NFTART (against WBNB) is: FAILED
    2022/06/03 13:56:22 [__main__] INFO: Pair address: 0x03A4eeA71075fE14BA09B27931c1D767D0a082BD, exchange: PancakeSwap, router: 0x05fF2B0DB69458A0750badebc
    4f9e13aDd608C7F
    2022/06/03 14:03:53 [__main__] ERROR: Pair checker test failed for MOONLIGHT#17: execution reverted: transferToken failed.
    2022/06/03 14:03:53 [__main__] INFO: PairChecker result for MOONLIGHT#17 (against WBNB) is: FAILED
    2022/06/03 14:03:53 [__main__] INFO: Pair address: 0xe6de19Ae48969aF0a6f78271e41D3CE47580eaFB, exchange: ApeSwap, router: 0xcF0feBd3f17CEf5b47b0cD257aCf6
    025c5BFf3b7
    2022/06/03 14:04:17 [__main__] ERROR: Pair checker test failed for BONFIRE#2: execution reverted: transferToken failed.
    2022/06/03 14:04:17 [__main__] INFO: PairChecker result for BONFIRE#2 (against WBNB) is: FAILED
    2022/06/03 14:04:17 [__main__] INFO: Pair address: 0xD3F478F0d5E98b01f757bc6cB54Db4C00b9838f2, exchange: PancakeSwap, router: 0x05fF2B0DB69458A0750badebc
    4f9e13aDd608C7F
    
  • Okay, so in my contract I have this transsferToken function:
      function transferToken(address token, address to, uint amount) private {
          (bool success, bytes memory data) = token.call(abi.encodeWithSelector(TRANSFER_SELECTOR, to, amount));
          require(success && (data.length == 0 || abi.decode(data, (bool))), 'transferToken failed.');
      }
  • ⇒ That must be where the error I get comes from.
  • And then… ohh my… I wrote the full checkPair() function in assembly 🤣… So now I need to figure out what I'm doing exactly.
  • So the steps we take are:
    • Mask the srcToken, amount and ifp values:
              srcToken := and(srcToken, 0xffffffffffffffffffffffffffffffffffffffff)
              amount := and(amount, 0xffffffffffffffffffffffffffff)
              ifp := and(ifp, 0xffff)
    • Get the free memory pointer location with let b := mload(0x40)
    • Add an offset to the free memory pointer of 0xc0 = 192 bits / 6 regs (32bytes each):
      mstore(0x40, add(b, 0xc0))
    • Store signature for getReserves() at b:
      mstore(b,shl(224,0x0902f1ac)
    • do a staticcall on the pair address with input data starting from b and length 4 bytes (ie. the signature only) and destination b with size of 2 registers:
      pop(staticcall(2300, pair, b, 0x04, b, 0x40))
    • ⇒ then b contains r0, and b+0x20 contains r1
    • We then declare a pointer starting just after r1: let ptr := add(b,0x40)
    • We put the signature for 'token0()' into ptr;
      mstore(ptr, shl(224, 0x0dfe1681))
    • And then do a staticcall to get the token0 stored at ptr:
      pop(staticcall(2300, pair, ptr, 0x04, ptr, 0x20))
    • Then we check if token0 address is the same a srcToken:
              let flipt := 1
              if eq(mload(ptr), srcToken) { flipt := 0 }
    • If not, then we need to flip the tokens in b / b+0x20:
              if eq(flipt,1) {
                // In this case we need to flipt the r0 and r1 value:
                mstore(ptr, mload(b))
                mstore(b, mload(add(b,0x20)))
                mstore(add(b,0x20), mload(ptr))
              }
    • Next we get ready to call transferToken(address,address,uint256) ⇒ sig: 'f5537ede' (?)
    • We put the sig 'a9059cbb' (for “transfer(address,uint256)”) into ptr:
      mstore(ptr, shl(224, 0xa9059cbb))
    • Then we put the address of the pair, and then the amount we want to transfer (of srcToken):
              // Then we push the destination address: (taking 32 bytes) (cf. https://ethereum.stackexchange.com/questions/67980/solidity-assembly-function-call-parameter-alignment)
              mstore(add(ptr, 0x04), pair)
              // Then we push the amount: taking 32 bytes
              mstore(add(ptr, 0x24), amount)
    • Then we call that transfer function on the srcToken address, returning nothing as result:
      pop(call(gas(), srcToken, 0, ptr, 0x44, ptr, 0x0))
    • OK, so we have “amount srcToken” inside the pair contract at that point for our address.
    • Then we compute the amount_out we should get in theory given the reserves r0, r1 and the ifp value
    • Then we get read for the call to the swap function:
      • Signature will be either for swap(uint256,uint256,address) if we don't support flashloan on that dex or swap(uint256,uint256,address,bytes) if there is flashloan support
    • Then we call the swap function on the pair contract:
              let c := 0
      
              if eq(stype,0) {
                c:= call(gas(), pair, 0, ptr, 0x64, ptr, 0x0)
              }
              if eq(stype,1) {
                // To store an array of bytes, in the first 32 bytes we store the offset to the start of the data,
                // from the current data definition line *not* taking the function signature into account,
                // ie. so from the address just after the function signature:
                // cf. https://docs.soliditylang.org/en/develop/abi-spec.html#use-of-dynamic-types
                // we pushed 4 x 32 bytes arguments in this data line, so the offset is 0x80
                mstore(add(ptr, 0x64), 0x80)  // offset to start of data in array
                mstore(add(ptr, 0x84), 0)  // Size of the array.
                c:= call(gas(), pair, 0, ptr, 0xa4, ptr, 0x0)
              }
  • If that call fails, we return a special return code:
            if eq(c,0) {
              let size := returndatasize()
              returndatacopy(b, 0, size)
    
              // add a code for the location:
              mstore(add(b, size), shl(255, 65)) // 65 => code for 'A'
              revert(b, add(size,1))
            } 
  • Then we apply the changes on our reserve values:
            mstore(b, add(mload(b), amount))
            mstore(add(b,0x20), sub(mload(add(b,0x20)), amountOut))
  • And we compute our amountOut for a swap back:
            amount := amountOut
            // amountOut := getAmountOut(amount, r1, r0, mask(ifp, 16))
            amountOut := getAmountOut(amount, mload(add(b,0x20)), mload(b), ifp)
  • Then we perform the swap in the opposite direction.
  • Note: to get the signature of a function in keccak-256 we can use the online site: https://emn178.github.io/online-tools/keccak_256.html
  • Note2: While doing my investigations here I got a couple of successful pair checks:
    2022/06/03 14:26:54 [__main__] INFO: PairChecker result for FTM#8 (against WBNB) is: PASSED
    2022/06/03 14:26:54 [__main__] INFO: Block 18367757: Found arb setup of 0.002772 WBNB with pairs on FTM#8/WBNB
    2022/06/03 14:45:16 [__main__] INFO: PairChecker result for AUTO#7 (against WBNB) is: PASSED
    2022/06/03 14:45:16 [__main__] INFO: Block 18368117: Found arb setup of 0.001228 WBNB with pairs on AUTO#7/WBNB
    
  • Okay, okay… so first of all… I don't think that's the contract I'm really using myself right now lol because I'm never calling the transferToken() function in this process anyway.
  • So do I have that contract available already ? let's check that.
  • Contract creation dates:
    • 386 days 21 hrs ago for 0x2F077C3fE54be5D5dDad3AaBdC1A063fEf0101EE
    • 387 days 2 hrs ago for 0x79Cbc72A24F760b4f3fBeaD112b5282941D99038
    • 388 days 6 hrs ago for 0x0A5EE1C7Bb11978585Cd0202cE06CB4E33b68d37
    • 432 days 2 hrs ago for 0x3db7e66639E5C94A725F6943FFBA6863c7DfC233
  • OK, so we have the latest version deployed, but that doesn't seem to be the latest version of the code.
  • Anyway, let's just check one of those failing pairs now: say WBNB/PIT at 0xB450CBF17F6723Ef9c1bf3C3f0e0aBA368D09bF5
  • Ohh…. Just checking at the bscscan page for this:

  • ⇒ But this is actually giving me an idea: when doing a pair check, I could progressively virtually increase the swap fee value to figure out what is the fee required to swap a given token 👍 Let's try that…
  • So I updated the code as follow to try to increate the swap fee:
            extra_fee = 0
    
            while extra_fee < 10000:
                try:
                    ifp = min(int(10000 - sfp - extra_fee), 10000)
                    opr = self.pair_checker_sc.build_function_call(
                        "checkPair", paddr, qaddr, target.address(), amount, ifp, stype
                    )
                    self.chain.get_gas_estimate(opr)
                    break
                except ValueError as err:
                    logger.error(
                        "Pair checker test failed for %s (extra_fee=%.2f%%): %s ",
                        target.symbol(),
                        extra_fee / 100.0,
                        str(err),
                    )
                    extra_fee += 100
    
            if extra_fee > 0:
                logger.warning("Token %s requires an extra swap fee of %.2f%%", target.symbol(), extra_fee / 100.0)
    
            clresult = TokenClassification.OK if extra_fee == 0 else TokenClassification.INVALID
  • Yet, even with that I'm still getting the same error until the end when trying to check some pairs:
    2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=0.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=1.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=2.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=3.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=4.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=5.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=6.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=7.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=8.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=9.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=10.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=11.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=12.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=13.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=14.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=15.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=16.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=17.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=18.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=19.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=20.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=21.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=22.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=23.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=24.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=25.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=26.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=27.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=28.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=29.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=30.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=31.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=32.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=33.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=34.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=35.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=36.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=37.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=38.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=39.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=40.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=41.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=42.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=43.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=44.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=45.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=46.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=47.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=48.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=49.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=50.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=51.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=52.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=53.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=54.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=55.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=56.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=57.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=58.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=59.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=60.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=61.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=62.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=63.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=64.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=65.00%): execution reverted: transferToken failed.
    2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=66.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=67.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=68.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=69.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=70.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=71.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=72.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=73.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=74.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=75.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=76.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=77.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=78.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=79.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=80.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=81.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=82.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=83.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=84.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=85.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=86.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=87.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=88.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=89.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=90.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=91.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=92.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=93.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=94.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=95.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=96.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=97.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:03 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=98.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:03 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=99.00%): execution reverted: transferToken failed.
    2022/06/03 15:55:03 [__main__] WARNING: Token FEG requires an extra swap fee of 100.00%
    2022/06/03 15:55:03 [__main__] INFO: PairChecker result for FEG (against WBNB) is: FAILED
    2022/06/03 15:55:03 [__main__] INFO: Pair address: 0x93D708BFea03c689F110dBe2E578D5568708F942, exchange: PancakeSwap, router: 0x05fF2B0DB69458A0750badebc
    4f9e13aDd608C7F, ifp: 80
    
  • So this execution reverted doesn't seem to be only a swap fee issue… hmmm, 🤔 I need to think more about that.
  • What I think I really need to do next is to rework my solidty pairchecker contract to ensure I'm using the latest version and provide more debug outputs somehow.
  • And while i'm at it, I would like to try to deploy my contracts directly from python… is that possible ? 😊 ⇒ investigating
  • So installing the py-solc-x package in our env: OK
  • Preparing a solc component:
    """Solidity Compiler class"""
    
    import logging
    
    from nvp.nvp_component import NVPComponent
    from nvp.nvp_context import NVPContext
    
    logger = logging.getLogger(__name__)
    
    
    def create_component(ctx: NVPContext):
        """Create an instance of the component"""
        return SolidityCompiler(ctx)
    
    
    class SolidityCompiler(NVPComponent):
        """SolidityCompiler component class"""
    
        def __init__(self, ctx):
            """blockchain constructor"""
            NVPComponent.__init__(self, ctx)
    
        def process_command(self, cmd):
            """Check if this component can process the given command"""
    
            if cmd == "compile":
                filename = self.get_param("contract_file")
                logger.info("Should file the file %s here", filename)
                return True
    
    
    if __name__ == "__main__":
        # Create the context:
        context = NVPContext()
    
        # Add our component:
        comp = context.register_component("solc", SolidityCompiler(context))
    
        context.define_subparsers(
            "main",
            {
                "compile": None,
            },
        )
    
        psr = context.get_parser("main.compile")
        psr.add_argument(
            "contract_file",
            type=str,
            help="Contract file to compile",
        )
    
        comp.run()
    
  • List installable compiler:
            if cmd == "list-installable-versions":
                vers = solcx.get_installable_solc_versions()
                logger.info("Found %d installable compiler versions:", len(vers))
                for ver in vers:
                    logger.info(" - %s", ver)
                return True
  • Compile the files from a given path:
            if cmd == "compile":
                filename = self.get_param("contract_file")
                if not filename.endswith(".sol"):
                    filename += ".sol"
    
                file_path = self.get_path(self.source_dir, filename)
                if not self.file_exists(file_path):
                    logger.warning("Cannot find source file %s", file_path)
                    return True
    
                logger.info("Should file the file %s here", file_path)
                return True
  • Specify compiler version to use:
            if cmd == "compile":
                filename = self.get_param("contract_file")
                if not filename.endswith(".sol"):
                    filename += ".sol"
    
                file_path = self.get_path(self.source_dir, filename)
                if not self.file_exists(file_path):
                    logger.warning("Cannot find source file %s", file_path)
                    return True
    
                vers = self.get_param("compiler_version")
                logger.info("Compiling %s...", file_path)
                result = solcx.compile_files([file_path], solc_version=vers)
                logger.info("Compilation result: %s", result)
                return True
  • Adding support to install a compiler version:
            if cmd == "install":
                vers = self.get_param("compiler_version")
                logger.info("Installing solidity compiler version %s...", vers)
                solcx.install_solc(version=vers)
                logger.info("Installation completed.")
                return True
  • Now we install the latest version of solidity:
    $ nvp solc install latest
  • One thing I noticed is that on windows I always get an error reported on the init of the solcx package due to that part of the solcx/install.py file:
    try:
        # try to set the result of `which`/`where` as the default
        _default_solc_binary = _get_which_solc()
    except Exception:
        # if not available, use the most recent solcx installed version
        if get_installed_solc_versions():
            set_solc_version(get_installed_solc_versions()[0], silent=True)
  • ⇒ So I'm now using an internal copy of that package to fix that issue manually.
  • Now trying a simple compilation:
    $ nvp solc compile test -v "0.8.14"
    2022/06/04 00:20:23 [__main__] INFO: Compiling D:\Projects\NervHome\contracts\sources\test.sol...
    2022/06/04 00:20:23 [__main__] INFO: Compilation result: {'/Projects/NervHome/contracts/sources/test.sol:Foo': {'abi': [{'inputs': [], 'name': 'bar',
    'outputs': [], 'stateMutability': 'nonpayable', 'type': 'function'}], 'bin-runtime': '6080604052348015600f57600080fd5b506004361060285760003560e01c8063febb0f7e14602d575b600080fd5b60336035565b005b56fea2646970667358221220277588ef74e40ad0c09328268351877ffcee83ff1e8a684583e48f2019ca87cb64736f6c634300080e0033'}}
  • ⇒ Feeeww this is finally working!
  • Arrfff… And now trying to install the solc compiler version 0.6.6 just for testing I get another error from the py-solc-x package:
    $ nvp solc install 0.6.6
    2022/06/04 07:32:47 [__main__] INFO: Installing solidity compiler version 0.6.6...
    2022/06/04 07:32:48 [solcx] INFO: Downloading from https://solc-bin.ethereum.org/windows-amd64/solc-windows-amd64-v0.6.6+commit.6c089d02.zip
    Traceback (most recent call last):
      File "D:\Projects\NervHome\nvh\crypto\blockchain\solidity_compiler.py", line 105, in <module>
        comp.run()
      File "D:\Projects\NervProj\nvp\nvp_component.py", line 77, in run
        res = self.process_command(cmd)
      File "D:\Projects\NervProj\nvp\nvp_component.py", line 67, in process_command
        return self.process_cmd_path(self.ctx.get_command_path())
      File "D:\Projects\NervHome\nvh\crypto\blockchain\solidity_compiler.py", line 48, in process_cmd_path
        solcx.install_solc(version=vers)
      File "D:\Projects\NervHome\nvh\crypto\solcx\install.py", line 440, in install_solc
        _install_solc_windows(version, filename, show_progress, solcx_binary_path)
      File "D:\Projects\NervHome\nvh\crypto\solcx\install.py", line 603, in _install_solc_windows
        temp_path.rename(install_path)
      File "D:\Projects\NervProj\.pyenvs\bsc_env\lib\pathlib.py", line 1232, in rename
        self._accessor.rename(self, target)
    OSError: [WinError 17] Impossible de déplacer le fichier vers un lecteur de disque différent: 'D:\\LiberKey\\MyApps\\Cygwin\\tmp\\solcx-tmp-36296' ->
    'C:\\Users\\kenshin\\.solcx\\solc-v0.6.6'
    2022/06/04 07:33:01 [nvp.components.runner] ERROR: Error occured in script command: ['D:\\Projects\\NervProj\\.pyenvs\\bsc_env\\python.exe', 'D:\\Proj
    ects\\NervHome/nvh/crypto/blockchain/solidity_compiler.py', 'install', '0.6.6'] (cwd=None)
    
  • So, that's too bad… 😠 And I'm now really starting to think I should handle the compilers on my own in the SolidityCompiler component.
  • So here is my own solc install management layer:
        def install_solc(self, version):
            """Install a specific version of the solcx compiler."""
            url = f"{self.base_url}/{self.platform}-amd64/list.json"
            vlist = self.make_get_request(url).json()
    
            releases = vlist["releases"]
    
            if version == "latest":
                version = vlist["latestRelease"]
                logger.info("Using latest solc version: %s", version)
    
            dest_path = self.get_path(self.compiler_dir, f"solc-{version}")
            if self.dir_exists(dest_path):
                logger.info("Solc compiler v%s is already installed.", version)
                return
    
            # make the folder:
            self.make_folder(dest_path)
    
            src_file = f"{self.base_url}/{self.platform}-amd64/{releases[version]}"
    
            # download the file:
            tools: ToolsManager = self.get_component("tools")
            ext = self.get_path_extension(src_file)
            filename = "solc"
            if ext in [".exe", ".zip"]:
                filename += ext
    
            dst_file = self.get_path(dest_path, filename)
            tools.download_file(src_file, dst_file)
    
            # if we have a zip we extract it:
            if ext == ".zip":
                tools.unzip_package(dst_file, dest_path)
                self.remove_file(dst_file)
                # we should have a solc.exe file now:
                dst_file = self.get_path(dest_path, "solc.exe")
                self.check(self.file_exists(dst_file), "Could not extract solc compiler %s", version)
  • And with that I can install the solc compilers gracefully on windows:
    $ nvp solc install 0.6.6
    2022/06/04 08:23:26 [__main__] INFO: Installing solidity compiler version 0.6.6...
    2022/06/04 08:23:26 [nvp.components.tools] INFO: Downloading file from https://solc-bin.ethereum.org/windows-amd64/solc-windows-amd64-v0.6.6+commit.6c
    089d02.zip...
    [==================================================] 7196296/7196296 100.000%
    2022/06/04 08:23:35 [__main__] INFO: Installation completed.
    
    $ nvp solc install 0.4.1
    2022/06/04 08:23:49 [__main__] INFO: Installing solidity compiler version 0.4.1...
    2022/06/04 08:23:50 [nvp.components.tools] INFO: Downloading file from https://solc-bin.ethereum.org/windows-amd64/solc-windows-amd64-v0.4.1+commit.4f
    c6fc2c.zip...
    [==================================================] 3689890/3689890 100.000%
    2022/06/04 08:23:54 [__main__] INFO: Installation completed.
    
    $ nvp solc install latest
    2022/06/04 08:24:21 [__main__] INFO: Installing solidity compiler version latest...
    2022/06/04 08:24:21 [__main__] INFO: Using latest solc version: 0.8.14
    2022/06/04 08:24:21 [nvp.components.tools] INFO: Downloading file from https://solc-bin.ethereum.org/windows-amd64/solc-windows-amd64-v0.8.14+commit.8
    0d49f37.exe...
    [==================================================] 8831488/8831488 100.000%
    2022/06/04 08:24:32 [__main__] INFO: Installation completed.
    
  • Now I need to be able to list those installed paths instead of the default version from py-solcx:
        def get_installable_solc_versions(self):
            """Get the list of installable versions"""
            vlist = self.get_compilers_registry()
            releases = vlist["releases"]
            return list(releases.keys())
    
  • And the list of installed versions:
        def get_installed_solc_versions(self):
            """Get the list of installed versions"""
            if self.installed_versions is None:
                flist = self.get_all_folders(self.compiler_dir)
                flist = [fname[5:] for fname in flist if fname.startswith("solc-")]
                flist.reverse()
                self.installed_versions = flist
            return self.installed_versions
  • OK And now I'm back on rails with my manual solc install management 👍!:
    $ nvp solc compile test -v 0.6.6
    2022/06/04 08:50:37 [__main__] INFO: Using compiler path D:\Projects\NervHome\tools\windows\solc-0.6.6\solc.exe
    2022/06/04 08:50:37 [__main__] INFO: Compiling D:\Projects\NervHome\contracts\sources\test.sol...
    2022/06/04 08:50:37 [__main__] INFO: Compilation result: {'D:/Projects/NervHome/contracts/sources/test.sol:Foo': {'abi': [{'inputs': [], 'name': 'bar'
    , 'outputs': [], 'stateMutability': 'nonpayable', 'type': 'function'}], 'bin': '6080604052348015600f57600080fd5b50606d80601d6000396000f3fe608060405234
    8015600f57600080fd5b506004361060285760003560e01c8063febb0f7e14602d575b600080fd5b60336035565b005b56fea2646970667358221220e71dd39daca43ee41d13f156dc66ba
    05b41804ac8e472effd1a3726292f3370f64736f6c63430006060033'}}
  • When compiling a contract I should write the corresponding ABI to a dedicated json file, and write the bytecode in another json file too…
  • OK: now writing ABI json file, and saving the contract bytecode into a data file as follow:
    {
      "bytecode": {
        "Foo": "6080604052348015600f57600080fd5b50606d80601d6000396000f3fe6080604052348015600f57600080fd5b506004361060285760003560e01c8063febb0f7e14602d575b600080fd5b60336035565b005b56fea2646970667358221220e71dd39daca43ee41d13f156dc66ba05b41804ac8e472effd1a3726292f3370f64736f6c63430006060033"
      },
      "source_file_hash": {
        "test.sol": 15787391762704133169
      },
      "contract_file": {
        "Foo": "test.sol"
      }
    }
  • Now re-compiling the same file with no change will be optimized:
    $ nvp solc compile test -v 0.6.6
    2022/06/04 21:59:09 [__main__] INFO: No change detected in test.sol
  • If I use the version value “auto” (which I will make the default) the system should select the compiler to use based on the solidity version pragma line in each source file.
  • ⇒ So now automatically finding the min compiler version with this code:
        def find_solidity_pragma_version(self, file_path):
            """Find the pragma version in a given file"""
            content = self.read_text_file(file_path)
            content = self.remove_comments(content)
    
            # Example statement:
            # pragma solidity >=0.6.6;
    
            vers = None
            pat = re.compile(r"^\s*pragma\s+solidity\s+.*([0-9]+\.[0-9]+\.[0-9]+)\s*;\s*$")
            for line in content.splitlines():
                match = pat.search(line)
                if match is not None:
                    vers = match.group(1)
                    break
    
            if vers is None:
                # No version found, using latest
                logger.info("No solidity version found in %s, using latest.", file_path)
                return "latest"
    
            return vers
  • And finally, pushing this one step further, I could also automatically install the compiler I specified if it's not installed already:
        def get_compiler_path(self, version, auto_install=True):
            """Retrieve a compiler path"""
            if version == "latest":
                version = self.get_latest_version()
                logger.info("Using latest solc version %s", version)
    
            ext = ".exe" if self.is_windows else ""
            solc_path = self.get_path(self.compiler_dir, f"solc-{version}", f"solc{ext}")
            if not self.file_exists(solc_path) and auto_install:
                self.install_solc(version)
    
            self.check(self.file_exists(solc_path), "Compiler version %s is not available.", version)
            return solc_path
  • And now bumping the compiler version to 0.7.0 in our test.sol file works jsut as expected:
    $ nvp solc compile test
    2022/06/04 22:45:03 [__main__] INFO: Auto selecting solidity compiler '0.7.0'
    2022/06/04 22:45:03 [nvp.components.tools] INFO: Downloading file from https://solc-bin.ethereum.org/windows-amd64/solc-windows-amd64-v0.7.0+commit.9e
    61f92b.zip...
    [==================================================] 7689749/7689749 100.000%
    2022/06/04 22:45:16 [__main__] INFO: Using compiler path D:\Projects\NervHome\tools\windows\solc-0.7.0\solc.exe
    2022/06/04 22:45:16 [__main__] INFO: Compiling D:\Projects\NervHome\contracts\sources\test.sol...
    2022/06/04 22:45:16 [__main__] INFO: Compilation result: {'D:/Projects/NervHome/contracts/sources/test.sol:Foo': {'abi': [{'inputs': [], 'name': 'bar'
    , 'outputs': [], 'stateMutability': 'pure', 'type': 'function'}, {'inputs': [], 'name': 'bar2', 'outputs': [], 'stateMutability': 'pure', 'type': 'fun
    ction'}], 'bin': '6080604052348015600f57600080fd5b5060818061001e6000396000f3fe6080604052348015600f57600080fd5b506004361060325760003560e01c806364bdaf57
    146037578063febb0f7e14603f575b600080fd5b603d6047565b005b60456049565b005b565b56fea264697066735822122002b40752aed826b87c8ab40efa8520cf1eb31cc0f0721638ca
    b9b775b61fa6ac64736f6c63430007000033'}}
    
  • ⇒ all good 👍!
  • Another feature I think would be nice to have would be to nice to have would be to compile all the source files available in the contracts/sources/ folder with a simple nvp solc compile all command. So let's see how we can add that.
  • And here is the main function used to handle multiple input files at the same time:
        def compile_files(self, pattern):
            """Compile multiple files"""
            filename = pattern
            if filename == "all":
                # We should collect all the solidity files in the source folder:
                filename = self.source_dir
    
            if self.is_relative_path(filename):
                # use the source dir as base folder:
                filename = self.get_path(self.source_dir, filename)
    
            source_files = []
    
            if self.dir_exists(filename):
                # We collect all the files recursively:
                source_files = self.get_all_files(filename, exp=r"\.sol", recursive=True)
                source_files = [self.get_path(filename, src) for src in source_files]
            else:
                # Assume we are targetting a single file here:
                if not filename.endswith(".sol"):
                    filename += ".sol"
    
                if self.file_exists(filename):
                    source_files.append(filename)
    
            if len(source_files) == 0:
                logger.info("No source file found with input '%s'", pattern)
                return
    
            src_base_dir = self.source_dir.replace("\\", "/") + "/"
            num = len(src_base_dir)
    
            # Compile each file:
            for src_file in source_files:
                src_file = src_file.replace("\\", "/")
                if src_file.startswith(src_base_dir):
                    src_file = src_file[num:]
    
                self.compile_file(src_file)
  • Another thing I might need is a simple flag to “force” the recompilation, this may not be strictly necessary, but it feels nice to have. So I'm just adding a “force” command line parameter:
        psr = context.get_parser("main.compile")
        psr.add_argument(
            "file_pat",
            type=str,
            nargs="?",
            default="all",
            help="Contract file to compile",
        )
        psr.add_argument(
            "-v",
            "--version",
            dest="compiler_version",
            default="auto",
            type=str,
            help="Compiler version to use",
        )
        psr.add_argument(
            "-f",
            "--force",
            dest="force",
            action="store_true",
            help="Force recompilation",
        )
  • And using that eventually in the compile_file() method:
            # Check the file hash:
            file_hash = self.compute_file_hash(file_path)
            if not forced and not self.is_source_file_changed(filename, file_hash):
                logger.info("No change detected in %s", filename)
                return True
  • So now it's time a try and compile our PairChecker contract again with this python framework: I'm going to call this version 5 to ensure we do not conflict with anything existing so far.
  • Cool, The contract PairCheckerV5 is building just fine with a simple command like nvp solc compile 👍!
  • Okay, so… now, what's next ? Next I need to add support to deploy a contract on a given chain with a given account: I think this should be handled by the BlockchainManager, right ? Let's implement that…
  • Adding initial support for deploy command in bchain:
            if cmd == "deploy":
                chain_name = self.get_param("chain")
                chain: EVMBlockchain = self.get_component(f"{chain_name}_chain")
    
                account = self.get_param("account")
    
                contract = self.get_param("contract")
                logger.info("Should deploy the contract %s on %s with account %s", contract, chain_name, account)
                return True
  • Initial test:
    $ nvp bchain deploy Foo -c bsc -a main
    2022/06/05 18:39:01 [__main__] INFO: Should deploy the contract Foo on bsc with account main
  • What I think I should do next, is to add support for the bsc testnet: before I was usually deploying my contracts directly on the main net to test them in real usage case, but now I really think I should improve a bit on my testing procedure.
  • I was considering using ganache-cli but this would imply another large nodejs sub-project management system, and I don't have the energy/time for that for the moment. So I think it's a simpler option to just try to run my contracts on the testnet of interest to see what is the behavior I get there.
  • Here is my initial config for a new testnet_bsc chain:
        "testnet_bsc": {
          "address": "yyy",
          "private_key": "xxx",
          "chain_id": 97,
    
          "provider_url": "https://data-seed-prebsc-1-s1.binance.org:8545/",
          // https://data-seed-prebsc-2-s1.binance.org:8545/
          // https://data-seed-prebsc-1-s2.binance.org:8545/
          // https://data-seed-prebsc-2-s2.binance.org:8545/
          // https://data-seed-prebsc-1-s3.binance.org:8545/
          // https://data-seed-prebsc-2-s3.binance.org:8545/
    
          "short_name": "testnet_bsc",
          "max_gas_price": 15,
          "bscscan_api_key": "zzz",
          "chain_db_name": "testnet_bsc_chain",
          "chain_db_user": "uuu",
          "chain_db_password": "ppp",
          "gas_price_weights": {
            "default": [20.0, 0.8, 0.2, 0.0],
            "fast": [15.0, 0.4, 0.6, 0.0],
            "high": [10.0, 0.0, 1.5, 0.0],
            "ultra": [8.0, 0.0, 0.9, 0.1]
          },
          "native_symbol": "BNB",
          "token_desc_symbol_map": {
            "BNB": "WBNB"
          }
        },
  • Now also registering support to create that chain as a dynamic component… hmmm, except that this would mean duplicating the class “BinanceSmartChain” just to load a different config on creation: I don't like that too much, and I think I could do better by passing the construction arguments to the component we are constructing.
  • ⇒ I think I have a nice solution here: I introduced the concept of “construction frames” in the NVPContext class when creating a new dynamic component:
            # We have a dyn component module name, so we try to load it:
            mname = dyn_comps[cname]
            args = mname.split(":")
            mname = args.pop(0)
            logger.debug("Loading dynamic component %s from module %s", cname, mname)
    
            comp_module = import_module(mname)
    
            # Add a construct frame for this component:
            self.construct_frames.append({
                "component_name": cname,
                "module": mname,
                "args": args
            })
            comp = comp_module.create_component(self)
            # Remove the construct frame:
            self.construct_frames.pop()
  • And thus I should be able to pass additional arguments as strings on construction of a given component as follow for instance:
        def __init__(self, ctx):
            """blockchain constructor"""
            # Retrieve the config to use from the construct_frame:
            args = ctx.get_construct_args()
            cfgname = "bsc" if args is None else args[0]
            logger.info("Using config %s to create BinanceSmartChain object", cfgname)
            cfg = ctx.get_config()["blockchains"][cfgname]
            EVMBlockchain.__init__(self, ctx, cfg)
  • So let's test creating our testnet_bsc chain with that:
    $ nvp bchain collect-gas-price -c testnet_bsc
    2022/06/05 22:26:40 [nvh.crypto.bsc.binance_smart_chain] INFO: Using config testnet_bsc to create BinanceSmartChain object
    2022/06/05 22:26:50 [nvh.crypto.blockchain.evm_blockchain] INFO: Block 19941298: num_tx=2, mode=18.000, mean=18.000, dev=0.000
  • Cool! So this works. Let's move to the next point: the simplified handling of the accounts. I need a standardized system to assign a simple name to an EVM account. THe best option for this is probably to store the accounts in a dedicated entry in out config, so let's do that:
        def set_account(self, aname):
            """Set the current account by name"""
            account = self.ctx.get_config()["crypto_accounts"][aname]
            self.set_account_address(account["address"])
            self.set_private_key(account["private_key"])
  • OK, now that we have a clean account management system, let's also build all the BSC exchanges I have as simple instances of the UniswapBase class with a custom config just like for the BinanceSmartChain class.
  • I have replace the 6 trivial class implementation for the following DEXes:
        "pancakeswap": "nvh.crypto.bsc.pancake_swap",
        "pancakeswap2": "nvh.crypto.bsc.pancake_swap_2",
        "bakeryswap": "nvh.crypto.bsc.bakery_swap",
        "apeswap": "nvh.crypto.bsc.ape_swap",
        "mdexswap": "nvh.crypto.bsc.mdex_swap",
        "julswap": "nvh.crypto.bsc.jul_swap
  • … With a simple invocation of the UniswapBase component constructor with arguments:
        "pancakeswap": "nvh.crypto.blockchain.uniswap_base:PancakeSwap",
        "pancakeswap2": "nvh.crypto.blockchain.uniswap_base:PancakeSwap2",
        "bakeryswap": "nvh.crypto.blockchain.uniswap_base:BakerySwap",
        "apeswap": "nvh.crypto.blockchain.uniswap_base:ApeSwap",
        "mdexswap": "nvh.crypto.blockchain.uniswap_base:MdexSwap",
        "julswap": "nvh.crypto.blockchain.uniswap_base:JulSwap"
  • Simply taking as argument the name of the config to use for the swap we want to create:
    def create_component(ctx: NVPContext):
        """Create an instance of the component"""
        # Retrieve the swap name as argument:
        args = ctx.get_construct_args()
        swap_name = args[0]
        cfg = ctx.get_config()["exchanges"][swap_name]
    
        # get the blockchain:
        chain_name = cfg["chain"]
        chain = ctx.get_component(f"{chain_name}_chain")
        return UniswapBase(ctx, chain, cfg)
    
  • Okay so now back to our deployment command nvp bchain deploy Foo -c bsc -a main: let's try to actually deploy the smart contract.
  • I implemented the following method in EVMBlockchain:
        def deploy_contract(self, sc_name, max_gas=None, gas_price="default", args=None):
            """Deploy a contract given by name with the current account"""
    
            # First we get the bytecode for that contract:
            solc: SolidityCompiler = self.get_component("solc")
            desc = solc.get_contract_desc(sc_name)
    
            contract = self.web3.eth.contract(abi=desc["abi"], bytecode=desc["bytecode"])
    
            # 5. Build constructor tx
            args = args or []
            opr = contract.constructor(*args)
    
            gas_est = None
            if max_gas is None:
                max_gas = self.get_gas_estimate(opr)
                logger.info("Estimated gas for contract deployment: %d", max_gas)
                # Add some additional gas just in case:
                gas_est = max_gas
                max_gas *= 1.2
    
            receipt = self.perform_operation(opr, max_gas=max_gas, gas_price=gas_price, gas_est=gas_est)
    
            logger.info("Contract %s deployed on %s at address: %s", sc_name, self.short_name, receipt.contractAddress)
    
  • But well, my first attempt failed because of insufficient funds:
    $ nvp bchain deploy Foo -c testnet_bsc
    2022/06/06 21:14:44 [nvh.crypto.blockchain.solidity_compiler] INFO: No change detected in test.sol
    2022/06/06 21:14:45 [nvh.crypto.blockchain.evm_blockchain] INFO: Estimated gas for contract deployment: 81293
    2022/06/06 21:14:46 [nvh.crypto.blockchain.evm_blockchain] INFO: Processing transaction: {'value': 0, 'from': '0xxxxxxxxxxxxx', 
    'chainId': 97, 'gas': 121939, 'gasPrice': 15000000000, 'nonce': 0, 'data': '0x6080604052348015600f57600080fd5b5060818061001e6000396000f3fe
    6080604052348015600f57600080fd5b506004361060325760003560e01c806364bdaf57146037578063febb0f7e14603f575b600080fd5b603d6047565b005b60456049565b005b565
    b56fea2646970667358221220750890c4ea9c357e18dcc938c60c82b3bfeaf7beed800344a92a34992ba74fa764736f6c63430006060033', 'to': b''}
    2022/06/06 21:14:46 [nvh.crypto.blockchain.evm_blockchain] INFO: Error while trying to perform operation: {'code': -32000, 'message': 'insufficient
     funds for gas * price + value'}
    Traceback (most recent call last):
      File "D:\Projects\NervHome\nvh\crypto\blockchain\blockchain_manager.py", line 244, in <module>
        comp.run()
      File "D:\Projects\NervProj\nvp\nvp_component.py", line 77, in run
        res = self.process_command(cmd)
      File "D:\Projects\NervHome\nvh\crypto\blockchain\blockchain_manager.py", line 94, in process_command
        chain.deploy_contract(contract)
      File "D:\Projects\NervHome\nvh\crypto\blockchain\evm_blockchain.py", line 1044, in deploy_contract
        logger.info("Contract %s deployed on %s at address: %s", sc_name, self.short_name, receipt.contractAddress)
    AttributeError: 'NoneType' object has no attribute 'contractAddress'
    2022/06/06 21:14:47 [nvp.communication.email_handler] INFO: Should send the email message <p style="color: #fd0202;">**WARNING:** an exception occu
    red in the following command:</p><p><em>['D:\\Projects\\NervProj\\.pyenvs\\bsc_env\\python.exe', 'D:\\Projects\\NervHome/nvh/crypto/blockchain/bloc
    kchain_manager.py', 'deploy', 'Foo', '-c', 'testnet_bsc']</em></p><p>cwd=None</p><p >=> Check the logs for details.</p>
    2022/06/06 21:14:48 [nvp.components.runner] ERROR: Error occured in script command: ['D:\\Projects\\NervProj\\.pyenvs\\bsc_env\\python.exe', 'D:\\P
    rojects\\NervHome/nvh/crypto/blockchain/blockchain_manager.py', 'deploy', 'Foo', '-c', 'testnet_bsc'] (cwd=None)
    
  • So let's give some virtual BNBs to our wallet first ;-)!
  • ⇒ We go on the faucet to get the BNBs: https://testnet.binance.org/faucet-smart
    • Hmmmm… okay: interesting: I'm only allowed to get 0.2 BNB every 24hours. Pufff.
  • Anyway, this is working this time:
    $ nvp bchain deploy Foo -c testnet_bsc
    2022/06/07 07:48:49 [nvh.crypto.blockchain.solidity_compiler] INFO: No change detected in test.sol
    2022/06/07 07:48:50 [nvh.crypto.blockchain.evm_blockchain] INFO: Estimated gas for contract deployment: 81293
    2022/06/07 07:48:50 [nvh.crypto.blockchain.evm_blockchain] INFO: Processing transaction: {'value': 0, 'from': '0xff4D71ceB45Ffd65360aA50c4740A55D49e51e16', 'chainId': 97, 'gas': 121939, 'gasPrice': 15000000000, 'nonce': 0, 'data': '0x6080604052348015600f57600080fd5b5060818061001e6000396000f3fe6080604052348015600f57600080fd5b506004361060325760003560e01c806364bdaf57146037578063febb0f7e14603f575b600080fd5b603d6047565b005b60456049565b005b565b56fea2646970667358221220750890c4ea9c357e18dcc938c60c82b3bfeaf7beed800344a92a34992ba74fa764736f6c63430006060033', 'to': b''}
    2022/06/07 07:48:51 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0x63987f4a1e177cf6f69fed7bf6440e2be52ca4ce2619c86e46525542ba0e442c
    2022/06/07 07:48:57 [nvh.crypto.blockchain.evm_blockchain] INFO: Contract Foo deployed on testnet_bsc at address: 0xdC905687fC99Bb4583AcE986efB0aB5D98fc0392
  • Actually, reconsidering this 0.2 BNB limit per 24h mentioned above, this seems to be “per address”, so I'm thinking I could create some additional test accounts, request the BNBs there, and then send them to my primary test account 😊 That's probably totally useless, but at the same time it will allow me to implement a transfer method the send funds between accounts, which is good to have.
  • So creating a new account works just fine:
    $ nvp bchain create wallet -n evm_test2
    2022/06/07 10:34:34 [nvh.crypto.blockchain.evm_blockchain] INFO: Saved new crypto account evm_test2 at 0x0398EBCD115e6DC4115bED2c76Ac162Be53c7b71
  • And from that I could get my 0.2 BNB for that new account.
Still not a terribly good system since the allowance is also per IP address apparently, so I have to use a VPN to make another request for that second account 😅. but anyway…
  • Now I should transfer most of those funds to my account evm_test, something like this:
    $ nvp bchain send -c testnet_bsc -a evm_test2 -v 0.18 -d evm_test
    2022/06/07 12:23:01 [nvh.crypto.blockchain.evm_blockchain] INFO: Trial 1: Gas estimate for transaction: 21000
    2022/06/07 12:23:01 [nvh.crypto.blockchain.evm_blockchain] INFO: Processing transaction: {'from': '0x0398EBCD115e6DC4115bED2c76Ac162Be53c7b71', 'chainId': 97, 'gas': 31500, 'gasPrice': 15000000000, 'nonce': 3, 'to': '0xff4D71ceB45Ffd65360aA50c4740A55D49e51e16', 'value': 180000000000000000}
    2022/06/07 12:23:02 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0x2efd092c5fc0ff795b3c6c4e99ba137ec6d2a95c24a66d2ac789770f92c74eeb
    2022/06/07 12:23:57 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent 0.180000 BNB to 'evm_test
  • The main function implemented to support the transfer of funds is as follow:
        def transfer(self, value, to_name, token=None):
            """Send some token value to a given address"""
            self.check(token is None, "No support to transfer token %s yet", token)
    
            # Check our balance:
            bal = self.get_native_balance()
            if bal <= value:
                logger.error("Balance to low to send funds: %f <= %f", bal, value)
                return False
    
            # Check if the to_addr is actually an account name:
            accounts = self.ctx.get_config()["crypto_accounts"]
            # logger.info("To name: %s", to_name)
            # logger.info("accounts: %s", accounts)
            if to_name in accounts:
                to_addr = accounts[to_name]["address"]
            else:
                to_addr = to_name
    
            # logger.info("To address: %s", to_addr)
    
            b0 = self.get_native_balance(account=to_addr, unit="wei")
    
            # Prepare the transfer operation:
            amount = self.web3.toWei(value, "ether")
            opr = TransferOperation(to_addr, amount)
    
            self.perform_operation(opr, max_gas=opr.estimateGas())
    
            b1 = self.get_native_balance(account=to_addr, unit="wei")
            received = b1 - b0
            self.check(received == amount, "Mismatch in received funds: %d != %d", received, amount)
            logger.info("Sent %f %s to '%s'", value, self.native_symbol, to_name)
  • oki, oki, so let's move forward now, and try to setup what we need to test the PairCheckerV5 contract.
  • First, of course, we need to deploy that contract:
    $ nvp bchain deploy -c testnet_bsc PairCheckerV5
      2022/06/07 12:39:23 [nvh.crypto.blockchain.solidity_compiler] INFO: No change detected in PairCheckerV5.sol
    2022/06/07 12:39:25 [nvh.crypto.blockchain.evm_blockchain] INFO: Estimated gas for contract deployment: 766978
    2022/06/07 12:39:25 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0xdb36d872ba0ec911217b7db1b5124b4383f1c655c40a1f89433d606e2d772879
    2022/06/07 12:39:47 [nvh.crypto.blockchain.evm_blockchain] INFO: Contract PairCheckerV5 deployed on testnet_bsc at address: 0x9a835728502d84F3f3bD526B8C91ce6f4d2b3E5A
  • Now to be able to test that contract, we need… a pair on 2 tokens obviously! 🤣
  • So let's first try we something we know should really work, like the pair for WBNB/BUSD:
  • In fact, I should add a dedicated command line to request this kind of pair testing directy (on mainnet):
            if cmd == "check-pair":
                pname = self.get_param("pair_name")
                tokens = pname.split("/")
                self.check(len(tokens) == 2, "Invalid token list: %s", pname)
    
                chain_name = self.get_param("chain")
                chain: EVMBlockchain = self.get_component(f"{chain_name}_chain")
    
                account = self.get_param("account")
                chain.set_account(account)
                
                # get the token objects:
                t0 = chain.get_token(tokens[0])
                t1 = chain.get_token(tokens[1])
    
                dex = chain.get_default_exchange()
    
                # get the pair row
                pair = chain.get_db().get_pair_row(tokens=(t0.address(), t1.address()), ex_id=dex.get_id())
    
                # We need to create a new copy of the Arbitrage manager component:
                arbman = self.create_component("arb_manager", args=[chain_name])
    
                # perform the pair check:
                # We assume that t0 will always be given as the quote token:
                arbman.check_pair(pair, t0)
    
                return True
  • Note: In the process I also added support to create_component on a given context without registering them automatically… Here this is used to create the arb_manager component directly with a chain setup. But yeah, I could also probably just get the default component and call setup_chain on it too.
  • Anyway, now trying to use that feature we get the expected result:
    $ nvp bchain check-pair WBNB/BUSD
    2022/06/09 11:19:10 [nvh.crypto.blockchain.arbitrage_manager] INFO: Creating ArbitrageManager on chain bsc
    2022/06/09 11:19:10 [nvh.crypto.blockchain.arbitrage_manager] INFO: PairChecker result for BUSD (against WBNB) is: PASSED
  • Next, let's try a pair where this is failing:
    $ nvp bchain check-pair WBNB/GINUX
    2022/06/09 11:23:55 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=0.00%): execution reverted: transferToken failed.
    2022/06/09 11:23:55 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=10.00%): execution reverted: transferToken failed.
    2022/06/09 11:23:55 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=20.00%): execution reverted: transferToken failed.
    2022/06/09 11:23:55 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=30.00%): execution reverted: transferToken failed.
    2022/06/09 11:23:56 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=40.00%): execution reverted: transferToken failed.
    2022/06/09 11:23:56 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=50.00%): execution reverted: transferToken failed.
    2022/06/09 11:23:56 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=60.00%): execution reverted: transferToken failed.
    2022/06/09 11:23:56 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=70.00%): execution reverted: transferToken failed.
    2022/06/09 11:23:57 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=80.00%): execution reverted: transferToken failed.
    2022/06/09 11:23:57 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=90.00%): execution reverted: transferToken failed.
    2022/06/09 11:23:57 [nvh.crypto.blockchain.arbitrage_manager] WARNING: Token GINUX requires an extra swap fee of 100.00%
    2022/06/09 11:23:57 [nvh.crypto.blockchain.arbitrage_manager] INFO: PairChecker result for GINUX (against WBNB) is: FAILED
    2022/06/09 11:23:57 [nvh.crypto.blockchain.arbitrage_manager] INFO: Pair address: 0x85B446d3EDC3A7fe4db8A88649c14fdcB4e911dE, exchange: PancakeSwap2, router: 0x10ED43C718714eb63d5aA57B78B54704E256024E, ifp: 975
  • Now let's try to replicate this on the testnet:
  • First we should deploy our own copy of the WBNB token:
    $ nvp bchain find-token WBNB
    2022/06/09 12:53:16 [__main__] INFO: Found token: { 'address': '0xbb4CdB9CBd36B01bD1cBaEBF2De08d9173bc095c',
      'classification': 0,
      'decimals': 18,
      'id': 3,
      'min_slippage': None,
      'name': 'Wrapped BNB',
      'swap_fees': None,
      'symbol': 'WBNB'}
  • Sending the funds of the day:
    $ nvp bchain send -c testnet_bsc -a evm_test2 -v 0.2 -d evm_test
    2022/06/09 13:08:03 [nvh.crypto.blockchain.evm_blockchain] INFO: Trial 1: Gas estimate for transaction: 21000
    2022/06/09 13:08:03 [nvh.crypto.blockchain.evm_blockchain] INFO: Processing transaction: {'from': '0x0398EBCD115e6DC4115bED2c76Ac162Be53c7b71', 'chainId': 97, 'gas': 31500, 'gasPrice': 15000000000, 'nonce': 4, 'to': '0xff4D71ceB45Ffd65360aA50c4740A55D49e51e16', 'value': 200000000000000000}
    2022/06/09 13:08:04 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0x44b120421afab204261f68c2747f5339f6a525f3931883a5accc14a1e1598f69 2022/06/09 13:08:17 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent 0.200000 BNB to 'evm_test'
  • So I grabbed the contract for WBNB at the address 0xbb4CdB9CBd36B01bD1cBaEBF2De08d9173bc095c on mainnet, and added the following function to it:
        function setBalance(address tgt, uint256 bal) public {
            balanceOf[tgt] = bal;
        }
  • With that, I should be able to set the balance of WBNB just as I wish ? ⇒ Update: not sure this is really necessary, and below I ended up commenting that function anyway for the moment.
I just found this page https://github.com/kewka/give-me-bnb so I'm now testing the docker image for it.
  • According to the website we should be able to just use this command:
    docker run --rm kewka/give-me-bnb give-me-bnb -proxy socks5://127.0.0.1:9050 -to 0xff4D71ceB45Ffd65360aA50c4740A55D49e51e16
  • Arrf… unfortunately this doesn't quite work:
    kenshin@neptune:~$ docker run --rm kewka/give-me-bnb give-me-bnb -proxy socks5://127.0.0.1:9050 -to 0xff4D71ceB45Ffd65360aA50c4740A55D49e51e16
    NewSocketTransaction: faucet error: Beep-bop, you're a robot!
    kenshin@neptune:~$ docker run --rm kewka/give-me-bnb give-me-bnb -to 0xff4D71ceB45Ffd65360aA50c4740A55D49e51e16
    NewSocketTransaction: faucet error: Beep-bop, you're a robot!
  • So now let's get back to compiling/deploying the WBNB token:
    nvp solc compile
  • Then I deploy the contract on the testnet:
    nvp bchain deploy -c testnet_bsc WBNB
  • And this is taking forever… 😒 my my my… In fact the transaction even failed eventually:
    2022/06/10 21:45:37 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0x957f7b402433a7c30ceecf8bdf60e4d0cbeb7740d34222227469be76bb80e80e 2022/06/10 21:46:02 [nvh.crypto.blockchain.evm_blockchain] ERROR: HTTPError occured while waiting for receipt of transaction 0x957f7b402433a7c30ceecf8bdf60e4d0cbeb7740d34222227469be76bb80e80e
    2022/06/10 21:47:03 [nvh.crypto.blockchain.evm_blockchain] ERROR: HTTPError occured while waiting for receipt of transaction 0x957f7b402433a7c30ceecf8bdf60e4d0cbeb7740d34222227469be76bb80e80e
    2022/06/10 21:48:17 [nvh.crypto.blockchain.evm_blockchain] ERROR: HTTPError occured while waiting for receipt of transaction 0x957f7b402433a7c30ceecf8bdf60e4d0cbeb7740d34222227469be76bb80e80e
    2022/06/10 21:48:17 [nvh.crypto.blockchain.evm_blockchain] INFO: Giving up on transaction 0x957f7b402433a7c30ceecf8bdf60e4d0cbeb7740d34222227469be76bb80e80e due to timeout.
    Traceback (most recent call last):
      File "D:\Projects\NervHome\nvh\crypto\blockchain\blockchain_manager.py", line 388, in <module>
        comp.run()
      File "D:\Projects\NervProj\nvp\nvp_component.py", line 81, in run
        res = self.process_command(cmd)
      File "D:\Projects\NervProj\nvp\nvp_component.py", line 71, in process_command
        return self.process_cmd_path(self.ctx.get_command_path())
      File "D:\Projects\NervHome\nvh\crypto\blockchain\blockchain_manager.py", line 94, in process_cmd_path
        chain.deploy_contract(contract)
      File "D:\Projects\NervHome\nvh\crypto\blockchain\evm_blockchain.py", line 1068, in deploy_contract
        logger.info("Contract %s deployed on %s at address: %s", sc_name, self.short_name, receipt.contractAddress)
    AttributeError: 'str' object has no attribute 'contractAddress'
    2022/06/10 21:48:17 [nvp.components.runner] ERROR: Error occured in script command: ['D:\\Projects\\NervProj\\.pyenvs\\bsc_env\\python.exe', 'D:\\Projects\\NervHome/nvh/crypto/blockchain/blockchain_manager.py', 'deploy', '-c', 'testnet_bsc', 'WBNB'] (cwd=None)
  • And when I try again, I get another obscure 'already known' issue:
    2022/06/10 21:50:34 [nvh.crypto.blockchain.evm_blockchain] INFO: Error while trying to perform operation: {'code': -32000, 'message': 'already known'}
    Traceback (most recent call last):
      File "D:\Projects\NervHome\nvh\crypto\blockchain\blockchain_manager.py", line 388, in <module>
        comp.run()
      File "D:\Projects\NervProj\nvp\nvp_component.py", line 81, in run
        res = self.process_command(cmd)
      File "D:\Projects\NervProj\nvp\nvp_component.py", line 71, in process_command
        return self.process_cmd_path(self.ctx.get_command_path())
      File "D:\Projects\NervHome\nvh\crypto\blockchain\blockchain_manager.py", line 94, in process_cmd_path
        chain.deploy_contract(contract)
      File "D:\Projects\NervHome\nvh\crypto\blockchain\evm_blockchain.py", line 1068, in deploy_contract
        logger.info("Contract %s deployed on %s at address: %s", sc_name, self.short_name, receipt.contractAddress)
    AttributeError: 'NoneType' object has no attribute 'contractAddress'
    2022/06/10 21:50:34 [nvp.components.runner] ERROR: Error occured in script command: ['D:\\Projects\\NervProj\\.pyenvs\\bsc_env\\python.exe', 'D:\\Projects\\NervHome/nvh/crypto/blockchain/blockchain_manager.py', 'deploy', '-c', 'testnet_bsc', 'WBNB'] (cwd=None)
  • Maybe I should try with a different RPC endpoint: ⇒ same error, “already known”, what does it mean 🤔 ?
  • So now changing the code a little to try to avoid that error. hmmm… transaction still not working… This is getting me thinking: could it be that the version of the compiler I'm using is too old and somehow not accepted anymore ? Let's try to upgrade it.
  • ⇒ I started to refactor the WBNB.sol code to make it compilable with version 0.6.6, but still, cannot get the transaction to got on testnet 😠
  • This is weird because I could successfully deploy PairCheckerV5 compiled with version 0.6.6 before! So let's try to deploy that one again:
    nvp bchain deploy -c testnet_bsc PairCheckerV5
  • ⇒ Nope, that one is not working anymore either, even with a gas price of 20 gwei.
  • Okay, so you know what ? the BSC tesnet RPC endpoints seem to be largely out of sync/unusable for the moment. And I lost enough time on this: so now let's move to ganache-cli instead! This means more work for me, but at least then I will have the control on my test blockchain!
  • ⇒ I will thus start with ganache-cli setup in a new article: this one is, as usual, far too big already 😄!
After changing the provider url to “https://data-seed-prebsc-1-s3.binance.org:8545/” the next day, I could finally get the testnet to handle my transactions again.
  • Update: After a successful transaction I finally got the WBNB contract created on testnet:
    $ nvp bchain deploy -c testnet_bsc WBNB
    2022/06/11 08:05:37 [nvh.crypto.blockchain.solidity_compiler] INFO: No change detected in WBNB.sol
    2022/06/11 08:05:37 [nvh.crypto.blockchain.evm_blockchain] INFO: Estimated gas for contract deployment: 827168
    2022/06/11 08:05:38 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0x1eefdbd8a74ceb1edb87c3141c86d108f881ddf89b736335f743caa56cc282d1 
    2022/06/11 08:05:42 [nvh.crypto.blockchain.evm_blockchain] INFO: Contract WBNB deployed on testnet_bsc at address: 0x11C5BF3130cD9Af91607ecbfab9906812b851b3e
  • blog/2022/0611_crypto_pair_check_system.txt
  • Last modified: 2022/06/11 07:16
  • by 127.0.0.1