====== Crypto: investigations around token pair checking system ====== {{tag>dev python crypto nervhome finance blockchain}} Continuing with our Arbitrage management system in this post, I wanted to focus on restoring my PairChecker contract, and this will mostly lead me to the re-implementation of a layer in python to manager the solidity compiler and then some initial experiment on the BSC testnet which is in the end giving me a pretty poor impression, so I won't spend long on that, and at the end of this post, I will eventually decide to move to ganache instead. ====== ====== ===== Restoring the PairChecker smart contract ===== * At the end of our previous article on the topic we could get artibrage setups above the min profit values like the following: 2022/06/02 11:52:56 [__main__] INFO: Quote token SAFEMOON value: 3.54682e-10 WBNB 2022/06/02 11:52:56 [__main__] INFO: Quote token DAI value: 0.00327899 WBNB 2022/06/02 11:52:56 [__main__] INFO: Quote token BANANA value: 0.000508487 WBNB 2022/06/02 11:55:38 [__main__] INFO: Block 18336166: Found arb setup of 0.002436 WBNB with pairs on WBNB/QUACK#3 2022/06/02 11:55:48 [__main__] INFO: Block 18336169: Found arb setup of 0.03963 WBNB with pairs on ASS#7/WBNB 2022/06/02 11:56:20 [__main__] INFO: Block 18336180: Found arb setup of 0.001731 WBNB with pairs on WBNB/GINUX * Yet, clearly, we are not going to find arb setup with a value of 0.039 WBNB lol... Typically this will only happen with tokens where we have a large burn/transaction fee, which will in the end make an arbitrage on those pair actually not profitable. * => That's why I created the **PairChecker** contract to verify which tokens are "really valid" tokens that we may use during an arbitrage. * So I think it's now time to restore that contract too to filter our arbitrage setups even further. * And here is the method I just added to check a given pair: def check_pair(self, pair, qtoken, amount=100000): """Check a pair given a pair row.""" paddr = pair[1] t0addr = pair[2] t1addr = pair[3] qaddr = qtoken.address() self.check(t0addr == qaddr or t1addr == qaddr, "Invalid quote token %s for pair %s/%s", qaddr, t0addr, t1addr) # Get the current balance of qtoken in the pair checker: b0 = qtoken.get_balance(as_value=False, account=self.pair_checker_address) t0 = self.chain.get_token(t0addr) t1 = self.chain.get_token(t1addr) self.check( b0 >= amount, "Initial balance of %s too low, cannot perform pair check for %s/%s.", qtoken.symbol(), t0.symbol(), t1.symbol(), ) # get the exchange: dex0 = self.chain.get_exchange(pair[8]) ifp = int(10000 - dex0.get_swap_fee_points(paddr)) amount = int(amount) stype = 1 if dex0.is_flash_loan_supported() else 0 target = t0 if qaddr == t1addr else t1 opr = self.pair_checker_sc.build_function_call("checkPair", paddr, qaddr, target.address(), amount, ifp, stype) try: self.chain.get_gas_estimate(opr) clresult = TokenClassification.OK except ValueError as err: logger.error("Pair checker test failed for %s: %s", target.symbol(), str(err)) clresult = TokenClassification.INVALID if clresult == TokenClassification.OK: # The pair was successfully checked. logger.info("PairChecker result for %s (against %s) is: PASSED", target.symbol(), qtoken.symbol()) else: logger.info("PairChecker result for %s (against %s) is: FAILED", target.symbol(), qtoken.symbol()) # target.set_classification(clresult) return clresult == TokenClassification.OK * Now using that method when we find a potential arb setup: if best_profit < self.min_profit: # Profit too low, so we ignore this arb setup. return # Get the tokens: t0 = self.chain.get_token(t0addr) t1 = self.chain.get_token(t1addr) # We should ensure that the tokens are "valid for arbitrage", so we check their classification: if (t0.is_class_unknown() or t1.is_class_unknown()) and not self.check_pair(best_p0, qtoken): # The target pair is not valid. return if t0.is_class_invalid() or t1.is_class_invalid(): # Invalid target token. return logger.info( "Block %d: Found arb setup of %.4g %s with pairs on %s/%s", self.last_block_number, best_profit, self.native_symbol, t0.symbol(), t1.symbol(), ) * Except that... this pair checker contract can only be used by its owner πŸ˜‚... so I need to change the account I use by default in the ArbitrageManager. * And this means I need to provide again another private_key into the system hmmm... πŸ€”: on linux/cygwin systems this is not really a problem because we can store the environment variables in a simple file but with batch files, this is less straightforward... ===== Adding support for data encryption/decryption ===== * I found [[https://stackoverflow.com/questions/232747/read-environment-variables-from-file-in-windows-batch-cmd-exe|this page]] with a potential solution: setlocal FOR /F "tokens=*" %%i in ('type Settings.txt') do SET %%i java -Dsetting1=%setting1% ... endlocal * But I still think I need to push this a bit further: I should use a pair of RSA keys to encrypt the data and store that data directly in my config file as usual: then I would only need to store 1 private key as environment variable: should be a good option I think! * Let's see how I can do that... * Found this page about encryption/decryption in python: https://www.geeksforgeeks.org/how-to-encrypt-and-decrypt-strings-in-python/ * => I will build an dedicated component for that: the **Encrypter** class: """Encrypter utility functions""" import logging import rsa from nvp.nvp_component import NVPComponent from nvp.nvp_context import NVPContext import nvp.core.utils as utl logger = logging.getLogger(__name__) def create_component(ctx: NVPContext): """Create an instance of the component""" return Encrypter(ctx) class Encrypter(NVPComponent): """Encrypter component used to send automatic messages ono Encrypter server""" def __init__(self, ctx: NVPContext): """Script runner constructor""" NVPComponent.__init__(self, ctx) # Get the config for this component: self.config = ctx.get_config()["encrypter"] def generate_keys(self, size): """Generate a pair of RSA keys""" logger.info("Generating keys of size %d...", size) pub_key, priv_key = rsa.newkeys(size) # pub_hex = utl.bytes_to_hex(pub_key.save_pkcs1()) # priv_hex = utl.bytes_to_hex(priv_key.save_pkcs1()) pub_str = pub_key.save_pkcs1() priv_str = priv_key.save_pkcs1() pub_b64 = utl.bytes_to_b64(pub_str) priv_b64 = utl.bytes_to_b64(priv_str) pub_2 = utl.b64_to_bytes(pub_b64) priv_2 = utl.b64_to_bytes(priv_b64) self.check(pub_2 == pub_str, "Mismatch in pub key: %s != %s", pub_2, pub_str) self.check(priv_2 == priv_str, "Mismatch in priv key: %s != %s", priv_2, priv_str) # logger.info("Public key:\n%s", pub_key.save_pkcs1().decode('utf-8')) logger.info("Public key:\n%s", pub_b64) logger.info("Private key:\n%s", priv_b64) logger.info("Done.") def process_command(self, cmd): """Check if this component can process the given command""" if cmd == 'gen-keys': size = self.get_param("key_size") # logger.info("Should generate key of size %d", size) self.generate_keys(size) return True return False if __name__ == "__main__": # Create the context: context = NVPContext() # Add our component: comp = context.register_component("encrypter", Encrypter(context)) context.define_subparsers("main", { 'gen-keys': None, }) psr = context.get_parser('main.gen-keys') psr.add_argument("-s", "--size", dest="key_size", type=int, default=2048, help="Specify the size of the rsa keys.") comp.run() * Generating RSA keys on windows with cli.bat gen-rsa-keys * Now on the storage of that new private key... We cannot store it in an environment variable in windows because it is too long. * But maybe we could store it in the registry ? => No, not a good idea since we don't have clear control on the acces rights. * => Instead we should put that in a file in the user home folder. * On cygwin we need to resolve the home to the real windows home. * **Update**: To generate the RSA keys I will now rather use: cli.bat encrypter gen-keys * And that command above will also write the private key to a file directly and set the correct access rights on that file. * Now, time to encrypt/decrypt some content: Let's add a tab on the nervGate app to handle that. * => **OK** it works just fine now and I can encrypt/decrypt data directly from the nervgate gui: {{ blog:2022:0603:encrypt_decrypt.png }} * So I encrypted my private keys, and I can now store them in my config file with very high confidence 😎! * **Note**: And the nice thing is that now my CryptoView shortcut is working against thanks to that change ;-)! ===== More investigations on PairChecker ===== * So now I get my PairChecker running... and I seem to get some failure from it: 2022/06/03 13:45:23 [__main__] ERROR: Pair checker test failed for BONFIRE#2: execution reverted: transferToken failed. 2022/06/03 13:45:23 [__main__] INFO: PairChecker result for BONFIRE#2 (against WBNB) is: FAILED 2022/06/03 13:46:30 [__main__] ERROR: Pair checker test failed for GINUX: execution reverted: transferToken failed. 2022/06/03 13:46:30 [__main__] INFO: PairChecker result for GINUX (against WBNB) is: FAILED 2022/06/03 13:47:17 [__main__] ERROR: Pair checker test failed for GINUX: execution reverted: transferToken failed. 2022/06/03 13:47:17 [__main__] INFO: PairChecker result for GINUX (against WBNB) is: FAILED 2022/06/03 13:47:56 [__main__] ERROR: Pair checker test failed for QUACK#3: execution reverted: transferToken failed. 2022/06/03 13:47:56 [__main__] INFO: PairChecker result for QUACK#3 (against WBNB) is: FAILED 2022/06/03 13:48:14 [__main__] ERROR: Pair checker test failed for QUACK#3: execution reverted: transferToken failed. 2022/06/03 13:48:14 [__main__] INFO: PairChecker result for QUACK#3 (against WBNB) is: FAILED 2022/06/03 13:48:19 [__main__] ERROR: Pair checker test failed for QUACK#3: execution reverted: transferToken failed. 2022/06/03 13:48:19 [__main__] INFO: PairChecker result for QUACK#3 (against WBNB) is: FAILED * => Question is, to what extent are those failure legit ? And could I do anything about these to increase my number of valid arb setups ? * So let's get back to some solidity code now, yeeeppeee πŸ₯³! * First, here are a few examples with more inputs that might be relevant to us: 2022/06/03 13:54:16 [__main__] ERROR: Pair checker test failed for PIG: execution reverted: transferToken failed. 2022/06/03 13:54:16 [__main__] INFO: PairChecker result for PIG (against WBNB) is: FAILED 2022/06/03 13:54:16 [__main__] INFO: Pair address: 0x51dCAF423FE39F620A13379Cd26821cF8d433308, exchange: PancakeSwap, router: 0x05fF2B0DB69458A0750badebc 4f9e13aDd608C7F 2022/06/03 13:54:20 [__main__] ERROR: Pair checker test failed for QUACK#3: execution reverted: transferToken failed. 2022/06/03 13:54:20 [__main__] INFO: PairChecker result for QUACK#3 (against WBNB) is: FAILED 2022/06/03 13:54:20 [__main__] INFO: Pair address: 0xa19C85572f0Bf8D32145244d1886E59025aD0d60, exchange: ApeSwap, router: 0xcF0feBd3f17CEf5b47b0cD257aCf6 025c5BFf3b7 2022/06/03 13:56:22 [__main__] ERROR: Pair checker test failed for NFTART: execution reverted: transferToken failed. 2022/06/03 13:56:22 [__main__] INFO: PairChecker result for NFTART (against WBNB) is: FAILED 2022/06/03 13:56:22 [__main__] INFO: Pair address: 0x03A4eeA71075fE14BA09B27931c1D767D0a082BD, exchange: PancakeSwap, router: 0x05fF2B0DB69458A0750badebc 4f9e13aDd608C7F 2022/06/03 14:03:53 [__main__] ERROR: Pair checker test failed for MOONLIGHT#17: execution reverted: transferToken failed. 2022/06/03 14:03:53 [__main__] INFO: PairChecker result for MOONLIGHT#17 (against WBNB) is: FAILED 2022/06/03 14:03:53 [__main__] INFO: Pair address: 0xe6de19Ae48969aF0a6f78271e41D3CE47580eaFB, exchange: ApeSwap, router: 0xcF0feBd3f17CEf5b47b0cD257aCf6 025c5BFf3b7 2022/06/03 14:04:17 [__main__] ERROR: Pair checker test failed for BONFIRE#2: execution reverted: transferToken failed. 2022/06/03 14:04:17 [__main__] INFO: PairChecker result for BONFIRE#2 (against WBNB) is: FAILED 2022/06/03 14:04:17 [__main__] INFO: Pair address: 0xD3F478F0d5E98b01f757bc6cB54Db4C00b9838f2, exchange: PancakeSwap, router: 0x05fF2B0DB69458A0750badebc 4f9e13aDd608C7F * Okay, so in my contract I have this **transsferToken** function: function transferToken(address token, address to, uint amount) private { (bool success, bytes memory data) = token.call(abi.encodeWithSelector(TRANSFER_SELECTOR, to, amount)); require(success && (data.length == 0 || abi.decode(data, (bool))), 'transferToken failed.'); } * => That must be where the error I get comes from. * And then... ohh my... I wrote the full checkPair() function in assembly 🀣... So now I need to figure out what I'm doing exactly. * So the steps we take are: * Mask the srcToken, amount and ifp values: srcToken := and(srcToken, 0xffffffffffffffffffffffffffffffffffffffff) amount := and(amount, 0xffffffffffffffffffffffffffff) ifp := and(ifp, 0xffff) * Get the free memory pointer location with ''let b := mload(0x40)'' * Add an offset to the free memory pointer of 0xc0 = 192 bits / 6 regs (32bytes each): mstore(0x40, add(b, 0xc0)) * Store signature for **getReserves()** at b: mstore(b,shl(224,0x0902f1ac) * do a staticcall on the pair address with input data starting from b and length 4 bytes (ie. the signature only) and destination b with size of 2 registers: pop(staticcall(2300, pair, b, 0x04, b, 0x40)) * => then b contains r0, and b+0x20 contains r1 * We then declare a pointer starting just after r1: ''let ptr := add(b,0x40)'' * We put the signature for 'token0()' into ptr; mstore(ptr, shl(224, 0x0dfe1681)) * And then do a staticcall to get the token0 stored at ptr: pop(staticcall(2300, pair, ptr, 0x04, ptr, 0x20)) * Then we check if token0 address is the same a srcToken: let flipt := 1 if eq(mload(ptr), srcToken) { flipt := 0 } * If not, then we need to flip the tokens in b / b+0x20: if eq(flipt,1) { // In this case we need to flipt the r0 and r1 value: mstore(ptr, mload(b)) mstore(b, mload(add(b,0x20))) mstore(add(b,0x20), mload(ptr)) } * Next we get ready to call ''transferToken(address,address,uint256)'' => sig: 'f5537ede' (?) * We put the sig 'a9059cbb' (for "transfer(address,uint256)") into ptr: mstore(ptr, shl(224, 0xa9059cbb)) * Then we put the address of the pair, and then the amount we want to transfer (of srcToken): // Then we push the destination address: (taking 32 bytes) (cf. https://ethereum.stackexchange.com/questions/67980/solidity-assembly-function-call-parameter-alignment) mstore(add(ptr, 0x04), pair) // Then we push the amount: taking 32 bytes mstore(add(ptr, 0x24), amount) * Then we call that transfer function on the srcToken address, returning nothing as result: pop(call(gas(), srcToken, 0, ptr, 0x44, ptr, 0x0)) * OK, so we have "amount srcToken" inside the pair contract at that point for our address. * Then we compute the amount_out we should get in theory given the reserves r0, r1 and the ifp value * Then we get read for the call to the swap function: * Signature will be either for ''swap(uint256,uint256,address)'' if we don't support flashloan on that dex or ''swap(uint256,uint256,address,bytes)'' if there is flashloan support * Then we call the swap function on the pair contract: let c := 0 if eq(stype,0) { c:= call(gas(), pair, 0, ptr, 0x64, ptr, 0x0) } if eq(stype,1) { // To store an array of bytes, in the first 32 bytes we store the offset to the start of the data, // from the current data definition line *not* taking the function signature into account, // ie. so from the address just after the function signature: // cf. https://docs.soliditylang.org/en/develop/abi-spec.html#use-of-dynamic-types // we pushed 4 x 32 bytes arguments in this data line, so the offset is 0x80 mstore(add(ptr, 0x64), 0x80) // offset to start of data in array mstore(add(ptr, 0x84), 0) // Size of the array. c:= call(gas(), pair, 0, ptr, 0xa4, ptr, 0x0) } * If that call fails, we return a special return code: if eq(c,0) { let size := returndatasize() returndatacopy(b, 0, size) // add a code for the location: mstore(add(b, size), shl(255, 65)) // 65 => code for 'A' revert(b, add(size,1)) } * Then we apply the changes on our reserve values: mstore(b, add(mload(b), amount)) mstore(add(b,0x20), sub(mload(add(b,0x20)), amountOut)) * And we compute our amountOut for a swap back: amount := amountOut // amountOut := getAmountOut(amount, r1, r0, mask(ifp, 16)) amountOut := getAmountOut(amount, mload(add(b,0x20)), mload(b), ifp) * Then we perform the swap in the opposite direction. * **Note**: to get the signature of a function in keccak-256 we can use the online site: https://emn178.github.io/online-tools/keccak_256.html * **Note2**: While doing my investigations here I got a couple of successful pair checks: 2022/06/03 14:26:54 [__main__] INFO: PairChecker result for FTM#8 (against WBNB) is: PASSED 2022/06/03 14:26:54 [__main__] INFO: Block 18367757: Found arb setup of 0.002772 WBNB with pairs on FTM#8/WBNB 2022/06/03 14:45:16 [__main__] INFO: PairChecker result for AUTO#7 (against WBNB) is: PASSED 2022/06/03 14:45:16 [__main__] INFO: Block 18368117: Found arb setup of 0.001228 WBNB with pairs on AUTO#7/WBNB * Okay, okay... so first of all... I don't think that's the contract I'm really using myself right now lol because I'm never calling the **transferToken()** function in this process anyway. * So do I have that contract available already ? let's check that. * Contract creation dates: * 386 days 21 hrs ago for 0x2F077C3fE54be5D5dDad3AaBdC1A063fEf0101EE * 387 days 2 hrs ago for 0x79Cbc72A24F760b4f3fBeaD112b5282941D99038 * 388 days 6 hrs ago for 0x0A5EE1C7Bb11978585Cd0202cE06CB4E33b68d37 * 432 days 2 hrs ago for 0x3db7e66639E5C94A725F6943FFBA6863c7DfC233 * OK, so we have the latest version deployed, but that doesn't seem to be the latest version of the code. * Anyway, let's just check one of those failing pairs now: say WBNB/PIT at 0xB450CBF17F6723Ef9c1bf3C3f0e0aBA368D09bF5 * Ohh.... Just checking at the bscscan page for this: {{ blog:2022:0603:wbnb_pit.png }} * => This really looks like some kind of **scam**! * Checking the contract code at: https://bscscan.com/address/0xa57ac35ce91ee92caefaa8dc04140c8e232c2e50#code * **OK** so that token is really an awful mess with fees and burns etc. Not something we want to use. * => But this is actually giving me an idea: when doing a pair check, I could progressively virtually increase the swap fee value to figure out what is the fee required to swap a given token πŸ‘ Let's try that... * So I updated the code as follow to try to increate the swap fee: extra_fee = 0 while extra_fee < 10000: try: ifp = min(int(10000 - sfp - extra_fee), 10000) opr = self.pair_checker_sc.build_function_call( "checkPair", paddr, qaddr, target.address(), amount, ifp, stype ) self.chain.get_gas_estimate(opr) break except ValueError as err: logger.error( "Pair checker test failed for %s (extra_fee=%.2f%%): %s ", target.symbol(), extra_fee / 100.0, str(err), ) extra_fee += 100 if extra_fee > 0: logger.warning("Token %s requires an extra swap fee of %.2f%%", target.symbol(), extra_fee / 100.0) clresult = TokenClassification.OK if extra_fee == 0 else TokenClassification.INVALID * Yet, even with that I'm still getting the same error until the end when trying to check some pairs: 2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=0.00%): execution reverted: transferToken failed. 2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=1.00%): execution reverted: transferToken failed. 2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=2.00%): execution reverted: transferToken failed. 2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=3.00%): execution reverted: transferToken failed. 2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=4.00%): execution reverted: transferToken failed. 2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=5.00%): execution reverted: transferToken failed. 2022/06/03 15:54:53 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=6.00%): execution reverted: transferToken failed. 2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=7.00%): execution reverted: transferToken failed. 2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=8.00%): execution reverted: transferToken failed. 2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=9.00%): execution reverted: transferToken failed. 2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=10.00%): execution reverted: transferToken failed. 2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=11.00%): execution reverted: transferToken failed. 2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=12.00%): execution reverted: transferToken failed. 2022/06/03 15:54:54 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=13.00%): execution reverted: transferToken failed. 2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=14.00%): execution reverted: transferToken failed. 2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=15.00%): execution reverted: transferToken failed. 2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=16.00%): execution reverted: transferToken failed. 2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=17.00%): execution reverted: transferToken failed. 2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=18.00%): execution reverted: transferToken failed. 2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=19.00%): execution reverted: transferToken failed. 2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=20.00%): execution reverted: transferToken failed. 2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=21.00%): execution reverted: transferToken failed. 2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=22.00%): execution reverted: transferToken failed. 2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=23.00%): execution reverted: transferToken failed. 2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=24.00%): execution reverted: transferToken failed. 2022/06/03 15:54:55 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=25.00%): execution reverted: transferToken failed. 2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=26.00%): execution reverted: transferToken failed. 2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=27.00%): execution reverted: transferToken failed. 2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=28.00%): execution reverted: transferToken failed. 2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=29.00%): execution reverted: transferToken failed. 2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=30.00%): execution reverted: transferToken failed. 2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=31.00%): execution reverted: transferToken failed. 2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=32.00%): execution reverted: transferToken failed. 2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=33.00%): execution reverted: transferToken failed. 2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=34.00%): execution reverted: transferToken failed. 2022/06/03 15:54:56 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=35.00%): execution reverted: transferToken failed. 2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=36.00%): execution reverted: transferToken failed. 2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=37.00%): execution reverted: transferToken failed. 2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=38.00%): execution reverted: transferToken failed. 2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=39.00%): execution reverted: transferToken failed. 2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=40.00%): execution reverted: transferToken failed. 2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=41.00%): execution reverted: transferToken failed. 2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=42.00%): execution reverted: transferToken failed. 2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=43.00%): execution reverted: transferToken failed. 2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=44.00%): execution reverted: transferToken failed. 2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=45.00%): execution reverted: transferToken failed. 2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=46.00%): execution reverted: transferToken failed. 2022/06/03 15:54:57 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=47.00%): execution reverted: transferToken failed. 2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=48.00%): execution reverted: transferToken failed. 2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=49.00%): execution reverted: transferToken failed. 2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=50.00%): execution reverted: transferToken failed. 2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=51.00%): execution reverted: transferToken failed. 2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=52.00%): execution reverted: transferToken failed. 2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=53.00%): execution reverted: transferToken failed. 2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=54.00%): execution reverted: transferToken failed. 2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=55.00%): execution reverted: transferToken failed. 2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=56.00%): execution reverted: transferToken failed. 2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=57.00%): execution reverted: transferToken failed. 2022/06/03 15:54:58 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=58.00%): execution reverted: transferToken failed. 2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=59.00%): execution reverted: transferToken failed. 2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=60.00%): execution reverted: transferToken failed. 2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=61.00%): execution reverted: transferToken failed. 2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=62.00%): execution reverted: transferToken failed. 2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=63.00%): execution reverted: transferToken failed. 2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=64.00%): execution reverted: transferToken failed. 2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=65.00%): execution reverted: transferToken failed. 2022/06/03 15:54:59 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=66.00%): execution reverted: transferToken failed. 2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=67.00%): execution reverted: transferToken failed. 2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=68.00%): execution reverted: transferToken failed. 2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=69.00%): execution reverted: transferToken failed. 2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=70.00%): execution reverted: transferToken failed. 2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=71.00%): execution reverted: transferToken failed. 2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=72.00%): execution reverted: transferToken failed. 2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=73.00%): execution reverted: transferToken failed. 2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=74.00%): execution reverted: transferToken failed. 2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=75.00%): execution reverted: transferToken failed. 2022/06/03 15:55:00 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=76.00%): execution reverted: transferToken failed. 2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=77.00%): execution reverted: transferToken failed. 2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=78.00%): execution reverted: transferToken failed. 2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=79.00%): execution reverted: transferToken failed. 2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=80.00%): execution reverted: transferToken failed. 2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=81.00%): execution reverted: transferToken failed. 2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=82.00%): execution reverted: transferToken failed. 2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=83.00%): execution reverted: transferToken failed. 2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=84.00%): execution reverted: transferToken failed. 2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=85.00%): execution reverted: transferToken failed. 2022/06/03 15:55:01 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=86.00%): execution reverted: transferToken failed. 2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=87.00%): execution reverted: transferToken failed. 2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=88.00%): execution reverted: transferToken failed. 2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=89.00%): execution reverted: transferToken failed. 2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=90.00%): execution reverted: transferToken failed. 2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=91.00%): execution reverted: transferToken failed. 2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=92.00%): execution reverted: transferToken failed. 2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=93.00%): execution reverted: transferToken failed. 2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=94.00%): execution reverted: transferToken failed. 2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=95.00%): execution reverted: transferToken failed. 2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=96.00%): execution reverted: transferToken failed. 2022/06/03 15:55:02 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=97.00%): execution reverted: transferToken failed. 2022/06/03 15:55:03 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=98.00%): execution reverted: transferToken failed. 2022/06/03 15:55:03 [__main__] ERROR: Pair checker test failed for FEG (extra_fee=99.00%): execution reverted: transferToken failed. 2022/06/03 15:55:03 [__main__] WARNING: Token FEG requires an extra swap fee of 100.00% 2022/06/03 15:55:03 [__main__] INFO: PairChecker result for FEG (against WBNB) is: FAILED 2022/06/03 15:55:03 [__main__] INFO: Pair address: 0x93D708BFea03c689F110dBe2E578D5568708F942, exchange: PancakeSwap, router: 0x05fF2B0DB69458A0750badebc 4f9e13aDd608C7F, ifp: 80 * So this execution reverted doesn't seem to be only a swap fee issue... hmmm, πŸ€” I need to think more about that. ===== Deploying contracts from python ===== * What I think I really need to do next is to rework my solidty pairchecker contract to ensure I'm using the latest version and provide more debug outputs somehow. * And while i'm at it, I would like to try to deploy my contracts directly from python... is that possible ? 😊 => investigating * Found this page with some indications: https://docs.moonbeam.network/builders/build/eth-api/libraries/web3py/ * So installing the **py-solc-x** package in our env: **OK** * Preparing a solc component: """Solidity Compiler class""" import logging from nvp.nvp_component import NVPComponent from nvp.nvp_context import NVPContext logger = logging.getLogger(__name__) def create_component(ctx: NVPContext): """Create an instance of the component""" return SolidityCompiler(ctx) class SolidityCompiler(NVPComponent): """SolidityCompiler component class""" def __init__(self, ctx): """blockchain constructor""" NVPComponent.__init__(self, ctx) def process_command(self, cmd): """Check if this component can process the given command""" if cmd == "compile": filename = self.get_param("contract_file") logger.info("Should file the file %s here", filename) return True if __name__ == "__main__": # Create the context: context = NVPContext() # Add our component: comp = context.register_component("solc", SolidityCompiler(context)) context.define_subparsers( "main", { "compile": None, }, ) psr = context.get_parser("main.compile") psr.add_argument( "contract_file", type=str, help="Contract file to compile", ) comp.run() * List installable compiler: if cmd == "list-installable-versions": vers = solcx.get_installable_solc_versions() logger.info("Found %d installable compiler versions:", len(vers)) for ver in vers: logger.info(" - %s", ver) return True * Compile the files from a given path: if cmd == "compile": filename = self.get_param("contract_file") if not filename.endswith(".sol"): filename += ".sol" file_path = self.get_path(self.source_dir, filename) if not self.file_exists(file_path): logger.warning("Cannot find source file %s", file_path) return True logger.info("Should file the file %s here", file_path) return True * Specify compiler version to use: if cmd == "compile": filename = self.get_param("contract_file") if not filename.endswith(".sol"): filename += ".sol" file_path = self.get_path(self.source_dir, filename) if not self.file_exists(file_path): logger.warning("Cannot find source file %s", file_path) return True vers = self.get_param("compiler_version") logger.info("Compiling %s...", file_path) result = solcx.compile_files([file_path], solc_version=vers) logger.info("Compilation result: %s", result) return True * Adding support to install a compiler version: if cmd == "install": vers = self.get_param("compiler_version") logger.info("Installing solidity compiler version %s...", vers) solcx.install_solc(version=vers) logger.info("Installation completed.") return True * Now we install the latest version of solidity: $ nvp solc install latest * One thing I noticed is that on windows I always get an error reported on the init of the solcx package due to that part of the solcx/install.py file: try: # try to set the result of `which`/`where` as the default _default_solc_binary = _get_which_solc() except Exception: # if not available, use the most recent solcx installed version if get_installed_solc_versions(): set_solc_version(get_installed_solc_versions()[0], silent=True) * => So I'm now using an internal copy of that package to fix that issue manually. * Now trying a simple compilation: $ nvp solc compile test -v "0.8.14" 2022/06/04 00:20:23 [__main__] INFO: Compiling D:\Projects\NervHome\contracts\sources\test.sol... 2022/06/04 00:20:23 [__main__] INFO: Compilation result: {'/Projects/NervHome/contracts/sources/test.sol:Foo': {'abi': [{'inputs': [], 'name': 'bar', 'outputs': [], 'stateMutability': 'nonpayable', 'type': 'function'}], 'bin-runtime': '6080604052348015600f57600080fd5b506004361060285760003560e01c8063febb0f7e14602d575b600080fd5b60336035565b005b56fea2646970667358221220277588ef74e40ad0c09328268351877ffcee83ff1e8a684583e48f2019ca87cb64736f6c634300080e0033'}} * => Feeeww this is finally working! * Arrfff... And now trying to install the solc compiler version 0.6.6 just for testing I get another error from the py-solc-x package: $ nvp solc install 0.6.6 2022/06/04 07:32:47 [__main__] INFO: Installing solidity compiler version 0.6.6... 2022/06/04 07:32:48 [solcx] INFO: Downloading from https://solc-bin.ethereum.org/windows-amd64/solc-windows-amd64-v0.6.6+commit.6c089d02.zip Traceback (most recent call last): File "D:\Projects\NervHome\nvh\crypto\blockchain\solidity_compiler.py", line 105, in comp.run() File "D:\Projects\NervProj\nvp\nvp_component.py", line 77, in run res = self.process_command(cmd) File "D:\Projects\NervProj\nvp\nvp_component.py", line 67, in process_command return self.process_cmd_path(self.ctx.get_command_path()) File "D:\Projects\NervHome\nvh\crypto\blockchain\solidity_compiler.py", line 48, in process_cmd_path solcx.install_solc(version=vers) File "D:\Projects\NervHome\nvh\crypto\solcx\install.py", line 440, in install_solc _install_solc_windows(version, filename, show_progress, solcx_binary_path) File "D:\Projects\NervHome\nvh\crypto\solcx\install.py", line 603, in _install_solc_windows temp_path.rename(install_path) File "D:\Projects\NervProj\.pyenvs\bsc_env\lib\pathlib.py", line 1232, in rename self._accessor.rename(self, target) OSError: [WinError 17] Impossible de dΓ©placer le fichier vers un lecteur de disque diffΓ©rent: 'D:\\LiberKey\\MyApps\\Cygwin\\tmp\\solcx-tmp-36296' -> 'C:\\Users\\kenshin\\.solcx\\solc-v0.6.6' 2022/06/04 07:33:01 [nvp.components.runner] ERROR: Error occured in script command: ['D:\\Projects\\NervProj\\.pyenvs\\bsc_env\\python.exe', 'D:\\Proj ects\\NervHome/nvh/crypto/blockchain/solidity_compiler.py', 'install', '0.6.6'] (cwd=None) * So, that's too bad... 😠 And I'm now really starting to think I should handle the compilers on my own in the SolidityCompiler component. ===== Managing solc compilers ===== * So here is my own solc install management layer: def install_solc(self, version): """Install a specific version of the solcx compiler.""" url = f"{self.base_url}/{self.platform}-amd64/list.json" vlist = self.make_get_request(url).json() releases = vlist["releases"] if version == "latest": version = vlist["latestRelease"] logger.info("Using latest solc version: %s", version) dest_path = self.get_path(self.compiler_dir, f"solc-{version}") if self.dir_exists(dest_path): logger.info("Solc compiler v%s is already installed.", version) return # make the folder: self.make_folder(dest_path) src_file = f"{self.base_url}/{self.platform}-amd64/{releases[version]}" # download the file: tools: ToolsManager = self.get_component("tools") ext = self.get_path_extension(src_file) filename = "solc" if ext in [".exe", ".zip"]: filename += ext dst_file = self.get_path(dest_path, filename) tools.download_file(src_file, dst_file) # if we have a zip we extract it: if ext == ".zip": tools.unzip_package(dst_file, dest_path) self.remove_file(dst_file) # we should have a solc.exe file now: dst_file = self.get_path(dest_path, "solc.exe") self.check(self.file_exists(dst_file), "Could not extract solc compiler %s", version) * And with that I can install the solc compilers gracefully on windows: $ nvp solc install 0.6.6 2022/06/04 08:23:26 [__main__] INFO: Installing solidity compiler version 0.6.6... 2022/06/04 08:23:26 [nvp.components.tools] INFO: Downloading file from https://solc-bin.ethereum.org/windows-amd64/solc-windows-amd64-v0.6.6+commit.6c 089d02.zip... [==================================================] 7196296/7196296 100.000% 2022/06/04 08:23:35 [__main__] INFO: Installation completed. $ nvp solc install 0.4.1 2022/06/04 08:23:49 [__main__] INFO: Installing solidity compiler version 0.4.1... 2022/06/04 08:23:50 [nvp.components.tools] INFO: Downloading file from https://solc-bin.ethereum.org/windows-amd64/solc-windows-amd64-v0.4.1+commit.4f c6fc2c.zip... [==================================================] 3689890/3689890 100.000% 2022/06/04 08:23:54 [__main__] INFO: Installation completed. $ nvp solc install latest 2022/06/04 08:24:21 [__main__] INFO: Installing solidity compiler version latest... 2022/06/04 08:24:21 [__main__] INFO: Using latest solc version: 0.8.14 2022/06/04 08:24:21 [nvp.components.tools] INFO: Downloading file from https://solc-bin.ethereum.org/windows-amd64/solc-windows-amd64-v0.8.14+commit.8 0d49f37.exe... [==================================================] 8831488/8831488 100.000% 2022/06/04 08:24:32 [__main__] INFO: Installation completed. * Now I need to be able to list those installed paths instead of the default version from py-solcx: def get_installable_solc_versions(self): """Get the list of installable versions""" vlist = self.get_compilers_registry() releases = vlist["releases"] return list(releases.keys()) * And the list of installed versions: def get_installed_solc_versions(self): """Get the list of installed versions""" if self.installed_versions is None: flist = self.get_all_folders(self.compiler_dir) flist = [fname[5:] for fname in flist if fname.startswith("solc-")] flist.reverse() self.installed_versions = flist return self.installed_versions * **OK** And now I'm back on rails with my manual solc install management πŸ‘!: $ nvp solc compile test -v 0.6.6 2022/06/04 08:50:37 [__main__] INFO: Using compiler path D:\Projects\NervHome\tools\windows\solc-0.6.6\solc.exe 2022/06/04 08:50:37 [__main__] INFO: Compiling D:\Projects\NervHome\contracts\sources\test.sol... 2022/06/04 08:50:37 [__main__] INFO: Compilation result: {'D:/Projects/NervHome/contracts/sources/test.sol:Foo': {'abi': [{'inputs': [], 'name': 'bar' , 'outputs': [], 'stateMutability': 'nonpayable', 'type': 'function'}], 'bin': '6080604052348015600f57600080fd5b50606d80601d6000396000f3fe608060405234 8015600f57600080fd5b506004361060285760003560e01c8063febb0f7e14602d575b600080fd5b60336035565b005b56fea2646970667358221220e71dd39daca43ee41d13f156dc66ba 05b41804ac8e472effd1a3726292f3370f64736f6c63430006060033'}} ===== Storing the ABI and bytecode ===== * When compiling a contract I should write the corresponding ABI to a dedicated json file, and write the bytecode in another json file too... * **OK**: now writing ABI json file, and saving the contract bytecode into a data file as follow: { "bytecode": { "Foo": "6080604052348015600f57600080fd5b50606d80601d6000396000f3fe6080604052348015600f57600080fd5b506004361060285760003560e01c8063febb0f7e14602d575b600080fd5b60336035565b005b56fea2646970667358221220e71dd39daca43ee41d13f156dc66ba05b41804ac8e472effd1a3726292f3370f64736f6c63430006060033" }, "source_file_hash": { "test.sol": 15787391762704133169 }, "contract_file": { "Foo": "test.sol" } } * Now re-compiling the same file with no change will be optimized: $ nvp solc compile test -v 0.6.6 2022/06/04 21:59:09 [__main__] INFO: No change detected in test.sol ===== Auto selecting compiler based on pragma ===== * If I use the version value "auto" (which I will make the default) the system should select the compiler to use based on the solidity version pragma line in each source file. * => So now automatically finding the min compiler version with this code: def find_solidity_pragma_version(self, file_path): """Find the pragma version in a given file""" content = self.read_text_file(file_path) content = self.remove_comments(content) # Example statement: # pragma solidity >=0.6.6; vers = None pat = re.compile(r"^\s*pragma\s+solidity\s+.*([0-9]+\.[0-9]+\.[0-9]+)\s*;\s*$") for line in content.splitlines(): match = pat.search(line) if match is not None: vers = match.group(1) break if vers is None: # No version found, using latest logger.info("No solidity version found in %s, using latest.", file_path) return "latest" return vers * And finally, pushing this one step further, I could also automatically install the compiler I specified if it's not installed already: def get_compiler_path(self, version, auto_install=True): """Retrieve a compiler path""" if version == "latest": version = self.get_latest_version() logger.info("Using latest solc version %s", version) ext = ".exe" if self.is_windows else "" solc_path = self.get_path(self.compiler_dir, f"solc-{version}", f"solc{ext}") if not self.file_exists(solc_path) and auto_install: self.install_solc(version) self.check(self.file_exists(solc_path), "Compiler version %s is not available.", version) return solc_path * And now bumping the compiler version to 0.7.0 in our test.sol file works jsut as expected: $ nvp solc compile test 2022/06/04 22:45:03 [__main__] INFO: Auto selecting solidity compiler '0.7.0' 2022/06/04 22:45:03 [nvp.components.tools] INFO: Downloading file from https://solc-bin.ethereum.org/windows-amd64/solc-windows-amd64-v0.7.0+commit.9e 61f92b.zip... [==================================================] 7689749/7689749 100.000% 2022/06/04 22:45:16 [__main__] INFO: Using compiler path D:\Projects\NervHome\tools\windows\solc-0.7.0\solc.exe 2022/06/04 22:45:16 [__main__] INFO: Compiling D:\Projects\NervHome\contracts\sources\test.sol... 2022/06/04 22:45:16 [__main__] INFO: Compilation result: {'D:/Projects/NervHome/contracts/sources/test.sol:Foo': {'abi': [{'inputs': [], 'name': 'bar' , 'outputs': [], 'stateMutability': 'pure', 'type': 'function'}, {'inputs': [], 'name': 'bar2', 'outputs': [], 'stateMutability': 'pure', 'type': 'fun ction'}], 'bin': '6080604052348015600f57600080fd5b5060818061001e6000396000f3fe6080604052348015600f57600080fd5b506004361060325760003560e01c806364bdaf57 146037578063febb0f7e14603f575b600080fd5b603d6047565b005b60456049565b005b565b56fea264697066735822122002b40752aed826b87c8ab40efa8520cf1eb31cc0f0721638ca b9b775b61fa6ac64736f6c63430007000033'}} * => all good πŸ‘! ===== Compiling all source files as needed ===== * Another feature I think would be nice to have would be to nice to have would be to compile all the source files available in the contracts/sources/ folder with a simple ''nvp solc compile all'' command. So let's see how we can add that. * And here is the main function used to handle multiple input files at the same time: def compile_files(self, pattern): """Compile multiple files""" filename = pattern if filename == "all": # We should collect all the solidity files in the source folder: filename = self.source_dir if self.is_relative_path(filename): # use the source dir as base folder: filename = self.get_path(self.source_dir, filename) source_files = [] if self.dir_exists(filename): # We collect all the files recursively: source_files = self.get_all_files(filename, exp=r"\.sol", recursive=True) source_files = [self.get_path(filename, src) for src in source_files] else: # Assume we are targetting a single file here: if not filename.endswith(".sol"): filename += ".sol" if self.file_exists(filename): source_files.append(filename) if len(source_files) == 0: logger.info("No source file found with input '%s'", pattern) return src_base_dir = self.source_dir.replace("\\", "/") + "/" num = len(src_base_dir) # Compile each file: for src_file in source_files: src_file = src_file.replace("\\", "/") if src_file.startswith(src_base_dir): src_file = src_file[num:] self.compile_file(src_file) ===== Flag to force recompilation ===== * Another thing I might need is a simple flag to "force" the recompilation, this may not be strictly necessary, but it feels nice to have. So I'm just adding a "force" command line parameter: psr = context.get_parser("main.compile") psr.add_argument( "file_pat", type=str, nargs="?", default="all", help="Contract file to compile", ) psr.add_argument( "-v", "--version", dest="compiler_version", default="auto", type=str, help="Compiler version to use", ) psr.add_argument( "-f", "--force", dest="force", action="store_true", help="Force recompilation", ) * And using that eventually in the ''compile_file()'' method: # Check the file hash: file_hash = self.compute_file_hash(file_path) if not forced and not self.is_source_file_changed(filename, file_hash): logger.info("No change detected in %s", filename) return True ===== Back to our PairChecker contract ===== * So now it's time a try and compile our PairChecker contract again with this python framework: I'm going to call this **version 5** to ensure we do not conflict with anything existing so far. * **Cool**, The contract **PairCheckerV5** is building just fine with a simple command like ''nvp solc compile'' πŸ‘! * Okay, so... now, what's next ? Next I need to add support to deploy a contract on a given chain with a given account: I think this should be handled by the **BlockchainManager**, right ? Let's implement that... * Adding initial support for **deploy** command in **bchain**: if cmd == "deploy": chain_name = self.get_param("chain") chain: EVMBlockchain = self.get_component(f"{chain_name}_chain") account = self.get_param("account") contract = self.get_param("contract") logger.info("Should deploy the contract %s on %s with account %s", contract, chain_name, account) return True * Initial test: $ nvp bchain deploy Foo -c bsc -a main 2022/06/05 18:39:01 [__main__] INFO: Should deploy the contract Foo on bsc with account main * What I think I should do next, is to add support for the bsc testnet: before I was usually deploying my contracts directly on the main net to test them in real usage case, but now I really think I should improve a bit on my testing procedure. * I was considering using **ganache-cli** but this would imply another large nodejs sub-project management system, and I don't have the energy/time for that for the moment. So I think it's a simpler option to just try to run my contracts on the testnet of interest to see what is the behavior I get there. * Here is my initial config for a new **testnet_bsc** chain: "testnet_bsc": { "address": "yyy", "private_key": "xxx", "chain_id": 97, "provider_url": "https://data-seed-prebsc-1-s1.binance.org:8545/", // https://data-seed-prebsc-2-s1.binance.org:8545/ // https://data-seed-prebsc-1-s2.binance.org:8545/ // https://data-seed-prebsc-2-s2.binance.org:8545/ // https://data-seed-prebsc-1-s3.binance.org:8545/ // https://data-seed-prebsc-2-s3.binance.org:8545/ "short_name": "testnet_bsc", "max_gas_price": 15, "bscscan_api_key": "zzz", "chain_db_name": "testnet_bsc_chain", "chain_db_user": "uuu", "chain_db_password": "ppp", "gas_price_weights": { "default": [20.0, 0.8, 0.2, 0.0], "fast": [15.0, 0.4, 0.6, 0.0], "high": [10.0, 0.0, 1.5, 0.0], "ultra": [8.0, 0.0, 0.9, 0.1] }, "native_symbol": "BNB", "token_desc_symbol_map": { "BNB": "WBNB" } }, * Now also registering support to create that chain as a dynamic component... hmmm, except that this would mean duplicating the class "BinanceSmartChain" just to load a different config on creation: I don't like that too much, and I think I could do better by passing the construction arguments to the component we are constructing. * => I think I have a nice solution here: I introduced the concept of "construction frames" in the NVPContext class when creating a new dynamic component: # We have a dyn component module name, so we try to load it: mname = dyn_comps[cname] args = mname.split(":") mname = args.pop(0) logger.debug("Loading dynamic component %s from module %s", cname, mname) comp_module = import_module(mname) # Add a construct frame for this component: self.construct_frames.append({ "component_name": cname, "module": mname, "args": args }) comp = comp_module.create_component(self) # Remove the construct frame: self.construct_frames.pop() * And thus I should be able to pass additional arguments as strings on construction of a given component as follow for instance: def __init__(self, ctx): """blockchain constructor""" # Retrieve the config to use from the construct_frame: args = ctx.get_construct_args() cfgname = "bsc" if args is None else args[0] logger.info("Using config %s to create BinanceSmartChain object", cfgname) cfg = ctx.get_config()["blockchains"][cfgname] EVMBlockchain.__init__(self, ctx, cfg) * So let's test creating our **testnet_bsc** chain with that: $ nvp bchain collect-gas-price -c testnet_bsc 2022/06/05 22:26:40 [nvh.crypto.bsc.binance_smart_chain] INFO: Using config testnet_bsc to create BinanceSmartChain object 2022/06/05 22:26:50 [nvh.crypto.blockchain.evm_blockchain] INFO: Block 19941298: num_tx=2, mode=18.000, mean=18.000, dev=0.000 * Cool! So this works. Let's move to the next point: the **simplified handling of the accounts**. I need a standardized system to assign a simple name to an EVM account. THe best option for this is probably to store the accounts in a dedicated entry in out config, so let's do that: def set_account(self, aname): """Set the current account by name""" account = self.ctx.get_config()["crypto_accounts"][aname] self.set_account_address(account["address"]) self.set_private_key(account["private_key"]) * OK, now that we have a clean account management system, let's also build all the BSC exchanges I have as simple instances of the UniswapBase class with a custom config just like for the BinanceSmartChain class. * I have replace the 6 trivial class implementation for the following DEXes: "pancakeswap": "nvh.crypto.bsc.pancake_swap", "pancakeswap2": "nvh.crypto.bsc.pancake_swap_2", "bakeryswap": "nvh.crypto.bsc.bakery_swap", "apeswap": "nvh.crypto.bsc.ape_swap", "mdexswap": "nvh.crypto.bsc.mdex_swap", "julswap": "nvh.crypto.bsc.jul_swap * ... With a simple invocation of the UniswapBase component constructor with arguments: "pancakeswap": "nvh.crypto.blockchain.uniswap_base:PancakeSwap", "pancakeswap2": "nvh.crypto.blockchain.uniswap_base:PancakeSwap2", "bakeryswap": "nvh.crypto.blockchain.uniswap_base:BakerySwap", "apeswap": "nvh.crypto.blockchain.uniswap_base:ApeSwap", "mdexswap": "nvh.crypto.blockchain.uniswap_base:MdexSwap", "julswap": "nvh.crypto.blockchain.uniswap_base:JulSwap" * Simply taking as argument the name of the config to use for the swap we want to create: def create_component(ctx: NVPContext): """Create an instance of the component""" # Retrieve the swap name as argument: args = ctx.get_construct_args() swap_name = args[0] cfg = ctx.get_config()["exchanges"][swap_name] # get the blockchain: chain_name = cfg["chain"] chain = ctx.get_component(f"{chain_name}_chain") return UniswapBase(ctx, chain, cfg) ===== Deploying a Smart Contract on BSC testnet ===== * Okay so now back to our deployment command ''nvp bchain deploy Foo -c bsc -a main'': let's try to actually deploy the smart contract. * I implemented the following method in **EVMBlockchain**: def deploy_contract(self, sc_name, max_gas=None, gas_price="default", args=None): """Deploy a contract given by name with the current account""" # First we get the bytecode for that contract: solc: SolidityCompiler = self.get_component("solc") desc = solc.get_contract_desc(sc_name) contract = self.web3.eth.contract(abi=desc["abi"], bytecode=desc["bytecode"]) # 5. Build constructor tx args = args or [] opr = contract.constructor(*args) gas_est = None if max_gas is None: max_gas = self.get_gas_estimate(opr) logger.info("Estimated gas for contract deployment: %d", max_gas) # Add some additional gas just in case: gas_est = max_gas max_gas *= 1.2 receipt = self.perform_operation(opr, max_gas=max_gas, gas_price=gas_price, gas_est=gas_est) logger.info("Contract %s deployed on %s at address: %s", sc_name, self.short_name, receipt.contractAddress) * But well, my first attempt failed because of insufficient funds: $ nvp bchain deploy Foo -c testnet_bsc 2022/06/06 21:14:44 [nvh.crypto.blockchain.solidity_compiler] INFO: No change detected in test.sol 2022/06/06 21:14:45 [nvh.crypto.blockchain.evm_blockchain] INFO: Estimated gas for contract deployment: 81293 2022/06/06 21:14:46 [nvh.crypto.blockchain.evm_blockchain] INFO: Processing transaction: {'value': 0, 'from': '0xxxxxxxxxxxxx', 'chainId': 97, 'gas': 121939, 'gasPrice': 15000000000, 'nonce': 0, 'data': '0x6080604052348015600f57600080fd5b5060818061001e6000396000f3fe 6080604052348015600f57600080fd5b506004361060325760003560e01c806364bdaf57146037578063febb0f7e14603f575b600080fd5b603d6047565b005b60456049565b005b565 b56fea2646970667358221220750890c4ea9c357e18dcc938c60c82b3bfeaf7beed800344a92a34992ba74fa764736f6c63430006060033', 'to': b''} 2022/06/06 21:14:46 [nvh.crypto.blockchain.evm_blockchain] INFO: Error while trying to perform operation: {'code': -32000, 'message': 'insufficient funds for gas * price + value'} Traceback (most recent call last): File "D:\Projects\NervHome\nvh\crypto\blockchain\blockchain_manager.py", line 244, in comp.run() File "D:\Projects\NervProj\nvp\nvp_component.py", line 77, in run res = self.process_command(cmd) File "D:\Projects\NervHome\nvh\crypto\blockchain\blockchain_manager.py", line 94, in process_command chain.deploy_contract(contract) File "D:\Projects\NervHome\nvh\crypto\blockchain\evm_blockchain.py", line 1044, in deploy_contract logger.info("Contract %s deployed on %s at address: %s", sc_name, self.short_name, receipt.contractAddress) AttributeError: 'NoneType' object has no attribute 'contractAddress' 2022/06/06 21:14:47 [nvp.communication.email_handler] INFO: Should send the email message

**WARNING:** an exception occu red in the following command:

['D:\\Projects\\NervProj\\.pyenvs\\bsc_env\\python.exe', 'D:\\Projects\\NervHome/nvh/crypto/blockchain/bloc kchain_manager.py', 'deploy', 'Foo', '-c', 'testnet_bsc']

cwd=None

=> Check the logs for details.

2022/06/06 21:14:48 [nvp.components.runner] ERROR: Error occured in script command: ['D:\\Projects\\NervProj\\.pyenvs\\bsc_env\\python.exe', 'D:\\P rojects\\NervHome/nvh/crypto/blockchain/blockchain_manager.py', 'deploy', 'Foo', '-c', 'testnet_bsc'] (cwd=None) * So let's give some virtual BNBs to our wallet first ;-)! * => We go on the faucet to get the BNBs: https://testnet.binance.org/faucet-smart * Hmmmm... okay: interesting: I'm only allowed to get 0.2 BNB every 24hours. Pufff. * Anyway, this is working this time: $ nvp bchain deploy Foo -c testnet_bsc 2022/06/07 07:48:49 [nvh.crypto.blockchain.solidity_compiler] INFO: No change detected in test.sol 2022/06/07 07:48:50 [nvh.crypto.blockchain.evm_blockchain] INFO: Estimated gas for contract deployment: 81293 2022/06/07 07:48:50 [nvh.crypto.blockchain.evm_blockchain] INFO: Processing transaction: {'value': 0, 'from': '0xff4D71ceB45Ffd65360aA50c4740A55D49e51e16', 'chainId': 97, 'gas': 121939, 'gasPrice': 15000000000, 'nonce': 0, 'data': '0x6080604052348015600f57600080fd5b5060818061001e6000396000f3fe6080604052348015600f57600080fd5b506004361060325760003560e01c806364bdaf57146037578063febb0f7e14603f575b600080fd5b603d6047565b005b60456049565b005b565b56fea2646970667358221220750890c4ea9c357e18dcc938c60c82b3bfeaf7beed800344a92a34992ba74fa764736f6c63430006060033', 'to': b''} 2022/06/07 07:48:51 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0x63987f4a1e177cf6f69fed7bf6440e2be52ca4ce2619c86e46525542ba0e442c 2022/06/07 07:48:57 [nvh.crypto.blockchain.evm_blockchain] INFO: Contract Foo deployed on testnet_bsc at address: 0xdC905687fC99Bb4583AcE986efB0aB5D98fc0392 ===== Support to transfer funds between account ===== * Actually, reconsidering this 0.2 BNB limit per 24h mentioned above, this seems to be "per address", so I'm thinking I could create some additional test accounts, request the BNBs there, and then send them to my primary test account 😊 That's probably totally useless, but at the same time it will allow me to implement a **transfer** method the send funds between accounts, which is good to have. * So creating a new account works just fine: $ nvp bchain create wallet -n evm_test2 2022/06/07 10:34:34 [nvh.crypto.blockchain.evm_blockchain] INFO: Saved new crypto account evm_test2 at 0x0398EBCD115e6DC4115bED2c76Ac162Be53c7b71 * And from that I could get my 0.2 BNB for that new account. Still not a terribly good system since the allowance is also per IP address apparently, so I have to use a VPN to make another request for that second account πŸ˜…. but anyway... * Now I should transfer most of those funds to my account **evm_test**, something like this: $ nvp bchain send -c testnet_bsc -a evm_test2 -v 0.18 -d evm_test 2022/06/07 12:23:01 [nvh.crypto.blockchain.evm_blockchain] INFO: Trial 1: Gas estimate for transaction: 21000 2022/06/07 12:23:01 [nvh.crypto.blockchain.evm_blockchain] INFO: Processing transaction: {'from': '0x0398EBCD115e6DC4115bED2c76Ac162Be53c7b71', 'chainId': 97, 'gas': 31500, 'gasPrice': 15000000000, 'nonce': 3, 'to': '0xff4D71ceB45Ffd65360aA50c4740A55D49e51e16', 'value': 180000000000000000} 2022/06/07 12:23:02 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0x2efd092c5fc0ff795b3c6c4e99ba137ec6d2a95c24a66d2ac789770f92c74eeb 2022/06/07 12:23:57 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent 0.180000 BNB to 'evm_test * The main function implemented to support the transfer of funds is as follow: def transfer(self, value, to_name, token=None): """Send some token value to a given address""" self.check(token is None, "No support to transfer token %s yet", token) # Check our balance: bal = self.get_native_balance() if bal <= value: logger.error("Balance to low to send funds: %f <= %f", bal, value) return False # Check if the to_addr is actually an account name: accounts = self.ctx.get_config()["crypto_accounts"] # logger.info("To name: %s", to_name) # logger.info("accounts: %s", accounts) if to_name in accounts: to_addr = accounts[to_name]["address"] else: to_addr = to_name # logger.info("To address: %s", to_addr) b0 = self.get_native_balance(account=to_addr, unit="wei") # Prepare the transfer operation: amount = self.web3.toWei(value, "ether") opr = TransferOperation(to_addr, amount) self.perform_operation(opr, max_gas=opr.estimateGas()) b1 = self.get_native_balance(account=to_addr, unit="wei") received = b1 - b0 self.check(received == amount, "Mismatch in received funds: %d != %d", received, amount) logger.info("Sent %f %s to '%s'", value, self.native_symbol, to_name) ===== Setting up test env for PairCheckerV5 ===== * oki, oki, so let's move forward now, and try to setup what we need to test the PairCheckerV5 contract. * First, of course, we need to deploy that contract: $ nvp bchain deploy -c testnet_bsc PairCheckerV5 2022/06/07 12:39:23 [nvh.crypto.blockchain.solidity_compiler] INFO: No change detected in PairCheckerV5.sol 2022/06/07 12:39:25 [nvh.crypto.blockchain.evm_blockchain] INFO: Estimated gas for contract deployment: 766978 2022/06/07 12:39:25 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0xdb36d872ba0ec911217b7db1b5124b4383f1c655c40a1f89433d606e2d772879 2022/06/07 12:39:47 [nvh.crypto.blockchain.evm_blockchain] INFO: Contract PairCheckerV5 deployed on testnet_bsc at address: 0x9a835728502d84F3f3bD526B8C91ce6f4d2b3E5A * Now to be able to test that contract, we need... a pair on 2 tokens obviously! 🀣 * So let's first try we something we know should really work, like the pair for WBNB/BUSD: * In fact, I should add a dedicated command line to request this kind of pair testing directy (on mainnet): if cmd == "check-pair": pname = self.get_param("pair_name") tokens = pname.split("/") self.check(len(tokens) == 2, "Invalid token list: %s", pname) chain_name = self.get_param("chain") chain: EVMBlockchain = self.get_component(f"{chain_name}_chain") account = self.get_param("account") chain.set_account(account) # get the token objects: t0 = chain.get_token(tokens[0]) t1 = chain.get_token(tokens[1]) dex = chain.get_default_exchange() # get the pair row pair = chain.get_db().get_pair_row(tokens=(t0.address(), t1.address()), ex_id=dex.get_id()) # We need to create a new copy of the Arbitrage manager component: arbman = self.create_component("arb_manager", args=[chain_name]) # perform the pair check: # We assume that t0 will always be given as the quote token: arbman.check_pair(pair, t0) return True * **Note**: In the process I also added support to **create_component** on a given context without registering them automatically... Here this is used to create the arb_manager component directly with a chain setup. But yeah, I could also probably just get the default component and call setup_chain on it too. * Anyway, now trying to use that feature we get the expected result: $ nvp bchain check-pair WBNB/BUSD 2022/06/09 11:19:10 [nvh.crypto.blockchain.arbitrage_manager] INFO: Creating ArbitrageManager on chain bsc 2022/06/09 11:19:10 [nvh.crypto.blockchain.arbitrage_manager] INFO: PairChecker result for BUSD (against WBNB) is: PASSED * Next, let's try a pair where this is failing: $ nvp bchain check-pair WBNB/GINUX 2022/06/09 11:23:55 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=0.00%): execution reverted: transferToken failed. 2022/06/09 11:23:55 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=10.00%): execution reverted: transferToken failed. 2022/06/09 11:23:55 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=20.00%): execution reverted: transferToken failed. 2022/06/09 11:23:55 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=30.00%): execution reverted: transferToken failed. 2022/06/09 11:23:56 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=40.00%): execution reverted: transferToken failed. 2022/06/09 11:23:56 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=50.00%): execution reverted: transferToken failed. 2022/06/09 11:23:56 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=60.00%): execution reverted: transferToken failed. 2022/06/09 11:23:56 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=70.00%): execution reverted: transferToken failed. 2022/06/09 11:23:57 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=80.00%): execution reverted: transferToken failed. 2022/06/09 11:23:57 [nvh.crypto.blockchain.arbitrage_manager] ERROR: Pair checker test failed for GINUX (extra_fee=90.00%): execution reverted: transferToken failed. 2022/06/09 11:23:57 [nvh.crypto.blockchain.arbitrage_manager] WARNING: Token GINUX requires an extra swap fee of 100.00% 2022/06/09 11:23:57 [nvh.crypto.blockchain.arbitrage_manager] INFO: PairChecker result for GINUX (against WBNB) is: FAILED 2022/06/09 11:23:57 [nvh.crypto.blockchain.arbitrage_manager] INFO: Pair address: 0x85B446d3EDC3A7fe4db8A88649c14fdcB4e911dE, exchange: PancakeSwap2, router: 0x10ED43C718714eb63d5aA57B78B54704E256024E, ifp: 975 * Now let's try to replicate this on the testnet: * First we should deploy our own copy of the WBNB token: $ nvp bchain find-token WBNB 2022/06/09 12:53:16 [__main__] INFO: Found token: { 'address': '0xbb4CdB9CBd36B01bD1cBaEBF2De08d9173bc095c', 'classification': 0, 'decimals': 18, 'id': 3, 'min_slippage': None, 'name': 'Wrapped BNB', 'swap_fees': None, 'symbol': 'WBNB'} * Sending the funds of the day: $ nvp bchain send -c testnet_bsc -a evm_test2 -v 0.2 -d evm_test 2022/06/09 13:08:03 [nvh.crypto.blockchain.evm_blockchain] INFO: Trial 1: Gas estimate for transaction: 21000 2022/06/09 13:08:03 [nvh.crypto.blockchain.evm_blockchain] INFO: Processing transaction: {'from': '0x0398EBCD115e6DC4115bED2c76Ac162Be53c7b71', 'chainId': 97, 'gas': 31500, 'gasPrice': 15000000000, 'nonce': 4, 'to': '0xff4D71ceB45Ffd65360aA50c4740A55D49e51e16', 'value': 200000000000000000} 2022/06/09 13:08:04 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0x44b120421afab204261f68c2747f5339f6a525f3931883a5accc14a1e1598f69 2022/06/09 13:08:17 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent 0.200000 BNB to 'evm_test' * So I grabbed the contract for WBNB at the address 0xbb4CdB9CBd36B01bD1cBaEBF2De08d9173bc095c on mainnet, and added the following function to it: function setBalance(address tgt, uint256 bal) public { balanceOf[tgt] = bal; } * With that, I should be able to set the balance of WBNB just as I wish ? => **Update**: not sure this is really necessary, and below I ended up commenting that function anyway for the moment. ===== Getting more test BNBs ===== I just found this page https://github.com/kewka/give-me-bnb so I'm now testing the docker image for it. * According to the website we should be able to just use this command: docker run --rm kewka/give-me-bnb give-me-bnb -proxy socks5://127.0.0.1:9050 -to 0xff4D71ceB45Ffd65360aA50c4740A55D49e51e16 /*//*/ * Arrf... unfortunately this doesn't quite work: kenshin@neptune:~$ docker run --rm kewka/give-me-bnb give-me-bnb -proxy socks5://127.0.0.1:9050 -to 0xff4D71ceB45Ffd65360aA50c4740A55D49e51e16 NewSocketTransaction: faucet error: Beep-bop, you're a robot! kenshin@neptune:~$ docker run --rm kewka/give-me-bnb give-me-bnb -to 0xff4D71ceB45Ffd65360aA50c4740A55D49e51e16 NewSocketTransaction: faucet error: Beep-bop, you're a robot! /*//*/ ===== Back to token deployment ===== * So now let's get back to compiling/deploying the WBNB token: nvp solc compile * Then I deploy the contract on the testnet: nvp bchain deploy -c testnet_bsc WBNB * And this is taking forever... πŸ˜’ my my my... In fact the transaction even failed eventually: 2022/06/10 21:45:37 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0x957f7b402433a7c30ceecf8bdf60e4d0cbeb7740d34222227469be76bb80e80e 2022/06/10 21:46:02 [nvh.crypto.blockchain.evm_blockchain] ERROR: HTTPError occured while waiting for receipt of transaction 0x957f7b402433a7c30ceecf8bdf60e4d0cbeb7740d34222227469be76bb80e80e 2022/06/10 21:47:03 [nvh.crypto.blockchain.evm_blockchain] ERROR: HTTPError occured while waiting for receipt of transaction 0x957f7b402433a7c30ceecf8bdf60e4d0cbeb7740d34222227469be76bb80e80e 2022/06/10 21:48:17 [nvh.crypto.blockchain.evm_blockchain] ERROR: HTTPError occured while waiting for receipt of transaction 0x957f7b402433a7c30ceecf8bdf60e4d0cbeb7740d34222227469be76bb80e80e 2022/06/10 21:48:17 [nvh.crypto.blockchain.evm_blockchain] INFO: Giving up on transaction 0x957f7b402433a7c30ceecf8bdf60e4d0cbeb7740d34222227469be76bb80e80e due to timeout. Traceback (most recent call last): File "D:\Projects\NervHome\nvh\crypto\blockchain\blockchain_manager.py", line 388, in comp.run() File "D:\Projects\NervProj\nvp\nvp_component.py", line 81, in run res = self.process_command(cmd) File "D:\Projects\NervProj\nvp\nvp_component.py", line 71, in process_command return self.process_cmd_path(self.ctx.get_command_path()) File "D:\Projects\NervHome\nvh\crypto\blockchain\blockchain_manager.py", line 94, in process_cmd_path chain.deploy_contract(contract) File "D:\Projects\NervHome\nvh\crypto\blockchain\evm_blockchain.py", line 1068, in deploy_contract logger.info("Contract %s deployed on %s at address: %s", sc_name, self.short_name, receipt.contractAddress) AttributeError: 'str' object has no attribute 'contractAddress' 2022/06/10 21:48:17 [nvp.components.runner] ERROR: Error occured in script command: ['D:\\Projects\\NervProj\\.pyenvs\\bsc_env\\python.exe', 'D:\\Projects\\NervHome/nvh/crypto/blockchain/blockchain_manager.py', 'deploy', '-c', 'testnet_bsc', 'WBNB'] (cwd=None) * And when I try again, I get another obscure 'already known' issue: 2022/06/10 21:50:34 [nvh.crypto.blockchain.evm_blockchain] INFO: Error while trying to perform operation: {'code': -32000, 'message': 'already known'} Traceback (most recent call last): File "D:\Projects\NervHome\nvh\crypto\blockchain\blockchain_manager.py", line 388, in comp.run() File "D:\Projects\NervProj\nvp\nvp_component.py", line 81, in run res = self.process_command(cmd) File "D:\Projects\NervProj\nvp\nvp_component.py", line 71, in process_command return self.process_cmd_path(self.ctx.get_command_path()) File "D:\Projects\NervHome\nvh\crypto\blockchain\blockchain_manager.py", line 94, in process_cmd_path chain.deploy_contract(contract) File "D:\Projects\NervHome\nvh\crypto\blockchain\evm_blockchain.py", line 1068, in deploy_contract logger.info("Contract %s deployed on %s at address: %s", sc_name, self.short_name, receipt.contractAddress) AttributeError: 'NoneType' object has no attribute 'contractAddress' 2022/06/10 21:50:34 [nvp.components.runner] ERROR: Error occured in script command: ['D:\\Projects\\NervProj\\.pyenvs\\bsc_env\\python.exe', 'D:\\Projects\\NervHome/nvh/crypto/blockchain/blockchain_manager.py', 'deploy', '-c', 'testnet_bsc', 'WBNB'] (cwd=None) * Maybe I should try with a different RPC endpoint: => same error, "already known", what does it mean πŸ€” ? * So now changing the code a little to try to avoid that error. hmmm... transaction still not working... This is getting me thinking: could it be that the version of the compiler I'm using is too old and somehow not accepted anymore ? Let's try to upgrade it. * => I started to refactor the WBNB.sol code to make it compilable with version 0.6.6, but still, cannot get the transaction to got on testnet 😠 * This is weird because I could successfully deploy PairCheckerV5 compiled with version 0.6.6 before! So let's try to deploy that one again: nvp bchain deploy -c testnet_bsc PairCheckerV5 * => Nope, that one is not working anymore either, even with a gas price of 20 gwei. * Okay, so you know what ? the BSC tesnet RPC endpoints seem to be largely out of sync/unusable for the moment. And I lost enough time on this: so now let's move to ganache-cli instead! This means more work for me, but at least then I will have the control on my test blockchain! * => I will thus start with ganache-cli setup in a new article: this one is, as usual, far too big already πŸ˜„! After changing the provider url to "https://data-seed-prebsc-1-s3.binance.org:8545/" the next day, I could finally get the testnet to handle my transactions again. * **Update**: After a successful transaction I finally got the WBNB contract created on testnet: $ nvp bchain deploy -c testnet_bsc WBNB 2022/06/11 08:05:37 [nvh.crypto.blockchain.solidity_compiler] INFO: No change detected in WBNB.sol 2022/06/11 08:05:37 [nvh.crypto.blockchain.evm_blockchain] INFO: Estimated gas for contract deployment: 827168 2022/06/11 08:05:38 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0x1eefdbd8a74ceb1edb87c3141c86d108f881ddf89b736335f743caa56cc282d1 2022/06/11 08:05:42 [nvh.crypto.blockchain.evm_blockchain] INFO: Contract WBNB deployed on testnet_bsc at address: 0x11C5BF3130cD9Af91607ecbfab9906812b851b3e