Crypto: Restoring automatic Linear Finance claims

Long story short: in the past year or so I have been implementing many python scripts/objects to handle interactions with some blockchain protocols automatically. And now that I have started to clean things up with the NervProj framework it's time for me to get back to these, and restore most of those scripts.

One good starting point I believe is the automatic claim system I built for Linear Finance on the Binance Smart Chain: This will force me to cover quite a lot of points already so this may take some time, but hey, we have to start somewhere right ?

• Before I start anything on this, as usual, some required “maintenance”: lately I noticed I had a problem with my coingecko data retrieval scripts (something related to the rate limit apparently) and I had errors in the logs:

  2022/05/20 22:25:07 [__main__] INFO: Writing 7 new price entries
  2022/05/20 22:25:07 [__main__] INFO: 2/71: Done updating lagging highres data for ethereum
  2022/05/20 22:25:10 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (1/10)...
  2022/05/20 22:25:11 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (2/10)...
  2022/05/20 22:25:13 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (3/10)...
  2022/05/20 22:25:14 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (4/10)...
  2022/05/20 22:25:16 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (5/10)...
  2022/05/20 22:25:17 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (6/10)...
  2022/05/20 22:25:18 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (7/10)...
  2022/05/20 22:25:20 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (8/10)...
  2022/05/20 22:25:21 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (9/10)...
  2022/05/20 22:25:23 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (10/10)...
  Traceback (most recent call last):
    File "/mnt/data1/dev/projects/NervHome/nvh/crypto/coingecko.py", line 687, in <module>
      comp.run()
    File "/mnt/data1/dev/projects/NervProj/nvp/nvp_component.py", line 69, in run
      res = self.process_command(cmd)
    File "/mnt/data1/dev/projects/NervHome/nvh/crypto/coingecko.py", line 579, in process_command
      self.update_highres_datasets()
    File "/mnt/data1/dev/projects/NervHome/nvh/crypto/coingecko.py", line 147, in update_highres_datasets
      self.update_highres_dataset(cid, 300)
    File "/mnt/data1/dev/projects/NervHome/nvh/crypto/coingecko.py", line 290, in update_highres_dataset
      self.update_price_dataset(PriceDB.HIGHRES, cid, start_stamp, range_dur, lambda _: period)
    File "/mnt/data1/dev/projects/NervHome/nvh/crypto/coingecko.py", line 493, in update_price_dataset
      if len(res2['prices']) > 0 and last_data[5] is not None:
  TypeError: 'NoneType' object is not subscriptable
  Traceback (most recent call last):
    File "/mnt/data1/dev/projects/NervProj/cli.py", line 5, in <module>
      ctx.run()
    File "/mnt/data1/dev/projects/NervProj/nvp/nvp_context.py", line 403, in run
      if comp.process_command(cmd):
    File "/mnt/data1/dev/projects/NervProj/nvp/components/runner.py", line 42, in process_command
      self.run_script(sname, proj)
    File "/mnt/data1/dev/projects/NervProj/nvp/components/runner.py", line 155, in run_script
      self.execute(cmd, cwd=cwd, env=env)
    File "/mnt/data1/dev/projects/NervProj/nvp/nvp_object.py", line 422, in execute
      subprocess.check_call(cmd, stdout=stdout, stderr=stderr, cwd=cwd, env=env)
    File "/mnt/data1/dev/projects/NervProj/tools/linux/python-3.10.2/lib/python3.10/subprocess.py", line 369, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['/mnt/data1/dev/projects/NervProj/.pyenvs/defi_env/bin/python3', 'nvh/crypto/coingecko.py', 'update-highres']' returned non-zero exit status 1.

• Those errors would go totally unnoticed if I do not check my logs regularly, so I need to do something about that!
• ⇒ I'm thinking that in the runner execute command I could catch that CalledProcessError exception and then send me an email and a notification on rochetchat about it: let's do that.

• First thing to change: let's remove my email_handler and rockerchat components from the statically loaded components: these should be dynamic components instead.
• So I moved my rocketchat component in the folder nvp/communication and just added the main section to run it on its own:

  if __name__ == "__main__":
      # Create the context:
      context = NVPContext()
  
      # Add our component:
      comp = context.register_component("rchat", RocketChat(context))
  
      context.define_subparsers("main", {
          'send': None,
      })
  
      psr = context.get_parser('main.send')
      psr.add_argument("message", type=str,
                       help="Simple message that we should send")
  
      comp.run()

• Next we add the script for it:

      "rchat": {
        "help": "Send a message on the configured rocketchat server",
        "custom_python_env": "social_env",
        "cmd": "${PYTHON} ${NVP_ROOT_DIR}/nvp/communication/rocketchat.py send",
        "python_path": ["${NVP_ROOT_DIR}"]
      }

• And now the following command is working a expected:

  $ nvp rchat "Hello manu"

• Same thing for the email_handler component, moved into nvp/communication and adding the main handling:

  if __name__ == "__main__":
      # Create the context:
      context = NVPContext()
  
      # Add our component:
      comp = context.register_component("email", EmailHandler(context))
  
      context.define_subparsers("main", {
          'send': None,
      })
  
      psr = context.get_parser('main.send')
      psr.add_argument("message", type=str,
                       help="HTML message that should be sent by email")
      psr.add_argument("-t", "--title", type=str,
                       help="Message title")
      psr.add_argument("-d", "--dest", type=str, dest='to_addrs',
                       help="Destination addresses")
      psr.add_argument("-f", "--from", type=str, dest='from_addr',
                       help="From address")
  
      comp.run()

• And the following command works fine:

  $ nvp email "Coucou manu, ceci est un email de test" -t "Test de nervProj" -d roche.emmanuel@gmail.com

• Now sending a notification/email in case of failure of a runner script:

          try:
              self.execute(cmd, cwd=cwd, env=env)
          except subprocess.SubprocessError:
              # And exception occured in the sub process, so we should send a notification:
              msg = ":warning: **WARNING:** an exception occured in the following command:\n"
              msg += f"{cmd}\n"
              msg += f"cwd={cwd}\n\n"
              msg += "=> Check the logs for details."
  
              rchat = self.get_component("rchat")
              rchat.send_message(msg)
  
              msg = "<p style=\"color: #fd0202;\">**WARNING:** an exception occured in the following command:</p>"
              msg += f"<p><em>{cmd}</em></p>"
              msg += f"<p>cwd={cwd}</p>"
              msg += "<p >=> Check the logs for details.</p>"
  
              email = self.get_component("email")
              email.send_message("[NervProj] Exception notification", msg)

• ⇒ Not the most detailed feedback, but at least it gets the job done: I should get a notification if something goes wrong, and from there I could act accordingly 👍! So this will be good enough for now.

• In my previous NervSeed giant project, I was executing the following script to claim the linear finance rewards:

      lfile="/mnt/array1/admin/logs/linear_finance.log"
      defi_linear -c 2>&1 | tee -a $lfile $mainlog

• defi_linear is defined as:

  defi_linear()
  {
    cd `defi_dir`
    nv_py_setup_paths
    defi_python linear.py "$@"
    cd - > /dev/null
  }

• So let's check the linear.py file:

  import argparse
  from nv.core.log_utils import *
  from nv.defi.bsc.LinearFinance import *
  
  parser = argparse.ArgumentParser()
  parser.add_argument("-p","--pending", dest='pending', action='store_true', help="Display pending linear rewards")
  parser.add_argument("-c","--collect", dest='collect', action='store_true', help="Collect pending linear rewards")
  
  args = parser.parse_args()
  
  dapp = LinearFinance()
  
  if args.pending:
      val, _ = dapp.getPendingRewards()
      logDEBUG("Pending rewards: %.6f LINA" % val)
      nvSendRocketChatMessage(":white_check_mark: 💰 *[linear.finance]*:Pending reward: %.6f LINA" % (val))
  
  if args.collect:
      dapp.collectPendingRewards()
  

• And this brings us to the LinearFinance class:

  from nv.core.utils import * 
  import os
  import json
  from datetime import date
  from nv.defi.bsc.BinanceSmartChain import *
  from decimal import Decimal
  from nv.defi.coingecko import *
  from nv.defi.bsc.PancakeSwap import *
  from decimal import Decimal
  import requests
  
  class LinearFinance(object):
  
      def __init__(self, chain = None):
          self.chain = chain
  
          self.configFile = os.path.join(nvGetRootPath(),"config/linear.finance.json")
  
          self.loadConfig()
          
          if self.chain is None:
              self.chain = BinanceSmartChain()
  
          # Retrieve the stacker
          self.chef = self.chain.getContract(self.config["chef_address"], abiFile="ABI/linear_bsc.json")
  
          # need to collect the JSON data from the URL:
          # https://reward-query.linear-finance.workers.dev/rewards/0xyyyyyyyyyyyyyyyyyyyyyyyyy
  
      def stop(self):
          if self.chain is not None:
              # Stop the chain:
              self.chain.stop()
  
      def loadConfig(self):
          with open(self.configFile) as fd:
              self.config = json.load(fd)
  
      def saveConfig(self):
          with open(self.configFile, "w") as outfile:
              # logDEBUG("Saving config file...")
              json.dump(self.config,outfile,indent=2)
  
      def getPendingRewards(self):
          # retrieve the pending rewards:
          r = requests.get('https://reward-query.linear-finance.workers.dev/rewards/%s' % self.chain.getAccountAddress())
  
          entries = r.json()
          logDEBUG("Retrieved period entries: %s" % entries)
  
          # get the last claimed period Id:
          lastId = self.chef.callFunction("userLastClaimPeriodIds", self.chain.getAccountAddress())
          logDEBUG("Last claimed period ID: %d" % lastId)
          
          # Filter the entries keeping only the unclaimed values:
          # unclaimed = filter(lambda entry: entry["periodId"]>lastId, entries)
          unclaimed = [entry for entry in entries if entry["periodId"]>lastId]
  
          total_lina = 0.0
          for entry in unclaimed:
              val = Decimal(entry['stakingReward']) / Decimal(10 ** 18)
              total_lina += float(val)
          
          logDEBUG("Unclaimed entries: %s" % unclaimed)
  
          return total_lina, unclaimed
  
      def collectPendingRewards(self, minAmount=200.0):
          val, entries = self.getPendingRewards()
          if val < minAmount:
              logDEBUG("Not claiming %f LINA: min amount is %f LINA." % (val, minAmount))
              return
          
          # if date.today().weekday() != 4:  # 4==Friday
          #     logDEBUG("Not claiming %f LINA: awaiting Friday." % val)
          #     return
  
          # Execute all the claimable operations:
          for entry in entries:
              signature = entry['signatures'][0]['signature']
              amount = int(entry['stakingReward'])
              pId = int(entry['periodId'])
              logDEBUG("Claiming LINA rewards for period %d..." % pId)
              op = self.chef.buildFunctionCall("claimReward", pId, amount, int(entry['feeReward']), signature)
  
              if self.chain.performOperation(op, self.config['harvest_max_gas']) is not None:
                  val = Decimal(amount)/Decimal(10**18)
                  logDEBUG("=> Claimed %f LINA from linear finance for period %d" % (val, pId))
                  nvSendRocketChatMessage(":white_check_mark: 💰 *[Linear.finance]*: Claimed %f LINA from linear finance for period %d" % (val, pId))        
  

• The heart of this script in the BinanceSmartChain object where we call the performOperation() method ⇒ So let's restore that class first.

• Okay so… I had some troubles already with the python packages 😅: installing web3 & solana packages in the same python env seems to bring some version compatibility issues.
• But anyway, now starting to build a initial EVMBlockchain base class: OK ⇒ This is an already pretty large class now, so I won't just copy all that code here.
• In the process, I also restored the EVMSmartContract class used to represent a given SmartContract on chain.
• And finally on top of that I added the BinanceSmartChain class implementation, which is pretty simple for now:

  """BSC blockchain."""
  
  import logging
  
  from nvh.crypto.blockchain.evm_blockchain import EVMBlockchain
  
  logger = logging.getLogger(__name__)
  
  
  class BinanceSmartChain(EVMBlockchain):
      """BinanceSmartChain component class"""
  
      def __init__(self, ctx):
          """blockchain constructor"""
          cfg = ctx.get_config()['blockchains']['bsc']
          EVMBlockchain.__init__(self, ctx, cfg)
  
      def get_contract_abi_url(self, address):
          """Retrieve the ABI URL for a given contract"""
          api_key = self.config.get("bscscan_api_key", None)
          url = f"https://api.bscscan.com/api?module=contract&action=getabi&address={address}"
  
          if api_key is not None:
              url = f"{url}&apikey={api_key}"
  
          return url, 1.0/5 if api_key is not None else 5.0
  

• Let's see what more I really need to support claiming from LinearFinance…
• So here is my first version of the LinearFinance component:

  """LinearFinance protocol."""
  
  import logging
  
  from nvp.nvp_context import NVPContext
  from nvp.nvp_component import NVPComponent
  from nvh.crypto.bsc.binance_smart_chain import BinanceSmartChain
  
  logger = logging.getLogger(__name__)
  
  
  class LinearFinance(NVPComponent):
      """LinearFinance component class"""
  
      def __init__(self, ctx, chain=None):
          """Component constructor"""
          NVPComponent.__init__(self, ctx)
  
          if chain is None:
              chain = self.ctx.get_component("bsc_chain")
  
          self.chain: BinanceSmartChain = chain
  
          self.config = self.ctx.get_config()["linear_finance"]
  
          logger.info("Loading Linear finance chef contract...")
          self.chef = self.chain.get_contract(self.config["chef_address"], abi_file="ABI/bsc_linear_finance.json")
  
      def process_command(self, cmd):
          """Check if this component can process the given command"""
  
          if cmd == 'pending':
              logger.info("Should report pending claims here.")
              return True
  
          if cmd == 'collect':
              logger.info("Should collect claims here.")
              return True
  
          return False
  
  
  if __name__ == "__main__":
      # Create the context:
      context = NVPContext()
  
      # Add our component:
      comp = context.register_component("linear_finance", LinearFinance(context))
  
      context.define_subparsers("main", {
          'pending': None,
          'collect': None
      })
  
      comp.run()
  

• let's see if we can run that… And after only a few minor bug fixes we get the correct outputs:

  $ nvp linearfinance pending
  2022/05/22 16:42:05 [nvh.crypto.blockchain.evm_blockchain] INFO: Initializing Web3 interface...
  2022/05/22 16:42:05 [__main__] INFO: Loading Linear finance chef contract...
  2022/05/22 16:42:05 [__main__] INFO: Should report pending claims here.

• Now let's retrieve the pending values: for this, I add the get_pending_rewards() method:

      def get_pending_rewards(self):
          """report the pending rewards"""
          # retrieve the pending rewards:
          addr = self.chain.get_account_address()
          resp = self.make_get_request(f'https://reward-query.linear-finance.workers.dev/rewards/{addr}')
  
          entries = resp.json()
          logger.info("Retrieved period entries: %s", entries)
  
          # get the last claimed period Id:
          last_id = self.chef.call_function("userLastClaimPeriodIds", addr)
          logger.info("Last claimed period ID: %d", last_id)
  
          # Filter the entries keeping only the unclaimed values:
          # unclaimed = filter(lambda entry: entry["periodId"]>last_id, entries)
          unclaimed = [entry for entry in entries if entry["periodId"] > last_id]
  
          total_lina = 0.0
          for entry in unclaimed:
              val = Decimal(entry['stakingReward']) / Decimal(10 ** 18)
              total_lina += float(val)
  
          logger.info("Unclaimed entries: %s", unclaimed)
          logger.info("Pending rewards: %.6f LINA", total_lina)
  
          return total_lina, unclaimed
  
      def process_command(self, cmd):
          """Check if this component can process the given command"""
  
          if cmd == 'pending':
              # logger.info("Should report pending claims here.")
              self.get_pending_rewards()
              return True
  
          if cmd == 'collect':
              logger.info("Should collect claims here.")
              return True
  
          return False

• And when calling this we get some valid outputs:

  $ nvp linearfinance pending
  2022/05/22 16:49:35 [nvh.crypto.blockchain.evm_blockchain] INFO: Initializing Web3 interface...
  2022/05/22 16:49:35 [__main__] INFO: Loading Linear finance chef contract...
  2022/05/22 16:49:36 [__main__] INFO: Retrieved period entries: [{'chainId': 56, 'periodId': 71, 'recipient': '0xyyyyyyyyyyyyyyyyyyyyyyyyy', 
  'stakingReward': '1488466002032167998429', 'feeReward': '82584159233580863', 'signatures': [{'signer': '0x82356456F23850b7E
  63A6729Fe4b2e5572a6Fd10', 'signature': '0xbdd9cab154a45da23f1009c027f9a3e09d2c19fbbf1fcf31f5a52040bab4203a1b6d796a2028336a56f6b8864aef0b03b
  f6a6db37a7b047b34493be2b032c50d1c'}]}, {'chainId': 56, 'periodId': 72, 'recipient': '0xyyyyyyyyyyyyyyyyyyyyyyyyy', 'stakingR
  eward': '1457397861844139565919', 'feeReward': '84433606472010255', 'signatures': [{'signer': '0x82356456F23850b7E63A6729Fe4b2e5572a6Fd10',
   'signature': '0x7dc952e523d0cb8dd35f177fc4b725782cda469576a18cf441d604ccccddb84a16138bfbac2d735583cb8d54fa1ebbe4473ca41d0dbf09a43fb219fe7a
  ae9ce91c'}]}]
  2022/05/22 16:49:36 [__main__] INFO: Last claimed period ID: 71
  2022/05/22 16:49:36 [__main__] INFO: Unclaimed entries: [{'chainId': 56, 'periodId': 72, 'recipient': '0xyyyyyyyyyyyyyyyyyyyyyyyyy', 
  'stakingReward': '1457397861844139565919', 'feeReward': '84433606472010255', 'signatures': [{'signer': '0x82356456F23850b7E63A6729
  Fe4b2e5572a6Fd10', 'signature': '0x7dc952e523d0cb8dd35f177fc4b725782cda469576a18cf441d604ccccddb84a16138bfbac2d735583cb8d54fa1ebbe4473ca41d
  0dbf09a43fb219fe7aae9ce91c'}]}]
  2022/05/22 16:49:36 [__main__] INFO: Pending rewards: 1457.397862 LINA

• ⇒ Allright! Now let's try to actually claim those rewards 🤞!

• I implemented the required function in LinearFinance:

      def collect_pending_rewards(self, min_amount=200.0):
          """Collect the pending rewards"""
          val, entries = self.get_pending_rewards()
          if val < min_amount:
              logger.info("Not claiming %f LINA: min amount is %f LINA.", val, min_amount)
              return
  
          # if date.today().weekday() != 4:  # 4==Friday
          #     logDEBUG("Not claiming %f LINA: awaiting Friday." % val)
          #     return
  
          # Execute all the claimable operations:
          for entry in entries:
              signature = entry['signatures'][0]['signature']
              amount = int(entry['stakingReward'])
              period = int(entry['periodId'])
              logger.info("Claiming LINA rewards for period %d...", period)
              opr = self.chef.build_function_call("claimReward", period, amount, int(entry['feeReward']), signature)
  
              if self.chain.perform_operation(opr, self.config['harvest_max_gas']) is not None:
                  val = Decimal(amount)/Decimal(10**18)
                  logger.info("=> Claimed %f LINA from linear finance for period %d", val, period)
                  rchat = self.get_component('rchat')
                  msg = ":white_check_mark: 💰 *[Linear.finance]*: "
                  msg += f"Claimed {val} LINA from linear finance for period {period}"
                  rchat.send_message(msg)

• And then also added some support functions in the EVMBlockchain class, with mainly the perform_operation method below:

      def get_gas_estimate(self, opr, value=None):
          """Get a gas estimate for a given operation"""
  
          params = {'from': self.account_address}
          if value is not None:
              params['value'] = value
          return opr.estimateGas(params)
  
      def get_default_gas_price(self):
          """Get a default gas price value"""
          # gasPrice = self.getGasOracle().getCurrentLowPrice()
          # logger.info("Using Oracle low Gas price: %d" % gasPrice)
          gas_price = 7
          logger.warning("Using fixed gas price of %d => should compute that value dynamically instead.", gas_price)
          return gas_price
  
      def get_next_transaction_nonce(self):
          """Retrieve the next transaction nonce for our current account"""
          return self.web3.eth.getTransactionCount(self.account_address)
  
      def wait_for_transaction_receipt(self, txh):
          """wait for the receipt of a given transaction."""
          return self.web3.eth.waitForTransactionReceipt(txh)
  
      def process_transaction(self, txobj):
          """Process a given transaction"""
          logger.info("Processing transaction: %s", txobj)
  
          self.check(self.private_key is not None, "Invalid private key.")
  
          signed_txn = self.web3.eth.account.sign_transaction(txobj, private_key=self.private_key)
          # logger.info("Signed txn hash: %s" % self.w3.toHex(signed_txn.hash))
          # logger.info("Signed txn raw: %s" % self.w3.toHex(signed_txn.rawTransaction))
          # logger.info("Signed txn r: %s" % self.w3.toHex(signed_txn.r))
          # logger.info("Signed txn s: %s" % self.w3.toHex(signed_txn.s))
          # logger.info("Signed txn v: %s" % signed_txn.v)
  
          # loDEBUG("Sending signed raw transaction...")
          txhash = self.web3.eth.sendRawTransaction(signed_txn.rawTransaction)
  
          # txhash2 = self.w3.toHex(self.w3.keccak(signed_txn.rawTransaction))
          # if txhash != txhash2:
          #     logWARN("Unexpected transaction hash: %s != %s" % (txhash, txhash2))
  
          logger.info("Sent transaction with hash: %s", txhash.hex())
          return txhash
  
      def send_transaction(self, opr, nonce=None, gas_est=None, gas_price=None, value=None):
          """Send a transaction on chain."""
          if gas_est is None:
              gas_est = self.get_gas_estimate(opr, value=value)
  
          if gas_price is None:
              gas_price = self.get_default_gas_price()
          if isinstance(gas_price, str):
              gas_price = self.compute_gas_price_estimate(gas_price)
  
          # if the gas price is too high then we cancel the transaction:
          if gas_price > self.max_gas_price:
              logger.error("Cancelling transaction: gas price is currently too high: %d > %d",
                           gas_price, self.max_gas_price)
              return None
  
          if nonce is None:
              nonce = self.get_next_transaction_nonce()
  
          params = {
              'from': self.account_address,
              'chainId': self.get_chain_id(),
              'gas': gas_est,
              'gas_price': self.web3.toWei(gas_price, 'gwei'),
              'nonce': nonce
          }
  
          if value is not None:
              params['value'] = value
  
          tx_obj = opr.buildTransaction(params)
  
          return self.process_transaction(tx_obj)
  
      def perform_operation(
              self, opr, max_gas, gas_price="default", gas_est_mult=1.5, show_errors=True, gas_est=None, value=None,
              num_trials=2, max_wait_count=3):
          """Perform an operation on chain"""
          try:
              trial = 0
              receipt = None
  
              while trial < num_trials:
                  trial += 1
  
                  if gas_est is None:
                      gas_est = self.get_gas_estimate(opr, value=value)
                      logger.info("Trial %d: Gas estimate for transaction: %d", trial, gas_est)
  
                  if gas_price is None:
                      gas_price = self.get_default_gas_price()
                  if isinstance(gas_price, str):
                      gas_price = self.compute_gas_price_estimate(gas_price)
  
                  if gas_est > max_gas:
                      logger.info("Trial %d: Gas estimate too high, not performing operation.", trial)
                      continue
                  else:
                      txhash = self.send_transaction(
                          opr, gas_price=gas_price, gas_est=int(gas_est * gas_est_mult),
                          value=value)
                      # logger.info("Received transaction hash: %s" % str(txhash))
                      # logger.info("Waiting for completion...")
  
                      # Add support to wait for a receipt as long as required once the transaction is sent
                      receipt = None
                      wait_count = 0
                      while receipt is None and (max_wait_count is None or wait_count < max_wait_count):
                          try:
                              wait_count += 1
                              receipt = self.wait_for_transaction_receipt(txhash)
                          except TimeExhausted:
                              logger.error("Timeout waiting for receipt of transaction %s", txhash.hex())
                          except ReadTimeout:
                              logger.error("ReadTimeout while waiting for receipt of transaction %s", txhash.hex())
  
                          # except Timeout:
                          #     logERROR("Timeout waiting for receipt of transaction %s" % txhash.hex())
  
                      if receipt is None:
                          logger.info("Giving up on transaction %s due to timeout.", txhash.hex())
                          return 'timeout'
  
                      if receipt['status'] == 0:
                          logger.error("Trial %d: Transaction was reverted by EVM: %s", trial, str(receipt))
                          receipt = None
                          continue
  
                      # We have a valid receipt, so we stop here:
                      break
  
              return receipt
  
          except ValueError as err:
              if show_errors:
                  logger.info("Error while trying to perform operation: %s", str(err))
              return None
  
      def compute_gas_price_estimate(self, mode="default"):
          """Compute an advanced gas price estimate."""
          logger.warning("Using fixed gas price value of 7 for now.")
          return 7

• Now unfortunately, this doesn't seem to work for some strange reason 🤔:

  $ nvp linearfinance collect
  2022/05/22 17:28:58 [nvh.crypto.blockchain.evm_blockchain] INFO: Initializing Web3 interface...
  2022/05/22 17:28:58 [__main__] INFO: Loading Linear finance chef contract...
  2022/05/22 17:29:00 [__main__] INFO: Claiming LINA rewards for period 72...
  2022/05/22 17:29:00 [nvh.crypto.blockchain.evm_blockchain] INFO: Trial 1: Gas estimate for transaction: 1052283
  2022/05/22 17:29:00 [nvh.crypto.blockchain.evm_blockchain] WARNING: Using fixed gas price value of 7 for now.
  D:\Projects\NervProj\.pyenvs\web3_env\lib\site-packages\web3\eth.py:656: UserWarning: There was an issue with the method eth_maxPriorityFee
  PerGas. Calculating using eth_feeHistory.
    warnings.warn(
  2022/05/22 17:29:00 [nvh.crypto.blockchain.evm_blockchain] INFO: Error while trying to perform operation: {'code': -32601, 'message': 'the
  method eth_feeHistory does not exist/is not available'}
  

• Okay so I downgraded from web3 version 5.29.1 to version 5.21.0 and now it seems that error is gone (I suspect it has something to do with the latest changes on the Ethereum blockchain, which is maybe not applicable yet to the BSC ?). And now I have another error, but that's because I was replacing everywhere “gasprice” with “gas_price” 😂:

  2022/05/22 17:42:40 [nvh.crypto.blockchain.evm_blockchain] INFO: Initializing Web3 interface...
  2022/05/22 17:42:40 [__main__] INFO: Loading Linear finance chef contract...
  2022/05/22 17:42:41 [__main__] INFO: Claiming LINA rewards for period 72...
  2022/05/22 17:42:41 [nvh.crypto.blockchain.evm_blockchain] INFO: Trial 1: Gas estimate for transaction: 1052271
  2022/05/22 17:42:41 [nvh.crypto.blockchain.evm_blockchain] WARNING: Using fixed gas price of 7 => should compute that value dynamically ins
  tead.
  2022/05/22 17:42:42 [nvh.crypto.blockchain.evm_blockchain] INFO: Processing transaction: {'value': 0, 'gasPrice': 5000000000, 'from': '0x00
  5a945c426fE1f2EB3c4B04585A2427F7a032A8', 'chainId': 56, 'gas': 1578406, 'gas_price': 7000000000, 'nonce': 3484, 'to': '0x9C86c4764E59A336C1
  08A6F85be48F8a9a7FaD85', 'data': '0x6209deb400000000000000000000000000000000000000000000000000000000000000480000000000000000000000000000000
  0000000000000004f017551b8f1c2e35f000000000000000000000000000000000000000000000000012bf7eaf0de8a0f000000000000000000000000000000000000000000
  000000000000000000008000000000000000000000000000000000000000000000000000000000000000417dc952e523d0cb8dd35f177fc4b725782cda469576a18cf441d60
  4ccccddb84a16138bfbac2d735583cb8d54fa1ebbe4473ca41d0dbf09a43fb219fe7aae9ce91c00000000000000000000000000000000000000000000000000000000000000
  '}
  Traceback (most recent call last):
    File "D:\Projects\NervHome\nvh\crypto\bsc\linear_finance.py", line 111, in <module>
      comp.run()
    File "D:\Projects\NervProj\nvp\nvp_component.py", line 69, in run
      res = self.process_command(cmd)
    File "D:\Projects\NervHome\nvh\crypto\bsc\linear_finance.py", line 93, in process_command
      self.collect_pending_rewards()
    File "D:\Projects\NervHome\nvh\crypto\bsc\linear_finance.py", line 75, in collect_pending_rewards
      if self.chain.perform_operation(opr, self.config['harvest_max_gas']) is not None:
    File "D:\Projects\NervHome\nvh\crypto\blockchain\evm_blockchain.py", line 373, in perform_operation
      txhash = self.send_transaction(
    File "D:\Projects\NervHome\nvh\crypto\blockchain\evm_blockchain.py", line 347, in send_transaction
      return self.process_transaction(tx_obj)
    File "D:\Projects\NervHome\nvh\crypto\blockchain\evm_blockchain.py", line 298, in process_transaction
      signed_txn = self.web3.eth.account.sign_transaction(txobj, private_key=self.private_key)
    File "D:\Projects\NervProj\.pyenvs\bsc_env\lib\site-packages\eth_utils\decorators.py", line 18, in _wrapper
      return self.method(obj, *args, **kwargs)
    File "D:\Projects\NervProj\.pyenvs\bsc_env\lib\site-packages\eth_account\account.py", line 748, in sign_transaction
      ) = sign_transaction_dict(account._key_obj, sanitized_transaction)
    File "D:\Projects\NervProj\.pyenvs\bsc_env\lib\site-packages\eth_account\_utils\signing.py", line 32, in sign_transaction_dict
      unsigned_transaction = serializable_unsigned_transaction_from_dict(transaction_dict)
    File "D:\Projects\NervProj\.pyenvs\bsc_env\lib\site-packages\eth_account\_utils\legacy_transactions.py", line 44, in serializable_unsigne
  d_transaction_from_dict
      assert_valid_fields(transaction_dict)
    File "D:\Projects\NervProj\.pyenvs\bsc_env\lib\site-packages\eth_account\_utils\legacy_transactions.py", line 102, in assert_valid_fields
  
      raise TypeError("Transaction must not include unrecognized fields: %r" % superfluous_keys)
  TypeError: Transaction must not include unrecognized fields: {'gas_price'}
  2022/05/22 17:42:42 [nvp.components.runner] ERROR: Error occured in script command: ['D:\\Projects\\NervProj\\.pyenvs\\bsc_env\\python.exe'
  , 'D:\\Projects\\NervHome/nvh/crypto/bsc/linear_finance.py', 'collect'] (cwd=None)

• And Great! with the simple fix for the error above, the transaction is now sent and processed correctly! 👍:

  $ nvp linearfinance collect
  2022/05/22 17:46:35 [nvh.crypto.blockchain.evm_blockchain] INFO: Initializing Web3 interface...
  2022/05/22 17:46:35 [__main__] INFO: Loading Linear finance chef contract...
  2022/05/22 17:46:37 [__main__] INFO: Claiming LINA rewards for period 72...
  2022/05/22 17:46:38 [nvh.crypto.blockchain.evm_blockchain] INFO: Trial 1: Gas estimate for transaction: 1052271
  2022/05/22 17:46:38 [nvh.crypto.blockchain.evm_blockchain] WARNING: Using fixed gas price of 7 => should compute that value dynamically ins
  tead.
  2022/05/22 17:46:38 [nvh.crypto.blockchain.evm_blockchain] INFO: Processing transaction: {'value': 0, 'from': '0x005a945c426fE1f2EB3c4B0458
  5A2427F7a032A8', 'chainId': 56, 'gas': 1578406, 'gasPrice': 7000000000, 'nonce': 3484, 'to': '0x9C86c4764E59A336C108A6F85be48F8a9a7FaD85',
  'data': '0x6209deb4000000000000000000000000000000000000000000000000000000000000004800000000000000000000000000000000000000000000004f017551b8
  f1c2e35f000000000000000000000000000000000000000000000000012bf7eaf0de8a0f0000000000000000000000000000000000000000000000000000000000000080000
  00000000000000000000000000000000000000000000000000000000000417dc952e523d0cb8dd35f177fc4b725782cda469576a18cf441d604ccccddb84a16138bfbac2d73
  5583cb8d54fa1ebbe4473ca41d0dbf09a43fb219fe7aae9ce91c00000000000000000000000000000000000000000000000000000000000000'}
  2022/05/22 17:46:38 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0xXXXXXXXXXXXXX[hidden hash]XXXXXXXXXXXXX
  2022/05/22 17:46:43 [__main__] INFO: => Claimed 1457.397862 LINA from linear finance for period 72
  

• Final step on the project: setup a script to collect the rewards automatically when available: I'm simply going to call the script above as part of my dayly cron script:

  lfile="${log_dir}/linear_finance.log"
  nvp linearfinance collect 2>&1 | tee -a $lfile
  

• ⇒ And this is all I need here: which this small command I should be able to automatically claim my LINA rewards each time there is a new one available 👍!