blog:2022:0522_crypto_linear_finance_claims [NervTech's Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Crypto: Restoring automatic Linear Finance claims ======

{{tag>dev python crypto nervhome finance bsc blockchain}}

Long story short: in the past year or so I have been implementing many python scripts/objects to handle interactions with some blockchain protocols automatically. And now that I have started to clean things up with the NervProj framework it's time for me to get back to these, and restore most of those scripts. 

One good starting point I believe is the automatic claim system I built for [[https://linear.finance/|Linear Finance]] on the **Binance Smart Chain**: This will force me to cover quite a lot of points already so this may take some time, but hey, we have to start somewhere right ?

====== ======

===== Before we start... =====

  * Before I start anything on this, as usual, some required "maintenance": lately I noticed I had a problem with my coingecko data retrieval scripts (something related to the rate limit apparently) and I had errors in the logs: <code>2022/05/20 22:25:07 [__main__] INFO: Writing 7 new price entries
2022/05/20 22:25:07 [__main__] INFO: 2/71: Done updating lagging highres data for ethereum
2022/05/20 22:25:10 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (1/10)...
2022/05/20 22:25:11 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (2/10)...
2022/05/20 22:25:13 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (3/10)...
2022/05/20 22:25:14 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (4/10)...
2022/05/20 22:25:16 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (5/10)...
2022/05/20 22:25:17 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (6/10)...
2022/05/20 22:25:18 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (7/10)...
2022/05/20 22:25:20 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (8/10)...
2022/05/20 22:25:21 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (9/10)...
2022/05/20 22:25:23 [nvp.nvp_object] ERROR: Received bad status 429 from get request to https://api.coingecko.com/api/v3/coins/litecoin/market_chart/range (params={'vs_currency': 'eth', 'from': 1653075901, 'to': 1653078306}), retrying (10/10)...
Traceback (most recent call last):
  File "/mnt/data1/dev/projects/NervHome/nvh/crypto/coingecko.py", line 687, in <module>
    comp.run()
  File "/mnt/data1/dev/projects/NervProj/nvp/nvp_component.py", line 69, in run
    res = self.process_command(cmd)
  File "/mnt/data1/dev/projects/NervHome/nvh/crypto/coingecko.py", line 579, in process_command
    self.update_highres_datasets()
  File "/mnt/data1/dev/projects/NervHome/nvh/crypto/coingecko.py", line 147, in update_highres_datasets
    self.update_highres_dataset(cid, 300)
  File "/mnt/data1/dev/projects/NervHome/nvh/crypto/coingecko.py", line 290, in update_highres_dataset
    self.update_price_dataset(PriceDB.HIGHRES, cid, start_stamp, range_dur, lambda _: period)
  File "/mnt/data1/dev/projects/NervHome/nvh/crypto/coingecko.py", line 493, in update_price_dataset
    if len(res2['prices']) > 0 and last_data[5] is not None:
TypeError: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/mnt/data1/dev/projects/NervProj/cli.py", line 5, in <module>
    ctx.run()
  File "/mnt/data1/dev/projects/NervProj/nvp/nvp_context.py", line 403, in run
    if comp.process_command(cmd):
  File "/mnt/data1/dev/projects/NervProj/nvp/components/runner.py", line 42, in process_command
    self.run_script(sname, proj)
  File "/mnt/data1/dev/projects/NervProj/nvp/components/runner.py", line 155, in run_script
    self.execute(cmd, cwd=cwd, env=env)
  File "/mnt/data1/dev/projects/NervProj/nvp/nvp_object.py", line 422, in execute
    subprocess.check_call(cmd, stdout=stdout, stderr=stderr, cwd=cwd, env=env)
  File "/mnt/data1/dev/projects/NervProj/tools/linux/python-3.10.2/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/mnt/data1/dev/projects/NervProj/.pyenvs/defi_env/bin/python3', 'nvh/crypto/coingecko.py', 'update-highres']' returned non-zero exit status 1.
</code>

  * Those errors would go totally unnoticed if I do not check my logs regularly, so I need to do something about that!
  * => I'm thinking that in the runner execute command I could catch that **CalledProcessError** exception and then send me an email and a notification on rochetchat about it: let's do that.

  * First thing to change: let's remove my **email_handler** and **rockerchat** components from the statically loaded components: these should be dynamic components instead.
  * So I moved my **rocketchat** component in the folder **nvp/communication** and just added the main section to run it on its own: <sxh python; highlight: []>if __name__ == "__main__":
    # Create the context:
    context = NVPContext()

    # Add our component:
    comp = context.register_component("rchat", RocketChat(context))

    context.define_subparsers("main", {
        'send': None,
    })

    psr = context.get_parser('main.send')
    psr.add_argument("message", type=str,
                     help="Simple message that we should send")

    comp.run()</sxh>

  * Next we add the script for it: <sxh javascript; highlight: []>    "rchat": {
      "help": "Send a message on the configured rocketchat server",
      "custom_python_env": "social_env",
      "cmd": "${PYTHON} ${NVP_ROOT_DIR}/nvp/communication/rocketchat.py send",
      "python_path": ["${NVP_ROOT_DIR}"]
    }</sxh>

  * And now the following command is working a expected: <sxh bash; highlight: []>$ nvp rchat "Hello manu"</sxh>

  * Same thing for the **email_handler** component, moved into **nvp/communication** and adding the main handling: <sxh python; highlight: []>if __name__ == "__main__":
    # Create the context:
    context = NVPContext()

    # Add our component:
    comp = context.register_component("email", EmailHandler(context))

    context.define_subparsers("main", {
        'send': None,
    })

    psr = context.get_parser('main.send')
    psr.add_argument("message", type=str,
                     help="HTML message that should be sent by email")
    psr.add_argument("-t", "--title", type=str,
                     help="Message title")
    psr.add_argument("-d", "--dest", type=str, dest='to_addrs',
                     help="Destination addresses")
    psr.add_argument("-f", "--from", type=str, dest='from_addr',
                     help="From address")

    comp.run()</sxh>

  * And the following command works fine: <sxh bash; highlight: []>$ nvp email "Coucou manu, ceci est un email de test" -t "Test de nervProj" -d roche.emmanuel@gmail.com</sxh>

  * Now sending a notification/email in case of failure of a runner script: <sxh python; highlight: []>        try:
            self.execute(cmd, cwd=cwd, env=env)
        except subprocess.SubprocessError:
            # And exception occured in the sub process, so we should send a notification:
            msg = ":warning: **WARNING:** an exception occured in the following command:\n"
            msg += f"{cmd}\n"
            msg += f"cwd={cwd}\n\n"
            msg += "=> Check the logs for details."

            rchat = self.get_component("rchat")
            rchat.send_message(msg)

            msg = "<p style=\"color: #fd0202;\">**WARNING:** an exception occured in the following command:</p>"
            msg += f"<p><em>{cmd}</em></p>"
            msg += f"<p>cwd={cwd}</p>"
            msg += "<p >=> Check the logs for details.</p>"

            email = self.get_component("email")
            email.send_message("[NervProj] Exception notification", msg)</sxh>

  * => Not the most detailed feedback, but at least it gets the job done: I should get a notification if something goes wrong, and from there I could act accordingly 👍! So this will be good enough for now.


===== Initial investigations =====

  * In my previous **NervSeed** giant project, I was executing the following script to claim the linear finance rewards: <sxh bash; highlight: []>    lfile="/mnt/array1/admin/logs/linear_finance.log"
    defi_linear -c 2>&1 | tee -a $lfile $mainlog</sxh>

  * **defi_linear** is defined as: <sxh bash; highlight: []>defi_linear()
{
  cd `defi_dir`
  nv_py_setup_paths
  defi_python linear.py "$@"
  cd - > /dev/null
}</sxh>

  * So let's check the linear.py file: <sxh python; highlight: []>import argparse
from nv.core.log_utils import *
from nv.defi.bsc.LinearFinance import *

parser = argparse.ArgumentParser()
parser.add_argument("-p","--pending", dest='pending', action='store_true', help="Display pending linear rewards")
parser.add_argument("-c","--collect", dest='collect', action='store_true', help="Collect pending linear rewards")

args = parser.parse_args()

dapp = LinearFinance()

if args.pending:
    val, _ = dapp.getPendingRewards()
    logDEBUG("Pending rewards: %.6f LINA" % val)
    nvSendRocketChatMessage(":white_check_mark: 💰 *[linear.finance]*:Pending reward: %.6f LINA" % (val))

if args.collect:
    dapp.collectPendingRewards()
</sxh>

  * And this brings us to the **LinearFinance** class: <sxh python; highlight: []>from nv.core.utils import * 
import os
import json
from datetime import date
from nv.defi.bsc.BinanceSmartChain import *
from decimal import Decimal
from nv.defi.coingecko import *
from nv.defi.bsc.PancakeSwap import *
from decimal import Decimal
import requests

class LinearFinance(object):

    def __init__(self, chain = None):
        self.chain = chain

        self.configFile = os.path.join(nvGetRootPath(),"config/linear.finance.json")

        self.loadConfig()
        
        if self.chain is None:
            self.chain = BinanceSmartChain()

        # Retrieve the stacker
        self.chef = self.chain.getContract(self.config["chef_address"], abiFile="ABI/linear_bsc.json")

        # need to collect the JSON data from the URL:
        # https://reward-query.linear-finance.workers.dev/rewards/0xyyyyyyyyyyyyyyyyyyyyyyyyy

    def stop(self):
        if self.chain is not None:
            # Stop the chain:
            self.chain.stop()

    def loadConfig(self):
        with open(self.configFile) as fd:
            self.config = json.load(fd)

    def saveConfig(self):
        with open(self.configFile, "w") as outfile:
            # logDEBUG("Saving config file...")
            json.dump(self.config,outfile,indent=2)

    def getPendingRewards(self):
        # retrieve the pending rewards:
        r = requests.get('https://reward-query.linear-finance.workers.dev/rewards/%s' % self.chain.getAccountAddress())

        entries = r.json()
        logDEBUG("Retrieved period entries: %s" % entries)

        # get the last claimed period Id:
        lastId = self.chef.callFunction("userLastClaimPeriodIds", self.chain.getAccountAddress())
        logDEBUG("Last claimed period ID: %d" % lastId)
        
        # Filter the entries keeping only the unclaimed values:
        # unclaimed = filter(lambda entry: entry["periodId"]>lastId, entries)
        unclaimed = [entry for entry in entries if entry["periodId"]>lastId]

        total_lina = 0.0
        for entry in unclaimed:
            val = Decimal(entry['stakingReward']) / Decimal(10 ** 18)
            total_lina += float(val)
        
        logDEBUG("Unclaimed entries: %s" % unclaimed)

        return total_lina, unclaimed

    def collectPendingRewards(self, minAmount=200.0):
        val, entries = self.getPendingRewards()
        if val < minAmount:
            logDEBUG("Not claiming %f LINA: min amount is %f LINA." % (val, minAmount))
            return
        
        # if date.today().weekday() != 4:  # 4==Friday
        #     logDEBUG("Not claiming %f LINA: awaiting Friday." % val)
        #     return

        # Execute all the claimable operations:
        for entry in entries:
            signature = entry['signatures'][0]['signature']
            amount = int(entry['stakingReward'])
            pId = int(entry['periodId'])
            logDEBUG("Claiming LINA rewards for period %d..." % pId)
            op = self.chef.buildFunctionCall("claimReward", pId, amount, int(entry['feeReward']), signature)

            if self.chain.performOperation(op, self.config['harvest_max_gas']) is not None:
                val = Decimal(amount)/Decimal(10**18)
                logDEBUG("=> Claimed %f LINA from linear finance for period %d" % (val, pId))
                nvSendRocketChatMessage(":white_check_mark: 💰 *[Linear.finance]*: Claimed %f LINA from linear finance for period %d" % (val, pId))        
</sxh>

  * The heart of this script in the **BinanceSmartChain** object where we call the ''performOperation()'' method => So let's restore that class first.

===== Restoring BinanceSmartChain class =====

  * Okay so... I had some troubles already with the python packages 😅: installing web3 & solana packages in the same python env seems to bring some version compatibility issues.
  * But anyway, now starting to build a initial **EVMBlockchain** base class: **OK** => This is an already pretty large class now, so I won't just copy all that code here.
  * In the process, I also restored the **EVMSmartContract** class used to represent a given SmartContract on chain.
  * And finally on top of that I added the **BinanceSmartChain** class implementation, which is pretty simple for now: <sxh python; highlight: []>"""BSC blockchain."""

import logging

from nvh.crypto.blockchain.evm_blockchain import EVMBlockchain

logger = logging.getLogger(__name__)


class BinanceSmartChain(EVMBlockchain):
    """BinanceSmartChain component class"""

    def __init__(self, ctx):
        """blockchain constructor"""
        cfg = ctx.get_config()['blockchains']['bsc']
        EVMBlockchain.__init__(self, ctx, cfg)

    def get_contract_abi_url(self, address):
        """Retrieve the ABI URL for a given contract"""
        api_key = self.config.get("bscscan_api_key", None)
        url = f"https://api.bscscan.com/api?module=contract&action=getabi&address={address}"

        if api_key is not None:
            url = f"{url}&apikey={api_key}"

        return url, 1.0/5 if api_key is not None else 5.0
</sxh>

  * Let's see what more I really need to support claiming from LinearFinance...
  * So here is my first version of the **LinearFinance** component: <sxh python; highlight: []>"""LinearFinance protocol."""

import logging

from nvp.nvp_context import NVPContext
from nvp.nvp_component import NVPComponent
from nvh.crypto.bsc.binance_smart_chain import BinanceSmartChain

logger = logging.getLogger(__name__)


class LinearFinance(NVPComponent):
    """LinearFinance component class"""

    def __init__(self, ctx, chain=None):
        """Component constructor"""
        NVPComponent.__init__(self, ctx)

        if chain is None:
            chain = self.ctx.get_component("bsc_chain")

        self.chain: BinanceSmartChain = chain

        self.config = self.ctx.get_config()["linear_finance"]

        logger.info("Loading Linear finance chef contract...")
        self.chef = self.chain.get_contract(self.config["chef_address"], abi_file="ABI/bsc_linear_finance.json")

    def process_command(self, cmd):
        """Check if this component can process the given command"""

        if cmd == 'pending':
            logger.info("Should report pending claims here.")
            return True

        if cmd == 'collect':
            logger.info("Should collect claims here.")
            return True

        return False


if __name__ == "__main__":
    # Create the context:
    context = NVPContext()

    # Add our component:
    comp = context.register_component("linear_finance", LinearFinance(context))

    context.define_subparsers("main", {
        'pending': None,
        'collect': None
    })

    comp.run()
</sxh>

  * let's see if we can run that... And after only a few minor bug fixes we get the correct outputs: <sxh bash; highlight: []>$ nvp linearfinance pending
2022/05/22 16:42:05 [nvh.crypto.blockchain.evm_blockchain] INFO: Initializing Web3 interface...
2022/05/22 16:42:05 [__main__] INFO: Loading Linear finance chef contract...
2022/05/22 16:42:05 [__main__] INFO: Should report pending claims here.</sxh>

===== Displaying the pending rewards =====


  * Now let's retrieve the pending values: for this, I add the ''get_pending_rewards()'' method: <sxh python; highlight: []>    def get_pending_rewards(self):
        """report the pending rewards"""
        # retrieve the pending rewards:
        addr = self.chain.get_account_address()
        resp = self.make_get_request(f'https://reward-query.linear-finance.workers.dev/rewards/{addr}')

        entries = resp.json()
        logger.info("Retrieved period entries: %s", entries)

        # get the last claimed period Id:
        last_id = self.chef.call_function("userLastClaimPeriodIds", addr)
        logger.info("Last claimed period ID: %d", last_id)

        # Filter the entries keeping only the unclaimed values:
        # unclaimed = filter(lambda entry: entry["periodId"]>last_id, entries)
        unclaimed = [entry for entry in entries if entry["periodId"] > last_id]

        total_lina = 0.0
        for entry in unclaimed:
            val = Decimal(entry['stakingReward']) / Decimal(10 ** 18)
            total_lina += float(val)

        logger.info("Unclaimed entries: %s", unclaimed)
        logger.info("Pending rewards: %.6f LINA", total_lina)

        return total_lina, unclaimed

    def process_command(self, cmd):
        """Check if this component can process the given command"""

        if cmd == 'pending':
            # logger.info("Should report pending claims here.")
            self.get_pending_rewards()
            return True

        if cmd == 'collect':
            logger.info("Should collect claims here.")
            return True

        return False</sxh>

  * And when calling this we get some valid outputs: <sxh bash; highlight: []>$ nvp linearfinance pending
2022/05/22 16:49:35 [nvh.crypto.blockchain.evm_blockchain] INFO: Initializing Web3 interface...
2022/05/22 16:49:35 [__main__] INFO: Loading Linear finance chef contract...
2022/05/22 16:49:36 [__main__] INFO: Retrieved period entries: [{'chainId': 56, 'periodId': 71, 'recipient': '0xyyyyyyyyyyyyyyyyyyyyyyyyy', 
'stakingReward': '1488466002032167998429', 'feeReward': '82584159233580863', 'signatures': [{'signer': '0x82356456F23850b7E
63A6729Fe4b2e5572a6Fd10', 'signature': '0xbdd9cab154a45da23f1009c027f9a3e09d2c19fbbf1fcf31f5a52040bab4203a1b6d796a2028336a56f6b8864aef0b03b
f6a6db37a7b047b34493be2b032c50d1c'}]}, {'chainId': 56, 'periodId': 72, 'recipient': '0xyyyyyyyyyyyyyyyyyyyyyyyyy', 'stakingR
eward': '1457397861844139565919', 'feeReward': '84433606472010255', 'signatures': [{'signer': '0x82356456F23850b7E63A6729Fe4b2e5572a6Fd10',
 'signature': '0x7dc952e523d0cb8dd35f177fc4b725782cda469576a18cf441d604ccccddb84a16138bfbac2d735583cb8d54fa1ebbe4473ca41d0dbf09a43fb219fe7a
ae9ce91c'}]}]
2022/05/22 16:49:36 [__main__] INFO: Last claimed period ID: 71
2022/05/22 16:49:36 [__main__] INFO: Unclaimed entries: [{'chainId': 56, 'periodId': 72, 'recipient': '0xyyyyyyyyyyyyyyyyyyyyyyyyy', 
'stakingReward': '1457397861844139565919', 'feeReward': '84433606472010255', 'signatures': [{'signer': '0x82356456F23850b7E63A6729
Fe4b2e5572a6Fd10', 'signature': '0x7dc952e523d0cb8dd35f177fc4b725782cda469576a18cf441d604ccccddb84a16138bfbac2d735583cb8d54fa1ebbe4473ca41d
0dbf09a43fb219fe7aae9ce91c'}]}]
2022/05/22 16:49:36 [__main__] INFO: Pending rewards: 1457.397862 LINA</sxh>

  * => Allright! Now let's try to actually claim those rewards 🤞!


===== Claiming pending rewards =====

  * I implemented the required function in LinearFinance: <sxh python; highlight: []>    def collect_pending_rewards(self, min_amount=200.0):
        """Collect the pending rewards"""
        val, entries = self.get_pending_rewards()
        if val < min_amount:
            logger.info("Not claiming %f LINA: min amount is %f LINA.", val, min_amount)
            return

        # if date.today().weekday() != 4:  # 4==Friday
        #     logDEBUG("Not claiming %f LINA: awaiting Friday." % val)
        #     return

        # Execute all the claimable operations:
        for entry in entries:
            signature = entry['signatures'][0]['signature']
            amount = int(entry['stakingReward'])
            period = int(entry['periodId'])
            logger.info("Claiming LINA rewards for period %d...", period)
            opr = self.chef.build_function_call("claimReward", period, amount, int(entry['feeReward']), signature)

            if self.chain.perform_operation(opr, self.config['harvest_max_gas']) is not None:
                val = Decimal(amount)/Decimal(10**18)
                logger.info("=> Claimed %f LINA from linear finance for period %d", val, period)
                rchat = self.get_component('rchat')
                msg = ":white_check_mark: 💰 *[Linear.finance]*: "
                msg += f"Claimed {val} LINA from linear finance for period {period}"
                rchat.send_message(msg)</sxh>

  * And then also added some support functions in the EVMBlockchain class, with mainly the **perform_operation** method below: <sxh python; highlight: []>    def get_gas_estimate(self, opr, value=None):
        """Get a gas estimate for a given operation"""

        params = {'from': self.account_address}
        if value is not None:
            params['value'] = value
        return opr.estimateGas(params)

    def get_default_gas_price(self):
        """Get a default gas price value"""
        # gasPrice = self.getGasOracle().getCurrentLowPrice()
        # logger.info("Using Oracle low Gas price: %d" % gasPrice)
        gas_price = 7
        logger.warning("Using fixed gas price of %d => should compute that value dynamically instead.", gas_price)
        return gas_price

    def get_next_transaction_nonce(self):
        """Retrieve the next transaction nonce for our current account"""
        return self.web3.eth.getTransactionCount(self.account_address)

    def wait_for_transaction_receipt(self, txh):
        """wait for the receipt of a given transaction."""
        return self.web3.eth.waitForTransactionReceipt(txh)

    def process_transaction(self, txobj):
        """Process a given transaction"""
        logger.info("Processing transaction: %s", txobj)

        self.check(self.private_key is not None, "Invalid private key.")

        signed_txn = self.web3.eth.account.sign_transaction(txobj, private_key=self.private_key)
        # logger.info("Signed txn hash: %s" % self.w3.toHex(signed_txn.hash))
        # logger.info("Signed txn raw: %s" % self.w3.toHex(signed_txn.rawTransaction))
        # logger.info("Signed txn r: %s" % self.w3.toHex(signed_txn.r))
        # logger.info("Signed txn s: %s" % self.w3.toHex(signed_txn.s))
        # logger.info("Signed txn v: %s" % signed_txn.v)

        # loDEBUG("Sending signed raw transaction...")
        txhash = self.web3.eth.sendRawTransaction(signed_txn.rawTransaction)

        # txhash2 = self.w3.toHex(self.w3.keccak(signed_txn.rawTransaction))
        # if txhash != txhash2:
        #     logWARN("Unexpected transaction hash: %s != %s" % (txhash, txhash2))

        logger.info("Sent transaction with hash: %s", txhash.hex())
        return txhash

    def send_transaction(self, opr, nonce=None, gas_est=None, gas_price=None, value=None):
        """Send a transaction on chain."""
        if gas_est is None:
            gas_est = self.get_gas_estimate(opr, value=value)

        if gas_price is None:
            gas_price = self.get_default_gas_price()
        if isinstance(gas_price, str):
            gas_price = self.compute_gas_price_estimate(gas_price)

        # if the gas price is too high then we cancel the transaction:
        if gas_price > self.max_gas_price:
            logger.error("Cancelling transaction: gas price is currently too high: %d > %d",
                         gas_price, self.max_gas_price)
            return None

        if nonce is None:
            nonce = self.get_next_transaction_nonce()

        params = {
            'from': self.account_address,
            'chainId': self.get_chain_id(),
            'gas': gas_est,
            'gas_price': self.web3.toWei(gas_price, 'gwei'),
            'nonce': nonce
        }

        if value is not None:
            params['value'] = value

        tx_obj = opr.buildTransaction(params)

        return self.process_transaction(tx_obj)

    def perform_operation(
            self, opr, max_gas, gas_price="default", gas_est_mult=1.5, show_errors=True, gas_est=None, value=None,
            num_trials=2, max_wait_count=3):
        """Perform an operation on chain"""
        try:
            trial = 0
            receipt = None

            while trial < num_trials:
                trial += 1

                if gas_est is None:
                    gas_est = self.get_gas_estimate(opr, value=value)
                    logger.info("Trial %d: Gas estimate for transaction: %d", trial, gas_est)

                if gas_price is None:
                    gas_price = self.get_default_gas_price()
                if isinstance(gas_price, str):
                    gas_price = self.compute_gas_price_estimate(gas_price)

                if gas_est > max_gas:
                    logger.info("Trial %d: Gas estimate too high, not performing operation.", trial)
                    continue
                else:
                    txhash = self.send_transaction(
                        opr, gas_price=gas_price, gas_est=int(gas_est * gas_est_mult),
                        value=value)
                    # logger.info("Received transaction hash: %s" % str(txhash))
                    # logger.info("Waiting for completion...")

                    # Add support to wait for a receipt as long as required once the transaction is sent
                    receipt = None
                    wait_count = 0
                    while receipt is None and (max_wait_count is None or wait_count < max_wait_count):
                        try:
                            wait_count += 1
                            receipt = self.wait_for_transaction_receipt(txhash)
                        except TimeExhausted:
                            logger.error("Timeout waiting for receipt of transaction %s", txhash.hex())
                        except ReadTimeout:
                            logger.error("ReadTimeout while waiting for receipt of transaction %s", txhash.hex())

                        # except Timeout:
                        #     logERROR("Timeout waiting for receipt of transaction %s" % txhash.hex())

                    if receipt is None:
                        logger.info("Giving up on transaction %s due to timeout.", txhash.hex())
                        return 'timeout'

                    if receipt['status'] == 0:
                        logger.error("Trial %d: Transaction was reverted by EVM: %s", trial, str(receipt))
                        receipt = None
                        continue

                    # We have a valid receipt, so we stop here:
                    break

            return receipt

        except ValueError as err:
            if show_errors:
                logger.info("Error while trying to perform operation: %s", str(err))
            return None

    def compute_gas_price_estimate(self, mode="default"):
        """Compute an advanced gas price estimate."""
        logger.warning("Using fixed gas price value of 7 for now.")
        return 7</sxh>

  * Now unfortunately, this doesn't seem to work for some strange reason 🤔: <sxh bash; highlight: []>$ nvp linearfinance collect
2022/05/22 17:28:58 [nvh.crypto.blockchain.evm_blockchain] INFO: Initializing Web3 interface...
2022/05/22 17:28:58 [__main__] INFO: Loading Linear finance chef contract...
2022/05/22 17:29:00 [__main__] INFO: Claiming LINA rewards for period 72...
2022/05/22 17:29:00 [nvh.crypto.blockchain.evm_blockchain] INFO: Trial 1: Gas estimate for transaction: 1052283
2022/05/22 17:29:00 [nvh.crypto.blockchain.evm_blockchain] WARNING: Using fixed gas price value of 7 for now.
D:\Projects\NervProj\.pyenvs\web3_env\lib\site-packages\web3\eth.py:656: UserWarning: There was an issue with the method eth_maxPriorityFee
PerGas. Calculating using eth_feeHistory.
  warnings.warn(
2022/05/22 17:29:00 [nvh.crypto.blockchain.evm_blockchain] INFO: Error while trying to perform operation: {'code': -32601, 'message': 'the
method eth_feeHistory does not exist/is not available'}
</sxh>

  * Okay so I downgraded from web3 version 5.29.1 to version 5.21.0 and now it seems that error is gone (I suspect it has something to do with the latest changes on the Ethereum blockchain, which is maybe not applicable yet to the BSC ?). And now I have another error, but that's because I was replacing everywhere "gasprice" with "gas_price" 😂: <sxh bash; highlight: []>2022/05/22 17:42:40 [nvh.crypto.blockchain.evm_blockchain] INFO: Initializing Web3 interface...
2022/05/22 17:42:40 [__main__] INFO: Loading Linear finance chef contract...
2022/05/22 17:42:41 [__main__] INFO: Claiming LINA rewards for period 72...
2022/05/22 17:42:41 [nvh.crypto.blockchain.evm_blockchain] INFO: Trial 1: Gas estimate for transaction: 1052271
2022/05/22 17:42:41 [nvh.crypto.blockchain.evm_blockchain] WARNING: Using fixed gas price of 7 => should compute that value dynamically ins
tead.
2022/05/22 17:42:42 [nvh.crypto.blockchain.evm_blockchain] INFO: Processing transaction: {'value': 0, 'gasPrice': 5000000000, 'from': '0x00
5a945c426fE1f2EB3c4B04585A2427F7a032A8', 'chainId': 56, 'gas': 1578406, 'gas_price': 7000000000, 'nonce': 3484, 'to': '0x9C86c4764E59A336C1
08A6F85be48F8a9a7FaD85', 'data': '0x6209deb400000000000000000000000000000000000000000000000000000000000000480000000000000000000000000000000
0000000000000004f017551b8f1c2e35f000000000000000000000000000000000000000000000000012bf7eaf0de8a0f000000000000000000000000000000000000000000
000000000000000000008000000000000000000000000000000000000000000000000000000000000000417dc952e523d0cb8dd35f177fc4b725782cda469576a18cf441d60
4ccccddb84a16138bfbac2d735583cb8d54fa1ebbe4473ca41d0dbf09a43fb219fe7aae9ce91c00000000000000000000000000000000000000000000000000000000000000
'}
Traceback (most recent call last):
  File "D:\Projects\NervHome\nvh\crypto\bsc\linear_finance.py", line 111, in <module>
    comp.run()
  File "D:\Projects\NervProj\nvp\nvp_component.py", line 69, in run
    res = self.process_command(cmd)
  File "D:\Projects\NervHome\nvh\crypto\bsc\linear_finance.py", line 93, in process_command
    self.collect_pending_rewards()
  File "D:\Projects\NervHome\nvh\crypto\bsc\linear_finance.py", line 75, in collect_pending_rewards
    if self.chain.perform_operation(opr, self.config['harvest_max_gas']) is not None:
  File "D:\Projects\NervHome\nvh\crypto\blockchain\evm_blockchain.py", line 373, in perform_operation
    txhash = self.send_transaction(
  File "D:\Projects\NervHome\nvh\crypto\blockchain\evm_blockchain.py", line 347, in send_transaction
    return self.process_transaction(tx_obj)
  File "D:\Projects\NervHome\nvh\crypto\blockchain\evm_blockchain.py", line 298, in process_transaction
    signed_txn = self.web3.eth.account.sign_transaction(txobj, private_key=self.private_key)
  File "D:\Projects\NervProj\.pyenvs\bsc_env\lib\site-packages\eth_utils\decorators.py", line 18, in _wrapper
    return self.method(obj, *args, **kwargs)
  File "D:\Projects\NervProj\.pyenvs\bsc_env\lib\site-packages\eth_account\account.py", line 748, in sign_transaction
    ) = sign_transaction_dict(account._key_obj, sanitized_transaction)
  File "D:\Projects\NervProj\.pyenvs\bsc_env\lib\site-packages\eth_account\_utils\signing.py", line 32, in sign_transaction_dict
    unsigned_transaction = serializable_unsigned_transaction_from_dict(transaction_dict)
  File "D:\Projects\NervProj\.pyenvs\bsc_env\lib\site-packages\eth_account\_utils\legacy_transactions.py", line 44, in serializable_unsigne
d_transaction_from_dict
    assert_valid_fields(transaction_dict)
  File "D:\Projects\NervProj\.pyenvs\bsc_env\lib\site-packages\eth_account\_utils\legacy_transactions.py", line 102, in assert_valid_fields

    raise TypeError("Transaction must not include unrecognized fields: %r" % superfluous_keys)
TypeError: Transaction must not include unrecognized fields: {'gas_price'}
2022/05/22 17:42:42 [nvp.components.runner] ERROR: Error occured in script command: ['D:\\Projects\\NervProj\\.pyenvs\\bsc_env\\python.exe'
, 'D:\\Projects\\NervHome/nvh/crypto/bsc/linear_finance.py', 'collect'] (cwd=None)</sxh>

  * And **Great!** with the simple fix for the error above, the transaction is now sent and processed correctly! 👍: <sxh bash; highlight: []>$ nvp linearfinance collect
2022/05/22 17:46:35 [nvh.crypto.blockchain.evm_blockchain] INFO: Initializing Web3 interface...
2022/05/22 17:46:35 [__main__] INFO: Loading Linear finance chef contract...
2022/05/22 17:46:37 [__main__] INFO: Claiming LINA rewards for period 72...
2022/05/22 17:46:38 [nvh.crypto.blockchain.evm_blockchain] INFO: Trial 1: Gas estimate for transaction: 1052271
2022/05/22 17:46:38 [nvh.crypto.blockchain.evm_blockchain] WARNING: Using fixed gas price of 7 => should compute that value dynamically ins
tead.
2022/05/22 17:46:38 [nvh.crypto.blockchain.evm_blockchain] INFO: Processing transaction: {'value': 0, 'from': '0x005a945c426fE1f2EB3c4B0458
5A2427F7a032A8', 'chainId': 56, 'gas': 1578406, 'gasPrice': 7000000000, 'nonce': 3484, 'to': '0x9C86c4764E59A336C108A6F85be48F8a9a7FaD85',
'data': '0x6209deb4000000000000000000000000000000000000000000000000000000000000004800000000000000000000000000000000000000000000004f017551b8
f1c2e35f000000000000000000000000000000000000000000000000012bf7eaf0de8a0f0000000000000000000000000000000000000000000000000000000000000080000
00000000000000000000000000000000000000000000000000000000000417dc952e523d0cb8dd35f177fc4b725782cda469576a18cf441d604ccccddb84a16138bfbac2d73
5583cb8d54fa1ebbe4473ca41d0dbf09a43fb219fe7aae9ce91c00000000000000000000000000000000000000000000000000000000000000'}
2022/05/22 17:46:38 [nvh.crypto.blockchain.evm_blockchain] INFO: Sent transaction with hash: 0xXXXXXXXXXXXXX[hidden hash]XXXXXXXXXXXXX
2022/05/22 17:46:43 [__main__] INFO: => Claimed 1457.397862 LINA from linear finance for period 72
</sxh>

===== Automating the claim process =====

  * Final step on the project: setup a script to collect the rewards automatically when available: I'm simply going to call the script above as part of my dayly cron script: <sxh bash; highlight: []>lfile="${log_dir}/linear_finance.log"
nvp linearfinance collect 2>&1 | tee -a $lfile
</sxh>

  * => And this is all I need here: which this small command I should be able to automatically claim my LINA rewards each time there is a new one available 👍!