I’ve a python app to trace and retailer utxo in a easy/flat construction that appears like this…

from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException 

# monitor the utxo
trackUtxoList = []

# rpc name 1
rpc_connection = AuthServiceProxy("http://%s:%s@127.0.0.1:8332"%(rpc_user, rpc_password))

for b in vary(blockStart, blockLimit): 

    # rpc name 2 (per block)
    blockhash = rpc_connection.getblockhash(b)

    # rpc name 3 (per block)
    block = rpc_connection.getblock(blockhash)

    for txid in block['tx']:
        
        # rpc name 4 (per tx)
        raw_tx = rpc_connection.getrawtransaction(txid,False,blockhash) 

        # rpc name 5 (per tx)
        decoded_tx = rpc_connection.decoderawtransaction(raw_tx)

        for output in decoded_tx['vout']:

            # rpc name 6 (per vout)
            for l in rpc_connection.deriveaddresses(output['scriptPubKey']['desc']):
                trackUtxoList.append({'deal with':str(l),'worth':output['value'],'block':b, 'txid':txid,'isUtxo':True})

        for output in decoded_tx['vin']:

            vinSearch = record(filter(lambda merchandise: merchandise['t'] == output['txid'], trackUtxoList))
            vinSearch[output['vout']]['isUtxo'] = False

The issue is that it runs very slowly for two causes that relate to my 2 questions…

1. My blocks listing is on an exterior laborious drive as a result of it is too massive for my laptop computer – fairly positive this may impression read-speeds and so I am questioning if it is attainable to someway cut up the blocks listing in order that for instance, I may pull 100k blocks onto my laptop computer, course of that, then delete and replica the subsequent 100k blocks onto my laptop computer and many others. I’ve already cut up the blocks listing (on exterior hdd) from the remainder of the information listing (on my laptop computer) however cannot discover any examples of individuals briefly splitting the blocks listing itself.

2. I am making 2 rpc calls per block, 2 per transaction and 1 per vout. To my eye it seems like calls 5 + 6 (as commented in my code instance) do not actually need to make use of my rpc_connection since they’re manipulations on information already pulled from the blockchain? Whereas the others are necessities/unavoidable? If that is right, are there any python libraries which are usually used to decode transactions and derive addresses with out utilizing an rpc name? I performed a bit of with Buterin’s https://pypi.org/venture/bitcoin/ however because it’s now not maintained I wasn’t positive whether or not it is my greatest guess.

Thanks for any ideas in any respect on the way to optimise my code to parse your entire blockchain