An Introduction to Tezos RPCs: a Basic Wallet

Date: 2018-11-15

In this technical blog post, we will briefly introduce Tezos RPCs through a simple example: we will show how the tezos-client program interacts with the tezos-node during a transfer command. Tezos RPCs are HTTP queries (GET or POST) to which tezos-node replies in JSON format. They are the only way for wallets to interact with the node. However, given the large number of RPCs accepted by the node, it is not always easy to understand which ones can be useful if you want to write a wallet. So, here, we use tezos-client as a simple example, that we will complete in another blog post for wallets that do not have access to the Tezos Protocol OCaml code.

As for the basic setup, we run a sandboxed node locally on port 9731, with two known addresses in its wallet, called bootstrap1 and bootstrap2.

Here is the command we are going to trace during this example:

tezos-client --addr 127.0.0.1 --port 9731 -l transfer 100 from bootstrap1 to bootstrap2

With this command, we send just 100 tezzies between the two accounts, paying only for the default fees (0.05 tz).

We use the -l option to request tezos-client to log all the RPC calls it uses on the standard error (the console).

The first query issued by tezos-client is:

>>>>0: http://127.0.0.1:9731/chains/main/blocks/head/context/contracts/tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx/counter
<<<<0: 200 OK
"2"

tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx is the Tezos address corresponding to bootstrap1 the payer of the operation. In Tezos, the payer is the address responsible for paying the fees and burn (storage) of the transaction. In our case, it is also the source of the transfer. Here, tezos-client requests the counter of the payer, because all operations must have a different counter. This is an important feature, here, it will prevent bootstrap2 from sending the same operation over and over, emptying the account of bootstrap1.

Here, the counter is 2, probably because we already issued some former operations, so the next operation should have a counter of 3. The request is done on the block head of the main chain, an alias for the last block baked on the chain.

The next query is:

>>>>1: http://127.0.0.1:9731/chains/main/blocks/head/context/contracts/tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx/manager_key
<<<<1: 200 OK
{ "manager": "tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx",
  "key": "edpkuBknW28nW72KG6RoHtYW7p12T6GKc7nAbwYX5m8Wd9sDVC9yav" }

This time, the client requests the key of the account manager. For a keyhash address (tz…), the manager is always itself, but this query is needed to know if the public key of the manager has been revealed. Here, the key field contains a public key, which means a revelation operation has already been published. Otherwise, the client would have had to also create this revelation operation prior to the transfer (or together, actually). The revelation is mandatory, because all the nodes need to know the public key of the manager to validate the signature of the transfer.

Let’s see the next query:

>>>>2: http://127.0.0.1:9731/monitor/bootstrapped
<<<<2: 200 OK
{ "block": "BLyypN89WuTQyLtExGP6PEuZiu5WFDxys3GTUf7Vz4KvgKcvo2E",
  "timestamp": "2018-10-13T00:32:47Z" }

This time, the client checks whether the node it is using is well connected to the network. A node is bootstrapped if it has enough connections to other nodes, and its chain is synchronized with them. This step is needed to prevent the operation from being sent on an obsolete fork of the chain.

Now, the next query requests the current configuration of the network.

>>>>3: http://127.0.0.1:9731/chains/main/blocks/head/context/constants
<<<<3: 200 OK
{ "proof_of_work_nonce_size": 8,
  "nonce_length": 32,
  "max_revelations_per_block": 32,
  "max_operation_data_length": 16384,
  "preserved_cycles": 5,
  "blocks_per_cycle": 4096,
  "blocks_per_commitment": 32,
  "blocks_per_roll_snapshot": 512,
  "blocks_per_voting_period": 32768,
  "time_between_blocks": [ "60", "75" ],
  "endorsers_per_block": 32, 
  "hard_gas_limit_per_operation": "400000",
  "hard_gas_limit_per_block": "4000000",
  "proof_of_work_threshold": "-1",
  "tokens_per_roll": "10000000000",
  "michelson_maximum_type_size": 1000,
  "seed_nonce_revelation_tip": "125000",
  "origination_burn": "257000",
  "block_security_deposit": "512000000",
  "endorsement_security_deposit": "64000000", 
  "block_reward": "16000000",
  "endorsement_reward": "2000000",
  "cost_per_byte": "1000",
  "hard_storage_limit_per_operation": "60000"
}

These constants may differ for different protocols, or different networks. They are for example different on mainnet, alphanet and zeronet. Among these constants, some of them are useful when issuing a transaction: mainly hard_gas_limit_per_operation and hard_storage_limit_per_operation . The first one is the maximum gas that can be set for a transaction, and the second one is the maximum storage that can be used. We don’t plan to use them directly, but we will use them to compute an approximation of the gas and storage that we will set for the transaction.

>>>>4: http://127.0.0.1:9731/chains/main/blocks/head/hash
<<<<4: 200 OK
"BLyypN89WuTQyLtExGP6PEuZiu5WFDxys3GTUf7Vz4KvgKcvo2E"

This query is a bit redundant with the /monitor/bootstrapped query, which already returned the last block baked on the chain. Anyway, it is useful if we are not working on the main chain.

The next query requests the chain_id of the main chain, which is typically useful to verify that we know the format of operations for this chain id:

>>>>5: http://127.0.0.1:9731/chains/main/chain_id
<<<<5: 200 OK
"NetXdQprcVkpaWU"

Finally, the client tries to simulate the transaction, using the maximal gas and storage limits requested earlier. Since it is in simulation mode, the transaction is only ran locally on the node, and immediately backtracked. It is used to know if the transactions executes successfully, and to know the gas and storage actually used (to avoid paying fees for an erroneous transaction) :

>>>>6: http://127.0.0.1:9731/chains/main/blocks/head/helpers/scripts/run_operation
{ "branch": "BLyypN89WuTQyLtExGP6PEuZiu5WFDxys3GTUf7Vz4KvgKcvo2E",
  "contents": [
    { "kind": "transaction",
      "source": "tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx",
      "fee": "50000",
      "counter": "3",
      "gas_limit": "400000",
      "storage_limit": "60000",
      "amount": "100000000",
      "destination": "tz1gjaF81ZRRvdzjobyfVNsAeSC6PScjfQwN" } 
    ],
  "signature":
    "edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"
}

The operation is related to a branch, and you can see that the branch field is here set to the hash of the last block head. The branch field is used to prevent an operation from being executed on an alternative head, and also for garbage collection: an operation can be inserted only in one of the 64 blocks after the branch block, or it will be deleted.

The result looks like this:

<<<<6: 200 OK
{ "contents": [ 
    { "kind": "transaction",
      "source": "tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx",
      "fee": "50000",
      "counter": "3",
      "gas_limit": "400000",
      "storage_limit": "60000",
      "amount": "100000000",
      "destination": "tz1gjaF81ZRRvdzjobyfVNsAeSC6PScjfQwN",
      "metadata": { 
        "balance_updates": [ 
         { "kind": "contract",
           "contract": "tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx",
           "change": "-50000" },
         { "kind": "freezer", 
           "category": "fees",
           "delegate": "tz1Ke2h7sDdakHJQh8WX4Z372du1KChsksyU",
           "level": 0, 
           "change": "50000" } 
          ],
        "operation_result":
          { "status": "applied",
            "balance_updates": [
             { "kind": "contract",
               "contract": "tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx",
               "change": "-100000000" },
             { "kind": "contract",
               "contract": "tz1gjaF81ZRRvdzjobyfVNsAeSC6PScjfQwN",
               "change": "100000000" } 
             ], 
        "consumed_gas": "100" } } } 
   ] 
}

Notice the consumed_gas field in the metadata section, that’s the gas that we can expect the transaction to use on the real chain. Here, there is no storage consumed, otherwise, a storage_size field would be present. The returned status is applied, meaning that the transaction could be successfully simulated by the node.

However, in the query, there was a field that we cannot easily infer: it is the signature field. Indeed, the tezos-client knows how to generate a signature for the transaction, knowing the public/private key of the manager. How can we do that in our wallet ? We will explain that in a next Tezos blog post.

Again, the tezos-client requests the last block head:

>>>>7: http://127.0.0.1:9731/chains/main/blocks/head/hash
<<<<7: 200 OK
"BLyypN89WuTQyLtExGP6PEuZiu5WFDxys3GTUf7Vz4KvgKcvo2E"

and the current chain id:

>>>>8: http://127.0.0.1:9731/chains/main/chain_id
<<<<8: 200 OK
"NetXdQprcVkpaWU"

The last simulation is a prevalidation of the transaction, with the exact same parameters (gas and storage) with which it will be submitted on the official blockchain:

>>>>9: http://127.0.0.1:9731/chains/main/blocks/head/helpers/preapply/operations
[ { "protocol": "PsYLVpVvgbLhAhoqAkMFUo6gudkJ9weNXhUYCiLDzcUpFpkk8Wt",
    "branch": "BLyypN89WuTQyLtExGP6PEuZiu5WFDxys3GTUf7Vz4KvgKcvo2E",
    "contents": [ 
     { "kind": "transaction", 
       "source": "tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx", 
       "fee": "50000",
       "counter": "3",
       "gas_limit": "200",
       "storage_limit": "0",
       "amount": "100000000",
       "destination": "tz1gjaF81ZRRvdzjobyfVNsAeSC6PScjfQwN" 
     } ], 
    "signature": "edsigu5Cb8WEmUZzoeGSL3sbSuswNFZoqRPq5nXA18Pg4RHbhnFqshL2Rw5QJBM94UxdWntQjmY7W5MqBDMhugLgqrRAWHyH5hD" 
} ]

Notice that, in this query, the gas_limit was set to 200. tezos-client is a bit conservative, adding 100 to the gas returned by the first simulation. Indeed, the gas can be different when the transaction is ran for inclusion, for example if a baker introduced another transaction before that interferes with this one (for example, a transaction that empties an account has an additionnal gas cost of 50).

<<<<9: 200 OK
[ { "contents": [
     { "kind": "transaction", 
       "source": "tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx",
       "fee": "50000",
       "counter": "3",
       "gas_limit": "200",
       "storage_limit": "0",
       "amount": "100000000",
       "destination": "tz1gjaF81ZRRvdzjobyfVNsAeSC6PScjfQwN",
       "metadata": { 
         "balance_updates": [ 
          { "kind": "contract",
            "contract": "tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx",
            "change": "-50000" },
          { "kind": "freezer",
            "category": "fees",
            "delegate": "tz1Ke2h7sDdakHJQh8WX4Z372du1KChsksyU",
            "level": 0,
            "change": "50000" } ],
         "operation_result": 
          { "status": "applied",
            "balance_updates": [ 
             { "kind": "contract",
               "contract": "tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx",
               "change": "-100000000" },
             { "kind": "contract",
               "contract": "tz1gjaF81ZRRvdzjobyfVNsAeSC6PScjfQwN",
               "change": "100000000" } ],
         "consumed_gas": "100" } 
     } } ], 
    "signature": "edsigu5Cb8WEmUZzoeGSL3sbSuswNFZoqRPq5nXA18Pg4RHbhnFqshL2Rw5QJBM94UxdWntQjmY7W5MqBDMhugLgqrRAWHyH5hD"
 } ]

Again, the tezos-client had to sign the transaction with the manager private key. This will be explained in a next blog post.

Since this prevalidation was successful, the client can now inject the transaction on the block chain:

>>>>10: http://127.0.0.1:9731/injection/operation?chain=main
"a75719f568f22f279b42fa3ce595c5d4d0227cc8cf2af351a21e50d2ab71ab3208000002298c03ed7d454a101eb7022bc95f7e5f41ac78d0860303c8010080c2d72f0000e7670f32038107a59a2b9cfefae36ea21f5aa63c00eff5b0ce828237f10bab4042a891d89e951de2c5ad4a8fa72e9514ee63fec9694a772b563bcac8ae0d332d57f24eae7d4a6fad784a8436b6ba03d05bf72e4408"
<<<<10: 200 OK
"ooUo7nUZAbZKhTuX5NC999BuHs9TZBmtoTrCWT3jFnW7vMdN25U"

We can see that this request does not contain the JSON encoding of the transaction, but a binary version (in hexadecimal format). This binary version is what is stored in the blockchain, to decrease the size of the storage. It contains both a binary encoding of the transaction, and the signature of the transaction. tezos-client knows this binary format, but if we want to create our own wallet, we will need a way to compute it by ourselves.

The node replies with the operation hash of the injected operation: the operation is now waiting for inclusion in the mempool of the node, and will be forwarded to other nodes so that the next baker can include it in the next block.

I hope you have now a better understanding of how a wallet can use Tezos RPCs to issue a transaction. We now have two remaining questions, for a next blog post:

How to generate the binary format of an operation, from the JSON encoding ? How to sign an operation, so that we can include this signature in the run, preapply and injection RPCs ?

If we can reply to these questions, we will also be able to sign operations offline.

Comments

lizhihohng (5 May 2019 at 6 h 59 min):

Before forge or sign a transaction, how to get a gas or gas limit, not a hard gas limit from contants?

Juliane (16 November 2019 at 15 h 29 min):

Good answer back in return of this difficulty with solid arguments and explaining all on the topic of that.

About OCamlPro:

OCamlPro is a R&D lab founded in 2011, with the mission to help industrial users benefit from experts with a state-of-the-art knowledge of programming languages theory and practice.

We provide audit, support, custom developer tools and training for both the most modern languages, such as Rust, Wasm and OCaml, and for legacy languages, such as COBOL or even home-made domain-specific languages;

We design, create and implement software with great added-value for our clients. High complexity is not a problem for our PhD-level experts. For example, we helped the French Income Tax Administration re-adapt and improve their internally kept M language, we designed a DSL to model and express revenue streams in the Cinema Industry, codename Niagara, and we also developed the prototype of the Tezos proof-of-stake blockchain from 2014 to 2018.

We have a long history of creating open-source projects, such as the Opam package manager, the LearnOCaml web platform, and contributing to other ones, such as the Flambda optimizing compiler, or the GnuCOBOL compiler.

We are also experts of Formal Methods, developing tools such as our SMT Solver Alt-Ergo (check our Alt-Ergo Users' Club) and using them to prove safety or security properties of programs.

Please reach out, we'll be delighted to discuss your challenges: contact@ocamlpro.com or book a quick discussion.

Most Recent Articles

2024

2023

[Show all]