An Introduction to Tezos RPCs: Signing Operations

Authors: Fabrice Le Fessant
Date: 2018-11-21
Category: Blockchains
Tags: blockchains, tezos


In a previous blogpost, we presented the RPCs used by tezos-client to send a transfer operation to a tezos-node. We were left with two remaining questions:

  • How to forge a binary operation, for signature

  • How to sign a binary operation

In this post, we will reply to these questions. We are still assuming a node running and waiting for RPCs on address 127.0.0.1:9731. Since we will ask this node to forge a request, we really need to trust it, as a malicious node could send a different binary transaction from the one we sent him.

Let’s take back our first operation:

{
  "branch": "BMHBtAaUv59LipV1czwZ5iQkxEktPJDE7A9sYXPkPeRzbBasNY8",
  "contents": [
    { "kind": "transaction",
      "source": "tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx",
      "fee": "50000",
      "counter": "3",
      "gas_limit": "200",
      "storage_limit": "0",
      "amount": "100000000",
      "destination": "tz1gjaF81ZRRvdzjobyfVNsAeSC6PScjfQwN"
   } ]
}

So, we need to translate this operation into a binary format, more amenable for signature. For that, we use a new RPC to forge operations. Under Linux, we can use the tool curl to send the request to the node:

$ curl -v -X POST http://127.0.0.1:9731/chains/main/blocks/head/helpers/forge/operations -H "Content-type: application/json" --data '{
  "branch": "BMHBtAaUv59LipV1czwZ5iQkxEktPJDE7A9sYXPkPeRzbBasNY8",
  "contents": [
    { "kind": "transaction",
      "source": "tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx",
      "fee": "50000",
      "counter": "3",
      "gas_limit": "200",
      "storage_limit": "0",
      "amount": "100000000",
      "destination": "tz1gjaF81ZRRvdzjobyfVNsAeSC6PScjfQwN"
  } ]
}'

Note that we use a POST request (request with content), with a Content-type header indicating that the content is in JSON format. We get the following body in the reply :

"ce69c5713dac3537254e7be59759cf59c15abd530d10501ccf9028a5786314cf08000002298c03ed7d454a101eb7022bc95f7e5f41ac78d0860303c8010080c2d72f0000e7670f32038107a59a2b9cfefae36ea21f5aa63c00"

This is the binary representation of our operation, in hexadecimal format, exactly what we were looking for to be able to include operations on the blockchain. However, this representation is not yet complete, since we also need the operation to be signed by the manager.

To sign this operation, we will first use tezos-client. That’s something that we can do if we want, for example, to sign an operation offline, for better security. Let’s assume that we have saved the content of the string (ce69...3c00 without the quotes) in a file operation.hex, we can ask tezos-client to sign it with:

$ tezos-client --addr 127.0.0.1 --port 9731 sign bytes 0x03$(cat operation.hex) for bootstrap1

The 0x03$(cat operation.hex) is the concatenation of the 0x03 prefix and the hexa content of the operation.hex, which is equivalent to 0x03ce69...3c00. The prefix is used (1) to indicate that the representation is hexadecimal (0x), and (2) that it should start with 03, which is a watermark for operations in Tezos.

We get the following reply in the console:

Signature: edsigtkpiSSschcaCt9pUVrpNPf7TTcgvgDEDD6NCEHMy8NNQJCGnMfLZzYoQj74yLjo9wx6MPVV29CvVzgi7qEcEUok3k7AuMg

Wonderful, we have a signature, in base58check format ! We can use this signature in the run_operation and preapply RPCs… but not in the injection RPC, which requires a binary format. So, to inject the operation, we need to convert to the hexadecimal version of the signature. For that, we will use the base58check package of Python (we could do it in OCaml, but then, we could just use tezos-client all along, no ?):

$ pip3 install base58check
$ python
>>>import base58check
>>>base58check.b58decode(b'edsigtkpiSSschcaCt9pUVrpNPf7TTcgvgDEDD6NCEHMy8NNQJCGnMfLZzYoQj74yLjo9wx6MPVV29CvVzgi7qEcEUok3k7AuMg').hex()
'09f5cd8612637e08251cae646a42e6eb8bea86ece5256cf777c52bc474b73ec476ee1d70e84c6ba21276d41bc212e4d878615f4a31323d39959e07539bc066b84174a8ff0de436e3a7'

All signatures in Tezos start with 09f5cd8612, which is used to generate the edsig prefix. Also, the last 4 bytes are used as a checksum (e436e3a7). Thus, the signature itself is after this prefix and before the checksum: 637e08251cae64...174a8ff0d.

Finally, we just need to append the binary operation with the binary signature for the injection, and put them into a string, and send that to the server for injection. If we have stored the hexadecimal representation of the signature in a file signature.hex, then we can use :

$ curl -v -H "Content-type: application/json" 'http://127.0.0.1:9731/injection/operation?chain=main' --data '"'$(cat operation.hex)$(cat signature.hex)'"'

and we receive the hash of this new operation:

"oo1iWZDczV8vw3XLunBPW6A4cjmdekYTVpRxRh77Fd1BVv4HV2R"

Again, we cheated a little, by using tezos-client to generate the signature. Let’s try to do it in Python, too !

First, we will need the secret key of bootstrap1. We can export from tezos-client to use it directly:

$ tezos-client show address bootstrap1 -S
Hash: tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx
Public Key: edpkuBknW28nW72KG6RoHtYW7p12T6GKc7nAbwYX5m8Wd9sDVC9yav
Secret Key: unencrypted:edsk3gUfUPyBSfrS9CCgmCiQsTCHGkviBDusMxDJstFtojtc1zcpsh

The secret key is exported on the last line by using the -S argument, and it usually starts with edsk. Again, it is in base58check, so we can use the same trick to extract its binary value:

$ python3
>>> import base58check
>>> base58check.b58decode(b'edsk3gUfUPyBSfrS9CCgmCiQsTCHGkviBDusMxDJstFtojtc1zcpsh').hex()[8:72]
'8500c86780141917fcd8ac6a54a43a9eeda1aba9d263ce5dec5a1d0e5df1e598'

This time, we directly extracted the key, by removing the first 8 hexa chars, and keeping only 64 hexa chars (using [8:72]), since the key is 32-bytes long. Let’s suppose that we save this value in a file bootstrap1.hex.

Now, we will use the following script to compute the signature:

import binascii

operation=binascii.unhexlify(open("operation.hex","rb").readline()[:-1])
seed = binascii.unhexlify(open("bootstrap1.hex","rb").readline()[:-1])

from pyblake2 import blake2b
h = blake2b(digest_size=32)
h.update(b'x03' + operation)
digest = h.digest()

import ed25519
sk = ed25519.SigningKey(seed)
sig = sk.sign(digest)
print(sig.hex())

The binascii module is used to read the files in hexadecimal (after removing the newlines), to get the binary representation of the operation and of the Ed25519 seed. Ed25519 is an elliptive curve used in Tezos to manage tz1 addresses, i.e. to sign data and check signatures.

The blake2b module is used to hash the message, before signature. Again, we add a watermark to the operation, i.e. x03, before hashing. We also have to specify the size of the hash, i.e. digest_size=32, because the Blake2b hashing function can generate hashes with different sizes.

Finally, we use the ed25519 module to transform the seed (private/secret key) into a signing key, and use it to sign the hash, that we print in hexadecimal. We obtain:

637e08251cae646a42e6eb8bea86ece5256cf777c52bc474b73ec476ee1d70e84c6ba21276d41bc212e4d878615f4a31323d39959e07539bc066b84174a8ff0d

This result is exactly the same as what we got using tezos-client !

SignTransaction-791x1024.jpg

We now have a complete wallet, i.e. the ability to create transactions and sign them without tezos-client. Of course, there are several limitations to this work: first, we have exposed the private key in clear, which is usually not a very good idea for security; also, Tezos supports three types of keys, tz1 for Ed25519 keys, tz2 for Secp256k1 keys (same as Bitcoin/Ethereum) and tz3 for P256 keys; finally, a realistic wallet would probably use cryptographic chips, on a mobile phone or an external device (Ledger, etc.).

Comments

Anthony (28 November 2018 at 2 h 01 min):

Fabrice, you talk about signing the operation using tezos-client, which can then be used with the run_operation, however when . you talk about doing it in a script, it doesn’t include the edsig or checksum or converted back into a usable form for run_operations. Can you explain how this is done in a script?

Thanks Anthony

Fabrice Le Fessant (29 November 2018 at 15 h 07 min):

You are right, run_operation needs an edsig signature, not the hexadecimal encoding. To generate the edsig, you just need to use the reverse operation of base58check.b58decode, i.e. base58check.b58encode, on the concatenation of 3 byte arrays:

1/ the 5-bytes prefix that will generate the initial edsig characters, i.e. 0x09f5cd8612 in hexadecimal 2/ the raw signature s 3/ the 4 initial bytes of a checksum: the checksum is computed as sha256(sha256(s))

Badalona (27 December 2018 at 13 h 13 min):

Hi Fabrice.

I will aprreciate if you show the coding of step 3. My checksum is always wrong.

Thanks

Alain (16 January 2019 at 16 h 26 min):

The checksum is on prefix + s. Here is a python3 script to do it:

./hex2edsig.py 637e08251cae646a42e6eb8bea86ece5256cf777c52bc474b73ec476ee1d70e84c6ba21276d41bc212e4d878615f4a31323d39959e07539bc066b84174a8ff0dedsigtkpiSSschcaCt9pUVrpNPf7TTcgvgDEDD6NCEHMy8NNQJCGnMfLZzYoQj74yLjo9wx6MPVV29CvVzgi7qEcEUok3k7AuMg

from pyblake2 import blake2b
import hashlib
import base58check
import ed25519
import sys

def sha256 (x) :
    return hashlib.sha256(x).digest()

def b58check (prefix, b) :
    x = prefix + b
    checksum = sha256(sha256(x))[0:4]
    return base58check.b58encode(x + checksum)

edsig_prefix = bytes([9, 245, 205, 134, 18])

hexsig = sys.argv[1]
bytessig = bytes.fromhex(hexsig)
b58sig = b58check (edsig_prefix, bytessig)
print(b58sig.decode('ascii'))

Anthony (29 November 2018 at 21 h 49 min):

Fabrice, Thanks for the information would you be able to show the coding as you have done in your blog? Thanks Anthony

Mark Robson (9 February 2020 at 23 h 51 min):

Great information, but can the article be updated to include the things discussed in the comments? As I can’t see the private key of bootstrap1 I can’t replicate locally. Been going around in circles on that point

stacey roberts (7 May 2020 at 13 h 53 min):

Can you help me to clear about how tezos can support to build a fully decentralized supply chain eco system?

leesadaisy (16 September 2020 at 10 h 25 min):

Hi there! Thanks for sharing useful info. Keep up your work.

Alice Jenifferze (17 September 2020 at 10 h 51 min):

Thanks for sharing!


About OCamlPro:

OCamlPro is a R&D lab founded in 2011, with the mission to help industrial users benefit from experts with a state-of-the-art knowledge of programming languages theory and practice.

  • We provide audit, support, custom developer tools and training for both the most modern languages, such as Rust, Wasm and OCaml, and for legacy languages, such as COBOL or even home-made domain-specific languages;
  • We design, create and implement software with great added-value for our clients. High complexity is not a problem for our PhD-level experts. For example, we helped the French Income Tax Administration re-adapt and improve their internally kept M language, we designed a DSL to model and express revenue streams in the Cinema Industry, codename Niagara, and we also developed the prototype of the Tezos proof-of-stake blockchain from 2014 to 2018.
  • We have a long history of creating open-source projects, such as the Opam package manager, the LearnOCaml web platform, and contributing to other ones, such as the Flambda optimizing compiler, or the GnuCOBOL compiler.
  • We are also experts of Formal Methods, developing tools such as our SMT Solver Alt-Ergo (check our Alt-Ergo Users' Club) and using them to prove safety or security properties of programs.

Please reach out, we'll be delighted to discuss your challenges: contact@ocamlpro.com or book a quick discussion.