An Introduction to Tezos RPCs: Signing Operations
In a previous blogpost, we presented the RPCs used by tezos-client to send a transfer operation to a tezos-node. We were left with two remaining questions:
-
How to forge a binary operation, for signature
-
How to sign a binary operation
In this post, we will reply to these questions. We are still assuming a node running and waiting for RPCs on address 127.0.0.1:9731. Since we will ask this node to forge a request, we really need to trust it, as a malicious node could send a different binary transaction from the one we sent him.
Let’s take back our first operation:
{
"branch": "BMHBtAaUv59LipV1czwZ5iQkxEktPJDE7A9sYXPkPeRzbBasNY8",
"contents": [
{ "kind": "transaction",
"source": "tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx",
"fee": "50000",
"counter": "3",
"gas_limit": "200",
"storage_limit": "0",
"amount": "100000000",
"destination": "tz1gjaF81ZRRvdzjobyfVNsAeSC6PScjfQwN"
} ]
}
So, we need to translate this operation into a binary format, more
amenable for signature. For that, we use a new RPC to forge
operations. Under Linux, we can use the tool curl
to send the
request to the node:
$ curl -v -X POST http://127.0.0.1:9731/chains/main/blocks/head/helpers/forge/operations -H "Content-type: application/json" --data '{
"branch": "BMHBtAaUv59LipV1czwZ5iQkxEktPJDE7A9sYXPkPeRzbBasNY8",
"contents": [
{ "kind": "transaction",
"source": "tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx",
"fee": "50000",
"counter": "3",
"gas_limit": "200",
"storage_limit": "0",
"amount": "100000000",
"destination": "tz1gjaF81ZRRvdzjobyfVNsAeSC6PScjfQwN"
} ]
}'
Note that we use a POST request (request with content), with a
Content-type
header indicating that the content is in JSON format. We
get the following body in the reply :
"ce69c5713dac3537254e7be59759cf59c15abd530d10501ccf9028a5786314cf08000002298c03ed7d454a101eb7022bc95f7e5f41ac78d0860303c8010080c2d72f0000e7670f32038107a59a2b9cfefae36ea21f5aa63c00"
This is the binary representation of our operation, in hexadecimal format, exactly what we were looking for to be able to include operations on the blockchain. However, this representation is not yet complete, since we also need the operation to be signed by the manager.
To sign this operation, we will first use tezos-client
. That’s
something that we can do if we want, for example, to sign an operation
offline, for better security. Let’s assume that we have saved the
content of the string (ce69...3c00
without the quotes) in a file
operation.hex
, we can ask tezos-client
to sign it with:
$ tezos-client --addr 127.0.0.1 --port 9731 sign bytes 0x03$(cat operation.hex) for bootstrap1
The 0x03$(cat operation.hex)
is the concatenation of the 0x03
prefix and the hexa content of the operation.hex
, which is equivalent
to 0x03ce69...3c00
. The prefix is used (1) to indicate that the
representation is hexadecimal (0x
), and (2) that it should start with
03
, which is a watermark for operations in Tezos.
We get the following reply in the console:
Signature: edsigtkpiSSschcaCt9pUVrpNPf7TTcgvgDEDD6NCEHMy8NNQJCGnMfLZzYoQj74yLjo9wx6MPVV29CvVzgi7qEcEUok3k7AuMg
Wonderful, we have a signature, in base58check
format ! We can use
this signature in the run_operation
and preapply
RPCs… but not in
the injection
RPC, which requires a binary format. So, to inject the
operation, we need to convert to the hexadecimal version of the
signature. For that, we will use the base58check
package of Python
(we could do it in OCaml, but then, we could just use tezos-client
all along, no ?):
$ pip3 install base58check
$ python
>>>import base58check
>>>base58check.b58decode(b'edsigtkpiSSschcaCt9pUVrpNPf7TTcgvgDEDD6NCEHMy8NNQJCGnMfLZzYoQj74yLjo9wx6MPVV29CvVzgi7qEcEUok3k7AuMg').hex()
'09f5cd8612637e08251cae646a42e6eb8bea86ece5256cf777c52bc474b73ec476ee1d70e84c6ba21276d41bc212e4d878615f4a31323d39959e07539bc066b84174a8ff0de436e3a7'
All signatures in Tezos start with 09f5cd8612
, which is used to
generate the edsig
prefix. Also, the last 4 bytes are used as a
checksum (e436e3a7
). Thus, the signature itself is after this prefix
and before the checksum: 637e08251cae64...174a8ff0d
.
Finally, we just need to append the binary operation with the binary
signature for the injection, and put them into a string, and send that
to the server for injection. If we have stored the hexadecimal
representation of the signature in a file signature.hex
, then we can
use :
$ curl -v -H "Content-type: application/json" 'http://127.0.0.1:9731/injection/operation?chain=main' --data '"'$(cat operation.hex)$(cat signature.hex)'"'
and we receive the hash of this new operation:
"oo1iWZDczV8vw3XLunBPW6A4cjmdekYTVpRxRh77Fd1BVv4HV2R"
Again, we cheated a little, by using tezos-client
to generate the
signature. Let’s try to do it in Python, too !
First, we will need the secret key of bootstrap1. We can export from
tezos-client
to use it directly:
$ tezos-client show address bootstrap1 -S
Hash: tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx
Public Key: edpkuBknW28nW72KG6RoHtYW7p12T6GKc7nAbwYX5m8Wd9sDVC9yav
Secret Key: unencrypted:edsk3gUfUPyBSfrS9CCgmCiQsTCHGkviBDusMxDJstFtojtc1zcpsh
The secret key is exported on the last line by using the -S
argument, and it usually starts with edsk
. Again, it is in
base58check
, so we can use the same trick to extract its binary
value:
$ python3
>>> import base58check
>>> base58check.b58decode(b'edsk3gUfUPyBSfrS9CCgmCiQsTCHGkviBDusMxDJstFtojtc1zcpsh').hex()[8:72]
'8500c86780141917fcd8ac6a54a43a9eeda1aba9d263ce5dec5a1d0e5df1e598'
This time, we directly extracted the key, by removing the first 8 hexa
chars, and keeping only 64 hexa chars (using [8:72]
), since the key
is 32-bytes long. Let’s suppose that we save this value in a file
bootstrap1.hex
.
Now, we will use the following script to compute the signature:
import binascii
operation=binascii.unhexlify(open("operation.hex","rb").readline()[:-1])
seed = binascii.unhexlify(open("bootstrap1.hex","rb").readline()[:-1])
from pyblake2 import blake2b
h = blake2b(digest_size=32)
h.update(b'x03' + operation)
digest = h.digest()
import ed25519
sk = ed25519.SigningKey(seed)
sig = sk.sign(digest)
print(sig.hex())
The binascii
module is used to read the files in hexadecimal (after
removing the newlines), to get the binary representation of the
operation and of the Ed25519 seed. Ed25519 is an elliptive curve used
in Tezos to manage tz1
addresses, i.e. to sign data and check
signatures.
The blake2b
module is used to hash the message, before
signature. Again, we add a watermark to the operation, i.e. x03
,
before hashing. We also have to specify the size of the hash,
i.e. digest_size=32
, because the Blake2b hashing function can generate
hashes with different sizes.
Finally, we use the ed25519 module to transform the seed (private/secret key) into a signing key, and use it to sign the hash, that we print in hexadecimal. We obtain:
637e08251cae646a42e6eb8bea86ece5256cf777c52bc474b73ec476ee1d70e84c6ba21276d41bc212e4d878615f4a31323d39959e07539bc066b84174a8ff0d
This result is exactly the same as what we got using tezos-client !
We now have a complete wallet, i.e. the ability to create transactions
and sign them without tezos-client. Of course, there are several
limitations to this work: first, we have exposed the private key in
clear, which is usually not a very good idea for security; also, Tezos
supports three types of keys, tz1
for Ed25519 keys, tz2
for
Secp256k1 keys (same as Bitcoin/Ethereum) and tz3
for P256 keys;
finally, a realistic wallet would probably use cryptographic chips, on
a mobile phone or an external device (Ledger, etc.).
Comments
Anthony (28 November 2018 at 2 h 01 min):
Fabrice, you talk about signing the operation using tezos-client, which can then be used with the run_operation, however when . you talk about doing it in a script, it doesn’t include the edsig or checksum or converted back into a usable form for run_operations. Can you explain how this is done in a script?
Thanks Anthony
Fabrice Le Fessant (29 November 2018 at 15 h 07 min):
You are right,
run_operation
needs anedsig
signature, not the hexadecimal encoding. To generate theedsig
, you just need to use the reverse operation ofbase58check.b58decode
, i.e.base58check.b58encode
, on the concatenation of 3 byte arrays:1/ the 5-bytes prefix that will generate the initial
edsig
characters, i.e.0x09f5cd8612
in hexadecimal 2/ the raw signatures
3/ the 4 initial bytes of a checksum: the checksum is computed assha256(sha256(s))
Badalona (27 December 2018 at 13 h 13 min):
Hi Fabrice.
I will aprreciate if you show the coding of step 3. My checksum is always wrong.
Thanks
Alain (16 January 2019 at 16 h 26 min):
The checksum is on prefix + s. Here is a python3 script to do it:
./hex2edsig.py 637e08251cae646a42e6eb8bea86ece5256cf777c52bc474b73ec476ee1d70e84c6ba21276d41bc212e4d878615f4a31323d39959e07539bc066b84174a8ff0dedsigtkpiSSschcaCt9pUVrpNPf7TTcgvgDEDD6NCEHMy8NNQJCGnMfLZzYoQj74yLjo9wx6MPVV29CvVzgi7qEcEUok3k7AuMg
from pyblake2 import blake2b
import hashlib
import base58check
import ed25519
import sys
def sha256 (x) :
return hashlib.sha256(x).digest()
def b58check (prefix, b) :
x = prefix + b
checksum = sha256(sha256(x))[0:4]
return base58check.b58encode(x + checksum)
edsig_prefix = bytes([9, 245, 205, 134, 18])
hexsig = sys.argv[1]
bytessig = bytes.fromhex(hexsig)
b58sig = b58check (edsig_prefix, bytessig)
print(b58sig.decode('ascii'))
Anthony (29 November 2018 at 21 h 49 min):
Fabrice, Thanks for the information would you be able to show the coding as you have done in your blog? Thanks Anthony
Mark Robson (9 February 2020 at 23 h 51 min):
Great information, but can the article be updated to include the things discussed in the comments? As I can’t see the private key of bootstrap1 I can’t replicate locally. Been going around in circles on that point
stacey roberts (7 May 2020 at 13 h 53 min):
Can you help me to clear about how tezos can support to build a fully decentralized supply chain eco system?
leesadaisy (16 September 2020 at 10 h 25 min):
Hi there! Thanks for sharing useful info. Keep up your work.
Alice Jenifferze (17 September 2020 at 10 h 51 min):
Thanks for sharing!
About OCamlPro:
OCamlPro is a R&D lab founded in 2011, with the mission to help industrial users benefit from experts with a state-of-the-art knowledge of programming languages theory and practice.
- We provide audit, support, custom developer tools and training for both the most modern languages, such as Rust, Wasm and OCaml, and for legacy languages, such as COBOL or even home-made domain-specific languages;
- We design, create and implement software with great added-value for our clients. High complexity is not a problem for our PhD-level experts. For example, we helped the French Income Tax Administration re-adapt and improve their internally kept M language, we designed a DSL to model and express revenue streams in the Cinema Industry, codename Niagara, and we also developed the prototype of the Tezos proof-of-stake blockchain from 2014 to 2018.
- We have a long history of creating open-source projects, such as the Opam package manager, the LearnOCaml web platform, and contributing to other ones, such as the Flambda optimizing compiler, or the GnuCOBOL compiler.
- We are also experts of Formal Methods, developing tools such as our SMT Solver Alt-Ergo (check our Alt-Ergo Users' Club) and using them to prove safety or security properties of programs.
Please reach out, we'll be delighted to discuss your challenges: contact@ocamlpro.com or book a quick discussion.
Most Recent Articles
2024
- Alt-Ergo 2.6 is Out!
- Flambda2 Ep. 3: Speculative Inlining
- opam 2.2.0 release!
- Flambda2 Ep. 2: Loopifying Tail-Recursive Functions
- Fixing and Optimizing the GnuCOBOL Preprocessor
- OCaml Backtraces on Uncaught Exceptions
- Opam 102: Pinning Packages
- Flambda2 Ep. 1: Foundational Design Decisions
- Behind the Scenes of the OCaml Optimising Compiler Flambda2: Introduction and Roadmap
- Lean 4: When Sound Programs become a Choice
- Opam 101: The First Steps
2023
- Maturing Learn-OCaml to version 1.0: Gateway to the OCaml World
- The latest release of Alt-Ergo version 2.5.1 is out, with improved SMT-LIB and bitvector support!
- 2022 at OCamlPro
- Autofonce, GNU Autotests Revisited
- Sub-single-instruction Peano to machine integer conversion
- Statically guaranteeing security properties on Java bytecode: Paper presentation at VMCAI 23
- Release of ocplib-simplex, version 0.5
- The Growth of the OCaml Distribution