Node Bootstrap

How to get started running on node on Flow

This guide is for getting a new node staked and running on Flow. For sporking documentation for existing node operators, see Spork Process.

Timing

New nodes are able to join the network each time a new epoch begins. In order to join the network at epoch N+1, the node must be registered with sufficient stake and authorized by the service account prior to the end of epoch N's Staking Auction Phase. Confirmation of a new node's inclusion in epoch N+1 is included in the EpochSetup event.

Nodes registered for epoch N+1 are able to participate in network communication on a limited basis starting in the Epoch Setup Phase of epoch N.

Flow Epoch Schedule

Once registered and confirmed to join the network at epoch N+1, the node must start up before epoch N+1 begins.

  • Verification & Access nodes may start up any time during the Epoch Setup Phase.
  • Consensus & Collection nodes must start up within the first 1000 views (~30mins) of the Epoch Setup Phase to participate in the Epoch Preparation Protocol.

Step 1 - Run Genesis Bootstrap

You will need to run this process for each node that you are operating

Download the Bootstrapping Kit

If you have downloaded the bootstrapping kit previously, ensure you check the hash below still matches. If not, re-download to ensure you are using the most up-to-date version.

1
curl -sL -O storage.googleapis.com/flow-genesis-bootstrap/boot-tools.tar
2
tar -xvf boot-tools.tar
1
sha256sum ./boot-tools/bootstrap
2
7ecab6488ddde7ef248f129080702214da6aea379ba464cf3226c7457a25cf9b ./boot-tools/bootstrap
3
4
sha256sum ./boot-tools/transit
5
c519c919f652f97a42b82cc649802c4ad1daabc08be25061287d9dea22cb4446 ./boot-tools/transit

Generate Your Node Keys

Network Address

Use a fully qualified domain name for the network address. Please also include the port number in the network address e.g. example.com:3569

Do not include in http:// format.

If you are running multiple nodes, please ensure you have different addresses for each node.

All your current keys and Flow genesis files should be in the bootstrap folder created earlier. Please take a back up of the entire folder.

1
## Skip this section if this is your first time ##
2
# If you joined our network previously, make sure to take a backup of your previously generated keys!
3
cp -r /path/to/bootstrap /path/to/bootstrap.bak
4
#########################################################
5
# Generate Keys
6
$ mkdir ./bootstrap
7
# YOUR_NODE_ADDRESS: FQDN associated to your instance
8
# YOUR_NODE_ROLE: The Flow nodes that you wish to run, it should be ONE of the following - [ access, collection, consensus, execution, verification ]
9
$ ./boot-tools/bootstrap key --address \"${YOUR_NODE_ADDRESS}:3569\" --role ${YOUR_NODE_ROLE} -o ./bootstrap
1
$./boot-tools/bootstrap key --address "consensus-001.nodes.flow.com:3569" --role consensus -o ./bootstrap
2
<nil> DBG will generate networking key
3
<nil> INF generated networking key
4
<nil> DBG will generate staking key
5
<nil> INF generated staking key
6
<nil> DBG will generate db encryption key
7
<nil> INF generated db encryption key
8
<nil> DBG assembling node information address=consensus-001.nodes.flow.com:3569
9
<nil> DBG encoded public staking and network keys networkPubKey=7f31ae79017a2a58a5e59af9184f440d08885a16614b2c4e361019fa72a9a1a42bf85b4e3f9674782f12ca06afd9782e9ccf19496baed069139385b82f8f40f6 stakingPubKey=829d086b292d84de8e7938fd2fafa8f51a6e025f429291835c20e59d9e25665febf24fa59de12a4df08be7e82c5413180cc7b1c73e01f26e05344506aaca4fa9cc009dc1c33f8ba3d7c7509e86d3d3e7341b43b9bf80bb9fba56ae0b3135dd72
10
<nil> INF wrote file bootstrap/public-root-information/node-id
11
<nil> INF wrote file bootstrap/private-root-information/private-node-info_ab6e0b15837de7e5261777cb65665b318cf3f94492dde27c1ea13830e989bbf9/node-info.priv.json
12
<nil> INF wrote file bootstrap/private-root-information/private-node-info_5e44ad5598bb0acb44784f629e84000ffea34d5552427247d9008ccf147fb87f/secretsdb-key
13
<nil> INF wrote file bootstrap/public-root-information/node-info.pub.ab6e0b15837de7e5261777cb65665b318cf3f94492dde27c1ea13830e989bbf9.json
14
<nil> DBG will generate machine account key
15
<nil> INF generated machine account key
16
<nil> DBG assembling machine account information address=consensus-001.nodes.flow.com:3569
17
<nil> INF encoded machine account public key for entry to Flow Port machineAccountPubKey=f847b8406e8969b869014cd1684770a8db02d01621dd1846cdf42fc2bca3444d2d55fe7abf740c548639cc8451bcae0cd6a489e6ff59bb6b38c2cfb83e095e81035e507b02038203e8
18
<nil> INF wrote file bootstrap/private-root-information/private-node-info_ab6e0b15837de7e5261777cb65665b318cf3f94492dde27c1ea13830e989bbf9/node-machine-account-key.priv.json
19
20
$tree ./bootstrap/
21
./bootstrap
22
├── private-root-information
23
│ └── private-node-info_ab6e0b15837de7e5261777cb65665b318cf3f94492dde27c1ea13830e989bbf9
24
│ ├── node-info.priv.json
25
│ ├── node-machine-account-key.priv.json
26
│ └── secretsdb-key
27
└── public-root-information
28
├── node-id
29
└── node-info.pub.ab6e0b15837de7e5261777cb65665b318cf3f94492dde27c1ea13830e989bbf9.json
30
31
3 directories, 4 files

For consensus and collection node types an additional key will be created for the Machine Account. For all other node types this will not be needed.

Machine Account Creation

If you are running a collection and consensus node, you will have an additional private key file (node-machine-account-key.priv.json) which contains the private key for your node's machine account. You can learn more about machine accounts here.

In Step 2 of this guide, when you submit a transaction to stake your node, you will need to provide the machine account public key, which can be found in the output of the previous bootstrap key command.

1
$./boot-tools/bootstrap key --address ${YOUR_NODE_ADDRESS} --role ${YOUR_NODE_ROLE} -o ./bootstrap
2
...
3
<nil> DBG encoded public machine account key machineAccountPubKey=1b9c00e6f0930792c5738d3397169f8a592416f334cf11e84e6327b98691f2b72158b40886a4c3663696f96cd15bfb5a08730e529f62a00c78e2405013a6016d
4
<nil> INF wrote file bootstrap/private-root-information/private-node-info_ab6e0b15837de7e5261777cb65665b318cf3f94492dde27c1ea13830e989bbf9/node-machine-account-key.priv.json

Copy the machine account public key somewhere safe. You will need it in a later step.

Step 2 - Stake Your Node

Stake your node via Flow Port

The node details (Node ID, Network Address, Networking Key and Staking Key) that need to be submitted when staking the node on Flow Port, can be found in the file: ./bootstrap/public-root-information/node-info.pub.<node-id>.json.

1
$cat ./bootstrap/public-root-information/node-info.pub.39fa54984b8eaa463e129919464f61c8cec3a4389478df79c44eb9bfbf30799a.json
2
{
3
"Role": "consensus",
4
"Address": "consensus-001.nodes.flow.com:3569",
5
"NodeID": "39fa54984b8eaa463e129919464f61c8cec3a4389478df79c44eb9bfbf30799a",
6
"Weight": 1000,
7
"NetworkPubKey": "d92e3d5880abe233cf9fe9104db34bbb31251468a541454722b3870c04156a1b0504aef443bcaad124b997384b8fe7052847ce1e6189af1392d865e6be69835b",
8
"StakingPubKey": "917826e018f056a00b778a58ae83054906957ffd4b6f1b7da083551f7a9f35e02b76ace50424ed7d2c9fc69207a59f0f08a031048f5641db94e77d0648b24d150dedd54bab7cd44b4aa60cfd54be418647b0b3965f8ae54c0bcb48ae9d705162"
9
}

If you are running a collection or consensus node, you will need to provide an additional field Machine Account Public Key. This value is found in the output of the bootstrap key command from Step 1.

Please let us know your node id via discord or email.

Finalize Machine Account Setup

If you are not running a collection or consensus node, you can skip this step.

You will now need to use the bootstrap utility to run machine-account with the created address to finalize the set up of your Machine account.

1
$ ./boot-tools/bootstrap machine-account --address ${YOUR_MACHINE_ACCOUNT_ADDRESS} -o ./bootstrap
1
$ ./boot-tools/bootstrap machine-account --address 0x1de23de44985c7e7 -o ./bootstrap
2
<nil> INF read machine account private key json
3
<nil> DBG encoded public machine account key machineAccountPubKey=2743786d1ff1bf7d7026d693a774210eaa54728343859baab62e2df7f71a370651f4c7fd239d07af170e484eedd4f3c2df47103f6c39baf2eb2a50f67bbcba6a
4
<nil> INF wrote file bootstrap/private-root-information/private-node-info_6f6e98c983dbd9aa69320452949b81abeab2ac591a247f55f19f4dbf0b477d26/node-machine-account-info.priv.json
5
6
$tree ./bootstrap/
7
./bootstrap
8
├── private-root-information
9
│ └── private-node-info_d60bd55ee616c5c297cae1d5cfb7f65e7e04014d9c4abe595af2fd83f3cfe160
10
│ ├── node-info.priv.json
11
│ ├── node-machine-account-info.priv.json
12
│ ├── node-machine-account-key.priv.json
13
│ └── secretsdb-key
14
└── public-root-information
15
├── node-id
16
└── node-info.pub.d60bd55ee616c5c297cae1d5cfb7f65e7e04014d9c4abe595af2fd83f3cfe160.json
17
18
3 directories, 5 files

After running this step, you should see the node-machine-account-info.priv.json file in your bootstrap directory as shown above.

Verify Machine Account Setup

After finalizing your machine account setup, you should verify its correctness with the check-machine-account command:

1
$ ./boot-tools/bootstrap check-machine-account --access-address access.mainnet.nodes.onflow.org:9000 -o ./bootstrap
2
<nil> DBG read machine account info from disk hash_algo=SHA3_256 key_index=0 machine_account_address=0x284463aa6e25877c machine_account_pub_key=f847b84051bad4512101640772bf5e05e8a49868d92eaf9ebed41030881d95485769afd28653c5c53216cdcda4554384bb3ff6396a2ac04842422d55f0562496ad8d952802038203e8 signing_algo=ECDSA_P256
3
<nil> DBG checking machine account configuration... machine_account_address=0x284463aa6e25877c role=consensus
4
<nil> DBG machine account balance: 0.10000000
5
<nil> INF 🤖 machine account is configured correctly

This command will detect and provide information about common misconfigurations, or confirm that the machine account is configured correctly.

Push transit keys (consensus node only)

If you are running a consensus node, run the following command to push the transit keys. Use the token mainnet-x-organization e.g. mainnet-17-companyname.

1
$ ./boot-tools/transit push-transit-key -b ./bootstrap -t mainnet-x-organization
2
<nil> INF generating transit keys
3
<nil> INF generating key pair
4
<nil> INF attempting to push transit public key to the transit servers
5
<nil> INF successfully pushed transit public key to the transit servers

Step 3 - Start Your Flow Node

Ensure you have configured your node using the Node Setup guide.

Confirming authorization

You can confirm your node's successful registration and authorization by executing a Cadence script to query the Staking Contract. At the end of the Staking Auction Phase, the members of the Proposed Identity Table are confirmed as authorized participants in the next epoch. Therefore, if your node ID appears in the Proposed Identity Table during the Staking Auction Phase, your node will be a participant in the next epoch.

You can read the current Proposed Identity Table using the getProposedTable script.

You can read the current epoch phase using the getEpochPhase script. (A return value of 0 indicates the Staking Auction Phase.)

Trusted Root Snapshot

Once your node has been registered and authorized by the service account, it will be able to participate in the next epoch.

Flow Epoch Schedule

A new node must bootstrap with a trusted root snapshot of the protocol state, where the node is a confirmed participant. Since new nodes are confirmed at the end of the Staking Auction Phase, this means that, if the node is registered to join at epoch N+1, it must use a root snapshot from within the Epoch Setup Phase of epoch N.

Dynamic Startup

Flow provides a mechanism called Dynamic Startup to simplify the process of obtaining the root snapshot. When using Dynamic Startup, the node can be started at any time during the Staking Auction Phase. The node will wait for the Epoch Setup Phase to begin, retrieve a valid root snapshot from a trusted Access Node, then bootstrap its state and join the network. This is the recommended way to start your node for the first time.

  1. Remove any root-protocol-state-snapshot.json file from your bootstrap folder. (If this file is present the node will attempt to bootstrap with it rather than Dynamic Startup.)
  2. Select a trusted Access Node to provide the root snapshot. You will need this node's secure GRPC server address and Networking Public Key.
  3. Configure Dynamic Startup by adding flags:
1
... \
2
--dynamic-startup-access-address=secure.mainnet.nodes.onflow.org:9001 \
3
--dynamic-startup-access-publickey=28a0d9edd0de3f15866dfe4aea1560c4504fe313fc6ca3f63a63e4f98d0e295144692a58ebe7f7894349198613f65b2d960abf99ec2625e247b1c78ba5bf2eae
  1. Start your node (see guide)

Once the node has bootstrapped, these flags may be removed.

Manually Provisioned Root Snapshot

You can also provision the root snapshot file manually, then start the node without configuring Dynamic Startup. A snapshot can be obtained from any Access Node using Flow CLI.

1
flow snapshot save /path/to/root/snapshot --network mainnet

The snapshot must be within the Epoch Setup Phase.

Since Collection and Consensus Nodes must start up in the first ~30mins of the Epoch Setup Phase (see Timing), the snapshot must be provisioned within this time window.

Once a valid root snapshot file is downloaded to the node's bootstrap folder, it can be started (see guide)