Alpha Release

This is an alpha release of the EOSIO History Tools. It includes database fillers (fill-pg, fill-rocksdb) which pull data from nodeos's State History Plugin, and a new query engine (wasm-ql-pg, wasm-ql-rocksdb) which supports queries defined by wasm, along with an emulation of the legacy /v1/ RPC API.

This alpha release is designed to solicit community feedback. There are several potential directions this toolset may take; we'd like feedback on which direction(s) may be most useful. Please create issues about changes you'd like to see going forward.

Since this is an alpha release, it will likely have incompatible changes in the future. Some of these may be driven by community feedback.

This release includes the following:

Alpha 0.4.0

This release upgrades fill-pg to support nodeos v2.1.0. The remaining tools are still disabled and have not been upgraded. The support for RocksDB is likely to be dropped entirely now that nodeos can use it as a backend store. The schema used by fill-pg has changed. At present no data migration tool is available, so data may be manually migrated, regenerated by replaying from the desired block number, or exist side by side in two schemas, with older blocks in the v0.3.0 schema and newer blocks in the new schema. Use of the --pg-schema option will facilitate the transition. Schema differences include:

• addition of version numbers in column names for fields which are part of versioned structures, e.g. in the action_trace table, receipt_receiver is now receipt0_receiver.
• the addition of an action_trace_v1 table, where all new action traces are written
• new columns in tables with fields from versioned structures indicating which version is populated for the row, named with the suffix _variant_populated.
• addition of a key_value table for storing data from the new nodeos storage mechanism of the same name.

Full details of the differences can be found via diff of plain text backups of each schema using pg_dump, not included here for brevity.

Alpha 0.3.0

This release adds temporary workarounds to fill-pg to support Nodeos 2.0. It also disables the remaining tools. If you would like to test rocksdb support or wasm-ql support, stick with Nodeos 1.8 and the Alpha 0.2.0 release of History Tools.

• Temporary fill-pg fixes
  • Removed the global_property table
  • Removed new_producers from the block_info table
• Temporarily disabled building everything except fill-pg

Alpha 0.2.0

• There are now 2 self-contained demonstrations in public Docker images. See container-demos for details.
  • Talk: this demonstrates using wasm-ql to provide messages from on-chain conversations to clients in threaded order.
  • Partial history: this demonstrates some of wasm-ql's chain and token queries on data drawn from one of the public EOSIO networks.
• Added RocksDB and removed LMDB. This has the following advantages:
  • Filling outperforms both PostgreSQL and LMDB by considerable margins, both for partial history and for full history on large well-known chains.
  • Database size for full history is much smaller than PostgreSQL.
• Database fillers have a new option --fill-trx to filter transaction traces.
• Database fillers no longer need --fill-skip-to when starting from partial history.
• Database fillers now automatically reconnect to the State History Plugin.
• wasm-ql now uses a thread pool to handle queries. --wql-threads controls the thread pool size.
• wasm-ql now uses eos-vm instead of SpiderMonkey. This simplifies the build process.
• wasm-ql can now serve static files. Enabled by the new --wql-static-dir option.
• SHiP connection handling moved to state_history_connection.hpp. This file may aid users needing to write custom solutions which connect to the State History Plugin.