Skip to main content

Use Molecule Serialization System

What is the UnsignedTx.json file used for?

If you open UnsignedTx.json file, you will find a piece of data with 0x prefix. This is the data generated by the serialization of UnsignedTransaction code.This use case could be used in a place where the highest level security is required when signing transactions offline.

You can use a tool called generate-message-tool to deserialize the data,then generate message enventually. If this message is similar with the signing message replyed by felix bot, see Sign the transaction offline , then it's proved that there is no problem with the offline signing process.

Serialization and deserialization are very common functions that used for network transfer and data storage.

You should use Molecule to implement the serialization and deserialization process. Molecule, as a widely used data structure in CKB, has its unique property that memory consumption could be minimized, see RFC:Serialization.

In this section, you will learn about the molecule format, the molecule serialization implementation and molecule serialization implementation.

molecule
Figure 9 molecule serialization implementation and molecule serialization implementation.

Use the molecule serialization implementation#

Create a schema file#

The UnsignedTx is described via the following molecule format data structure, see RFC: Serialization for more information about molecule format. You can find the complete schema file here.

/felix/schema/UnsignedTransaction.mol
struct SighashAllSigning {    signing_script: Script,}

union SigningMethods {    SighashAllSigning,}
table UnsignedTransaction {    signing_method: SigningMethods,    tx: Transaction,    input_txs: TransactionVec,    cell_dep_txs: TransactionVec,    headers: HeaderVec,}

The UnsignedTx object inclueds the following objects:

  • signing_method: SighashAllSigning is the default signing solution used in CKB now,see Sign the transfer transaction.
  • tx: The complete transfer transaction.
  • input_txs: The input cell is not enough to validate, it is required to display the full transaction for validating the correctness of the input cell.
  • cell_dep_txs: The same as input_txs, the cell_dep is not enough to validate, it is required to display the full transaction.
  • headers: The block header which includes input_txs.

Compile the schema#

Moleculec-es is a ECMAScript plugin for the molecule serialization system. Use molecule-es to compile the schema and generate the javascript file to use.You can find the compiled files here. The finalized file UnsignedTransaction.umd.js includes molecule serialization implementation and molecule deserialization implementation.

$ git clone https://github.com/nervosnetwork/moleculec-es.git$ cd moleculec-es$ cargo install moleculec$ moleculec --language - --schema-file "your schema file" --format json > /tmp/schema.json$ moleculec-es -hasBigInt -inputFile /tmp/schema.json -outputFile "your JS file"$ rollup -f umd -n bundle -i UnsignedTransaction.js -o UnsignedTransaction.umd.js // convert `esm` format to `umd` format

Convert the plain JavaScript object#

The plain JavaScript object should be converted to another JavaScript object which can be serialized by molecule serialization implementation.

The normalizer function can convert a plain JavaScript object which can be validated by validator function to another JavaScript object which can be serialized into serialized ArrayBuffer data in molecule format，For each CKB data structure, there is a corresponding normalizer function, see normalizers.

/felix/lib/server.js
const { Reader,normalizers } = require("ckb-js-toolkit");
const SighashAllSigning = {    signing_script: normalizers.NormalizeScript(tx.outputs[0].lock)   }   const signing_method =    {type: "SighashAllSigning",    value: SighashAllSigning}
   const Unsignedtx = Object();
   Unsignedtx.signing_method = signing_method;      Unsignedtx.tx = normalizers.NormalizeTransaction(tx);
    const INPUT_TX_HASH = tx.inputs[0].previous_output.tx_hash;   const input_txs = (await rpc.get_transaction(INPUT_TX_HASH)).transaction;
    const CELL_DEP_TX_HASH = tx.cell_deps[0].out_point.tx_hash;   const cell_dep_txs = (await rpc.get_transaction(CELL_DEP_TX_HASH)).transaction
   const txstatus = (await rpc.get_transaction(INPUT_TX_HASH)).tx_status;   const headers = (await rpc.get_block(txstatus.block_hash)).header;
   const normalizedinput_txs = normalizers.NormalizeTransaction(input_txs);    Unsignedtx.input_txs = new Array(normalizedinput_txs);    const normalizedcell_dep_txs = normalizers.NormalizeTransaction(cell_dep_txs);   Unsignedtx.cell_dep_txs = new Array(normalizedcell_dep_txs);     const normalizedheaders = normalizers.NormalizeHeader(headers);   Unsignedtx.headers = new Array(normalizedheaders);

Generate serialized ArrayBuffer data in molecule format#

Use UnsignedTransaction function in UnsignedTransaction.umd.js to generate serialized ArrayBuffer data, then use Reader class in CKB-JS-Toolkit to convert the result's format.

/felix/lib/server.js
const UnsignedTransaction = require ("../schema/UnsignedTransaction.umd.js");const serializedUnsignedTx = new Reader(    UnsignedTransaction.SerializeUnsignedTransaction(Unsignedtx)    ).serializeJson();

Reply the serialized data#

Reply the UnsignedTx.json file.

/felix/lib/server.js
const readable = toStream(Buffer.from(serializedUnsignedTx));
const writerStream = fs.createWriteStream('UnsignedTx.json');readable.pipe(writerStream);
reply.document(fs.createReadStream('UnsignedTx.json'));

Use the molecule deserialization implementation#

There is a tool called generate-message-tool which displayed how to use the molecule deserialization implementation. Use the tool to deserialize UnsignedTx.json, restore the previous transfer transaction, generate the txSkeleton and finally use common.prepareSigningEntries(txSkeleton) to generate message.

Project Structure#

Clone and open the project, put the UnsignedTx.json in it, you will see the following files:

$ git clone https://github.com/zengbing15/generate-message-tool.git$ cd generate-message-tool
generate-message-toolβ”œβ”€β”€ binβ”‚   └── index.jsβ”œβ”€β”€ schemaβ”‚   β”œβ”€β”€ UnsignedTransaction.molβ”‚   β”œβ”€β”€ UnsignedTransaction.jsonβ”‚   β”œβ”€β”€ UnsignedTransaction.jsβ”‚   └── UnsignedTransaction.umd.jsβ”œβ”€β”€ package.jsonβ”œβ”€β”€ UnsignedTx.jsonβ”œβ”€β”€ .gitignore└── README.md

Use molecule deserialization implementation#

A transaction object includes the following objects: see A transfer transaction on CKB Testnet

  • version
  • cell_deps
  • header_deps
  • inputs
  • outputs
  • outputs_data
  • witnesses

Use UnsignedTransaction.umd.js to deserialize the ArrayBuffer data and generate the transaction object,the corresponding interfaces are exposed with exports in the file.

/generate-message-tool/schema/UnsignedTransaction.umd.js
  exports.Block = Block;  exports.Byte32 = Byte32;  exports.Byte32Vec = Byte32Vec;  exports.Bytes = Bytes;  ......

Then call the corresponding getXXX() method to deserialize the result, by the way, use Reader class to convert format.

/generate-message-tool/bin/index.js
const UnsignedTransaction = require ("../schema/UnsignedTransaction.umd.js");
/* Read UnsignedTx.json file */
let rawdata = fs.readFileSync('UnsignedTx.json');let unsignedtx = rawdata.toString();const wholetx = new Object();const UnsignedTx = new UnsignedTransaction.UnsignedTransaction(new Reader(unsignedtx));
const tx = UnsignedTx.getTx();
// versionwholetx.version = "0x"+tx.getRaw().getVersion().toBigEndianUint32().toString(16);
// cell_depsconst cellDeps_arraybuffer = new Array();for ( var i=0; i < tx.getRaw().getCellDeps().length(); i++){  cellDeps_arraybuffer.push({    "out_point":{      "tx_hash":tx.getRaw().getCellDeps().indexAt(i).getOutPoint().getTxHash().raw(),      "index":tx.getRaw().getCellDeps().indexAt(i).getOutPoint().getIndex()    },    "dep_type":tx.getRaw().getCellDeps().indexAt(i).getDepType()    });  }  // "dep_type" = uint8(1) means that "dep_type" is "dep_group"wholetx.cell_deps = new Array();for ( var i=0; i < tx.getRaw().getCellDeps().length(); i++){  wholetx.cell_deps.push({    "out_point":{      "tx_hash":"0x"+ Buffer.from(cellDeps_arraybuffer[i].out_point.tx_hash).toString("hex"),      "index":"0x"+cellDeps_arraybuffer[i].out_point.index.toBigEndianUint32().toString(16)    },    "dep_type":"dep_group"   });  }
for ( var i=0; i < tx.getRaw().getHeaderDeps().length(); i++){  outputsData_arraybuffer.push(tx.getRaw().getHeaderDeps().indexAt(i).raw());   }// Because headerDeps_arraybuffer = []wholetx.header_deps = [];
// inputsconst inputs_arraybuffer = new Array();for ( var i=0; i < tx.getRaw().getInputs().length(); i++){  inputs_arraybuffer.push({    "since":tx.getRaw().getInputs().indexAt(i).getSince().raw(),    "previous_output":{      "tx_hash":tx.getRaw().getInputs().indexAt(i).getPreviousOutput().getTxHash().raw(),      "index":tx.getRaw().getInputs().indexAt(i).getPreviousOutput().getIndex()    },     });  }  wholetx.inputs = new Array();for ( var i=0; i < tx.getRaw().getInputs().length(); i++){  wholetx.inputs.push({    "since":"0x"+Buffer.from(inputs_arraybuffer[i].since).toString("hex"),    "previous_output":{      "tx_hash":"0x"+Buffer.from(inputs_arraybuffer[i].previous_output.tx_hash).toString('hex'),      "index":"0x"+inputs_arraybuffer[i].previous_output.index.toLittleEndianUint32().toString(16)    },   });  }  //outputs  const outputs_arraybuffer = new Array();for ( var i=0; i < tx.getRaw().getOutputs().length(); i++){  outputs_arraybuffer.push({    "capacity":tx.getRaw().getOutputs().indexAt(i).getCapacity().toLittleEndianBigUint64(),    "lock": {      "code_hash":tx.getRaw().getOutputs().indexAt(i).getLock().getCodeHash().raw(),      "hash_type":tx.getRaw().getOutputs().indexAt(i).getLock().getHashType(),      "args":tx.getRaw().getOutputs().indexAt(i).getLock().getArgs().raw()    },   });  }  wholetx.outputs = new Array();for ( var i=0; i < tx.getRaw().getOutputs().length(); i++){  wholetx.outputs.push({    "capacity":"0x"+ outputs_arraybuffer[i].capacity.toString(16),    "lock": {      "code_hash":"0x"+Buffer.from(outputs_arraybuffer[i].lock.code_hash).toString("hex"),      "hash_type":"type",      "args":"0x"+Buffer.from(outputs_arraybuffer[i].lock.args).toString("hex")    },   });  }
//outputs_data  const outputsData_arraybuffer = new Array();for(var i=0; i < tx.getRaw().getOutputsData().length(); i++){  outputsData_arraybuffer.push(tx.getRaw().getOutputsData().indexAt(i).raw());   }wholetx.outputs_data = new Array()for(var i=0; i < tx.getRaw().getOutputsData().length(); i++){  wholetx.outputs_data.push("0x"+Buffer.from(outputsData_arraybuffer[i]).toString("hex"));   }
//witnessesconst witness_arraybuffer =  new Array();for(var i=0; i < tx.getWitnesses().length(); i++){  witness_arraybuffer.push(tx.getWitnesses().indexAt(i).raw());   }wholetx.witnesses = new Array()for(var i=0; i < tx.getWitnesses().length(); i++){wholetx.witnesses.push("0x"+Buffer.from(witness_arraybuffer[i]).toString("hex"));}
console.log(JSON.stringify(wholetx,null,2));

Generate the signing message#

You can use common.prepareSigningEntries(txSkeleton) to generate the message, see the following steps:

  • Use transaction object to generate the txSkeleton object.
  • Use objectToTransactionSkeleton to convert txSkeleton object to TransactionSkeleton type
  • Use common.prepareSigningEntries(txSkeleton) to generate message
/generate-message-tool/bin/index.js
const {objectToTransactionSkeleton} = require("@ckb-lumos/helpers");async function main() {      const rpc = new RPC("http://localhost:8114");  const INPUT_TX_HASH = wholetx.inputs[0].previous_output.tx_hash;  const transaction = (await rpc.get_transaction(INPUT_TX_HASH)).transaction;  const txstatus = (await rpc.get_transaction(INPUT_TX_HASH)).tx_status;  const blockheader = (await rpc.get_block(txstatus.block_hash)).header;
      const obj = new Object();  obj.cellProvider = { indexer };  obj.cellDeps = transaction.cell_deps;  obj.headerDeps = transaction.header_deps;  obj.inputs = List([    { "cell_output": transaction.outputs[1],       "out_point": wholetx.inputs[0].previous_output,      "block_hash": txstatus.block_hash ,      "block_number": blockheader.number,       "data": transaction.outputs_data[1]}]);   obj.outputs = new Array();   for ( var i=0; i < wholetx.outputs.length; i++){    obj.outputs.push({ "cell_output": wholetx.outputs[i],"data":wholetx.outputs_data[i]});     }  // witness = {lock is 0, input_type is null, output_type is null}  obj.witnesses = List(["0x55000000100000005500000055000000410000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"]);  obj.fixedEntries = [];  obj.signingEntries = [];  obj.inputSinces = {};
  let txSkeleton = objectToTransactionSkeleton(obj);
  txSkeleton = common.prepareSigningEntries(txSkeleton);  const signingEntriesArray = txSkeleton.signingEntries.toArray();  console.log("The generated message is "+ signingEntriesArray[0].message);}
main();
info

Submit the minimal data#

Have you found that in the development process, almost all data is processed offline, and only the tx_hash is committed online?

Well, this is a important principle for development DApps on CKB layer1: Submit the minimal data as much as possible.

Only the truly important common knowledge needs the global consensus,the data that can be derived offline is completely unnecessary to submit.The online miminal data is able to ensure the certainty of offline data.So you can enjoy the benefits of a blockchain and at the same time avoid the performance disadvantages of the blockchain.