Proof-of-concept implementation of ASN.1 mapping #62

2026-07-07T17:07:03+09:00

uwiger commented

2026-07-07 17:07:03 +09:00

This branch contains a proof-of-concept exploration of representing
the RLP (gmser_rlp) and gmserialization layers using ASN.1 with
DER encoding.

Goals of the experiment:

Model the static serialization templates and dynamic types in ASN.1.
Demonstrate reliable detection of legacy RLP data vs. new DER data
(first byte >= 0xC0 for RLP lists vs. 0x30 for DER SEQUENCE).
Evaluate wire-size overhead of standard DER TLV encoding compared
to the extremely compact RLP prefix encoding.
Verify that the DER path produces deterministic output suitable
for hashing (idempotent serialization).

Key files:

GajumaruSerialization.asn -- the ASN.1 schema
GajumaruSerialization.{erl,hrl} -- generated encoder/decoder
detect_demo.erl, size_comparison.erl -- supporting test code

Findings (high level):

Detection via first byte works cleanly.
DER is fully deterministic for hashing when the same value is given.
Small objects pay a significant size overhead (often 3-6x) due to
TLV tags/lengths and field names in the generic path.
Concrete SEQUENCE definitions are much more compact than the
generic templateFields representation.
Large payloads have acceptable relative overhead.

This is NOT production code. It was created to answer feasibility
questions around ASN.1 modeling, format detection, determinism,
and size characteristics.

Branch created from uw-new-acs for isolated experimentation.

This branch contains a proof-of-concept exploration of representing the RLP (gmser_rlp) and gmserialization layers using ASN.1 with DER encoding. Goals of the experiment: - Model the static serialization templates and dynamic types in ASN.1. - Demonstrate reliable detection of legacy RLP data vs. new DER data (first byte >= 0xC0 for RLP lists vs. 0x30 for DER SEQUENCE). - Evaluate wire-size overhead of standard DER TLV encoding compared to the extremely compact RLP prefix encoding. - Verify that the DER path produces deterministic output suitable for hashing (idempotent serialization). Key files: - GajumaruSerialization.asn -- the ASN.1 schema - GajumaruSerialization.{erl,hrl} -- generated encoder/decoder - detect_demo.erl, size_comparison.erl -- supporting test code Findings (high level): - Detection via first byte works cleanly. - DER is fully deterministic for hashing when the same value is given. - Small objects pay a significant size overhead (often 3-6x) due to TLV tags/lengths and field names in the generic path. - Concrete SEQUENCE definitions are much more compact than the generic templateFields representation. - Large payloads have acceptable relative overhead. This is NOT production code. It was created to answer feasibility questions around ASN.1 modeling, format detection, determinism, and size characteristics. Branch created from uw-new-acs for isolated experimentation.

uwiger added 9 commits 2026-07-07 17:07:03 +09:00

Add support for extended-auth account types

Gajumaru Serialization Tests / tests (push) Successful in 11s

Details

7a8e840793

Add auth tx and account sig store

Gajumaru Serialization Tests / tests (push) Successful in 11s

Details

dbfc013c8a

Add missing tag for proposal_gossip_tx

Gajumaru Serialization Tests / tests (push) Successful in 11s

Details

a368c64f7e

Add is_account/1 and account_pubkey/1 helpers

Gajumaru Serialization Tests / tests (push) Successful in 11s

Details

a84dcc880d

Extended-auth account ids use an {account, Subtype} tag. These helpers
let callers test account membership and extract the 32-byte pubkey
without pattern-matching on specialize/1 results.

Add account_auth_update_tx tag

Gajumaru Serialization Tests / tests (push) Successful in 11s

Details

79b43b0665

Add ac_receipt and AC side tx tags from ac793dcaf6

Gajumaru Serialization Tests / tests (push) Successful in 12s

Details

faa20bd00b

Cherry-pick the chain-object tags from ac793dcaf6 while keeping the
extended-auth account types (account_sig_store, auth_tx, etc.).

Add ac_acct_state serialization tag

Gajumaru Serialization Tests / tests (push) Successful in 10s

Details

0e157f824a

Fix ac_acct_state tag and restore key_block mapping

Gajumaru Serialization Tests / tests (push) Successful in 11s

Details

c80505c810

The previous commit accidentally removed key_block (tag 100) instead of
adding ac_acct_state (tag 99).

PoC: ASN.1 / DER serialization experiment

Gajumaru Serialization Tests / tests (push) Successful in 10s

Details

6d7ab1e4ad

This branch contains a proof-of-concept exploration of representing
the RLP (gmser_rlp) and gmserialization layers using ASN.1 with
DER encoding.

Goals of the experiment:
- Model the static serialization templates and dynamic types in ASN.1.
- Demonstrate reliable detection of legacy RLP data vs. new DER data
  (first byte >= 0xC0 for RLP lists vs. 0x30 for DER SEQUENCE).
- Evaluate wire-size overhead of standard DER TLV encoding compared
  to the extremely compact RLP prefix encoding.
- Verify that the DER path produces deterministic output suitable
  for hashing (idempotent serialization).

Key files:
- GajumaruSerialization.asn   -- the ASN.1 schema
- GajumaruSerialization.{erl,hrl} -- generated encoder/decoder
- detect_demo.erl, size_comparison.erl -- supporting test code

Findings (high level):
- Detection via first byte works cleanly.
- DER is fully deterministic for hashing when the same value is given.
- Small objects pay a significant size overhead (often 3-6x) due to
  TLV tags/lengths and field names in the generic path.
- Concrete SEQUENCE definitions are much more compact than the
  generic templateFields representation.
- Large payloads have acceptable relative overhead.

This is NOT production code. It was created to answer feasibility
questions around ASN.1 modeling, format detection, determinism,
and size characteristics.

Branch created from uw-new-acs for isolated experimentation.

uwiger requested review from zxq9 2026-07-07 17:07:03 +09:00

uwiger added 1 commit 2026-07-07 17:50:31 +09:00

Add thin ASN.1-driven RLP production layer + equivalence tests

Gajumaru Serialization Tests / tests (push) Successful in 10s

Details

9817af8a46

Implements gmser_asn1_rlp:encode/1 that accepts terms shaped according
to the GajumaruSerialization.asn model (e.g. GajumaruData with
templateFields or concrete CHOICEs like signedTx) and emits exactly
the same RLP bytes as the legacy gmserialization + gmser_rlp stack.

Purpose:
- Use ASN.1 as a formal, multi-language friendly schema and type model.
- Provide a thin, portable "production layer" that other languages can
  implement (ASN.1-generated types + equivalent RLP emitter).
- Preserve the existing compact RLP wire format and on-the-wire
  equivalence (no DER bloat for the legacy path).

The implementation walks the ASN.1-shaped value and applies the same
encoding rules as gmserialization:encode_field/2 (minimal unsigned
integers, positional values for static maps, etc.) followed by
gmser_rlp:encode/1.

Includes EUnit equivalence tests covering:
- Simple fields, zero/empty values
- Lists and tuples
- List-of-tuples (e.g. type_info)
- Concrete signedTx
- ContractV3 (including bool in structured data)

All tests assert byte-for-byte identity with legacy serialization and
that the resulting bytes are still accepted by legacy decoders.

This continues the proof-of-concept on the uw-asn1 branch.

uwiger added 1 commit 2026-07-08 17:53:43 +09:00

Updated asn1 experiment, now exploring PER and OER

Gajumaru Serialization Tests / tests (push) Successful in 12s

Details

4036133476

Gajumaru Serialization Tests / tests (push) Successful in 12s

Details

This pull request can be merged automatically.

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin uw-asn1:uw-asn1

git checkout uw-asn1

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: QPQ-AG/gmserialization#62