Compare commits

...

66 Commits

Author SHA1 Message Date
Ulf Wiger 4036133476 Updated asn1 experiment, now exploring PER and OER
Gajumaru Serialization Tests / tests (push) Successful in 12s
2026-07-08 10:53:31 +02:00
Ulf Wiger 9817af8a46 Add thin ASN.1-driven RLP production layer + equivalence tests
Gajumaru Serialization Tests / tests (push) Successful in 10s
Implements gmser_asn1_rlp:encode/1 that accepts terms shaped according
to the GajumaruSerialization.asn model (e.g. GajumaruData with
templateFields or concrete CHOICEs like signedTx) and emits exactly
the same RLP bytes as the legacy gmserialization + gmser_rlp stack.

Purpose:
- Use ASN.1 as a formal, multi-language friendly schema and type model.
- Provide a thin, portable "production layer" that other languages can
  implement (ASN.1-generated types + equivalent RLP emitter).
- Preserve the existing compact RLP wire format and on-the-wire
  equivalence (no DER bloat for the legacy path).

The implementation walks the ASN.1-shaped value and applies the same
encoding rules as gmserialization:encode_field/2 (minimal unsigned
integers, positional values for static maps, etc.) followed by
gmser_rlp:encode/1.

Includes EUnit equivalence tests covering:
- Simple fields, zero/empty values
- Lists and tuples
- List-of-tuples (e.g. type_info)
- Concrete signedTx
- ContractV3 (including bool in structured data)

All tests assert byte-for-byte identity with legacy serialization and
that the resulting bytes are still accepted by legacy decoders.

This continues the proof-of-concept on the uw-asn1 branch.
2026-07-07 10:48:30 +02:00
Ulf Wiger 6d7ab1e4ad PoC: ASN.1 / DER serialization experiment
Gajumaru Serialization Tests / tests (push) Successful in 10s
This branch contains a proof-of-concept exploration of representing
the RLP (gmser_rlp) and gmserialization layers using ASN.1 with
DER encoding.

Goals of the experiment:
- Model the static serialization templates and dynamic types in ASN.1.
- Demonstrate reliable detection of legacy RLP data vs. new DER data
  (first byte >= 0xC0 for RLP lists vs. 0x30 for DER SEQUENCE).
- Evaluate wire-size overhead of standard DER TLV encoding compared
  to the extremely compact RLP prefix encoding.
- Verify that the DER path produces deterministic output suitable
  for hashing (idempotent serialization).

Key files:
- GajumaruSerialization.asn   -- the ASN.1 schema
- GajumaruSerialization.{erl,hrl} -- generated encoder/decoder
- detect_demo.erl, size_comparison.erl -- supporting test code

Findings (high level):
- Detection via first byte works cleanly.
- DER is fully deterministic for hashing when the same value is given.
- Small objects pay a significant size overhead (often 3-6x) due to
  TLV tags/lengths and field names in the generic path.
- Concrete SEQUENCE definitions are much more compact than the
  generic templateFields representation.
- Large payloads have acceptable relative overhead.

This is NOT production code. It was created to answer feasibility
questions around ASN.1 modeling, format detection, determinism,
and size characteristics.

Branch created from uw-new-acs for isolated experimentation.
2026-07-07 10:05:30 +02:00
Ulf Wiger c80505c810 Fix ac_acct_state tag and restore key_block mapping
Gajumaru Serialization Tests / tests (push) Successful in 11s
The previous commit accidentally removed key_block (tag 100) instead of
adding ac_acct_state (tag 99).
2026-07-07 08:13:37 +02:00
Ulf Wiger 0e157f824a Add ac_acct_state serialization tag
Gajumaru Serialization Tests / tests (push) Successful in 10s
2026-07-07 08:07:58 +02:00
Ulf Wiger faa20bd00b Add ac_receipt and AC side tx tags from ac793dcaf6
Gajumaru Serialization Tests / tests (push) Successful in 12s
Cherry-pick the chain-object tags from ac793dcaf6 while keeping the
extended-auth account types (account_sig_store, auth_tx, etc.).
2026-07-06 22:27:28 +02:00
Ulf Wiger 79b43b0665 Add account_auth_update_tx tag
Gajumaru Serialization Tests / tests (push) Successful in 11s
2026-07-06 15:30:19 +02:00
Ulf Wiger a84dcc880d Add is_account/1 and account_pubkey/1 helpers
Gajumaru Serialization Tests / tests (push) Successful in 11s
Extended-auth account ids use an {account, Subtype} tag. These helpers
let callers test account membership and extract the 32-byte pubkey
without pattern-matching on specialize/1 results.
2026-07-06 12:26:38 +02:00
Ulf Wiger a368c64f7e Add missing tag for proposal_gossip_tx
Gajumaru Serialization Tests / tests (push) Successful in 11s
2026-07-06 10:55:20 +02:00
Ulf Wiger dbfc013c8a Add auth tx and account sig store
Gajumaru Serialization Tests / tests (push) Successful in 11s
2026-07-06 10:19:38 +02:00
Ulf Wiger 7a8e840793 Add support for extended-auth account types
Gajumaru Serialization Tests / tests (push) Successful in 11s
2026-07-05 08:38:42 +02:00
uwiger eecff86500 Merge pull request 'Add zompify support, verup to 0.2.0' (#59) from uw-zomp-vsn-0.2.0 into master
Gajumaru Serialization Tests / tests (push) Successful in -2m7s
Reviewed-on: #59
2026-04-17 14:25:34 +09:00
Ulf Wiger 2ac9363d30 Add zompify support, verup to 0.2.0
Gajumaru Serialization Tests / tests (push) Successful in -2m7s
2026-04-16 23:31:12 +02:00
uwiger 4698b54832 Merge pull request 'Add keypair encoding, fix seckey size checks' (#57) from uw-account_seckey into master
Gajumaru Serialization Tests / tests (push) Successful in -2m43s
Reviewed-on: #57
2026-03-30 05:26:09 +09:00
Ulf Wiger 4cc6adee2e Add unsafe_encode/2 to forego size check
Gajumaru Serialization Tests / tests (push) Successful in -2m49s
2026-03-25 22:10:06 +01:00
Ulf Wiger 4f97dd1bd1 Add keypair encoding, fix seckey size checks
Gajumaru Serialization Tests / tests (push) Successful in -2m51s
2026-03-24 23:06:29 +01:00
uwiger 05bbf058be Map 'hash' as known api type, add some more tests (#56)
Gajumaru Serialization Tests / tests (push) Successful in -2m53s
I noticed that the 'hash' serialization type wasn't in the `known_type()` set, although it was supported otherwise.
This PR adds it to the `known_type()` and also exports the type `gmser_api_encoder:known_type()`.

Also, some more tests are added to try to detect this sort of thing in the future.

Co-authored-by: Ulf Wiger <ulf@wiger.net>
Reviewed-on: #56
2026-03-24 21:23:20 +09:00
uwiger 2b24b17af3 Merge pull request 'Fix OTP 28 dialyzer warnings, rewrite gmser_dyn decoder' (#55) from uw-gmser_dyn-rewrite into master
Gajumaru Serialization Tests / tests (push) Successful in -2m54s
Reviewed-on: #55
2026-03-24 21:20:30 +09:00
Ulf Wiger 00699b08b7 Fix OTP 28 dialyzer warnings, rewrite gmser_dyn decoder
Gajumaru Serialization Tests / tests (push) Successful in -3m56s
gmser_dyn no longer tries to compress output by omitting type tags.
Decoding streams using custom template codes can either use 'strict'
decoding, in which case matching templates must be registered on the
decoding end; in `strict => false` mode, the stream can still be decoded
without valudation if the custom template is missing.
2026-02-18 21:10:43 +01:00
Ulf Wiger 8734e67c66 WIP refactor gmser_dyn 2026-02-15 12:12:04 +01:00
uwiger dda5cac7a9 Merge pull request 'Serialization got broken by previous PR' (#54) from uw-fix-serialization into master
Gajumaru Serialization Tests / tests (push) Successful in 49m58s
Reviewed-on: #54
2025-04-29 17:22:22 +09:00
Ulf Wiger 07d61722b4 Serialization got broken by previous PR
Gajumaru Serialization Tests / tests (push) Successful in 49m58s
2025-04-29 10:18:59 +02:00
uwiger 4ac7531351 Merge pull request 'uw-switch-semantics' (#53) from uw-switch-semantics into master
Gajumaru Serialization Tests / tests (push) Successful in 49m56s
Reviewed-on: #53
2025-04-29 03:57:16 +09:00
Ulf Wiger f996253e6b Add forgotten exports, expand(Types) function
Gajumaru Serialization Tests / tests (push) Successful in 49m55s
2025-04-28 12:12:43 +02:00
Ulf Wiger b9a51acf55 Add gmser_dyn_types.erl
Gajumaru Serialization Tests / tests (push) Successful in 49m54s
2025-04-28 11:59:27 +02:00
Ulf Wiger 5df23c05c1 test case for 'switch'
Gajumaru Serialization Tests / tests (push) Failing after 49m55s
2025-04-28 11:51:23 +02:00
Ulf Wiger b358dfe914 Add switch semantics
Gajumaru Serialization Tests / tests (push) Failing after 49m54s
2025-04-28 11:36:02 +02:00
uwiger 0288719ae1 Merge pull request 'Save options, test cases for missing labels' (#52) from uw-save-options into master
Gajumaru Serialization Tests / tests (push) Successful in 49m42s
Reviewed-on: #52
2025-04-24 06:46:26 +09:00
Ulf Wiger 795c7f7860 Save options, test cases for missing labels
Gajumaru Serialization Tests / tests (push) Successful in 49m42s
2025-04-23 23:36:03 +02:00
uwiger 0d77ca0388 Fix function_clause bug (#51)
Gajumaru Serialization Tests / tests (push) Successful in 49m16s
Co-authored-by: Ulf Wiger <ulf@wiger.net>
Reviewed-on: #51
2025-04-14 19:10:58 +09:00
uwiger ed204f8526 Merge pull request 'uw-dyn-options' (#50) from uw-dyn-options into master
Gajumaru Serialization Tests / tests (push) Failing after 49m16s
Reviewed-on: #50
2025-04-14 18:59:38 +09:00
Ulf Wiger a949d166f6 Add options for deserialization of missing labels
Gajumaru Serialization Tests / tests (push) Failing after 49m16s
2025-04-14 11:54:48 +02:00
Ulf Wiger ba771836fb Document static, make anyint standard
Gajumaru Serialization Tests / tests (push) Successful in 49m9s
2025-04-11 16:39:06 +02:00
uwiger c403fa89a1 Merge pull request 'Fix dynamic encoding type tag emission. Support 'negint', 'alt', 'items'' (#48) from uw-dynamic-encoding2 into master
Gajumaru Serialization Tests / tests (push) Successful in 49m5s
Reviewed-on: #48
2025-04-11 00:01:54 +09:00
Ulf Wiger dd3e731480 Support 'negint', 'items' and 'alt'
Gajumaru Serialization Tests / tests (push) Successful in 49m5s
2025-04-10 16:58:03 +02:00
Ulf Wiger 6563ef9de7 WIP add 'items', fix some layout issues 2025-04-10 16:58:03 +02:00
uwiger bff07885fb Merge pull request 'Dynamic serialization using gmser_dyn' (#47) from uw-dynamic-encoding into master
Gajumaru Serialization Tests / tests (push) Successful in 48m57s
Reviewed-on: #47
2025-04-07 18:31:05 +09:00
Ulf Wiger dd1c2455f0 Fix type-driven encode, more docs
Gajumaru Serialization Tests / tests (push) Successful in 48m53s
2025-04-05 21:44:36 +02:00
Ulf Wiger 3ede4f22e1 Register individual types, more docs 2025-04-05 13:20:30 +02:00
Ulf Wiger 4663a0f57e gmser_dyn.erl fairly complete
Gajumaru Serialization Tests / tests (push) Successful in 48m37s
2025-03-30 23:00:10 +02:00
zxq9 ac64e01b0f Update runner paths
Gajumaru Serialization Tests / tests (push) Successful in 47m16s
2025-02-04 15:58:21 +09:00
zxq9 9d2ecc00d3 Update zx dep
Gajumaru Serialization Tests / tests (push) Successful in -4m20s
2025-01-23 20:05:22 +09:00
zxq9 356d86cd72 Fix dep reference
Gajumaru Serialization Tests / tests (push) Successful in -4m21s
2025-01-23 20:01:15 +09:00
zxq9 f3da241917 zomp (#46)
Gajumaru Serialization Tests / tests (push) Failing after -4m25s
Reviewed-on: #46
Co-authored-by: Craig Everett <zxq9@zxq9.com>
Co-committed-by: Craig Everett <zxq9@zxq9.com>
2025-01-23 19:44:58 +09:00
zxq9 2db9ea6134 Package for zx (#45)
Gajumaru Serialization Tests / tests (push) Successful in -4m22s
This packages the library for deployment on zx (migrating from the old otpr-aeserialization-* line).

It also adds a native Erlang fallback to invokation of the Blake2 algorithm, picking enacl if it is present, but proceeding with eblake2 if not.

Reviewed-on: #45
Reviewed-by: dimitar.p.ivanov <dimitarivanov@qpq.swiss>
Reviewed-by: Ulf Wiger <ulfwiger@qpq.swiss>
Co-authored-by: Craig Everett <zxq9@zxq9.com>
Co-committed-by: Craig Everett <zxq9@zxq9.com>
2025-01-22 19:36:50 +09:00
zxq9 4b5cfcb67a Remove old type references, update naming, add license notices (#43)
Gajumaru Serialization Tests / tests (push) Successful in -4m21s
Removed the oracle type references and updated the dependency list to point to git.qpq.swiss.

Reviewed-on: #43
Co-authored-by: Craig Everett <zxq9@zxq9.com>
Co-committed-by: Craig Everett <zxq9@zxq9.com>
2025-01-22 13:22:03 +09:00
Ulf Wiger d5eb3ad0fb Merge branch 'uw-add-contract-source' into 'master'
Add contract source (cx)

See merge request ioecs/aeserialization!27
2024-10-30 16:02:22 +00:00
Ulf Wiger d28e63693c Add contract source (cx) 2024-10-15 08:07:24 +02:00
Ulf Wiger f4999b6c2f Merge branch 'uw-add-delegates-obj' into 'master'
Define channel_delegates chain obj

See merge request ioecs/aeserialization!26
2024-10-14 13:11:19 +00:00
Ulf Wiger b5929e4d2e Define channel_delegates chain obj 2024-10-14 14:48:05 +02:00
Hans Svensson 664f1f0c44 Merge branch 'add_data_extend' into 'master'
Add data_extend_tx

See merge request ioecs/aeserialization!25
2024-09-24 08:19:22 +00:00
Hans Svensson 9cb67eeb98 Add data_extend_tx 2024-09-24 09:47:25 +02:00
Dimitar Ivanov 7457d027fd Merge branch 'add_entries' into 'master'
Add entries

See merge request ioecs/aeserialization!24
2024-07-26 11:50:00 +00:00
Dimitar Ivanov c31bcac249 Add entries 2024-07-26 14:39:47 +03:00
Ulf Wiger c0bf7b5b74 Merge branch 'uw-native_tokens' into 'master'
Add native_token encoding support

See merge request ioecs/aeserialization!23
2024-07-26 11:36:12 +00:00
Ulf Wiger 5139f130f5 Add native token chain obj info 2024-07-15 17:38:23 +02:00
Ulf Wiger 8336d8113c Add native_token encoding support 2024-07-15 16:39:04 +02:00
Hans Svensson db7a9f62f2 Merge branch 'add_testimonies' into 'master'
Add testimonies and AC serialization

See merge request ioecs/aeserialization!22
2024-06-28 13:59:06 +00:00
Hans Svensson 8346df41f2 Add testimonies and AC serialization 2024-06-28 13:59:06 +00:00
Hans Svensson 2488ff3978 Add structured delegation signatures (#21)
* Bump enacl dependency

* Add serialization of delegation signature data
2023-12-06 15:23:57 +01:00
Ulf Wiger da8308821c Merge pull request #19 from davidyuk/secret-key
Add serialization of account secret key
2023-11-13 16:13:54 +01:00
Denis Davidyuk 213589ad9f Add serialization of account secret key 2023-09-15 10:08:29 +04:00
Ulf Wiger 270ebecc57 Merge pull request #20 from aeternity/uw-encode-maps
Add #{items := list()} type for encode/decode
2023-08-18 14:40:49 +02:00
Ulf Wiger 893ebb69f6 Update src/aeserialization.erl with duplicates check
From code review

Co-authored-by: Radosław Rowicki <35342116+radrow@users.noreply.github.com>
2023-08-17 20:43:59 +02:00
Ulf Wiger 05c4f87619 Update src/aeserialization.erl with type comment
From code review

Co-authored-by: Radosław Rowicki <35342116+radrow@users.noreply.github.com>
2023-08-17 20:42:48 +02:00
Ulf Wiger 91d6159ebd Add #{items := list()} type for encode/decode 2023-08-17 13:04:06 +02:00
60 changed files with 15247 additions and 268 deletions
-37
View File
@@ -1,37 +0,0 @@
version: 2.1
executors:
aebuilder:
docker:
- image: aeternity/builder:bionic-otp23
user: builder
working_directory: ~/src
jobs:
build:
executor: aebuilder
steps:
- checkout
- restore_cache:
keys:
- dialyzer-cache-v1-{{ .Branch }}-{{ .Revision }}
- dialyzer-cache-v1-{{ .Branch }}-
- dialyzer-cache-v1-
- run:
name: Build
command: rebar3 compile
- run:
name: Static Analysis
command: rebar3 dialyzer
- run:
name: Eunit
command: rebar3 eunit
- run:
name: Common Tests
command: rebar3 ct
- save_cache:
key: dialyzer-cache-v1-{{ .Branch }}-{{ .Revision }}
paths:
- _build/default/rebar3_20.3.8_plt
- store_artifacts:
path: _build/test/logs
+14
View File
@@ -0,0 +1,14 @@
name: Gajumaru Serialization Tests
run-name: ${{ gitea.actor }} testing Gajumaru Serialization
on: [push, workflow_dispatch]
jobs:
tests:
runs-on: linux_amd64
steps:
- name: Check out repository code
uses: actions/checkout@v4
- name: eunit
run: |
. /home/act_runner/.erts/27.2.1/activate
./rebar3 eunit
+1 -1
View File
@@ -7,7 +7,7 @@ _*
*.swp *.swp
*.swo *.swo
.erlang.cookie .erlang.cookie
ebin ebin/*.beam
log log
erl_crash.dump erl_crash.dump
.rebar .rebar
+1
View File
@@ -0,0 +1 @@
{"src/*", [debug_info, {i, "include/"}, {outdir, "ebin/"}]}.
+1
View File
@@ -1,5 +1,6 @@
ISC License ISC License
Copyright (c) 2025, QPQ AG
Copyright (c) 2017, aeternity developers Copyright (c) 2017, aeternity developers
Permission to use, copy, modify, and/or distribute this software for any Permission to use, copy, modify, and/or distribute this software for any
+263 -5
View File
@@ -1,9 +1,267 @@
aeserialization # GM Serialization
=====
Serialization helpers for Aeternity node. Serialization helpers for the Gajumaru.
Build For an overview of the static serializer, see [this document](doc/static.md).
-----
## Build
$ rebar3 compile $ rebar3 compile
## Test
$ rebar3 eunit
## Dynamic encoding
The module `gmser_dyn` offers dynamic encoding support, encoding most 'regular'
Erlang data types into an internal RLP representation.
Main API:
* `encode(term()) -> iolist()`
* `encode_typed(template(), term()) -> iolist()`
* `decode(iolist()) -> term()`
* `serialize(term()) -> binary()`
* `serialize_typed(template(), term()) -> binary()`
* `deserialize(binary()) -> term()`
In the examples below, we use the `decode` functions, to illustrate
how the type information is represented. The fully serialized form is
produced by the `serialize` functions.
The basic types supported by the encoder are:
* `integer()` (`anyint`, code: 246)
* `neg_integer()` (`negint`, code: 247)
* `non_neg_integer()` (`int` , code: 248)
* `binary()` (`binary`, code: 249)
* `boolean()` (`bool` , code: 250)
* `list()` (`list` , code: 251)
* `map()` (`map` , code: 252)
* `tuple()` (`tuple` , code: 253)
* `gmser_id:id()` (`id` , code: 254)
* `atom()` (`label` , code: 255)
(The range of codes is chosen because the `gmser_chain_objects` codes
range from 10 to 200, and also to stay within 1 byte.)
When encoding `map` types, the map elements are first sorted.
When specifying a map type for template-driven encoding, use
the `#{items => [{Key, ValueType} | {opt, Key, ValueType}]}` construct.
The key names are included in the encoding, and are match against the item
specs during decoding. If the key names don't match, the decoding fails, unless
for an `{opt, K, V}` item, in which case that item spec is skipped.
```erlang
T = #{items => [{a,int},{opt,b,int},{c,int}]}
E1 = gmser_dyn:encode_typed(T, #{a => 1, b => 2, c => 3}) ->
[<<0>>,<<1>>,[<<252>>,
[[[<<255>>,<<97>>],[<<248>>,<<1>>]],
[[<<255>>,<<98>>],[<<248>>,<<2>>]],
[[<<255>>,<<99>>],[<<248>>,<<3>>]]]]]
E2 = gmser_dyn:encode_typed(T, #{a => 1, c => 3}) ->
[<<0>>,<<1>>,[<<252>>,
[[[<<255>>,<<97>>],[<<248>>,<<1>>]],
[[<<255>>,<<99>>],[<<248>>,<<3>>]]]]]
gmser_dyn:decode_typed(T,E2) ->
#{c => 3,a => 1}
```
## Labels
Labels correspond to (existing) atoms in Erlang.
Decoding of a label results in a call to `binary_to_existing_atom/2`, so will
fail if the corresponding atom does not already exist.
This behavior can be modified using the option `#{missing_labels => fail | create | convert}`,
where `fail` is the default, as described above, `convert` means that missing atoms are
converted to binaries, and `create` means that the atom is created dynamically.
The option can be passed e.g.:
```erlang
gmser_dyn:deserialize(Binary, gmser_dyn:set_opts(#{missing_labels => convert}))
```
or
```erlang
gmser_dyn:deserialize(Binary, gmser_dyn:set_opts(#{missing_labels => convert}, Types))
```
By calling `gmser_dyn:register_types/1`, after having added options to the type map,
the options can be made to take effect automatically.
It's possible to cache labels for more compact encoding.
Note that when caching labels, the same cache mapping needs to be used on the
decoder side.
Labels are encoded as `[<<255>>, << AtomToBinary/binary >>]`.
If a cached label is used, the encoding becomes `[<<255>, [Ix]]`, where
`Ix` is the integer-encoded index value of the cached label.
## Examples
Dynamically encoded objects have the basic structure `[<<0>>,V,Obj]`, where `V` is the
integer-coded version, and `Obj` is the top-level encoding on the form `[Tag,Data]`.
```erlang
E = fun(T) -> io:fwrite("~w~n", [gmser_dyn:encode(T)]) end.
E(17) -> [<<0>>,<<1>>,[<<248>>,<<17>>]]
E(<<"abc">>) -> [<<0>>,<<1>>,[<<249>>,<<97,98,99>>]]
E(true) -> [<<0>>,<<1>>,[<<250>>,<<1>>]]
E(false) -> [<<0>>,<<1>>,[<<250>>,<<0>>]]
E([1,2]) -> [<<0>>,<<1>>,[<<251>>,[[<<248>>,<<1>>],[<<248>>,<<2>>]]]]
E({1,2}) -> [<<0>>,<<1>>,[<<253>>,[[<<248>>,<<1>>],[<<248>>,<<2>>]]]]
E(#{a=>1, b=>2}) ->
[<<0>>,<<1>>,[<<252>>,[[[<<255>>,<<97>>],[<<248>>,<<1>>]],[[<<255>>,<<98>>],[<<248>>,<<2>>]]]]]
E(gmser_id:create(account,<<1:256>>)) ->
[<<0>>,<<1>>,[<<254>>,<<1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1>>]]
```
Note that tuples and list are encoded the same way, except for the initial type tag.
Maps are encoded as `[<Map>, [KV1, KV2, ...]]`, where `[KV1, KV2, ...]` is the sorted
list of key-value tuples from `map:to_list(Map)`, but with the `tuple` type tag omitted.
## Template-driven encoding
Templates can be provided to the encoder by either naming an already registered
type, or by passing a template directly. In both cases, the encoder will enforce
the type information in the template.
If the template has been registered, the encoder uses the registered type specification
to drive the encoding. The code of the registered template is embedded in the encoded
output:
```erlang
gmser_dyn:encode_typed({int,int,int}, {1,2,3}) ->
[<<0>>,<<1>>,[<<253>>,
[[<<248>>,<<1>>],[<<248>>,<<2>>],[<<248>>,<<3>>]]]]
Types = gmser_dyn_types:add_type(t3,1013,{int,int,int}).
gmser_dyn:encode_typed(t3, {1,2,3}, Types) ->
[<<0>>,<<1>>,[[<<3,245>>,<<253>>],
[[<<248>>,<<1>>],[<<248>>,<<2>>],[<<248>>,<<3>>]]]]
```
Note that the original `<<253>>` type code is wrapped as `[<<3,245>>,<<253>>]`,
where `<<3,245>>` corresponds to the custom code `1013`.
Using the default option `#{strict => true}`, the decoder will extract the custom
type spec, and validate the encoded data against it. If the custom code is missing,
the decoder aborts. Using `#{strict => false}`, the custom code is used if it exists,
but otherwise, it's ignored, and the encoded data is decoded using the dynamic type
info.
### Alternative types
The dynamic encoder supports a few additions to the `gmserialization` template
language: `any`, `#{list => Type}`, `#{alt => [AltTypes]}` and `#{switch => [AltTypes]}`.
#### `any`
The `any` type doesn't have an associated code, but enforces dynamic encoding.
#### `list`
The original list type notation expects a key-value list, e.g.
`[{name, binary}, {age, int}]`
```erlang
EL = gmser_dyn:encode_typed([{name,binary},{age,int}], [{name,<<"Ulf">>},{age,29}]) ->
[<<0>>,<<1>>,[<<251>>,
[[<<253>>,[[<<255>>,<<110,97,109,101>>],[<<249>>,<<85,108,102>>]]],
[<<253>>,[[<<255>>,<<97,103,101>>],[<<248>>,<<29>>]]]]]]
```
Note that the encoding explicitly lays out a `[{Key, Value}]` structure, all
dynamically typed. This means it can be dynamically decoded without templates.
```erlang
gmser_dyn:decode(EL).
[{name,<<"Ulf">>},{age,29}]
```
In order to specify something like Erlang's `[integer()]` type, we can use
the following:
```erlang
gmser_dyn:encode_typed(#{list => int}, [1,2,3,4]) ->
[<<0>>,<<1>>,[<<251>>,
[[<<248>>,<<1>>],[<<248>>,<<2>>],[<<248>>,<<3>>],[<<248>>,<<4>>]]]]
```
#### `alt`
The `#{alt => [Type]}` construct also enforces dynamic encoding, and will try
to encode as each type in the list, in the specified order, until one matches.
```erlang
gmser_dyn:encode_typed(#{alt => [negint,int]}, 5) -> [<<0>>,<<1>>,[<<247>>,<<5>>]]
gmser_dyn:encode_typed(#{alt => [negint,int]}, 5) -> [<<0>>,<<1>>,[<<248>>,<<5>>]]
gmser_dyn:encode_typed(anyint,-5) -> [<<0>>,<<1>>,[<<246>>,[<<247>>,<<5>>]]]
gmser_dyn:encode_typed(anyint,5) -> [<<0>>,<<1>>,[<<246>>,[<<248>>,<<5>>]]]
```
#### `switch`
The `switch` type allows for encoding a 'tagged' object, where the tag determines
the type.
```erlang
E1 = gmser_dyn:encode_typed(#{switch => #{name => binary, age => int}}, #{age => 29}) ->
[<<0>>,<<1>>,[<<252>>,[[[<<255>>,<<97,103,101>>],[<<248>>,<<29>>]]]]]
gmser_dyn:decode_typed(#{switch => #{name => binary, age => int}}, E1) ->
#{age => 29}
E2 = gmser_dyn:encode_typed(#{switch => #{name => binary, age => int}}, #{name => <<"Ulf">>}) ->
[<<0>>,<<1>>,[<<252>>,[[[<<255>>,<<110,97,109,101>>],[<<249>>,<<85,108,102>>]]]]]
gmser_dyn:decode_typed(#{switch => #{name => binary, age => int}}, E1) ->
#{name => <<"Ulf">>}
```
A practical use of `switch` would be in a protocol schema:
```erlang
t_msg(_) ->
#{switch => #{ call => t_call
, reply => t_reply
, notification => t_notification }}.
t_call(_) ->
#{items => [ {id, anyint}
, {req, t_req} ]}.
t_reply(_) ->
#{alt => [#{items => [ {id, anyint}
, {result, t_result} ]},
#{items => [ {id, anyint}
, {code, anyint}
, {message, binary} ]}
]}.
```
In this scenario, messages are 'taggged' as 1-element maps, e.g.:
```erlang
async_request(Msg) ->
Id = erlang:unique_integer(),
gmmp_cp:to_server(
whereis(gmmp_core_connector),
#{call => #{ id => Id
, req => Msg }}),
Id.
```
### Notes
Note that `anyint` is a standard type. The static serializer supports only
positive integers (`int`), as negative numbers are forbidden on-chain.
For dynamic encoding e.g. in messaging protocols, handling negative numbers can
be useful, so the `negint` type was added as a dynamic type. To encode a full-range
integer, the `alt` construct is needed.
(Floats are not supported, as they are non-deterministic. Rationals and fixed-point
numbers could easily be handled as high-level types, e.g. as `{int,int}`.)
+212
View File
@@ -0,0 +1,212 @@
# ASN.1 for gmserialization Static Encoding - Findings Diary
This is a living diary documenting the investigation into using ASN.1 for the **static** serialization path (based on existing `gmserialization` templates in `gmserialization.erl` and `gmser_chain_objects.erl`). Dynamic encoding is out of scope.
Focus areas:
- Modeling static templates with ASN.1 (portability goal).
- Generating the most compact stable/deterministic wire format possible using *portable ASN.1 techniques* (UPER etc.).
- Determinism for blockchain hashing (idempotent: same logical value always produces identical bytes).
- (Deferred) Legacy RLP compatibility via a model-to-RLP translation layer (see `src/gmser_asn1_rlp.erl`).
The single source of truth is the ASN.1 schema in `GajumaruSerialization.asn`.
---
## 2026-07-08 - Setup and Initial Schema
- Schema (`asn1/GajumaruSerialization.asn`) models:
- `GajumaruData` as top-level (tag + vsn + content).
- `Content` CHOICE with `templateFields` (generic) and concrete types (e.g. `SignedTx`, `ContractV*`).
- `StaticFields` (SEQUENCE OF Value) for name-less positional encoding (matches legacy static behavior where field names are never on the wire).
- `Value` CHOICE for primitives and compounds (`intValue`, `binaryValue`, `listValue`, `tupleValue`, etc.).
- Supports all static template types: `int`, `bool`, `binary`, `id`, `[T]`, tuples, `#{items => [...]}`.
- Initial schema comments were DER-oriented (migration path). Updated to emphasize compact UPER + portability.
- Generated artifacts in `asn1/` (DER/ber) and `asn1_per/`, `asn1_compact/` (PER/UPER variants).
- Key files:
- `asn1/GajumaruSerialization.asn` (source)
- `src/gmser_asn1_rlp.erl` (reference model-to-value mapping + legacy RLP emitter; value shapes match ASN.1)
- Tests in `test/gmser_chain_objects_tests.erl` and inside `gmser_asn1_rlp` for equivalence.
## 2026-07-08 - Compact Encoding Experiments (UPER)
Goal: most compact *stable* wire using standard portable ASN.1 (not custom non-portable rules, not RLP).
- Tried standard DER → too verbose (tiny case: ~36 bytes vs legacy RLP 5 bytes).
- Switched to **UPER (Unaligned PER)** — the most compact *standard* ASN.1 encoding rule with good canonical/deterministic properties.
- Compiled via `asn1ct:compile(..., [uper])`.
- Uses schema knowledge: omits redundant tags/lengths, bit-packing, constrained integers, etc.
- Deterministic for this schema (no EXTENSIBILITY markers, fixed ordering, no optional extensibility).
- Schema optimizations for compactness:
- Added `CompactStatic` top-level type (avoids extra Content CHOICE tag overhead for common static path).
- `staticFields` (pure `SEQUENCE OF Value`) — no IA5String names on wire.
- Constrained `tag`/`vsn` (INTEGER (0..65535), (0..255)) for better packing.
- Prefer concrete SEQUENCES (e.g. `SignedTx`) or `staticFields` over generic `templateFields` (names add cost).
- Kept `TemplateFields` for debug/transition only.
- Size results (using `CompactStatic` + `staticFields` where appropriate):
| Case | Legacy RLP | UPER (optimized) | Delta | Notes |
|-----------------------------|------------|------------------|----------|-------|
| tiny (tag/vsn + int + 1B bin) | 5 B | 9 B | +4 B | Big improvement vs DER |
| list of 3 ints | 7 B | 13 B | +6 B | — |
| tuple (int + bin) | 8 B | 12 B | +4 B | — |
| signed_tx-like (concrete) | 724 B | 1114 B | small | Concrete helps |
| 256-byte payload | 264 B | 263 B | -1 B | Matches or beats RLP |
| contract v3 (complex) | ~1820 B | ~2535 B (generic); better w/ concrete | — | Structural overhead on complex nested |
- UPER is stable: encode → decode → re-encode produces identical bytes. Roundtrips work.
- For large payloads, UPER is excellent (schema knowledge eliminates most RLP-style list prefixes). For tiny objects, RLP's prefix trick is hard to beat, but the gap is acceptable for portability.
- OER (Octet Encoding Rules) also compiled but was larger (18 B on tiny case).
## 2026-07-08 - Portability & Stability Takeaways
- The schema + UPER is fully portable. Other languages can:
1. Compile the `.asn` with their ASN.1 tool.
2. Build a value matching `CompactStatic` / `staticFields` / concrete types.
3. Call their UPER encoder → identical compact bytes.
- No Erlang-specific runtime required for the new wire format.
- Determinism comes from:
- UPER canonical packing rules.
- Constrained types in schema.
- Explicit staticFields (no map iteration, names omitted).
- Same rules as legacy for ints (minimal), ordering, etc.
- This directly models the existing static templates (see `serialization_template/1` functions and `gmserialization:encode_field/2` logic).
- Concrete types in schema give best compactness for known objects.
- Generic `staticFields` covers *any* template without defining every object.
## Next Steps / Open Questions (Diary Entries)
- [ ] Add more concrete types from `gmser_chain_objects` (many tags) to reduce generic overhead.
- [ ] Add more ASN.1 constraints (SIZE, value ranges) to help PER pack tighter.
- [ ] Measure on real on-chain objects (key_block, etc.).
- [ ] Decide on top-level header for the new format (keep tag/vsn?).
- [ ] (Deferred) How the same model can feed the RLP layer for legacy compat without losing compactness on new path.
- [ ] Consider if a custom "ASN.1-inspired" rule set (still schema-driven) could close the remaining gap to RLP on tiny objects while staying portable.
## 2026-07-08 - Schema Updated for Bignums
- Updated `GajumaruSerialization.asn`:
- Introduced `BigInt ::= INTEGER (0..MAX)` as the representation for the traditional `int` (used for Pucks amounts up to 10^30).
- `Value` CHOICE now uses `bigIntValue` for the bignum case.
- Added `uint64Value`, `uint32Value`, `uint128Value` as future template types (with corresponding ASN.1 subtypes).
- Updated header comments to document the bignum nature of `int`.
- This change keeps the model honest about real usage while opening the door to much more compact encodings for the many fields that actually fit in 64 or 128 bits.
- Next: We should extend the Erlang-side `type()` in `gmserialization.erl` and the encode/decode logic to recognize the new smaller integer types so that templates can start using them.
---
*This file should be kept as a living diary. Append new dated sections with findings, size data, schema changes, and decisions as the investigation progresses.*
## 2026-07-08 - Handling of `int` as Bignums (Pucks, amounts, etc.)
Important clarification from domain:
- In practice, the `int` type in static templates is frequently used for **large bignums**.
- Example: Amount fields (balances, transaction amounts, etc.) are denominated in "Pucks".
- Maximum value mentioned: 1 × 10^30.
- This is ~ 2^99.66, i.e., requires up to ~13 bytes in minimal unsigned encoding.
Current legacy handling (in `gmserialization.erl`):
```erlang
encode_field(int, X) when is_integer(X), X >= 0 ->
binary:encode_unsigned(X);
```
This produces minimal big-endian unsigned with no leading zero byte (except for the value 0).
Implications for ASN.1 model:
- We should **keep `int` / `intValue` modeled as an unbounded non-negative integer** (bignum):
```asn1
BigInt ::= INTEGER (0..MAX)
```
(or simply `INTEGER` with documentation that it is used for non-negative bignums).
- Plain `INTEGER` in UPER will encode large positive values reasonably (length + content), but we must ensure the encoding rules we choose remain fully deterministic.
- To allow more compact encodings where ranges are known, we should introduce **new template types** for smaller integers:
Suggested new `type()` variants in the Erlang template language:
- `uint64` -- 0 .. 2^64-1
- `uint32` -- 0 .. 2^32-1
- `uint16`, `uint8`, `uint128` etc. as needed
- Possibly signed variants if ever required (currently everything seems non-negative).
Corresponding in ASN.1 (inside Value CHOICE or as reusable types):
```asn1
Uint64 ::= INTEGER (0..18446744073709551615)
Uint32 ::= INTEGER (0..4294967295)
Uint128 ::= INTEGER (0..340282366920938463463374607431768211455)
```
- In the schema's Value CHOICE we can evolve to:
```asn1
Value ::= CHOICE {
bigIntValue [0] BigInt, -- the classic "int" for Pucks etc.
uint64Value [7] Uint64,
uint32Value [8] Uint32,
...
-- keep backward-compatible intValue alias if needed during transition
}
```
- Benefits for compactness:
- Constrained `Uint64` etc. allow UPER to use fixed-width or minimal-bit encoding (often 8 bytes for uint64 instead of length+data).
- Still fully portable and deterministic.
- Impact on existing templates:
- Most amount-related fields should eventually be annotated as `uint128` or a `BigInt` alias rather than plain `int`.
- Small counters, versions, indices can use `uint32` / `uint64`.
- This may require extending the `type()` in `gmserialization.erl` and the encoders.
## 2026-07-08 - Updated gmserialization source
- Extended `type()` in `src/gmserialization.erl` with `uint128`, `uint64`, `uint32`, `uint16`, `uint8` (keeping `int` for bignums).
- Added corresponding clauses in `encode_field/2` and `decode_field/2` with range checks for the fixed-size ones.
- Updated `src/gmser_asn1_rlp.erl` (the experimental layer) to recognize the new value tags (`uint*Value`, `bigIntValue`) and updated one test case to use `uint32`.
- Updated `doc/static.md` example types.
- `int` remains fully backward compatible for bignum use.
- All existing tests + new type usage pass.
- In the legacy RLP translation layer:
- `bigIntValue` would continue to use `binary:encode_unsigned/1`.
- Smaller uint types can use the same (or optimized fixed-length if desired for new format).
Next action items:
- Update `GajumaruSerialization.asn` to introduce `BigInt`, `Uint*` types and adjust Value.
- Decide on naming in the Erlang template DSL (`int` remains bignum alias? or rename?).
- Add example in the schema for a balance field.
- Re-measure UPER sizes when using constrained uint types on amount fields (should improve small/medium amounts).
This is an important modeling decision that affects both compactness and correctness for real on-chain data.
## 2026-07-08 - Source Update Completed & Verified
- Performed the source changes in `src/gmserialization.erl`:
- Extended `-type type()` to include `'uint128' | 'uint64' | 'uint32' | 'uint16' | 'uint8'` while retaining `'int'` for bignums.
- Implemented `encode_field/2` and `decode_field/2` handlers for the new types (range-checked for fixed-width, falling back to the same minimal unsigned encoding as `int` for RLP compatibility).
- Synced `src/gmser_asn1_rlp.erl`:
- Added support for new ASN.1 value variants (`{uint*Value, ...}`, `{bigIntValue, ...}`) in `encode_asn1_value/1`.
- Updated test data and comments to use the new types.
- Updated `doc/static.md` to reflect the extended type language.
- Verified:
- Legacy templates using `int` continue to work unchanged.
- New types (e.g. `{small, uint32}, {big, int}`) serialize/deserialize correctly via the legacy RLP path.
- All 7 equivalence tests in `gmser_asn1_rlp` still pass.
- `gmser_chain_objects_tests` (12 tests) still pass.
- The ASN.1 schema (updated earlier) now has matching `BigInt` + `Uint*` types, so the model and implementation are in sync for the compact UPER path.
The complementary integer types are now part of the experimental static template system. This enables the ASN.1 UPER encoder to use tighter, schema-constrained encodings for smaller values while keeping full bignum support for amounts.
Next diary items (still open):
- Extend more concrete types in the schema.
- Add actual UPER-based encode path in the library (beyond the RLP layer).
- Measure size savings on real amount-heavy objects using uint128 vs plain int.
+267
View File
@@ -0,0 +1,267 @@
-- GajumaruSerialization.asn
--
-- ASN.1 model for the structures serialized by gmserialization.erl
-- (the layer on top of RLP).
--
-- Purpose:
-- * Provide a formal, toolable description of the data for portability
-- (other languages use ASN.1 compilers to get types/parsers).
-- * Define compact canonical wire format using standard ASN.1 techniques
-- (primarily Unaligned PER / UPER with constraints for packing).
-- * Static encoding only (templates from gmserialization).
-- * Legacy RLP compatibility is achieved via separate translation layer
-- (deferred in current focus).
--
-- Detection of legacy vs new:
-- Legacy data (produced by gmserialization:serialize / gmser_chain_objects:serialize
-- or gmser_dyn:serialize) is always an RLP *list* at the top level.
-- This means the first byte is in the range 0xC0 .. 0xFF.
--
-- DER-encoded data using the types below will start with 0x30 (SEQUENCE,
-- constructed, universal tag) for the outermost GajumaruData (or any
-- top-level SEQUENCE we define). 0x30 < 0xC0, so a single-byte prefix
-- check reliably distinguishes the two formats.
--
-- Usage in Erlang (after compiling with asn1ct for compact wire):
-- {ok, Mod} = asn1ct:compile(GajumaruSerialization.asn, [uper]).
-- {ok, Bytes} = 'GajumaruSerialization':encode('GajumaruData', Asn1Value).
-- Bytes is the compact UPER wire format (deterministic with this schema).
-- For legacy RLP, use the model-to-RLP layer instead (see gmser_asn1_rlp).
--
-- Notes on type mapping from gmserialization.erl (static templates):
-- - int (bignum) -> BigInt ::= INTEGER (0..MAX)
-- In practice used for amounts/balances in Pucks (up to 1*10^30).
-- Encoded as non-negative bignum.
-- - New smaller integer types will be added to templates (uint64, uint32, ...)
-- for cases where range is known → enables tighter UPER packing.
-- - binary -> OCTET STRING
-- - bool -> BOOLEAN
-- - id -> Id (SEQUENCE)
-- - [T] -> SEQUENCE OF T
-- - {T1,...} -> SEQUENCE (fixed-size, order matters)
-- - #{items := [{K,T},...]} -> SEQUENCE with the fields in template order
-- (static maps carry *values only*, no keys on the wire)
-- - tag + vsn are preserved as the first two fields for compatibility
-- with existing dispatch logic.
--
-- WARNING:
-- Any cryptographic hash or signature computed over a serialized object
-- will change when switching from RLP to DER for that object. Plan a
-- coordinated upgrade.
GajumaruSerialization DEFINITIONS
AUTOMATIC TAGS ::=
BEGIN
EXPORTS ALL;
-- ============================================================
-- Top-level wrapper used for new DER data.
-- This is what a decoder will see first.
-- ============================================================
GajumaruData ::= SEQUENCE {
-- Constrained for better PER packing
tag INTEGER (0..65535),
vsn INTEGER (0..255),
content Content
}
-- Preferred top-level for the compact static wire format.
-- Avoids the extra Content CHOICE tag when the structure is known to be static.
CompactStatic ::= SEQUENCE {
tag INTEGER (0..65535),
vsn INTEGER (0..255),
fields StaticFields
}
-- Content can be a specific structured type (preferred) or a generic
-- representation of a template-driven object.
Content ::= CHOICE {
-- Generic fallback that can represent any [{Field, Type}] template
-- without having a pre-defined SEQUENCE for every object.
templateFields [0] TemplateFields,
-- Static-optimized: no field names on wire (matches legacy static behavior exactly)
-- Preferred for compact wire format of known templates.
staticFields [10] StaticFields,
-- Examples of concrete versioned types (extend as needed)
account [1] Account,
signedTx [2] SignedTx,
contract [3] ContractCode
-- Add more alternatives for other tags from gmser_chain_objects
}
-- ============================================================
-- Generic template-driven representation
-- (useful during transition or for unregistered types)
-- ============================================================
TemplateFields ::= SEQUENCE OF TemplateField
TemplateField ::= SEQUENCE {
-- Field name is included for debuggability / generic processing.
-- In the original static encoding the name is NOT on the wire;
-- only position and type matter. We include it here for convenience.
name IA5String OPTIONAL,
value Value
}
-- Optimized for static wire format: just the values in order, no names.
-- This matches the legacy static encoding where maps/records are positional only.
StaticFields ::= SEQUENCE OF Value
Value ::= CHOICE {
-- "int" in static templates is used for bignums in practice
-- (e.g. balances and amounts in Pucks, up to 1*10^30).
-- We keep it as unbounded non-negative integer.
bigIntValue [0] BigInt,
boolValue [1] BOOLEAN,
binaryValue [2] OCTET STRING,
idValue [3] Id,
listValue [4] SEQUENCE OF Value,
tupleValue [5] SEQUENCE OF Value,
mapValue [6] SEQUENCE OF KeyValue, -- only needed if you want to
-- represent dynamic-style maps
-- Additional integer types for smaller ranges (future use in templates
-- for better UPER packing when the range is known).
uint64Value [7] Uint64,
uint32Value [8] Uint32,
uint128Value [9] Uint128
}
-- Bignum integer (non-negative). Used for the traditional "int" in static
-- templates. Max practical value mentioned: 1*10^30 (≈ 2^100 bits).
BigInt ::= INTEGER (0..MAX)
-- Convenience sized unsigned integer types.
Uint64 ::= INTEGER (0..18446744073709551615)
Uint32 ::= INTEGER (0..4294967295)
Uint128 ::= INTEGER (0..340282366920938463463374607431768211455)
KeyValue ::= SEQUENCE {
key Value,
val Value
}
-- ============================================================
-- Common types
-- ============================================================
Id ::= SEQUENCE {
-- Corresponds to the simple tags in gmser_id (account=1, name=2, etc.)
-- and the extended account subtype (high bit in legacy).
type INTEGER (0..255),
value OCTET STRING (SIZE (32))
}
-- ============================================================
-- Concrete object examples (derived from usage in the codebase)
-- Add / evolve per version as you introduce new vsns.
-- ============================================================
-- Example: a very simple account-like object used in tests
Account ::= SEQUENCE {
foo INTEGER,
bar OCTET STRING
}
-- Simplified signed transaction
SignedTx ::= SEQUENCE {
signatures SEQUENCE OF OCTET STRING,
transaction OCTET STRING
}
-- Contract code objects (see gmser_contract_code.erl)
-- We model the three versions that exist today.
ContractCode ::= CHOICE {
v1 [0] ContractV1,
v2 [1] ContractV2,
v3 [2] ContractV3
}
ContractV1 ::= SEQUENCE {
sourceHash OCTET STRING,
-- typeInfo is a list of 4-tuples in legacy:
-- {typeHash, name, argType, outType}
typeInfo SEQUENCE OF TypeInfoV1,
byteCode OCTET STRING
}
ContractV2 ::= SEQUENCE {
sourceHash OCTET STRING,
typeInfo SEQUENCE OF TypeInfoV1,
byteCode OCTET STRING,
compilerVersion OCTET STRING
}
ContractV3 ::= SEQUENCE {
sourceHash OCTET STRING,
typeInfo SEQUENCE OF TypeInfoV3,
byteCode OCTET STRING,
compilerVersion OCTET STRING,
payable BOOLEAN
}
TypeInfoV1 ::= SEQUENCE {
typeHash OCTET STRING,
name OCTET STRING,
argType OCTET STRING,
outType OCTET STRING
}
TypeInfoV3 ::= SEQUENCE {
typeHash OCTET STRING,
name OCTET STRING,
payable BOOLEAN,
argType OCTET STRING,
outType OCTET STRING
}
-- ============================================================
-- Notes for implementers
-- ============================================================
-- 1. Detection (recommended decoder entry point):
--
-- decode(Binary) ->
-- case Binary of
-- <<B, _/binary>> when B >= 16#C0 ->
-- decode_legacy_rlp(Binary); % existing gmser_* path
-- _ ->
-- {ok, Term} =
-- 'GajumaruSerialization':decode('GajumaruData', Binary),
-- Term
-- end.
--
-- This works because:
-- - All current top-level output from serialize() is an RLP list
-- (first byte 0xC0-0xFF).
-- - GajumaruData and the concrete choices above are SEQUENCEs
-- (first byte 0x30 for short form, or 0x30 0x81/0x82... for long).
--
-- 2. When you add a new object type or version, prefer adding a
-- concrete SEQUENCE alternative in the Content CHOICE rather than
-- always falling back to TemplateFields. This gives you better
-- validation and generated types.
--
-- 3. For the compact wire format we use UPER (unaligned PER).
-- It is stable/deterministic for a given schema (no extensibility
-- markers on these types, fixed order).
--
-- 4. INTEGER uses ASN.1 PER encoding (canonical for the constraints).
-- This may differ from legacy RLP minimal unsigned; new format
-- will have different hashes (expected when introducing new encoding).
--
-- 5. Dynamic encoding (gmser_dyn) is not in scope here.
--
-- 6. To generate the compact wire:
-- asn1ct:compile(GajumaruSerialization, [uper]).
-- {ok, CompactBytes} = 'GajumaruSerialization':encode('GajumaruData', Value).
--
-- Use staticFields (not templateFields) for best compactness on static data.
END
File diff suppressed because it is too large Load Diff
+76
View File
@@ -0,0 +1,76 @@
%% Generated by the Erlang ASN.1 compiler. Version: 5.4.3
%% Purpose: Erlang record definitions for each named and unnamed
%% SEQUENCE and SET, and macro definitions for each value
%% definition in module GajumaruSerialization.
-ifndef(_GAJUMARUSERIALIZATION_HRL_).
-define(_GAJUMARUSERIALIZATION_HRL_, true).
-record('GajumaruData', {
tag,
vsn,
content
}).
-record('TemplateField', {
name = asn1_NOVALUE,
value
}).
-record('KeyValue', {
key,
val
}).
-record('Id', {
type,
value
}).
-record('Account', {
foo,
bar
}).
-record('SignedTx', {
signatures,
transaction
}).
-record('ContractV1', {
sourceHash,
typeInfo,
byteCode
}).
-record('ContractV2', {
sourceHash,
typeInfo,
byteCode,
compilerVersion
}).
-record('ContractV3', {
sourceHash,
typeInfo,
byteCode,
compilerVersion,
payable
}).
-record('TypeInfoV1', {
typeHash,
name,
argType,
outType
}).
-record('TypeInfoV3', {
typeHash,
name,
payable,
argType,
outType
}).
-endif. %% _GAJUMARUSERIALIZATION_HRL_
+27
View File
@@ -0,0 +1,27 @@
-module(detect_demo).
-export([run/0]).
run() ->
application:ensure_all_started(gmserialization),
code:add_path("asn1"),
Sample = {'GajumaruData', 10, 1,
{templateFields, [
{'TemplateField', <<"foo">>, {intValue, 42}},
{'TemplateField', <<"bar">>, {binaryValue, <<"hello">>}}
]}},
{ok, Der} = 'GajumaruSerialization':encode('GajumaruData', Sample),
io:format("DER first byte: ~p (0x~2.16.0B)~n", [binary:at(Der,0), binary:at(Der,0)]),
{ok, _Dec} = 'GajumaruSerialization':decode('GajumaruData', Der),
io:format("DER roundtrip OK~n"),
Legacy = gmser_chain_objects:serialize(account, 1,
[{foo,int},{bar,binary}],
[{foo,42},{bar,<<"hello">>}]),
<<L0>> = binary:part(Legacy,0,1),
<<D0>> = binary:part(Der,0,1),
io:format("Legacy 0x~2.16.0B (>= 0xC0 -> legacy: ~p)~n", [L0, L0 >= 16#C0]),
io:format("DER 0x~2.16.0B (< 0xC0 -> new DER: ~p)~n", [D0, D0 < 16#C0]),
ok.
+114
View File
@@ -0,0 +1,114 @@
-module(size_comparison).
-export([run/0]).
run() ->
application:ensure_all_started(gmserialization),
code:add_path("asn1"),
io:format("=== Size Comparison: Legacy RLP vs DER ===~n~n"),
Compare = fun(Desc, LegacyBin, DerBin) ->
L = byte_size(LegacyBin),
D = byte_size(DerBin),
Overhead = D - L,
Pct = case L of 0 -> 0; _ -> round(Overhead * 100 / L) end,
io:format("~s~n", [Desc]),
io:format(" Legacy: ~p bytes ~w~n", [L, LegacyBin]),
Show = binary:part(DerBin, 0, min(18, D)),
io:format(" DER: ~p bytes ~w~n", [D, Show]),
io:format(" Overhead: +~p bytes (~p%)~n~n", [Overhead, Pct])
end,
%% Case 1: tag+vsn + small int + small binary
T1 = [{foo,int},{bar,binary}],
V1 = [{foo,1},{bar,<<2>>}],
Leg1 = gmser_chain_objects:serialize(account, 1, T1, V1),
DerVal1 = {'GajumaruData', 10, 1, {templateFields, [
{'TemplateField', <<"foo">>, {intValue, 1}},
{'TemplateField', <<"bar">>, {binaryValue, <<2>>}}
]}},
{ok, Der1} = 'GajumaruSerialization':encode('GajumaruData', DerVal1),
Compare("Case 1: tag+vsn + small int + tiny binary (2 fields)", Leg1, Der1),
%% Case 2: zero + empty binary
V2 = [{foo,0},{bar,<<>>}],
Leg2 = gmser_chain_objects:serialize(account, 1, T1, V2),
DerVal2 = {'GajumaruData', 10, 1, {templateFields, [
{'TemplateField', <<"foo">>, {intValue, 0}},
{'TemplateField', <<"bar">>, {binaryValue, <<>>}}
]}},
{ok, Der2} = 'GajumaruSerialization':encode('GajumaruData', DerVal2),
Compare("Case 2: zero int + empty binary", Leg2, Der2),
%% Case 3: list of ints
T3 = [{xs,[int]}],
V3 = [{xs,[1,2,3]}],
Leg3 = gmser_chain_objects:serialize(account, 1, T3, V3),
DerVal3 = {'GajumaruData', 10, 1, {templateFields, [
{'TemplateField', <<"xs">>, {listValue, [
{intValue,1},{intValue,2},{intValue,3}
]}}
]}},
{ok, Der3} = 'GajumaruSerialization':encode('GajumaruData', DerVal3),
Compare("Case 3: list of 3 small ints", Leg3, Der3),
%% Case 4: tuple (int, binary)
T4 = [{p,{int,binary}}],
V4 = [{p,{42,<<"hi">>}}],
Leg4 = gmser_chain_objects:serialize(account, 1, T4, V4),
DerVal4 = {'GajumaruData', 10, 1, {templateFields, [
{'TemplateField', <<"p">>, {tupleValue, [
{intValue,42}, {binaryValue,<<"hi">>}
]}}
]}},
{ok, Der4} = 'GajumaruSerialization':encode('GajumaruData', DerVal4),
Compare("Case 4: fixed tuple (int + 2-byte binary)", Leg4, Der4),
%% Case 5: 256-byte payload
Bin5 = crypto:strong_rand_bytes(256),
T5 = [{data,binary}],
V5 = [{data,Bin5}],
Leg5 = gmser_chain_objects:serialize(account, 1, T5, V5),
DerVal5 = {'GajumaruData', 10, 1, {templateFields, [
{'TemplateField', <<"data">>, {binaryValue, Bin5}}
]}},
{ok, Der5} = 'GajumaruSerialization':encode('GajumaruData', DerVal5),
Compare("Case 5: 256-byte binary payload only", Leg5, Der5),
%% Case 6: Concrete SignedTx style (no field names in DER)
T6 = [{signatures,[binary]},{tx,binary}],
V6 = [{signatures,[<<"sig1">>,<<"sig2">>]},{tx,<<"txbody123">>}],
Leg6 = gmser_chain_objects:serialize(signed_tx, 1, T6, V6),
DerVal6 = {'GajumaruData', 11, 1, {signedTx, {'SignedTx',
[<<"sig1">>, <<"sig2">>], <<"txbody123">>}}},
{ok, Der6} = 'GajumaruSerialization':encode('GajumaruData', DerVal6),
Compare("Case 6: SignedTx-like (concrete DER, no names)", Leg6, Der6),
%% Case 7: 33-byte id-like value
IdBin = <<1, 0:256>>,
T7 = [{owner,binary}],
V7 = [{owner,IdBin}],
Leg7 = gmser_chain_objects:serialize(account, 1, T7, V7),
DerVal7 = {'GajumaruData', 10, 1, {templateFields, [
{'TemplateField', <<"owner">>, {binaryValue, IdBin}}
]}},
{ok, Der7} = 'GajumaruSerialization':encode('GajumaruData', DerVal7),
Compare("Case 7: 33-byte value (id-like)", Leg7, Der7),
%% Case 8: Deeper nesting / more fields
T8 = [{a,int},{b,binary},{c,[int]},{d,{int,int}}],
V8 = [{a,123456},{b,<<"abcdef">>},{c,[10,20,30]},{d,{7,8}}],
Leg8 = gmser_chain_objects:serialize(account, 1, T8, V8),
DerVal8 = {'GajumaruData', 10, 1, {templateFields, [
{'TemplateField', <<"a">>, {intValue, 123456}},
{'TemplateField', <<"b">>, {binaryValue, <<"abcdef">>}},
{'TemplateField', <<"c">>, {listValue, [{intValue,10},{intValue,20},{intValue,30}]}},
{'TemplateField', <<"d">>, {tupleValue, [{intValue,7},{intValue,8}]}}
]}},
{ok, Der8} = 'GajumaruSerialization':encode('GajumaruData', DerVal8),
Compare("Case 8: 4 fields mixed (int, bin, list, tuple)", Leg8, Der8),
io:format("=== Analysis ===~n"),
io:format("Note: Generic templateFields path includes IA5String field names.~n"),
io:format("Concrete types (e.g. SignedTx) avoid name overhead.~n"),
ok.
+267
View File
@@ -0,0 +1,267 @@
-- GajumaruSerialization.asn
--
-- ASN.1 model for the structures serialized by gmserialization.erl
-- (the layer on top of RLP).
--
-- Purpose:
-- * Provide a formal, toolable description of the data for portability
-- (other languages use ASN.1 compilers to get types/parsers).
-- * Define compact canonical wire format using standard ASN.1 techniques
-- (primarily Unaligned PER / UPER with constraints for packing).
-- * Static encoding only (templates from gmserialization).
-- * Legacy RLP compatibility is achieved via separate translation layer
-- (deferred in current focus).
--
-- Detection of legacy vs new:
-- Legacy data (produced by gmserialization:serialize / gmser_chain_objects:serialize
-- or gmser_dyn:serialize) is always an RLP *list* at the top level.
-- This means the first byte is in the range 0xC0 .. 0xFF.
--
-- DER-encoded data using the types below will start with 0x30 (SEQUENCE,
-- constructed, universal tag) for the outermost GajumaruData (or any
-- top-level SEQUENCE we define). 0x30 < 0xC0, so a single-byte prefix
-- check reliably distinguishes the two formats.
--
-- Usage in Erlang (after compiling with asn1ct for compact wire):
-- {ok, Mod} = asn1ct:compile(GajumaruSerialization.asn, [uper]).
-- {ok, Bytes} = 'GajumaruSerialization':encode('GajumaruData', Asn1Value).
-- Bytes is the compact UPER wire format (deterministic with this schema).
-- For legacy RLP, use the model-to-RLP layer instead (see gmser_asn1_rlp).
--
-- Notes on type mapping from gmserialization.erl (static templates):
-- - int (bignum) -> BigInt ::= INTEGER (0..MAX)
-- In practice used for amounts/balances in Pucks (up to 1*10^30).
-- Encoded as non-negative bignum.
-- - New smaller integer types will be added to templates (uint64, uint32, ...)
-- for cases where range is known → enables tighter UPER packing.
-- - binary -> OCTET STRING
-- - bool -> BOOLEAN
-- - id -> Id (SEQUENCE)
-- - [T] -> SEQUENCE OF T
-- - {T1,...} -> SEQUENCE (fixed-size, order matters)
-- - #{items := [{K,T},...]} -> SEQUENCE with the fields in template order
-- (static maps carry *values only*, no keys on the wire)
-- - tag + vsn are preserved as the first two fields for compatibility
-- with existing dispatch logic.
--
-- WARNING:
-- Any cryptographic hash or signature computed over a serialized object
-- will change when switching from RLP to DER for that object. Plan a
-- coordinated upgrade.
GajumaruSerialization DEFINITIONS
AUTOMATIC TAGS ::=
BEGIN
EXPORTS ALL;
-- ============================================================
-- Top-level wrapper used for new DER data.
-- This is what a decoder will see first.
-- ============================================================
GajumaruData ::= SEQUENCE {
-- Constrained for better PER packing
tag INTEGER (0..65535),
vsn INTEGER (0..255),
content Content
}
-- Preferred top-level for the compact static wire format.
-- Avoids the extra Content CHOICE tag when the structure is known to be static.
CompactStatic ::= SEQUENCE {
tag INTEGER (0..65535),
vsn INTEGER (0..255),
fields StaticFields
}
-- Content can be a specific structured type (preferred) or a generic
-- representation of a template-driven object.
Content ::= CHOICE {
-- Generic fallback that can represent any [{Field, Type}] template
-- without having a pre-defined SEQUENCE for every object.
templateFields [0] TemplateFields,
-- Static-optimized: no field names on wire (matches legacy static behavior exactly)
-- Preferred for compact wire format of known templates.
staticFields [10] StaticFields,
-- Examples of concrete versioned types (extend as needed)
account [1] Account,
signedTx [2] SignedTx,
contract [3] ContractCode
-- Add more alternatives for other tags from gmser_chain_objects
}
-- ============================================================
-- Generic template-driven representation
-- (useful during transition or for unregistered types)
-- ============================================================
TemplateFields ::= SEQUENCE OF TemplateField
TemplateField ::= SEQUENCE {
-- Field name is included for debuggability / generic processing.
-- In the original static encoding the name is NOT on the wire;
-- only position and type matter. We include it here for convenience.
name IA5String OPTIONAL,
value Value
}
-- Optimized for static wire format: just the values in order, no names.
-- This matches the legacy static encoding where maps/records are positional only.
StaticFields ::= SEQUENCE OF Value
Value ::= CHOICE {
-- "int" in static templates is used for bignums in practice
-- (e.g. balances and amounts in Pucks, up to 1*10^30).
-- We keep it as unbounded non-negative integer.
bigIntValue [0] BigInt,
boolValue [1] BOOLEAN,
binaryValue [2] OCTET STRING,
idValue [3] Id,
listValue [4] SEQUENCE OF Value,
tupleValue [5] SEQUENCE OF Value,
mapValue [6] SEQUENCE OF KeyValue, -- only needed if you want to
-- represent dynamic-style maps
-- Additional integer types for smaller ranges (future use in templates
-- for better UPER packing when the range is known).
uint64Value [7] Uint64,
uint32Value [8] Uint32,
uint128Value [9] Uint128
}
-- Bignum integer (non-negative). Used for the traditional "int" in static
-- templates. Max practical value mentioned: 1*10^30 (≈ 2^100 bits).
BigInt ::= INTEGER (0..MAX)
-- Convenience sized unsigned integer types.
Uint64 ::= INTEGER (0..18446744073709551615)
Uint32 ::= INTEGER (0..4294967295)
Uint128 ::= INTEGER (0..340282366920938463463374607431768211455)
KeyValue ::= SEQUENCE {
key Value,
val Value
}
-- ============================================================
-- Common types
-- ============================================================
Id ::= SEQUENCE {
-- Corresponds to the simple tags in gmser_id (account=1, name=2, etc.)
-- and the extended account subtype (high bit in legacy).
type INTEGER (0..255),
value OCTET STRING (SIZE (32))
}
-- ============================================================
-- Concrete object examples (derived from usage in the codebase)
-- Add / evolve per version as you introduce new vsns.
-- ============================================================
-- Example: a very simple account-like object used in tests
Account ::= SEQUENCE {
foo INTEGER,
bar OCTET STRING
}
-- Simplified signed transaction
SignedTx ::= SEQUENCE {
signatures SEQUENCE OF OCTET STRING,
transaction OCTET STRING
}
-- Contract code objects (see gmser_contract_code.erl)
-- We model the three versions that exist today.
ContractCode ::= CHOICE {
v1 [0] ContractV1,
v2 [1] ContractV2,
v3 [2] ContractV3
}
ContractV1 ::= SEQUENCE {
sourceHash OCTET STRING,
-- typeInfo is a list of 4-tuples in legacy:
-- {typeHash, name, argType, outType}
typeInfo SEQUENCE OF TypeInfoV1,
byteCode OCTET STRING
}
ContractV2 ::= SEQUENCE {
sourceHash OCTET STRING,
typeInfo SEQUENCE OF TypeInfoV1,
byteCode OCTET STRING,
compilerVersion OCTET STRING
}
ContractV3 ::= SEQUENCE {
sourceHash OCTET STRING,
typeInfo SEQUENCE OF TypeInfoV3,
byteCode OCTET STRING,
compilerVersion OCTET STRING,
payable BOOLEAN
}
TypeInfoV1 ::= SEQUENCE {
typeHash OCTET STRING,
name OCTET STRING,
argType OCTET STRING,
outType OCTET STRING
}
TypeInfoV3 ::= SEQUENCE {
typeHash OCTET STRING,
name OCTET STRING,
payable BOOLEAN,
argType OCTET STRING,
outType OCTET STRING
}
-- ============================================================
-- Notes for implementers
-- ============================================================
-- 1. Detection (recommended decoder entry point):
--
-- decode(Binary) ->
-- case Binary of
-- <<B, _/binary>> when B >= 16#C0 ->
-- decode_legacy_rlp(Binary); % existing gmser_* path
-- _ ->
-- {ok, Term} =
-- 'GajumaruSerialization':decode('GajumaruData', Binary),
-- Term
-- end.
--
-- This works because:
-- - All current top-level output from serialize() is an RLP list
-- (first byte 0xC0-0xFF).
-- - GajumaruData and the concrete choices above are SEQUENCEs
-- (first byte 0x30 for short form, or 0x30 0x81/0x82... for long).
--
-- 2. When you add a new object type or version, prefer adding a
-- concrete SEQUENCE alternative in the Content CHOICE rather than
-- always falling back to TemplateFields. This gives you better
-- validation and generated types.
--
-- 3. For the compact wire format we use UPER (unaligned PER).
-- It is stable/deterministic for a given schema (no extensibility
-- markers on these types, fixed order).
--
-- 4. INTEGER uses ASN.1 PER encoding (canonical for the constraints).
-- This may differ from legacy RLP minimal unsigned; new format
-- will have different hashes (expected when introducing new encoding).
--
-- 5. Dynamic encoding (gmser_dyn) is not in scope here.
--
-- 6. To generate the compact wire:
-- asn1ct:compile(GajumaruSerialization, [uper]).
-- {ok, CompactBytes} = 'GajumaruSerialization':encode('GajumaruData', Value).
--
-- Use staticFields (not templateFields) for best compactness on static data.
END
Binary file not shown.
File diff suppressed because it is too large Load Diff
+82
View File
@@ -0,0 +1,82 @@
%% Generated by the Erlang ASN.1 compiler. Version: 5.4.3
%% Purpose: Erlang record definitions for each named and unnamed
%% SEQUENCE and SET, and macro definitions for each value
%% definition in module GajumaruSerialization.
-ifndef(_GAJUMARUSERIALIZATION_HRL_).
-define(_GAJUMARUSERIALIZATION_HRL_, true).
-record('GajumaruData', {
tag,
vsn,
content
}).
-record('CompactStatic', {
tag,
vsn,
fields
}).
-record('TemplateField', {
name = asn1_NOVALUE,
value
}).
-record('KeyValue', {
key,
val
}).
-record('Id', {
type,
value
}).
-record('Account', {
foo,
bar
}).
-record('SignedTx', {
signatures,
transaction
}).
-record('ContractV1', {
sourceHash,
typeInfo,
byteCode
}).
-record('ContractV2', {
sourceHash,
typeInfo,
byteCode,
compilerVersion
}).
-record('ContractV3', {
sourceHash,
typeInfo,
byteCode,
compilerVersion,
payable
}).
-record('TypeInfoV1', {
typeHash,
name,
argType,
outType
}).
-record('TypeInfoV3', {
typeHash,
name,
payable,
argType,
outType
}).
-endif. %% _GAJUMARUSERIALIZATION_HRL_
+247
View File
@@ -0,0 +1,247 @@
-- GajumaruSerialization.asn
--
-- ASN.1 model for the structures serialized by gmserialization.erl
-- (the layer on top of RLP).
--
-- Purpose:
-- * Provide a formal, toolable description of the data for portability
-- (other languages use ASN.1 compilers to get types/parsers).
-- * Define compact canonical wire format using standard ASN.1 techniques
-- (primarily Unaligned PER / UPER with constraints for packing).
-- * Static encoding only (templates from gmserialization).
-- * Legacy RLP compatibility is achieved via separate translation layer
-- (deferred in current focus).
--
-- Detection of legacy vs new:
-- Legacy data (produced by gmserialization:serialize / gmser_chain_objects:serialize
-- or gmser_dyn:serialize) is always an RLP *list* at the top level.
-- This means the first byte is in the range 0xC0 .. 0xFF.
--
-- DER-encoded data using the types below will start with 0x30 (SEQUENCE,
-- constructed, universal tag) for the outermost GajumaruData (or any
-- top-level SEQUENCE we define). 0x30 < 0xC0, so a single-byte prefix
-- check reliably distinguishes the two formats.
--
-- Usage in Erlang (after compiling with asn1ct for compact wire):
-- {ok, Mod} = asn1ct:compile(GajumaruSerialization.asn, [uper]).
-- {ok, Bytes} = 'GajumaruSerialization':encode('GajumaruData', Asn1Value).
-- Bytes is the compact UPER wire format (deterministic with this schema).
-- For legacy RLP, use the model-to-RLP layer instead (see gmser_asn1_rlp).
--
-- Notes on type mapping from gmserialization.erl:
-- - int -> INTEGER (new data uses canonical DER INTEGER)
-- - binary -> OCTET STRING
-- - bool -> BOOLEAN
-- - id -> Id (SEQUENCE) (cleaner than the legacy packed 33-byte form)
-- - [T] -> SEQUENCE OF T
-- - {T1,...} -> SEQUENCE (fixed-size, order matters)
-- - #{items := [{K,T},...]} -> SEQUENCE with the fields in template order
-- (static maps carry *values only*, no keys on the wire)
-- - tag + vsn are preserved as the first two fields for compatibility
-- with existing dispatch logic.
--
-- Legacy integers had strict unsigned-minimal no-leading-zero (except for 0)
-- and RLP-level rules. New DER data does not need to follow those rules.
--
-- WARNING:
-- Any cryptographic hash or signature computed over a serialized object
-- will change when switching from RLP to DER for that object. Plan a
-- coordinated upgrade.
GajumaruSerialization DEFINITIONS
AUTOMATIC TAGS ::=
BEGIN
EXPORTS ALL;
-- ============================================================
-- Top-level wrapper used for new DER data.
-- This is what a decoder will see first.
-- ============================================================
GajumaruData ::= SEQUENCE {
-- Constrained for better PER packing
tag INTEGER (0..65535),
vsn INTEGER (0..255),
content Content
}
-- Preferred top-level for the compact static wire format.
-- Avoids the extra Content CHOICE tag when the structure is known to be static.
CompactStatic ::= SEQUENCE {
tag INTEGER (0..65535),
vsn INTEGER (0..255),
fields StaticFields
}
-- Content can be a specific structured type (preferred) or a generic
-- representation of a template-driven object.
Content ::= CHOICE {
-- Generic fallback that can represent any [{Field, Type}] template
-- without having a pre-defined SEQUENCE for every object.
templateFields [0] TemplateFields,
-- Static-optimized: no field names on wire (matches legacy static behavior exactly)
-- Preferred for compact wire format of known templates.
staticFields [10] StaticFields,
-- Examples of concrete versioned types (extend as needed)
account [1] Account,
signedTx [2] SignedTx,
contract [3] ContractCode
-- Add more alternatives for other tags from gmser_chain_objects
}
-- ============================================================
-- Generic template-driven representation
-- (useful during transition or for unregistered types)
-- ============================================================
TemplateFields ::= SEQUENCE OF TemplateField
TemplateField ::= SEQUENCE {
-- Field name is included for debuggability / generic processing.
-- In the original static encoding the name is NOT on the wire;
-- only position and type matter. We include it here for convenience.
name IA5String OPTIONAL,
value Value
}
-- Optimized for static wire format: just the values in order, no names.
-- This matches the legacy static encoding where maps/records are positional only.
StaticFields ::= SEQUENCE OF Value
Value ::= CHOICE {
intValue [0] INTEGER,
boolValue [1] BOOLEAN,
binaryValue [2] OCTET STRING,
idValue [3] Id,
listValue [4] SEQUENCE OF Value,
tupleValue [5] SEQUENCE OF Value,
mapValue [6] SEQUENCE OF KeyValue -- only needed if you want to
-- represent dynamic-style maps
}
KeyValue ::= SEQUENCE {
key Value,
val Value
}
-- ============================================================
-- Common types
-- ============================================================
Id ::= SEQUENCE {
-- Corresponds to the simple tags in gmser_id (account=1, name=2, etc.)
-- and the extended account subtype (high bit in legacy).
type INTEGER (0..255),
value OCTET STRING (SIZE (32))
}
-- ============================================================
-- Concrete object examples (derived from usage in the codebase)
-- Add / evolve per version as you introduce new vsns.
-- ============================================================
-- Example: a very simple account-like object used in tests
Account ::= SEQUENCE {
foo INTEGER,
bar OCTET STRING
}
-- Simplified signed transaction
SignedTx ::= SEQUENCE {
signatures SEQUENCE OF OCTET STRING,
transaction OCTET STRING
}
-- Contract code objects (see gmser_contract_code.erl)
-- We model the three versions that exist today.
ContractCode ::= CHOICE {
v1 [0] ContractV1,
v2 [1] ContractV2,
v3 [2] ContractV3
}
ContractV1 ::= SEQUENCE {
sourceHash OCTET STRING,
-- typeInfo is a list of 4-tuples in legacy:
-- {typeHash, name, argType, outType}
typeInfo SEQUENCE OF TypeInfoV1,
byteCode OCTET STRING
}
ContractV2 ::= SEQUENCE {
sourceHash OCTET STRING,
typeInfo SEQUENCE OF TypeInfoV1,
byteCode OCTET STRING,
compilerVersion OCTET STRING
}
ContractV3 ::= SEQUENCE {
sourceHash OCTET STRING,
typeInfo SEQUENCE OF TypeInfoV3,
byteCode OCTET STRING,
compilerVersion OCTET STRING,
payable BOOLEAN
}
TypeInfoV1 ::= SEQUENCE {
typeHash OCTET STRING,
name OCTET STRING,
argType OCTET STRING,
outType OCTET STRING
}
TypeInfoV3 ::= SEQUENCE {
typeHash OCTET STRING,
name OCTET STRING,
payable BOOLEAN,
argType OCTET STRING,
outType OCTET STRING
}
-- ============================================================
-- Notes for implementers
-- ============================================================
-- 1. Detection (recommended decoder entry point):
--
-- decode(Binary) ->
-- case Binary of
-- <<B, _/binary>> when B >= 16#C0 ->
-- decode_legacy_rlp(Binary); % existing gmser_* path
-- _ ->
-- {ok, Term} =
-- 'GajumaruSerialization':decode('GajumaruData', Binary),
-- Term
-- end.
--
-- This works because:
-- - All current top-level output from serialize() is an RLP list
-- (first byte 0xC0-0xFF).
-- - GajumaruData and the concrete choices above are SEQUENCEs
-- (first byte 0x30 for short form, or 0x30 0x81/0x82... for long).
--
-- 2. When you add a new object type or version, prefer adding a
-- concrete SEQUENCE alternative in the Content CHOICE rather than
-- always falling back to TemplateFields. This gives you better
-- validation and generated types.
--
-- 3. For the compact wire format we use UPER (unaligned PER).
-- It is stable/deterministic for a given schema (no extensibility
-- markers on these types, fixed order).
--
-- 4. INTEGER uses ASN.1 PER encoding (canonical for the constraints).
-- This may differ from legacy RLP minimal unsigned; new format
-- will have different hashes (expected when introducing new encoding).
--
-- 5. Dynamic encoding (gmser_dyn) is not in scope here.
--
-- 6. To generate the compact wire:
-- asn1ct:compile(GajumaruSerialization, [uper]).
-- {ok, CompactBytes} = 'GajumaruSerialization':encode('GajumaruData', Value).
--
-- Use staticFields (not templateFields) for best compactness on static data.
END
Binary file not shown.
File diff suppressed because it is too large Load Diff
+82
View File
@@ -0,0 +1,82 @@
%% Generated by the Erlang ASN.1 compiler. Version: 5.4.3
%% Purpose: Erlang record definitions for each named and unnamed
%% SEQUENCE and SET, and macro definitions for each value
%% definition in module GajumaruSerialization.
-ifndef(_GAJUMARUSERIALIZATION_HRL_).
-define(_GAJUMARUSERIALIZATION_HRL_, true).
-record('GajumaruData', {
tag,
vsn,
content
}).
-record('CompactStatic', {
tag,
vsn,
fields
}).
-record('TemplateField', {
name = asn1_NOVALUE,
value
}).
-record('KeyValue', {
key,
val
}).
-record('Id', {
type,
value
}).
-record('Account', {
foo,
bar
}).
-record('SignedTx', {
signatures,
transaction
}).
-record('ContractV1', {
sourceHash,
typeInfo,
byteCode
}).
-record('ContractV2', {
sourceHash,
typeInfo,
byteCode,
compilerVersion
}).
-record('ContractV3', {
sourceHash,
typeInfo,
byteCode,
compilerVersion,
payable
}).
-record('TypeInfoV1', {
typeHash,
name,
argType,
outType
}).
-record('TypeInfoV3', {
typeHash,
name,
payable,
argType,
outType
}).
-endif. %% _GAJUMARUSERIALIZATION_HRL_
+235
View File
@@ -0,0 +1,235 @@
-- GajumaruSerialization.asn
--
-- ASN.1 model for the structures serialized by gmserialization.erl
-- (the layer on top of RLP).
--
-- Purpose:
-- * Provide a formal, toolable description of the data.
-- * Enable migration from the legacy RLP-based format to DER.
--
-- Detection of legacy vs new:
-- Legacy data (produced by gmserialization:serialize / gmser_chain_objects:serialize
-- or gmser_dyn:serialize) is always an RLP *list* at the top level.
-- This means the first byte is in the range 0xC0 .. 0xFF.
--
-- DER-encoded data using the types below will start with 0x30 (SEQUENCE,
-- constructed, universal tag) for the outermost GajumaruData (or any
-- top-level SEQUENCE we define). 0x30 < 0xC0, so a single-byte prefix
-- check reliably distinguishes the two formats.
--
-- Usage in Erlang (after compiling with asn1ct):
-- {ok, Mod} = asn1ct:compile(GajumaruSerialization.asn, [ber, der]).
-- {ok, Term} = 'GajumaruSerialization':decode('GajumaruData', DerBinary).
--
-- Notes on type mapping from gmserialization.erl:
-- - int -> INTEGER (new data uses canonical DER INTEGER)
-- - binary -> OCTET STRING
-- - bool -> BOOLEAN
-- - id -> Id (SEQUENCE) (cleaner than the legacy packed 33-byte form)
-- - [T] -> SEQUENCE OF T
-- - {T1,...} -> SEQUENCE (fixed-size, order matters)
-- - #{items := [{K,T},...]} -> SEQUENCE with the fields in template order
-- (static maps carry *values only*, no keys on the wire)
-- - tag + vsn are preserved as the first two fields for compatibility
-- with existing dispatch logic.
--
-- Legacy integers had strict unsigned-minimal no-leading-zero (except for 0)
-- and RLP-level rules. New DER data does not need to follow those rules.
--
-- WARNING:
-- Any cryptographic hash or signature computed over a serialized object
-- will change when switching from RLP to DER for that object. Plan a
-- coordinated upgrade.
GajumaruSerialization DEFINITIONS
AUTOMATIC TAGS ::=
BEGIN
EXPORTS ALL;
-- ============================================================
-- Top-level wrapper used for new DER data.
-- This is what a decoder will see first.
-- ============================================================
GajumaruData ::= SEQUENCE {
-- Constrained for better PER packing
tag INTEGER (0..65535),
vsn INTEGER (0..255),
content Content
}
-- Content can be a specific structured type (preferred) or a generic
-- representation of a template-driven object.
Content ::= CHOICE {
-- Generic fallback that can represent any [{Field, Type}] template
-- without having a pre-defined SEQUENCE for every object.
templateFields [0] TemplateFields,
-- Static-optimized: no field names on wire (matches legacy static behavior exactly)
-- Preferred for compact wire format of known templates.
staticFields [10] StaticFields,
-- Examples of concrete versioned types (extend as needed)
account [1] Account,
signedTx [2] SignedTx,
contract [3] ContractCode
-- Add more alternatives for other tags from gmser_chain_objects
}
-- ============================================================
-- Generic template-driven representation
-- (useful during transition or for unregistered types)
-- ============================================================
TemplateFields ::= SEQUENCE OF TemplateField
TemplateField ::= SEQUENCE {
-- Field name is included for debuggability / generic processing.
-- In the original static encoding the name is NOT on the wire;
-- only position and type matter. We include it here for convenience.
name IA5String OPTIONAL,
value Value
}
-- Optimized for static wire format: just the values in order, no names.
-- This matches the legacy static encoding where maps/records are positional only.
StaticFields ::= SEQUENCE OF Value
Value ::= CHOICE {
intValue [0] INTEGER,
boolValue [1] BOOLEAN,
binaryValue [2] OCTET STRING,
idValue [3] Id,
listValue [4] SEQUENCE OF Value,
tupleValue [5] SEQUENCE OF Value,
mapValue [6] SEQUENCE OF KeyValue -- only needed if you want to
-- represent dynamic-style maps
}
KeyValue ::= SEQUENCE {
key Value,
val Value
}
-- ============================================================
-- Common types
-- ============================================================
Id ::= SEQUENCE {
-- Corresponds to the simple tags in gmser_id (account=1, name=2, etc.)
-- and the extended account subtype (high bit in legacy).
type INTEGER (0..255),
value OCTET STRING (SIZE (32))
}
-- ============================================================
-- Concrete object examples (derived from usage in the codebase)
-- Add / evolve per version as you introduce new vsns.
-- ============================================================
-- Example: a very simple account-like object used in tests
Account ::= SEQUENCE {
foo INTEGER,
bar OCTET STRING
}
-- Simplified signed transaction
SignedTx ::= SEQUENCE {
signatures SEQUENCE OF OCTET STRING,
transaction OCTET STRING
}
-- Contract code objects (see gmser_contract_code.erl)
-- We model the three versions that exist today.
ContractCode ::= CHOICE {
v1 [0] ContractV1,
v2 [1] ContractV2,
v3 [2] ContractV3
}
ContractV1 ::= SEQUENCE {
sourceHash OCTET STRING,
-- typeInfo is a list of 4-tuples in legacy:
-- {typeHash, name, argType, outType}
typeInfo SEQUENCE OF TypeInfoV1,
byteCode OCTET STRING
}
ContractV2 ::= SEQUENCE {
sourceHash OCTET STRING,
typeInfo SEQUENCE OF TypeInfoV1,
byteCode OCTET STRING,
compilerVersion OCTET STRING
}
ContractV3 ::= SEQUENCE {
sourceHash OCTET STRING,
typeInfo SEQUENCE OF TypeInfoV3,
byteCode OCTET STRING,
compilerVersion OCTET STRING,
payable BOOLEAN
}
TypeInfoV1 ::= SEQUENCE {
typeHash OCTET STRING,
name OCTET STRING,
argType OCTET STRING,
outType OCTET STRING
}
TypeInfoV3 ::= SEQUENCE {
typeHash OCTET STRING,
name OCTET STRING,
payable BOOLEAN,
argType OCTET STRING,
outType OCTET STRING
}
-- ============================================================
-- Notes for implementers
-- ============================================================
-- 1. Detection (recommended decoder entry point):
--
-- decode(Binary) ->
-- case Binary of
-- <<B, _/binary>> when B >= 16#C0 ->
-- decode_legacy_rlp(Binary); % existing gmser_* path
-- _ ->
-- {ok, Term} =
-- 'GajumaruSerialization':decode('GajumaruData', Binary),
-- Term
-- end.
--
-- This works because:
-- - All current top-level output from serialize() is an RLP list
-- (first byte 0xC0-0xFF).
-- - GajumaruData and the concrete choices above are SEQUENCEs
-- (first byte 0x30 for short form, or 0x30 0x81/0x82... for long).
--
-- 2. When you add a new object type or version, prefer adding a
-- concrete SEQUENCE alternative in the Content CHOICE rather than
-- always falling back to TemplateFields. This gives you better
-- validation and generated types.
--
-- 3. INTEGER in DER is signed and uses a different minimal encoding
-- than the legacy unsigned big-endian no leading zero form. This is
-- fine for new data.
--
-- 4. If you need to preserve exact legacy integer wire bytes inside
-- the new format (e.g. for some hash preimage reason), you can
-- carry selected integers as OCTET STRING (legacy unsigned minimal bytes).
--
-- 5. Dynamic encoding (gmser_dyn) is intentionally not modeled here
-- in full, because it is runtime-schema driven (type codes 246-255,
-- labels, alt/switch, etc.). You can still use the generic
-- TemplateFields + Value for some dynamic cases, or model specific
-- message schemas as additional CHOICE arms.
--
-- 6. Compile with DER for canonical output:
-- asn1ct:compile(GajumaruSerialization, [der]).
--
-- The ber option is also accepted; der implies the stricter rules.
END
Binary file not shown.
File diff suppressed because it is too large Load Diff
+76
View File
@@ -0,0 +1,76 @@
%% Generated by the Erlang ASN.1 compiler. Version: 5.4.3
%% Purpose: Erlang record definitions for each named and unnamed
%% SEQUENCE and SET, and macro definitions for each value
%% definition in module GajumaruSerialization.
-ifndef(_GAJUMARUSERIALIZATION_HRL_).
-define(_GAJUMARUSERIALIZATION_HRL_, true).
-record('GajumaruData', {
tag,
vsn,
content
}).
-record('TemplateField', {
name = asn1_NOVALUE,
value
}).
-record('KeyValue', {
key,
val
}).
-record('Id', {
type,
value
}).
-record('Account', {
foo,
bar
}).
-record('SignedTx', {
signatures,
transaction
}).
-record('ContractV1', {
sourceHash,
typeInfo,
byteCode
}).
-record('ContractV2', {
sourceHash,
typeInfo,
byteCode,
compilerVersion
}).
-record('ContractV3', {
sourceHash,
typeInfo,
byteCode,
compilerVersion,
payable
}).
-record('TypeInfoV1', {
typeHash,
name,
argType,
outType
}).
-record('TypeInfoV3', {
typeHash,
name,
payable,
argType,
outType
}).
-endif. %% _GAJUMARUSERIALIZATION_HRL_
+225
View File
@@ -0,0 +1,225 @@
-- GajumaruSerialization.asn
--
-- ASN.1 model for the structures serialized by gmserialization.erl
-- (the layer on top of RLP).
--
-- Purpose:
-- * Provide a formal, toolable description of the data.
-- * Enable migration from the legacy RLP-based format to DER.
--
-- Detection of legacy vs new:
-- Legacy data (produced by gmserialization:serialize / gmser_chain_objects:serialize
-- or gmser_dyn:serialize) is always an RLP *list* at the top level.
-- This means the first byte is in the range 0xC0 .. 0xFF.
--
-- DER-encoded data using the types below will start with 0x30 (SEQUENCE,
-- constructed, universal tag) for the outermost GajumaruData (or any
-- top-level SEQUENCE we define). 0x30 < 0xC0, so a single-byte prefix
-- check reliably distinguishes the two formats.
--
-- Usage in Erlang (after compiling with asn1ct):
-- {ok, Mod} = asn1ct:compile(GajumaruSerialization.asn, [ber, der]).
-- {ok, Term} = 'GajumaruSerialization':decode('GajumaruData', DerBinary).
--
-- Notes on type mapping from gmserialization.erl:
-- - int -> INTEGER (new data uses canonical DER INTEGER)
-- - binary -> OCTET STRING
-- - bool -> BOOLEAN
-- - id -> Id (SEQUENCE) (cleaner than the legacy packed 33-byte form)
-- - [T] -> SEQUENCE OF T
-- - {T1,...} -> SEQUENCE (fixed-size, order matters)
-- - #{items := [{K,T},...]} -> SEQUENCE with the fields in template order
-- (static maps carry *values only*, no keys on the wire)
-- - tag + vsn are preserved as the first two fields for compatibility
-- with existing dispatch logic.
--
-- Legacy integers had strict unsigned-minimal no-leading-zero (except for 0)
-- and RLP-level rules. New DER data does not need to follow those rules.
--
-- WARNING:
-- Any cryptographic hash or signature computed over a serialized object
-- will change when switching from RLP to DER for that object. Plan a
-- coordinated upgrade.
GajumaruSerialization DEFINITIONS
AUTOMATIC TAGS ::=
BEGIN
EXPORTS ALL;
-- ============================================================
-- Top-level wrapper used for new DER data.
-- This is what a decoder will see first.
-- ============================================================
GajumaruData ::= SEQUENCE {
tag INTEGER,
vsn INTEGER,
content Content
}
-- Content can be a specific structured type (preferred) or a generic
-- representation of a template-driven object.
Content ::= CHOICE {
-- Generic fallback that can represent any [{Field, Type}] template
-- without having a pre-defined SEQUENCE for every object.
templateFields [0] TemplateFields,
-- Examples of concrete versioned types (extend as needed)
account [1] Account,
signedTx [2] SignedTx,
contract [3] ContractCode
-- Add more alternatives for other tags from gmser_chain_objects
}
-- ============================================================
-- Generic template-driven representation
-- (useful during transition or for unregistered types)
-- ============================================================
TemplateFields ::= SEQUENCE OF TemplateField
TemplateField ::= SEQUENCE {
-- Field name is included for debuggability / generic processing.
-- In the original static encoding the name is NOT on the wire;
-- only position and type matter. We include it here for convenience.
name IA5String OPTIONAL,
value Value
}
Value ::= CHOICE {
intValue [0] INTEGER,
boolValue [1] BOOLEAN,
binaryValue [2] OCTET STRING,
idValue [3] Id,
listValue [4] SEQUENCE OF Value,
tupleValue [5] SEQUENCE OF Value,
mapValue [6] SEQUENCE OF KeyValue -- only needed if you want to
-- represent dynamic-style maps
}
KeyValue ::= SEQUENCE {
key Value,
val Value
}
-- ============================================================
-- Common types
-- ============================================================
Id ::= SEQUENCE {
-- Corresponds to the simple tags in gmser_id (account=1, name=2, etc.)
-- and the extended account subtype (high bit in legacy).
type INTEGER (0..255),
value OCTET STRING (SIZE (32))
}
-- ============================================================
-- Concrete object examples (derived from usage in the codebase)
-- Add / evolve per version as you introduce new vsns.
-- ============================================================
-- Example: a very simple account-like object used in tests
Account ::= SEQUENCE {
foo INTEGER,
bar OCTET STRING
}
-- Simplified signed transaction
SignedTx ::= SEQUENCE {
signatures SEQUENCE OF OCTET STRING,
transaction OCTET STRING
}
-- Contract code objects (see gmser_contract_code.erl)
-- We model the three versions that exist today.
ContractCode ::= CHOICE {
v1 [0] ContractV1,
v2 [1] ContractV2,
v3 [2] ContractV3
}
ContractV1 ::= SEQUENCE {
sourceHash OCTET STRING,
-- typeInfo is a list of 4-tuples in legacy:
-- {typeHash, name, argType, outType}
typeInfo SEQUENCE OF TypeInfoV1,
byteCode OCTET STRING
}
ContractV2 ::= SEQUENCE {
sourceHash OCTET STRING,
typeInfo SEQUENCE OF TypeInfoV1,
byteCode OCTET STRING,
compilerVersion OCTET STRING
}
ContractV3 ::= SEQUENCE {
sourceHash OCTET STRING,
typeInfo SEQUENCE OF TypeInfoV3,
byteCode OCTET STRING,
compilerVersion OCTET STRING,
payable BOOLEAN
}
TypeInfoV1 ::= SEQUENCE {
typeHash OCTET STRING,
name OCTET STRING,
argType OCTET STRING,
outType OCTET STRING
}
TypeInfoV3 ::= SEQUENCE {
typeHash OCTET STRING,
name OCTET STRING,
payable BOOLEAN,
argType OCTET STRING,
outType OCTET STRING
}
-- ============================================================
-- Notes for implementers
-- ============================================================
-- 1. Detection (recommended decoder entry point):
--
-- decode(Binary) ->
-- case Binary of
-- <<B, _/binary>> when B >= 16#C0 ->
-- decode_legacy_rlp(Binary); % existing gmser_* path
-- _ ->
-- {ok, Term} =
-- 'GajumaruSerialization':decode('GajumaruData', Binary),
-- Term
-- end.
--
-- This works because:
-- - All current top-level output from serialize() is an RLP list
-- (first byte 0xC0-0xFF).
-- - GajumaruData and the concrete choices above are SEQUENCEs
-- (first byte 0x30 for short form, or 0x30 0x81/0x82... for long).
--
-- 2. When you add a new object type or version, prefer adding a
-- concrete SEQUENCE alternative in the Content CHOICE rather than
-- always falling back to TemplateFields. This gives you better
-- validation and generated types.
--
-- 3. INTEGER in DER is signed and uses a different minimal encoding
-- than the legacy unsigned big-endian no leading zero form. This is
-- fine for new data.
--
-- 4. If you need to preserve exact legacy integer wire bytes inside
-- the new format (e.g. for some hash preimage reason), you can
-- carry selected integers as OCTET STRING (legacy unsigned minimal bytes).
--
-- 5. Dynamic encoding (gmser_dyn) is intentionally not modeled here
-- in full, because it is runtime-schema driven (type codes 246-255,
-- labels, alt/switch, etc.). You can still use the generic
-- TemplateFields + Value for some dynamic cases, or model specific
-- message schemas as additional CHOICE arms.
--
-- 6. Compile with DER for canonical output:
-- asn1ct:compile(GajumaruSerialization, [der]).
--
-- The ber option is also accepted; der implies the stricter rules.
END
Binary file not shown.
File diff suppressed because it is too large Load Diff
+76
View File
@@ -0,0 +1,76 @@
%% Generated by the Erlang ASN.1 compiler. Version: 5.4.3
%% Purpose: Erlang record definitions for each named and unnamed
%% SEQUENCE and SET, and macro definitions for each value
%% definition in module GajumaruSerialization.
-ifndef(_GAJUMARUSERIALIZATION_HRL_).
-define(_GAJUMARUSERIALIZATION_HRL_, true).
-record('GajumaruData', {
tag,
vsn,
content
}).
-record('TemplateField', {
name = asn1_NOVALUE,
value
}).
-record('KeyValue', {
key,
val
}).
-record('Id', {
type,
value
}).
-record('Account', {
foo,
bar
}).
-record('SignedTx', {
signatures,
transaction
}).
-record('ContractV1', {
sourceHash,
typeInfo,
byteCode
}).
-record('ContractV2', {
sourceHash,
typeInfo,
byteCode,
compilerVersion
}).
-record('ContractV3', {
sourceHash,
typeInfo,
byteCode,
compilerVersion,
payable
}).
-record('TypeInfoV1', {
typeHash,
name,
argType,
outType
}).
-record('TypeInfoV3', {
typeHash,
name,
payable,
argType,
outType
}).
-endif. %% _GAJUMARUSERIALIZATION_HRL_
+225
View File
@@ -0,0 +1,225 @@
-- GajumaruSerialization.asn
--
-- ASN.1 model for the structures serialized by gmserialization.erl
-- (the layer on top of RLP).
--
-- Purpose:
-- * Provide a formal, toolable description of the data.
-- * Enable migration from the legacy RLP-based format to DER.
--
-- Detection of legacy vs new:
-- Legacy data (produced by gmserialization:serialize / gmser_chain_objects:serialize
-- or gmser_dyn:serialize) is always an RLP *list* at the top level.
-- This means the first byte is in the range 0xC0 .. 0xFF.
--
-- DER-encoded data using the types below will start with 0x30 (SEQUENCE,
-- constructed, universal tag) for the outermost GajumaruData (or any
-- top-level SEQUENCE we define). 0x30 < 0xC0, so a single-byte prefix
-- check reliably distinguishes the two formats.
--
-- Usage in Erlang (after compiling with asn1ct):
-- {ok, Mod} = asn1ct:compile(GajumaruSerialization.asn, [ber, der]).
-- {ok, Term} = 'GajumaruSerialization':decode('GajumaruData', DerBinary).
--
-- Notes on type mapping from gmserialization.erl:
-- - int -> INTEGER (new data uses canonical DER INTEGER)
-- - binary -> OCTET STRING
-- - bool -> BOOLEAN
-- - id -> Id (SEQUENCE) (cleaner than the legacy packed 33-byte form)
-- - [T] -> SEQUENCE OF T
-- - {T1,...} -> SEQUENCE (fixed-size, order matters)
-- - #{items := [{K,T},...]} -> SEQUENCE with the fields in template order
-- (static maps carry *values only*, no keys on the wire)
-- - tag + vsn are preserved as the first two fields for compatibility
-- with existing dispatch logic.
--
-- Legacy integers had strict unsigned-minimal no-leading-zero (except for 0)
-- and RLP-level rules. New DER data does not need to follow those rules.
--
-- WARNING:
-- Any cryptographic hash or signature computed over a serialized object
-- will change when switching from RLP to DER for that object. Plan a
-- coordinated upgrade.
GajumaruSerialization DEFINITIONS
AUTOMATIC TAGS ::=
BEGIN
EXPORTS ALL;
-- ============================================================
-- Top-level wrapper used for new DER data.
-- This is what a decoder will see first.
-- ============================================================
GajumaruData ::= SEQUENCE {
tag INTEGER,
vsn INTEGER,
content Content
}
-- Content can be a specific structured type (preferred) or a generic
-- representation of a template-driven object.
Content ::= CHOICE {
-- Generic fallback that can represent any [{Field, Type}] template
-- without having a pre-defined SEQUENCE for every object.
templateFields [0] TemplateFields,
-- Examples of concrete versioned types (extend as needed)
account [1] Account,
signedTx [2] SignedTx,
contract [3] ContractCode
-- Add more alternatives for other tags from gmser_chain_objects
}
-- ============================================================
-- Generic template-driven representation
-- (useful during transition or for unregistered types)
-- ============================================================
TemplateFields ::= SEQUENCE OF TemplateField
TemplateField ::= SEQUENCE {
-- Field name is included for debuggability / generic processing.
-- In the original static encoding the name is NOT on the wire;
-- only position and type matter. We include it here for convenience.
name IA5String OPTIONAL,
value Value
}
Value ::= CHOICE {
intValue [0] INTEGER,
boolValue [1] BOOLEAN,
binaryValue [2] OCTET STRING,
idValue [3] Id,
listValue [4] SEQUENCE OF Value,
tupleValue [5] SEQUENCE OF Value,
mapValue [6] SEQUENCE OF KeyValue -- only needed if you want to
-- represent dynamic-style maps
}
KeyValue ::= SEQUENCE {
key Value,
val Value
}
-- ============================================================
-- Common types
-- ============================================================
Id ::= SEQUENCE {
-- Corresponds to the simple tags in gmser_id (account=1, name=2, etc.)
-- and the extended account subtype (high bit in legacy).
type INTEGER (0..255),
value OCTET STRING (SIZE (32))
}
-- ============================================================
-- Concrete object examples (derived from usage in the codebase)
-- Add / evolve per version as you introduce new vsns.
-- ============================================================
-- Example: a very simple account-like object used in tests
Account ::= SEQUENCE {
foo INTEGER,
bar OCTET STRING
}
-- Simplified signed transaction
SignedTx ::= SEQUENCE {
signatures SEQUENCE OF OCTET STRING,
transaction OCTET STRING
}
-- Contract code objects (see gmser_contract_code.erl)
-- We model the three versions that exist today.
ContractCode ::= CHOICE {
v1 [0] ContractV1,
v2 [1] ContractV2,
v3 [2] ContractV3
}
ContractV1 ::= SEQUENCE {
sourceHash OCTET STRING,
-- typeInfo is a list of 4-tuples in legacy:
-- {typeHash, name, argType, outType}
typeInfo SEQUENCE OF TypeInfoV1,
byteCode OCTET STRING
}
ContractV2 ::= SEQUENCE {
sourceHash OCTET STRING,
typeInfo SEQUENCE OF TypeInfoV1,
byteCode OCTET STRING,
compilerVersion OCTET STRING
}
ContractV3 ::= SEQUENCE {
sourceHash OCTET STRING,
typeInfo SEQUENCE OF TypeInfoV3,
byteCode OCTET STRING,
compilerVersion OCTET STRING,
payable BOOLEAN
}
TypeInfoV1 ::= SEQUENCE {
typeHash OCTET STRING,
name OCTET STRING,
argType OCTET STRING,
outType OCTET STRING
}
TypeInfoV3 ::= SEQUENCE {
typeHash OCTET STRING,
name OCTET STRING,
payable BOOLEAN,
argType OCTET STRING,
outType OCTET STRING
}
-- ============================================================
-- Notes for implementers
-- ============================================================
-- 1. Detection (recommended decoder entry point):
--
-- decode(Binary) ->
-- case Binary of
-- <<B, _/binary>> when B >= 16#C0 ->
-- decode_legacy_rlp(Binary); % existing gmser_* path
-- _ ->
-- {ok, Term} =
-- 'GajumaruSerialization':decode('GajumaruData', Binary),
-- Term
-- end.
--
-- This works because:
-- - All current top-level output from serialize() is an RLP list
-- (first byte 0xC0-0xFF).
-- - GajumaruData and the concrete choices above are SEQUENCEs
-- (first byte 0x30 for short form, or 0x30 0x81/0x82... for long).
--
-- 2. When you add a new object type or version, prefer adding a
-- concrete SEQUENCE alternative in the Content CHOICE rather than
-- always falling back to TemplateFields. This gives you better
-- validation and generated types.
--
-- 3. INTEGER in DER is signed and uses a different minimal encoding
-- than the legacy unsigned big-endian no leading zero form. This is
-- fine for new data.
--
-- 4. If you need to preserve exact legacy integer wire bytes inside
-- the new format (e.g. for some hash preimage reason), you can
-- carry selected integers as OCTET STRING (legacy unsigned minimal bytes).
--
-- 5. Dynamic encoding (gmser_dyn) is intentionally not modeled here
-- in full, because it is runtime-schema driven (type codes 246-255,
-- labels, alt/switch, etc.). You can still use the generic
-- TemplateFields + Value for some dynamic cases, or model specific
-- message schemas as additional CHOICE arms.
--
-- 6. Compile with DER for canonical output:
-- asn1ct:compile(GajumaruSerialization, [der]).
--
-- The ber option is also accepted; der implies the stricter rules.
END
Binary file not shown.
File diff suppressed because it is too large Load Diff
+76
View File
@@ -0,0 +1,76 @@
%% Generated by the Erlang ASN.1 compiler. Version: 5.4.3
%% Purpose: Erlang record definitions for each named and unnamed
%% SEQUENCE and SET, and macro definitions for each value
%% definition in module GajumaruSerialization.
-ifndef(_GAJUMARUSERIALIZATION_HRL_).
-define(_GAJUMARUSERIALIZATION_HRL_, true).
-record('GajumaruData', {
tag,
vsn,
content
}).
-record('TemplateField', {
name = asn1_NOVALUE,
value
}).
-record('KeyValue', {
key,
val
}).
-record('Id', {
type,
value
}).
-record('Account', {
foo,
bar
}).
-record('SignedTx', {
signatures,
transaction
}).
-record('ContractV1', {
sourceHash,
typeInfo,
byteCode
}).
-record('ContractV2', {
sourceHash,
typeInfo,
byteCode,
compilerVersion
}).
-record('ContractV3', {
sourceHash,
typeInfo,
byteCode,
compilerVersion,
payable
}).
-record('TypeInfoV1', {
typeHash,
name,
argType,
outType
}).
-record('TypeInfoV3', {
typeHash,
name,
payable,
argType,
outType
}).
-endif. %% _GAJUMARUSERIALIZATION_HRL_
+70
View File
@@ -0,0 +1,70 @@
# ASN.1 for Static Serialization - Compact Wire Format
## Goal
Use portable ASN.1 techniques to produce the most compact *deterministic* (stable/idempotent for hashing) wire format for gmserialization **static** encoding, based on existing templates.
Focus is on the wire format itself (not RLP translation for legacy, which is deferred).
## Approach
- The `asn1/GajumaruSerialization.asn` is the single source of truth (abstract syntax).
- Use **Unaligned PER (UPER)** as the standard compact canonical encoding rule provided by the ASN.1 framework.
- Portable across languages/tools that support ASN.1 UPER.
- Deterministic for a fixed schema (no extensibility, consistent packing).
- Optimize the schema for packing:
- Constrain INTEGER ranges.
- Provide `staticFields` (SEQUENCE OF Value) -- no field names (names are never on the wire for static case).
- Provide `CompactStatic` top-level type to avoid unnecessary CHOICE overhead for the common static path.
- Concrete SEQUENCEs for well-known objects (SignedTx, ContractV* etc.) when possible.
- Encode with the generated ASN.1 module: ` 'GajumaruSerialization':encode('CompactStatic', Value) `.
## Results (example sizes)
Using current optimized UPER:
- Tiny object (tag/vsn + int + 1-byte bin): 9 bytes (legacy RLP = 5)
- List of 3 ints: 13 bytes (legacy = 7)
- Signed tx example: 11 bytes (legacy = 7)
- 256-byte payload: ~263 bytes (legacy ~264) -- matches or slightly better
PER/UPER overhead is mainly the structural tags for the generic case. Concrete types and `staticFields` + `CompactStatic` minimize it.
Compared to DER (previous orientation): dramatically better (e.g. tiny case was ~36B in DER).
## Usage in Erlang (for the compact format)
```erlang
% Build value according to schema (using staticFields for best compactness)
Value = {'CompactStatic', Tag, Vsn, [
{'intValue', 42},
{'binaryValue', <<"data">>}
% ...
]},
{ok, CompactBytes} = 'GajumaruSerialization':encode('CompactStatic', Value).
```
Compile the schema with:
```
asn1ct:compile("GajumaruSerialization.asn", [uper]).
```
## Schema Notes
- `staticFields` should be used for generic static templates (mirrors legacy positional encoding).
- Concrete types (e.g. `signedTx`) are preferred when the structure is fixed.
- `TemplateFields` (with names) is kept for debug/transition but not optimal for wire size.
- The model directly reflects the static `template()` types from `gmserialization.erl`.
## Portability
Any language with an ASN.1 UPER codec can produce and consume the exact same bytes by using the schema and the same value construction rules.
## Stability
- UPER encoding of this schema is stable (tested roundtrip + re-encode identical).
- No random/padding choices.
- Same input value always produces identical bytes.
## Limitations / Future
- For very small objects, hand-crafted RLP is still smaller because it has almost no structural overhead.
- If an even more compact custom encoding is desired while keeping the model, a custom "encoding rule" can be implemented driven by the schema (similar to how the RLP layer works, but targeting a new bit-packed format).
- Dynamic encoder (gmser_dyn) is out of scope.
See also: `asn1/GajumaruSerialization.asn`, `asn1_compact/`, tests in `src/gmser_asn1_rlp.erl` (for value shapes), `doc/static.md`.
+84
View File
@@ -0,0 +1,84 @@
# Static Serialization
The `gmserialization` and `gmser_chain_objects` modules implement the
static serialization support used in the Gajumaru blockchain.
The purpose is to produce fully deterministic serialization, in order
to maintain predictable hashing.
Example:
```erlang
%% deterministic canonical serialization.
-spec serialize_to_binary(signed_tx()) -> binary_signed_tx().
serialize_to_binary(#signed_tx{tx = Tx, signatures = Sigs}) ->
gmser_chain_objects:serialize(
?SIG_TX_TYPE,
?SIG_TX_VSN,
serialization_template(?SIG_TX_VSN),
[ {signatures, lists:sort(Sigs)}
, {transaction, aetx:serialize_to_binary(Tx)}
]).
-spec deserialize_from_binary(binary()) -> signed_tx().
deserialize_from_binary(SignedTxBin) when is_binary(SignedTxBin) ->
[ {signatures, Sigs}
, {transaction, TxBin}
] = gmser_chain_objects:deserialize(
?SIG_TX_TYPE,
?SIG_TX_VSN,
serialization_template(?SIG_TX_VSN),
SignedTxBin),
assert_sigs_size(Sigs),
#signed_tx{ tx = aetx:deserialize_from_binary(TxBin)
, signatures = Sigs
}.
serialization_template(?SIG_TX_VSN) ->
[ {signatures, [binary]}
, {transaction, binary}
].
```
The terms that can be encoded using these templates are given by
this type in `gmserialization.erl`:
```erlang
-type encodable_term() :: non_neg_integer()
| binary()
| boolean()
| [encodable_term()] %% Of any length
| #{atom() => encodable_term()}
| tuple() %% Any arity, containing encodable_term().
| gmser_id:id().
```
The template 'language' is defined by these types:
```erlang
-type template() :: [{field_name(), type()}].
-type field_name() :: atom().
-type type() :: 'int' % bignum (non-negative, for amounts etc. up to 10^30 Pucks)
| 'uint128' | 'uint64' | 'uint32' | 'uint16' | 'uint8'
| 'bool'
| 'binary'
| 'id' %% As defined in aec_id.erl
| [type()] %% Length one in the type. This means a list of any length.
| #{items := [{field_name(), type()}]} %% Record with named fields
%% represented as a map.
%% Encoded as a list in the given
%% order.
| tuple(). %% Any arity, containing type(). This means a static size array.
```
The `gmser_chain_objects.erl` module specifies a serialization code for each
object that can go on-chain. E.g.:
```erlang
tag(signed_tx) -> 11;
...
rev_tag(11) -> signed_tx;
```
The `tag` and `vsn` are laid out in the beginning of the serialized object.
+11
View File
@@ -0,0 +1,11 @@
{application,gmserialization,
[{description,"Serialization of data for the Gajumaru"},
{vsn,"0.2.0"},
{registered,[]},
{applications,[kernel,stdlib,crypto,base58]},
{env,[]},
{modules,[gmser_api_encoder,gmser_chain_objects,
gmser_contract_code,gmser_delegation,gmser_id,
gmser_rlp,gmserialization]},
{licenses,[]},
{links,[]}]}.
+15 -2
View File
@@ -1,4 +1,17 @@
{erl_opts, [debug_info]}. {erl_opts, [debug_info]}.
{deps, [ {base58, {git, "https://github.com/aeternity/erl-base58.git", {ref, "60a3356"}}} {deps,
, {enacl, {git, "https://github.com/aeternity/enacl.git", {ref, "793ddb5"}}} [{base58,
{git,
"https://git.qpq.swiss/QPQ-AG/erl-base58.git",
{ref, "e6aa62eeae3d4388311401f06e4b939bf4e94b9c"}}},
{enacl,
{git,
"https://git.qpq.swiss/QPQ-AG/enacl.git",
{ref, "4eb7ec70084ba7c87b1af8797c4c4e90c84f95a2"}}},
{eblake2, "1.0.0"}
]}.
{dialyzer,
[ {plt_apps, all_deps},
{base_plt_apps, [erts, kernel, stdlib, enacl, base58, eblake2]}
]}. ]}.
+16
View File
@@ -0,0 +1,16 @@
{"1.2.0",
[{<<"base58">>,
{git,"https://git.qpq.swiss/QPQ-AG/erl-base58.git",
{ref,"e6aa62eeae3d4388311401f06e4b939bf4e94b9c"}},
0},
{<<"eblake2">>,{pkg,<<"eblake2">>,<<"1.0.0">>},0},
{<<"enacl">>,
{git,"https://git.qpq.swiss/QPQ-AG/enacl.git",
{ref,"4eb7ec70084ba7c87b1af8797c4c4e90c84f95a2"}},
0}]}.
[
{pkg_hash,[
{<<"eblake2">>, <<"EC8AD20E438AAB3F2E8D5D118C366A0754219195F8A0F536587440F8F9BCF2EF">>}]},
{pkg_hash_ext,[
{<<"eblake2">>, <<"3C4D300A91845B25D501929A26AC2E6F7157480846FAB2347A4C11AE52E08A99">>}]}
].
Executable
BIN
View File
Binary file not shown.
-111
View File
@@ -1,111 +0,0 @@
%%% -*- erlang-indent-level:4; indent-tabs-mode: nil -*-
%%%-------------------------------------------------------------------
%%% @copyright (C) 2018, Aeternity Anstalt
%%% @doc
%%% ADT for identifiers
%%% @end
%%%-------------------------------------------------------------------
-module(aeser_id).
-export([ create/2
, specialize/1
, specialize/2
, specialize_type/1
, is_id/1
]).
%% For aec_serialization
-export([ encode/1
, decode/1
]).
-record(id, { tag
, val
}).
-type tag() :: 'account' | 'oracle' | 'name'
| 'commitment' | 'contract' | 'channel'.
-type val() :: <<_:256>>.
-opaque(id() :: #id{}).
-export_type([ id/0
, tag/0
, val/0
]).
-define(PUB_SIZE, 32).
-define(TAG_SIZE, 1).
-define(SERIALIZED_SIZE, 33). %% ?TAG_SIZE + ?PUB_SIZE
-define(IS_TAG(___TAG___), ___TAG___ =:= account;
___TAG___ =:= oracle;
___TAG___ =:= name;
___TAG___ =:= commitment;
___TAG___ =:= contract;
___TAG___ =:= channel
).
-define(IS_VAL(___VAL___), byte_size(___VAL___) =:= 32).
%%%===================================================================
%%% API
%%%===================================================================
-spec create(tag(), val()) -> id().
create(Tag, Val) when ?IS_TAG(Tag), ?IS_VAL(Val) ->
#id{ tag = Tag
, val = Val};
create(Tag, Val) when ?IS_VAL(Val) ->
error({illegal_tag, Tag});
create(Tag, Val) when ?IS_TAG(Tag)->
error({illegal_val, Val});
create(Tag, Val) ->
error({illegal_tag_and_val, Tag, Val}).
-spec specialize(id()) -> {tag(), val()}.
specialize(#id{tag = Tag, val = Val}) ->
{Tag, Val}.
-spec specialize(id(), tag()) -> val().
specialize(#id{tag = Tag, val = Val}, Tag) when ?IS_TAG(Tag), ?IS_VAL(Val) ->
Val.
-spec specialize_type(id()) -> tag().
specialize_type(#id{tag = Tag}) when ?IS_TAG(Tag) ->
Tag.
-spec is_id(term()) -> boolean().
is_id(#id{}) -> true;
is_id(_) -> false.
-spec encode(id()) -> binary().
encode(#id{tag = Tag, val = Val}) ->
Res = <<(encode_tag(Tag)):?TAG_SIZE/unit:8, Val/binary>>,
true = ?SERIALIZED_SIZE =:= byte_size(Res),
Res.
-spec decode(binary()) -> id().
decode(<<Tag:?TAG_SIZE/unit:8, Val:?PUB_SIZE/binary>>) ->
#id{ tag = decode_tag(Tag)
, val = Val}.
%%%===================================================================
%%% Internal functions
%%%===================================================================
encode_tag(account) -> 1;
encode_tag(name) -> 2;
encode_tag(commitment) -> 3;
encode_tag(oracle) -> 4;
encode_tag(contract) -> 5;
encode_tag(channel) -> 6;
encode_tag(Other) -> error({illegal_id_tag_name, Other}).
decode_tag(1) -> account;
decode_tag(2) -> name;
decode_tag(3) -> commitment;
decode_tag(4) -> oracle;
decode_tag(5) -> contract;
decode_tag(6) -> channel;
decode_tag(X) -> error({illegal_id_tag, X}).
@@ -1,43 +1,56 @@
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
%%% @copyright (C) 2025, QPQ AG
%%% @copyright (C) 2017, Aeternity Anstalt %%% @copyright (C) 2017, Aeternity Anstalt
%%% @doc %%% @doc
%%% API encoding for the Aeternity node. %%% API encoding for the Gajumaru
%%% @end %%% @end
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
-module(aeser_api_encoder). -module(gmser_api_encoder).
-vsn("0.2.0").
-export([encode/2, -export([encode/2,
decode/1, decode/1,
safe_decode/2, safe_decode/2,
byte_size_for_type/1]). byte_size_for_type/1]).
-export_type([encoded/0]). -export([encode_keypair/1,
safe_decode_keypair/1]).
-export([unsafe_encode/2]). %% Encode without size checks
-export_type([encoded/0,
known_type/0]).
-type known_type() :: key_block_hash -type known_type() :: key_block_hash
| micro_block_hash | micro_block_hash
| block_pof_hash | block_pof_hash
| block_tx_hash | block_tx_hash
| block_state_hash | block_state_hash
| block_witness_hash
| channel | channel
| contract_bytearray | contract_bytearray
| contract_pubkey | contract_pubkey
| contract_store_key | contract_store_key
| contract_store_value | contract_store_value
| contract_source
| transaction | transaction
| tx_hash | tx_hash
| oracle_pubkey
| oracle_query
| oracle_query_id
| oracle_response
| account_pubkey | account_pubkey
| {account_pubkey, 0..6}
| account_seckey
| associate_chain
| entry
| signature | signature
| name | name
| native_token
| commitment | commitment
| peer_pubkey | peer_pubkey
| hash
| state | state
| poi | poi
| state_trees | state_trees
| call_state_tree | call_state_tree
| mp_tree_hash
| bytearray. | bytearray.
-type extended_type() :: known_type() | block_hash | {id_hash, [known_type()]}. -type extended_type() :: known_type() | block_hash | {id_hash, [known_type()]}.
@@ -46,14 +59,70 @@
-type payload() :: binary(). -type payload() :: binary().
-type encoded() :: binary(). -type encoded() :: binary().
-type keypair() :: #{public := <<_:(32*8)>>, secret := <<_:(64*8)>>}.
-type encoded_keypair() :: #{binary() => binary()}.
-export_type([ keypair/0
, encoded_keypair/0 ]).
-define(BASE58, 1). -define(BASE58, 1).
-define(BASE64, 2). -define(BASE64, 2).
-spec encode(known_type(), payload() | aeser_id:id()) -> encoded(). -spec encode_keypair(keypair()) -> encoded_keypair().
encode_keypair(#{public := Pub, secret := Sec}) ->
case Sec of
<<Seed:32/binary, Pub1:32/binary>> when Pub1 =:= Pub ->
#{ <<"pub">> => encode(account_pubkey, Pub)
, <<"priv">> => encode(account_seckey, Seed) };
_ ->
erlang:error(invalid_keypair)
end.
-spec safe_decode_keypair(encoded_keypair()) -> {'ok', keypair()} | {'error', atom()}.
safe_decode_keypair(#{<<"pub">> := EncPub, <<"priv">> := EncPriv}) ->
case safe_decode(account_pubkey, EncPub) of
{ok, Pub} ->
case safe_decode(account_seckey, EncPriv) of
{ok, Seed} when byte_size(Seed) =:= 32 ->
case enacl:sign_seed_keypair(Seed) of
#{public := Pub, secret := _} = KP ->
{ok, KP};
_ ->
{error, illegal_encoding}
end;
{ok, <<Seed:32/binary, Pub:32/binary>>} ->
case enacl:sign_seed_keypair(Seed) of
#{public := Pub} = KP ->
{ok, KP};
_ ->
{error, illegal_encoding}
end;
{ok, _} ->
{error, illegal_encoding};
{error, _} = Error1 ->
Error1
end;
Error ->
Error
end.
-spec encode(known_type(), payload() | gmser_id:id()) -> encoded().
encode(id_hash, Payload) -> encode(id_hash, Payload) ->
{IdType, Val} = aeser_id:specialize(Payload), case gmser_id:to_map(Payload) of
encode(id2type(IdType), Val); #{type := account, subtype := SubT, value := Val} ->
encode({account_pubkey, SubT}, Val);
#{type := IdType, value := Val} ->
encode(id2type(IdType), Val)
end;
encode(Type, Payload) -> encode(Type, Payload) ->
case type_size_check(Type, Payload) of
ok ->
unsafe_encode(Type, Payload);
{error, Reason} ->
erlang:error(Reason)
end.
unsafe_encode(Type, Payload) ->
Pfx = type2pfx(Type), Pfx = type2pfx(Type),
Enc = case type2enc(Type) of Enc = case type2enc(Type) of
?BASE58 -> base58_check(Payload); ?BASE58 -> base58_check(Payload);
@@ -61,6 +130,7 @@ encode(Type, Payload) ->
end, end,
<<Pfx/binary, "_", Enc/binary>>. <<Pfx/binary, "_", Enc/binary>>.
-spec decode(binary()) -> {known_type(), payload()}. -spec decode(binary()) -> {known_type(), payload()}.
decode(Bin0) -> decode(Bin0) ->
case split(Bin0) of case split(Bin0) of
@@ -76,6 +146,13 @@ decode(Bin0) ->
erlang:error(missing_prefix) erlang:error(missing_prefix)
end. end.
type_size_check(account_seckey, Bin) ->
case byte_size(Bin) of
Sz when Sz =:= 32; Sz =:= 64 ->
ok;
_ ->
{error, incorrect_size}
end;
type_size_check(Type, Bin) -> type_size_check(Type, Bin) ->
case byte_size_for_type(Type) of case byte_size_for_type(Type) of
not_applicable -> ok; not_applicable -> ok;
@@ -87,14 +164,14 @@ type_size_check(Type, Bin) ->
end end
end. end.
-spec safe_decode(extended_type(), encoded()) -> {'ok', payload() | aeser_id:id()} -spec safe_decode(extended_type(), encoded()) -> {'ok', payload() | gmser_id:id()}
| {'error', atom()}. | {'error', atom()}.
safe_decode({id_hash, AllowedTypes}, Enc) -> safe_decode({id_hash, AllowedTypes}, Enc) ->
try decode(Enc) of try decode(Enc) of
{ActualType, Dec} -> {ActualType, Dec} ->
case lists:member(ActualType, AllowedTypes) of case lists:member(ActualType, AllowedTypes) of
true -> true ->
try {ok, aeser_id:create(type2id(ActualType), Dec)} try {ok, gmser_id:create(type2id(ActualType), Dec)}
catch error:_ -> {error, invalid_prefix} catch error:_ -> {error, invalid_prefix}
end; end;
false -> false ->
@@ -161,44 +238,54 @@ sha256_hash(Bin) ->
id2type(account) -> account_pubkey; id2type(account) -> account_pubkey;
id2type(associate_chain) -> associate_chain;
id2type(channel) -> channel; id2type(channel) -> channel;
id2type(commitment) -> commitment; id2type(commitment) -> commitment;
id2type(contract) -> contract_pubkey; id2type(contract) -> contract_pubkey;
id2type(name) -> name; id2type(name) -> name;
id2type(oracle) -> oracle_pubkey. id2type(native_token) -> native_token;
id2type(entry) -> entry.
type2id({account_pubkey, SubT}) -> {account, SubT};
type2id(account_pubkey) -> account; type2id(account_pubkey) -> account;
type2id(associate_chain) -> associate_chain;
type2id(channel) -> channel; type2id(channel) -> channel;
type2id(commitment) -> commitment; type2id(commitment) -> commitment;
type2id(contract_pubkey) -> contract; type2id(contract_pubkey) -> contract;
type2id(name) -> name; type2id(name) -> name;
type2id(oracle_pubkey) -> oracle. type2id(native_token) -> native_token;
type2id(entry) -> entry.
type2enc(key_block_hash) -> ?BASE58; type2enc(key_block_hash) -> ?BASE58;
type2enc(micro_block_hash) -> ?BASE58; type2enc(micro_block_hash) -> ?BASE58;
type2enc(block_pof_hash) -> ?BASE58; type2enc(block_pof_hash) -> ?BASE58;
type2enc(block_tx_hash) -> ?BASE58; type2enc(block_tx_hash) -> ?BASE58;
type2enc(block_state_hash) -> ?BASE58; type2enc(block_state_hash) -> ?BASE58;
type2enc(block_witness_hash) -> ?BASE58;
type2enc(channel) -> ?BASE58; type2enc(channel) -> ?BASE58;
type2enc(contract_pubkey) -> ?BASE58; type2enc(contract_pubkey) -> ?BASE58;
type2enc(contract_bytearray) -> ?BASE64; type2enc(contract_bytearray) -> ?BASE64;
type2enc(contract_store_key) -> ?BASE64; type2enc(contract_store_key) -> ?BASE64;
type2enc(contract_store_value) -> ?BASE64; type2enc(contract_store_value) -> ?BASE64;
type2enc(contract_source) -> ?BASE64;
type2enc(transaction) -> ?BASE64; type2enc(transaction) -> ?BASE64;
type2enc(tx_hash) -> ?BASE58; type2enc(tx_hash) -> ?BASE58;
type2enc(oracle_pubkey) -> ?BASE58; type2enc({account_pubkey, _}) -> ?BASE58;
type2enc(oracle_query) -> ?BASE64;
type2enc(oracle_query_id) -> ?BASE58;
type2enc(oracle_response) -> ?BASE64;
type2enc(account_pubkey) -> ?BASE58; type2enc(account_pubkey) -> ?BASE58;
type2enc(account_seckey) -> ?BASE58;
type2enc(associate_chain) -> ?BASE58;
type2enc(signature) -> ?BASE58; type2enc(signature) -> ?BASE58;
type2enc(commitment) -> ?BASE58; type2enc(commitment) -> ?BASE58;
type2enc(peer_pubkey) -> ?BASE58; type2enc(peer_pubkey) -> ?BASE58;
type2enc(name) -> ?BASE58; type2enc(name) -> ?BASE58;
type2enc(native_token) -> ?BASE58;
type2enc(state) -> ?BASE64; type2enc(state) -> ?BASE64;
type2enc(poi) -> ?BASE64; type2enc(poi) -> ?BASE64;
type2enc(state_trees) -> ?BASE64; type2enc(state_trees) -> ?BASE64;
type2enc(call_state_tree) -> ?BASE64; type2enc(call_state_tree) -> ?BASE64;
type2enc(mp_tree_hash) -> ?BASE58;
type2enc(hash) -> ?BASE58;
type2enc(entry) -> ?BASE64;
type2enc(bytearray) -> ?BASE64. type2enc(bytearray) -> ?BASE64.
@@ -207,26 +294,37 @@ type2pfx(micro_block_hash) -> <<"mh">>;
type2pfx(block_pof_hash) -> <<"bf">>; type2pfx(block_pof_hash) -> <<"bf">>;
type2pfx(block_tx_hash) -> <<"bx">>; type2pfx(block_tx_hash) -> <<"bx">>;
type2pfx(block_state_hash) -> <<"bs">>; type2pfx(block_state_hash) -> <<"bs">>;
type2pfx(block_witness_hash) -> <<"ws">>;
type2pfx(channel) -> <<"ch">>; type2pfx(channel) -> <<"ch">>;
type2pfx(contract_pubkey) -> <<"ct">>; type2pfx(contract_pubkey) -> <<"ct">>;
type2pfx(contract_bytearray) -> <<"cb">>; type2pfx(contract_bytearray) -> <<"cb">>;
type2pfx(contract_store_key) -> <<"ck">>; type2pfx(contract_store_key) -> <<"ck">>;
type2pfx(contract_store_value) -> <<"cv">>; type2pfx(contract_store_value) -> <<"cv">>;
type2pfx(contract_source) -> <<"cx">>;
type2pfx(transaction) -> <<"tx">>; type2pfx(transaction) -> <<"tx">>;
type2pfx(tx_hash) -> <<"th">>; type2pfx(tx_hash) -> <<"th">>;
type2pfx(oracle_pubkey) -> <<"ok">>; type2pfx({account_pubkey,0}) -> <<"a0">>;
type2pfx(oracle_query) -> <<"ov">>; type2pfx({account_pubkey,1}) -> <<"a1">>;
type2pfx(oracle_query_id) -> <<"oq">>; type2pfx({account_pubkey,2}) -> <<"a2">>;
type2pfx(oracle_response) -> <<"or">>; type2pfx({account_pubkey,3}) -> <<"a3">>;
type2pfx({account_pubkey,4}) -> <<"a4">>;
type2pfx({account_pubkey,5}) -> <<"a5">>;
type2pfx({account_pubkey,6}) -> <<"a6">>;
type2pfx(account_pubkey) -> <<"ak">>; type2pfx(account_pubkey) -> <<"ak">>;
type2pfx(account_seckey) -> <<"sk">>;
type2pfx(associate_chain) -> <<"ac">>;
type2pfx(signature) -> <<"sg">>; type2pfx(signature) -> <<"sg">>;
type2pfx(commitment) -> <<"cm">>; type2pfx(commitment) -> <<"cm">>;
type2pfx(peer_pubkey) -> <<"pp">>; type2pfx(peer_pubkey) -> <<"pp">>;
type2pfx(name) -> <<"nm">>; type2pfx(name) -> <<"nm">>;
type2pfx(native_token) -> <<"nt">>;
type2pfx(state) -> <<"st">>; type2pfx(state) -> <<"st">>;
type2pfx(poi) -> <<"pi">>; type2pfx(poi) -> <<"pi">>;
type2pfx(state_trees) -> <<"ss">>; type2pfx(state_trees) -> <<"ss">>;
type2pfx(call_state_tree) -> <<"cs">>; type2pfx(call_state_tree) -> <<"cs">>;
type2pfx(mp_tree_hash) -> <<"mt">>;
type2pfx(hash) -> <<"hs">>;
type2pfx(entry) -> <<"en">>;
type2pfx(bytearray) -> <<"ba">>. type2pfx(bytearray) -> <<"ba">>.
pfx2type(<<"kh">>) -> key_block_hash; pfx2type(<<"kh">>) -> key_block_hash;
@@ -234,26 +332,37 @@ pfx2type(<<"mh">>) -> micro_block_hash;
pfx2type(<<"bf">>) -> block_pof_hash; pfx2type(<<"bf">>) -> block_pof_hash;
pfx2type(<<"bx">>) -> block_tx_hash; pfx2type(<<"bx">>) -> block_tx_hash;
pfx2type(<<"bs">>) -> block_state_hash; pfx2type(<<"bs">>) -> block_state_hash;
pfx2type(<<"ws">>) -> block_witness_hash;
pfx2type(<<"ch">>) -> channel; pfx2type(<<"ch">>) -> channel;
pfx2type(<<"cb">>) -> contract_bytearray; pfx2type(<<"cb">>) -> contract_bytearray;
pfx2type(<<"ck">>) -> contract_store_key; pfx2type(<<"ck">>) -> contract_store_key;
pfx2type(<<"cv">>) -> contract_store_value; pfx2type(<<"cv">>) -> contract_store_value;
pfx2type(<<"ct">>) -> contract_pubkey; pfx2type(<<"ct">>) -> contract_pubkey;
pfx2type(<<"cx">>) -> contract_source;
pfx2type(<<"tx">>) -> transaction; pfx2type(<<"tx">>) -> transaction;
pfx2type(<<"th">>) -> tx_hash; pfx2type(<<"th">>) -> tx_hash;
pfx2type(<<"ok">>) -> oracle_pubkey; pfx2type(<<"a0">>) -> {account_pubkey, 0};
pfx2type(<<"ov">>) -> oracle_query; pfx2type(<<"a1">>) -> {account_pubkey, 1};
pfx2type(<<"oq">>) -> oracle_query_id; pfx2type(<<"a2">>) -> {account_pubkey, 2};
pfx2type(<<"or">>) -> oracle_response; pfx2type(<<"a3">>) -> {account_pubkey, 3};
pfx2type(<<"a4">>) -> {account_pubkey, 4};
pfx2type(<<"a5">>) -> {account_pubkey, 5};
pfx2type(<<"a6">>) -> {account_pubkey, 6};
pfx2type(<<"ak">>) -> account_pubkey; pfx2type(<<"ak">>) -> account_pubkey;
pfx2type(<<"sk">>) -> account_seckey;
pfx2type(<<"ac">>) -> associate_chain;
pfx2type(<<"sg">>) -> signature; pfx2type(<<"sg">>) -> signature;
pfx2type(<<"cm">>) -> commitment; pfx2type(<<"cm">>) -> commitment;
pfx2type(<<"pp">>) -> peer_pubkey; pfx2type(<<"pp">>) -> peer_pubkey;
pfx2type(<<"nm">>) -> name; pfx2type(<<"nm">>) -> name;
pfx2type(<<"nt">>) -> native_token;
pfx2type(<<"st">>) -> state; pfx2type(<<"st">>) -> state;
pfx2type(<<"pi">>) -> poi; pfx2type(<<"pi">>) -> poi;
pfx2type(<<"ss">>) -> state_trees; pfx2type(<<"ss">>) -> state_trees;
pfx2type(<<"cs">>) -> call_state_tree; pfx2type(<<"cs">>) -> call_state_tree;
pfx2type(<<"mt">>) -> mp_tree_hash;
pfx2type(<<"hs">>) -> hash;
pfx2type(<<"en">>) -> entry;
pfx2type(<<"ba">>) -> bytearray. pfx2type(<<"ba">>) -> bytearray.
-spec byte_size_for_type(known_type()) -> non_neg_integer() | not_applicable. -spec byte_size_for_type(known_type()) -> non_neg_integer() | not_applicable.
@@ -263,26 +372,31 @@ byte_size_for_type(micro_block_hash) -> 32;
byte_size_for_type(block_pof_hash) -> 32; byte_size_for_type(block_pof_hash) -> 32;
byte_size_for_type(block_tx_hash) -> 32; byte_size_for_type(block_tx_hash) -> 32;
byte_size_for_type(block_state_hash) -> 32; byte_size_for_type(block_state_hash) -> 32;
byte_size_for_type(block_witness_hash) -> 32;
byte_size_for_type(channel) -> 32; byte_size_for_type(channel) -> 32;
byte_size_for_type(contract_pubkey) -> 32; byte_size_for_type(contract_pubkey) -> 32;
byte_size_for_type(contract_bytearray) -> not_applicable; byte_size_for_type(contract_bytearray) -> not_applicable;
byte_size_for_type(contract_store_key) -> not_applicable; byte_size_for_type(contract_store_key) -> not_applicable;
byte_size_for_type(contract_store_value) -> not_applicable; byte_size_for_type(contract_store_value) -> not_applicable;
byte_size_for_type(contract_source) -> not_applicable;
byte_size_for_type(transaction) -> not_applicable; byte_size_for_type(transaction) -> not_applicable;
byte_size_for_type(tx_hash) -> 32; byte_size_for_type(tx_hash) -> 32;
byte_size_for_type(oracle_pubkey) -> 32;
byte_size_for_type(oracle_query) -> not_applicable;
byte_size_for_type(oracle_query_id) -> 32;
byte_size_for_type(oracle_response) -> not_applicable;
byte_size_for_type(account_pubkey) -> 32; byte_size_for_type(account_pubkey) -> 32;
byte_size_for_type({account_pubkey, _}) -> 32;
byte_size_for_type(account_seckey) -> 32;
byte_size_for_type(associate_chain) -> 32;
byte_size_for_type(signature) -> 64; byte_size_for_type(signature) -> 64;
byte_size_for_type(name) -> not_applicable; byte_size_for_type(name) -> not_applicable;
byte_size_for_type(native_token) -> 32;
byte_size_for_type(commitment) -> 32; byte_size_for_type(commitment) -> 32;
byte_size_for_type(peer_pubkey) -> 32; byte_size_for_type(peer_pubkey) -> 32;
byte_size_for_type(state) -> 32; byte_size_for_type(state) -> 32;
byte_size_for_type(poi) -> not_applicable; byte_size_for_type(poi) -> not_applicable;
byte_size_for_type(state_trees) -> not_applicable; byte_size_for_type(state_trees) -> not_applicable;
byte_size_for_type(call_state_tree) -> not_applicable; byte_size_for_type(call_state_tree) -> not_applicable;
byte_size_for_type(mp_tree_hash) -> 32;
byte_size_for_type(hash) -> 32;
byte_size_for_type(entry) -> not_applicable;
byte_size_for_type(bytearray) -> not_applicable. byte_size_for_type(bytearray) -> not_applicable.
+315
View File
@@ -0,0 +1,315 @@
%%%-------------------------------------------------------------------
%%% @copyright (C) 2026, QPQ AG (experiment)
%%% @doc
%%% Thin RLP production layer driven by ASN.1-shaped values.
%%%
%%% This module implements a "thin translation" from structures that
%%% mirror the ASN.1 definitions in asn1/GajumaruSerialization.asn
%%% to the exact same RLP wire format produced by the legacy
%%% gmserialization + gmser_rlp + gmser_chain_objects stack.
%%%
%%% Goal:
%%% - Use ASN.1 as a formal, multi-language-friendly schema.
%%% - Keep the compact legacy RLP on the wire (no DER bloat).
%%% - Provide a reference implementation that other languages can
%%% port (types from ASN.1 compiler + this thin RLP emitter).
%%%
%%% The layer does NOT use the ASN.1 BER/DER codec at runtime for
%%% the wire format. It walks ASN.1-like Erlang terms and emits
%%% RLP using the same rules as gmserialization:encode_field/2
%%% and gmser_rlp:encode/1.
%%%
%%% Supported shapes (matching the ASN.1 value notation):
%%% {'GajumaruData', Tag, Vsn, Content}
%%% Content is a CHOICE:
%%% {templateFields, [ {'TemplateField', Name, Value}, ... ]}
%%% {signedTx, {'SignedTx', Sigs, Tx}}
%%% {account, {'Account', Foo, Bar}}
%%% ...
%%% Value is one of:
%%% {bigIntValue, integer()} % bignum (the original "int" for Pucks etc.)
%%% {uint128Value, integer()}
%%% {uint64Value, integer()}
%%% {uint32Value, integer()}
%%% {uint16Value, integer()}
%%% {uint8Value, integer()}
%%% {binaryValue, binary()}
%%% {boolValue, boolean()}
%%% {listValue, [Value]}
%%% {tupleValue, [Value]} % or tuple, both accepted
%%% {idValue, ...} % basic support
%%%
%%% For templateFields (used for generic/static equivalence), field
%%% *names* are ignored on the wire (matching legacy static behavior
%%% where only values are sent in template order).
%%%
%%% Equivalence with legacy is the primary contract of this module.
%%% @end
%%%-------------------------------------------------------------------
-module(gmser_asn1_rlp).
-vsn("0.1.0-experiment").
-export([encode/1]).
%% For tests and other-language ports, these helpers are useful
-export([encode_basic/2,
encode_asn1_value/1]).
-ifdef(TEST).
-include_lib("eunit/include/eunit.hrl").
-endif.
%%%===================================================================
%%% API
%%%===================================================================
-spec encode(term()) -> binary().
encode({'GajumaruData', Tag, Vsn, Content}) ->
TagB = encode_basic(int, Tag),
VsnB = encode_basic(int, Vsn),
Payload = encode_content(Content),
gmser_rlp:encode([TagB, VsnB | Payload]);
encode(Other) ->
error({unsupported_asn1_top_level, Other}).
%%%===================================================================
%%% Internal: content (the CHOICE after tag/vsn)
%%%===================================================================
encode_content({templateFields, FieldList}) when is_list(FieldList) ->
%% For wire compatibility with legacy static serialization we
%% emit only the values (in order). Names are not sent on the wire.
[encode_asn1_value(Val) || {_, _Name, Val} <- FieldList];
encode_content({signedTx, {'SignedTx', Sigs, Tx}}) ->
SigsEnc = [encode_basic(binary, S) || S <- Sigs],
TxEnc = encode_basic(binary, Tx),
[SigsEnc, TxEnc];
encode_content({account, {'Account', Foo, Bar}}) ->
[encode_basic(int, Foo), encode_basic(binary, Bar)];
encode_content({contract, Contract}) ->
encode_contract(Contract);
encode_content(Other) ->
error({unsupported_content_choice, Other}).
encode_contract({v1, {'ContractV1', Hash, TypeInfo, ByteCode}}) ->
[encode_basic(binary, Hash),
[encode_type_info_v1(TI) || TI <- TypeInfo],
encode_basic(binary, ByteCode)];
encode_contract({v2, {'ContractV2', Hash, TypeInfo, ByteCode, CompilerVsn}}) ->
[encode_basic(binary, Hash),
[encode_type_info_v1(TI) || TI <- TypeInfo],
encode_basic(binary, ByteCode),
encode_basic(binary, CompilerVsn)];
encode_contract({v3, {'ContractV3', Hash, TypeInfo, ByteCode, CompilerVsn, Payable}}) ->
[encode_basic(binary, Hash),
[encode_type_info_v3(TI) || TI <- TypeInfo],
encode_basic(binary, ByteCode),
encode_basic(binary, CompilerVsn),
encode_basic(bool, Payable)].
encode_type_info_v1({_TypeInfoV1, H, N, A, O}) ->
[encode_basic(binary, H),
encode_basic(binary, N),
encode_basic(binary, A),
encode_basic(binary, O)];
encode_type_info_v1(T) when is_tuple(T), tuple_size(T) =:= 4 ->
%% Accept plain 4-tuple as well for convenience
[encode_basic(binary, element(I, T)) || I <- lists:seq(1,4)].
encode_type_info_v3({_TypeInfoV3, H, N, P, A, O}) ->
[encode_basic(binary, H),
encode_basic(binary, N),
encode_basic(bool, P),
encode_basic(binary, A),
encode_basic(binary, O)];
encode_type_info_v3(T) when is_tuple(T), tuple_size(T) =:= 5 ->
%% TypeInfoV3 layout: binary, binary, bool, binary, binary
[encode_basic(binary, element(1,T)),
encode_basic(binary, element(2,T)),
encode_basic(bool, element(3,T)),
encode_basic(binary, element(4,T)),
encode_basic(binary, element(5,T))].
%%%===================================================================
%%% Value encoding (recursive, mirrors ASN.1 Value CHOICE)
%%%===================================================================
-spec encode_asn1_value(term()) -> binary() | [term()].
encode_asn1_value({bigIntValue, I}) -> encode_basic(int, I);
encode_asn1_value({uint128Value, I}) -> encode_basic(int, I);
encode_asn1_value({uint64Value, I}) -> encode_basic(int, I);
encode_asn1_value({uint32Value, I}) -> encode_basic(int, I);
encode_asn1_value({uint16Value, I}) -> encode_basic(int, I);
encode_asn1_value({uint8Value, I}) -> encode_basic(int, I);
encode_asn1_value({binaryValue, B}) -> encode_basic(binary, B);
encode_asn1_value({boolValue, B}) -> encode_basic(bool, B);
encode_asn1_value({listValue, L}) when is_list(L) ->
[encode_asn1_value(E) || E <- L];
encode_asn1_value({tupleValue, T}) when is_list(T) ->
[encode_asn1_value(E) || E <- T];
encode_asn1_value({tupleValue, T}) when is_tuple(T) ->
[encode_asn1_value(E) || E <- tuple_to_list(T)];
encode_asn1_value({idValue, {'Id', Type, Val}}) ->
%% Basic support: encode as the legacy 33-byte id form if possible,
%% otherwise fall back to treating the value as binary.
try
Id = gmser_id:create(decode_id_tag(Type), Val),
gmser_id:encode(Id)
catch _:_ ->
encode_basic(binary, Val)
end;
encode_asn1_value({idValue, Bin}) when is_binary(Bin) ->
%% Convenience: bare 33-byte id value
encode_basic(binary, Bin);
encode_asn1_value(Other) ->
error({unsupported_asn1_value, Other}).
decode_id_tag(1) -> account;
decode_id_tag(2) -> name;
decode_id_tag(3) -> commitment;
decode_id_tag(5) -> contract;
decode_id_tag(6) -> channel;
decode_id_tag(7) -> associate_chain;
decode_id_tag(8) -> native_token;
decode_id_tag(9) -> entry;
decode_id_tag(T) when is_integer(T) -> error({unknown_id_tag, T}).
%%%===================================================================
%%% Basic encoders matching gmserialization rules
%%%===================================================================
-spec encode_basic(atom(), term()) -> binary().
encode_basic(int, X) when is_integer(X), X >= 0 ->
binary:encode_unsigned(X);
encode_basic(binary, X) when is_binary(X) ->
X;
encode_basic(bool, true) -> <<1:8>>;
encode_basic(bool, false) -> <<0:8>>;
encode_basic(id, Val) ->
try gmser_id:encode(Val)
catch _:_ -> error({illegal, id, Val})
end;
encode_basic(Type, Val) ->
error({unsupported_basic_type, Type, Val}).
%%%===================================================================
%%% EUnit equivalence tests
%%%===================================================================
-ifdef(TEST).
%% These tests assert that encoding an ASN.1-shaped value produces
%% *exactly* the same bytes as the legacy gmserialization stack.
%% This is the key property for a thin RLP production layer.
equivalence_simple_fields_test() ->
T = [{foo, uint32}, {bar, binary}],
V = [{foo, 1}, {bar, <<2>>}],
Legacy = gmser_chain_objects:serialize(account, 1, T, V),
Asn1 = {'GajumaruData', 10, 1, {templateFields, [
{'TemplateField', <<"foo">>, {uint32Value, 1}},
{'TemplateField', <<"bar">>, {binaryValue, <<2>>}}
]}},
New = encode(Asn1),
?assertEqual(Legacy, New),
%% Also check we can roundtrip via legacy decoder
Dec = gmser_chain_objects:deserialize(account, 1, T, New),
?assertEqual(V, Dec).
equivalence_zero_and_empty_test() ->
T = [{foo, int}, {bar, binary}],
V = [{foo, 0}, {bar, <<>>}],
Legacy = gmser_chain_objects:serialize(account, 1, T, V),
Asn1 = {'GajumaruData', 10, 1, {templateFields, [
{'TemplateField', <<"foo">>, {bigIntValue, 0}},
{'TemplateField', <<"bar">>, {binaryValue, <<>>}}
]}},
?assertEqual(Legacy, encode(Asn1)).
equivalence_list_field_test() ->
T = [{xs, [int]}],
V = [{xs, [1,2,3]}],
Legacy = gmser_chain_objects:serialize(account, 1, T, V),
Asn1 = {'GajumaruData', 10, 1, {templateFields, [
{'TemplateField', <<"xs">>, {listValue, [
{bigIntValue, 1}, {bigIntValue, 2}, {bigIntValue, 3}
]}}
]}},
?assertEqual(Legacy, encode(Asn1)).
equivalence_tuple_field_test() ->
T = [{p, {int, binary}}],
V = [{p, {42, <<"hi">>}}],
Legacy = gmser_chain_objects:serialize(account, 1, T, V),
Asn1 = {'GajumaruData', 10, 1, {templateFields, [
{'TemplateField', <<"p">>, {tupleValue, [
{bigIntValue, 42}, {binaryValue, <<"hi">>}
]}}
]}},
?assertEqual(Legacy, encode(Asn1)).
equivalence_signed_tx_concrete_test() ->
T = [{signatures, [binary]}, {tx, binary}],
V = [{signatures, [<<"sig1">>, <<"sig2">>]}, {tx, <<"txbody123">>}],
Legacy = gmser_chain_objects:serialize(signed_tx, 1, T, V),
Asn1 = {'GajumaruData', 11, 1, {signedTx, {'SignedTx',
[<<"sig1">>, <<"sig2">>], <<"txbody123">>}}},
?assertEqual(Legacy, encode(Asn1)).
equivalence_list_of_tuples_test() ->
%% Corresponds to type_info style: list of 4-tuples
T = [{type_info, [{binary, binary, binary, binary}]}],
V = [{type_info, [
{<<"h1">>, <<"n1">>, <<"a1">>, <<"o1">>},
{<<"h2">>, <<"n2">>, <<"a2">>, <<"o2">>}
]}],
Legacy = gmser_chain_objects:serialize(account, 1, T, V),
Asn1 = {'GajumaruData', 10, 1, {templateFields, [
{'TemplateField', <<"type_info">>, {listValue, [
{tupleValue, [{binaryValue,<<"h1">>},{binaryValue,<<"n1">>},
{binaryValue,<<"a1">>},{binaryValue,<<"o1">>}]},
{tupleValue, [{binaryValue,<<"h2">>},{binaryValue,<<"n2">>},
{binaryValue,<<"a2">>},{binaryValue,<<"o2">>}]}
]}}
]}},
?assertEqual(Legacy, encode(Asn1)).
equivalence_contract_v3_test() ->
T = [ {source_hash, binary}
, {type_info, [{binary, binary, bool, binary, binary}]}
, {byte_code, binary}
, {compiler_version, binary}
, {payable, bool}
],
TI = [{<<"h">>, <<"n">>, true, <<"a">>, <<"o">>}],
V = [ {source_hash, <<"hash">>}
, {type_info, TI}
, {byte_code, <<"code">>}
, {compiler_version, <<"vsn">>}
, {payable, true}
],
Legacy = gmser_chain_objects:serialize(contract, 3, T, V),
Asn1 = {'GajumaruData', 40, 3, {contract, {v3, {'ContractV3',
<<"hash">>,
[ {<<"h">>, <<"n">>, true, <<"a">>, <<"o">>} ],
<<"code">>,
<<"vsn">>,
true
}}}},
?assertEqual(Legacy, encode(Asn1)).
-endif.
@@ -1,20 +1,22 @@
%%% -*- erlang-indent-level:4; indent-tabs-mode: nil -*- %%% -*- erlang-indent-level:4; indent-tabs-mode: nil -*-
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
%%% @copyright (C) 2025, QPQ AG
%%% @copyright (C) 2018, Aeternity Anstalt %%% @copyright (C) 2018, Aeternity Anstalt
%%% @doc %%% @doc
%%% Functions for serializing chain objects to binary format. %%% Functions for serializing chain objects to binary format.
%%% @end %%% @end
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
-module(aeser_chain_objects). -module(gmser_chain_objects).
-vsn("0.2.0").
-export([ serialize/4 -export([ serialize/4
, deserialize/4 , deserialize/4
, deserialize_type_and_vsn/1 , deserialize_type_and_vsn/1
]). ]).
-type template() :: aeserialization:template(). -type template() :: gmserialization:template().
-type fields() :: aeserialization:fields(). -type fields() :: gmserialization:fields().
%%%=================================================================== %%%===================================================================
%%% API %%% API
@@ -22,15 +24,15 @@
-spec serialize(atom(), non_neg_integer(), template(), fields()) -> binary(). -spec serialize(atom(), non_neg_integer(), template(), fields()) -> binary().
serialize(Type, Vsn, Template, Fields) -> serialize(Type, Vsn, Template, Fields) ->
aeserialization:serialize(tag(Type), Vsn, Template, Fields). gmserialization:serialize(tag(Type), Vsn, Template, Fields).
deserialize_type_and_vsn(Binary) -> deserialize_type_and_vsn(Binary) ->
{Tag, Vsn, Fields} = aeserialization:deserialize_tag_and_vsn(Binary), {Tag, Vsn, Fields} = gmserialization:deserialize_tag_and_vsn(Binary),
{rev_tag(Tag), Vsn, Fields}. {rev_tag(Tag), Vsn, Fields}.
-spec deserialize(atom(), non_neg_integer(), template(), binary()) -> fields(). -spec deserialize(atom(), non_neg_integer(), template(), binary()) -> fields().
deserialize(Type, Vsn, Template, Binary) -> deserialize(Type, Vsn, Template, Binary) ->
aeserialization:deserialize(Type, tag(Type), Vsn, Template, Binary). gmserialization:deserialize(Type, tag(Type), Vsn, Template, Binary).
%%%=================================================================== %%%===================================================================
%%% Internal functions %%% Internal functions
@@ -39,12 +41,7 @@ deserialize(Type, Vsn, Template, Binary) ->
tag(account) -> 10; tag(account) -> 10;
tag(signed_tx) -> 11; tag(signed_tx) -> 11;
tag(spend_tx) -> 12; tag(spend_tx) -> 12;
tag(oracle) -> 20; tag(data_extend_tx) -> 13;
tag(oracle_query) -> 21;
tag(oracle_register_tx) -> 22;
tag(oracle_query_tx) -> 23;
tag(oracle_response_tx) -> 24;
tag(oracle_extend_tx) -> 25;
tag(name) -> 30; tag(name) -> 30;
tag(name_commitment) -> 31; tag(name_commitment) -> 31;
tag(name_claim_tx) -> 32; tag(name_claim_tx) -> 32;
@@ -57,8 +54,10 @@ tag(contract) -> 40;
tag(contract_call) -> 41; tag(contract_call) -> 41;
tag(contract_create_tx) -> 42; tag(contract_create_tx) -> 42;
tag(contract_call_tx) -> 43; tag(contract_call_tx) -> 43;
tag(contract_source) -> 44;
tag(channel_create_tx) -> 50; tag(channel_create_tx) -> 50;
tag(channel_set_delegates_tx) -> 501; tag(channel_set_delegates_tx) -> 501;
tag(channel_delegates) -> 502;
tag(channel_deposit_tx) -> 51; tag(channel_deposit_tx) -> 51;
tag(channel_withdraw_tx) -> 52; tag(channel_withdraw_tx) -> 52;
tag(channel_force_progress_tx) -> 521; tag(channel_force_progress_tx) -> 521;
@@ -85,27 +84,54 @@ tag(contracts_mtree) -> 621;
tag(calls_mtree) -> 622; tag(calls_mtree) -> 622;
tag(channels_mtree) -> 623; tag(channels_mtree) -> 623;
tag(nameservice_mtree) -> 624; tag(nameservice_mtree) -> 624;
tag(oracles_mtree) -> 625;
tag(accounts_mtree) -> 626; tag(accounts_mtree) -> 626;
tag(acs_mtree) -> 627;
tag(entries_mtree) -> 628;
tag(compiler_sophia) -> 70; tag(compiler_sophia) -> 70;
tag(ga_attach_tx) -> 80; tag(ga_attach_tx) -> 80;
tag(ga_meta_tx) -> 81; tag(ga_meta_tx) -> 81;
tag(paying_for_tx) -> 82; tag(paying_for_tx) -> 82;
tag(ga_meta_tx_auth_data) -> 810; tag(ga_meta_tx_auth_data) -> 810;
tag(associate_chain) -> 90;
tag(ac_state) -> 91;
tag(ac_burn) -> 92;
tag(ac_create_tx) -> 93;
tag(ac_deposit_tx) -> 94;
tag(ac_update_cops_tx) -> 95;
tag(ac_rollup_tx) -> 96;
tag(ac_proposal_tx) -> 97;
tag(ac_receipt) -> 98;
tag(ac_acct_state) -> 99;
tag(key_block) -> 100; tag(key_block) -> 100;
tag(micro_block) -> 101; tag(micro_block) -> 101;
tag(light_micro_block) -> 102; tag(light_micro_block) -> 102;
tag(pof) -> 200. tag(testimony) -> 110;
tag(testimony_tx) -> 111;
tag(nt_native_token) -> 120;
tag(nt_create_tx) -> 121;
tag(nt_mint_tx) -> 122;
tag(nt_finalize_tx) -> 123;
tag(nt_trade_tx) -> 124;
tag(nt_burn_tx) -> 125;
tag(entry) -> 140;
tag(entry_create_tx) -> 141;
tag(entry_transfer_tx) -> 142;
tag(entry_destroy_tx) -> 143;
tag(account_key_store) -> 144;
tag(account_create_tx) -> 145;
tag(account_sig_store) -> 146;
tag(auth_tx) -> 147;
tag(proposal_gossip_tx) -> 148;
tag(account_auth_update_tx) -> 149;
tag(pof) -> 200;
%% Gajumaru AC side transactions
tag(ac_side_withdraw_tx) -> 300;
tag(ac_side_rollup_tx) -> 301.
rev_tag(10) -> account; rev_tag(10) -> account;
rev_tag(11) -> signed_tx; rev_tag(11) -> signed_tx;
rev_tag(12) -> spend_tx; rev_tag(12) -> spend_tx;
rev_tag(20) -> oracle; rev_tag(13) -> data_extend_tx;
rev_tag(21) -> oracle_query;
rev_tag(22) -> oracle_register_tx;
rev_tag(23) -> oracle_query_tx;
rev_tag(24) -> oracle_response_tx;
rev_tag(25) -> oracle_extend_tx;
rev_tag(30) -> name; rev_tag(30) -> name;
rev_tag(31) -> name_commitment; rev_tag(31) -> name_commitment;
rev_tag(32) -> name_claim_tx; rev_tag(32) -> name_claim_tx;
@@ -118,8 +144,10 @@ rev_tag(40) -> contract;
rev_tag(41) -> contract_call; rev_tag(41) -> contract_call;
rev_tag(42) -> contract_create_tx; rev_tag(42) -> contract_create_tx;
rev_tag(43) -> contract_call_tx; rev_tag(43) -> contract_call_tx;
rev_tag(44) -> contract_source;
rev_tag(50) -> channel_create_tx; rev_tag(50) -> channel_create_tx;
rev_tag(501) -> channel_set_delegates_tx; rev_tag(501) -> channel_set_delegates_tx;
rev_tag(502) -> channel_delegates;
rev_tag(51) -> channel_deposit_tx; rev_tag(51) -> channel_deposit_tx;
rev_tag(52) -> channel_withdraw_tx; rev_tag(52) -> channel_withdraw_tx;
rev_tag(521) -> channel_force_progress_tx; rev_tag(521) -> channel_force_progress_tx;
@@ -146,14 +174,46 @@ rev_tag(621) -> contracts_mtree;
rev_tag(622) -> calls_mtree; rev_tag(622) -> calls_mtree;
rev_tag(623) -> channels_mtree; rev_tag(623) -> channels_mtree;
rev_tag(624) -> nameservice_mtree; rev_tag(624) -> nameservice_mtree;
rev_tag(625) -> oracles_mtree;
rev_tag(626) -> accounts_mtree; rev_tag(626) -> accounts_mtree;
rev_tag(627) -> acs_mtree;
rev_tag(628) -> entries_mtree;
rev_tag(70) -> compiler_sophia; rev_tag(70) -> compiler_sophia;
rev_tag(80) -> ga_attach_tx; rev_tag(80) -> ga_attach_tx;
rev_tag(81) -> ga_meta_tx; rev_tag(81) -> ga_meta_tx;
rev_tag(82) -> paying_for_tx; rev_tag(82) -> paying_for_tx;
rev_tag(810) -> ga_meta_tx_auth_data; rev_tag(810) -> ga_meta_tx_auth_data;
rev_tag(90) -> associate_chain;
rev_tag(91) -> ac_state;
rev_tag(92) -> ac_burn;
rev_tag(93) -> ac_create_tx;
rev_tag(94) -> ac_deposit_tx;
rev_tag(95) -> ac_update_cops_tx;
rev_tag(96) -> ac_rollup_tx;
rev_tag(97) -> ac_proposal_tx;
rev_tag(98) -> ac_receipt;
rev_tag(99) -> ac_acct_state;
rev_tag(100) -> key_block; rev_tag(100) -> key_block;
rev_tag(101) -> micro_block; rev_tag(101) -> micro_block;
rev_tag(102) -> light_micro_block; rev_tag(102) -> light_micro_block;
rev_tag(200) -> pof. rev_tag(110) -> testimony;
rev_tag(111) -> testimony_tx;
rev_tag(120) -> nt_native_token;
rev_tag(121) -> nt_create_tx;
rev_tag(122) -> nt_mint_tx;
rev_tag(123) -> nt_finalize_tx;
rev_tag(124) -> nt_trade_tx;
rev_tag(125) -> nt_burn_tx;
rev_tag(140) -> entry;
rev_tag(141) -> entry_create_tx;
rev_tag(142) -> entry_transfer_tx;
rev_tag(143) -> entry_destroy_tx;
rev_tag(144) -> account_key_store;
rev_tag(145) -> account_create_tx;
rev_tag(146) -> account_sig_store;
rev_tag(147) -> auth_tx;
rev_tag(148) -> proposal_gossip_tx;
rev_tag(149) -> account_auth_update_tx;
rev_tag(200) -> pof;
%% Gajumaru AC side transactions
rev_tag(300) -> ac_side_withdraw_tx;
rev_tag(301) -> ac_side_rollup_tx.
@@ -1,12 +1,14 @@
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
%%% @copyright (C) 2025, QPQ AG
%%% @copyright (C) 2019, Aeternity Anstalt %%% @copyright (C) 2019, Aeternity Anstalt
%%% @doc %%% @doc
%%% Serialization of contract code %%% Serialization of contract code
%%% @end %%% @end
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
-module(aeser_contract_code). -module(gmser_contract_code).
-vsn("0.2.0").
-include("aeser_contract_code.hrl"). -include("gmser_contract_code.hrl").
-export([ deserialize/1 -export([ deserialize/1
, serialize/1 , serialize/1
@@ -16,14 +18,15 @@
serialize(CodeMap) -> serialize(CodeMap) ->
serialize(CodeMap, ?SOPHIA_CONTRACT_VSN_3). serialize(CodeMap, ?SOPHIA_CONTRACT_VSN_3).
-spec serialize(map(), non_neg_integer()) -> binary(). -spec serialize(map(), non_neg_integer()) -> binary().
serialize(CodeMap = #{ byte_code := ByteCode serialize(CodeMap = #{ byte_code := ByteCode
, type_info := TypeInfo }, SophiaContractVersion) -> , type_info := TypeInfo }, SophiaContractVersion) ->
%% Source hash %% Source hash
SourceHash = case CodeMap of SourceHash =
case CodeMap of
#{ source_hash := SHash } -> SHash; #{ source_hash := SHash } -> SHash;
#{ contract_source := SrcStr } -> #{ contract_source := SrcStr } -> blake2(32, list_to_binary(SrcStr))
enacl:generichash(32, list_to_binary(SrcStr))
end, end,
%% Compiler version %% Compiler version
@@ -42,20 +45,35 @@ serialize(CodeMap = #{ byte_code := ByteCode
|| SophiaContractVersion > ?SOPHIA_CONTRACT_VSN_1 ] ++ || SophiaContractVersion > ?SOPHIA_CONTRACT_VSN_1 ] ++
[ {payable, Payable} [ {payable, Payable}
|| SophiaContractVersion > ?SOPHIA_CONTRACT_VSN_2 ], || SophiaContractVersion > ?SOPHIA_CONTRACT_VSN_2 ],
aeser_chain_objects:serialize(compiler_sophia, gmser_chain_objects:serialize(compiler_sophia,
SophiaContractVersion, SophiaContractVersion,
serialization_template(SophiaContractVersion), serialization_template(SophiaContractVersion),
Fields). Fields).
% NOTE:
% This form significantly favors the presence of enacl and slows fallback
% invokation of eblake2. `try' is really fast; the error throwing machinery
% is comparatively slow. The assumption here is that in cases where you want
% eblake2 performance isn't the problem you're solving (you're probably not
% syncing a new node, for example).
blake2(Size, Bin) ->
try
enacl:generichash(Size, Bin)
catch error:undef ->
{ok, Hash} = eblake2:blake2b(Size, Bin),
Hash
end.
-spec deserialize(binary()) -> map(). -spec deserialize(binary()) -> map().
deserialize(Binary) -> deserialize(Binary) ->
case aeser_chain_objects:deserialize_type_and_vsn(Binary) of case gmser_chain_objects:deserialize_type_and_vsn(Binary) of
{compiler_sophia = Type, ?SOPHIA_CONTRACT_VSN_1 = Vsn, _Rest} -> {compiler_sophia = Type, ?SOPHIA_CONTRACT_VSN_1 = Vsn, _Rest} ->
Template = serialization_template(Vsn), Template = serialization_template(Vsn),
[ {source_hash, Hash} [ {source_hash, Hash}
, {type_info, TypeInfo} , {type_info, TypeInfo}
, {byte_code, ByteCode} , {byte_code, ByteCode}
] = aeser_chain_objects:deserialize(Type, Vsn, Template, Binary), ] = gmser_chain_objects:deserialize(Type, Vsn, Template, Binary),
#{ source_hash => Hash #{ source_hash => Hash
, type_info => TypeInfo , type_info => TypeInfo
, byte_code => ByteCode , byte_code => ByteCode
@@ -68,7 +86,7 @@ deserialize(Binary) ->
, {type_info, TypeInfo} , {type_info, TypeInfo}
, {byte_code, ByteCode} , {byte_code, ByteCode}
, {compiler_version, CompilerVersion} , {compiler_version, CompilerVersion}
] = aeser_chain_objects:deserialize(Type, Vsn, Template, Binary), ] = gmser_chain_objects:deserialize(Type, Vsn, Template, Binary),
#{ source_hash => Hash #{ source_hash => Hash
, type_info => TypeInfo , type_info => TypeInfo
, byte_code => ByteCode , byte_code => ByteCode
@@ -83,7 +101,7 @@ deserialize(Binary) ->
, {byte_code, ByteCode} , {byte_code, ByteCode}
, {compiler_version, CompilerVersion} , {compiler_version, CompilerVersion}
, {payable, Payable} , {payable, Payable}
] = aeser_chain_objects:deserialize(Type, Vsn, Template, Binary), ] = gmser_chain_objects:deserialize(Type, Vsn, Template, Binary),
#{ source_hash => Hash #{ source_hash => Hash
, type_info => TypeInfo , type_info => TypeInfo
, byte_code => ByteCode , byte_code => ByteCode
+65
View File
@@ -0,0 +1,65 @@
%%%-------------------------------------------------------------------
%%% @copyright (C) 2025, QPQ AG
%%% @copyright (C) 2023, Aeternity Anstalt
%%% @doc
%%% Serialization of delegation signatures
%%% @end
%%%-------------------------------------------------------------------
-module(gmser_delegation).
-vsn("0.2.0").
-export([ aens_preclaim_sig/3
, aens_name_sig/4
, aens_sig/3
]).
%% Delegation signatures are prefixed with a unique tag, ensuring not to
%% collide with serialized transactions.
-define(DELEGATION_TAG, 16#1a01).
-define(TYPE_AENS, 1).
-define(TYPE_AENS_NAME, 2).
-define(TYPE_AENS_PRECLAIM, 3).
-define(TYPE_ORACLE, 4).
-define(TYPE_ORACLE_RESPONSE, 5).
-define(VSN, 1).
-type sig_data() :: binary().
-spec aens_preclaim_sig(binary(), gmser_id:id(), gmser_id:id()) -> sig_data().
aens_preclaim_sig(NetworkId, Account, Contract) ->
assert_id(account, Account),
assert_id(contract, Contract),
Template = [{account, id}, {contract, id}],
Fields = [{account, Account}, {contract, Contract}],
serialize(?TYPE_AENS_PRECLAIM, NetworkId, Template, Fields).
-spec aens_name_sig(binary(), gmser_id:id(), gmser_id:id(), gmser_id:id()) -> sig_data().
aens_name_sig(NetworkId, Account, Name, Contract) ->
assert_id(account, Account),
assert_id(name, Name),
assert_id(contract, Contract),
Template = [{account, id}, {name, id}, {contract, id}],
Fields = [{account, Account}, {name, Name}, {contract, Contract}],
serialize(?TYPE_AENS_NAME, NetworkId, Template, Fields).
-spec aens_sig(binary(), gmser_id:id(), gmser_id:id()) -> sig_data().
aens_sig(NetworkId, Account, Contract) ->
assert_id(account, Account),
assert_id(contract, Contract),
Template = [{account, id}, {contract, id}],
Fields = [{account, Account}, {contract, Contract}],
serialize(?TYPE_AENS, NetworkId, Template, Fields).
%% ------------------------------------------------------------------------
%% -- Internal functions
%% ------------------------------------------------------------------------
serialize(Type, NetworkId, Template, Fields) ->
Data = gmserialization:serialize(Type, ?VSN, Template, Fields),
<<?DELEGATION_TAG:16, NetworkId/binary, Data/binary>>.
assert_id(Type, AeserId) ->
Type = gmser_id:specialize_type(AeserId).
+1318
View File
File diff suppressed because it is too large Load Diff
+72
View File
@@ -0,0 +1,72 @@
-module(gmser_dyn_types).
-vsn("0.2.0").
-export([ add_type/3 %% (Tag, Code, Template) -> Types1
, add_type/4 %% (Tag, Code, Template, Types) -> Types1
, from_list/2
, expand/1 ]).
-export([ next_code/1 ]).
next_code(#{codes := Codes}) ->
lists:max(maps:keys(Codes)) + 1.
-spec add_type(Tag, Code, Template) -> Types
when Tag :: gmser_dyn:tag()
, Code :: gmser_dyn:code()
, Template :: gmser_dyn:template()
, Types :: gmser_dyn:types().
add_type(Tag, Code, Template) ->
add_type(Tag, Code, Template, gmser_dyn:registered_types()).
add_type(Tag, Code, Template, Types) ->
elem_to_type({Tag, Code, Template}, Types).
from_list(L, Types) ->
lists:foldl(fun elem_to_type/2, Types, L).
expand(#{vsn := V, templates := Templates0} = Types) ->
Templates =
maps:map(
fun(_, F) when is_function(F, 0) ->
F();
(_, F) when is_function(F, 1) ->
F(V);
(_, T) ->
T
end, Templates0),
Types#{templates := Templates}.
elem_to_type({Tag, Code, Template}, Acc) when is_atom(Tag), is_integer(Code) ->
#{codes := Codes, rev := Rev, templates := Temps} = Acc,
case {is_map_key(Tag, Rev), is_map_key(Code, Codes)} of
{false, false} ->
Acc#{ codes := Codes#{Code => Tag}
, rev := Rev#{Tag => Code}
, templates => Temps#{Tag => Template}
};
{true, _} -> error({duplicate_tag, Tag});
{_, true} -> error({duplicate_code, Code})
end;
elem_to_type({modify, {Tag, Template}}, Acc) ->
#{codes := _, rev := Rev, templates := Templates} = Acc,
_ = maps:get(Tag, Rev),
Templates1 = Templates#{Tag := Template},
Acc#{templates := Templates1};
elem_to_type({labels, Lbls}, Acc) ->
lists:foldl(fun add_label/2, Acc, Lbls);
elem_to_type({vsn, V}, Acc) ->
Acc#{vsn => V};
elem_to_type(Elem, _) ->
error({invalid_type, Elem}).
add_label({L, Code}, #{labels := Lbls, rev_labels := RevLbls} = Acc)
when is_atom(L), is_integer(Code), Code > 0 ->
case {is_map_key(L, Lbls), is_map_key(Code, RevLbls)} of
{false, false} ->
Acc#{labels := Lbls#{L => Code},
rev_labels := RevLbls#{Code => L}};
{true, _} -> error({duplicate_label, L});
{_, true} -> error({duplicate_label_code, Code})
end;
add_label(Elem, _) ->
error({invalid_label, Elem}).
+181
View File
@@ -0,0 +1,181 @@
%%% -*- erlang-indent-level:4; indent-tabs-mode: nil -*-
%%%-------------------------------------------------------------------
%%% @copyright (C) 2025, QPQ AG
%%% @copyright (C) 2018, Aeternity Anstalt
%%% @doc
%%% ADT for identifiers
%%% @end
%%%-------------------------------------------------------------------
-module(gmser_id).
-vsn("0.2.0").
-export([ create/2
, specialize/1
, specialize/2
, specialize_type/1
, is_account/1
, account_pubkey/1
, to_map/1
, is_id/1
]).
-export([ t_id/1
]).
%% For aec_serialization
-export([ encode/1
, decode/1
]).
-record(id, { tag
, val
}).
-type subtype() :: 0..6.
-type id_map() :: #{ type := simple_tag()
, subtype => subtype()
, value := binary() }.
-type tag() :: {'account', subtype()} | simple_tag().
-type simple_tag() :: 'account'
| 'associate_chain'
| 'channel'
| 'commitment'
| 'contract'
| 'name'
| 'native_token'
| 'entry'.
-type val() :: <<_:256>>.
-type id() :: #id{}.
-export_type([ id/0
, tag/0
, val/0
]).
-define(PUB_SIZE, 32).
-define(TAG_SIZE, 1).
-define(SERIALIZED_SIZE, 33). %% ?TAG_SIZE + ?PUB_SIZE
-define(IS_TAG(___TAG___), ___TAG___ =:= account;
___TAG___ =:= name;
___TAG___ =:= commitment;
___TAG___ =:= contract;
___TAG___ =:= channel;
___TAG___ =:= associate_chain;
___TAG___ =:= entry;
___TAG___ =:= native_token
).
-define(IS_VAL(___VAL___), byte_size(___VAL___) =:= 32).
%%%===================================================================
%%% API
%%%===================================================================
-spec create(tag(), val()) -> id().
create(Tag, Val) when ?IS_TAG(Tag), ?IS_VAL(Val) ->
#id{ tag = Tag
, val = Val};
create({account,I}, Val) when is_binary(Val), I >= 0, I =< 6 ->
#id{ tag = {account, I}, val = Val};
create(Tag, Val) when ?IS_VAL(Val) ->
error({illegal_tag, Tag});
create(Tag, Val) when ?IS_TAG(Tag)->
error({illegal_val, Val});
create(Tag, Val) ->
error({illegal_tag_and_val, Tag, Val}).
-spec specialize(id()) -> {tag(), val()}.
specialize(#id{tag = {Tag,_}, val = Val}) ->
{Tag, Val};
specialize(#id{tag = Tag, val = Val}) ->
{Tag, Val}.
-spec specialize(id(), tag()) -> val().
specialize(#id{tag = {Tag, _}, val = Val}, Tag) when is_binary(Val) ->
Val;
specialize(#id{tag = Tag, val = Val}, Tag) when ?IS_TAG(Tag), ?IS_VAL(Val) ->
Val.
-spec specialize_type(id()) -> tag().
specialize_type(#id{tag = {Tag, _}}) when ?IS_TAG(Tag) ->
Tag;
specialize_type(#id{tag = Tag}) when ?IS_TAG(Tag) ->
Tag.
-spec is_account(id() | term()) -> boolean().
is_account(#id{tag = account}) ->
true;
is_account(#id{tag = {account, _}}) ->
true;
is_account(_) ->
false.
-spec account_pubkey(id()) -> val().
account_pubkey(#id{tag = account, val = Val}) when ?IS_VAL(Val) ->
Val;
account_pubkey(#id{tag = {account, _}, val = Val}) when ?IS_VAL(Val) ->
Val.
-spec to_map(id()) -> id_map().
to_map(#id{tag = {Tag, SubType}, val = Val}) when ?IS_TAG(Tag) ->
#{ type => Tag
, subtype => SubType
, value => Val };
to_map(#id{tag = Tag, val = Val}) when ?IS_TAG(Tag) ->
#{ type => Tag
, value => Val }.
-spec is_id(term()) -> boolean().
is_id(#id{}) -> true;
is_id(_) -> false.
-spec encode(id()) -> binary().
encode(#id{tag = {account, N}, val = Val}) when N =< 2#111_1111 ->
Ext = 2#1000_0000 bor N,
<<Ext:8, Val/binary>>;
encode(#id{tag = Tag, val = Val}) ->
Res = <<(encode_tag(Tag)):?TAG_SIZE/unit:8, Val/binary>>,
true = ?SERIALIZED_SIZE =:= byte_size(Res),
Res.
-spec decode(binary()) -> id().
decode(<<Ext:8, Rest/binary>>) when Ext >= 2#1000_0000 ->
%% Extended account id type
Type = Ext band 2#0111_1111,
#id{ tag = {account, Type}
, val = Rest };
decode(<<Tag:?TAG_SIZE/unit:8, Val:?PUB_SIZE/binary>>) ->
#id{ tag = decode_tag(Tag)
, val = Val}.
-spec t_id(any()) -> id().
t_id(#id{} = Id) -> Id.
%%%===================================================================
%%% Internal functions
%%%===================================================================
encode_tag(account) -> 1;
encode_tag(name) -> 2;
encode_tag(commitment) -> 3;
encode_tag(contract) -> 5;
encode_tag(channel) -> 6;
encode_tag(associate_chain) -> 7;
encode_tag(native_token) -> 8;
encode_tag(entry) -> 9;
encode_tag(Other) -> error({illegal_id_tag_name, Other}).
decode_tag(1) -> account;
decode_tag(2) -> name;
decode_tag(3) -> commitment;
decode_tag(5) -> contract;
decode_tag(6) -> channel;
decode_tag(7) -> associate_chain;
decode_tag(8) -> native_token;
decode_tag(9) -> entry;
decode_tag(X) -> error({illegal_id_tag, X}).
+5 -1
View File
@@ -1,14 +1,18 @@
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
%%% @copyright (C) 2025, QPQ AG
%%% @copyright (C) 2017, Aeternity Anstalt %%% @copyright (C) 2017, Aeternity Anstalt
%%% @doc %%% @doc
%%% Implementation of the Recursive Length Prefix. %%% Implementation of the Recursive Length Prefix.
%%% %%%
%%% https://zxq9.com/archives/2749
%%% https://github.com/ethereum/wiki/wiki/RLP %%% https://github.com/ethereum/wiki/wiki/RLP
%%% %%%
%%% @end %%% @end
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
-module(aeser_rlp). -module(gmser_rlp).
-vsn("0.2.0").
-export([ decode/1 -export([ decode/1
, decode_one/1 , decode_one/1
, encode/1 , encode/1
@@ -1,6 +1,6 @@
{application, aeserialization, {application, gmserialization,
[{description, "Serialization of data for Aeternity"}, [{description, "Serialization of data for the Gajumaru"},
{vsn, "0.1.0"}, {vsn, "zomp"},
{registered, []}, {registered, []},
{applications, {applications,
[kernel, [kernel,
+14
View File
@@ -0,0 +1,14 @@
%% -*- erlang-mode; erlang-indent-level: 4; indent-tabs-mode: nil -*-
[{application, Name, Opts}] = CONFIG.
case lists:keyfind(vsn, 1, Opts) of
{vsn, "zomp"} ->
ZompMetaF = filename:join(filename:dirname(filename:dirname(SCRIPT)), "zomp.meta"),
{ok, ZMeta} = file:consult(ZompMetaF),
{_, {_, _, {Vmaj,Vmin,Vpatch}}} = lists:keyfind(package_id, 1, ZMeta),
VsnStr = unicode:characters_to_list(io_lib:fwrite("~w.~w.~w", [Vmaj, Vmin, Vpatch])),
Opts1 = lists:keyreplace(vsn, 1, Opts, {vsn, VsnStr}),
[{application, Name, Opts1}];
_ ->
CONFIG
end.
@@ -1,16 +1,20 @@
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
%%% @copyright (C) 2025, QPQ AG
%%% @copyright (C) 2018, Aeternity Anstalt %%% @copyright (C) 2018, Aeternity Anstalt
%%% @doc %%% @doc
%%% Functions for serializing generic objects to/from binary format. %%% Functions for serializing generic objects to/from binary format.
%%% @end %%% @end
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
-module(aeserialization). -module(gmserialization).
-vsn("0.2.0").
-export([ decode_fields/2 -export([ decode_fields/2
, decode_field/2
, deserialize/5 , deserialize/5
, deserialize_tag_and_vsn/1 , deserialize_tag_and_vsn/1
, encode_fields/2 , encode_fields/2
, encode_field/2
, serialize/4 ]). , serialize/4 ]).
%%%=================================================================== %%%===================================================================
@@ -21,21 +25,30 @@
, fields/0 , fields/0
]). ]).
-export_type([ encodable_term/0 ]).
-type template() :: [{field_name(), type()}]. -type template() :: [{field_name(), type()}].
-type field_name() :: atom(). -type field_name() :: atom().
-type type() :: 'int' -type type() :: 'int' % bignum (non-negative, arbitrary size; used for Pucks amounts etc. up to 10^30)
| 'uint128'
| 'uint64'
| 'uint32'
| 'uint16'
| 'uint8'
| 'bool' | 'bool'
| 'binary' | 'binary'
| 'id' %% As defined in aec_id.erl | 'id' %% As defined in aec_id.erl
| [type()] %% Length one in the type. This means a list of any length. | [type()] %% Length one in the type. This means a list of any length.
| #{items := [{field_name(), type()}]} %% Record with named fields represented as a map. Encoded as a list in the given order.
| tuple(). %% Any arity, containing type(). This means a static size array. | tuple(). %% Any arity, containing type(). This means a static size array.
-type encodable_term() :: non_neg_integer() -type encodable_term() :: non_neg_integer()
| binary() | binary()
| boolean() | boolean()
| [encodable_term()] %% Of any length | [encodable_term()] %% Of any length
| #{atom() => encodable_term()}
| tuple() %% Any arity, containing encodable_term(). | tuple() %% Any arity, containing encodable_term().
| aeser_id:id(). | gmser_id:id().
-type fields() :: [{field_name(), encodable_term()}]. -type fields() :: [{field_name(), encodable_term()}].
@@ -47,13 +60,13 @@
serialize(Tag, Vsn, Template, Fields) -> serialize(Tag, Vsn, Template, Fields) ->
List = encode_fields([{tag, int}, {vsn, int}|Template], List = encode_fields([{tag, int}, {vsn, int}|Template],
[{tag, Tag}, {vsn, Vsn}|Fields]), [{tag, Tag}, {vsn, Vsn}|Fields]),
aeser_rlp:encode(List). gmser_rlp:encode(List).
%% Type isn't strictly necessary, but will give a better error reason %% Type isn't strictly necessary, but will give a better error reason
-spec deserialize(atom(), non_neg_integer(), non_neg_integer(), -spec deserialize(atom(), non_neg_integer(), non_neg_integer(),
template(), binary()) -> fields(). template(), binary()) -> fields().
deserialize(Type, Tag, Vsn, Template0, Binary) -> deserialize(Type, Tag, Vsn, Template0, Binary) ->
Decoded = aeser_rlp:decode(Binary), Decoded = gmser_rlp:decode(Binary),
Template = [{tag, int}, {vsn, int}|Template0], Template = [{tag, int}, {vsn, int}|Template0],
case decode_fields(Template, Decoded) of case decode_fields(Template, Decoded) of
[{tag, Tag}, {vsn, Vsn}|Left] -> [{tag, Tag}, {vsn, Vsn}|Left] ->
@@ -66,7 +79,7 @@ deserialize(Type, Tag, Vsn, Template0, Binary) ->
-spec deserialize_tag_and_vsn(binary()) -> -spec deserialize_tag_and_vsn(binary()) ->
{non_neg_integer(), non_neg_integer(), fields()}. {non_neg_integer(), non_neg_integer(), fields()}.
deserialize_tag_and_vsn(Binary) -> deserialize_tag_and_vsn(Binary) ->
[TagBin, VsnBin|Fields] = aeser_rlp:decode(Binary), [TagBin, VsnBin|Fields] = gmser_rlp:decode(Binary),
Template = [{tag, int}, {vsn, int}], Template = [{tag, int}, {vsn, int}],
[{tag, Tag}, {vsn, Vsn}] = decode_fields(Template, [TagBin, VsnBin]), [{tag, Tag}, {vsn, Vsn}] = decode_fields(Template, [TagBin, VsnBin]),
{Tag, Vsn, Fields}. {Tag, Vsn, Fields}.
@@ -101,25 +114,73 @@ decode_fields(Template, Values) ->
encode_field([Type], L) when is_list(L) -> encode_field([Type], L) when is_list(L) ->
[encode_field(Type, X) || X <- L]; [encode_field(Type, X) || X <- L];
encode_field(#{items := Items}, Map) ->
lists:map(
fun({K, Type}) ->
V = maps:get(K, Map),
encode_field(Type, V)
end, Items);
encode_field(Type, T) when tuple_size(Type) =:= tuple_size(T) -> encode_field(Type, T) when tuple_size(Type) =:= tuple_size(T) ->
Zipped = lists:zip(tuple_to_list(Type), tuple_to_list(T)), Zipped = lists:zip(tuple_to_list(Type), tuple_to_list(T)),
[encode_field(X, Y) || {X, Y} <- Zipped]; [encode_field(X, Y) || {X, Y} <- Zipped];
encode_field(uint128, X) when is_integer(X), X >= 0, X < (1 bsl 128) ->
binary:encode_unsigned(X);
encode_field(uint64, X) when is_integer(X), X >= 0, X < (1 bsl 64) ->
binary:encode_unsigned(X);
encode_field(uint32, X) when is_integer(X), X >= 0, X < (1 bsl 32) ->
binary:encode_unsigned(X);
encode_field(uint16, X) when is_integer(X), X >= 0, X < (1 bsl 16) ->
binary:encode_unsigned(X);
encode_field(uint8, X) when is_integer(X), X >= 0, X < (1 bsl 8) ->
binary:encode_unsigned(X);
encode_field(int, X) when is_integer(X), X >= 0 -> encode_field(int, X) when is_integer(X), X >= 0 ->
binary:encode_unsigned(X); binary:encode_unsigned(X);
encode_field(binary, X) when is_binary(X) -> X; encode_field(binary, X) when is_binary(X) -> X;
encode_field(bool, true) -> <<1:8>>; encode_field(bool, true) -> <<1:8>>;
encode_field(bool, false) -> <<0:8>>; encode_field(bool, false) -> <<0:8>>;
encode_field(id, Val) -> encode_field(id, Val) ->
try aeser_id:encode(Val) try gmser_id:encode(Val)
catch _:_ -> error({illegal, id, Val}) catch _:_ -> error({illegal, id, Val})
end; end;
encode_field(Type, Val) -> error({illegal, Type, Val}). encode_field(Type, Val) -> error({illegal, Type, Val}).
decode_field([Type], List) when is_list(List) -> decode_field([Type], List) when is_list(List) ->
[decode_field(Type, X) || X <- List]; [decode_field(Type, X) || X <- List];
decode_field(#{items := Items}, List) when length(List) =:= length(Items) ->
Zipped = lists:zip(Items, List),
lists:foldl(
fun({{K, Type}, V}, Map) ->
maps:is_key(K, Map) andalso error(badarg, duplicate_field),
Map#{ K => decode_field(Type, V) }
end, #{}, Zipped);
decode_field(Type, List) when length(List) =:= tuple_size(Type) -> decode_field(Type, List) when length(List) =:= tuple_size(Type) ->
Zipped = lists:zip(tuple_to_list(Type), List), Zipped = lists:zip(tuple_to_list(Type), List),
list_to_tuple([decode_field(X, Y) || {X, Y} <- Zipped]); list_to_tuple([decode_field(X, Y) || {X, Y} <- Zipped]);
decode_field(uint128, X) when is_binary(X) ->
I = binary:decode_unsigned(X),
if I < (1 bsl 128) -> I;
true -> error({illegal, uint128, X})
end;
decode_field(uint64, X) when is_binary(X) ->
I = binary:decode_unsigned(X),
if I < (1 bsl 64) -> I;
true -> error({illegal, uint64, X})
end;
decode_field(uint32, X) when is_binary(X) ->
I = binary:decode_unsigned(X),
if I < (1 bsl 32) -> I;
true -> error({illegal, uint32, X})
end;
decode_field(uint16, X) when is_binary(X) ->
I = binary:decode_unsigned(X),
if I < (1 bsl 16) -> I;
true -> error({illegal, uint16, X})
end;
decode_field(uint8, X) when is_binary(X) ->
I = binary:decode_unsigned(X),
if I < (1 bsl 8) -> I;
true -> error({illegal, uint8, X})
end;
decode_field(int, <<0:8, X/binary>> = B) when X =/= <<>> -> decode_field(int, <<0:8, X/binary>> = B) when X =/= <<>> ->
error({illegal, int, B}); error({illegal, int, B});
decode_field(int, X) when is_binary(X) -> binary:decode_unsigned(X); decode_field(int, X) when is_binary(X) -> binary:decode_unsigned(X);
@@ -127,7 +188,7 @@ decode_field(binary, X) when is_binary(X) -> X;
decode_field(bool, <<1:8>>) -> true; decode_field(bool, <<1:8>>) -> true;
decode_field(bool, <<0:8>>) -> false; decode_field(bool, <<0:8>>) -> false;
decode_field(id, Val) -> decode_field(id, Val) ->
try aeser_id:decode(Val) try gmser_id:decode(Val)
catch _:_ -> error({illegal, id, Val}) catch _:_ -> error({illegal, id, Val})
end; end;
decode_field(Type, X) -> error({illegal, Type, X}). decode_field(Type, X) -> error({illegal, Type, X}).
@@ -1,12 +1,13 @@
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
%%% @copyright (C) 2025, QPQ AG
%%% @copyright (C) 2018, Aeternity Anstalt %%% @copyright (C) 2018, Aeternity Anstalt
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
-module(aeser_api_encoder_tests). -module(gmser_api_encoder_tests).
-include_lib("eunit/include/eunit.hrl"). -include_lib("eunit/include/eunit.hrl").
-define(TEST_MODULE, aeser_api_encoder). -define(TEST_MODULE, gmser_api_encoder).
-define(TYPES, [ {key_block_hash , 32} -define(TYPES, [ {key_block_hash , 32}
, {micro_block_hash , 32} , {micro_block_hash , 32}
, {block_tx_hash , 32} , {block_tx_hash , 32}
@@ -15,17 +16,45 @@
, {contract_pubkey , 32} , {contract_pubkey , 32}
, {transaction , not_applicable} , {transaction , not_applicable}
, {tx_hash , 32} , {tx_hash , 32}
, {oracle_pubkey , 32}
, {oracle_query_id , 32}
, {account_pubkey , 32} , {account_pubkey , 32}
, {signature , 64} , {signature , 64}
, {name , not_applicable} , {name , not_applicable}
, {native_token , 32}
, {commitment , 32} , {commitment , 32}
, {peer_pubkey , 32} , {peer_pubkey , 32}
, {hash , 32}
, {state , 32} , {state , 32}
, {poi , not_applicable}]). , {poi , not_applicable}]).
encode_decode_test_() -> encode_decode_test_() ->
encode_decode_test_(?TYPES).
encode_decode_known_types_test_() ->
KnownTypes = known_types(),
SizedTypes = [{T, ?TEST_MODULE:byte_size_for_type(T)} || T <- KnownTypes],
encode_decode_test_(SizedTypes).
prefixes_are_known_types_test() ->
MappedPfxs = mapped_prefixes(),
KnownTypes = known_types(),
lists:foreach(
fun({Pfx, Type}) ->
case lists:member(Type, KnownTypes) of
true -> ok;
false ->
error({not_a_known_type, Pfx, Type})
end
end, MappedPfxs),
lists:foreach(
fun(Type) ->
case lists:keyfind(Type, 2, MappedPfxs) of
{_, _} -> ok;
false ->
error({has_no_mapped_prefix, Type})
end
end, KnownTypes).
encode_decode_test_(Types) ->
[{"Byte sizes are correct", [{"Byte sizes are correct",
fun() -> fun() ->
lists:foreach( lists:foreach(
@@ -33,7 +62,7 @@ encode_decode_test_() ->
{_Type, _, ByteSize} = {Type, ByteSize, {_Type, _, ByteSize} = {Type, ByteSize,
?TEST_MODULE:byte_size_for_type(Type)} ?TEST_MODULE:byte_size_for_type(Type)}
end, end,
?TYPES) Types)
end end
}, },
{"Serialize/deserialize known types", {"Serialize/deserialize known types",
@@ -50,7 +79,7 @@ encode_decode_test_() ->
{Type, Key} = ?TEST_MODULE:decode(EncodedKey), {Type, Key} = ?TEST_MODULE:decode(EncodedKey),
{ok, Key} = ?TEST_MODULE:safe_decode(Type, EncodedKey) {ok, Key} = ?TEST_MODULE:safe_decode(Type, EncodedKey)
end, end,
?TYPES) Types)
end end
}, },
{"Key size check works", {"Key size check works",
@@ -58,17 +87,18 @@ encode_decode_test_() ->
lists:foreach( lists:foreach(
fun({_Type, not_applicable}) -> ok; fun({_Type, not_applicable}) -> ok;
({Type, ByteSize}) -> ({Type, ByteSize}) ->
CheckIlligalSize = CheckIllegalSize =
fun(S) -> fun(S) ->
Key = <<42:S/unit:8>>, Key = <<42:S/unit:8>>,
EncodedKey = ?TEST_MODULE:encode(Type, Key), ?assertError(incorrect_size, ?TEST_MODULE:encode(Type, Key)),
EncodedKey = ?TEST_MODULE:unsafe_encode(Type, Key), %% no size check
{error, invalid_encoding} = ?TEST_MODULE:safe_decode(Type, EncodedKey) {error, invalid_encoding} = ?TEST_MODULE:safe_decode(Type, EncodedKey)
end, end,
CheckIlligalSize(0), CheckIllegalSize(0),
CheckIlligalSize(ByteSize - 1), CheckIllegalSize(ByteSize - 1),
CheckIlligalSize(ByteSize + 1) CheckIllegalSize(ByteSize + 1)
end, end,
?TYPES) Types)
end end
}, },
{"Missing prefix", {"Missing prefix",
@@ -91,7 +121,7 @@ encode_decode_test_() ->
<<_WholePrefix:3/unit:8, RestOfKey2/binary>> = EncodedKey, <<_WholePrefix:3/unit:8, RestOfKey2/binary>> = EncodedKey,
{error, invalid_encoding} = ?TEST_MODULE:safe_decode(Type, RestOfKey2) {error, invalid_encoding} = ?TEST_MODULE:safe_decode(Type, RestOfKey2)
end, end,
?TYPES) Types)
end end
}, },
{"Piece of encoded key", {"Piece of encoded key",
@@ -110,7 +140,7 @@ encode_decode_test_() ->
{error, invalid_encoding} = ?TEST_MODULE:safe_decode(Type, HalfKey), {error, invalid_encoding} = ?TEST_MODULE:safe_decode(Type, HalfKey),
{error, invalid_encoding} = ?TEST_MODULE:safe_decode(Type, RestOfKey) {error, invalid_encoding} = ?TEST_MODULE:safe_decode(Type, RestOfKey)
end, end,
?TYPES) Types)
end end
}, },
{"Encode/decode binary with only zeros", {"Encode/decode binary with only zeros",
@@ -131,8 +161,67 @@ encode_decode_test_() ->
Encoded1 = base58:binary_to_base58(Bin), Encoded1 = base58:binary_to_base58(Bin),
Decoded1 = base58:base58_to_binary(Encoded1), Decoded1 = base58:base58_to_binary(Encoded1),
?assertEqual(Bin, Decoded1) ?assertEqual(Bin, Decoded1)
end, ?TYPES) end, Types)
end, end,
Bins) Bins)
end} end},
{"Encode/decode keypairs",
fun() ->
KP1 = enacl:sign_keypair(),
Enc1 = ?TEST_MODULE:encode_keypair(KP1),
{ok, KP1} = ?TEST_MODULE:safe_decode_keypair(Enc1),
KP2 = enacl:sign_keypair(),
Enc2 = ?TEST_MODULE:encode_keypair(KP2),
{ok, KP2} = ?TEST_MODULE:safe_decode_keypair(Enc2),
BadEnc = Enc1#{~"priv" => maps:get(~"priv", Enc2)},
{error, illegal_encoding} = ?TEST_MODULE:safe_decode_keypair(BadEnc)
end
},
{"Encode AND decode both 32-byte and 64-byte account_seckey",
fun() ->
%% Originally, we could encode a 64-byte seckey, but decode would fail.
#{public := Pub, secret := Sec} = enacl:sign_keypair(),
<<Seed:32/binary, Pub:32/binary>> = Sec,
EncSeed = ?TEST_MODULE:encode(account_seckey, Seed),
EncSec = ?TEST_MODULE:encode(account_seckey, Sec),
{ok, Seed} = ?TEST_MODULE:safe_decode(account_seckey, EncSeed),
{ok, Sec} = ?TEST_MODULE:safe_decode(account_seckey, EncSec)
end
}
]. ].
known_types() ->
Forms = get_forms(),
[{type, _, union, Types}] =
[Def || {attribute, _, type, {known_type, Def, []}} <- Forms],
lists:flatmap(fun known_type_entry/1, Types).
known_type_entry({atom, _, Name}) ->
[Name];
known_type_entry({type, _, tuple,
[{atom, _, account_pubkey},
{type, _, range, [{integer, _, Lo}, {integer, _, Hi}]}]}) ->
[{account_pubkey, N} || N <- lists:seq(Lo, Hi)];
known_type_entry(Other) ->
error({unsupported_known_type, Other}).
mapped_prefixes() ->
Forms = get_forms(),
[Clauses] = [Cs || {function,_,pfx2type,1,Cs} <- Forms],
Abst = [{B, A} || {clause,_,[B],[],[A]} <- Clauses],
lists:map(
fun({B, A}) ->
{eval_expr(B), eval_expr(A)}
end, Abst).
get_forms() ->
get_forms(code:which(?TEST_MODULE)).
get_forms(Beam) ->
{ok, {_, [{abstract_code, {raw_abstract_v1, Forms}}]}} =
beam_lib:chunks(Beam, [abstract_code]),
Forms.
eval_expr(Expr) ->
{value, Val, []} = erl_eval:expr(Expr, []),
Val.
@@ -1,11 +1,12 @@
%%% -*- erlang-indent-level:4; indent-tabs-mode: nil -*- %%% -*- erlang-indent-level:4; indent-tabs-mode: nil -*-
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
%%% @copyright (C) 2025, QPQ AG
%%% @copyright (C) 2018, Aeternity Anstalt %%% @copyright (C) 2018, Aeternity Anstalt
%%% @doc %%% @doc
%%% EUnit tests for aeser_chain_objects %%% EUnit tests for gmser_chain_objects
%%% @end %%% @end
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
-module(aeser_chain_objects_tests). -module(gmser_chain_objects_tests).
-include_lib("eunit/include/eunit.hrl"). -include_lib("eunit/include/eunit.hrl").
@@ -78,10 +79,10 @@ deserialize(Template, Bin) ->
deserialize(Template, Bin, ?DEFAULT_TAG, ?DEFAULT_VERSION). deserialize(Template, Bin, ?DEFAULT_TAG, ?DEFAULT_VERSION).
deserialize(Template, Bin, Tag, Vsn) -> deserialize(Template, Bin, Tag, Vsn) ->
aeser_chain_objects:deserialize(Tag, Vsn, Template, Bin). gmser_chain_objects:deserialize(Tag, Vsn, Template, Bin).
serialize(Template, Bin) -> serialize(Template, Bin) ->
serialize(Template, Bin, ?DEFAULT_TAG, ?DEFAULT_VERSION). serialize(Template, Bin, ?DEFAULT_TAG, ?DEFAULT_VERSION).
serialize(Template, Bin, Tag, Vsn) -> serialize(Template, Bin, Tag, Vsn) ->
aeser_chain_objects:serialize(Tag, Vsn, Template, Bin). gmser_chain_objects:serialize(Tag, Vsn, Template, Bin).
@@ -1,7 +1,11 @@
-module(aeser_contract_code_tests). %%%-------------------------------------------------------------------
%%% @copyright (C) 2025, QPQ AG
%%%-------------------------------------------------------------------
-module(gmser_contract_code_tests).
-include_lib("eunit/include/eunit.hrl"). -include_lib("eunit/include/eunit.hrl").
-include("aeser_contract_code.hrl"). -include("gmser_contract_code.hrl").
-define(DUMMY_CODE_MAP_1, -define(DUMMY_CODE_MAP_1,
#{ byte_code => <<"DUMMY CODE">> #{ byte_code => <<"DUMMY CODE">>
@@ -22,14 +26,14 @@
, payable => true} ). , payable => true} ).
vsn_1_test() -> vsn_1_test() ->
aeser_contract_code:deserialize( gmser_contract_code:deserialize(
aeser_contract_code:serialize(?DUMMY_CODE_MAP_1, ?SOPHIA_CONTRACT_VSN_1)). gmser_contract_code:serialize(?DUMMY_CODE_MAP_1, ?SOPHIA_CONTRACT_VSN_1)).
vsn_2_test() -> vsn_2_test() ->
aeser_contract_code:deserialize( gmser_contract_code:deserialize(
aeser_contract_code:serialize(?DUMMY_CODE_MAP_2, ?SOPHIA_CONTRACT_VSN_2)). gmser_contract_code:serialize(?DUMMY_CODE_MAP_2, ?SOPHIA_CONTRACT_VSN_2)).
vsn_3_test() -> vsn_3_test() ->
aeser_contract_code:deserialize( gmser_contract_code:deserialize(
aeser_contract_code:serialize(?DUMMY_CODE_MAP_3, ?SOPHIA_CONTRACT_VSN_3)). gmser_contract_code:serialize(?DUMMY_CODE_MAP_3, ?SOPHIA_CONTRACT_VSN_3)).
+49
View File
@@ -0,0 +1,49 @@
%%%-------------------------------------------------------------------
%%% @copyright (C) 2025, QPQ AG
%%% @copyright (C) 2023, Aeternity Anstalt
%%%-------------------------------------------------------------------
-module(gmser_delegation_tests).
-include_lib("eunit/include/eunit.hrl").
-define(TEST_MODULE, gmser_delegation).
-define(ACCOUNT, gmser_id:create(account, <<1:256>>)).
-define(CONTRACT, gmser_id:create(contract, <<2:256>>)).
-define(NAME, gmser_id:create(name, <<3:256>>)).
-define(NETWORK_ID, <<"my_fancy_network"/utf8>>).
encode_correct_test_() ->
[{"Encode preclaim sig",
fun() ->
gmser_delegation:aens_preclaim_sig(?NETWORK_ID, ?ACCOUNT, ?CONTRACT)
end},
{"Encode name sig",
fun() ->
gmser_delegation:aens_name_sig(?NETWORK_ID, ?ACCOUNT, ?NAME, ?CONTRACT)
end},
{"Encode aens wildcard sig",
fun() ->
gmser_delegation:aens_sig(?NETWORK_ID, ?ACCOUNT, ?CONTRACT)
end}
].
encode_fail_test_() ->
[{"Bad encoding preclaim sig",
fun() ->
?assertError(_, gmser_delegation:aens_preclaim_sig(?NETWORK_ID, <<42:256>>, ?CONTRACT)),
?assertError(_, gmser_delegation:aens_preclaim_sig(?NETWORK_ID, ?CONTRACT, ?ACCOUNT))
end},
{"Bad encoding name sig",
fun() ->
?assertError(_, gmser_delegation:aens_name_sig(?NETWORK_ID, ?ACCOUNT, <<42:256>>, ?CONTRACT)),
?assertError(_, gmser_delegation:aens_name_sig(?NETWORK_ID, ?NAME, ?ACCOUNT, ?CONTRACT))
end},
{"Bad encoding aens wildcard sig",
fun() ->
?assertError(_, gmser_delegation:aens_sig(?NETWORK_ID, ?ACCOUNT, <<42:256>>)),
?assertError(_, gmser_delegation:aens_sig(?NETWORK_ID, ?CONTRACT, ?CONTRACT))
end}
].
+44
View File
@@ -0,0 +1,44 @@
-module(gmser_id_tests).
-include_lib("eunit/include/eunit.hrl").
-define(PUBKEY, <<12345:32/unit:8>>).
is_account_test() ->
{"is_account recognizes standard and extended account ids",
fun() ->
?assert(gmser_id:is_account(gmser_id:create(account, ?PUBKEY))),
?assert(gmser_id:is_account(gmser_id:create({account, 0}, ?PUBKEY))),
?assert(gmser_id:is_account(gmser_id:create({account, 5}, ?PUBKEY))),
?assertNot(gmser_id:is_account(gmser_id:create(contract, ?PUBKEY))),
?assertNot(gmser_id:is_account(not_an_id))
end}.
account_pubkey_test() ->
{"account_pubkey returns the 32-byte account hash",
fun() ->
?assertEqual(?PUBKEY,
gmser_id:account_pubkey(gmser_id:create(account, ?PUBKEY))),
?assertEqual(?PUBKEY,
gmser_id:account_pubkey(gmser_id:create({account, 3}, ?PUBKEY))),
?assertEqual(?PUBKEY,
gmser_id:account_pubkey(gmser_id:create({account, 6}, ?PUBKEY)))
end}.
account_pubkey_matches_specialize_test() ->
{"account_pubkey agrees with specialize/2 for account ids",
fun() ->
Id = gmser_id:create({account, 2}, ?PUBKEY),
?assertEqual(gmser_id:specialize(Id, account),
gmser_id:account_pubkey(Id))
end}.
extended_account_roundtrip_test() ->
{"extended account ids round-trip through encode/decode",
fun() ->
Id = gmser_id:create({account, 4}, ?PUBKEY),
?assert(gmser_id:is_account(Id)),
Id1 = gmser_id:decode(gmser_id:encode(Id)),
?assertEqual(?PUBKEY, gmser_id:account_pubkey(Id1)),
?assertEqual(account, gmser_id:specialize_type(Id1))
end}.
@@ -1,10 +1,11 @@
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
%%% @copyright (C) 2025, QPQ AG
%%% @copyright (C) 2017, Aeternity Anstalt %%% @copyright (C) 2017, Aeternity Anstalt
%%% @doc Tests for Recursive Length Prefix %%% @doc Tests for Recursive Length Prefix
%%% @end %%% @end
%%%------------------------------------------------------------------- %%%-------------------------------------------------------------------
-module(aeser_rlp_tests). -module(gmser_rlp_tests).
-include_lib("eunit/include/eunit.hrl"). -include_lib("eunit/include/eunit.hrl").
@@ -13,7 +14,7 @@
-define(BYTE_ARRAY_OFFSET , 128). -define(BYTE_ARRAY_OFFSET , 128).
-define(LIST_OFFSET , 192). -define(LIST_OFFSET , 192).
-define(TEST_MODULE, aeser_rlp). -define(TEST_MODULE, gmser_rlp).
rlp_one_byte_test() -> rlp_one_byte_test() ->
B = <<42>>, B = <<42>>,
+17
View File
@@ -0,0 +1,17 @@
{name,"Gajumaru Serialization"}.
{type,lib}.
{modules,[]}.
{prefix,none}.
{desc,"Serialization helpers for the Gajumaru."}.
{author,"Hans Svensson"}.
{package_id,{"otpr","gmserialization",{0,2,0}}}.
{deps,[{"otpr","eblake2",{1,0,1}},{"otpr","base58",{0,1,1}}]}.
{key_name,none}.
{a_email,[]}.
{c_email,[]}.
{copyright,"QPQ AG"}.
{file_exts,[]}.
{license,skip}.
{repo_url,"https://git.qpq.swiss/QPQ-AG/gmserialization"}.
{tags,["blockchain","crypto","gm","gajumaru"]}.
{ws_url,[]}.
Executable
+49
View File
@@ -0,0 +1,49 @@
#!/bin/sh
set -e
APP=$(basename "$PWD")
SRC="_build/default/lib/$APP"
DST="$PWD/_build/zomp/lib/$APP"
IGNORE_FILE="zomp.ignore"
mkdir -p "$DST"
# Remove broken symlinks
find "$SRC" -type l ! -exec test -e {} \; -delete || true
# Build ignore matcher
IGNORE_TEMP=$(mktemp)
trap "rm -f $IGNORE_TEMP" EXIT
# Expand globs in zomp.ignore to patterns suitable for grep
if [ -e "$IGNORE_FILE" ]; then
grep -v '^\s*#' "$IGNORE_FILE" | sed 's#/#\\/#g' | sed 's/\./\\./g' | sed 's/\*/.*/g' > "$IGNORE_TEMP"
fi
# Copy Git-tracked and Zomp-allowed files
git ls-files -z | while IFS= read -r -d '' file; do
# Skip if ignored
echo "$file" | grep -Eq -f "$IGNORE_TEMP" && continue
# Only copy if file exists in the build dir
if [ -e "$SRC/$file" ]; then
mkdir -p "$DST/$(dirname "$file")"
cp -a "$SRC/$file" "$DST/$file"
fi
done
rm "$IGNORE_TEMP"
# Copy metadata
cp "$PWD/zomp.meta" "$DST/"
cp "$PWD/Emakefile" "$DST/"
# copy generated schema
SCHEMA="$SRC/priv/gmhc_schema.json"
if [ -e "$SCHEMA" ]; then
mkdir -p "$DST/priv"
cp -a "$SCHEMA" "$DST/priv/$(basename "$SCHEMA")"
fi
# Clean up beam files just in case
[ -d "$DST/ebin" ] && find "$DST/ebin" -name '*.beam' -exec rm -f {} + || true