QPQ-AG/mnesia_rocksdb
11
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity
97 Commits 10 Branches 2 Tags
996ae8271784a3c2898f83d78cabd71bbe5a1583
Commit Graph
45 Commits
Author SHA1 Message Date
dragan d6b86524fd Fix wrong index preparation in mrdb:index_read_/3 
In the case when index is of type {_}, index is returned as is
instead of putting it into a tuple {Ix, ordered}.

An example

Given the following index plugin implementation:
-module(key6).

-record(cgh, {key, val}).

-export([k6/3
        , add_index/0
        , create_table/0
        , tst_bad_type/0
    ]).

k6(_, _, #cgh{key = {Name, K8, _Cid, _Vn}}) ->
    [list_to_binary([atom_to_binary(Name),
     list_to_binary(lists:sublist(K8, 6))])].

add_index() ->
    mnesia_schema:add_index_plugin({k6}, key6, k6).

create_table() ->
    mnesia:create_table(cgh,
    [{attributes, record_info(fields, cgh)},
     {type, ordered_set},
     {index, [{k6}]},
     {rocksdb_copies, [node()]},
     {local_content, true}]).

tst_bad_type() ->
    E1 = #cgh{key = {n1, "u1vgrjkh",{ecgi,238,6,213020,50},3}, val = ok},
    mnesia:dirty_write(E1),
    Index1 = list_to_binary([atom_to_binary(n1),
                             list_to_binary(lists:sublist("u1vgrjkh", 6))]),
    mrdb:index_read(cgh, Index1, {k6}).

-- testing --
Eshell V14.2.2 (press Ctrl+G to abort, type help(). for help)
(rock@bang)2> mnesia:create_schema([node()]).
ok
(rock@bang)3> mnesia:start().
ok
(rock@bang)4> mnesia_rocksdb:register().
{ok,rocksdb_copies}
(rock@bang)5> l(key6).
{module,key6}
(rock@bang)6> key6:add_index().
{atomic,ok}
(rock@bang)7> key6:create_table().
{atomic,ok}
(rock@bang)8> key6:tst_bad_type().
** exception exit: {aborted,{bad_type,{cgh,index,{k6}}}}
     in function  mnesia:abort/1 (mnesia.erl, line 362)
     in call from mrdb:ensure_ref/1 (~/projects/erlang/github/mnesia_rocksdb/src/mrdb.erl, line 609)
     in call from mrdb:index_read_/3 (~/projects/erlang/github/mnesia_rocksdb/src/mrdb.erl, line 945)

-- after patch --

(rock@bang)9> c(mrdb).
Recompiling ~/projects/erlang/github/mnesia_rocksdb/src/mrdb.erl
{ok,mrdb}
(rock@bang)10> key6:tst_bad_type().
[{cgh,{n1,"u1vgrjkh",{ecgi,238,6,213020,50},3},ok}]
(rock@bang)11>
 2024-08-12 14:32:20 +02:00
Thomas Arts 16d0533acf Fix dialyzer errors and warnings 2024-03-19 14:08:14 +01:00
Ulf Wiger 71ebaf2d66 New args for rocksdb:transaction_iterator() 2024-03-18 17:40:42 +01:00
Ulf Wiger 57f02078bc Fix typo 2023-10-05 11:53:41 +02:00
Ulf Wiger 791cec41db Adding/deleting indexes cleans up index metadata 2023-10-05 11:53:41 +02:00
Ulf Wiger 465a220bfe Update comment about mutex implementation 2022-11-03 13:37:52 +01:00
Ulf Wiger 3635eac717 Test case for inner retries; add mrdb_stats.erl; update docs 2022-11-03 12:41:14 +01:00
Ulf Wiger 95abe4e36e Mutex server with fifo queues 2022-11-01 10:11:20 +01:00
Ulf Wiger 7c729bd932 Use a serializing mutex 2022-11-01 10:09:48 +01:00
Ulf Wiger d1a6bf22d5 Don't try to release dummy batch ref 2022-10-31 16:48:23 +01:00
Ulf Wiger b65e82ed71 Merge pull request #33 from aeternity/uw-batch-on-demand 
Begin dirty activity with batch ref dummy
 2022-10-26 13:16:48 +02:00
Ulf Wiger ce2be519b4 Begin dirty activity with batch ref dummy 2022-10-24 15:15:01 +02:00
Ulf Wiger 8073a0daa5 Tx push at retry before mutex instead of after 2022-10-19 13:19:48 +02:00
Ulf Wiger b908998e6b Check pdict for dumper state at close_table 2022-08-02 17:07:53 +02:00
Ulf Wiger dfc0125800 More dets-like load/close behavior 2022-07-25 15:33:00 +02:00
Ulf Wiger 296abb23bb Fix double encoding of info data 2022-07-22 16:33:32 +02:00
Ulf Wiger 7057f4dcbd Remove rocksdb opts refault, safer index consistency check 2022-07-22 15:31:55 +02:00
Ulf Wiger c4235be94a Be more lenient about mnesia asking more than once 2022-07-15 14:14:06 +02:00
Ulf Wiger fde2e1194e Fix type to satisfy Dialyzer 2022-07-13 11:35:40 +02:00
Ulf Wiger c4f7b7ac02 Migration chunk handling was broken. Add progress reporting support 2022-07-13 11:19:47 +02:00
Ulf Wiger b236fe8b4b Make app.src vsn tag automatic 2022-07-11 14:10:08 +02:00
Ulf Wiger 93a296f6b9 Debugging direct access API 2022-07-08 11:23:12 +02:00
Ulf Wiger 0da3169a01 Fix typo in mrdb.erl 2022-06-29 12:44:55 +02:00
Ulf Wiger e351a02175 Fix batch- and tx-handling refs, formatting annot 2022-06-29 12:19:29 +02:00
Ulf Wiger b70654d0a2 Default to accessing data via the table owner process 2022-04-13 14:24:53 +02:00
Ulf Wiger 0830cecbf9 Add test/mrdb_bench.erl, remove some debug logging 2022-04-07 18:48:24 +02:00
Ulf Wiger 2339232f59 Fix table load bug, fail properly on mnesia-ordered updates 2022-04-06 14:52:58 +02:00
Ulf Wiger d5dafb5b7e Refactor to support column families, direct rocksdb access 
Expose low-level helpers, fix dialyzer warnings

WIP column families and mrdb API

Basic functionality in place

started adding documentation

remove doc/ from .gitignore

add doc/* files

recognize pre-existing tabs at startup

wip: most of the functionality in place (not yet merge ops)

wip: adding transaction support

wip: add transaction test case (currently dumps core)

First draft, mnesia plugin user guide

Fix note formatting

WIP working on indexing

Index iterators, dialyzer, xref fixes

open db with optimistic transactions

Use rocksdb-1.7.0

Use seanhinde rocksdb patch, enable rollback

Call the right transaction_get() function

WIP add 'snap_tx' activity type

tx restart using mrdb_mutex

Fix test suite sync bugs

WIP instrumented for debugging

WIP working on migration test case

Add migration test suite

Migration works, subscribe to schema changes

WIP fix batch handling

Manage separate batches per db_ref

Add mrdb:fold/3

Add some docs, erlang_ls config

Use seanhinde's rocksdb vsn
 2022-03-25 15:48:19 +01:00
Ulf Wiger c1d949b47e Merge pull request #6 from arpunk/include-sext 
Include sext as dependency
 2021-11-17 15:05:59 +01:00
Ulf Wiger 79d6467d26 Expose low-level helpers, fix dialyzer warnings 2020-12-22 10:36:24 +01:00
Ulf Wiger 35c79aaa32 WIP update deps, fix try-catch warning, for rocksdb 1.6.0 2020-10-19 10:46:02 +02:00
Tino Breddin 984d12479d Add test for error store support 2019-11-18 16:43:08 +01:00
Tino Breddin a2f863804d Formatting improvements 2019-11-15 16:31:09 +01:00
Tino Breddin 63bae181e3 Add caller error store to error handling 
During initialization of the backend the caller may provide a new
configuration to define a target ETS table for storing errors which the
backend encountered at runtime. These errors are written during handling
of write errors.
 2019-11-15 16:28:39 +01:00
Ulf Wiger 6f41733461 Add tests for error handling 
Document reason for breaking out update functions
 2019-09-30 14:02:35 +02:00
Ulf Wiger f93c1aba93 fix on_write_error options 2019-09-24 16:10:13 +02:00
Ulf Wiger c8da9ce31b Check return values on update operations. 
- put(), write() and delete() wrapped, checking for non-ok returns
- Option {on_write_error, verbose | warning | error | fatal}
  added to the rocksdb_opts user property. The corresponding function
  mnesia_lib:Op(Fmt, Args) will be called if an error return is spotted.
 2019-09-24 15:45:07 +02:00
arpunk 8662ba331a include sext as dependency 2018-04-02 09:50:49 -05:00
Ulf Wiger dd23a80e27 use only normal iterator function 2018-03-06 22:11:41 +01:00
Ulf Wiger 0345e3558b On select, mnesia_rocksdb would only calculate a key prefix for the 
first select clause in a multi-clause select pattern. This could give
erroneous results if the first pattern was more specific (on the key)
than following patterns. This change calculates a prefix for all clauses
and keeps the shortest common prefix.
 2018-02-16 18:43:59 +01:00
Ulf Wiger af4c6e8fd2 More renaming, ldb -> rdb 2018-02-06 20:03:01 +01:00
Ulf Wiger d326daf251 Merge pull request #2 from aeternity/uw-use-official-sext 
Use official sext instead of copy
 2018-02-06 14:32:52 +01:00
Ulf Wiger 3fccd1cd4f Use official sext instead of copy 2018-02-06 14:30:17 +01:00
Ulf Wiger a70f84f645 ensure rocksdb started at backend init 2018-02-06 13:58:17 +01:00
Ulf Wiger 6c9f5b565f first version, based on mnesia_eleveldb 2018-02-06 09:43:57 +01:00