45 Commits

Author SHA1 Message Date
dragan
d6b86524fd Fix wrong index preparation in mrdb:index_read_/3
In the case when index is of type {_}, index is returned as is
instead of putting it into a tuple {Ix, ordered}.

An example

Given the following index plugin implementation:
-module(key6).

-record(cgh, {key, val}).

-export([k6/3
        , add_index/0
        , create_table/0
        , tst_bad_type/0
    ]).

k6(_, _, #cgh{key = {Name, K8, _Cid, _Vn}}) ->
    [list_to_binary([atom_to_binary(Name),
     list_to_binary(lists:sublist(K8, 6))])].

add_index() ->
    mnesia_schema:add_index_plugin({k6}, key6, k6).

create_table() ->
    mnesia:create_table(cgh,
    [{attributes, record_info(fields, cgh)},
     {type, ordered_set},
     {index, [{k6}]},
     {rocksdb_copies, [node()]},
     {local_content, true}]).

tst_bad_type() ->
    E1 = #cgh{key = {n1, "u1vgrjkh",{ecgi,238,6,213020,50},3}, val = ok},
    mnesia:dirty_write(E1),
    Index1 = list_to_binary([atom_to_binary(n1),
                             list_to_binary(lists:sublist("u1vgrjkh", 6))]),
    mrdb:index_read(cgh, Index1, {k6}).

-- testing --
Eshell V14.2.2 (press Ctrl+G to abort, type help(). for help)
(rock@bang)2> mnesia:create_schema([node()]).
ok
(rock@bang)3> mnesia:start().
ok
(rock@bang)4> mnesia_rocksdb:register().
{ok,rocksdb_copies}
(rock@bang)5> l(key6).
{module,key6}
(rock@bang)6> key6:add_index().
{atomic,ok}
(rock@bang)7> key6:create_table().
{atomic,ok}
(rock@bang)8> key6:tst_bad_type().
** exception exit: {aborted,{bad_type,{cgh,index,{k6}}}}
     in function  mnesia:abort/1 (mnesia.erl, line 362)
     in call from mrdb:ensure_ref/1 (~/projects/erlang/github/mnesia_rocksdb/src/mrdb.erl, line 609)
     in call from mrdb:index_read_/3 (~/projects/erlang/github/mnesia_rocksdb/src/mrdb.erl, line 945)

-- after patch --

(rock@bang)9> c(mrdb).
Recompiling ~/projects/erlang/github/mnesia_rocksdb/src/mrdb.erl
{ok,mrdb}
(rock@bang)10> key6:tst_bad_type().
[{cgh,{n1,"u1vgrjkh",{ecgi,238,6,213020,50},3},ok}]
(rock@bang)11>
2024-08-12 14:32:20 +02:00
Thomas Arts
16d0533acf Fix dialyzer errors and warnings 2024-03-19 14:08:14 +01:00
Ulf Wiger
71ebaf2d66 New args for rocksdb:transaction_iterator() 2024-03-18 17:40:42 +01:00
Ulf Wiger
57f02078bc Fix typo 2023-10-05 11:53:41 +02:00
Ulf Wiger
791cec41db Adding/deleting indexes cleans up index metadata 2023-10-05 11:53:41 +02:00
Ulf Wiger
465a220bfe Update comment about mutex implementation 2022-11-03 13:37:52 +01:00
Ulf Wiger
3635eac717 Test case for inner retries; add mrdb_stats.erl; update docs 2022-11-03 12:41:14 +01:00
Ulf Wiger
95abe4e36e Mutex server with fifo queues 2022-11-01 10:11:20 +01:00
Ulf Wiger
7c729bd932 Use a serializing mutex 2022-11-01 10:09:48 +01:00
Ulf Wiger
d1a6bf22d5 Don't try to release dummy batch ref 2022-10-31 16:48:23 +01:00
Ulf Wiger
b65e82ed71
Merge pull request #33 from aeternity/uw-batch-on-demand
Begin dirty activity with batch ref dummy
2022-10-26 13:16:48 +02:00
Ulf Wiger
ce2be519b4 Begin dirty activity with batch ref dummy 2022-10-24 15:15:01 +02:00
Ulf Wiger
8073a0daa5 Tx push at retry before mutex instead of after 2022-10-19 13:19:48 +02:00
Ulf Wiger
b908998e6b Check pdict for dumper state at close_table 2022-08-02 17:07:53 +02:00
Ulf Wiger
dfc0125800 More dets-like load/close behavior 2022-07-25 15:33:00 +02:00
Ulf Wiger
296abb23bb Fix double encoding of info data 2022-07-22 16:33:32 +02:00
Ulf Wiger
7057f4dcbd Remove rocksdb opts refault, safer index consistency check 2022-07-22 15:31:55 +02:00
Ulf Wiger
c4235be94a Be more lenient about mnesia asking more than once 2022-07-15 14:14:06 +02:00
Ulf Wiger
fde2e1194e Fix type to satisfy Dialyzer 2022-07-13 11:35:40 +02:00
Ulf Wiger
c4f7b7ac02 Migration chunk handling was broken. Add progress reporting support 2022-07-13 11:19:47 +02:00
Ulf Wiger
b236fe8b4b Make app.src vsn tag automatic 2022-07-11 14:10:08 +02:00
Ulf Wiger
93a296f6b9 Debugging direct access API 2022-07-08 11:23:12 +02:00
Ulf Wiger
0da3169a01 Fix typo in mrdb.erl 2022-06-29 12:44:55 +02:00
Ulf Wiger
e351a02175 Fix batch- and tx-handling refs, formatting annot 2022-06-29 12:19:29 +02:00
Ulf Wiger
b70654d0a2 Default to accessing data via the table owner process 2022-04-13 14:24:53 +02:00
Ulf Wiger
0830cecbf9 Add test/mrdb_bench.erl, remove some debug logging 2022-04-07 18:48:24 +02:00
Ulf Wiger
2339232f59 Fix table load bug, fail properly on mnesia-ordered updates 2022-04-06 14:52:58 +02:00
Ulf Wiger
d5dafb5b7e Refactor to support column families, direct rocksdb access
Expose low-level helpers, fix dialyzer warnings

WIP column families and mrdb API

Basic functionality in place

started adding documentation

remove doc/ from .gitignore

add doc/* files

recognize pre-existing tabs at startup

wip: most of the functionality in place (not yet merge ops)

wip: adding transaction support

wip: add transaction test case (currently dumps core)

First draft, mnesia plugin user guide

Fix note formatting

WIP working on indexing

Index iterators, dialyzer, xref fixes

open db with optimistic transactions

Use rocksdb-1.7.0

Use seanhinde rocksdb patch, enable rollback

Call the right transaction_get() function

WIP add 'snap_tx' activity type

tx restart using mrdb_mutex

Fix test suite sync bugs

WIP instrumented for debugging

WIP working on migration test case

Add migration test suite

Migration works, subscribe to schema changes

WIP fix batch handling

Manage separate batches per db_ref

Add mrdb:fold/3

Add some docs, erlang_ls config

Use seanhinde's rocksdb vsn
2022-03-25 15:48:19 +01:00
Ulf Wiger
c1d949b47e
Merge pull request #6 from arpunk/include-sext
Include sext as dependency
2021-11-17 15:05:59 +01:00
Ulf Wiger
79d6467d26 Expose low-level helpers, fix dialyzer warnings 2020-12-22 10:36:24 +01:00
Ulf Wiger
35c79aaa32 WIP update deps, fix try-catch warning, for rocksdb 1.6.0 2020-10-19 10:46:02 +02:00
Tino Breddin
984d12479d Add test for error store support 2019-11-18 16:43:08 +01:00
Tino Breddin
a2f863804d Formatting improvements 2019-11-15 16:31:09 +01:00
Tino Breddin
63bae181e3 Add caller error store to error handling
During initialization of the backend the caller may provide a new
configuration to define a target ETS table for storing errors which the
backend encountered at runtime. These errors are written during handling
of write errors.
2019-11-15 16:28:39 +01:00
Ulf Wiger
6f41733461 Add tests for error handling
Document reason for breaking out update functions
2019-09-30 14:02:35 +02:00
Ulf Wiger
f93c1aba93 fix on_write_error options 2019-09-24 16:10:13 +02:00
Ulf Wiger
c8da9ce31b Check return values on update operations.
- put(), write() and delete() wrapped, checking for non-ok returns
- Option {on_write_error, verbose | warning | error | fatal}
  added to the rocksdb_opts user property. The corresponding function
  mnesia_lib:Op(Fmt, Args) will be called if an error return is spotted.
2019-09-24 15:45:07 +02:00
arpunk
8662ba331a
include sext as dependency 2018-04-02 09:50:49 -05:00
Ulf Wiger
dd23a80e27 use only normal iterator function 2018-03-06 22:11:41 +01:00
Ulf Wiger
0345e3558b On select, mnesia_rocksdb would only calculate a key prefix for the
first select clause in a multi-clause select pattern. This could give
erroneous results if the first pattern was more specific (on the key)
than following patterns. This change calculates a prefix for all clauses
and keeps the shortest common prefix.
2018-02-16 18:43:59 +01:00
Ulf Wiger
af4c6e8fd2 More renaming, ldb -> rdb 2018-02-06 20:03:01 +01:00
Ulf Wiger
d326daf251
Merge pull request #2 from aeternity/uw-use-official-sext
Use official sext instead of copy
2018-02-06 14:32:52 +01:00
Ulf Wiger
3fccd1cd4f Use official sext instead of copy 2018-02-06 14:30:17 +01:00
Ulf Wiger
a70f84f645 ensure rocksdb started at backend init 2018-02-06 13:58:17 +01:00
Ulf Wiger
6c9f5b565f first version, based on mnesia_eleveldb 2018-02-06 09:43:57 +01:00