From d9dbabf736a575b5f34aec33d1ccb87a2a5ea4ea Mon Sep 17 00:00:00 2001 From: Jesper Louis Andersen Date: Sat, 29 Nov 2014 16:12:21 +0100 Subject: [PATCH] Improve ordering of the README file. --- README.md | 69 +++++++++++++++++++++++++++++-------------------------- 1 file changed, 36 insertions(+), 33 deletions(-) diff --git a/README.md b/README.md index 6c3b9af..1cb43b2 100644 --- a/README.md +++ b/README.md @@ -15,38 +15,6 @@ In addition, I would like to thank Steve Vinoski and Sverker Eriksson for provid * Write Eunit/Common Test cases which verifies that the byte-output of the functions matches the expected output from the NaCl library. -# Rationale - -Doing crypto right in Erlang is not that easy. For one, the crypto system has to be rather fast, which rules out Erlang as the main vehicle. Second, cryptographic systems must be void of timing attacks. This mandates we write the code in a language where we can avoid cache timing attacks. This leaves only C as a contender, more or less. The obvious way to handle this is by the use of NIF implementations, but most C code will run to its conclusion once set off for processing. This is a major problem for a system which needs to keep its latency in check. The solution taken by this library is to use the new Dirty Scheduler API of Erlang in order to provide a safe way to handle the long-running cryptographic processing. It keeps the cryptographic primitives on the dirty schedulers and thus it avoids the major problem. - -Focus has first and foremost been on the correct use of dirty schedulers, without any regard for speed. The plan is to extend the underlying implementation, while keeping the API stable. In a future version, we might want to make simple short-lived crypto-calls directly on the Erlang scheduler rather than moving these to a separate scheduler and paying the price of scheduler invocation. - -Also, while the standard `crypto` bindings in Erlang does a great job at providing cryptographic primitives, these are based on OpenSSL, which is known to be highly problematic in many ways. It is not as easy to use the OpenSSL library correctly as it is with these bindings. Rather than providing a low-level cipher suite, NaCl provides intermediate level primitives constructed as to protect the user against typical low-level cryptographic gotchas and problems. - -## Scheduler handling - -The major problem which a NIF library has to address is the problem of blocking Erlang schedulers. A long-running NIF messes with the scheduler in many ways, the worst of which is breaking it. To avoid this, we have to address long-running work on the NIF. The current method used is to care about the *progress* of the system rather than the *precision*. That is, we guarantee the system will always quickly progress toward a new process, even when running many cryptographic NIFs are run in a given process. However, we don't care about the precision of the progress. A cryptographic NIF may get either a free ride on the reduction budget, or be penalized more than it should be. - -The current approach is to switch between blocking NIF calls and dirty scheduler use at a breakoff threshold. Currently, we use the meaurements obtained by assuming a schedule of 100μs is 1/10th of a 1ms budget. And then we set a reduction budget based on these values. 100μs is roughly set at 200 reductions. And to be on the safe side, we multiply these values by two to handle older CPUs as well too. Measurements are obtained by running: - - enacl_timing:all(). - -The current "typical modern machine" is: - - Intel Core i7-4900QM - -I'm interested in machines for which the schedules end up being far off. That is, machines for which the current CPU schedule takes more than 250μs. This is especially interesting for virtual machines. - -# Testing - -Every primitive has been stress-tested through the use of Erlang QuickCheck with both *positive* and *negative* testing. This has been used to check against memory leaks as well as correct invocation. Please report any error so we can extend the test cases to include a randomized test which captures the problem so we generically catch every problem in a given class of errors. - -Positive and negative testing refers to Type I and Type II errors in statistical testing. This means false positives—given a *valid* input the function rejects it; as well as false negatives—given an *invalid* input the functions fails to reject that input. - -The problem however, is that while we are testing the API level, we can't really test the strength of the cryptographic primitives. We can verify their correctness by trying different standard correctness tests for the primitives, verifying that the output matches the expected one given a specific input. But there is no way we can show that the cryptographic primitive has the strength we want. Thus, we opted to mostly test the API and its invocation for stability. - -Also, in addition to correctness, testing the system like this makes sure we have no memory leaks as they will show themselves under the extensive QuickCheck test cases we run. It has been verified there are no leaks in the code. - # Overview The NaCl cryptographic library provides a number of different cryptographic primitives. In the following, we split up the different generic primitives and explain them briefly. @@ -74,4 +42,39 @@ This implements cryptography where there is a shared secret key between parties. ## Low-level functions * *Hashing:* Cryptographically secure hashing -* *String comparison:* Implements guaranteed constant-time string comparisons to protect against timing attacks. \ No newline at end of file +* *String comparison:* Implements guaranteed constant-time string comparisons to protect against timing attacks. + +# Rationale + +Doing crypto right in Erlang is not that easy. For one, the crypto system has to be rather fast, which rules out Erlang as the main vehicle. Second, cryptographic systems must be void of timing attacks. This mandates we write the code in a language where we can avoid cache timing attacks. This leaves only C as a contender, more or less. The obvious way to handle this is by the use of NIF implementations, but most C code will run to its conclusion once set off for processing. This is a major problem for a system which needs to keep its latency in check. The solution taken by this library is to use the new Dirty Scheduler API of Erlang in order to provide a safe way to handle the long-running cryptographic processing. It keeps the cryptographic primitives on the dirty schedulers and thus it avoids the major problem. + +Focus has first and foremost been on the correct use of dirty schedulers, without any regard for speed. The plan is to extend the underlying implementation, while keeping the API stable. In a future version, we might want to make simple short-lived crypto-calls directly on the Erlang scheduler rather than moving these to a separate scheduler and paying the price of scheduler invocation. + +Also, while the standard `crypto` bindings in Erlang does a great job at providing cryptographic primitives, these are based on OpenSSL, which is known to be highly problematic in many ways. It is not as easy to use the OpenSSL library correctly as it is with these bindings. Rather than providing a low-level cipher suite, NaCl provides intermediate level primitives constructed as to protect the user against typical low-level cryptographic gotchas and problems. + +## Scheduler handling + +The major problem which a NIF library has to address is the problem of blocking Erlang schedulers. A long-running NIF messes with the scheduler in many ways, the worst of which is breaking it. To avoid this, we have to address long-running work on the NIF. The current method used is to care about the *progress* of the system rather than the *precision*. That is, we guarantee the system will always quickly progress toward a new process, even when running many cryptographic NIFs are run in a given process. However, we don't care about the precision of the progress. A cryptographic NIF may get either a free ride on the reduction budget, or be penalized more than it should be. + +The current approach is to switch between blocking NIF calls and dirty scheduler use at a breakoff threshold. Currently, we use the meaurements obtained by assuming a schedule of 100μs is 1/10th of a 1ms budget. And then we set a reduction budget based on these values. 100μs is roughly set at 200 reductions. And to be on the safe side, we multiply these values by two to handle older CPUs as well too. Measurements are obtained by running: + + enacl_timing:all(). + +The current "typical modern machine" is: + + Intel Core i7-4900QM + +When running benchmarks, we warm the CPU a bit before conducting the benchmark. Also, the script `benchmark.sh` can be used (altered to your CPU type), to disable the powersave mode of CPUs in order to obtain realistic benchmarks. Do note nothing was done to get a realistic disable of Intel's Turbo Boost functionality and this is a one-core benchmark. + +I'm interested in machines for which the schedules end up being far off. That is, machines for which the current CPU schedule takes more than 250μs. This is especially interesting for virtual machines. If you are running on very slow machines, you may have to tune the reduction counts and threshold sizes to get good latency on the system. + +# Testing + +Every primitive has been stress-tested through the use of Erlang QuickCheck with both *positive* and *negative* testing. This has been used to check against memory leaks as well as correct invocation. Please report any error so we can extend the test cases to include a randomized test which captures the problem so we generically catch every problem in a given class of errors. + +Positive and negative testing refers to Type I and Type II errors in statistical testing. This means false positives—given a *valid* input the function rejects it; as well as false negatives—given an *invalid* input the functions fails to reject that input. + +The problem however, is that while we are testing the API level, we can't really test the strength of the cryptographic primitives. We can verify their correctness by trying different standard correctness tests for the primitives, verifying that the output matches the expected one given a specific input. But there is no way we can show that the cryptographic primitive has the strength we want. Thus, we opted to mostly test the API and its invocation for stability. + +Also, in addition to correctness, testing the system like this makes sure we have no memory leaks as they will show themselves under the extensive QuickCheck test cases we run. It has been verified there are no leaks in the code. +