From c4d8801495b5483040ea3a7a7b7725baaca77cc0 Mon Sep 17 00:00:00 2001 From: chriseth Date: Tue, 25 Sep 2018 19:20:59 +0200 Subject: [DOCS] Update contributing. --- docs/contributing.rst | 46 ++++++++++++++++++++++++---------------------- 1 file changed, 24 insertions(+), 22 deletions(-) (limited to 'docs') diff --git a/docs/contributing.rst b/docs/contributing.rst index d1ed9c6b..88924fae 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -47,13 +47,14 @@ in addition to *what* you did (unless it is a tiny change). If you need to pull in any changes from ``develop`` after making your fork (for example, to resolve potential merge conflicts), please avoid using ``git merge`` -and instead, ``git rebase`` your branch. +and instead, ``git rebase`` your branch. This will help us review your change +more easily. -Additionally, if you are writing a new feature, please ensure you write appropriate -Boost test cases and place them under ``test/``. +Additionally, if you are writing a new feature, please ensure you add appropriate +test cases under ``test/`` (see below). However, if you are making a larger change, please consult with the `Solidity Development Gitter channel -`_ (different from the one mentioned above, this on is +`_ (different from the one mentioned above, this one is focused on compiler and language development instead of language use) first. New features and bugfixes should be added to the ``Changelog.md`` file: please @@ -69,8 +70,8 @@ Thank you for your help! Running the compiler tests ========================== -Solidity includes different types of tests. They are included in the application -called ``soltest``. Some of them require the ``cpp-ethereum`` client in testing mode, +Solidity includes different types of tests. Most of them are bundled in the application +called ``soltest``. Some of them require the ``aleth`` client in testing mode, some others require ``libz3`` to be installed. ``soltest`` reads test contracts that are annotated with expected results @@ -79,10 +80,10 @@ tests the root test directory has to be specified using the ``--testpath`` comma line option, e.g. ``./build/test/soltest -- --testpath ./test``. To disable the z3 tests, use ``./build/test/soltest -- --no-smt --testpath ./test`` and -to run a subset of the tests that do not require ``cpp-ethereum``, use +to run a subset of the tests that do not require ``aleth``, use ``./build/test/soltest -- --no-ipc --testpath ./test``. -For all other tests, you need to install `cpp-ethereum `_ and run it in testing mode: ``eth --test -d /tmp/testeth``. +For all other tests, you need to install `aleth `_ and run it in testing mode: ``aleth --test -d /tmp/testeth``. Then you run the actual tests: ``./build/test/soltest -- --ipcpath /tmp/testeth/geth.ipc --testpath ./test``. @@ -91,19 +92,19 @@ To run a subset of tests, filters can be used: where ``TestName`` can be a wildcard ``*``. Alternatively, there is a testing script at ``scripts/tests.sh`` which executes all tests and runs -``cpp-ethereum`` automatically if it is in the path (but does not download it). +``aleth`` automatically if it is in the path (but does not download it). -Travis CI even runs some additional tests (including ``solc-js`` and testing third party Solidity frameworks) that require compiling the Emscripten target. +The CI even runs some additional tests (including ``solc-js`` and testing third party Solidity frameworks) that require compiling the Emscripten target. .. note :: - While any version of ``cpp-ethereum`` should be usable, this cannot be guaranteed, and it is suggested to use the same version that is used by the Solidity continuous integration tests. - Currently the CI uses ``d661ac4fec0aeffbedcdc195f67f5ded0c798278`` of ``cpp-ethereum``. + Some versions of ``aleth`` cannot be used for testing. We suggest using the same version that is used by the Solidity continuous integration tests. + Currently the CI uses ``d661ac4fec0aeffbedcdc195f67f5ded0c798278`` of ``aleth``. Writing and running syntax tests -------------------------------- -As mentioned above, syntax tests are stored in individual contracts. These files must contain annotations, stating the expected result(s) of the respective test. +As mentioned above, syntax tests are stored in individual files. These files must contain annotations, stating the expected result(s) of the respective test. The test suite will compile and check them against the given expectations. Example: ``./test/libsolidity/syntaxTests/double_stateVariable_declaration.sol`` @@ -115,10 +116,10 @@ Example: ``./test/libsolidity/syntaxTests/double_stateVariable_declaration.sol`` uint128 variable; } // ---- - // DeclarationError: Identifier already declared. + // DeclarationError: (36-52): Identifier already declared. -A syntax test must contain at least the contract under test itself, followed by the separator ``----``. The additional comments above are used to describe the -expected compiler errors or warnings. This section can be empty in case that the contract should compile without any errors or warnings. +A syntax test must contain at least the contract under test itself, followed by the separator ``// ----``. The following comments are used to describe the +expected compiler errors or warnings. This section can be empty in case the contract should compile without any errors or warnings. In the above example, the state variable ``variable`` was declared twice, which is not allowed. This will result in a ``DeclarationError`` stating that the identifier was already declared. @@ -131,7 +132,7 @@ editing of failing contracts using your preferred text editor. Let's try to brea uint256 variable; } // ---- - // DeclarationError: Identifier already declared. + // DeclarationError: (36-52): Identifier already declared. Running ``./test/isoltest`` again will result in a test failure: @@ -144,13 +145,13 @@ Running ``./test/isoltest`` again will result in a test failure: } Expected result: - DeclarationError: Identifier already declared. + DeclarationError: (36-52): Identifier already declared. Obtained result: Success -which prints the expected result next to the obtained result, but also provides a way to change edit / update / skip the current contract or to even quit. -``isoltest`` offers several options for failing tests: +``isoltest`` prints the expected result next to the obtained result, but also provides a way to change edit / update / skip the current contract or to even quit. +It offers several options for failing tests: - edit: ``isoltest`` will try to open the editor that was specified before using ``isoltest --editor /path/to/editor``. If no path was set, this will result in a runtime error. In case an editor was specified, this will open it such that the contract can be adjusted. - update: Updates the contract under test. This will either remove the annotation which contains the exception not met or will add missing expectations. The test will then be run again. @@ -177,7 +178,8 @@ and re-run the test. It will now pass again: .. note:: Please choose a name for the contract file, that is self-explainatory in the sense of what is been tested, e.g. ``double_variable_declaration.sol``. - Do not put more than one contract into a single file. ``isoltest`` is currently not able to recognize them individually. + Do not put more than one contract into a single file, unless you are testing inheritance or cross-contract calls. + Each file should test one aspect of your new feature. Running the Fuzzer via AFL @@ -276,7 +278,7 @@ use the tool ``scripts/uniqueErrors.sh`` to filter out the unique errors. Whiskers ======== -*Whiskers* is a templating system similar to `Mustache `_. It is used by the +*Whiskers* is a string templating system similar to `Mustache `_. It is used by the compiler in various places to aid readability, and thus maintainability and verifiability, of the code. The syntax comes with a substantial difference to Mustache: the template markers ``{{`` and ``}}`` are -- cgit From 998de848362ac1a3d664f25c0e837ea05bb370d6 Mon Sep 17 00:00:00 2001 From: chriseth Date: Thu, 27 Sep 2018 15:24:45 +0200 Subject: fixup! [DOCS] Update contributing. --- docs/contributing.rst | 38 ++++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 16 deletions(-) (limited to 'docs') diff --git a/docs/contributing.rst b/docs/contributing.rst index 88924fae..c065893b 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -70,29 +70,32 @@ Thank you for your help! Running the compiler tests ========================== +There is a script at ``scripts/tests.sh`` which executes most of the tests and +runs ``aleth`` automatically if it is in the path, but does not download it, +so it most likely will not work right away. Please read on for the details. + Solidity includes different types of tests. Most of them are bundled in the application called ``soltest``. Some of them require the ``aleth`` client in testing mode, some others require ``libz3`` to be installed. -``soltest`` reads test contracts that are annotated with expected results -stored in ``./test/libsolidity/syntaxTests``. In order for soltest to find these -tests the root test directory has to be specified using the ``--testpath`` command -line option, e.g. ``./build/test/soltest -- --testpath ./test``. +To run a basic set of tests that neither require ``aleth`` nor ``libz3``, run +``./scripts/soltest.sh --no-ipc --no-smt``. This script will run ``build/test/soltest`` +internally. -To disable the z3 tests, use ``./build/test/soltest -- --no-smt --testpath ./test`` and -to run a subset of the tests that do not require ``aleth``, use -``./build/test/soltest -- --no-ipc --testpath ./test``. +The option ``--no-smt`` disables the tests that require ``libz3`` and +``--no-ipc`` disables those that requir ``aleth``. -For all other tests, you need to install `aleth `_ and run it in testing mode: ``aleth --test -d /tmp/testeth``. +If you want to run the ipc tests (those test the semantics of the generated code), +you need to install `aleth `_ and run it in testing mode: ``aleth --test -d /tmp/testeth``. -Then you run the actual tests: ``./build/test/soltest -- --ipcpath /tmp/testeth/geth.ipc --testpath ./test``. +Then you run the actual tests: ``./scripts/soltest.sh --ipcpath /tmp/testeth/geth.ipc``. To run a subset of tests, filters can be used: -``soltest -t TestSuite/TestName -- --ipcpath /tmp/testeth/geth.ipc --testpath ./test``, +``./scripts/soltest.sh -t TestSuite/TestName --ipcpath /tmp/testeth/geth.ipc``, where ``TestName`` can be a wildcard ``*``. -Alternatively, there is a testing script at ``scripts/tests.sh`` which executes all tests and runs -``aleth`` automatically if it is in the path (but does not download it). +The script ``scripts/tests.sh`` also runs commandline tests and compilation tests +in addition to those found in ``soltest``. The CI even runs some additional tests (including ``solc-js`` and testing third party Solidity frameworks) that require compiling the Emscripten target. @@ -104,7 +107,9 @@ The CI even runs some additional tests (including ``solc-js`` and testing third Writing and running syntax tests -------------------------------- -As mentioned above, syntax tests are stored in individual files. These files must contain annotations, stating the expected result(s) of the respective test. +Syntax tests check that the compiler generates the correct error messages for invalid code +and properly accepts valid code. +They are stored in individual files. These files must contain annotations, stating the expected result(s) of the respective test. The test suite will compile and check them against the given expectations. Example: ``./test/libsolidity/syntaxTests/double_stateVariable_declaration.sol`` @@ -119,7 +124,8 @@ Example: ``./test/libsolidity/syntaxTests/double_stateVariable_declaration.sol`` // DeclarationError: (36-52): Identifier already declared. A syntax test must contain at least the contract under test itself, followed by the separator ``// ----``. The following comments are used to describe the -expected compiler errors or warnings. This section can be empty in case the contract should compile without any errors or warnings. +expected compiler errors or warnings. The number range denotes the location in the source where the error occurred. +The section after the separator can be empty in case the contract should compile without any errors or warnings. In the above example, the state variable ``variable`` was declared twice, which is not allowed. This will result in a ``DeclarationError`` stating that the identifier was already declared. @@ -153,7 +159,7 @@ Running ``./test/isoltest`` again will result in a test failure: ``isoltest`` prints the expected result next to the obtained result, but also provides a way to change edit / update / skip the current contract or to even quit. It offers several options for failing tests: -- edit: ``isoltest`` will try to open the editor that was specified before using ``isoltest --editor /path/to/editor``. If no path was set, this will result in a runtime error. In case an editor was specified, this will open it such that the contract can be adjusted. +- edit: ``isoltest`` will try to open the contract in an editor so you can adjust it. It will either use the editor given on the command line (as ``isoltest --editor /path/to/editor``), in the environment variable ``EDITOR`` or just ``/usr/bin/editor`` (in this order). - update: Updates the contract under test. This will either remove the annotation which contains the exception not met or will add missing expectations. The test will then be run again. - skip: Skips the execution of this particular test. - quit: Quits ``isoltest``. @@ -177,7 +183,7 @@ and re-run the test. It will now pass again: .. note:: - Please choose a name for the contract file, that is self-explainatory in the sense of what is been tested, e.g. ``double_variable_declaration.sol``. + Please choose a name for the contract file that explains what it tests, e.g. ``double_variable_declaration.sol``. Do not put more than one contract into a single file, unless you are testing inheritance or cross-contract calls. Each file should test one aspect of your new feature. -- cgit From 593d303ced57022747ce4debc94dd26b1f9b2b5c Mon Sep 17 00:00:00 2001 From: chriseth Date: Mon, 1 Oct 2018 12:54:04 +0200 Subject: fixup! fixup! [DOCS] Update contributing. --- docs/contributing.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) (limited to 'docs') diff --git a/docs/contributing.rst b/docs/contributing.rst index c065893b..95a0f69d 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -83,10 +83,10 @@ To run a basic set of tests that neither require ``aleth`` nor ``libz3``, run internally. The option ``--no-smt`` disables the tests that require ``libz3`` and -``--no-ipc`` disables those that requir ``aleth``. +``--no-ipc`` disables those that require ``aleth``. If you want to run the ipc tests (those test the semantics of the generated code), -you need to install `aleth `_ and run it in testing mode: ``aleth --test -d /tmp/testeth``. +you need to install `aleth `_ and run it in testing mode: ``aleth --test -d /tmp/testeth`` (make sure to rename it). Then you run the actual tests: ``./scripts/soltest.sh --ipcpath /tmp/testeth/geth.ipc``. @@ -109,7 +109,8 @@ Writing and running syntax tests Syntax tests check that the compiler generates the correct error messages for invalid code and properly accepts valid code. -They are stored in individual files. These files must contain annotations, stating the expected result(s) of the respective test. +They are stored in individual files inside ``tests/libsolidity/syntaxTests``. +These files must contain annotations, stating the expected result(s) of the respective test. The test suite will compile and check them against the given expectations. Example: ``./test/libsolidity/syntaxTests/double_stateVariable_declaration.sol`` -- cgit From b93c11f7a11d4da9c2a3968cc2d9407d538e8dc6 Mon Sep 17 00:00:00 2001 From: chriseth Date: Mon, 1 Oct 2018 13:34:36 +0200 Subject: fixup! fixup! fixup! [DOCS] Update contributing. --- docs/contributing.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) (limited to 'docs') diff --git a/docs/contributing.rst b/docs/contributing.rst index 95a0f69d..361570a0 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -126,7 +126,8 @@ Example: ``./test/libsolidity/syntaxTests/double_stateVariable_declaration.sol`` A syntax test must contain at least the contract under test itself, followed by the separator ``// ----``. The following comments are used to describe the expected compiler errors or warnings. The number range denotes the location in the source where the error occurred. -The section after the separator can be empty in case the contract should compile without any errors or warnings. +In case the contract should compile without any errors or warning, the section after the separator has to be empty +and the separator can be left out completely. In the above example, the state variable ``variable`` was declared twice, which is not allowed. This will result in a ``DeclarationError`` stating that the identifier was already declared. @@ -160,8 +161,8 @@ Running ``./test/isoltest`` again will result in a test failure: ``isoltest`` prints the expected result next to the obtained result, but also provides a way to change edit / update / skip the current contract or to even quit. It offers several options for failing tests: -- edit: ``isoltest`` will try to open the contract in an editor so you can adjust it. It will either use the editor given on the command line (as ``isoltest --editor /path/to/editor``), in the environment variable ``EDITOR`` or just ``/usr/bin/editor`` (in this order). -- update: Updates the contract under test. This will either remove the annotation which contains the exception not met or will add missing expectations. The test will then be run again. +- edit: ``isoltest`` tries to open the contract in an editor so you can adjust it. It either uses the editor given on the command line (as ``isoltest --editor /path/to/editor``), in the environment variable ``EDITOR`` or just ``/usr/bin/editor`` (in this order). +- update: Updates the contract under test. This either removes the annotation which contains the exception not met or adds missing expectations. The test will then be run again. - skip: Skips the execution of this particular test. - quit: Quits ``isoltest``. -- cgit