Improve README and action.yml

dorny · Jan 24, 2021 · 92cb68e · 92cb68e
1 parent 8fdf2ae
commit 92cb68e
Show file tree

Hide file tree

Showing 7 changed files with 299 additions and 80 deletions.
diff --git a/LICENSE b/LICENSE
@@ -1,7 +1,7 @@
 
 The MIT License (MIT)
 
-Copyright (c) 2020 Michal Dorner and contributors
+Copyright (c) 2021 Michal Dorner and contributors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
@@ -19,4 +19,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-THE SOFTWARE.
+THE SOFTWARE.
diff --git a/README.md b/README.md
@@ -1,12 +1,180 @@
-**WIP**
+# Test Reporter
 
-# Test Check Report
+This [Github Action](https://github.com/features/actions) displays test results from popular testing frameworks directly in GitHub.
+- [x] Parses test results in XML or JSON format and creates nice report (Github Check Run)
+- [x] Annotates code where it failed based on message and stack trace captured during test execution
+- [x] Sets output variable conclusion to `success` if all tests passed or `failure` if any test failed
 
-Goal of this project is to create [Github Action](https://github.com/features/actions)
-that could presents test results from popular testing frameworks as Github check run with code annotation in places where test failed.
+**Supported languages / frameworks:**
+- .NET / [xUnit](https://xunit.net/) / [NUnit](https://nunit.org/) / [MSTest](https://github.com/Microsoft/testfx-docs)
+- Dart / [test](https://pub.dev/packages/test)
+- Flutter / [test](https://pub.dev/packages/test)
+- JavaScript / [JEST](https://jestjs.io/)
 
-Support for following test reports are planned for initial release:
-- [ ] dart-json: [`dart test --file-reporter=\"json:test-results.json`](https://pub.dev/packages/test)
-- [ ] dotnet-trx: [`dotnet test --logger "trx;LogFileName=test-results.trx"`](https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test#examples)
-- [ ] flutter-machine: [`flutter test --machine > test-results.json`](https://flutter.dev/docs/cookbook/testing/unit/introduction)
-- [ ] jest-junit: [`jest --ci --reporters=jest-junit`](https://github.com/jest-community/jest-junit#readme)
+For more information see [Supported formats](#supported-formats) section.
+
+**Support is planned for:**
+- Java / [JUnit 5](https://junit.org/junit5/)
+
+Do you miss support for your favorite language or framework?
+Please create [Issue](https://github.com/dorny/test-reporter/issues/new) or contribute with PR.
+
+## Example
+
+```yaml
+jobs:
+  build-test:
+    name: 'Build & Test'
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2          # checkout the repo
+      - run: npm ci                        # install packages
+      - run: npm test                      # run tests (configured to use jest-junit reporter)
+
+      - name: 'Test Report'
+        uses: dorny/test-reporter@v1
+        if: always()                       # run this step even if previous step failed
+        with:
+          name: 'JEST Tests'               # Name of the check run which will be created
+          path: '__reports__/jest-*.xml'   # Path to test report
+          reporter: 'jest-junit'           # Format of test report
+```
+
+## Usage
+
+```yaml
+- uses: dorny/test-reporter@v1
+  with:
+
+    # Name of the Check Run which will be created
+    name: ''
+
+    # Path to test report
+    # Supports wildcards via [fast-glob](https://github.com/mrmlnc/fast-glob)
+    # Path may match multiple result files of same format
+    path: ''
+
+    # Format of test report. Supported options:
+    #   dart-json
+    #   dotnet-trx
+    #   flutter-machine
+    #   jest-junit
+    reporter: ''
+
+    # Enables code annotations with error message and stack trace
+    annotations: 'true'
+
+    # Set action as failed if test report contain any failed test
+    fail-on-error: 'true'
+
+    # Relative path under $GITHUB_WORKSPACE where the repository was checked out.
+    working-directory: ''
+
+    # Personal access token used to interact with Github API
+    # Default: ${{ github.token }}
+    token: ''
+```
+
+## Supported formats
+
+<details>
+  <summary>dart-json</summary>
+
+Test run must be configured to use [JSON](https://github.com/dart-lang/test/blob/master/pkgs/test/doc/configuration.md#reporter) reporter.
+You can configure it in `dart_test.yaml`:
+
+```yml
+file_reporters:
+  json: reports/test-results.json
+```
+
+Or with CLI arguments:
+
+[`dart test --file-reporter="json:test-results.json"`](https://pub.dev/packages/test)
+
+For more information see:
+- [test package](https://pub.dev/packages/test)
+- [test configuration](https://github.com/dart-lang/test/blob/master/pkgs/test/doc/configuration.md)
+</details>
+
+<details>
+  <summary>dotnet-trx</summary>
+
+Test execution must be configured to produce *Visual Studio Test Results* files (TRX).
+To get test results in TRX format you can execute your tests with CLI arguments:
+
+`dotnet test --logger "trx;LogFileName=test-results.trx"`
+
+Or you can configure TRX test output in `*.csproj` or `Directory.Build.props`:
+```xml
+<PropertyGroup>
+  <VSTestLogger>trx%3bLogFileName=$(MSBuildProjectName).trx</VSTestLogger>
+  <VSTestResultsDirectory>$(MSBuildThisFileDirectory)/reports</VSTestResultsDirectory>
+</PropertyGroup>
+```
+
+Supported testing frameworks:
+- [xUnit](https://xunit.net/)
+- [NUnit](https://nunit.org/)
+- [MSTest](https://github.com/Microsoft/testfx-docs)
+
+For more information see [dotnet test](https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test#examples)
+</details>
+
+<details>
+  <summary>flutter-machine</summary>
+
+Test run must be configured to use [JSON](https://github.com/dart-lang/test/blob/master/pkgs/test/doc/configuration.md#reporter) reporter.
+You can configure it in `dart_test.yaml`:
+```yml
+file_reporters:
+  json: reports/test-results.json
+```
+
+Or with (undocumented) CLI argument:
+
+`flutter test --machine > test-results.json`
+
+
+According to documentation `dart_test.yaml` should be at the root of the package, next to the package's pubspec.
+It works in dart projects but the file is ignored by flutter. With flutter it works when `dart_test.yaml` is placed inside your `test` folder.
+
+For more information see:
+- [test package](https://pub.dev/packages/test)
+- [test configuration](https://github.com/dart-lang/test/blob/master/pkgs/test/doc/configuration.md)
+- [flutter-cli](https://flutter.dev/docs/reference/flutter-cli)
+- [unit testing introduction](https://flutter.dev/docs/cookbook/testing/unit/introduction)
+
+</details>
+
+<details>
+  <summary>jest-junit</summary>
+
+[JEST](https://jestjs.io/) testing framework support requires usage of [jest-junit](https://github.com/jest-community/jest-junit) reporter.
+It will create test results in junit XML format which can be then processed by this action.
+You can use following example configuration in `package.json`:
+```json
+"scripts": {
+  "test": "jest --ci --reporters=default --reporters=jest-junit"
+},
+"devDependencies": {
+  "jest": "^26.5.3",
+  "jest-junit": "^12.0.0"
+},
+"jest-junit": {
+  "outputDirectory": "__reports__",
+  "outputName": "jest-junit.xml",
+  "ancestorSeparator": " › ",
+  "uniqueOutputName": "false",
+  "suiteNameTemplate": "{filepath}",
+  "classNameTemplate": "{classname}",
+  "titleTemplate": "{title}"
+}
+```
+
+Configuration of `uniqueOutputName`, `suiteNameTemplate`, `classNameTemplate`, `titleTemplate` is important for proper visualization of test results.
+</details>
+
+## License
+
+The scripts and documentation in this project are released under the [MIT License](https://github.com/dorny/test-reporter/blob/master/LICENSE)
diff --git a/action.yml b/action.yml
@@ -1,26 +1,22 @@
-name: 'Test Check Reporter'
+name: Test Reporter
 description: |
-  Presents test results from popular testing frameworks as Github check run.
-  Supports:
-    - dotnet (trx logger)
-    - flutter (JSON output with '--machine' flag )
-    - jest (jest-junit reporter)
-
-author: 'Michal Dorner <dorner.michal@gmail.com>'
+  Displays test results from popular testing frameworks directly in GitHub.
+  Supports .NET (xUnit, NUnit, MSTest), Dart, Flutter and JavaScript (JEST).
+author: Michal Dorner <dorner.michal@gmail.com>
 inputs:
   annotations:
-    description: 'Annotate code where exceptions in tests were thrown'
+    description: Enables code annotations with error message and stack trace captured during test execution
     required: true
     default: 'true'
   fail-on-error:
-    description: 'Set this action as failed if test report contains any failed test'
+    description: Set this action as failed if test report contain any failed test
     required: true
     default: 'true'
   name:
-    description: 'Name of the check run'
+    description: Name of the check run
     required: true
   path:
-    description: 'Path to test report'
+    description: Path to test report
     required: true
   reporter:
     description: |
@@ -31,11 +27,11 @@ inputs:
         - jest-junit
     required: true
   token:
-    description: 'GitHub Access Token'
+    description: GitHub Access Token
     required: false
     default: ${{ github.token }}
   working-directory:
-    description: 'Relative path under $GITHUB_WORKSPACE where the repository was checked out.'
+    description: Relative path under $GITHUB_WORKSPACE where the repository was checked out
     required: false
 outputs:
   conclusion: