-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #46 from dorny/improve-readme
Improve README and action.yml
- Loading branch information
Showing
14 changed files
with
364 additions
and
85 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,12 +1,183 @@ | ||
| **WIP** | ||
| # Test Reporter | ||
|
|
||
| # Test Check Report | ||
| This [Github Action](https://github.com/features/actions) displays test results from popular testing frameworks directly in GitHub. | ||
| - [x] Parses test results in XML or JSON format and creates nice report (Github Check Run) | ||
| - [x] Annotates code where it failed based on message and stack trace captured during test execution | ||
| - [x] Sets output variable conclusion to `success` if all tests passed or `failure` if any test failed | ||
|
|
||
| Goal of this project is to create [Github Action](https://github.com/features/actions) | ||
| that could presents test results from popular testing frameworks as Github check run with code annotation in places where test failed. | ||
| **Supported languages / frameworks:** | ||
| - .NET / [xUnit](https://xunit.net/) / [NUnit](https://nunit.org/) / [MSTest](https://github.com/Microsoft/testfx-docs) | ||
| - Dart / [test](https://pub.dev/packages/test) | ||
| - Flutter / [test](https://pub.dev/packages/test) | ||
| - JavaScript / [JEST](https://jestjs.io/) | ||
|
|
||
| Support for following test reports are planned for initial release: | ||
| - [ ] dart-json: [`dart test --file-reporter=\"json:test-results.json`](https://pub.dev/packages/test) | ||
| - [ ] dotnet-trx: [`dotnet test --logger "trx;LogFileName=test-results.trx"`](https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test#examples) | ||
| - [ ] flutter-machine: [`flutter test --machine > test-results.json`](https://flutter.dev/docs/cookbook/testing/unit/introduction) | ||
| - [ ] jest-junit: [`jest --ci --reporters=jest-junit`](https://github.com/jest-community/jest-junit#readme) | ||
| For more information see [Supported formats](#supported-formats) section. | ||
|
|
||
| **Support is planned for:** | ||
| - Java / [JUnit 5](https://junit.org/junit5/) | ||
|
|
||
| Do you miss support for your favorite language or framework? | ||
| Please create [Issue](https://github.com/dorny/test-reporter/issues/new) or contribute with PR. | ||
|
|
||
| ## Example | ||
|
|
||
| ```yaml | ||
| jobs: | ||
| build-test: | ||
| name: 'Build & Test' | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v2 # checkout the repo | ||
| - run: npm ci # install packages | ||
| - run: npm test # run tests (configured to use jest-junit reporter) | ||
|
|
||
| - name: 'Test Report' | ||
| uses: dorny/test-reporter@v1 | ||
| if: always() # run this step even if previous step failed | ||
| with: | ||
| name: 'JEST Tests' # Name of the check run which will be created | ||
| path: 'reports/jest-*.xml' # Path to test report | ||
| reporter: 'jest-junit' # Format of test report | ||
| ``` | ||
| ## Usage | ||
| ```yaml | ||
| - uses: dorny/test-reporter@v1 | ||
| with: | ||
|
|
||
| # Name of the Check Run which will be created | ||
| name: '' | ||
|
|
||
| # Path to test report | ||
| # Supports wildcards via [fast-glob](https://github.com/mrmlnc/fast-glob) | ||
| # Path may match multiple result files of same format | ||
| path: '' | ||
|
|
||
| # Format of test report. Supported options: | ||
| # dart-json | ||
| # dotnet-trx | ||
| # flutter-machine | ||
| # jest-junit | ||
| reporter: '' | ||
|
|
||
| # Enables code annotations with error message and stack trace captured during test execution | ||
| annotations: 'true' | ||
|
|
||
| # Set action as failed if test report contain any failed test | ||
| fail-on-error: 'true' | ||
|
|
||
| # Relative path under $GITHUB_WORKSPACE where the repository was checked out. | ||
| working-directory: '' | ||
|
|
||
| # Personal access token used to interact with Github API | ||
| # Default: ${{ github.token }} | ||
| token: '' | ||
| ``` | ||
| ## Supported formats | ||
| <details> | ||
| <summary>dart-json</summary> | ||
| Test run must be configured to use [JSON](https://github.com/dart-lang/test/blob/master/pkgs/test/doc/configuration.md#reporter) reporter. | ||
| You can configure it in `dart_test.yaml`: | ||
|
|
||
| ```yml | ||
| file_reporters: | ||
| json: reports/test-results.json | ||
| ``` | ||
|
|
||
| Or with CLI arguments: | ||
|
|
||
| [`dart test --file-reporter="json:test-results.json"`](https://pub.dev/packages/test) | ||
|
|
||
| For more information see: | ||
| - [test package](https://pub.dev/packages/test) | ||
| - [test configuration](https://github.com/dart-lang/test/blob/master/pkgs/test/doc/configuration.md) | ||
| </details> | ||
|
|
||
| <details> | ||
| <summary>dotnet-trx</summary> | ||
|
|
||
| Test execution must be configured to produce *Visual Studio Test Results* files (TRX). | ||
| To get test results in TRX format you can execute your tests with CLI arguments: | ||
|
|
||
| `dotnet test --logger "trx;LogFileName=test-results.trx"` | ||
|
|
||
| Or you can configure TRX test output in `*.csproj` or `Directory.Build.props`: | ||
| ```xml | ||
| <PropertyGroup> | ||
| <VSTestLogger>trx%3bLogFileName=$(MSBuildProjectName).trx</VSTestLogger> | ||
| <VSTestResultsDirectory>$(MSBuildThisFileDirectory)/reports</VSTestResultsDirectory> | ||
| </PropertyGroup> | ||
| ``` | ||
|
|
||
| Supported testing frameworks: | ||
| - [xUnit](https://xunit.net/) | ||
| - [NUnit](https://nunit.org/) | ||
| - [MSTest](https://github.com/Microsoft/testfx-docs) | ||
|
|
||
| For more information see [dotnet test](https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test#examples) | ||
| </details> | ||
|
|
||
| <details> | ||
| <summary>flutter-machine</summary> | ||
|
|
||
| Test run must be configured to use [JSON](https://github.com/dart-lang/test/blob/master/pkgs/test/doc/configuration.md#reporter) reporter. | ||
| You can configure it in `dart_test.yaml`: | ||
| ```yml | ||
| file_reporters: | ||
| json: reports/test-results.json | ||
| ``` | ||
|
|
||
| Or with (undocumented) CLI argument: | ||
|
|
||
| `flutter test --machine > test-results.json` | ||
|
|
||
|
|
||
| According to documentation `dart_test.yaml` should be at the root of the package, next to the package's pubspec. | ||
| It works in dart projects but the file is ignored by flutter. With flutter it works when `dart_test.yaml` is placed inside your `test` folder. | ||
|
|
||
| For more information see: | ||
| - [test package](https://pub.dev/packages/test) | ||
| - [test configuration](https://github.com/dart-lang/test/blob/master/pkgs/test/doc/configuration.md) | ||
| - [flutter-cli](https://flutter.dev/docs/reference/flutter-cli) | ||
| - [unit testing introduction](https://flutter.dev/docs/cookbook/testing/unit/introduction) | ||
|
|
||
| </details> | ||
|
|
||
| <details> | ||
| <summary>jest-junit</summary> | ||
|
|
||
| [JEST](https://jestjs.io/) testing framework support requires usage of [jest-junit](https://github.com/jest-community/jest-junit) reporter. | ||
| It will create test results in junit XML format which can be then processed by this action. | ||
| You can use following example configuration in `package.json`: | ||
| ```json | ||
| "scripts": { | ||
| "test": "jest --ci --reporters=default --reporters=jest-junit" | ||
| }, | ||
| "devDependencies": { | ||
| "jest": "^26.5.3", | ||
| "jest-junit": "^12.0.0" | ||
| }, | ||
| "jest-junit": { | ||
| "outputDirectory": "reports", | ||
| "outputName": "jest-junit.xml", | ||
| "ancestorSeparator": " › ", | ||
| "uniqueOutputName": "false", | ||
| "suiteNameTemplate": "{filepath}", | ||
| "classNameTemplate": "{classname}", | ||
| "titleTemplate": "{title}" | ||
| } | ||
| ``` | ||
|
|
||
| Configuration of `uniqueOutputName`, `suiteNameTemplate`, `classNameTemplate`, `titleTemplate` is important for proper visualization of test results. | ||
| </details> | ||
|
|
||
| ## See also | ||
| - [paths-filter](https://github.com/dorny/paths-filter) - Conditionally run actions based on files modified by PR, feature branch or pushed commits | ||
|
|
||
| ## License | ||
|
|
||
| The scripts and documentation in this project are released under the [MIT License](https://github.com/dorny/test-reporter/blob/master/LICENSE) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.