Add docs for guiding the developers on finding, diagnosing, and fixing scenarios in Autoware
Closed this issue · 3 comments
Checklist
- I've read the contribution guidelines.
- I've searched other issues and no duplicate issues were found.
- I've agreed with the maintainers that I can plan this task.
Description
This issue aims to provide comprehensive guidance for developers interested in contributing to Autoware by identifying, troubleshooting, and resolving issues within various test scenarios.
It will serve as a resource for planning developers looking for jobs, new-comers evaluating Autoware's test success, and contributors aiming to fix broken scenarios without causing regressions.
Purpose
The purpose of this documentation is to streamline the process for developers to engage with Autoware's testing framework effectively.
It will enable contributors to easily find scenarios, understand their results, reproduce issues, and confidently submit fixes ensuring no disruption to other parts of the software.
Target Audience
- Planning developers seeking opportunities in new developments but unsure where to find relevant jobs.
- New-comers interested in understanding the success rates and test situations in Autoware.
- Developers who encounter a broken scenario and want to fix it but don't know where to start.
Possible approaches
- Scenario Discovery: Where are the scenarios located, how do I find and see them?
- Success Evaluation: How do I know which scenarios are failed, which has passed?
- Failure Analysis: Why did the scenario fail? What metric was not achieved?
- Environment Setup: How do I reproduce the failing scenarios on my own machine? Including all commit-sha's from scenario.
- Regression Testing: If I fix a scenario, how do I ensure it doesn't break the rest? How to test all scenarios on my own machine and make sure things didn't get worse?
Definition of done
- Comprehensive guidelines on where and how to find scenarios are documented.
- Criteria for evaluating the success of a scenario are clearly defined.
- Instructions on identifying and analyzing the reasons for scenario failures are provided.
- Steps to reproduce the exact scenario environment are detailed.
- Procedures for running regression tests to ensure no disruptions are described.
I've updated the Possible approaches section.
This pull request has been automatically marked as stale because it has not had recent activity.
The PR has been merged, so I am closing this issue.