احسب ان هذه الفرصة هي خير و نعمة من الله فالحمدلله الذي رزقني هذا فما توفيقي الا به و ما توكلي الا عليه
👋 Hello and welcome to my project for Google Summer of Code (GSoC)! I'm excited to share with you the journey of building contract testing capabilities for Keploy, a powerful open-source tool designed for automated end-to-end test generation, mocking, and test management.
Through this project, I worked on enhancing Keploy by implementing contract testing features that automate the validation of interactions between different services. This ensures seamless communication through APIs, reducing the risk of integration errors and service downtimes.
In the sections below, I will walk you through the features of the project, how the schema matching and scoring systems work, and the value this project adds to the software development process. I'm also excited to share how this experience has helped me grow technically and professionally during GSoC.
- My Proposal
- Key Features
- Generate Contracts
- Download Contracts
- Validate Contracts
- Support for Multiple HTTP Methods
- OpenAPI Integration
- Mock Generation for Testing
- Matching and Scoring
- Sample
- Code
- Value Added
You can find my proposal in this repo or ask me if you have any questions.
The generate command allows users to create contracts for specified services. It auto-generates contracts based on the HTTP methods (like GET, POST, etc.), request body, response status, and body. This checks if the services behave according to expectations in terms of their input/output interactions.
Command Example:
keploy contract generateWhen executed, this command will:
- Capture HTTP requests and responses
- Generate contracts in OpenAPI formats
- Save these contracts for future use (validation, mocking, etc.)
The download command fetches contracts for specific services from either remote repositories or local configurations. This helps maintain consistency across different environments (development, testing, production).
Command Example:
keploy contract download --driven "consumer" --path /local/pathWhen executed, this command will:
- Download contracts from the specified path or repository
- Ensure consistent versioning across testing and production environments
- Make these contracts available for validation or mock generation
The validate command checks whether the service's behavior matches the generated contract. It ensures no unexpected changes occur in the service's communication patterns, reducing API-breaking changes.
Command Example:
keploy contract validate --driven "consumer" --path /local/pathWhen executed, this command will:
- Compare the current service behavior against the contract
- Log and report any discrepancies
- Help identify if any API-breaking changes have occurred
This project supports multiple HTTP methods, including GET, POST, PUT, DELETE, and PATCH. It automatically maps request parameters such as headers, query parameters, and path variables, validating the structure of the response.
The system integrates with OpenAPI for contract generation and validation. HTTP requests and responses are converted into OpenAPI documentation, ensuring standardized contract storage and validation in OpenAPI format.
The mock feature generates mock data based on service contracts. This allows testing services in isolation without needing real service dependencies, improving the testing process.
The matching process compares mock and test services' request and response bodies, headers, parameters, and status codes. The goal is to ensure that both mock and test operations behave similarly, adhering to the expected contract.
The matching is carried out by:
- Comparing the operation types (GET, POST, etc.) to ensure they match
- Comparing the request bodies and validating JSON structures using schema comparison tools
- Comparing parameters such as headers, query parameters, and path variables to ensure they align
- Comparing response bodies, status codes, and content
- Handling discrepancies using diff tools to highlight differences in the request/response payloads
The scoring system evaluates how closely the test service adheres to the contract. It assigns a score based on the similarity of the expected and actual outputs. Here's how the scoring is calculated:
- Operation Matching: If the mock and test operations (GET, POST, etc.) match, a higher score is assigned.
- Request Body Matching: The system compares the JSON structures of the request body. If the structures are identical, a higher score is given. The system uses JSON diff tools to calculate the differences between the expected and actual JSON.
- Response Body Matching: The similarity between the response bodies (in terms of status code, body content, and headers) also impacts the score. A close match will result in a higher score, while differences will reduce the score.
- Parameter Matching: Parameters such as headers, query parameters, and path variables are compared. Matching parameters contribute to a higher score.
The score is normalized and expressed as a percentage. A score close to 100% indicates that the test service adheres closely to the contract, while lower scores indicate discrepancies.
For instance:
- A match in operation types contributes a fixed percentage (e.g., 30%).
- A match in request body structure adds another percentage (e.g., 40%).
- A match in response body structure adds the remaining percentage (e.g., 30%).
In case of mismatches or errors during comparison, the system logs detailed reports and differences between the expected and actual values. This helps developers understand where the mismatches occurred and take appropriate action.
You can run a sample from this repo
You can take a look here
This project automates contract validation, reducing the manual effort required in integration testing. It automatically checks service compatibility against existing contracts.
By ensuring services do not break integration points, contract testing reduces integration issues in microservice architectures.
The project highlights and prevents changes that violate the service contract, ensuring API stability and backward compatibility.
By validating contracts early, teams can resolve integration issues before they reach production, leading to faster and more reliable development cycles.
Mock data generation allows developers to simulate service interactions, enabling isolated testing without dependencies on real services.
By using OpenAPI standards, this system ensures interoperability with various tools and services, automating the generation of OpenAPI documentation from service behaviors.
The system ensures that contract testing can be consistently applied across different environments (development, testing, production).
Detailed reports are generated during validation, highlighting passed, failed, and missed test cases for easy debugging and triaging.
I would like to extend my deepest gratitude to my amazing mentors who guided me throughout this journey:
- Gourav - Your wisdom and feedback were instrumental in helping me grow both technically and professionally.
- Shivam - Thank you for your constant support and for always being available to answer my questions.
I truly appreciate all the time and effort each of you put into making this project a success. Your mentorship has been invaluable, and I am forever grateful!