YiVal stands at the intersection of flexibility and adaptability in the AI development landscape. Crafted meticulously for those who seek a tailored experimentation experience, it effortlessly caters to both hands-on developers and those who lean into automation.
At its core, YiVal is more than just a framework—it's a philosophy. A philosophy that believes in offering tools that can be seamlessly molded to fit unique experimentation needs. With a robust architecture and versatile components, Yival ensures that every AI journey, regardless of its complexity, is smooth and efficient.
Web UI
Dive into the world of YiChing and discover your fortune on our index page. A fun and interactive way to get started with Yival.
- Experiment Result Analysis: Gain insights into aggregated outputs for each combination, evaluator outcomes, average latency, token usage, and sample test case results. The best combinations will be highlighted for ease of reference.
![Screenshot 2023-08-16 at 10 51 57 PM](https://private-user-images.githubusercontent.com/1544154/261210942-054e7659-ceb1-4048-af4e-301958b0b675.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjEyMzk1NTksIm5iZiI6MTcyMTIzOTI1OSwicGF0aCI6Ii8xNTQ0MTU0LzI2MTIxMDk0Mi0wNTRlNzY1OS1jZWIxLTQwNDgtYWY0ZS0zMDE5NThiMGI2NzUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDcxNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA3MTdUMTgwMDU5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MzMzMzU3MDFhYzc3MDIzYzRjODM0NGFlNWI3ZmRhMGU1MWQ4NWQ2OWQxMGZkZWU2NmJkMzQ4OGE5M2E4OTVjMCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.YpZJ_CQA50Lz0ot-0pXlA9harHmuyvAJtgxn4LmxGM4)
- Data Analysis Page: Delve deep into your experiment data, extracting meaningful insights and patterns that can guide further experimentation.
![Screenshot 2023-08-16 at 10 54 50 PM](https://private-user-images.githubusercontent.com/1544154/261211436-3440b51c-f607-477d-9092-94be94b4ebbe.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjEyMzk1NTksIm5iZiI6MTcyMTIzOTI1OSwicGF0aCI6Ii8xNTQ0MTU0LzI2MTIxMTQzNi0zNDQwYjUxYy1mNjA3LTQ3N2QtOTA5Mi05NGJlOTRiNGViYmUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDcxNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA3MTdUMTgwMDU5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YTEyMDNkZTMxMDMzYzVkNmNjMTI3MTcyZjU4NWEzMGQ3OWY2MmMxNjgxYzViNDIxODQ4NDkxY2FhMGI5MjZhZCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.IUiRid5ehEH7nWP4tnfwG7_233xT2uhdX1qFkiFjC5M)
- Detailed Test Results: A granular look at each test case result for every combination, providing a comprehensive understanding of the experiment's outcomes.
![Screenshot 2023-08-16 at 10 57 22 PM](https://private-user-images.githubusercontent.com/1544154/261211893-8f1f9e04-e94c-473e-b7f8-83e6ce0f16e8.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjEyMzk1NTksIm5iZiI6MTcyMTIzOTI1OSwicGF0aCI6Ii8xNTQ0MTU0LzI2MTIxMTg5My04ZjFmOWUwNC1lOTRjLTQ3M2UtYjdmOC04M2U2Y2UwZjE2ZTgucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDcxNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA3MTdUMTgwMDU5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9OTU5NzQwNzdkNWUxZWU1NjNmZmEwYWM1ZWMyOGRjM2ZjYWE5NGMyZGYwZGEzMjIyYjZmNThmM2RhYzBkZWRhYiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.OC3B5vkPemr0PV6ywLMYgGjRyLOrMEqvTqTV8xnJIuw)
- Improver Experiment Result Analysis: After the improvement phase, see the aggregated outputs for each combination. This includes evaluator outputs, average latency, token usage, and sample test cases, with the best combinations highlighted.
![Screenshot 2023-08-16 at 10 57 58 PM](https://private-user-images.githubusercontent.com/1544154/261211992-fd087b34-d3d4-48bb-800e-68cf09e47e5d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjEyMzk1NTksIm5iZiI6MTcyMTIzOTI1OSwicGF0aCI6Ii8xNTQ0MTU0LzI2MTIxMTk5Mi1mZDA4N2IzNC1kM2Q0LTQ4YmItODAwZS02OGNmMDllNDdlNWQucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDcxNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA3MTdUMTgwMDU5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YWYzZjdmYTkyOTdiZDhkN2RmMGM0ZDY2MDUxM2U4NDJjOGJmY2JmMjk2MjM2YmIzMTVlNDcwMWYxYTZiODE4MiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.XgF6sG38aM7FmgcYmJDZnVfGEedfd96JxFJIwsgPrS0)
- Improver Detailed Test Results: Post-improvement, this page offers a detailed view of each test case result for every combination, showcasing the enhancement in results.
![Screenshot 2023-08-16 at 10 58 18 PM](https://private-user-images.githubusercontent.com/1544154/261212051-3145de90-04b4-4cd6-8405-fae0ecb40545.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjEyMzk1NTksIm5iZiI6MTcyMTIzOTI1OSwicGF0aCI6Ii8xNTQ0MTU0LzI2MTIxMjA1MS0zMTQ1ZGU5MC0wNGI0LTRjZDYtODQwNS1mYWUwZWNiNDA1NDUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDcxNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA3MTdUMTgwMDU5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ZDU0MjllZGUxZTc0YmY4MWM5YzllZjdlMGMxNzJhMmQ5NTc3NTk3YjVlMGRhNWEyMTlmMmNkYWY5MzlhY2I0ZiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.EJpLZNzbxeuFch0i0EUuInLG9S-d0uFtttOdIsZyA3s)
- Export Data: Securely store and export your experiment for future reference or to share with peers.
![Screenshot 2023-08-16 at 10 59 01 PM](https://private-user-images.githubusercontent.com/1544154/261212196-2664fd03-0a3c-43ff-b065-8ea6cf440158.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjEyMzk1NTksIm5iZiI6MTcyMTIzOTI1OSwicGF0aCI6Ii8xNTQ0MTU0LzI2MTIxMjE5Ni0yNjY0ZmQwMy0wYTNjLTQzZmYtYjA2NS04ZWE2Y2Y0NDAxNTgucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDcxNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA3MTdUMTgwMDU5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ODc2ZGNmYTFhMzhmYzYyNjlmYTM5YTA3ZDdjZDVjNmFiZmMzNzcwYzhhM2Y3NDFhMTg3ZDIxYjA2ZGQ3ZjQ3NSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.DFlbwfS__V4Xs5m8nuRPg1-LFWgXCdGywj5WeYNgXn4)
- Rating: Human touch matters. Add ratings to each test case on the experiment results page based on configurable criteria.
![Screenshot 2023-08-16 at 10 59 18 PM](https://private-user-images.githubusercontent.com/1544154/261212248-87161a42-711a-4fc5-bb87-93e79d745554.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjEyMzk1NTksIm5iZiI6MTcyMTIzOTI1OSwicGF0aCI6Ii8xNTQ0MTU0LzI2MTIxMjI0OC04NzE2MWE0Mi03MTFhLTRmYzUtYmI4Ny05M2U3OWQ3NDU1NTQucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDcxNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA3MTdUMTgwMDU5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YTI1NGI3NDFiMWZkODZhZDY4OTllZjNkZjE1MTQxMDc0NmIwZWJlNjFjZTk1OGVkYWIyNmQ3N2RhMGEwMDFlNCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.qQuFIr8Obl5ofmQzCYU-6tEFccWVylK9-N7HThwNFis)
- Interactive Mode: Flexibility at its best. Enter new test cases for combinations, tailoring your experimentation in real-time.
![Screenshot 2023-08-16 at 11 02 37 PM](https://private-user-images.githubusercontent.com/1544154/261213118-f2ed3997-5f3c-4376-89a8-3ed3c5df0720.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjEyMzk1NTksIm5iZiI6MTcyMTIzOTI1OSwicGF0aCI6Ii8xNTQ0MTU0LzI2MTIxMzExOC1mMmVkMzk5Ny01ZjNjLTQzNzYtODlhOC0zZWQzYzVkZjA3MjAucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDcxNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA3MTdUMTgwMDU5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NWI1NjMwNTRhYmM0ZmJiNzdmZjY0OTVhYjRkODFkZjQ2YTU0Nzg2MzNjMzBmM2VhYTRkMGM4NGI3NTZjMjI2OCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.2iwItoz7zlJcnS0XsBFMJMxDnHBJuleAYiVi6LOWzek)
The flowchart below depicts the core flow of ideal AI Generated Content (AIGC) app development That Yival tries to support. This emphasis on iteration ensures a continuous cycle of improvement, allowing developers to hone their applications to perfection.
flowchart TD
A{Test Data}
B[Core App Development]
C[Evaluation]
D[Refinement]
A --> B
B --> C
C -->|Check| D
D -->|YiVal Supports Iteration| B
subgraph "Core App Development"
B1[Retrieve Data]
B2[Collaborate with Models]
B3[Select Best Model]
B4[Prompt Development]
end
subgraph Evaluation
C1[Human Evaluator]
C2[Auto Evaluator]
C --> C1
C --> C2
end
subgraph Refinement
D1[Human Improver]
D2[Auto Improver]
D --> D1
D --> D2
end
pip install yival
To get started with a demo for basic interactive mode of YiVal, run the following command:
yival demo --basic_interactive
Once started, navigate to the following address in your web browser:
http://127.0.0.1:8073/interactive
For more details on this demo, check out the Basic Interactive Mode Demo.
yival demo --qa_expected_results
Once started, navigate to the following address in your web browser: http://127.0.0.1:8073/
For more details on this demo, check out the Question Answering with expected result evaluator.