py-Vtest

An OI test-data maker & std test tool.

Mode

vmake: Automatically generate test-data with user-provided standard solution and Maker (described below).
vcheck: Execute a solution on set of test-data like an OnlineJudge does. WARNING: No sandbox protection, don't run any untrusted code!

Maker

A Maker is an executable that generates the input of a single test case.

How to write a Maker?

Read the subtask ID (starting from 1), a single integer from stdin.
Output the input data to stdout.

Usage

Create vtest.conf.

The format of vtest.conf:

<name> <subtask count>
<data path>
<case count for subtask #1>
<case count for subtask #2>
...
<case count for subtask #n>

Example:

AplusB 5
data
5
15
20
20
40

The file structure will be like this

.
├─ <data path> // Auto Generated
│  ├─ <name>.1.1.in
│  ├─ <name>.1.1.out
│  ├─ ...
│  ├─ <name>.<subtask count>.<n>.in
│  └─ <name>.<subtask count>.<n>.out
│
├─ mk_<name>  // Compile yourself
├─ std_<name> // Compile yourself
├─ run_<name> // Compile yourself
├─ vmake.py   // Downloaded
└─ vcheck.py  // Downloaded

Example

Here is a example of generating test-data of the A + B problem and testing a solution of it.

Create an empty folder.
Create vtest.conf with the following content:
```
AplusB 2
data
6
4
```
Download vmake.py and vcheck.py.

You can use the following commands if you prefer CLI or simply click Download Zip in the project repository page and extract vmake.py and vcheck.py:
```
wget https://github.com/ZTL-UwU/py-vtest/raw/main/vmake.py
wget https://github.com/ZTL-UwU/py-vtest/raw/main/vcheck.py
```
Create a standard solutoion as an executable named std_AplusB.

For example, the following code is a C++ version of a standard solution of A + B problem, compile it into std_AplusB:
```
// std_AplusB.cpp
#include <iostream>

int main() {
    long long a, b;
    std::cin >> a >> b;
    std::cout << a + b;
    return 0;
}
```
Compile commands:
```
g++ std_AplusB.cpp -o std_AplusB
```

Create an executable named mk_AplusB which is a Maker (described above).

For example, the following is a C++ version of the test-data Maker, compile it into mk_AplusB:

// mk_AplusB.cpp
#include <iostream>
#include <random>

int main() {
    int subtask_id;
    std::cin >> subtask_id;

    if (subtask_id == 1) {
        std::mt19937 rng(std::random_device{}());
        std::cout << rng() << " " << rng();
    }

    if (subtask_id == 2) {
        // In this subtask, we will generate larger inputs
    // which can hack solutions without using long long.
        std::mt19937_64 rng(std::random_device{}());
        std::cout << rng() << " " << rng();
    }

    return 0;
}

Compile commands:

g++ mk_AplusB.cpp -o mk_AplusB

Run vmake.py.

You can use the following command or simply double-click on vmake.py:

python3 vmake.py

The output is similar to the following:

Start Making data for AplusB.

Making subtask #1
    [ 10%] Made case #1.1: (9.0ms)
    [ 20%] Made case #1.2: (2.17ms)
    [ 30%] Made case #1.3: (5.08ms)
    [ 40%] Made case #1.4: (2.53ms)
    [ 50%] Made case #1.5: (4.01ms)
    [ 60%] Made case #1.6: (3.81ms)
Making subtask #2
    [ 70%] Made case #2.1: (1.99ms)
    [ 80%] Made case #2.2: (3.06ms)
    [ 90%] Made case #2.3: (2.04ms)
    [100%] Made case #2.4: (3.4ms)

Summary:
    Slowest case: #1.1 (9.0ms)

Now you can see the generated data in the data folder.

.
├─ data
|  ├─ AplusB.1.1.in
|  ├─ AplusB.1.1.out
|  ├─ ...
|  ├─ AplusB.2.4.in
|  └─ AplusB.2.4.out

Lets try another solution without using long long (who cannot pass the test).

This is a C++ version of a wrong solution, compile it into run_AplusB:

// AplusB_wrong.cpp
#include <iostream>

int main() {
    int a, b;
    std::cin >> a >> b;
    std::cout << a + b;
    return 0;
}

Compile commands:

g++ AplusB_wrong.cpp -o run_AplusB

Run vcheck.py

You can use the following command or simply double-click on vcheck.py:

python3 vcheck.py