rohanpadhye/JQF

How to write a generic generator?

Closed this issue · 3 comments

Since I need to test a lot of different classes I don't want to write a generator for each one manually, so I wrote the following simple generic generator:

package caohch1;

import com.pholser.junit.quickcheck.generator.Generator;
import com.pholser.junit.quickcheck.random.SourceOfRandomness;
import com.pholser.junit.quickcheck.generator.GenerationStatus;
import java.lang.reflect.Constructor;
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
import java.util.*;

public class GenericGenerator<T> extends Generator<T> {
    private final Class<T> type;

    public GenericGenerator(Class<T> type) {
        super(type);
        this.type = type;
    }

    @Override
    public T generate(SourceOfRandomness random, GenerationStatus status) {
        random.nextBytes(4); 

        try {
            T instance = createInstance(random);
            if (instance != null) {
                populateFields(instance, random);
            }
            return instance;
        } catch (Exception e) {
            return null;
        }
    }

    private T createInstance(SourceOfRandomness random) {
        Constructor<?>[] constructors = type.getDeclaredConstructors();
        Arrays.sort(constructors, (c1, c2) -> 
            Integer.compare(c1.getParameterCount(), c2.getParameterCount()));

        for (Constructor<?> constructor : constructors) {
            constructor.setAccessible(true);
            try {
                Class<?>[] paramTypes = constructor.getParameterTypes();
                Object[] params = new Object[paramTypes.length];
                
                for (int i = 0; i < paramTypes.length; i++) {
                    params[i] = generateRandomValue(paramTypes[i], random);
                }
                
                @SuppressWarnings("unchecked")
                T instance = (T) constructor.newInstance(params);
                return instance;
            } catch (Exception e) {
                continue;
            }
        }
        return null;
    }

    private void populateFields(T instance, SourceOfRandomness random) {
        try {
            for (Field field : instance.getClass().getFields()) {
                if (!Modifier.isStatic(field.getModifiers()) && 
                    !Modifier.isFinal(field.getModifiers())) {
                    Object value = generateRandomValue(field.getType(), random);
                    if (value != null) {
                        field.set(instance, value);
                    }
                }
            }
        } catch (Exception e) {
            // Ignore field population errors
        }
    }

    private Object generateRandomValue(Class<?> type, SourceOfRandomness random) {
        random.nextBytes(1);
        
        if (type == int.class || type == Integer.class) {
            return random.nextInt();
        }
        if (type == long.class || type == Long.class) {
            return random.nextLong();
        }
        if (type == boolean.class || type == Boolean.class) {
            return random.nextBoolean();
        }
        if (type == byte.class || type == Byte.class) {
            return random.nextByte(Byte.MIN_VALUE, Byte.MAX_VALUE);
        }
        if (type == char.class || type == Character.class) {
            return (char) random.nextChar(Character.MIN_VALUE, Character.MAX_VALUE);
        }
        if (type == double.class || type == Double.class) {
            return random.nextDouble();
        }
        if (type == float.class || type == Float.class) {
            return random.nextFloat();
        }
        if (type == String.class) {
            int length = random.nextInt();
            StringBuilder sb = new StringBuilder(length);
            for (int i = 0; i < length; i++) {
                sb.append(random.nextChar(Character.MIN_VALUE, Character.MAX_VALUE));
            }

            return sb.toString();
        }
        if (type.isEnum()) {
            Object[] constants = type.getEnumConstants();
            return constants[random.nextInt(constants.length)];
        }
        if (List.class.isAssignableFrom(type)) {
            return new ArrayList<>();
        }
        if (Set.class.isAssignableFrom(type)) {
            return new HashSet<>();
        }
        if (Map.class.isAssignableFrom(type)) {
            return new HashMap<>();
        }
        
        try {
            return new GenericGenerator<>(type).generate(random, null);
        } catch (Exception e) {
            return null;
        }
    }
}

I use a simple case to test this generator

public class Test {
    @Fuzz
    public void fuzzLocalDateTime(@From(GenericGenerator.class) String var1, @From(GenericGenerator.class) String var2) throws IllegalArgumentException, DateTimeParseException {
      LocalDateTime.parse(var1, DateTimeFormatter.ofPattern(var2));
    }
}

However, after I run ../bin/jqf-zest -c "$DEPENDENCY":. Test fuzzLocalDateTime I observe that the code coverage is stuck and doesn't increase as following:

Semantic Fuzzing with Zest
--------------------------

Test name:            Test#fuzzLocalDateTime
Instrumentation:      Janala
Results directory:    ...../fuzz-results
Elapsed time:         9s (no time limit)
Number of executions: 11,578 (no trial limit)
Valid inputs:         11,578 (100.00%)
Cycles completed:     10
Unique failures:      0
Queue size:           3 (1 favored last cycle)
Current parent input: 0 (favored) {627/1000 mutations}
Execution speed:      2,116/sec now | 1,178/sec overall
Total coverage:       147 branches (0.22% of map)
Valid coverage:       147 branches (0.22% of map)

Is there an existing implementation for a generic generator?

JQF is built on top of JUnit-Quickcheck, which has support for implicit generators from public constructors via @From(Ctor.class) and settable fields via @From(Fields.class).

JQF is built on top of JUnit-Quickcheck, which has support for implicit generators from public constructors via @From(Ctor.class) and settable fields via @From(Fields.class).

Thanks for your quick reply! I replaced @From(GenericGenerator.class) with @From(Fields.class) in fuzzLocalDateTime but I still observe that the code coverage is stuck and doesn't increase. Is it normal?