need help understanding

[freedom1b2830]

I have a few questions.

designations:

(ret to fuzzer) this is a function call (with @ Fuzz annotation) by a fuzzer
>>SEE<< the highlights of my question

Questions/Suggestions

1)I suggest adding null values to the generator for wrapper classes (int(Integer),long(Long) and others) (to substitute fuzzing as an argument)

args:null ->(ret to fuzzer)

2)add "possible" interpretations of some data types as others:

Long loo=140;//RANDOM
String ret=Long.toString(loo);
(ret to fuzzer)

or in a line (with the name of the function in my code):

Long->String->i1;
Long->String->i2;
invoke fuzzCreateContextPath(String i1, String i2);

3)

//CHECK

I want to get 3 obvious errors

1)java.lang.NullPointerException
2)java.lang.NumberFormatException
3)java.lang.ArithmeticException: / by zero

but i get only 1 exception type but with different argument data

.id_000000[0]: 񥂉🤒񔫰󏶰򀼏񶿋񗦈񓠅􁴟뎆󆊊񒦳􎕔񨪦򜟟񞿐򰇐𡗋󡽈ꩫ񭛃򋡀𦞪𧹟
id_000000[1]: 悖ᯪ䏺窚遖㟩⎛汪⼾嘬臑铸퍞Ạ胎矿垊䖿頳摗ꂌ쀐ꋐ᱅懗
id_000000 ::= FAILURE (java.lang.NumberFormatException)
E
Time: 0,177
There was 1 failure:
1) fuzzCreateContextPath(freedom1b2830.fforum.test.Pages)
java.lang.NumberFormatException: For input string: "񥂉🤒񔫰󏶰򀼏񶿋񗦈񓠅􁴟뎆󆊊񒦳􎕔񨪦򜟟񞿐򰇐𡗋󡽈ꩫ񭛃򋡀𦞪𧹟"
	at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
	at java.base/java.lang.Integer.parseInt(Integer.java:668)
	at java.base/java.lang.Integer.valueOf(Integer.java:999)
	at freedom1b2830.fforum.test.Pages.fuzzCreateContextPath(Pages.java)

FAILURES!!!
Tests run: 1,  Failures: 1

1. can't progress past this exception NumberFormatException
   here is the message:

Test name:            freedom1b2830.fforum.test.Pages#fuzzCreateContextPath
Results directory:    /home/dev/.eclipse/WORKSPACE/fforum/target/fuzz-results/freedom1b2830.fforum.test.Pages/fuzzCreateContextPath
Elapsed time:         30s (no time limit)
Number of executions: 99 341 (no trial limit)
Valid inputs:         0 (0,00%)
Cycles completed:     0
Unique failures:      3 // >>SEE<< all 3 type 1 exceptions
Queue size:           0 (0 favored last cycle)
Current parent input: <seed>
Execution speed:      4 233/sec now | 3 252/sec overall
Total coverage:       0 branches (0,00% of map)
Valid coverage:       0 branches (0,00% of map)
Fuzzing stopped due to guidance exception: Too many trials without coverage; likely all assumption violations
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:02 min
[INFO] Finished at: 2022-05-30T21:49:05Z
[INFO] ------------------------------------------------------------------------

>>SEE<<

[ERROR] Failed to execute goal edu.berkeley.cs.jqf:jqf-maven-plugin:1.8:fuzz (default-cli) on project fforum: Internal error: Too many trials without coverage; likely all assumption violations -> [Help 1]

my code:

@RunWith(JQF.class)
public class Pages {
	public CopyOnWriteArrayList<Class<? extends Exception>> exceptions = new CopyOnWriteArrayList<>();

	public @Fuzz void fuzzCreateContextPath(String i1, String i2) {//NPE from
		try {
			Integer intI1 = Integer.valueOf(i1);//NPE here, NumberFormatException here
			Integer intI2 = Integer.valueOf(i2);//NPE here, NumberFormatException here
			int ext = intI1 / intI2; //ArithmeticException
		} catch (Exception e) {

                        // >>SEE<<
                        //CHECK start
                       //here is a check that the exception is unique (>> the question is how to skip the caught exception, and write down what is 'throw e;' )
                        //in the eclipse IDE, there is a switch in the debug tool to catch all_caught(try{}catch)/uncaught exceptions. How do I set this up in JQF ?
			Class<? extends Exception> ccc = e.getClass();
			if (!exceptions.contains(ccc)) {
				exceptions.add(ccc);
				throw e;
			}
                        //CHECK end
		}
	}
}

[freedom1b2830]

I will supplement my questions with illustrations with the help of yed

[rohanpadhye]

Thanks for the questions. One of the design decisions of JQF was to be compatible with JUnit-Quickcheck tests, so we borrow the interpretation of type generators etc. from there.

1)I suggest adding null values to the generator for wrapper classes (int(Integer),long(Long) and others) (to substitute fuzzing as an argument)

Null-input generation is not supported by default in JUnit-QuickCheck, but can be added via annotations such as @NullAllowed or @Nullable; see docs: https://pholser.github.io/junit-quickcheck/site/1.0/usage/null.html.

JQF did not support these annotations for some reason, so I just pushed a commit that will allow these to be used with JQF-style generators. Hope this solves your use case! (Note that Integer.valueOf() does not throw an NPE for null args, only a NumberFormatException with a message "Cannot parse null string").

2)add "possible" interpretations of some data types as others:

Again, we choose to use the default generators from junit-quickcheck, which do random uniform sampling. If you want to add special values, you can roll out a custom generator that uses tokens pulled out of a dictionary of special values. For an example, see DictionaryBackedStringGenerator and its sample usage when fuzzing Ant's build.xml input with a dict file.

3)java.lang.ArithmeticException: / by zero

Getting this exception in your example would be possible but incredibly unlikely. JQF would have to first generate a valid String i1 which is of an integer representation and then generate a String i2 which is exactly the single character 0. The reason this is unlikely is because by default, JQF does not instrument the internals of JDK classes such as Integer and cannot track the fuzzing progress ("coverage") within the implementation of Integer.valueOf(). This might work if you had your own string-to-integer parser. For fuzzing JDK internals, you will have to use the javaagent approach (not the Maven plugin) and mess with JVM command-line flags to allow JQF to instrument the JDK classes such as java.lang.Integer. This is possible but not recommended, and I do not have the bandwidth to document the hackery required for this.

[freedom1b2830]

please check the code
if (true) {

[freedom1b2830]

I'm wondering how to implement data mutation in the generator
how to track the place where the data allows you to "pass" the code further (pass checks, etc.)