Deal with dependencies requested by LLMs
Opened this issue · 2 comments
ahumenberger commented
Assume an LLM responds with something like
package com.eval;
import org.junit.jupiter.api.Test;
import org.mockito.InMockito;
import static org.junit.jupiter.api.Assertions.*;
class PlainTest {
@Test
void testPlain() {
InMockito.when(Plain::plain).thenReturn(null);
Plain.plain();
InMockito.verify(Plain::plain);
}
}
The test file needs mockito to run, and we need to make sure that this is available when executing the tests.
Response from custom-fireworks_accounts_fireworks_models_qwen2-72b-instruct
bauersimon commented
What's also bad is that we just go mod tidy
all dependencies for Go... so LLMs can use whatever they want, but for Java everything except Java is forbidden.
ahumenberger commented
llama-3-70b-instruct
responded with tests using hamcrest
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.CoreMatchers.equalTo;
import static org.hamcrest.MatcherAssert.assertThat;