This is a working repository on a reinforcement learning approach to generating synthetic learning behavior data.
EdNet is the dataset of all student-system interactions collected over 2 years by Santa, a multi-platform AI tutoring service with more than 780K users in Korea available through Android, iOS and web.
The goal is to use EdNet to extract an expert policy that will be used to train an imitation learning agent. This agent will then model student behavior and learning habits that can be used to expand on the EdNet dataset more efficiently than by using conventional methods.
EdNet consists of four datasets with increasing resolution in terms of the actions of students being tracked.
timestampis the moment the question was given, represented as Unix timestamp in milliseconds.solving_idrepresents each learning session of students corresponds to each bunle. It is a form of single integer, starting from1.question_idis the ID of the question that given to student, which is a form ofq{integer}.user_answeris the answer that the student submitted, recorded as a character betweenaanddinclusively.elapsed_timeis the time that the students spends on each question in milliseconds.
action_typeis one of the following:enter,respond, andsubmit.enteris recorded when student first receives and views a question bundle through UI.respondis recorded when the student selects an answer choice to one of the questions in the bundle. A student can respond to the same question multiple times. In this case, only the last response before submitting his final answer is considered as his response.submitis recorded when the student submits his final answers to the the given bundle.
item_idis The ID of item involved with the action. For EdNet-KT2, only the IDs of questions and bundles are recorded. A bundle is assigned for actions of typeenterandsubmit.sourceshows where the student solve a question or watch a lecture in Santa UI. There are several sources in Santa that students can solve questions or watch lectures. For KT2, only the sources that provides question-solving environments are recorded.- In
sprint, students choose a part that they want to study. After that, they can only solve questions belongs to the part that they choose, until they change to different part or select different source. - For each day, Santa recommends questions and lectures based on each student's current knowledge status, i.e. correctness probabilities predicted by the Collaborative Filtering model. Such source is called Today's Recommendation. Questions that belong to particular parts can be recommended,
todays_recommendation::sprint,todays_recommendation::review_quiz - Once the number of incorrect answers to questions with particular tags exceeds certain threshold, Santa suggests lectures and questions with corresponding tags. Such suggestion is recorded as
adaptive_offer. It also offers lectures and questions if the average correctness rate of questions with particular tags decreased by more than a certain threshold. - All Parts is a source that students solve questions that Santa recommends following a certain algorithm, from all possible candidates. This is recorded as
tutor. - The student can re-do the questions that he already solved before using review system, which is recorded as
in_review.
- In
user_answeris recorded whenaction_typeisrespond, which stands for the student's submitted answer. It is one of the alphabetsa,b,c, andd.platformshows where the student used Santa, which is either mobile or web.
- Reading Explanations
- After each student solves given questions, corresponding explanations are given to him. The sources of explanations for questions from
sprintandrevieware recorded asafter_sprintandafter_review, respectively. One can also re-read explanations of questions that he solved frommy_note. - Whenever a student enters the explanation view in Santa UI or exits the view, a corresponding action of type
enterandquitwith explanation ID asitem_idis recorded. Note that the explanation ID is exactly same as the bundle ID.
- After each student solves given questions, corresponding explanations are given to him. The sources of explanations for questions from
- Watching Lectures
- There are two possible sources that the student can watch a lecture:
archive,adaptive_offer, andtodays_recommendatin::lecture. He can access all the possible lectures fromarchive. Also, Santa can suggest lectures by Today's recommendation or adaptive offer along with questions. - Whenever a student plays a lecture video or stops watching the video, a corresponding action of type
enterandquitwith lecture ID asitem_idis recorded.
- There are two possible sources that the student can watch a lecture:
erase_choice,undo_erase_choice- For user's convenience, a student can hide an answer choice by erasing it. He can also undo his action to consider the choice again. The act of erasing a choice and undoing it are given as actions of type
erase_choiceandundo_erase_choicerespectively. The answer choice erased/un-erased is supplied in theuser_answercolumn.
- For user's convenience, a student can hide an answer choice by erasing it. He can also undo his action to consider the choice again. The act of erasing a choice and undoing it are given as actions of type
play_audio,pause_audio,play_video,pause_video- A student can play or pause a given multimedia asset. For videos, he can also navigate to different moments of the video by moving his cursor to different places. Such actions are denoted as one of the action types
play_audio,pause_audio,play_videoorpause_video. A columncursor_timeis added to EdNet-KT4, to represent the moment where he has played or paused the media.
- A student can play or pause a given multimedia asset. For videos, he can also navigate to different moments of the video by moving his cursor to different places. Such actions are denoted as one of the action types
pay,refund- By default, a free user is offered 10 questions of Part 2 and 5 each daily. By purchasing a payment item, the student have full access to questions of all parts. A table of payment items is provided separately (see below). Items of type
passallows solving all questions for the timedurationin milliseconds. Items of typepaygoallows student to solve the specific number of bundles denoted by columnnumber_of_bundles.
- By default, a free user is offered 10 questions of Part 2 and 5 each daily. By purchasing a payment item, the student have full access to questions of all parts. A table of payment items is provided separately (see below). Items of type
enroll_coupon- A student may enter his promotion coupon code to receive corresponding benefits. The ID of his coupon and the time he entered the coupon is recorded as an action of type
enroll_coupon. A table of coupons is provided separately (see below).
- A student may enter his promotion coupon code to receive corresponding benefits. The ID of his coupon and the time he entered the coupon is recorded as an action of type
From the sections above, the following list of actions is extracted:
enter(view a question bundle or the explanation)respond(select answer)submit(submit answer)quit(quit explanation view)erase_choiceundo_erase_choiceplay_audiopause_audioplay_videopause_videopayrefundenroll_coupon
The agent will take actions in an environment. At each timestep, the environment can be described by states and it is assumed that the agent has access to all states at any point in time. Furthermore, the imitation learning agent will learn how the expert policy influences the states.
timestamp(the moment the question was given)solving_idquestion_iditem_idsourceafter_sprintafter_reviewarchiveadaptive_offertodays_recommendatin::lecture