tobyli
Toby Li is an Assistant Professor at Notre Dame with interests in HCI, conversational agents, intelligent interfaces, and human-AI collaboration
University of Notre DameNotre Dame, IN
Pinned Repositories
PrivacyStreams
Access and process various types of personal data in Android with a set of easy, uniform, and privacy-friendly APIs.
wikibrain
The WikiBrain Java library enables researchers and developers to incorporate state-of-the-art Wikipedia-based algorithms and technologies in a few lines of code.
consensus
A GDM tool, version 0.2
EmoLight
gabriel
Cognitive assistance with Google Glass
gabriel_android-client
Screen2Vec
Screen2Vec is a new self-supervised technique for generating more comprehensive semantic embeddings of GUI screens and components using their textual content, visual design and layout patterns, and app meta-data.
snanalyzer
A simulator for social network privacy research: simulate the history sniffing attacks and history poisoning defenses
Sugilite_development
SUGILITE is a new programming-by-demonstration (PBD) system that enables users to create automation on smartphones. SUGILITE uses Android’s accessibility API to support automating arbitrary tasks in any Android app (or even across multiple apps). When the user gives verbal commands that SUGILITE does not know how to execute, the user can demonstrate by directly manipulating the regular apps’ user interface. By leveraging the verbal instructions, the demonstrated procedures, and the apps’ UI hierarchy structures, SUGILITE can automatically generalize the script from the recorded actions, so SUGILITE learns how to perform tasks with different variations and parameters from a single demonstration. Extensive error handling and context checking support forking the script when new situations are encountered, and provide robustness if the apps change their user interface. Our lab study suggests that users with little or no programming knowledge can successfully automate smartphone tasks using SUGILITE.
wikibrain
The WikiBrain Java library enables researchers and developers to incorporate state-of-the-art Wikipedia-based algorithms and technologies in a few lines of code.
tobyli's Repositories
tobyli/Screen2Vec
Screen2Vec is a new self-supervised technique for generating more comprehensive semantic embeddings of GUI screens and components using their textual content, visual design and layout patterns, and app meta-data.
tobyli/Sugilite_development
SUGILITE is a new programming-by-demonstration (PBD) system that enables users to create automation on smartphones. SUGILITE uses Android’s accessibility API to support automating arbitrary tasks in any Android app (or even across multiple apps). When the user gives verbal commands that SUGILITE does not know how to execute, the user can demonstrate by directly manipulating the regular apps’ user interface. By leveraging the verbal instructions, the demonstrated procedures, and the apps’ UI hierarchy structures, SUGILITE can automatically generalize the script from the recorded actions, so SUGILITE learns how to perform tasks with different variations and parameters from a single demonstration. Extensive error handling and context checking support forking the script when new situations are encountered, and provide robustness if the apps change their user interface. Our lab study suggests that users with little or no programming knowledge can successfully automate smartphone tasks using SUGILITE.
tobyli/wikibrain
The WikiBrain Java library enables researchers and developers to incorporate state-of-the-art Wikipedia-based algorithms and technologies in a few lines of code.
tobyli/EmoLight
tobyli/snanalyzer
A simulator for social network privacy research: simulate the history sniffing attacks and history poisoning defenses
tobyli/consensus
A GDM tool, version 0.2
tobyli/gabriel
Cognitive assistance with Google Glass
tobyli/gabriel_android-client
tobyli/PrivacyStreams.github.io
The PrivacyStreams website.
tobyli/tranX
A general-purpose neural semantic parser for mapping natural language queries into machine executable code
tobyli/CHI2017Demo
Demo of CHI 2017 paper: "Interaction Proxies for Runtime Repair and Enhancement of Mobile Application Accessibility"
tobyli/CSrankings
A web app for ranking computer science departments according to their research output in selective venues, and for finding active faculty across a wide range of areas.
tobyli/ucremodeler
tobyli/CSStipendRankings
https://csstipendrankings.org
tobyli/GeoTweetsAnalyzer
A course project from 5980 - sniff & analyze tweets with geotags
tobyli/homeaway-fromhome
tobyli/InstructableServer
tobyli/screen2vec-survey
tobyli/sempre
Semantic Parser with Execution
tobyli/SugiliteCommunicationTest
tobyli/tensor2tensor
A library for generalized sequence to sequence models