Working on ESMTopic and trying to understand how to include a topological inductive bias into a generative diffusion model for protein generation using an ESMFold backbone.
Also working on fine-tuning ESM-2 models using Low Rank Adaptation (LoRA) and Quantized LoRA (QLoRA) for RNA binding site prectors for protein sequences.
Also working on understanding equivariant model architectures and how they relate to ESM-2, ESMFold, and ESM-IF1.
I would also like to eventually apply some of my knowledge and ideas to LLMs and developing a TrOCR like model for Hebrew and Yiddish OCR and translation.