Support model examples

[llCurious]

Hi, I wonder does Haiku plans to support more diverse models like Bert, LLAMA, etc. It would be awesom to have these examples provided.

[davisyoshida]

@llCurious I have a haiku port of LLaMA if you want to use it: https://github.com/davisyoshida/llama-haiku

[llCurious]

That's awesom. Thank you for providing the implementation example.

[tomhennigan]

Hi @llCurious, so far we've kept the examples in Haiku itself fairly sparse and the community have provided a number of different implementations of popular papers outside of the core Haiku repo.

[llCurious]

Yep, make sense. I wonder whether you know the community provides example for Bert model?

[davisyoshida]

@llCurious There are Flax implementations of BERT etc if you need to experiment with them.

[llCurious]

Yep, this would be of great help to me. Thx a lot @davisyoshida

[davisyoshida]

Here's the directory with the code for all the HuggingFace models. For example you can look at bert/modeling_flax_bert.py for BERT.