Does this work with LLama 3?

[thomasgauthier]

Question in the title.

Big thanks for making this accessible to the community!

[ChenxinAn-fdu]

Thank you for bringing up this issue! We are actively testing DCA with Llama3 8B/70B. The results will be updated next week!

[ChenxinAn-fdu]

Hi! The language modeling results for Llama3 8b/70b have been updated! DCA is still very effective! ChunkLlama3 shows an obvious performance gain over ChunkLlama2 in PPL.

Updates of needle in a haystack, few-shot, and zero-shot results are expected in two days. These are running a bit slow.

[ChenxinAn-fdu]

All results of ChunkLlama3 8b/70b has been updated. (see the results)

Generally, ChunkLlama3-8b achieves 100% retrieval accuracy across all document depths.
Results on real-world tasks show that ChunkLlama3-70b achieves performance on par with GPT-4 (2023/06/13) and Llama2 Long 70b!