From 8796f519f119d6fa9ae8a05ccc35acbacf4fb527 Mon Sep 17 00:00:00 2001 From: Ryan Taylor Date: Sat, 4 May 2024 13:26:30 -0400 Subject: [PATCH] minor fix --- readme.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/readme.md b/readme.md index 236d247..d69200c 100644 --- a/readme.md +++ b/readme.md @@ -27,7 +27,7 @@ Infini-Transformer ([https://arxiv.org/abs/2404.07143](https://arxiv.org/abs/240 - Efficient fine-tuning for task-specific adaptation - Includes a Mixture-of-Depths ([https://arxiv.org/abs/2404.02258](https://arxiv.org/abs/2404.02258)) transformer layer that incorporates Infini-Attention - Implementation of RoPE ([https://arxiv.org/abs/2104.09864](https://arxiv.org/abs/2104.09864)) that conforms to Infini-Attention's and Mixture-of-Depth's memory-efficient designs (PoSE to be added soon) -- - Implementation of YaRN ([https://arxiv.org/abs/2309.00071](https://arxiv.org/abs/2309.00071)) that conforms to Infini-Attention's and Mixture-of-Depth's memory-efficient designs (PoSE to be added soon) +- Implementation of YaRN ([https://arxiv.org/abs/2309.00071](https://arxiv.org/abs/2309.00071)) that conforms to Infini-Attention's and Mixture-of-Depth's memory-efficient designs (PoSE to be added soon) ## Directory structure