Skip to content

Latest commit

 

History

History
44 lines (32 loc) · 1.72 KB

README.md

File metadata and controls

44 lines (32 loc) · 1.72 KB
Raw
doji logo

Transformers

A Unity package to run pretrained transformer models with Unity Sentis

OpenUPM · Documentation (coming soon) · Feedback/Questions

About

This is essentially a C# port of Hugging Face’s transformers library.

There are two use cases for this package right now:

  • It's used by the com.doji.diffusers package to run Stable Diffusion models in Unity (most SD models use a ClipTokenizer for prompting, newer pipelines require additional ones like T5Tokenizer)
  • To run small LLMs like Phi-3 in Unity (WIP)

Roadmap:

Tokenizers

  • CLIPTokenizer
  • LLamaTokenizer
  • GPT2Tokenizer
  • T5Tokenizer

LLMs

  • Phi-3

The intention is to provide a similar API like Hugging Face's transformers library, so usage in Unity will look something like this:

var tokenizer = AutoTokenizer.FromPretrained("julienkay/Phi-3-mini-4k-instruct_no_cache_uint8");
var model = Phi3ForCausalLM.FromPretrained("julienkay/Phi-3-mini-4k-instruct_no_cache_uint8");

var inputs = tokenizer.Encode("<input-prompt>");
var outputs = model.Generate(inputs);
var predictedText = tokenizer.Decode(outputs);