Skip to content

@llm-jp llm-jp

Change the repository type filter

All

    Repositories list

    45 repositories

    • text2dataset

      Public
      Easily turn large English text datasets into Japanese text datasets using open LLMs.
      Python
      •
      MIT License
      •1•17•6•0•Updated Jan 20, 2025Jan 20, 2025

    • scripts

      Public
      Shell
      •
      Apache License 2.0
      •1•2•4•12•Updated Jan 20, 2025Jan 20, 2025

    • megatron-lm-bert

      Public
      Ongoing research training transformer models at scale
      Python
      •
      Other
      •2.5k•0•0•0•Updated Jan 19, 2025Jan 19, 2025

    • llm-jp-membership-inference

      Public
      This is the repository for llm jp membership inference attack.
      Python
      •
      Apache License 2.0
      •0•0•0•0•Updated Jan 18, 2025Jan 18, 2025

    • nlp2025-ws

      Public
      NLP2025ワークショップ「大規模言語モデルのファインチューニング技術と評価」
      Python
      •0•0•0•0•Updated Jan 14, 2025Jan 14, 2025

    • llm-human-eval-jp

      Public
      0•0•0•0•Updated Jan 10, 2025Jan 10, 2025

    • swallow-evaluation-vllm

      Public
      Shell
      •0•0•0•0•Updated Jan 4, 2025Jan 4, 2025

    • lm-evaluation-harness

      Public
      A framework for few-shot evaluation of language models.
      Python
      •
      MIT License
      •2k•0•0•0•Updated Jan 4, 2025Jan 4, 2025

    • awesome-japanese-llm

      Public
      日本語LLMまとめ - Overview of Japanese LLMs
      japanesegenerative-modeljapanese-languagelanguage-modelslanguage-modelgenerative-modelsmultimodalvision-and-languagevision-languagefoundation-models
      TypeScript
      •
      Apache License 2.0
      •31•1.1k•3•0•Updated Dec 31, 2024Dec 31, 2024

    • llm-jp-eval

      Public
      Python
      •
      Apache License 2.0
      •38•114•5•3•Updated Dec 30, 2024Dec 30, 2024

    • nii-sakura-cluster-configs

      Public
      Shell
      •0•3•0•0•Updated Dec 24, 2024Dec 24, 2024

    • Megatron-LM

      Public
      Python
      •
      Other
      •1•6•0•0•Updated Dec 19, 2024Dec 19, 2024

    • tuning-competition

      Public
      CSS
      •0•0•0•0•Updated Dec 16, 2024Dec 16, 2024

    • tuning-competition-baseline

      Public
      Python
      •0•1•0•0•Updated Dec 16, 2024Dec 16, 2024

    • llm-jp-VILA

      Public
      Python
      •
      Apache License 2.0
      •0•8•0•0•Updated Dec 16, 2024Dec 16, 2024

    • FastChat2

      Public
      An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
      Python
      •
      Apache License 2.0
      •4.6k•0•0•0•Updated Nov 25, 2024Nov 25, 2024

    • Toxicity_Filter_fastText

      Public
      A fastText-based classifier for removing toxic texts from corpora, trained on automatically generated labeled data.
      Python
      •0•0•0•0•Updated Nov 23, 2024Nov 23, 2024

    • Toxicity_Filter

      Public
      A DeBERTa-based toxicity filter trained on LLM-jp Toxicity Dataset
      Python
      •0•0•0•0•Updated Nov 5, 2024Nov 5, 2024

    • moe-recipes

      Public
      Shell
      •0•2•0•0•Updated Oct 24, 2024Oct 24, 2024

    • transformers

      Public
      🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
      Python
      •
      Apache License 2.0
      •28k•0•0•0•Updated Oct 14, 2024Oct 14, 2024

    • llm-leaderboard

      Public
      Project of llm evaluation to Japanese tasks
      Python
      •37•1•0•0•Updated Aug 11, 2024Aug 11, 2024

    • FastChat

      Public
      An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
      Python
      •
      Apache License 2.0
      •4.6k•0•0•0•Updated Aug 9, 2024Aug 9, 2024

    • experiments

      Public
      Issue-Only Pretrain Task Management Repository
      0•0•68•0•Updated Aug 8, 2024Aug 8, 2024

    • llm-jp-tokenizer

      Public
      Roff
      •
      Apache License 2.0
      •6•33•1•0•Updated Aug 7, 2024Aug 7, 2024

    • llmjp_topic_model

      Public
      Python
      •0•0•0•0•Updated Jul 19, 2024Jul 19, 2024

    • modelwg

      Public
      LLM-jp Model-WG Working Directory
      Shell
      •
      Apache License 2.0
      •0•6•0•4•Updated Jul 9, 2024Jul 9, 2024

    • DeepSpeed-MII

      Public
      MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
      Python
      •
      Apache License 2.0
      •176•0•0•0•Updated Jul 5, 2024Jul 5, 2024

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •5.2k•0•0•0•Updated Jun 21, 2024Jun 21, 2024

    • nii-geniac-gcp-lustre

      Public
      GENIAC用のLustreクラスタ構築スクリプト
      Jinja
      •0•0•0•0•Updated Jun 20, 2024Jun 20, 2024

    • nii-geniac-hpc-toolkit

      Public
      HCL
      •0•1•0•0•Updated Jun 13, 2024Jun 13, 2024