Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add v0.6.0 release notes #1991

Merged
merged 2 commits into from
Sep 10, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified assets/wechat.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1 +1,86 @@
# DB-GPT V0.6.0, Defining new standards for AI-native data applications.
# DB-GPT V0.6.0, Defining new standards for AI-native data applications.

## Introduction

DB-GPT is an open source AI native data application development framework with AWEL and agents. In the V0.6.0 version, we further provide flexible and scalable AI native data application management and development capabilities around large models, which can help enterprises quickly build and deploy intelligent AI data applications, and achieve enterprise digital transformation and business growth through intelligent data analysis, insights and decisions


### The V0.6.0 version mainly adds and enhances the following core features

- AWEL protocol upgrade 2.0, supporting more complex orchestration

- Supports the creation and lifecycle management of data applications, and supports multiple application construction modes, such as: multi-agent automatic planning mode, task flow orchestration mode, single agent mode, and native application mode

- GraphRAG supports graph community aggregation and hybrid retrieval, and the graph index cost is reduced by 50% compared to Microsoft GraphRAG.
csunny marked this conversation as resolved.
Show resolved Hide resolved

- Supports multiple Agent Memories, such as perceptual memory, short-term memory, long-term memory, mixed memory, etc.

- Supports intent recognition and prompt management, and newly added support for Text2NLU and Text2GQL fine-tuning

- GPT-Vis front-end visualization upgrade to support richer visualization charts

## Features

**AWEL protocol upgrade 2.0 supports more complex orchestration and optimizes front-end visualization and interaction capabilities.**

AWEL (Agentic Workflow Expression Language) is an agent-based workflow expression language designed specifically for large model application development, providing powerful functions and flexibility. Through the AWEL API, developers can focus on large model application logic development without having to pay attention to cumbersome model, environment and other details. In AWEL2.0, we support more complex orchestration and visualization


<p align="center">
<img src={'/img/app/agent_prompt_awel_v0.6.jpg'} width="800px" />
</p>

**Supports the creation and life cycle management of data applications, and supports multiple modes to build applications, such as: multi-agent automatic planning mode, task flow orchestration mode, single agent mode, and native application mode**

<p align="center">
<img src={'/img/app/app_manage_mode_v0.6.jpg'} width="800px" />
</p>

<p align="center">
<img src={'/img/app/app_manage_app_v0.6.jpg'} width="800px" />
</p>

**GraphRAG supports graph community summarization and hybrid retrieval.**

The graph construction and retrieval performance have obvious advantages compared to community solutions, and it supports cool visualization. GraphRAG is an enhanced retrieval generation system based on knowledge graphs. Through the construction and retrieval of knowledge graphs, it further enhances the accuracy of retrieval and the stability of recall, while reducing the illusion of large models and enhancing the effects of domain applications. DB-GPT combines with TuGraph to build efficient retrieval enhancement generation capabilities

<p align="center">
<img src={'/img/app/graph_rag_pipeline_v0.6.png'} width="800px" />
</p>

Based on the universal RAG framework launched in DB-GPT version 0.5.6 that integrates vector index, graph index, and full-text index, DB-GPT version 0.6.0 has enhanced the capabilities of graph index (GraphRAG) to support graph community summary and hybrid retrieval. ability. In the new version, we introduced TuGraph’s built-in Leiden community discovery algorithm, combined with large models to extract community subgraph summaries, and finally used similarity recall of community summaries to cope with generalized questioning scenarios, namely QFS (Query Focused Summarization). question. In addition, in the knowledge extraction stage, we upgraded the original triple extraction to graph extraction with point edge information summary, and optimized cross-text block associated information extraction through text block history to further enhance the information density of the knowledge graph.


Based on the above design, we used the open source knowledge graph corpus (OSGraph) provided by the TuGraph community and the product introduction materials of DB-GPT and TuGraph (about 43k tokens in total), and conducted comparative tests with Microsoft's GraphRAG system. Finally, DB-GPT It only consumes 50% of the token overhead and generates a knowledge graph of the same scale. And on the premise that the quality of the question and answer test is equivalent, the global search performance has been significantly improved.

<p align="center">
<img src={'/img/app/graph_rag_v0.6.png'} width="800px" />
</p>

For the final generated knowledge graph, we used AntV's G6 engine to upgrade the front-end rendering logic, which can intuitively preview the knowledge graph data and community segmentation results.

<p align="center">
<img src={'/img/app/graph_rag_display_v0.6.png'} width="800px" />
</p>


**GPT-Vis: GPT-Vis is an interactive visualization solution for LLM and data, supporting rich visual chart display and intelligent recommendations**

<p align="center">
<img src={'/img/app/app_chat_v0.6.jpg'} width="800px" />
</p>

**Text2GQL and Text2NLU fine-tuning: Newly supports fine-tuning from natural language to graph language, as well as fine-tuning for semantic classification.**

<p align="center">
<img src={'/img/ft/ft_pipeline.jpg'} width="800px" />
</p>

## Acknowledgements
This iteration is inseparable from the participation of developers and users in the community, and it also further cooperates with the [TuGraph](https://github.com/TuGraph-family) and [AntV](https://github.com/antvis) communities. Thanks to all the contributors who made this release possible!

@Aries-ckt, @Dreammy23, @Hec-gitHub, @JxQg, @KingSkyLi, @M1n9X, @bigcash, @chaplinthink, @csunny, @dusens, @fangyinc, @huangjh131, @hustcc, @lhwan, @whyuds and @yhjun1026


## Reference
- [中文手册](https://www.yuque.com/eosphoros/dbgpt-docs/ym574wh2hddunfbd)
4 changes: 3 additions & 1 deletion docs/docs/application/graph_rag.md
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
# GraphRAG
# GraphRAG

[GraphRAG](../cookbook/rag/graph_rag_app_develop.md)
101 changes: 100 additions & 1 deletion docs/docs/changelog/Released_V0.6.0.md
Original file line number Diff line number Diff line change
@@ -1 +1,100 @@
# DB-GPT V0.6.0, Defining new standards for AI-native data applications.
# DB-GPT V0.6.0, Defining new standards for AI-native data applications.

## Introduction

DB-GPT is an open source AI native data application development framework with AWEL and agents. In the V0.6.0 version, we further provide flexible and scalable AI native data application management and development capabilities around large models, which can help enterprises quickly build and deploy intelligent AI data applications, and achieve enterprise digital transformation and business growth through intelligent data analysis, insights and decisions


### The V0.6.0 version mainly adds and enhances the following core features

- AWEL protocol upgrade 2.0, supporting more complex orchestration

- Supports the creation and lifecycle management of data applications, and supports multiple application construction modes, such as: multi-agent automatic planning mode, task flow orchestration mode, single agent mode, and native application mode

- GraphRAG supports graph community aggregation and hybrid retrieval, and the graph index cost is reduced by 50% compared to Microsoft GraphRAG.

- Supports multiple Agent Memories, such as perceptual memory, short-term memory, long-term memory, mixed memory, etc.

- Supports intent recognition and prompt management, and newly added support for Text2NLU and Text2GQL fine-tuning

- GPT-Vis front-end visualization upgrade to support richer visualization charts

<p align="center">
<img src={'/img/app/app_chat_v0.6.jpg'} width="800px" />
</p>


## Features

**AWEL protocol upgrade 2.0 supports more complex orchestration and optimizes front-end visualization and interaction capabilities.**

AWEL (Agentic Workflow Expression Language) is an agent-based workflow expression language designed specifically for large model application development, providing powerful functions and flexibility. Through the AWEL API, developers can focus on large model application logic development without having to pay attention to cumbersome model, environment and other details. In AWEL2.0, we support more complex orchestration and visualization


<p align="center">
<img src={'/img/app/agent_prompt_awel_v0.6.jpg'} width="800px" />
</p>

**Supports the creation and life cycle management of data applications, and supports multiple modes to build applications, such as: multi-agent automatic planning mode, task flow orchestration mode, single agent mode, and native application mode**

<p align="center">
<img src={'/img/app/app_manage_mode_v0.6.jpg'} width="800px" />
</p>

<p align="center">
<img src={'/img/app/app_manage_app_v0.6.jpg'} width="800px" />
</p>

**GraphRAG supports graph community summarization and hybrid retrieval.**

The graph construction and retrieval performance have obvious advantages compared to community solutions, and it supports cool visualization. GraphRAG is an enhanced retrieval generation system based on knowledge graphs. Through the construction and retrieval of knowledge graphs, it further enhances the accuracy of retrieval and the stability of recall, while reducing the illusion of large models and enhancing the effects of domain applications. DB-GPT combines with TuGraph to build efficient retrieval enhancement generation capabilities

<p align="center">
<img src={'/img/app/graph_rag_pipeline_v0.6.png'} width="800px" />
</p>

Based on the universal RAG framework launched in DB-GPT version 0.5.6 that integrates vector index, graph index, and full-text index, DB-GPT version 0.6.0 has enhanced the capabilities of graph index (GraphRAG) to support graph community summary and hybrid retrieval. ability. In the new version, we introduced TuGraph’s built-in Leiden community discovery algorithm, combined with large models to extract community subgraph summaries, and finally used similarity recall of community summaries to cope with generalized questioning scenarios, namely QFS (Query Focused Summarization). question. In addition, in the knowledge extraction stage, we upgraded the original triple extraction to graph extraction with point edge information summary, and optimized cross-text block associated information extraction through text block history to further enhance the information density of the knowledge graph.


Based on the above design, we used the open source knowledge graph corpus (OSGraph) provided by the TuGraph community and the product introduction materials of DB-GPT and TuGraph (about 43k tokens in total), and conducted comparative tests with Microsoft's GraphRAG system. Finally, DB-GPT It only consumes 50% of the token overhead and generates a knowledge graph of the same scale. And on the premise that the quality of the question and answer test is equivalent, the global search performance has been significantly improved.

<p align="center">
<img src={'/img/app/graph_rag_v0.6.png'} width="800px" />
</p>

For the final generated knowledge graph, we used AntV's G6 engine to upgrade the front-end rendering logic, which can intuitively preview the knowledge graph data and community segmentation results.

<p align="center">
<img src={'/img/app/graph_rag_display_v0.6.png'} width="800px" />
</p>


**GPT-Vis: GPT-Vis is an interactive visualization solution for LLM and data, supporting rich visual chart display and intelligent recommendations**

<p align="center">
<img src={'/img/app/app_chat_v0.6.jpg'} width="800px" />
</p>

**Text2GQL and Text2NLU fine-tuning: Newly supports fine-tuning from natural language to graph language, as well as fine-tuning for semantic classification.**

<p align="center">
<img src={'/img/ft/ft_pipeline.jpg'} width="800px" />
</p>

## How to Upgrade?

[Upgrade to v0.6.0](../upgrade/v0.6.0.md)

## User Manual
- [Apps](../application/apps/app_manage.md)
- [AWEL](../awel/awel.md)
- [GraphRAG](../application/graph_rag.md)

## Acknowledgements
This iteration is inseparable from the participation of developers and users in the community, and it also further cooperates with the [TuGraph](https://github.com/TuGraph-family) and [AntV](https://github.com/antvis) communities. Thanks to all the contributors who made this release possible!

@Aries-ckt, @Dreammy23, @Hec-gitHub, @JxQg, @KingSkyLi, @M1n9X, @bigcash, @chaplinthink, @csunny, @dusens, @fangyinc, @huangjh131, @hustcc, @lhwan, @whyuds and @yhjun1026


## Reference
- [中文手册](https://www.yuque.com/eosphoros/dbgpt-docs/ym574wh2hddunfbd)
Binary file added docs/static/img/app/graph_rag_display_v0.6.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/static/img/app/graph_rag_pipeline_v0.6.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/static/img/app/graph_rag_v0.6.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/static/img/ft/ft_pipeline.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading