Skip to content

Commit

Permalink
Update the default directory value where the database is located in t…
Browse files Browse the repository at this point in the history
…he data export section.

Add a graph project-level data migration plan.
Update the approach for handling cross-major version upgrades in the quick start section.
  • Loading branch information
zhangwh807 committed Nov 15, 2024
1 parent f7840ac commit 5771c05
Show file tree
Hide file tree
Showing 6 changed files with 99 additions and 11 deletions.
48 changes: 46 additions & 2 deletions docs/en-US/source/13.best-practices/3.data_migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@ The data in the original TuGraph service needs to be migrated. Based on the syst
1. Compatible migration: When the system environment before and after the migration is consistent and the TuGraph software is compatible, you can directly use the backup and recovery method to migrate data;
2. Upgrade and migration: When the system environment before and after the migration is inconsistent or the TuGraph software is not compatible, it is necessary to migrate the data by first exporting the data and then re-importing;
3. Online migration: When data migration is performed on a high-availability cluster and the network environment of the cluster is good, the original cluster can be smoothly switched to the new cluster by adding or deleting nodes.
The following article will introduce these three schemes in detail.
4. Graph project-level data migration : It includes the migration of subgraph data within the same TuGraph service as well as between different TuGraph services.

Please note that the incompatibility of TuGraph software is mainly reflected in storage methods when transitioning across major versions. The following sections will provide a detailed introduction to these four solutions.

## 2. Compatible Migration
Compatible migration means that when the system environment remains unchanged and the TuGraph software version is compatible, the data and stored procedures of the original service can be used in the new service, so it can be directly migrated.
Expand Down Expand Up @@ -79,4 +81,46 @@ Use the following command to join the new node to the cluster. After joining the
Stop the original node service, and send subsequent application requests directly to the new cluster
```bash
lgraph_server -c /usr/local/etc/lgraph_ha.json --directory db.cp --ha_conf 192.168.0.1:9090,192.168.0.2:9090,192.168.0.3:9090 -d stop
```
```

## 5. Graph Project-Level Data Migration

Graph project-level data migration includes subgraph data migration within the same TuGraph service as well as subgraph data migration between different TuGraph services. For subgraph data migration within the same TuGraph service, you can migrate a graph project from the current service to a new project. The new project will be completely consistent with the migrated project in terms of models, nodes, and edges data. For subgraph data migration between different TuGraph services, you can migrate a graph project from another TuGraph service to the current TuGraph service. The models and data will remain consistent before and after the migration.

### 5.1. TuGraph Data Directory Description

As shown in the figure, data is by default located in the `/var/lib/lgraph/data` directory. Within this path, each directory named with a long string represents the data storage location of a specific graph project. By entering one of these directories, you can find the data.mdb file, which stores the specific models and data for that graph project.The data storage directory for graph projects currently does not support naming by project names, so different projects can be distinguished by their creation time.

![dataPath](../../../images/dataPath.png)

### 5.2. Graph Project Data Migration

Execute the following statement in the graph query of any graph project to perform the migration:

```
CALL db.importor.fullFileImportor("${newGraphName}","data.mdb",false)
```

Parameter Description:
1. The first parameter is the name of the new subgraph.
2. The second parameter is the absolute file path of the graph project to be migrated on the server. For path details, please refer to the "TuGraph Data Directory Description".
3. The third parameter specifies whether the file is a remote file. If it is a local file on the server, set this parameter to false or leave it unset.

Note:
If the name of the new subgraph specified in the first parameter already exists in an existing graph project, the original subgraph data will be overwritten by the migrated data. If the subgraph does not exist, a new subgraph will be created. It is advisable to avoid using duplicate names to prevent data loss due to overwriting.

Example:

```
CALL db.importor.fullFileImportor("test","/var/lib/lgraph/data/9DF16A038435BBAAE36E53D1E09DED17/data.mdb",false)
```

Execute the migration statement within the graph query of any graph project.

![execute-data-migration-statements](../../../images/execute-data-migration-statements.png)

After executing the migration statement, check the new subgraph.

![graph-migration-result](../../../images/graph-migration-result.png)

To achieve cross-TuGraph service data migration, you can first copy the target data directory to the current TuGraph server. By executing the aforementioned migration steps, you can perform seamless data and model migration while the service remains operational.
1 change: 1 addition & 0 deletions docs/en-US/source/3.quick-start/1.preparation.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ The TuGraph service can be started in two ways. The first method integrates the
# which you can modify according to your actual situation.
```

Please note that due to storage incompatibility between major versions, continuing to use the persistent directory from the old version's host during a cross-major version upgrade can cause the TuGraph service to fail to start. To avoid this issue, please change the path of the persistent directory during the upgrade. If you wish to use the previous data in the new version, please refer to the "Upgrade migration" section in the [Data Migration](../13.best-practices/3.data_migration.md). This issue does not affect initial installation and deployment.
4. Open by browser
Expand Down
4 changes: 2 additions & 2 deletions docs/en-US/source/6.utility-tools/2.data-export.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ $ lgraph_export -d {database_dir} -e {export_destination_dir} -g {graph_to_use}

Details:

- `-d {database_dir}` specifies the directory of the database from which the data will be exported. The default value is./testdb '.
- `-d {database_dir}` specifies the directory of the database from which the data will be exported. The default value is `/var/lib/lgraph/data`.
- `-e {export_destination_dir}` specifies the directory where the export file is stored. The default value is./exportdir.
- `-g {graph_to_use}` specifies the type of graph database. default is' default '.
- `-g {graph_to_use}` specifies the type of graph database. default is 'default '.
- `-u {username}` Specifies the name of the user who performs the export operation.
- `-p {password}` Specifies the password of the user who performs the export operation.
- `-s {field_separator}` specifies the separator for the exported file. The default is comma.
Expand Down
52 changes: 47 additions & 5 deletions docs/zh-CN/source/13.best-practices/3.data_migration.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
# 数据迁移

## 1. 简介
数据迁移是指将数据从一个系统、存储介质或应用程序迁移到另一个系统、存储介质或应用程序的过程。当TuGraph要升级或者系统硬件环境发生变化时,
需要对原TuGraph服务中的数据进行迁移。以系统硬件环境和软件版本为依据进行划分,本文将数据迁移分为三种方案
数据迁移是指将数据从一个系统、存储介质或应用程序迁移到另一个系统、存储介质或应用程序的过程。当TuGraph要升级或者系统硬件环境发生变化时,
需要对原TuGraph服务中的数据进行迁移。以系统硬件环境和软件版本为依据进行划分,本文将数据迁移分为四种方案
1. 兼容迁移:当迁移前后系统环境一致且TuGraph软件兼容时,可以直接使用备份恢复的方式迁移数据;
2. 升级迁移:当迁移前后系统环境不一致或TuGraph软件不兼容时,需要使用先导出数据再重新导入的方式迁移数据;
3. 在线迁移:当对高可用集群进行数据迁移且集群网络环境良好时,可以使用增删节点的的方式将原集群平滑切换到新集群。
接下来本文将详细介绍这三种方案。
3. 在线迁移:当对高可用集群进行数据迁移且集群网络环境良好时,可以使用增删节点的的方式将原集群平滑切换到新集群;
4. 图项目级数据迁移:包括在相同的TuGraph服务间进行子图数据迁移,以及在不同的TuGraph服务间进行子图数据迁移。

请注意,TuGraph软件的不兼容性主要体现在跨大版本时的存储方式不兼容。接下来本文将详细介绍这四种方案。

## 2. 兼容迁移
兼容迁移指的是在系统环境不变,且TuGraph软件版本兼容时,原服务的数据和存储过程可以在新服务中使用,所以可以直接迁移。
Expand Down Expand Up @@ -79,4 +81,44 @@
停止原节点服务,后续应用请求直接发给新集群
```bash
lgraph_server -c /usr/local/etc/lgraph_ha.json --directory db.cp --ha_conf 192.168.0.1:9090,192.168.0.2:9090,192.168.0.3:9090 -d stop
```
```
## 5. 图项目级数据迁移
图项目级数据迁移包括在相同的TuGraph服务间进行子图数据迁移,以及在不同的TuGraph服务间进行子图数据迁移。 对于同一TuGraph服务内的子图数据迁移,可以将当前服务中的图项目迁移到一个新的项目中,新项目与被迁移的项目在模型和节点、边数据上完全一致。 对于不同TuGraph服务间的子图数据迁移,可以将其他TuGraph服务中的图项目迁移到当前的TuGraph服务中,迁移前后的模型和数据同样一致。

### 5.1. TuGraph 数据目录说明
如下图,数据默认在 `/var/lib/lgraph/data` 目录下。在该路径下,每个以长字符串命名的目录代表一个特定的图项目的数据存储位置。进入到具体的长字符串目录中,可以看到 data.mdb 文件,这些文件存储了该图项目的具体模型和数据。由于图项目的数据存储目录当前不支持以项目名称命名,可以通过项目的创建时间来区分不同的项目。

![dataPath](../../../images/dataPath.png)

### 5.2. 子图间数据迁移

在任意图项目的图查询中执行如下语句进行迁移:

```
CALL db.importor.fullFileImportor("${新的图项目名称}","data.mdb文件",false)
```

参数说明:
1. 第一个参数是新子图名称。
2. 第二个参数是服务器上待迁移图项目的文件绝对路径。有关路径位置,请参考“TuGraph 数据目录说明”。
3. 第三个是文件是否是远程文件,如果是服务器的本地文件就设置为false或者不设置。

注意:
如果第一个参数指定的新子图名称已经存在于已创建的图项目中,则原有的子图数据将会被迁移过来的数据覆盖。如果子图不存在,则会创建一个新的子图。建议避免使用重名,以防止覆盖导致的数据丢失。

示例:

```
CALL db.importor.fullFileImportor("test","/var/lib/lgraph/data/9DF16A038435BBAAE36E53D1E09DED17/data.mdb",false)
```

在任意图项目的图查询中执行迁移语句:

![execute-data-migration-statements](../../../images/execute-data-migration-statements.png)

执行迁移语句后查看新子图

![graph-migration-result](../../../images/graph-migration-result.png)


要实现跨 TuGraph 服务的数据迁移,可以先将目标数据目录复制到当前的 TuGraph 服务器上。通过执行上述迁移步骤,在服务正常运行的情况下进行数据和模型无缝迁移。
3 changes: 2 additions & 1 deletion docs/zh-CN/source/3.quick-start/1.preparation.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,11 +95,12 @@ TuGraph 可以通过 Docker Image 快速安装,或者通过 rpm/deb 包本地
# /var/lib/lgraph/data是容器内的默认数据目录,/var/log/lgraph_log是容器内的默认日志目录
# 命令将数据目录和日志目录挂载到了宿主机的/root/tugraph/上进行持久化,您可以根据实际情况修改。
```
请注意,由于大版本之间存储不兼容,在进行跨大版本升级时,如果继续使用旧版本宿主机上的持久化目录,会导致TuGraph服务无法启动。为避免此问题,请在跨大版本升级时更改持久化目录的路径。如果希望在新版本中使用之前的数据,请参考[数据迁移](../13.best-practices/3.data_migration.md)中的"升级迁移"章节。首次安装和部署则不受此问题影响。

5. 前端访问

访问tugraph-db-browser: `http://x.x.x.x:7070`,数据库地址格式为 `bolt://ip:bolt_port`(老版本不用填),默认用户名为 `admin`,密码为 `73@TuGraph`
首次登录会默认跳转修改密码页面,请尽快修改默认密码避免安全风险。
首次登录会默认跳转修改密码页面,请尽快修改默认密码避免安全风险。

### 2.2.新旧前端说明
进入容器,可以通过修改配置文件"/usr/local/etc/lgraph.json"中的"web"参数来选择使用老版本或新版本的前端。对于老版本,可以将"web"的值设为"/usr/local/share/lgraph/resource";对于新版本,可以将"web"的值设为"/usr/local/share/lgraph/browser-resource"。完成配置文件的修改后,请执行命令 `docker restart tugraph` 以使更改生效。需要注意的是,新版本是默认选项。
Expand Down
2 changes: 1 addition & 1 deletion docs/zh-CN/source/6.utility-tools/2.data-export.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ $ lgraph_export -d {database_dir} -e {export_destination_dir} -g {graph_to_use}

其中:

- `-d {database_dir}` 指定需要进行数据导出的数据库所在目录,默认值为 `./testdb`
- `-d {database_dir}` 指定需要进行数据导出的数据库所在目录,默认值为 `/var/lib/lgraph/data`
- `-e {export_destination_dir}` 指定导出文件存放的目录,默认值为 `./exportdir`
- `-g {graph_to_use}` 指定图数据库的种类,默认为 `default`
- `-u {username}` 指定进行该导出操作的用户的用户名。
Expand Down

0 comments on commit 5771c05

Please sign in to comment.