Skip to content

Commit

Permalink
Refine docs
Browse files Browse the repository at this point in the history
  • Loading branch information
cjdsellers committed Jan 15, 2025
1 parent edf5df5 commit e284162
Show file tree
Hide file tree
Showing 3 changed files with 29 additions and 21 deletions.
28 changes: 16 additions & 12 deletions docs/concepts/data.md
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ NautilusTrader defines an internal data format specified in the `nautilus_model`
These models are serialized into Arrow record batches and written to Parquet files.
Nautilus backtesting is most efficient when using these Nautilus-format Parquet files.

However, migrating the data model between schema changes or to custom schemas can be challenging.
However, migrating the data model between [precision modes](../getting_started/installation.md#precision-mode) and schema changes can be challenging.
This guide explains how to handle data migrations using our utility tools.

### Migration tools
Expand Down Expand Up @@ -380,34 +380,38 @@ Converts JSON back to Parquet format:

### Migration Process

The following example migration process uses trades data.
The following example migration process uses trades data. All commands should be run from
the root of the `nautilus_persistence` crate directory.

1. Convert from old schema to JSON:
**1. Convert from old schema to JSON**:
```bash
cargo run --bin to_json trades.parquet
# Creates trades.json and trades.metadata.json
```

2. Switch to new schema version:
This will create `trades.json` and `trades.metadata.json` files.

**2. Switch to new schema version**:
```bash
git checkout <new-version>
```

or change precision mode:
```bash
export HIGH_PRECISION=true
make install
# Build nautilus_trader package with high-precision mode enabled
make install-debug
```

3. Convert back to Parquet
This will build the `nautilus_trader` package with **high-precision** mode enabled.

**3. Convert back to Parquet**:
```bash
cargo run --bin to_parquet trades.json
# Creates trades.parquet with new schema
```

This will creates a `trades.parquet` file with new schema.

### Best Practices

- Always test migrations with a small dataset first
- Maintain backups of original files
- Verify data integrity after migration
- Always test migrations with a small dataset first.
- Maintain backups of original files.
- Verify data integrity after migration.
4 changes: 2 additions & 2 deletions docs/concepts/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ The following value types are backed by either 128-bit or 64-bit raw integer val

### High-precision mode

When the `high-precision` feature flag is *enabled* (default), values use the specification:
When the `high-precision` feature flag is **enabled** (default), values use the specification:

| Type | Raw backing | Max precision | Min value | Max value |
|:-------------|:------------|:--------------|:--------------------|:-------------------|
Expand All @@ -230,7 +230,7 @@ When the `high-precision` feature flag is *enabled* (default), values use the sp

### Low-precision mode

When the `high-precision` feature flag is *disabled*, values use the specification:
When the `high-precision` feature flag is **disabled**, values use the specification:

| Type | Raw backing | Max precision | Min value | Max value |
|:-------------|:------------|:--------------|:--------------------|:-------------------|
Expand Down
18 changes: 11 additions & 7 deletions docs/getting_started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,19 +226,23 @@ Performance benchmarks comparing the modes are pending.
The precision mode is controlled by:
- The `HIGH_PRECISION` environment variable during compilation
- The corresponding Rust feature flag `high-precision`
- The `HIGH_PRECISION` environment variable during compilation.
- The corresponding Rust feature flag `high-precision`.
#### High-precision mode (default)
```bash
# High-precision mode (default)
export HIGH_PRECISION=true
make install
make install-debug
```
#### Low-precision mode
# Low-precision mode
```bash
export HIGH_PRECISION=false
make install
make install-debug
```
info:::
:::info
See the [Value Types](../concepts/overview.md#value-types) guide for more details.
:::

0 comments on commit e284162

Please sign in to comment.