Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use static linking on Linux and MacOS, dynamic on Windows #81

Merged
merged 12 commits into from
Jan 21, 2025
5 changes: 0 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -123,8 +123,3 @@ jobs:
nimble --version
nimble install -y
nimble test

# static linking is not supported on windows
if [[ "${{ matrix.target.os }}" != "windows" ]]; then
nimble test_static
fi
56 changes: 38 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Nim-RocksDB
# Nim-RocksDb

[![Build Status (Travis)](https://img.shields.io/travis/status-im/nim-rocksdb/master.svg?label=Linux%20/%20macOS "Linux/macOS build status (Travis)")](https://travis-ci.org/status-im/nim-rocksdb)
[![Windows build status (Appveyor)](https://img.shields.io/appveyor/ci/nimbus/nim-rocksdb/master.svg?label=Windows "Windows build status (Appveyor)")](https://ci.appveyor.com/project/nimbus/nim-rocksdb)
Expand All @@ -10,48 +10,67 @@ A Nim wrapper for [Facebook's RocksDB](https://github.com/facebook/rocksdb), a p

## Current status

Nim-RocksDB provides a wrapper for the low-level functions in the librocksdb c library.
Nim-RocksDb provides a wrapper for the low-level functions in the librocksdb c
library.

## Installation

Nim-RocksDB requires Nim and the Nimble package manager. For Windows you will need Visual Studio 2015 Update 3 or greater with the English language pack.
Nim-RocksDb requires Nim and the Nimble package manager. For Windows you will
need Visual Studio 2015 Update 3 or greater with the English language pack.

To get started run:
```
nimble install
nimble install rocksdb
```

This will download and install the RocksDB dynamic libraries for your platform and copy them into the `build/` directory of the project. When including this library in your application you may want to copy these libraries into another location or set the LD_LIBRARY_PATH environment variable (DYLD_LIBRARY_PATH on MacOS, PATH on Windows) to include the `build/` directory so that your application can find them on startup.
This will download and install the RocksDB libraries for your platform and copy
them into the `build/` directory of the project. On Linux and MacOS only static
linking to the RocksDb libraries is supported and on Windows only dynamic linking
is supported.

Alternatively you can use the `rocksdb_static_linking` flag to statically link the library into your application.
On Windows you may want to copy the dll into another location or set your PATH
to include the `build/` directory so that your application can find the dll on
startup.

### Static linking
### Compression libraries

To build the RocksDB static libraries run:
```
./scripts/build_static_deps.sh
```
RocksDb supports using a number of compression libraries. This library builds
and only supports the following compression libraries:
- lz4
- zstd

On Linux and MacOS these libraries are staticly linked into the final binary
along with the RocksDb static library. On Windows they are staticly linked into
the RocksDb dll.


### Static linking

To statically link RocksDB, you would do something like:
On Linux and MacOS your Nim program will need to use the C++ linker profile
because RocksDb is a C++ library. For example:

```
nim c -d:rocksdb_static_linking --threads:on your_program.nim
when defined(macosx):
switch("clang.linkerexe", "clang++")
when defined(linux):
switch("gcc.linkerexe", "g++")
```

See the config.nims file which contains the static linking configuration which is switched on with the `rocksdb_static_linking` flag. Note that static linking is currently not supported on windows.
Note that static linking is currently not supported on windows.

## Usage

See [simple_example](examples/simple_example.nim)

### Contribution

Any contribution intentionally submitted for inclusion in the work by you shall be dual licensed as above, without any
additional terms or conditions.
Any contribution intentionally submitted for inclusion in the work by you shall
be dual licensed as above, without any additional terms or conditions.

## Versioning

The library generally follows the upstream RocksDb version number, adding one more number for tracking changes to the Nim wrapper itself.
The library generally follows the upstream RocksDb version number, adding one
more number for tracking changes to the Nim wrapper itself.

## License

Expand All @@ -65,7 +84,8 @@ or

* Apache License, Version 2.0, ([LICENSE-APACHEv2](LICENSE-APACHEv2) or http://www.apache.org/licenses/LICENSE-2.0)

at your option. This file may not be copied, modified, or distributed except according to those terms.
at your option. This file may not be copied, modified, or distributed except
according to those terms.

### Dependency License

Expand Down
8 changes: 2 additions & 6 deletions config.nims
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# nim-rocksdb
# Copyright (c) 2019-2023 Status Research & Development GmbH
# Copyright (c) 2019-2025 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
Expand All @@ -12,17 +12,13 @@ when fileExists("nimble.paths"):
include "nimble.paths"
# end Nimble config

when defined(rocksdb_static_linking):
when not defined(rocksdb_dynamic_linking) and not defined(windows):
# use the C++ linker profile because it's a C++ library
when defined(macosx):
switch("clang.linkerexe", "clang++")
else:
switch("gcc.linkerexe", "g++")

switch("dynlibOverride", "rocksdb")
switch("dynlibOverride", "lz4")
switch("dynlibOverride", "zstd")

--styleCheck:
usages
--styleCheck:
Expand Down
39 changes: 13 additions & 26 deletions rocksdb.nimble
Original file line number Diff line number Diff line change
Expand Up @@ -11,35 +11,22 @@ installDirs = @["build"]
### Dependencies
requires "nim >= 2.0", "results", "tempfile", "unittest2"

template build() =
when defined(windows):
exec ".\\scripts\\build_dlls_windows.bat"
else:
exec "scripts/build_static_deps.sh"

before install:
build()

task format, "Format nim code using nph":
exec "nimble install [email protected]"
exec "nph ."

task clean, "Remove temporary files":
exec "rm -rf build"
exec "make -C vendor/rocksdb clean"

task test, "Run tests":
let runTests = "nim c -d:nimDebugDlOpen -r --threads:on tests/test_all.nim"
when defined(linux):
exec "export LD_LIBRARY_PATH=build; " & runTests
when defined(macosx):
exec "export DYLD_LIBRARY_PATH=build; " & runTests
build()
when defined(windows):
exec runTests

task test_static, "Run tests after static linking dependencies":
when defined(windows):
echo "Static linking is not supported on windows"
quit(1)

exec "scripts/build_static_deps.sh"
exec "nim c -d:rocksdb_static_linking -r --threads:on tests/test_all.nim"

before install:
when defined(linux):
exec "scripts/build_shared_deps_linux.sh"
when defined(macosx):
exec "scripts/build_shared_deps_osx.sh"
when defined(windows):
exec ".\\scripts\\build_dlls_windows.bat"
exec "nim c -d:nimDebugDlOpen -r --threads:on tests/test_all.nim"
else:
exec "nim c -r --threads:on tests/test_all.nim"
24 changes: 15 additions & 9 deletions rocksdb/lib/librocksdb.nim
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2018-2024 Status Research & Development GmbH
# Copyright 2018-2025 Status Research & Development GmbH
# Licensed under either of
#
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
Expand All @@ -22,6 +22,8 @@

## This file exposes the low-level C API of RocksDB

import std/[os, strutils]

{.push raises: [].}

type
Expand Down Expand Up @@ -87,23 +89,27 @@ type
rocksdb_memory_consumers_t* = object
rocksdb_wait_for_compact_options_t* = object

## DB operations

when defined(rocksdb_static_linking):
{.pragma: importrocks, importc, cdecl.}
when defined(windows):
const librocksdb = "librocksdb.dll"
elif defined(macosx):
const librocksdb = "librocksdb.dylib"
else:
const librocksdb = "librocksdb.so"

import std/[os, strutils]
when defined(rocksdb_dynamic_linking) or defined(windows):
{.push importc, cdecl, dynlib: librocksdb.}
else:
const
topLevelPath = currentSourcePath.parentDir().parentDir().parentDir()
libsDir = topLevelPath.replace('\\', '/') & "/build/lib"
libsDir = topLevelPath.replace('\\', '/') & "/build/"

{.passl: libsDir & "/librocksdb.a".}
{.passl: libsDir & "/liblz4.a".}
{.passl: libsDir & "/libzstd.a".}

when defined(windows):
{.passl: "-lshlwapi -lrpcrt4".}
else:
{.pragma: importrocks, importc, cdecl, dynlib: librocksdb.}

{.push importc, cdecl.}

include ./rocksdb_gen.nim
8 changes: 0 additions & 8 deletions rocksdb/lib/rocksdb.h
Original file line number Diff line number Diff line change
Expand Up @@ -46,15 +46,7 @@

#ifdef C2NIM
# def ROCKSDB_LIBRARY_API
# dynlib librocksdb
# cdecl
# if defined(windows)
# define librocksdb "librocksdb.dll"
# elif defined(macosx)
# define librocksdb "librocksdb.dylib"
# else
# define librocksdb "librocksdb.so"
# endif
# mangle uint32_t uint32
# mangle uint16_t uint16
# mangle uint8_t uint8
Expand Down
Loading
Loading