Optimization question

[krll-kov]

Hi. I wonder if it's a good idea to .lookupFunction<>() when you call it.

For example, we have a GetAsyncKeyState function

and a while loop.

while(GetAsyncKeyState(45 & 0x8000)) {
  await Future.delayed(Duration(milliseconds: 1));
  // do something
  }

So now each iteration of a while loop we call a GetAsyncKeyState function, which itself tries to lookup for a function in user32.dll library. Isn't it better to store a lookup result somewhere outside the function so that we don't look for it every time? Firstly i thought that because of a final keyword it stays the same, but then i checked the result of lookupFunction and each time variable recreates itself with new lookup result. Or maybe i'm wrong and it's working as needed. Can someone explain this moment better, please?

[ilopX]

Hi @kirill-21!
Here is my benchmark.
Optimization works, 💪 but only slightly.

My results

Benchmark code

import 'dart:ffi';

import 'package:win32/win32.dart';

void main() {
  final benchmarks = runAllBenchmarks(passes: 100000);
  printAll(benchmarks);
}

List<Benchmark> runAllBenchmarks({required int passes}) {
  final defaultFuncName = 'GetAsyncKeyState()';
  final memorizedFuncName = 'MemorizedGetAsyncKeyState()';
  final key = 45 & 0x8000;

  return [
    Benchmark(defaultFuncName, 'load & first call', () {
      GetAsyncKeyState(key);
    }),
    Benchmark(defaultFuncName, 'second call', () {
      GetAsyncKeyState(key);
    }),
    Benchmark(defaultFuncName, 'third call', () {
      GetAsyncKeyState(key);
    }),
    Benchmark(memorizedFuncName, 'load', () {
      loadOptimizedGetAsyncKeyState();
    }),
    Benchmark(memorizedFuncName, 'first call', () {
      MemorizedGetAsyncKeyState(key);
    }),
    Benchmark(memorizedFuncName, 'second call', () {
      MemorizedGetAsyncKeyState(key);
    }),
    Benchmark(memorizedFuncName, 'third call', () {
      MemorizedGetAsyncKeyState(key);
    }),
    Benchmark(defaultFuncName, 'calls($passes)', () {
      for (var i = 0; i < passes; i++) {
        GetAsyncKeyState(key);
      }
    }),
    Benchmark(memorizedFuncName, 'calls($passes)', () {
      for (var i = 0; i < passes; i++) {
        MemorizedGetAsyncKeyState(key);
      }
    }),
  ];
}

void printAll(List<Benchmark> benchmarks) {
  for (var i = 0; i < benchmarks.length; i++) {
    final isLastBenchmark = i == benchmarks.length - 1;

    final benchmarkText = benchmarks[i].toPrintableString(
      renderBiceps: isLastBenchmark,
    );
    print(benchmarkText);

    if (i == 2 || i == 6) {
      print('');
    }
  }
}

late DynamicLibrary _user32;
late int Function(int vKey) _memorizedAsyncKeyState;

void loadOptimizedGetAsyncKeyState() {
  _user32 = DynamicLibrary.open('user32.dll');
  _memorizedAsyncKeyState = _user32.lookupFunction<Int16 Function(Int32 vKey),
      int Function(int vKey)>('GetAsyncKeyState');
}

int MemorizedGetAsyncKeyState(int vKey) {
  return _memorizedAsyncKeyState(vKey);
}

class Benchmark {
  final String name;
  final String effect;
  late final double timeMs;

  Benchmark(
    this.name,
    this.effect,
    void Function() func,
  ) {
    final stopwatch = Stopwatch()..start();
    func();
    stopwatch.stop();
    timeMs = (stopwatch.elapsedTicks / 10000);
  }

  String toPrintableString({bool renderBiceps = false}) {
    final fName = name.padRight(32);
    final fEffect = effect.padLeft(18);
    final fTimeMs = timeMs.toStringAsFixed(4).padLeft(12);
    final biceps = renderBiceps ? _greenText('💪') : '';
    return '$fName$fEffect${_blueText(fTimeMs)} ms $biceps';
  }

  String _blueText(String text) {
    return '\x1B[34m$text\x1B[0m';
  }

  String _greenText(String text) {
    return '\x1B[32m$text\x1B[0m';
  }
}

[krll-kov]

Thank you. Well, it's 17 times faster so the way to lookup for a function outside seems much better. I think if you use much more complex winapi the difference is going to be significant

[timsneath]

It's a really interesting conundrum, and one I considered early on. Caching the lookup is of course possible, but calling some load() function as in the benchmark is painful. But looking at ffigen, it uses the late keyword for lazy initialization of the lookupFunction.

This would turn:

int GetAsyncKeyState(int vKey) {
  final _GetAsyncKeyState = _user32.lookupFunction<Int16 Function(Int32 vKey),
      int Function(int vKey)>('GetAsyncKeyState');
  return _GetAsyncKeyState(vKey);
}

into something like:

int GetAsyncKeyState(int vKey) => _getAsyncKeyState(vKey);

late final _getAsyncKeyState =
    _user32.lookupFunction<Int16 Function(Int32 vKey), int Function(int vKey)>(
        'GetAsyncKeyState');

I wonder if that would be as fast as the memoized version in @ilopX's benchmark, but without the need to explicitly cache? @ilopX, would you be willing to try this out and see how it compares in your benchmark?

[ilopX]

Yes, that's an interesting idea. Late memoized getter works greats

dart benckmark.dart

benchmark.exe

code

import 'dart:ffi';

import 'package:win32/win32.dart';

void main() {
  final benchmarks = runAllBenchmarks(passes: 100000);
  printAll(benchmarks);
}

List<Benchmark> runAllBenchmarks({required int passes}) {
  final defaultFuncName = 'GetAsyncKeyState()';
  final memoizedFuncName = 'MemoizedGetAsyncKeyState()';
  final lateMemoizedFuncName = 'LateMemoizedGetAsyncKeyState()';
  final key = 45 & 0x8000;

  return [
    Benchmark(defaultFuncName, 'first load & first call', () {
      GetAsyncKeyState(key);
    }, '⌛\n'),

    Benchmark(defaultFuncName, 'second call', () {
      GetAsyncKeyState(key);
    }),
    Benchmark(defaultFuncName, 'third call', () {
      GetAsyncKeyState(key);
    }, '\n'),

    Benchmark(memoizedFuncName, 'load only', () {
      loadOptimizedGetAsyncKeyState();
    }, '⌛'),

    Benchmark(memoizedFuncName, 'first call', () {
      MemoizedGetAsyncKeyState(key);
    }),
    Benchmark(memoizedFuncName, 'second call', () {
      MemoizedGetAsyncKeyState(key);
    }),
    Benchmark(memoizedFuncName, 'third call', () {
      MemoizedGetAsyncKeyState(key);
    }, '\n'),

    Benchmark(lateMemoizedFuncName, 'load & first call', () {
      LateMemoizedGetAsyncKeyState(key);
    }, '⌛ 🕘'),
    Benchmark(lateMemoizedFuncName, 'second call', () {
      LateMemoizedGetAsyncKeyState(key);
    }),
    Benchmark(lateMemoizedFuncName, 'third call', () {
      LateMemoizedGetAsyncKeyState(key);
    }, '\n'),

    Benchmark(defaultFuncName, 'calls($passes)', () {
      for (var i = 0; i < passes; i++) {
        GetAsyncKeyState(key);
      }
    }),
    Benchmark(memoizedFuncName, 'calls($passes)', () {
      for (var i = 0; i < passes; i++) {
        MemoizedGetAsyncKeyState(key);
      }
    }, '💪'),
    Benchmark(lateMemoizedFuncName, 'calls($passes)', () {
      for (var i = 0; i < passes; i++) {
        LateMemoizedGetAsyncKeyState(key);
      }
    }, '💪'),
  ];
}

/// momoized
DynamicLibrary _user32 = DynamicLibrary.open('user32.dll');
late int Function(int vKey) _memorizedAsyncKeyState;

void loadOptimizedGetAsyncKeyState() {
  _memorizedAsyncKeyState = _user32.lookupFunction<Int16 Function(Int32 vKey),
      int Function(int vKey)>('GetAsyncKeyState');
}

int MemoizedGetAsyncKeyState(int vKey) {
  return _memorizedAsyncKeyState(vKey);
}

/// late momoized
DynamicLibrary _user32_2 = DynamicLibrary.open('user32.dll');
int LateMemoizedGetAsyncKeyState(int vKey) => _getAsyncKeyState(vKey);

late final _getAsyncKeyState =
_user32_2.lookupFunction<Int16 Function(Int32 vKey), int Function(int vKey)>(
    'GetAsyncKeyState');

void printAll(List<Benchmark> benchmarks) {
  for (var i = 0; i < benchmarks.length; i++) {
    final isLastBenchmark = i == benchmarks.length - 1;

    final benchmarkText = benchmarks[i].toPrintableString(
      renderBiceps: isLastBenchmark,
    );
    print(benchmarkText);
  }
}

class Benchmark {
  final String name;
  final String effect;
  late final double timeMs;
  final String emojiIcon;
  Benchmark(
      this.name,
      this.effect,
      void Function() func,
      [this.emojiIcon = '']
      ) {
    final stopwatch = Stopwatch()..start();
    func();
    stopwatch.stop();
    timeMs = (stopwatch.elapsedTicks / 10000);
  }

  String toPrintableString({bool renderBiceps = false}) {
    final fName = name.padRight(32);
    final fEffect = effect.padLeft(25);
    final fTimeMs = timeMs.toStringAsFixed(4).padLeft(12);
    return '$fName'
        '$fEffect'
        '${TextColor.blueText(fTimeMs)} ms '
        '${TextColor.greenText(emojiIcon)}';
  }
}

class TextColor {
  static String blueText(String text) {
    return '\x1B[34m$text\x1B[0m';
  }

  static String greenText(String text) {
    return '\x1B[32m$text\x1B[0m';
  }
}

[timsneath]

Great -- that's sold me. Thank you so much for taking the time to benchmark this, @ilopX!

@kirill-21 -- thank you for the grit of sand in the oyster shell and for bringing this insight to the surface.

I'll prepare a patch that uses late memoization based on this thread.