add support for Tracy profiler

[dazKind]

Problem Statement

As a Haxe developer I want to be able to reliably profile the performance of my engine & hxcpp apps in a ergonomic, lightweight and battletested manner.

I have looked at HxTelemetry and HxScout and found them lacking in usability and support. The app is constantly crashing for me, like 10 years old and not officially opensource? Also I question the use of the AMF protocol these days.

Solution

This PR adds support for Tracy Profiler

Tracy is a real time, nanosecond resolution, remote telemetry, hybrid frame and sampling profiler for games and other applications.

Anyway, we looked around and we found the Tracy Profiler which seems to satisfy my need perfectly. While I'm happy to run & maintain my own fork for my engine, I think this is something the whole community can benefit from, this is why I'm opening a PR for review & discussion.

Personally this has already helped me in several projects exposing bugs and optimizing a heavily threaded hxcpp app.

How to use

To activate the tracy integration, compile your app with:

-D HXCPP_TRACY
-D HXCPP_TELEMETRY
-D HXCPP_STACK_TRACE
-D HXCPP_STACK_LINE

and use the following call at the very end of your mainloop, so it gets called every frame:

cpp.vm.tracy.TracyProfiler.frameMark();

Note: See HaxeFoundation/haxe#11772 or copy that file containing the functions into your project

Then start either Tracy's UI-App or cmdline server listening on localhost and start your hxcpp-made binary.

Some notes about the integration

Haxe-Code

We integrate Tracy into hxcpp as a Telemetry option which utilizes hx::StackPosition and offer a set of static functions to set zones and other tracy functionality. Through this all your haxe-code will be captured in a profiler-session.

There are however native parts of hxcpp that wont be visible by default in Tracy (bc there are no ZoneScopes).

Note: Exceptions are in a few spots in the GC-Code, so GC becomes visible for us.

Note: Hxcpp's native calls will become visible if you use the option to capture callstacks.

Note: We capture source-locations and their filepaths. By default these are relative to your project and thus the sourcecode preview / browsing in Tracy wont work since it expects absolute paths. To solve this you can use -D absolute-path in your builds.

externs

The same is true about externs you might be using in your project. If you want to make these visible, you need to @:include('hx/TelemetryTracy.h') and you gain access to Tracy's C-Macros that you can use in your extern's c/cpp-code. Please refer to the official Tracy documentation: https://github.com/wolfpld/tracy/releases/latest/download/tracy.pdf

externs with static/dynamic libs

Another special case are static or dynamic libs your externs might be using. For these you will have to make more changes that are beyond the scope of this doc here, please refer to Tracy's official documentation over here: https://github.com/wolfpld/tracy/releases/latest/download/tracy.pdf

Optional Features

Memory Profiling

The following define adds tracking (de-)allocations of hxcpp's small & large object heap.

-D HXCPP_TRACY_MEMORY

Capture Callstacks

By default we only track zones. If you wanna inspect the actual callstack per zone, you should use the following define:

-D HXCPP_TRACY_INCLUDE_CALLSTACKS

Note: This will inflate the telemetry data A LOT and cost more performance. Please be aware.

On Demand Profiling

By default this integration will start sampling & collecting telemetry with the start of your application. You can change this behavior by the following define and your app will only generate telemetry if the Tracy Profiler app is open and reachable.

-D HXCPP_TRACY_ON_DEMAND

Short-lived Application Support

In cases where you dont have a mainloop or a very short-lived application you can use the following define to let your application stay around to complete sending telemetry data it has collected.

-D HXCPP_TRACY_NO_EXIT

Questions relevant for hxcpp & this PR

• Licenses? Are they cool for hxcpp:
  • Tracy Client Code: https://github.com/wolfpld/tracy/blob/master/LICENSE
  • libbacktrace: https://github.com/wolfpld/tracy/blob/master/public/libbacktrace/LICENSE

[dazKind]

Ok, seems like 1426a88 fixes the pipelines. That also means that the tracy include has sideeffects for mysql on macos. Interesting.

[Aidan63]

The tracy source files should be moved to project/thirdparty as thats where all the other vendored libraries live. Would also be good to not need the hxcpp.h manually added to TracyClient.cpp to ease updating later on.

Now that we've got a better idea of what we need to do I wonder if a better approach is to make a tracy implementation of the hxcpp telemetry api. I.e. Have a TracyTelemetry.cpp which implements all the tlm and __hxcpp_hxt prefixed functions. tlmSampleEnter, for example, is called whenever a new stack frame is created so that can be used to issue a tracy call.
Many of those functions may just be no-ops for now, but the allocation ones will be useful once I've figured out allocation tracking.

Also wonder if we can do something similar for the hxcpp profiler api, might be able to provide a tracy implementation of that for custom zones, advantage being cpp.Profiler is already in the std library.

[Aidan63]

Another plus of a dedicated TracyTelemetry.cpp file is then only that needs to include tracy headers, so we dodge that mac sql issue you mentioned entirely.

[dazKind]

Ok, we updated the setup.

• Sources have been moved to project/thirdparty
• little tweaks to the build.xmls
• things are refactored into TracyTelemetry.cpp and useful tracy functions are exposed as globals now.

Updating the description&docs here now.

There are few more things left to be done concerning memory, but we are discovering more and more cool things that this integration allows, e.g. like source-preview:

[dazKind]

Ok, things have settled and we can move out of draft now.

Major kudos for @Aidan63 who was instrumental in making this a reality and taking it way beyond the initial scope in a relatively short amount of time!

Working on this has been a blast!

[BNTFryingPan]

i used hxscout once forever ago and had a hard time figuring it out, so this looks like it would be a big improvement

[Vortex2Oblivion]

The hxscout killer?

[EliteMasterEric]

Tested this with Funkin' and it worked brilliantly with minimal setup.

Note: This works on Haxe 4.3.6 also, as long as the cpp.vm.tracy.TraceyProfiler class from the linked project is included somewhere in the workspace (you don't need to use a fork of Haxe).

[Simn]

I don't really feel qualified to review this properly, but if @Aidan63 says it's good then I'm happy to merge it. I've already merged HaxeFoundation/haxe#11772, is there anything else we should do before merging here?

[dazKind]

Good morning,

I've already merged HaxeFoundation/haxe#11772, is there anything else we should do before merging here?

Nothing else required.

[barisyild]

Tracy profiler is increasing the memory usage of my openfl application from megabytes to gigabytes.

I think there might be a memory leak somewhere.

[dazKind]

Tracy profiler is increasing the memory usage of my openfl application from megabytes to gigabytes.
I think there might be a memory leak somewhere.

No, this is actually expected. Make sure you have either the UI or the cmdline capturer active & connected to your app.

By default your app will start collecting telemetry data from its startup till it receives an active connection to either the Tracy UI application or one of the shell loggers and then it will transmit & release all that data.

If you only want to capture telemetry data ONLY when a connections is active, use the mentioned compile flag mentioned in On Demand Profiling above.

I can only recommend you study documentation here: https://github.com/wolfpld/tracy/releases/latest/download/tracy.pdf

[barisyild]

Tracy profiler is increasing the memory usage of my openfl application from megabytes to gigabytes.

I think there might be a memory leak somewhere.

No, this is actually expected. Make sure you have either the UI or the cmdline capturer active & connected to your app.

By default your app will start collecting telemetry data from its startup till it receives an active connection to either the Tracy UI application or one of the shell loggers and then it will transmit & release all that data.

If you only want to capture telemetry data ONLY when a connections is active, use the mentioned compile flag mentioned in On Demand Profiling above.

I can only recommend you study documentation here: https://github.com/wolfpld/tracy/releases/latest/download/tracy.pdf

I'm already trying it this way, please apply the change I made in OpenFL and run the example.

openfl/openfl#2745

[dazKind]

I have no idea what example you are talking about or why you think this is problematic behavior. You reported that there might be a leak. I'm sorry I lack context to understand what brings you to that conclusion.

Do you crash or what is it exactly that you consider a problem there? If you activate Tracy there is a lot of data generated and you need a computer that is able to transfer and crunch all these numbers.

If you cant provide some more background information and/or a minimal example that reproduces what you consider an issue I cant help you.