Export to and import from CSV #135
Comments
|
The only "CSV flavour" that was meant - temporarily - as an exchange format was the "Sky Dolly CSV" flavour. You cannot "mix and match" different CSV flavours, as they are not compatible:
You get the idea. So while export as "Sky Dolly CSV" and then importing it again worked (almost without loss of data, apart from floating point rounding errors due to conversion to text and back to floating numbers) this format will be completely removed from the upcoming v0.17. The good news: already in the v0.16 you have a way superior format for exchange of data between Sky Dolly instances, and that format is called:
It is the exact same format like the Sky Dolly logbook (after all, it is called "Sky Dolly Logbook"), with the only difference that when exporting a flight (which may contain multiple aircraft, aka "formation flight") the exported logbook will only contain that one specific flight. (Locations from your current logbook are also not contained, not even the default ones). There is absolutely no loss of data, as the data is exported and imported 1:1. And even better: when you export a flight with a Sky Dolly version X and later you want to import it with a Sky Dolly version Y, where Y > X, the old exported logbook will be automatically migrated to the newer version Y. In fact, with the very same migration logic that is also applied to your actual logbook (an SQLite database really). That is why I abandoned the "Sky Dolly CSV" format completely. To summarise:
"Oh, but how can I edit my flight data? That was much easier with CSV!" - I hear you say. And you are correct that editing CSV data with Excel (and then import it again into Sky Dolly) is somewhat more intuitive. But fear not: as mentioned already the "logbook" is a standard SQLite database file which can be opened with any SQLite database browser, e.g. (open source) Sure, you need at least some basic knowledge about SQL ("structured query language"): https://www.sqlitetutorial.net/ But once you get the basics you will quickly realise that editing / transforming / insert more sample points using SQL is so much more powerful! The most important tables:
Likewise PRIMARY_FLIGHT_CONTROL, LIGHTS, HANDLE etc. contain the recorded flight controls (yoke, rudder, pedals, flaps, ...), all associated with the corresponding AIRCRAFT. Does that make sense? |
|
In fact, I once wrote a little "tutorial" ("workshop") here: I hope that gets you going. The tables might have slightly different columns theses days - but you should find your way around. And yes, while this "workshop" was meant for the actual logbook it applies 1:1 also to exported logbooks. |
|
Hi Oliver,
Absolutely! After realizing the impossibility to use CSV, I began investigating further, and stumbled upon your SQL Lite tutorial. I installed the DB Browser (SQLite), and could investigate the “Test flight.sdlog” database that I exported from my short flight. I have worked with MySQL for some time, years ago in my professional career (I’m retired now). So this is not entirely unfamiliar for me.
The goal I have is this: years ago, for FSX I made a tool that imports IGC files (I’m a glider pilot). I then make a huge number of calculations in Excel to deduct these parameters: Timestamp, Time, Latitude, Longitude, Altitude, Pitch, Bank, Heading, OnGround, Gear, Ailerons, Rudder, Elevators, Spoilers, Flaps, Date, for every second of the flight (sometimes over 8 hours per flight!). You see: far more parameters than present in an IGC file! Especially the data for attitude and airplane controls need to be accurately calculated to be able to faithfully simulate a glider, that does steep winch starts, banks up to 45° in thermals, can do loopings, dive-starts a sustainer motor, etc… The tool produced a CSV file, which was then converted to an FRC file, that was used by FS Recorder (by Matthias Neusinger) for playback in FSX. That way I could nicely simulate starting, flying and landing real glider flights, in the “crude” FSX landscapes…
I investigated a lot of possibilities to port this to MSFS, eventually I thought SkyDolly should be the tool to use. At first I thought: YES, it can import IGC files, so I can get rid of my Excel calculations. But alas, although SD is able to simulate a glider flight, it is not at all realistic as far as the glider’s attitude is concerned: it spirals in thermals with almost no banking, without responsive controls, no wheel out in and out animation, it often starts and lands above or under the ground, …. So I still need my Excel!
Now I will have to study of a way to convert my resulting Excel table directly to SQL, or first export to CSV and then convert to SQL… All this should run automatically after Excel has finished its calculations… Today the path to go still remains unclear for me. BUT, the reward in succeeding will be satisfying, I guess!... 😉
Thanks for your extensive reply, especially the fact that CSV support will not be continued, so I don’t need to spend more time on that…
2 questions:
1. Do you think it is possible to have up to 36000 records (with each 15 or so numbers) in an sdlog file? 36000 is 10 hours of flight with 1s logging interval…
2. Matthias Neusinger’s FS Recorder had the ability to insert rolling on-screen comments at user defined timestamps. That was nice to indicate motor start events, arrival at turnpoints, etc… Is there a remote possibility this can/could be done with SkyDolly?
Beste regards,
Hugo Jansen
|
|
Hello Hugo,
So you should find your way around the Sky Dolly "logbook" tables then :) I was never a big fan of databases during my studies as a software engineer ("data graves" I called them). Turned out I ended up being a PL/SQL developer for 12 years in my previous job, ha ha. To be fair, that really mostly involved writing "application code" - plus the occasional SQL statement, of course. Anyways, back to the topic...
You have not been, by any chance, invovled in the following discussion: https://www.fsdeveloper.com/forum/threads/smooth-ai-movement-the-old-favorite.17374/page-2 Because I stumbled over this old FSX/SimConnect related discussion a couple of months ago (or even a year by now) while desperately trying to find means to get "smooth movement", specifically for AI aircraft (also see this issue: #86). Turns out this is an old and well-known problem with SimConnect. The code developed by user "B21" is now in fact available here: https://xp-soaring.github.io/fsx/dev/sim_logger/index.html It uses "slew" with slew factors that have been heuristically calibrated with the desired speed (in feet / second). The problem with "slew": it disables all aircraft animations. Probably not a big issue for gliders (well, they have ailerons, too...), but I did not try out this solution just yet with MSFS... but the solution to get smooth AI aircraft movemements really seems to lie in applying velocity vectors instead of repeatedly ("as fast as possible") setting absolute world coordinates. But again, this is another issue...
Well, Sky Dolly is open source anyway, so I cannot hide it: the "flight augmentation code" is outright silly in its current form (deliberately, knowingly). I merely consider it a "temporary placeholder". In fact, the current bank angle "guesstimation" really just takes the angle between the current and next flight path line segment and does a linear interpolation of the bank angle - based on a maximum bank angle of 25 degrees and an assumed maximum turn angle of 90 degrees or so. And yes, that is based on the assumption that the aircraft would be an A320-kind of tube airliner ;) https://www.pprune.org/tech-log/377244-a320-321-ap-bank-angle-limits.html The "flight augmentation" implementation is here: https://github.com/till213/SkyDolly/blob/main/src/Flight/src/FlightAugmentation.cpp Other "hardcoded" assumptions: the gear raises and lowers at a fixed timestamp after and before the first and last timestamp (again, under the assumption that we have an imported flight from e.g. flightaware.com or flightradar24.com), and similar with the flaps. Refer e.g. to Again, all this was meant to be replaced by some "aircraft behaviour", configurable e.g. by some JSON setup file, with different profiles that could be assigned during import etc. As far as the bank angle calculation goes, I just did some research and I think I came across a formula that should give more realistic results, e.g. here: So all I really need to calculate - given the GPS path - is the velocity (easy: we usually have the timestamps for each sample point) and the turn radius (can be estimated by the current three sample points). Once we have those values we simply plug them into the formula and should have a good estimate of the required bank angle.
Yes, another known issue, also see #27 During my initial attempts someone in the MSFS forums suggested to use compensation for undulation: https://en.wikipedia.org/wiki/Geoid Especially for the IGC import case I was hopeful, as the altitude is (also) given in relation to the "WGS84 reference elipsoid". The undulation file that comes with Sky Dolly (and takes almost half of the installation size, see [Sky Dolly Directory]/Resources/geoids/egm2008-5.pgm - it is 18 MB in size. This is the smallest "undulation file" there is (*)) is supposed to give the difference of the reference altitude to the "magnetic altitude" (? the details escape me right now ;) ?) or the real altitude... uh... anyway, that compensation with those undulation values also did not properly align gliders imported via IGC format onto the surface. Either because most recorders still write "reference pressure altitudes" into their IGC files (according to the specs the reported altitude should really be "GPS altitudes", with WGS84 reference elipsoid) - or the earth model in MSFS is off my given or take a few metres anyway, due to floating point rounding errors etc. And CSV files made available via flightaware.com and flightradar24.com provide only altitude data with "standard pressure" (at least in the free version of it - one of them should also have "GPS altitudes", but only with a valid (paid) subscription). So what I really intend to do is to guesstimate the sample points that are supposed to be on ground, place the aircraft at each of those points and "force it to be on ground" (using SimConnect -> initPosition), record the actual altitude and overwrite the imported altitude with the "real" ground altitude. Of course I guess this needs to be "interactive", with some kind of wizard dialog, as we need to give MSFS time to actually load the scenery at all involved airports.
Just to be very clear about this, also in case others come across this issue here: ONLY the "Sky Dolly CSV" will be removed in the v0.17 - all other existing CSV import and export formats (namely "position & attitude", "flightradar24", "flight recorder") will remain unchanged (and other CSV flavours may even be added in the future). The "Sky Dolly CSV" format was in fact the very first possibility to store and restore flights - but it was my intention from the very beginning that I wanted to have a "proper persistence", such as SQLite. The data in Sky Dolly is "hierarchical" (Flight -> Aircraft -> Position | Engine | Lights | ...), so a "flat format" (like CSV) is never a natural fit for such data structures. Not to mention that with almost each iteration of Sky Dolly previous CSV files got invalid, as I kept adding and moving data fields... Finally as for your questions:
At least from Sky Dolly's point of view there is no real limit: all timestamps are 64bit (milliseconds), so while I did not do the math this should give you plenty of recording time ;) As for SQLite: I guess the file can easily grow above the 4 GB file size limit (that is just limit that comes to mind, from earlier days ;)) But you may see some performance degradations perhaps. And yes, at this point Sky Dolly first records everything into memory, and only at the very end a "commit" is done. I thought about a "ring buffer" implemenation, but that is at the bottom of the "backlog" currently (also see above issues that I want to tackle first). The good news: Sky Dolly organises the data into "groups", and only data that has actually changed gets recorded. So most of the time you will only record the data that goes into the POSITION table (and even this I intend to optimise even further).
So when you mean "text scrollers" in Sky Dolly itself: certainly, that would be doable. However the ability to show text in the flight simulator does not work anymore since a long time (and possibly never did in MSFS). The corresponding SimConnect function has been deprecated from the very beginning, if I remember correctly. |
|
Sorry, your previous reply somehow got marked as "duplicate comment": either that was me (and I don't know how to undo it), or it was automatically done by github (I tried to revert that, too, by removing the quoted part of mine). In any case, that was not intentional! UPDATE: "Edit / Unhide" did the trick ;) |
|
Oliver, that was me! I replied to your email, not knowing that reply would go public on GitHub, as the email return address said “noreply”, that’s why I assumed it. I’m completely new to GitHub (although I know of it a long time of course), I became a member only yesterday…
As I don’t want to start elaborate discussions about this subject now, I thought I would better hide it…
In the mean time, I’ve noticed you wrote a loooong (!) reply, I will answer that shortly!
Best regards,
Hugo
[Edited email addresses removed]
|
|
Hi Oliver,
[Editor's note: original parts from me properly quoted and shortened - also removed "private, casual converstation parts from hugojans' original reply, as they might not have been intended]
Whaw, I have to sit down now to digest all this information! I will reply inline below…
[Parts about programming background removed]
Your problem is slightly different from mine: you need two or more airplanes to execute the same patterns without jitter between them. In fact, I had a similar problem when I wanted to simulate a tug plane in front of the glider, but I solved that by offsetting the glider’s parameters so the tug always stays in front of the glider. I then introduced some random minor deviations to these parameters, so that the tug plane moves not exactly the same as the glider (which is doesn’t do in reality either). Mind you, all these calculations are done in Excel, not C++…
The code developed by user "B21" is now in fact available here:
https://xp-soaring.github.io/fsx/dev/sim_logger/index.html
Yes, now I remember I played with that a long time ago! Quite nice. But B21’s goal was different from mine: he wants to replay glider competitions, where the gliders are positioned in the sky by slew, and then they fly a racing task, if I remember well.
My goal was/is to simulate an entire glider flight, from standing still on the ground, the tow or winch start, releasing the cable, retracting the gear, pull up or dive as needed while flying cross-country, make nice thermal circles with the correct bank angle and nose pitch attitude, with all the control surfaces moving as would be expected (far greater movements than in a powered airplane), arriving at the downwind leg of the destination, extend the gear, make the two final turns towards the final, extend airbrakes, flare and land smoothly. Not easy, with only an IGC file as input! So I had to dive into the physics, and search for the correct formula for f.e. the bank angle, which depends on airspeed and circle diameter. I also had to assume a number of things, like when to retract and extend the gear and the flaps, as these are not recorded In the IGC file. Also, for gliders with a “turbo” (sustainer) motor, that have to be dive-started, careful examination of the location and speed is needed to determine when the motor is started and stopped. In Excel I maintained database tables for the starting and landing locations and their altitudes, in FSX the IGC locations more often than not did NOT correspond with the runways in FSX… Also, I had tables for every FSX glider used, were their relevant parameters were stored. And then finally, there was a table that united data from all these tables into the flight simulation for the current flight. An export module created the CSV file with all FSX parameters, in 1s interval, the FRC file for the replay tool and an .flt file and a .pln file for FSX, along with track and barogram PNG files…
All in all, a gigantic task, which I undertook (and underestimated!) when the corona-lockdowns forced us to stay at home, I worked more than 2 years on it… If you’re interested, I can show you some video fragments of such simulated flights (I hope I have them still lying around, otherwise I will need to produce them again)…
It uses "slew" with slew factors that have been heuristically calibrated with the desired speed (in feet / second).
But alas, although SD is able to simulate a glider flight, it is not at all realistic as far as the glider’s attitude is concerned
Well, Sky Dolly is open source anyway, so I cannot hide it: the "flight augmentation code" is outright silly in its current form (deliberately, knowingly). I merely consider it a "temporary placeholder"
Haha, a glider *routinely* banks at 40-45° in thermals, it can even do 60° in some circumstances (for example, abrupt avoidance turns when flying very close to a mountain ridge or in the vicinity of other gliders). That’s why it is important to do it with the correct calculations ( <https://www.omnicalculator.com/physics/bank-angle#in-the-sky-the-angle-of-banking-of-an-aircraft> Angle of Banking Calculator (omnicalculator.com), <https://aviation.stackexchange.com/questions/65628/what-is-the-formula-for-the-bank-angle-required-for-a-turn-in-line-abreast-forma> aircraft physics - What is the formula for the bank angle required for a turn in line-abreast formation? - Aviation Stack Exchange, ….)
And yes, that is based on the assumption that the aircraft would be an A320-kind of tube airliner ;)
https://www.pprune.org/tech-log/377244-a320-321-ap-bank-angle-limits.html
The "flight augmentation" implementation is here:
https://github.com/till213/SkyDolly/blob/main/src/Flight/src/FlightAugmentation.cpp
Wow, that’s quite some C++ code! It would take me a fair amount of time to study all this….
Other "hardcoded" assumptions: the gear raises and lowers at a fixed timestamp after and before the first and last timestamp
For the glider flight simulations, I made these assumptions, based on the real world (!): wheel retracted when the tow cable is released by the glider at the max. tow height (500 or 600m AGL), or when the cable (automatically or manually) detaches at the end of the winch tow. Wheel out at 200 to 250m AGL (above ground). Airbrakes 50% extended when entering final, retract when glider comes to a standstill after landing.
As far as the bank angle calculation goes, I just did some research and I think I came across a formula that should give more realistic results, e.g. here:
<https://aviation.stackexchange.com/questions/65628/what-is-the-formula-for-the-bank-angle-required-for-a-turn-in-line-abreast-forma> https://aviation.stackexchange.com/questions/65628/what-is-the-formula-for-the-bank-angle-required-for-a-turn-in-line-abreast-forma --> Ah yes, also found that just 2 minutes ago… 😉
So all I really need to calculate - given the GPS path - is the velocity (easy: we usually have the timestamps for each sample point) and the turn radius (can be estimated by the current three sample points).
Correct! BUT, there is a caveat! I inspected numerous IGC files, from all sorts of IGC recorders and different gliders, and I noticed that many IGC files have missing recordings!! Assuming modern recorders always record at 1s intervals, more than once the import algorithm finds interval of 2s, 3s, sometimes even 10s… VERY annoying, to say the least, and very visible in the simulation, as the glider then moves with great jerks… So one has the find these uncommon intervals and try to interpolate the missing 1s values for lat, lon, altitude, speed, …. One more hurdle to take! You can find hundreds of IGC files online, just pick a number of them and try… Older recorders had intervals of 5s, or even 10s, these flight are very difficult to simulate of course…
>it often starts and lands above or under the ground
So what I really intend to do is to guesstimate the sample points that are supposed to be on ground, place the aircraft at each of those points and "force it to be on ground" (using SimConnect -> initPosition), record the actual altitude and overwrite the imported altitude with the "real" ground altitude.
Interesting method you’re thinking about! What I did in FSX: I placed the glider-to-use on the correct spot (runway or grass location) in the simulator, I noted FSX lat/lon/alt, and calculated the offsets to the real IGC lat/lon/alt start parameters. Then I did the same for the landing location. Then I corrected all IGC locations in between linearly so that a smooth trajectory between start and end location is obtained. That’s why I created a table with start and landing locations, even on one airfield you can have multiple start and landing locations (on the concrete runway and on the grass next to the runway, and depending on start direction into the wind. Mostly landing point also is not equal to start point, and here you need distance after touchdown point to have the glider slowing down from landing speed (about 90-100 km/h) to 0 km/h.
The "Sky Dolly CSV" format was in fact the very first possibility to store and restore flights
SQL is indeed the best way to store all relevant data, for sure! (Just for me this is at this moment an extra hurdle…) I never fully liked working in Excel, but things evolved that way, and I’m not such an expert in C++ as you are…
At least from Sky Dolly's point of view there is no real limit: all timestamps are 64bit (milliseconds), so while I did not do the math this should give you plenty of recording time ;)
OK!
I thought about a "ring buffer" implemenation, but that is at the bottom of the "backlog" currently (also see above issues that I want to tackle first).
OK!
The good news: Sky Dolly organises the data into "groups", and only data that has actually changed gets recorded.
Nice idea, but I guess that a glider flight produces constantly changing data, this is not a tube liner…
So when you mean "text scrollers" in Sky Dolly itself: certainly, that would be doable. However the ability to show text in the flight simulator does not work anymore
Ah, that’s a pity, didn’t know that. In FSX it still works though…
Final remark: I will further investigate the matter from my side, but it will take some time!
[Personal comments removed]
|
Yes, replies to conversational emails - and apparently alo when github.com's email address sais "noreply" - go public on github (perhaps because there is another "reply-to" address given somewhere in the email header). As I usually check online on github.com and write my replies here (as this also allows to use "markdown" etc.) I did not know that the sender's address is "noreply" (the one from github.com). I will take the liberty and reformat your last reply here on github, in order that it is a bit clearer who wrote what initially. Unless you want this to be deleted - in case feel free to edit the parts you want see removed yourself (or the entire message). In fact, I was just about to refer to our discussion about the "flight augmentation" and "bank angle", because someone else just opened an issue about this topic ;) |
Just a small remark: altering tables (via DDL - data definition language - such as CREATE TABLE, DROP TABLE etc.) does an implicit "commit", at least with "proper databases" like Oracle (not sure about SQLite). That means your newly started transaction gets immediately committed and hence ends with the DROP TABLE statement (without having actually inserted or updated any data). With SQLite I think you have a default transaction anyway once you open a connection, so in your example SQL you still get to insert the desired values (that are then also committed with your actual "COMMIT" statement). So in your script I would put all DDL at the very beginning, that is drop and (re-)create all tables first, and then do one transaction only (that's the whole point of a transaction, that you insert all data completely, or not at all. Currently you could end up with only position values, but no "handle" (gears, flaps, ...) values. But that's just a general remark about SQL and not really related to the actual topic at hand ;)
Exactly ;)
As mentioned previously already depending on the specific simulation variable the minima / maxima are different: some are "position values" ranging from +/- 16k, others are "percent" ranging from 0 to 100 and yet others are "normalised percent" values going from 0.0 to 1.0. You need to either consult the SimConnect documentation (see link given above) or record a flight with Sky Dolly and move all elements to their extrema to see the minima / maxima values (e.g. with the SQLite DB Browser: "select * from table position where...", or with the built-in table browser etc.). But you seem to have figured that out already :) And Sky Dolly does something else as well: during replay it "freezes" the aircraft, so any simulation that would either position or attitude is stopped by MSFS (otherwise both applications would "fight" against each other). Also, MSFS seems to have two calculation modes: "on ground" and "airborne". E.g. the simulation of ailerons is different ("simplified") on ground than in the air.
Again, that's true: you also need to provide the velocity X/Y/Z values (feet per second - but again, check the code where it registers those simulation variables: I usually went with the default units, as suggested by the SimConnect documentation). And believe me, I have been there, too ("aircraft falling out of the sky after replay stops" ;)
You should be able to calculate the distance between two given position points: the Euclidean distance should be good enough for those small distances (Sky Dolly always calculates distances properly with the Haversine formula): (the later is actually calculated now by the 3rd-party GeographicLib - attribution where attribution is due ;)) Once you have the distance you also calculate the time delta given the two timestamps, and this gives you the required velocity. Just make sure you convert it to "feet per second" (conversion formulates are easily found "on the web" - or in the Sky Dolly source code: search for a source named "Conversion" or similar ;))
As far as replay is concerned you can completely ignore the "location" table: that one stores the pre-defined locations, as used by the "Location" module (for "teleportation" to that location: the indicated airspeed etc. is just used to "initialise" the aircraft, once teleported. Again, not relevant at all in the "replay" context.
The documented values should be correct, as far as I remember: what you could have experienced is perhaps that the "Simvar Watcher" example app used different units? But for "min/max" values that should not make a difference... Again, when in doubt simply record a flight with Sky Dolly, move the relevant control surfaces to their full extend and check the recorded values.
I certainly will - in about 4 days (am staying with a friend right now: small holidays in Lucerne ;)) |
|
Forgot to tell: spoilers_handle_percent in 'secondary_flight_control' also works correctly: 0 = airbrakes in, 100 = airbrakes full out. |
|
I have to apologize to having accused you to employ SLEW mode to move the aircraft, now I know this is NOT the case! In fact, I can make all controls visually operative if I multiply the rudder and elevator values I once used for the FSX / FSConvertor replays by about 2, and the aileron values by 3 (each time constrained to +/- 16383). Gives a quite realistic glider behaviour for the Asobo LS8 18m! Now I still need to get realistic horizontal and vertical speed indications (now 0). And the take-off and landing locations and attitudes are not yet correct. |
No worries ;) In fact, I already thought about using slew mode (at least for the AI aircraft in "formation flights", to avoid the dreaded "jitter" in their movement). But as you correctly say that also disables any flaps, gear etc. animations. But that was a solution that someone else applied for his replay tool back in the days with FSX, and it worked great as far as the movement goes. One can also set the "aircraft velocity" via the local aircraft coordinate system (e.g. Z-axis points "forward" etc.). That sounds great (and works, I already tried it with the sample "SimWatcher" application that comes with the MSFS SDK examples). The problem: I would have to change the aircraft attitude such that its nose points to the point where it should fly next. But if the aircraft is approaching a runway and its nose is "pointing upwards" when it should fly towards a point below it. Well, perhaps I could "offset" this "upward movement" by also setting the "down-vector velocity" accordingly. I have to re-visit basic vector school math to figure it all out ;) There are also "world coordinates" that - according to the SimConnect documentation - should be even "writeable". But they aren't. At least the aircraft does not move at all when setting them. They would be perfect, because all I would have to do is to calculate the difference (vector) of the current aircraft position and the target destination (according to the recorded flight path) and calculate the X, Y and Z velocities accordingly. But again, bummer: the aircraft does not move when setting those "world vector velocities"... Anyway, back to your questions :)
Yes, this sounds about right: 16383 (+/-) is indeed the maximum (minimum) value for many "simulation variables". The other thing you need to take into account is the unit, e.g. "feet per second" instead of the (perhaps expected) "meters per second". Sky Dolly needs to "register" those "simulation variables" of interest with MSFS, including the unit. I usually chose the "default unit" (as given in the SimConnect documentation respectively in the example code), e.g. here: That is, mostly "feet / second", "knots", "radians" etc.. The minima / maxima are documented e.g. here: As a heads-up: in the upcoming Sky Dolly v0.18 I will "split" the "Position" table into "Position" (latitude, longitude, altitude essentially) and into a new "Attitude" table (pitch, bank, roll angle and newly also a boolean "on ground"). The reason: I will lower the sample rate for the position down to 1 Hz (during recording only), while the attitude will still be sampled at the "simulation frame rate" (as before). My aim is to have a natural "low pass filter" for the position samples, that is if there is some "stutter" in the framerate that this gets naturally "smoothened out" during recording. And 1 Hz should still be plenty to give a "visually accurate flight path". The good news: your current "logbooks" will be automatically "migrated" once loaded into the new Sky Dolly version (that's also the reason why I dropped the CSV "Sky Dolly" format and introduced the "logbook" import/export format). But your INSERT statement will need to be "split" as well. I will update the example that I gave on the MSFS forum accordingly, and post a link here once done.
As you suspected in an earlier post already you also need to insert the corresponding velocity values. If I remember correctly you need to specify them as "feet per second". And no, Sky Dolly does not generate any values during replay (except for interpolation between sampled points) it simply sends back the recorded values as previously recorded. And yes, the velocity vectors make the velocity needle show the expected velocity (and wheels turn on the runway, otherwise they stand still). I hope the same is true for the vertical speed indication. |
|
OK, one thing I missed in the SkyDolly database is a description of which units you decided to use, it would be nice to have them somewhere (in the DB), because MSFS can use multiple units for a variable. Thanks for the link to your code, I should have looked that up... ;) I assumed the Velocity_X/Y/Z were in feet/sec, so I was right. I'm still working on calculating these, because before I sent all lat/lon/alt IGC values to FS Recorder (in FSX), that tool then calculated these velocities, now I have to do it myself. This involves some trigonometry and converting clockwise heading angles to trigonometry anti-clockwise angles by Alpha = MOD(450 - HDG, 360) (Excel formula). And yes, I already figured out that pitch, bank and heading should be given in degrees, not in radians as the SDK suggests... "On ground": I have put that in the 'location' table, but it doesn't work, my glider is still some feet above ground at the start and finish... Where can I set the initial attitude (especially pitch) and location/altitude of the glider? I guess 'altitude_above_ground' and 'start_on_ground' in the 'aircraft' table, but the initial pitch is not there, the LS8 always has a too much nose-up attitude...? One small recommendation: the table 'handle' should be better named 'handles', as it concerns multiple handles... Sky Dolly v0.18 I will "split" the "Position" table into "Position": no problem, I will adapt my insert statements accordingly, I'm still in the "study" phase, you know... ;) One other thing: when you select a glider (as the LS8) to replay, MSFS automatically assigns it a tow plane, a tow rope and a wingtip helper person. That's nice for normal flying: you waggle the tail and the tug starts to pull, the helper runs a while and then releases the wing. When high enough, the glider releases the tow rope with the key combo 'Ctrl+T'. But that's not what we want when we are going to replay a previously recorded glider flight! If we don't release the cable before replay, it stays attached to the nose of the glider and to the start position, obviously 'impossible' in the real world. I solve it now by pressing Ctrl+T before starting the replay. It would be nice that SkyDolly did that automatically, it knows a glider will be replayed. I searched the SDK, it seems there is no simvar for that, but there is the event TOW_PLANE_RELEASE (in AIRCRAFT MISCELLANEOUS EVENTS). There is a simvar TOW CONNECTION (true if a towline is connected to both tow plane and glider), but that is read-only. One small thing regarding the SkyDolly replay interface: you have a time slider, with at the right end the current time. But you don't have an indication of the TOTAL time of the recording. Would be nice to have that, also in the minimal view... All-in-all: I admire what you have accomplished with Sky Dolly, and I love to use it to convert my FSX tool to the MSFS world. Also, it's magical to be able to scroll through the replay with the time slider! I read your discussion about 'event based sampling' and 'cubic spline interpolation' methods, that's familiar stuff for me (as an electronics engineer), and that assures me this is the right tool for the job! Although the old FSX FS Recorder did similar things, Sky Dolly is far superior to that... My ultimate goal would be to get rid of my Excel conversions, but therefor I would need to also implement a SQL database and a lot of code to get the same result as with the Excel (that is now a lot of tables together with a lot of VB code...). |
Having the units stored in the database - together with the actual values - would only be useful if you could actually change them in the application. But that is not the case. The units are "hard-coded" in the code that "registers" the simulation variables with MSFS (whenever a connection is made with MSFS). So having the units stored in the database would be a "data redundancy". However I could document them, together with the database scheme etc. - but there is a lot to document first in the "online documentation" ;)
All you need to calculate is the distance between two sample (GPS) points, and the delta time (given their timestamp). This leads you straight to the desired velocity. So I don't see why you would require the "heading". As outlined previously you can use a (good) approximation with Pythagoras (the distances between sample points are small, so the error is negligible), or calculate the correct "distance on a sphere" using the Haversine formula, both linked in my previous post. In fact, I do this in my code, too, see: https://github.com/till213/SkyDolly/blob/main/src/Flight/src/FlightAugmentation.cpp -> FlightAugmentation::augmentAttitudeAndVelocity
Again, it all depends how the code "registers" those variables. It is possible that the SDK suggested radians in some places, but I thought that I followed the "general examples" when choosing units.
Again, forget the "location" table: it is not used at all for replay. Its only purpose is to store user-defined "locations" (sic) that you can "teleport" to, in the Location module. However I will actually add an additional "on ground" attribute to the upcoming "attitude" table. However it will (currently) only be honored when seeking to the very beginning of the replay, because there I issue a specific "initial position" command that actually takes an extra "on ground" attribute as well (the reason: when using this "initial position" SimConnect command it also "re-initialises" the scenery, by placing all AI aircraft again etc. - that is why I currently use this special command only when seeking to the very beginning of the timeline).
I am not quite sure whether I understand this question: the initial position and attitude are simply stored in the first sample point, typically with timestamp = 0.
Most values in the aircraft table are "for your information only" - except the on_ground attribute, that is (currently) used for placing the aircraft firmly on the ground, when initially loading that flight and when seeking back to the start. Again, I will store this attribute on the upcoming "attitude" table (as well), and hopefully can improve the "ground placement" when seeking in general.
All table names are singular. That is simply a common naming convention ;) Likewise we have:
Also see e.g. here:
And many more such (academic ;)) discussions. I simply followed (intuitively) what I have always done when naming database tables. There is neither wrong or right here. Of course consistency is king ;)
I will keep you updated here. I will reply to your second part a bit later :) Thank you for all your feedback, very much appreciated! |
|
[Editor's note: the following message has been edited for readability, by adding proper quotes (formatting) and removing redundant text (as posted above already)]
All you need to calculate is the distance between two sample (GPS) points, and the delta time (given their timestamp). This leads you straight to the desired velocity. So I don't see why you would require the "heading".
OK, but MSFS expects 3 velocities: X, Y and Z, where Z is the longitudinal main speed along the flight heading, X is the sideways (lateral, left/right) drift of the plane, and Y is the vertical speed (very important when a glider gets a winch tow!). So what do you mean with “the desired velocity”, is that velocity_Z? And if not, where do I enter that desired speed?? Now I’m a bit puzzled…
Here points A and B are 2 recorded coordinates. In A, the airplane has a heading towards point D. Obviously the wind is blowing from the left of the plane, therefor it is drifting to the right (orange line A-C, perpendicular to heading line), which gives lateral speed velocity_x. Velocity_z is then line C-B, which is a bit lower than the speed over ground (line A-B). Maybe I’m wrong in this…?
*SDK: *
**VELOCITY BODY X**
True lateral speed, relative to aircraft X axis.
Feet ( <https://docs.flightsimulator.com/html/Introduction/> ft) per second
**VELOCITY BODY Y**
True vertical speed, relative to aircraft Y axis.
Feet ( <https://docs.flightsimulator.com/html/Introduction/> ft) per second
**VELOCITY BODY Z**
True longitudinal speed, relative to aircraft Z axis.
Feet ( <https://docs.flightsimulator.com/html/Introduction/> ft) per second
As outlined previously you can use a (good) approximation with Pythagoras (the distances between sample points are small, so the error is negligible), or calculate the correct "distance on a sphere" using the Haversine formula, [...] In fact, I do this in my code, too, see:
will have a look at that!
Again, it all depends how the code "registers" those variables. It is possible that the SDK suggested radians in some places, but I thought that I followed the "general examples" when choosing units.
OK!
Again, forget the "location" table: it is not used at all for replay. Its only purpose is to store user-defined "locations" (sic) that you can "teleport" to, in the Location module.
Good to know!
However I will actually add an additional "on ground" attribute to the upcoming "attitude" table.
Nice!
(the reason: when using this "initial position" SimConnect command it also "re-initialises" the scenery, by placing all AI aircraft again etc. - that is why I currently use this special command only when seeking to the very beginning of the timeline).
Yes, I also read that in the SDK (or was it a forum discussion, don’t remember?....), you’re 100% right!
> Where can I set the initial attitude (especially pitch) and location/altitude of the glider?
I am not quite sure whether I understand this question: the initial position and attitude are simply stored in the first sample point, typically with timestamp = 0.
OK, I get it. I used the location, altitude and pitch values of the glider in FSX, for MSFS I will obviously have to use other values… So work to do here…
Most values in the aircraft table are "for your information only" - except the on_ground attribute, that is (currently) used for placing the aircraft firmly on the ground, when initially loading that flight and when seeking back to the start.
Apparently the ‘tail number’ is not used. But can I use ‘altitude_above_ground‘, or is that only informative?
Again, I will store this attribute on the upcoming "attitude" table (as well), and hopefully can improve the "ground placement" when seeking in general.
OK!
All table names are singular. That is simply a common naming convention
All right, I agree, consistency is king! I can live with singular names… 😉
|
|
Here is my reply to the second part of your second-to-last question. But one quick note before my reply: you seem to reply via email, which is fine. And I assume you use colours, to highlight your "inline" reply. The problem is: github converts all your messages to "text only": specifically your colour (if you use any in the first place) is removed, making it very hard to read your messages, specifically when it comes to distinguish between what you intend to quote, and where your actual "inline" reply starts (and ends). So either you use "traditional email quoting", as typically started with the ">" (larger than) character instead of colours, or you reply here on github.com instead (follow the link in the automated email). Replying on github.com has the advantage that you can use "markdown" syntax - and properly quote previous replies ;)
But now for my actual reply... :)
This might already help! In fact, there are quite a bunch of simulation variables that are - technically - also writeable (e.g. "flap position set" and the like), but I actually set them via "SimConnect events". The reason: depending on the variable "events" may work better: especially 3rd-party aircraft do not always react to changed "simulation variables" - most of the time those aircraft at least report the proper values via "simulation variables", but now always (I am looking at you, Fenix A320 ;)). So this would not be a "first" that I store the value via "simulation variable", but actually set it (during replay) via "events". I will take a note of the tow cable issue, that should be doable!
As I am currently "reworking" the time widget - to also include the actual date, in case the duration is longer than 24 hours - I will also think of this. Would a tooltip be sufficient? I am trying to keep the user interface as "clutter-free" as possible. Also, the duration of the flight is shown in the logbook (you can move around the columns, by the way: the logbook table layout is persisted, too). But I agree that the logbook may not always be visible, specifically when the user interface is in "minimal user interface" mode. So I try to think of something clever here...
Spline interpolation is almost as old as computer graphics (well, the math behind it is for sure older than computer graphics ;)). In fact, I just stumbled over a new kind of spline: Akima splines. They have the advantage that they "oscillate less" than e.g. Hermite (cubic) splines and hence feel more like a natural flight path. To be clear: when sampling the position each second the difference would be negligible. But when importing a flight with a sample interval of, say, 30 seconds or even more (e.g. a CSV import file originating from flightaware.com or flightradar24.com) then the difference will be significant! And yes, my master thesis was in computer graphics: 3d graphics, to be specific. "Pointshop 3D" is still available online, as open source: https://cgl.ethz.ch/pointshop3d/ It let's you draw (and carve) on a 3d object: the new thing (at the time): the 3d object is not modelled with triangles, but a "point cloud" instead (each point is rendered as an ellipse, to make the impression of a smooth surface). To be clear: the "point renderer" was written by one of the PhD students who was accompanying my master thesis. Also the whole "painting" and "sculpting" theory was researched by those two PhD students. So my task (master thesis) was to write an "extensible application" that would implement all those draw tool operations, read/write the "surfel format" and in general would provide all "interactive operations". And guess what: the first thing that I did was "rip away" all C and Microsoft MFC related code (Microsoft Foundation Classes - a terrible attempt to do "object-oriented programming" in a non-object oriented language like C - and terrible in every other aspect, too ;)) and I chose Qt as the underlying graphical application toolkit. Qt was new to me at the time (and I barely knew C++ - only from some Unix system programming lectures - at the ETH the progrmaming language Oberon was all the rage at the time). So the PhD student who had written the initial "surfel (surface element) renderer" in C and with MFC agreed to use C++ and Qt, but... "you are on your own when you get stuck!" Turned out that my decision to use Qt was the right one: it is such a nice toolkit, shielding you (for the most part) from all the C++ madness that existed at the time (early 2000 - and still today ;)). And already back then I used a "plugin architecture", that is, you put your "plugins" (DLLs essentially) into a specific "plugin directory" and the application scans for their availability "at runtime" - so you can extend an existing application with (well-defined) extension points, without having to re-compile the original application. That also enforces a strict "application architecture", with a well defined "dependency pyramid" etc. But I am rambling, sorry... ;) On the web site you will also find somewhere a picture of my team, and my (much) "younger me" ;) Good times... (Oh and yes, Sky Dolly is also using the Qt toolkit - with the very same "plugin architecture" like Pointshop 3D - just with a much more modern C++ flavour than back in the days, and 20+ more years of programming experience in all sorts of programming languages ;))
Well, eventually perhaps you can create some video of your imported and edited flights and I can have a look at it :) |
You are right, I was a bit sloppy here with my explanation respectively currently I indeed only calculate the "forward velocity" - that is, the Z axis -> "velocity_Z" - in my "flight augmentation" code (for those flight imports where no velocity is given): I simply calculate the distance between sample point A and B (convert to meters) and given the timestamps this gives me the "forward velocity" -> Z axis. Only. That is, I currently do not calculate the lateral and vertical velocities. If you come up with a good formula (given the GPS coordinates with timestamps, and yes, in this case probably also the heading is needed), then feel free to post them here and I will consider them as well in my "flight augmentation" code (that needs much more love anyway, as previously discussed with regards to bank angle etc.). |
> Got it (I hope)! I've learned something now (was new to GitHub) ... ;)
_> Nice, one step further towards replaying glider flights! I guess Sky Dolly could detect the used airplane is a glider, then generate the TOW_PLANE_RELEASE event automatically? On the other hand: in my current Excel IGC-to-FSX calculations, I also simulate the tow plane from the glider recorded data, assuming the tow plane is a fixed distance in front of the glider (standard 50m). Of course, it then has to bank and go up/down earlier than the glider, as in real life. Additionally, I calculated a (fictional) return-to-start path for the tow plane after the glider releases from the tow plane. Rather difficult calculations, but I succeeded! Nice to see the tow plane separates from the glider and eventually lands at the airport. In FSX you could even switch between the two airplanes, so you could sit in the towplane cockpit, watching it landing... Would be a possible future 'upgrade', as Sky Dolly is able to replay multiple airplanes... Also, for winch starts, it would be nice to only release the winch cable when the glider releases at the top of the tow... Also for the future..._
> A tooltip: not really what I had in mind, I was thinking of the max time always visible at the right end of the slider, and the current time (that now is at the right), somewhere above or below the time line. This is only a minor, but very useful, increase in 'clutter' in my opinion.
> Phew... quite impressive I must say! It's clear you have a lot more programming skills than I have, me only being a parttime and self-thought programmer (as a side-effect of having worked with electronics professionally). I have touched only the surface of MFC and C++, and have used C to program microcontrollers with an embedded real-time OS, that's 'small' stuff compared to what you did. I ofte see Qt mentioned, but have no knowledge about it (I always saw it related to QuickTime, the Apple media player...).
> If I find the time the next days, I will try to create a few. But I have to tell you that in the next two weeks I am participating in a real-life glider competition in Germany, so I will not be able to do very much (unless it rains a lot, which I hope not...). In any way, I will only take with me a small underpowered laptop, so I surely am not able to run MSFS on that! |
I used the formulae in a glider replay, and verified they produced accurate and realistic speed indications! Even when flying the glider in the landing pattern with a strong crosswind, so the nose pointing into the wind, the speed indications were correct! |
|
In a - with Sky Dolly recorded - flight, I noticed that velocity_x also can be negative. That was a surprise, as I assumed speeds were always positive, so that's what I calculated in the formula I sent you before. The SDK does not give any clue, but I have the impression that for a velocity_x to the right of the velocity_z vector the speed is positive, to the left is negative (please confirm or deny...). So in my calculations I had to find out if the x-vector is pointing to the left or the right of the z-vector. I have found a solution: LateralAngleRad = WorksheetFunction.Atan2(XC - XA, YC - YA) VelocityX = Sqr(((XC - XA) * Cos((YC + YA) / 2)) ^ 2 + (YC - YA) ^ 2) * EarthRadiusFeet / TimeInterval * LateralSign So VelocityX is multiplied by LateralSign (-1 or +1). I have tested this with a real-life converted flight, the result is convincing. Included is a zip file with the previous Excelsheet ("Determine Velocity_X en Velocity_Z in MSFS 14.xlsm"), that only gives a positive velocity_x, and the new "Determine Velocity_X en Velocity_Z in MSFS 15.xlsm", that implements the above calculations. Determine Velocity_X en Velocity_Z in MSFS.zip For Velocity_Y, this is simply done by taking the altitudes (in feet) of points A and B, then Velocity_Y (in ft/s) = (Alt B - Alt A) / (tB - tA) (where in my application the time interval tB - tA is 1s). |
|
I took some videos of my flight of 10 May 2024 in the Kempencup-contest in Weelde (EBWE), my gliding club. This was a flight of nearly 4 hours. I included 2024-05-10-NAV-5FA-01 - 08.sdlog. See Google Drive map 2024-05-10-NAV-5FA-01 D-KKAB Tow start in EBWE (Weelde) Kempencup Contest (MSFS SkyDolly) (link below). I used my Excel IGC-to-FSX tool to create a text file with the relevant parameters for Sky Dolly, which I imported in a new Excelsheet. The Velocity-parameters I had to create myself with the above method. In order to have the glider start and land on the MSFS runway, I had to manipulate the FSX coordinates, as well as the altitudes. After all these manual interventions, I could extract the data to be inserted in the sdlog file. This is surely not a final solution, as the glider attitude (pitch) is not yet as it should be (especially visible at the start). But it is enough to already produce a quite life-like simulation! There are other folders with flights replayed by FS Recorder in FSX. In the tow start flights you can see the simulated tow plane, derived from the glider's IGC log file. In the end I hope to replicate that also in MSFS/Sky Dolly. In FSX I could not simulate the tow/winch cable. You can see all videos at https://drive.google.com/drive/folders/1CB7OJSeZqnzy-RxrHv9Mr5x1NUJWTQiT?usp=sharing. If you don't have access, let me know. Have fun! |
While I cannot confirm or deny this (sorry, but I just HAD to paraphrase all those Hollywood secret agency movies here ;)) your explanation sounds very plausible: the different signs (+ or -) simply indicate the direction: "left" or "right", "up" or "down", "forwards" or "backwards". Now which is which is up to you (I would have to check this myself as well), respectively you can also open the "Simulation Variables" (Sky Dolly: press key V) and have a look at the shown values. That works also during recording, so you can "interactively" try it out. Note that some values in this dialog are reformated for readability (e.g. the latitude and longitude values), but for the most part the values are "as is", as recorded from the flight simulator. |
Thank you so much for sharing! I will gladly have a look at those files. Just to repeat: I intend to greatly enhance Sky Dolly's "flight augmentation" code, e.g.
I cannot make any promises about the "when" (or the "what" either, for that matter), as I develop this in my spare time. But I can assure that those features are on my "I want to have this, too"-list :) |
This is a classic confusion, also among professional developers ;) The Qt toolkit (originally created by a Norvegian company called Trolltech - yes, really ;)) is a so-called "cross-platform" development toolkit, providing user interface elements (windows, combo boxes, buttons, text fields, ...), but also other useful functionality for deskopt (and theses days also mobile) applications such as file access, networking, multi-threading and inter-process communication. In short: everything a modern (cross-platform) desktop application needs. In fact, I mostly develop Sky Dolly on macOS, and only for the flight simulator specific functionality ("SimConnect") I switch to Windows. The original authors pronounce Qt as "cute", but I (and others) simply pronounce it as "kiu-tee" (two letters). QT on the other hand (note: all capital letters) stands for QuickTime, a (discontinued) multimedia technology developed by Apple. But appart from those same letters - Qt vs QT - those two technologies (toolkits) have nothing in common :) Happy (real-world!) flying, by the way! I understand that you are now "in the air" (or in the coming days)... |
|
Next week I travel to a contest in Germany, a 700km drive, that takes me about a full day at max. 100km/h with a glider trailer behind the car... The contest is during 10 days, I hope the weather then will be nice.... If not, I may consult my e-mails more often... |
|
There is a very strange issue with the standard Asobo LS8 18 and the VelocityY calculations. For the moment this puzzles me, but I will not have the time anymore to investigate this before my journey. Nevertheless I share it with you.
The 2 videos show the same flight, each about 4 minutes:
When I use the LS8 Livery Aviators Club, then this bending does NOT occur! So I guess this is not just a livery change, but also has another flight model... ?? The same with a Discus glider, no bending! So for me, the best is to set VelocityY = 0 and use the standard Asobo LS8 18, that behaves the most 'natural', with slight wing flexing, as is normal. But WHY does VelocityY <> 0 not work???? When I record an LS8 flight with Sky Dolly, I never see this happen! Even not when I'm making the wildest movements with the glider. And I see varying VelocityY-values recorded, surely not 0! So that can only mean that my calculation of VelocityY is not correct. I further looked at it, and thought it might be this: VelocityY = VelocityZ * sin(radians(pitch)). But that's even worse... Now I thinking: I replay the real flight, and then record a second LS8 that flies the same path. Than I can compare the Velocity-values between the 2 gliders. Hopefully that provides a clue? Link to the videos: https://drive.google.com/drive/folders/1yMWw2IqPXP4GPvoYnqiKTZKMczHqe3P8?usp=sharing Side note: I have an FSX flight plan .pln file of this flight. MSFS just loads that fine, as you can see it in the VFR-chart. What you see is the contest task to fly. Just a pity MSFS does not like corners in the flight plan, it rounds them, which is unwanted for a contest flight!... ;) |
Thank you for your shared videos and specifically also the example Sky Dolly logbook (by the way: I downloaded them by now, you can "unshare" the OneDrive folder if you wish). While importing your Debugging the import process I quickly found out why: your logbook claims to have been created by Sky Dolly version v0.16 (according to the METADATA table) which is probably correct. But to my initial horror I noticed that the POSITION table did not contain the column INDICATED_ALTITUDE:
"Huh? Did I forget to add an important migration step when opening logbooks created by Sky Dolly v0.16?" - was my initial reaction. Because I knew that I had added the INDICATED_ALTITUDE column only at some later stage (that value is not used for the actual replay, but rather to define the initial position of aircraft, especially for when creating "AI aircraft" in a formation: because some SimConnect API expect GPS altitude, others require indicated (pressure) altitude - go figure ;)). Anyway, it quickly turned out that I had indeed added the column INDICATED_ALTITUDE already in Sky Dolly v0.9 and that the proper migration steps are all in place. And then I remembered your custom SQL script where you actually first drop and then re-create all tables (such as the POSITION table). Now the thing is you need to provide all columns - including the INDICATED_ALTITUDE column ;) If you only have GPS altitude at hand then simply fill the INDICATED_ALTITUDE column with the same values as the ALTITUDE column (or make some approximate calculations, based on some assumed default pressure and some magic formulae as found on the internet, for exampe: https://aviation.stackexchange.com/questions/37786/how-does-one-calculate-true-altitude ;) But the important thing is really: you need to create the exact table structure (all columns) for the logbook (= Sky Dolly) version that is indicated in the METADATA table (column APP_VERSION). -> The simplest thing is not to drop any tables, but instead write ("import") your calculated data via SQL into an existing logbook that has been created by Sky Dolly. Like this you can be sure that your table structure ("schema") is up to date and correct. This is what it should look like (in the current Sky Dolly v0.17 - in the upcoming v018 it will already be different again, as discussed earlier (split of lat/lon/alt and attitude data into two distinct tables):
However I also noticed some bugs in the actual Sky Dolly code:
So thanks to your shared logbook I actually found those issues! I have already fixed those issues in the (potentially) upcoming Sky Dolly v0.17.3 (but latest in the next v0.18). So again, thanks for sharing your logbook! |
|
The absolute times are implicitly given by the start date & time ("zulu" and "local time" zones) in the FLIGHT table:
The aboslute times can then be easily calculated, by simply adding the timestamp (milliseconds) to the start dates and times. Note however that the time stamp is typically "real-world" when recording a flight - that is, based on your actual computer clock. I mention this because you will also stumble over the end dates and times:
and especially if you would look at the times of a recorded flight (recorded from MSFS, that is) you would often notice that the "simulation end times" do not match with the actually recorded timestamps! This is because the simulation time often runs a bit "faster" in MSFS than in the real-world (despite the "simulation rate" being set to 1x). However if you create / import real-world data and are updating the timestamps according to some real-world flight (e.g. coming from an IGC file) then you can expect more realistic timestamps. Otherwise simply adjust the end dates and times in the FLIGHT table accordingly, if consistency is important here. |
|
The CREATION_TIME is meant to be the "real-world time" (stored in "zulu time", but displayed in the Sky Dolly user interface in your specific local time zone) that indicates when the recording has started or when a flight has been imported. Or in other words: there is no MSFS equivalent "simulation variable" for the CREATION_TIME, it is really an application-specific value. So you can set it to any time / date you want. But again, typically it is meant to be the "creation date of the flight data" (again, either the real-world recording time or time of file import etc.). In fact, it is the date and time being used in the logbook: the "year / month / day" date picker on the left uses this "creation date" field. So it helps you keep track of your recordings.
Adding additional columns should be fine: Sky Dolly will simply ignore those columns when reading data.
Ah, now I get it: you meant the time as shown in the user interface :) This is absolutely doable (and I remember that I had already thought about such a "time display option", to be able to toggle between "relative" (timestamp counting from 0 - as is the case now) and "absolute" times (showing the time in the format dd:mm:ss, that is counting from the START_LOCAL_SIM_TIME or START_ZULU_SIM_TIME etc.). In fact, I am actually reworking (extending) the time display right now in the upcoming Sky Dolly v0.18: I will also display the date in case the recording time is longer than 24 hours: In fact, I stumbled over this myself just recently, by "programmatically" (via SQL) creating a "timelapse recording" with a duration of one year (because why not ;)). -> And so a "timestamp" counter simply doesn't cut it anymore ;) So at the same time I will think about different time (date) display options, too (but no promises made whether they will "fit in" into the upcoming v0.18 - but I definitively took a note and find this feature very useful / natural to have).
Yes, I will think about also showing the end time. My aim is to keep the user interface as "clutter-free as possible" (in case the recording takes longer than 24 hours there will be an additional date field already), especially in the "minimal user interface" mode - but I think of something :) Thank you for all your input! And good luck with the weather! :) |
|
Hi Oliver,
OK, more time indications would be a good thing. It’s nice to reflect over these things…
Concerning the Velocities X,Y,Z: after many trials I always get excessively flexing wings with the LS8, especially with speeds over 140-150 km/h. The best solution still is: calculate and use Velocity_Z, but keep Velocity_X and Velocity_Y at 0! Strange… But all meters then work as expected, and the glider’s attitude is totally natural, even with some minor wing flexing. So I’ll stick to that. The only thing what is improved is that for the Velocity_Z now also the pitch has an influence: in a steep dive the ground speed (from the recorded GPS coordinates) does not change, but Velocity_Z of course must increase then!
Problem is that I can’t find any detailed explanation what these velocities are and how they should be calculated for MSFS, the SDK does not give a clue either. And doing maths yourself – with some assumptions - also doesn’t yield the good results. Probably some physicist or mathematician can solve this…? Or maybe I should ask it on Avsim (with little hope on a serious answer though…)?
To be continued after the contest… 😉 And indeed, hoping for better weather than we have now…
Regards,
Hugo
From: Oliver Knoll ***@***.***>
Sent: zondag 26 mei 2024 12:09
To: till213/SkyDolly ***@***.***>
Cc: hugojans ***@***.***>; Author ***@***.***>
Subject: Re: [till213/SkyDolly] Export to and import from CSV (Issue #135)
I correctly entered the START_LOCAL_SIM_TIME, START_ZULU_SIM_TIME, END_LOCAL_SIM_TIME, END_ZULU_SIM_TIME in the flight table. I don't know if creation_time is important, but I set it the same as the START_LOCAL_SIM_TIME.
The CREATION_TIME is meant to be the "real-world time" (stored in "zulu time", but displayed in the Sky Dolly user interface in your specific local time zone) that indicates when the recording has started or when a flight has been imported.
Or in other words: there is no MSFS equivalent "simulation variable" for the CREATION_TIME, it is really an application-specific value.
So you can set it to any time / date you want. But again, typically it is meant to be the "creation date of the flight data" (again, either the real-world recording time or time of file import etc.).
In fact, it is the date and time being used in the logbook: the "year / month / day" date picker on the left uses this "creation date" field. So it helps you keep track of your recordings.
Maybe you noticed that I introduced a datetime_local column in the position table.
Adding additional columns should be fine: Sky Dolly will simply ignore those columns when reading data.
But what I meant with my question is: when replaying, the (minimal) user interface shows the actual timestamp. For most people that's more than enough, surely when you only replay MSFS flights. But for the purpose of glider flight analysis, where data is coming from a source outside of MSFS, it is highly desirable to also have the real-world recorded time at the current location visible in the UI (maybe near the timestamp).
Ah, now I get it: you meant the time as shown in the user interface :) This is absolutely doable (and I remember that I had already thought about such a "time display option", to be able to toggle between "relative" (timestamp counting from 0 - as is the case now) and "absolute" times (showing the time in the format dd:mm:ss, that is counting from the START_LOCAL_SIM_TIME or START_ZULU_SIM_TIME etc.).
In fact, I am actually reworking (extending) the time display right now in the upcoming Sky Dolly v0.18: I will also display the date in case the recording time is longer than 24 hours: In fact, I stumbled over this myself just recently, by "programmatically" (via SQL) creating a "timelapse recording" with a duration of one year (because why not ;)).
-> And so a "timestamp" counter simply doesn't cut it anymore ;)
So at the same time I will think about different time (date) display options, too (but no promises made whether they will "fit in" into the upcoming v0.18 - but I definitively took a note and find this feature very useful / natural to have).
You could even combine this with my previous suggestion to also show the max. timestamp, I envisage it like this: for example in the middle above or below the timeline the current timestamp AND recorded (or derived from start time + current timestamp * 1000) time in hh:mm:ss format, and at the right end of the timeline the max. timestamp AND end time hh:mm:ss. For me this would be ideal, but it is up to you to decide, after all it's your "child"... ;)
Yes, I will think about also showing the end time. My aim is to keep the user interface as "clutter-free as possible" (in case the recording takes longer than 24 hours there will be an additional date field already), especially in the "minimal user interface" mode - but I think of something :)
Thank you for all your input! And good luck with the weather! :)
—
Reply to this email directly, view it on GitHub <#135 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/BIHQ6GSBWEQCBNOM5A2Z7QDZEGYDNAVCNFSM6AAAAABHD7OMNOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZSGE3DENJXGA> .
You are receiving this because you authored the thread. <https://github.com/notifications/beacon/BIHQ6GVTL5ODXMGTZSJHYFTZEGYDNA5CNFSM6AAAAABHD7OMNOWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTT7CY4AU.gif> Message ID: ***@***.*** ***@***.***> >
|
Today I recorded a short flight and exported it as a CSV file with "Position and attitude" data.
Test flight position and attitude.csv
At a later time, I wanted to import that same CSV file, but SkyDolly (version 0.16.3) issues an error: "The file ... could not be imported." I tried to import in the formats "SkyDolly (OBSOLETE)" and "Flight Recorder", without success.
My ultimate goal is to create a CSV file that can correctly be imported to and replayed by SkyDolly. But if importing it's own exported file is not even possible, I'm a bit worried this is not possible. Please help!
The text was updated successfully, but these errors were encountered: