Discussion: General Roadmap

[wokket]

G'day guys,

I just wanted to open a general discussion around where you want to head with this, your priorities, things we want to defer until later etc (and fair warning I haven't looked at the code yet, so Erik may have already ticked these boxes.

I've spent a bit of time pondering this and have some general ideas, but am obviously only one voice, and fully expect others to have had different experiences/use cases with RoundhousE so feel free to shoot me down here with your own experiences :)

I agree that we're more likely to make progress if we're working on features we actually use, so here's a quick brain dump on the features I personally use and love:

• The general directory structure based suite of RunOnce, RunAlways, RunIfChanged scripts
• Environment specific scripts
• Versioning info stored in the database (does anyone use something other than in-database versioning?)
• Version number sourced from a provided .DLL
• GO batch separation support
• SQL Server (on prem and Azure) only
  • Improving the support for connecting to Azure SQL would be nice (tokens, managed identity etc)
• DropCreate and Update modes. I have used RestoreAndUpdate in other environments too, but it feels less useful in a Cloud-First world to me.
• I have used tokens in the past for limited scenarios so I can see it's value.
• I have used the AdminConnection feature in the past for server-level setup of the database, but also ran into bugs around transaction scoping etc.
• .Net Global Tool installer (awesome!)
  • I currently build a docker image (based on Alpine) that uses .net tool installer to get Roundhouse on... Rather than having a formal docker image maybe we can just have some docco/a sample showing how to build a working docker based migration so there's less things to maintain?

wrt features I don't use, and I would be comfortable leaving until later (perhaps forever):

• MSbuild support. .Net is increasingly moving away from MSBuild, and I'd rather see support for an AzDo/GitHub task, or Docker-based deploys etc
• Oracle support. As above, I don't back myself to get it right, and have no means (or desire to learn) how to test it.
• MySQL had some funky keyword generation stuff going on back in RoundhousE,... It'd be nice to stay simple.

In other general thoughts (with no idea of how involved it would actually be):

• If we can somehow do 'pluggable' database support, so someone using SQL Server doesn't need to pull a binary with (say) Oracle or Postgres dlls. This would both be a smaller/faster install and save any (future) licencing issues for those that don't need it? No idea how we'd make it work yet...
• Separate the actual migration/execution engine from the Console app that's running it? Would this get us any benefits re: hosting roundhouse inside your application for those people that want in-app migrations?
• Maintaining Back-Compat with Roundhosue for the versioning info would make migrating to grate easier
  • Same file diffing logic
  • Sample showing how to point grate at the Roundhouse schema ?

Anyway that's some early thinking from my side, I'm off to look at some code :)

[erikbra]

Hi, @wokket , and welcome to the journey! Congratulations on the first issue!

Thanks a lot for a lot of input. There are a lot of points here. Some are already implemented, some are outstanding. I'll try to give a quick run-through of the features I think are already implemented in grate below, and I suggest creating issues on the others, so that we can detail the features, and track the implementation of them.

✔️ The general directory structure based suite of RunOnce, RunAlways, RunIfChanged scripts
✔️ Environment specific scripts
✔️ Versioning info stored in the database (does anyone use something other than in-database versioning?)
❌Version number sourced from a provided .DLL - I've simplified to just use a string from an argument for now
✔️ GO batch separation support
✔️ SQL Server (on prem and Azure) only - support Sql Server, PostgreSQL, MariaDB/MySQL now. Oracle if someone can create a connection string
✔️ Improving the support for connecting to Azure SQL would be nice (tokens, managed identity etc) - As long as it can fit in the connection string, this one should be good.
❌ DropCreate and Update modes. I have used RestoreAndUpdate in other environments too, but it feels less useful in a Cloud-First world to me. These are unfamiliar to me - maybe elaborate in a separate issue?
❌ I have used tokens in the past for limited scenarios so I can see it's value. Not implemented yet, but no big job, I think
✔️I have used the AdminConnection feature in the past for server-level setup of the database, but also ran into bugs around transaction scoping etc. This one should be good
❌ .Net Global Tool installer (awesome!) Not done yet. But only work.
❌ I currently build a docker image (based on Alpine) that uses .net tool installer to get Roundhouse on... Rather than having a formal docker image maybe we can just have some docco/a sample showing how to build a working docker based migration so there's less things to maintain? Great idea. We could also consider other ways of using docker and grate. Mounting volumes, having a watch mode? Other?

wrt features I don't use, and I would be comfortable leaving until later (perhaps forever):

❌ MSbuild support. .Net is increasingly moving away from MSBuild, and I'd rather see support for an AzDo/GitHub task, or Docker-based deploys etc I think MSBuild supprt is a bit exotic as well, and have never used it. Open to exploring the other options. We should detail the workflow scenario more here.
❌ Oracle support. As above, I don't back myself to get it right, and have no means (or desire to learn) how to test it. If someone wants to own it - go ahead, if not, it's a no-fix for my part. I've given up :D
✔️MySQL had some funky keyword generation stuff going on back in RoundhousE,... It'd be nice to stay simple. _ I think this one is OK, but I haven't tested the implementation thorougly._

n other general thoughts (with no idea of how involved it would actually be):

❌ If we can somehow do 'pluggable' database support, so someone using SQL Server doesn't need to pull a binary with (say) Oracle or Postgres dlls. This would both be a smaller/faster install and save any (future) licencing issues for those that don't need it? No idea how we'd make it work yet... RoundhousE is modular, one nuget package per database. We could split out, if there is a need. But, how big a hit is it, if you compile everything into a single binary, does it matter if it's 15 or 21 Megs? I see arguments in both directions. Open to hearing more scenarios here.

❌ Separate the actual migration/execution engine from the Console app that's running it? Would this get us any benefits re: hosting roundhouse inside your application for those people that want in-app migrations? _ I'm eager to hear some more scenarios here. I've never RoundhousE as anything other than a command-line tool. But, others use it as a library, I think._

✔️Maintaining Back-Compat with Roundhosue for the versioning info would make migrating to grate easier This is done. Just specify 'RoundhousE' as schema name.
✔️Same file diffing logic
❌ Sample showing how to point grate at the Roundhouse schema ? Definitely. This is a quick win. I think we should just create a wiki page listing the options needed.

Love to hear your further thougts. Does it make sense to create separate issues for each of the points with a ❌ on them? So we can discuss each one in depth?