doctrine:schema:validate returns DROP TABLE doctrine_migration_versions; #1406
Comments
|
+1: |
derrabus
changed the title
|
Somebody posted a workaround for Symfony applications relying on the Doctrine bundle: doctrine:
dbal:
schema_filter: ~^(?!doctrine_)~Can people here please test? |
|
|
Possible next steps:
|
|
The bug exists on ORM 3.1 with DBAL 3.8 as well, it is not per se a DBAL 4.0 issue. |
|
OK so possible next steps:
If somebody could dump their DBAL schema filter at runtime using the DBAL configuration object on ORM 2 then on ORM 3, it might help. |
|
Can somebody here please confirm that they get the issue with ORM 2 when using the |
No |
|
|
|
This is a real problem for our project, especially for production. because it causes an even bigger problem: |
|
Here is a shell script to trigger both a succeeding workflow with ORM 2.19 and a failing one with ORM 3.0. Script#!/usr/bin/env sh
# Set up new Symfony 7 project
composer create-project symfony/skeleton:"7.0.*" my_test_project
cd my_test_project
# Install Doctrine bundles
composer require --no-interaction "doctrine/doctrine-bundle:~2.11" "doctrine/doctrine-migrations-bundle:~3.3" "doctrine/orm:~2.19" "doctrine/dbal:~3.8"
# Use sqlite for local testing
echo "DATABASE_URL=\"sqlite:///%kernel.project_dir%/var/data.db\"" > .env.local
# Install empty migration. This creates the sqlite database with the doctrine_migration_versions table
bin/console doctrine:migrations:generate
bin/console doctrine:migrations:migrate
# Validate schema: succeeds
bin/console doctrine:schema:validate -v
# Upgrade to ORM 3.*
composer require "doctrine/orm:~3.0"
# Validate schema: fails
bin/console doctrine:schema:validate -vThe difference causing the change is that within the callstack of the command, the variable Backtrace ORM 2.19Backtrace ORM 3.0 |
|
Related PRs:
I think we should leverage asset filtering to do this. If it causes issues with command from |
|
Just remembered this solution that does exactly that (using a dynamic filter). Something better that might benefit not just Symfony users might be to do it directly from code inside |
|
Proposed If you run for example |
|
@gjuric yes, @yapro already established that in #1406 (comment) Can you try the solution in my last command? I'm currently working on integrated the solution from my previous comment in the migrations bundle. |
|
Here is my attempt: doctrine/DoctrineMigrationsBundle#526 |
|
For now, I manually add the migration table to the schema with a doctrine event listener on Here's a gist if anyone interested : https://gist.github.com/Brewal/d0fe0792a69e7e5fdf3fb06898b20d35 |
|
Adding a listener for |
|
If it relies solely on the ORM's event system and not on Symfony or the DoctrineBundle, I can see how it would be superior to the asset filtering solution, in that it could benefit to other frameworks such as Laminas or Laravel as well. @Brewal @stof are you willing to open a pull request? Otherwise I might work on it soon. If I understand properly, the ORM command works by diffing 2 schemas. @FlyingDR's relies on asset filtering at the DBAL level, while Symfony's event system, which means it works only for Symfony (I think? Can @Brewal's solution relies on the ORM event system, which will kick in only for the ORM. I hope we are not forgetting about other cases relying on schema diffing and not relying on the ORM somehow. Are there other pros or cons I'm forgetting ? |
Removing the table from the schemas won't detect changes to |
|
Gave it a try, and I'm starting to see some cons:
I think I'd rather we go with @FlyingDR's solution, where only |
|
I gave it a try as well and ended up opening three PRs where each PR requires the previous one.
I don't believe so as
Hence the choice to remove the table from the comparison schema instead of adding it to the generated (metadata) schema. This doesn't catch changes to One thing that is missing is documentation on how to use #1418 without DoctrineMigrationsBundle. I feel like we can and should add that to the docs of this repository. |
|
As long as the issue of generating a Also, checking
The reason is that the DiffGenerator temporarily changes the configuration of DBAL to ignore that table from the introspection when generating the diff (so that it does not think it exists): (to be exact, it permanently changes the config until the end of the PHP process, making this class unsuitable for usages outside thediff command that exits the process once generated)
|
|
Oh right, this is not a workaround for Postgres users indeed. We'll have to give this more thought. |
|
Also: running |
|
Well… the
Do you mean because your CI filesystem is readonly or what? Please elaborate. A |
|
Just Upgrading doctrine/migrations (3.8.1 => 3.8.2) https://github.com/doctrine/migrations/releases/tag/3.8.2 But the |
I seem to have misunderstood this part:
We have a couple of use cases for The other use case could be compared to I usually use If Also: a couple of projects are not using the migration system for security reasons. If the |
The mapping validation is not planned for any changes, no.
Yes… and more importantly, the exit code of the command being negated in my proposal would make your deployment fail right after the file is created.
Our plan would be to have a mode for the migrations package. When in changelog mode, everything would work as it works now. When in "no-changelog" mode, things would work "in memory": no migration file, and the metadata table would be ignored. So basically it would work like
Can you elaborate on the "security reasons"? |
In some environments, database schema changes need to be controlled more tightly, using a two-man rule, and all schema changes are deployed by hand. The database user which Doctrine uses, have only a limited set of grants, and do not allow for schema changes. Those projects do not have the Doctrine Migrations Bundle installed at all. |
I don't fall into this camp, but this is a good question |
|
I see. With the new design you would still have to install |
|
Coming from doctrine/orm#11750 The fact that I thought about the @greg0ire workaround And maybe to avoid any error and then check that no file was generated. But Imho the right way could be to
This way, schema users could run |
|
Sounds fine, but bear in mind that in the end, we would like to remove the SchemaTool from the ORM altogether, so every user would end up being a migrations user, it's just that there would be a changelog mode you could turn off. So there would be no |
|
@greg0ire we did check removing SchemaTool until we found out that Migrations relies on it completly for generating the diffs 😆 One option we could consider is removing the doctrine:schema:* cmmands from the ORM and only making SchemaTool available programatically. @VincentLanglet I think your thoughts on separating the commands make total sense, just wondering how we can do this in a way where people upgrading sillently loose their mapping validation. schema:validate might need to be split into two new commands, leaving the original as-is and then remove it after deprecation. |
Oh right… if we go with that option, should we move |
|
For me, the previous answers did not work. Here is my workaround using MySQL until another solution is found :
<?php
declare(strict_types=1);
namespace App\Doctrine\DBAL\Schema;
use Doctrine\DBAL\Schema\{MySQLSchemaManager, Table};
class CustomMySqlSchemaManager extends MySQLSchemaManager
{
#[\Override]
public function createTable(Table $table): void
{
if ($table->getName() === 't___doctrine_migration_versions') {
return;
}
parent::createTable($table);
}
}
<?php
declare(strict_types=1);
namespace DoctrineMigrations;
use Doctrine\DBAL\Schema\Schema;
use Doctrine\Migrations\AbstractMigration;
final class Version20221104163712 extends AbstractMigration
{
/**
* @return non-empty-string
*/
#[\Override]
public function getDescription(): string
{
return 'Create the `t___doctrine_migration_versions` table ';
}
#[\Override]
public function up(Schema $schema): void
{
// language=MySQL
$sql = '
CREATE TABLE t___doctrine_migration_versions(
version VARCHAR(191) NOT NULL,
executed_at DATETIME NOT NULL,
execution_time INT UNSIGNED NOT NULL,
PRIMARY KEY (version)
) DEFAULT CHARACTER SET utf8mb4 COLLATE `utf8mb4_unicode_ci` ENGINE = InnoDB;
';
$this->addSql($sql);
}
#[\Override]
public function down(Schema $schema): void
{
// language=MySQL
$sql = '
DROP TABLE t___doctrine_migration_versions;
';
$this->addSql($sql);
}
#[\Override]
public function isTransactional(): bool
{
return false;
}
}
doctrine:
dbal:
connections:
default:
schema_filter: /^(?!t___)/
# config/packages/doctrine_migrations.yaml
doctrine_migrations:
storage:
table_storage:
table_name: 't___doctrine_migration_versions' |
|
I just released |
|
@greg0ire looks good on an application I just checked where this was an issue before |
I can confirm I don't have the issue in 3.4.0 |
|
Thank you both :) |
|
Not sure if related, but after upgrading to 3.4.0 from 3.3.1, migrations have stopped working.
A
|
|
Yes, I think my code needs a few slight adjustments I've described in doctrine/DoctrineMigrationsBundle#584 |
|
It seems it does not work with multiple entity manager connections |
|
I have the same issue as @cosminsandu. I solved it by adding an extra tag in a compile pass for the extra entity manager which also uses migrations: <?php
namespace App\DependencyInjection\Compiler;
use Symfony\Component\DependencyInjection\Compiler\CompilerPassInterface;
use Symfony\Component\DependencyInjection\ContainerBuilder;
class DoctrineMigrationsFilterCompilePass implements CompilerPassInterface
{
public function process(ContainerBuilder $container): void
{
$schemaFilter = $container->findDefinition('doctrine_migrations.schema_filter_listener');
$schemaFilter->addTag('doctrine.dbal.schema_filter', ['connection' => 'your_custom_connection']);
}
}Then add it to the Kernel with a higher priority than the // src/Kernel.php
namespace App;
use App\DependencyInjection\Compiler\DoctrineMigrationsFilterCompilePass;
use Symfony\Bundle\FrameworkBundle\Kernel\MicroKernelTrait;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\HttpKernel\Kernel as BaseKernel;
class Kernel extends BaseKernel
{
use MicroKernelTrait;
// ...
protected function build(ContainerBuilder $container): void
{
$container->addCompilerPass(new DoctrineMigrationsFilterCompilePass (), priority: 1);
}
} |
and others
BC Break Report
Summary
When I updated doctrine/orm to version 3.0.0, my CI started to fail due to the doctrine:schema check:validate, which returned DROP TABLE doctrine_migration_versions;
I found that adding schema_filter can fix this problem: "
^(?!doctrine_migration_versions$)" in the dbal configuration, indeed, then the error no longer appears, but after that my migrations stopped tracking the current state and think that at the moment no migration has been performed.In version 3.0, the saveMode argument was removed in \Doctrine\ORM\Tools\SchemaTool::getUpdateSchemaSql and because of this, doctrine:schema:validate tracks deleted tables and returns sql for this action.
Previous behavior
Current behavior
How to reproduce
The text was updated successfully, but these errors were encountered: