This maintenance release adds the support for skip events. Is now is possible to skip events (insert,delete,update) for single tables or for entire schemas. A new optional source parameter skip_events: is available for the sources with type mysql. Under skip events there are three keys one per each DML operation. Is possible to list an entire schema or single tables in the form of schema.table. The example snippet disables the inserts on the table delphis_mediterranea.
The maintenance release makes the multiprocess logging safe. Now each replica process logs in a separate file. The --full option now is working. Previously the option had no effect causing the maintenance to run always a conventional vacuum. This release fixes the issues reported in ticket #73 and #75 by pg_chameleon’s users. The bug reported in ticket #73 caused a wrong data type tokenisation when an alter table adds a column with options (e.
The maintenance release 2.0.6 fixes a crash occurring when a new column is added on the source database with the default value NOW(). The maintenance introduced in the version 2.0.5 is now less aggressive. In particular the run_maintenance command now executes a conventional VACUUM on the source’s log tables, unless the switch --full is specified. In that case a VACUUM FULL is executed. The detach has been disabled and may be completely removed in the future releases because very fragile and prone to errors.
The maintenance release 2.0.5 a regression which prevented some tables to be synced with sync_tables when the parameter limit_tables was set. Previously having two or more schemas mapped with only one schema listed in limit_tables prevented the other schema’s tables to be synchronised with sync_tables. This release add two new commands to improve the general performance and the management. The command stop_all_replicas stops all the running sources within the target postgresql database.
The maintenance release 2.0.4 fix the wrong handling of the ALTER TABLE when generating the MODIFY translation. The regression was added in the version 2.0.3 and can result in a broken replica process. This version improves the way to handle the replica from tables with dropped columns in the future. The python-mysql-replication library with this commit adds a way to manage the replica with the tables having columns dropped before the read replica is started.
The bugfix release 2.0.3 fixes the issue #63 changing all the fields i_binlog_position to bigint. Previously binlog files larger than 2GB would cause an integer overflow during the phase of write rows in the PostgreSQL database. The issue can affect also MySQL databases with smaller max_binlog_size as it seems that this value is a soft limit. As this change requires a replica catalogue upgrade is very important to follow the upgrade instructions provided below.
When I started writing pg_chameleon I had in mind a tool making simple to migrate from different dbms engines to PostgreSQL. The name was inspired by the Culture Club’s Karma chameleon song which suggested me the idea of a tool capable to move data between different database technologies. Then tool’s scope changed becoming a replica system rather a migration system but I’ve kept the name as I think the idea is still valid.
This bugfix relase adds a missing functionality which wasn’t added during the application development and fixes a bug in the sync_tables command. Previously the parameter batch_retention was ignored making the replayed batches to accumulate in the table sch_chameleon.t_replica_batch with the conseguent performance degradation over time. This release solves the issue re enabling the batch_retention. Please note that after upgrading there will be an initial replay lag building. This is normal as the first cleanup will have to remove a lot of rows.
The first maintenance release of pg_chameleon v2 adds a performance improvement in the read replica process when the variables limit_tables or skip_tables are set. Previously all the rows were read from the replica stream as the BinLogStreamReader do not allow the usage of the tables in the form of schema_name.table_name. This caused a large amount of useless data hitting the replica log tables as reported in the issue #58.
The version 2.0 of pg_chameleon the MySQL to PostgreSQL replica system is now available. Several improvements come with this new milestone which is compatible with python 3.3+. Replicates multiple MySQL schemas within the same MySQL cluster into a target PostgreSQL database. The source and target schema names can be different. Conservative approach to the replica. Tables which generate errors are automatically excluded from the replica. Daemonised init_replica,refresh_schema,sync_tables processes. Daemonised replica process with two separated subprocess, one for the read and one for the replay.