global_lib api documentation

class global_lib.config_dir[source]

Bases: object

Class used to setup the local user configuration directory. The class constructor sets only the class variables for the method set_config. The function get_python_lib() is used to determine the python library where pg_chameleon is installed.


The method loops the list self.conf_dirs creating it only if missing.

The method checks the freshness of the config-example.yaml file and copies the new version from the python library determined in the class constructor with get_python_lib().

If the configuration file is missing the method copies the file with a different message.

class global_lib.global_config(config_name='default', debug_mode=False)[source]

Bases: object

This class parses the configuration file specified by the parameter config_name and sets the class variables used by the replica_engine class. The constructor checks if the configuration file is present and if is missing emits an error message followed by the sys.exit() command. If the configuration file is successful parsed the class variables are set from the configuration values.

The function get_python_lib() is used to determine the library directory where pg_chameleon is installed in order to get the sql files. The configuration files are searched in the $HOME/.pg_chameleon/config. Should any parameter be missing in config the class constructor emits an error message specifying the parameter with reference to config-example.yaml.

Parameters:config_name – the configuration file to use. If omitted is set to default.

The method tries to set the parameter source_name determined from the configuration file. The value is used to query the replica catalog in order to get the source sstatus in method list_config().

Parameters:config_name – the configuration file to use. If omitted is set to default.
class global_lib.replica_engine(config, debug_mode=False, nolock=False)[source]

Bases: object

This class is wraps the the mysql and postgresql engines in order to perform the various activities required for the replica. The constructor inits the global configuration class and setup the mysql and postgresql engines as class objects. The class sets the logging using the configuration parameter.


register the configuration source in the replica catalogue


checks if the process is running.

Parameters:write_pid – determines whether the pid file is written or not. Used if we just need to check if the replica is running.

Copy the data for the replicated tables from mysql to postgres. After the copy the master’s coordinates are saved in postgres.

Parameters:truncate_tables – determines whether the existing tables should be truncated before running a copy table data

Creates the indices on the PostgreSQL schema using the metadata extracted from MySQL.


Creates the database schema on PostgreSQL using the metadata extracted from MySQL.


Creates the service schema sch_chameleon on the PostgreSQL database. The service schema is required for having the replica working correctly.


The method terminates the replica, remove the source from the table t_sources and resets the sequences in the postgresql database, leaving the replica snapshot capable to continue the activity directly on PostgreSQL


Drops the service schema. The action discards any replica data.


remove the configuration source and all the replica informations associated with the source from the replica catalogue


The method removes the exit file in order to let the replica start again.


The method initialise a replica.

It calls the pg_engine methods set_source_id which sets the source identifier and change the source status.

The pg_engine method clean_batch_data is used to remove any unreplayed row in the tables t_log_replica_1(2).

The class methods create_schema, copy_table_data and create_indices are called in sequence to initialise the replica.


List the available configurations stored in ~/.pg_chameleon/config/


The method starts and run the replica loop. Before starting the loop checks if the replica is already running with check_running with write_pid=True.

It also checks whether the exit file is present or not. If present skips the replica start, otherwise start the while loop which runs the mysql_engine method run_replica.

When the mysql_engine.run_replica() completes it checks if there is the exit file and eventually exit the loop. Otherwise sleeps for the amount or seconds set in sleep_loop.


Threaded version of run replica.


list the replica status using the configuration files and the replica catalogue


the method writes the exit file in the pid directory and waits for the replica process’s end. If allow_restart is true the exit file is removed.

Parameters:allow_restart – determines whether the exit file is removed or not in order to allow the replica to start again.

syncronise single tables with the mysql master. The process attempts to drop the existing tables or add them if not present. The tables are stored in the replica catalogue with their master’s coordinates and are ignored until the replica process reaches the correct position. :param table: comma separated list of table names to synchronise


Upgrade the service schema to the latest version.


The method is used to wait for the replica’s end monitoring the pid file. The replica status is determined using the check_running method passing the write_pid=false.

There is a 5 seconds sleep between each check.