MYSQL::Activate Slave server Again to sync with existing Master::Cluster Dissolve::Windows

Scenario: Activating existing slave again to sync with master server after cluster dissolved task happened in the master servers (group replication Innodb Cluster).

Steps:

1.Stop slave to ensure no process to master server.

mysql> STOP SLAVE;

The STOP SLAVE command in MySQL is used to stop the slave replication process on a slave server in a master-slave replication setup. When executed on the slave server, it halts the replication process and stops the slave from applying any new changes received from the master.

Query OK, 0 rows affected, 1 warning (0.00 sec)

2.Change master command to set the new master server ip and credentials details.

 mysql> CHANGE MASTER TO MASTER_HOST='MASTERDBSERVERIP', MASTER_PORT=3306, MASTER_USER='db_repl',

    ->  MASTER_PASSWORD='Password',MASTER_DELAY = 0,MASTER_AUTO_POSITION=1;

Query OK, 0 rows affected, 1 warning (0.05 sec)

 The CHANGE MASTER TO command in MySQL is used on a slave server in a master-slave replication setup to configure and change the replication settings. It is used to specify the master server's connection details and position from which the slave should start replicating.

3.Then start slave to sync master copy to slave.

mysql> START SLAVE;

Query OK, 0 rows affected (0.00 sec)

The START SLAVE command in MySQL is used to start the replication process on a slave server in a master-slave replication setup. When executed on the slave server, it initiates the connection to the master server and begins replicating the changes received from the master.

4.Check show slave status results.

The SHOW SLAVE STATUS command in MySQL is used to display detailed information about the current status of a slave server in a master-slave replication setup. When executed on the slave server, it provides a comprehensive overview of the slave's replication state and configuration.

mysql> show slave status \G;

*************************** 1. row ***************************

               Slave_IO_State: Waiting for master to send event

                  Master_Host: MASTERDBSERVERIP

                  Master_User: db_repl

                  Master_Port: 3306

                Connect_Retry: 60

              Master_Log_File: MASTERDBSERVERNAME-bin.001763

          Read_Master_Log_Pos: 843694033

               Relay_Log_File: SLAVEDBSERVERNAME-relay-bin.000002

                Relay_Log_Pos: 10832949

        Relay_Master_Log_File: MASTERDBSERVERNAME-bin.001763

             Slave_IO_Running: Yes

            Slave_SQL_Running: Yes

              Replicate_Do_DB:

          Replicate_Ignore_DB:

           Replicate_Do_Table:

       Replicate_Ignore_Table:

      Replicate_Wild_Do_Table:

  Replicate_Wild_Ignore_Table:

                   Last_Errno: 0

                   Last_Error:

                 Skip_Counter: 0

          Exec_Master_Log_Pos: 843694033

              Relay_Log_Space: 10833158

              Until_Condition: None

               Until_Log_File:

                Until_Log_Pos: 0

           Master_SSL_Allowed: No

           Master_SSL_CA_File:

           Master_SSL_CA_Path:

              Master_SSL_Cert:

            Master_SSL_Cipher:

               Master_SSL_Key:

        Seconds_Behind_Master: 0

Master_SSL_Verify_Server_Cert: No

                Last_IO_Errno: 0

                Last_IO_Error:

               Last_SQL_Errno: 0

               Last_SQL_Error:

  Replicate_Ignore_Server_Ids:

             Master_Server_Id: 13

                  Master_UUID: 08538d8c-ed98-11ea-852d-005056844240

             Master_Info_File: mysql.slave_master_info

                    SQL_Delay: 0

          SQL_Remaining_Delay: NULL

      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates

           Master_Retry_Count: 86400

                  Master_Bind:

      Last_IO_Error_Timestamp:

     Last_SQL_Error_Timestamp:

               Master_SSL_Crl:

           Master_SSL_Crlpath:

           Retrieved_Gtid_Set: 08538d8c-ed98-11ea-852d-005056844240:942835-942878

            Executed_Gtid_Set: 08538d8c-ed98-11ea-852d-005056844240:1-942878,

28179efb-72c5-11e4-b496-0050569242d9:1-13995703,

29a42f08-7c8e-11ed-969f-005056846111:1-69949262:70421923-70422753,

329fcb1a-c0dd-11eb-82b1-005056847352:1-66160,

6a6ae23d-183f-11eb-ae83-005056846111:1-114606407:114931406-114935574,

ae4d4435-ecbc-11ea-a835-0050568415e0:1-348529,

d1f71bf8-c0dd-11eb-82b1-005056847352:1-258121803,

d9014b6b-1812-11eb-976b-005056846111:1-292573,

dfc8fe40-336a-11ed-a8ea-005056847352:1-48770981

                Auto_Position: 1

         Replicate_Rewrite_DB:

                 Channel_Name:

           Master_TLS_Version:

       Master_public_key_path:

        Get_master_public_key: 0

            Network_Namespace:

1 row in set (0.00 sec)

Interpret the Output: The SHOW SLAVE STATUS command provides a result set with various columns containing information about the replication status. Some of the important columns and their meanings include:

1. Slave_IO_State: The current state of the slave I/O thread.
2. Master_Host and Master_Port: The hostname and port of the master server.
3. Slave_IO_Running and Slave_SQL_Running: Whether the slave I/O thread and slave SQL thread are running (Yes) or stopped (No).
4. Seconds_Behind_Master: The replication lag in seconds, indicating how far behind the slave is compared to the master.
5. Last_Error: Any error message related to replication.
6. Master_Log_File and Read_Master_Log_Pos: The binary log file and position from which the slave is currently reading.
7. Relay_Log_File and Relay_Log_Pos: The relay log file and position, indicating the last processed event on the slave.
8. Exec_Master_Log_Pos: The position of the last executed event on the slave.

These columns provide insights into the replication status, lag, error messages, and positions within the replication process.

 

MYSQL::Audit Requirement:: Keyring file rotation in MYSQL windows ::Master -SLave

 MYSQL::Audit Requirement:: Keyring file rotation in MYSQL windows ::Master -Slave

The ALTER INSTANCE ROTATE INNODB MASTER KEY statement in MySQL is used to rotate the master key for InnoDB tablespace encryption in a master-slave replication setup. This statement is specific to the InnoDB storage engine and is used when you have enabled encryption for InnoDB tablespaces using the keyring plugin.

Note down the keyring file timestamp before rotation.

When you execute the ALTER INSTANCE ROTATE INNODB MASTER KEY statement on the master server, it performs the following actions:

Generates a new master key: The statement generates a new master key for InnoDB tablespace encryption. The master key is used to encrypt and decrypt the InnoDB tablespaces.

Encrypts new data with the new master key: After generating the new master key, the statement starts encrypting new data that is written to the InnoDB tablespaces using the new key.

Re-encrypts existing data: The statement also initiates a background process that gradually re-encrypts existing data in the InnoDB tablespaces using the new master key. This process ensures that all data is eventually re-encrypted with the new key.

Synchronizes key rotation with replication: The ALTER INSTANCE ROTATE INNODB MASTER KEY statement ensures that the rotation of the master key is synchronized with the replication process in a master-slave setup. It ensures that the new key and the re-encryption process are propagated to the slave servers, allowing them to use the new master key for decryption.

Need to execute this command in master server only.

mysql>ALTER INSTANCE ROTATE INNODB MASTER KEY;

By rotating the InnoDB master key, you enhance the security of your encrypted data by periodically changing the encryption key. This helps protect against potential security threats or key exposure.

Note down the keyring file timestamp after rotation to check latest timestamp come.

It's important to note that this statement should be executed on the master server in a master-slave replication setup, and the key rotation and re-encryption process will be automatically propagated to the slave servers.

Note: These steps can be used in Innodb Cluster group replication also. For 4TB database, it took less than 10 seconds only.

MYSQL::Enterprise backup getting failed after cluster dissolve::

Error: ERROR: Unexpected number of rows from MySQL query 'SELECT member_role FROM performance_schema.replication_group_members WHERE member_id=@@server_uuid': got 0, expected == 1.

Anyone experienced this error. MySQL enterprise full backup, differential and incremental backup having this error in the backup log. Please advise any to need to be taken or can we ignore.

=========================================================================

230507 15:35:23 MAIN    INFO: SSL: attempting to use an SSL connection.

230507 15:35:23 MAIN    INFO: Group replication setup detected.

230507 15:35:23 MAIN   ERROR: Unexpected number of rows from MySQL query 'SELECT member_role FROM performance_schema.replication_group_members WHERE member_id=@@server_uuid': got 0, expected == 1.

230507 15:35:23 MAIN WARNING: This backup operation cannot update the backup_history table.

230507 15:35:23 MAIN    INFO: Full Image Backup operation completed successfully.

230507 15:35:23 MAIN    INFO: Backup image created successfully.

230507 15:35:23 MAIN    INFO: Image Path = Z:\DBNAME_PROD_WEEKLY_FULL_BKP\FULL_20230605_220000.img

230507 15:35:23 MAIN    INFO: MySQL binlog position: filename SERVERNAME-bin.001803, position 136791673

230507 15:35:23 MAIN    INFO: GTID_EXECUTED is 08538d8c-ed98-11ea-852d-005056844240:1-1658361,28179efb-72c5-11e4-b496-0050569242d9:1-13995703,29a42f08-7c8e-11ed-969f-005056846111:1-69949262:70421923-70422753,329fcb1a-c0dd-11eb-82b1-005056847352:1-66160,6a6ae23d-183f-11eb-ae83-005056846111:1-114606407:114931406-114935574,ae4d4435-ecbc-11ea-a835-0050568415e0:1-348529,d1f71bf8-c0dd-11eb-82b1-005056847352:1-258121803,d9014b6b-1812-11eb-976b-005056846111:1-292573,dfc8fe40-336a-11ed-a8ea-005056847352:1-48770981

-------------------------------------------------------------

   Parameters Summary         

-------------------------------------------------------------

   Start LSN                  : 13530353956352

   End LSN                    : 13531799802630

-------------------------------------------------------------

mysqlbackup completed OK! with 2 warnings

=========================================================================

Solution:

mysql> SELECT * FROM performance_schema.replication_group_members;

+---------------------------+-----------+-------------+-------------+--------------+-------------+----------------+

| CHANNEL_NAME              | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION |

+---------------------------+-----------+-------------+-------------+--------------+-------------+----------------+

| group_replication_applier |           |             |        NULL | OFFLINE      |             |                |

+---------------------------+-----------+-------------+-------------+--------------+-------------+----------------+

1 row in set (0.00 sec)

UNINSTALL PLUGIN GROUP_REPLICATION;

=================================================================

The UNINSTALL PLUGIN GROUP_REPLICATION; command is used in MySQL to uninstall or remove the Group Replication plugin. Group Replication is a MySQL plugin that enables you to create a highly available replication group of MySQL servers. It provides a multi-master replication solution with automatic conflict detection and resolution.

When you execute the UNINSTALL PLUGIN GROUP_REPLICATION; command, it removes the Group Replication plugin from the MySQL server. This means that the server will no longer participate in a Group Replication setup and will not be part of a replication group.

======================================================

mysql> uninstall plugin group_replication;

ERROR 1290 (HY000): The MySQL server is running with the --super-read-only option so it cannot execute this statement

mysql> set global super_read_only=0;

  Query OK, 0 rows affected (0.00 sec)

mysql> uninstall plugin group_replication;

Query OK, 0 rows affected (0.03 sec)

mysql> set global super_read_only=1;

Query OK, 0 rows affected (0.00 sec)

Then backup task started to run successfully.

Here are a few things to keep in mind when using the UNINSTALL PLUGIN GROUP_REPLICATION; command:

1. Replication Group: Before uninstalling the Group Replication plugin, ensure that the server you're uninstalling from is not an essential member of the replication group. Otherwise, removing the plugin from a member server can disrupt the replication group's functionality.

2. Backup: Make sure you have a backup of your data or a plan in place to ensure data availability during the uninstallation process. Removing the plugin doesn't affect the data stored on the server, but it's always a good practice to have backups in case of any unexpected issues.

3. Impact: Uninstalling the Group Replication plugin only affects the specific server where the command is executed. Other servers in the replication group will continue functioning as long as they still have the plugin installed and configured.