Migrate one CMA from a Provider-1 R70.40 to another Provider-1

Posted on November 8, 2011

1



I wanted to do some tests with only one CMA of our live Provider1 in Lab, that is why I tried to migrate just one piece. I could do a complete mds backup and restore in Lab, but if the provider has a lot of CMAs then it requires in Lab the same Hardware or it will be really really slow.. The following process is actually a working solution for CMA backup and restore on the same Provider1 if there is no global policy.

Steps not part of this post:

– Create Provider-1 Administrators
– Redefine and reassign them to customers after the upgrade.
– Provider-1 SmartConsole Clients
– Redefine and reassign to customers after the upgrade.
– Policy assignment to customers
– Assign policies to customers after the upgrade.
– Global Communities statuses.
– execute the command: mdsenv; fwm mds rebuild_global_communities_status all

1. Get the export tools

# cd /var/opt/export_tool/
# ftp 10.10.10.64
Connected to 10.10.10.64 (10.10.10.64).
220 Welcome to Baby FTP Server
Name (10.10.10.64:admin): anonymous
331 User name ok, need password.
Password:***
230 User logged in.
Remote system type is UNIX.
ftp> bin
200 Type set to I.
ftp> get SPLAT_p1_upgrade_tools.tgz
local: SPLAT_p1_upgrade_tools.tgz remote: SPLAT_p1_upgrade_tools.tgz
227 Entering Passive Mode (10,10,10,64,249,6).
150 Opening BINARY mode data connection for file transfer.
226 Transfer complete
180862 bytes received in 0.0313 secs (5.6e+03 Kbytes/sec)
ftp> quit
220 Goodbye.
# tar -xzvf SPLAT_p1_upgrade_tools.tgz

2. Export the global properties on source Management

If you analyse the script you can see that it copies the content of the /opt/CPmds-R70/conf/ folder to a temp folder and archive it to the destination folder. See the part from the export_database script:

# cd /var/opt/export_tool/export_tools/
# less ./export_database.sh
...
# the database structure will be created under $FWDIR/tmp/export_db
createTempDirStructure()
{
# Target directory structure is being built in $FWDIR/tmp which is always located on /var (for MDS, CMA and SMC)
# where we expect to have enough space
tempDir=$FWDIR/tmp
targetDirBaseName=export_db
targetDir=$tempDir/$targetDirBaseName
if [ -d $targetDir ]; then
$RM -rf $targetDir
fi
$MKDIR $targetDir# copy FWDIR files
$MKDIR $targetDir/conf
$CPR $FWDIR/conf/* $targetDir/conf 1>>$debugFile 2>&1
# includes lists, vpe, crls, cpsc, ahclientd, SMC_Files (handled specifically in migrate_assist)
# and local.arp (handled in SMC export)
…

If we start the script with bash -x ,we can see what it exactly does:

# cd /var/opt/export_tool/export_tools/
# bash -x ./export_database.sh /var/log/CMA_EXPORT –g

+ tempDir=/opt/CPmds-R70/tmp
+ targetDirBaseName=export_db
+ targetDir=/opt/CPmds-R70/tmp/export_db
+ '[' -d /opt/CPmds-R70/tmp/export_db ']'
+ /bin/mkdir /opt/CPmds-R70/tmp/export_db
+ /bin/mkdir /opt/CPmds-R70/tmp/export_db/conf
+ /bin/cp -rL /opt/CPmds-R70/conf/CPMIFtpExCache.C ..list of files is trimmed.. /opt/CPmds-R70/conf/wf_tagged_objects.C /opt/CPmds-R70/tmp/export_db/conf

Lets export the global policies:

# mkdir /var/log/CMA_EXPORT
# cd /var/log/CMA_EXPORT
# pwd
/var/log/CMA_EXPORT
# cd /var/opt/export_tool/export_tools/
# ./export_database.sh /var/log/CMA_EXPORT -g
Your source management should NOT be running while you export its database
because unsaved changes could be lost or log file corruption may occur.
Do you want to continue [yes/no] ? yes
Exporting global database...
Output file /var/log/CMA_EXPORT/exported_global_db.28Oct2011-155805.tgz was created successfully

Comment:Even if with mdsstop command we stop the mds and the cmas it give us this message… “Your source management should NOT be running while you export its database because unsaved changes could be lost or log file corruption may occur.” What should I stop now??

3.  Export the cma on source MDS

If you analyze the script you can see that it copies the content of /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/conf/ folder to a temp folder and archive it to the destination folder. If we start the script with bash -x,we can see what ti exactly does:

# cd /var/opt/export_tool/export_tools/
# bash -x ./export_database.sh /var/log/CMA_EXPORT -c mycma01
…
+ tempDir=/opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/tmp
+ targetDirBaseName=export_db
+ targetDir=/opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/tmp/export_db
+ '[' -d /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/tmp/export_db ']'
+ /bin/mkdir /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/tmp/export_db
+ /bin/mkdir /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/tmp/export_db/conf
+ /bin/cp -rL /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/conf/AMT.conf ..list of files is trimmed.. /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/conf/wf_tagged_objects.C /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/tmp/export_db/conf

Lets export the cma policies

# cd /var/opt/export_tool/export_tools/
# ./export_database.sh /var/log/CMA_EXPORT -c mycma01

Your source management should NOT be running while you export its database
because unsaved changes could be lost or log file corruption may occur.
Do you want to continue [yes/no] ? yes
Do you wish to export the log database [yes/no] ? no
Do you wish to export the SmartMap database [yes/no] ? no
Exporting CMA mycma01 database...
Output file /var/log/CMA_EXPORT/exported_mycma01.28Oct2011-153404.tgz was created successfully

4. Import the cma on target MDS

Install the same Version of Management

Install R70 and R70.40 R70 installs only the following Blades:

  • R70.40
  • IPS Event Analysis Blade
  • Workflow Blade
  • CPSG 80 Series Blade
  • SmartProvisioning support for IP Appliances

And installs the following packages:

  • Security Gateway / Security Management R70.40
  • NGX Compatibility Package R70.40
  • UTM-1 Edge compatibility Package R70.40
  • VSX NGX Compatibility Package R70.40
  • V40 Compatibility Package R70.40
  • MDS R70.40
  • CPSG 80 Series compatibility package R70.40 package
  • SecurePlatform R70.40

R70.40 updates these components:

**********************************************************
Welcome to Check Point R70.40 Installation
***********************************************************
The following components will be installed:
* R70.40
* IPS Event Analysis Blade
* Workflow Blade
* CPSG 80 Series Blade
* SmartProvisioning support for IP Appliances
Installation Application is about to stop all Check Point Processes.
Do you wish to continue (y/n) [y] ? y
stopping Check Point Processes...
Installing Security Gateway / Security Management R70.40...Done!
Installing NGX Compatibility Package R70.40...Done!
Installing UTM-1 Edge compatibility Package R70.40...Done!
Installing VSX NGX Compatibility Package R70.40...Done!
Installing V40 Compatibility Package R70.40...Done!
Installing MDS R70.40...Done!
Installing CPSG 80 Series compatibility package R70.40 package...Done!
Installing SecurePlatform R70.40... Done!

*******************************************************************************
The updated Inspect files have been installed successfully.
To complete the installation, please re-install the Security Policy on all
your gateways for the CMAs.
*******************************************************************************

Installing IPS Event Analysis Blade package... Done!
Installing Workflow Blade package...Done!
Installing CPSG 80 Series Blade package...Done!
Installing SmartProvisioning support for IP Appliances package...Done!

To complete initial configuration follow the next steps after reboot:
1. Install the IPS Event Analysis Blade / Event Correlation Blade on a dedicated server.
2. Connect with SmartDashboard, create and configure the new host to run the IPS Event Analysis / Event Correlation server.
3. Connect with the IPS Event Analysis / Eventia Analyzer Client, configure Correlation Units and install event policy.
Full installation instructions are provided in the Release Notes.

************************************************************************
Package Name                                                    Status
------------                                                    ------
Security Gateway / Security Management R70.40                   Succeeded
NGX Compatibility Package R70.40                                Succeeded
UTM-1 Edge compatibility Package R70.40                         Succeeded
VSX NGX Compatibility Package R70.40                            Succeeded
V40 Compatibility Package R70.40                                Succeeded
MDS R70.40                                                      Succeeded
CPSG 80 Series compatibility package R70.40                     Succeeded
SecurePlatform R70.40                                           Succeeded
IPS Event Analysis Blade                                        Succeeded
Workflow Blade                                                  Succeeded
CPSG 80 Series Blade                                            Succeeded
SmartProvisioning support for IP Appliances                     Succeeded

************************************************************************

Installation Program Completed Successfully.
Do you wish to reboot your machine (y/n) ?

Comments:What are the differences between a component and a plugin and a package and a blade? :-)

Create the customer and the cma

Without starting the cma. See Problem 3 if you start it.

If the CMA contains higher LIBSW version for th VPN-1 or UTM-1 Edge-s, than this must first upgraded to the version the source CMA has!

Here is the link for that:

https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk31448

an article in case you install it for more CMA within the same Provider1

http://sofaware.infopop.cc/eve/forums/a/tpc/f/6406072361/m/5011005663

# mdscmd addcustomer mycustomer01 -n mycma01 -i 10.10.10.13
mycustomer01 was updated successfully
mycma01_._._mycustomer01 was updated successfully
#

Migrate the configuration

With scp upload the tgz file to the target CMA to /var/tmp.

# cma_migrate /var/tmp/exported_mycma01.28Oct2011-153404.tgz /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1

Are you sure you want to migrate the management at /var/tmp/exported_mycma01.28Oct2011-153404.tgz
into the CMA /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1 [yes/no] ? yes

Verifying data before importing. Please wait ...
Source management version detected:
R70
======================================================================
>>> Executing Source Version Upgrade Path Checker

======================================================================
>>> Executing Plug-in Mismatch Detector

======================================================================
>>> Executing License Upgrade Tool

======================================================================
>>> Executing Global Encryption Rule Detector

======================================================================
>>> Executing Global Policy on Source Database Detector

======================================================================
>>> Executing Multiple CMAs with the same ICA Keys Detector

======================================================================
>>> Executing Default Global Services Inconsistency Detector

======================================================================
>>> Executing Globally Used Gateways Customer Name Inconsistency Detector

======================================================================
>>> Executing Globally Used Gateways Names Inconsistency Detector

======================================================================
>>> Executing Firmware References Detector

======================================================================
>>> Executing SmartCenter Pre Upgrade Verifier

Pre-migrate verification ended successfully.
A log file was created: /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/log/pre_migrate.log

Proceeding with migration.

Starting the migration process
Source management version detected:
R70
Starting the migration process
Migrating files from /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/tmp//export_db to /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1 ...
Removing non-required log-related files
Merging cp.license files using 'cplic put' command:
/opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/conf/vip_index.conf.tmp will be saved as /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/conf/vip_index.conf.tmp.premigrate
Setting updated IP address...
Setting SIC in registry
Converting CA database format...
Setting FQDN...
Command is: /opt/CPmds-R70/customers/mycma01/CPshrd-R70/bin/cp_conf ca fqdn 10.10.10.13
Your previous fwopsec.conf file was copied to fwopsec.conf.user. If it contains manual adjustments, please update them in the new fwopsec.conf
	 Attribute 'hosted_by_mds' repaired in object 'mycma01'. 

Error:   Failed to update CMA database according to the installed Plug-ins.

Migration completed.
[Expert@provider1]#
[Expert@provider1]# mdsstat
+--------------------------------------------------------------------------------------+
|                              Processes status checking                               |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| Type| Name            | IP address      | FWM       | FWD      | CPD      | CPCA     |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| MDS |        -        | 10.10.10.15    | up 2743   | up 2742  | up 2741  | up 3249  |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| CMA |cma1             | 10.10.10.33    | up 2729   | up 2728  | up 2718  | up 3247  |
| CMA |mycma01          | 10.10.10.13    | down      | down     | down     | down     |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| Total customer add-ons checked: 2     1 up   1 down                                  |
| Tip: Run mdsstat -h for legend                                                       |
+--------------------------------------------------------------------------------------+
[Expert@provider1]#
[Expert@provider1]# mdsstart
Adding Virtual IPs ..
Number of customers started so far: 1 out of 2
Number of customers started so far: 2 out of 2

MDS: Starting MDS Server
[1] 8612
[2] 8613
[3] 8614
MDS Server Started

[Expert@provider1]# mdsstat
+--------------------------------------------------------------------------------------+
|                              Processes status checking                               |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| Type| Name            | IP address      | FWM       | FWD      | CPD      | CPCA     |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| MDS |        -        | 10.10.10.15    | up 2743   | up 2742  | up 2741  | up 3249  |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| CMA |cma1             | 10.10.10.33    | up 2729   | up 2728  | up 2718  | up 3247  |
| CMA |mycma01..        | 10.10.10.13    | up 8585   | up 8584  | up 8563  | up 8677  |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| Total customer add-ons checked: 2     2 up   0 down                                  |
| Tip: Run mdsstat -h for legend                                                       |
+--------------------------------------------------------------------------------------+
[Expert@provider1]#

Delete a Customer and its CMA

This step is absolutely not part of the migration process, I just used it many times till I got a working migrated CMA…

Expert@provider1]# mdsquerydb Customers
mycustomer01
mycustomer02
 [Expert@provider1]#
[Expert@provider1]# mdscmd deletecustomer mycustomer02
mycustomer02 was deleted successfully

CMA Migration Problems

1. Problem 1. – Plug-in Mismatch Detector issue

After issuing the cma_migrate command the Plug-in Mismatch Detector finds an error and stops the migration process:

======================================================================
>>> Executing Plug-in Mismatch Detector

Error:   Some plug-ins exist only on the source database. Plug-In Mismatch.

Error:   The source database and the target CMA do not share the same plug-ins
----------------------------------------------------------------------
Plug-in Mismatch Detector completed with errors.

======================================================================

1.1. Trial1 – copy the source CMAs HKLM_registry.data file

What is the HKLM_registry.data?

The official Checkpoint documentation mentions not too much about this file. In the “architecture and Processes” Section of the R70 Provider-1 pdf we can read the followings:

Check Point Registry

Information related to the installation and versioning issues of different components that is requested by different Check Point processes, is centrally stored in a registry file. The registry is stored in $CPDIR/registry/HKLM_registry.data (where the value of CPDIR environment variable is different whether you are in the MDS environment or whether you are in different CMA environments. This means that there are separate registry files for the MDS and for the CMAs. ”

Or there is an old article for this file: skI3336

Path to the file: $CPDIR/registry/HKLM_registry.data

You can modify this file with the ckp_regedit command. There is an article about a special case where it was required to use it. Here is the link: http://3-4-5-6.blogspot.com/search/label/Checkpoint%20registry%20moification

As this file contains informations about the plugins I copy the source CMAs HKLM_registry.datafile to the target CMA and try the cma_migrate again, but make a backup of it first.

# cp /var/opt/CPmds-R70/customers/mycma01/CPshrd-R70/registry/HKLM_registry.data /var/opt/CPmds-R70/customers/mycma01/CPshrd-R70/registry/HKLM_registry.data_backup
# cp /var/tmp/export_db/registry/HKLM_registry.data /var/opt/CPmds-R70/customers/mycma01/CPshrd-R70/registry/
#

Result:

[Expert@provider1]# cma_migrate /var/tmp/exported_mycma01.28Oct2011-153404.tgz /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1

Are you sure you want to migrate the management at /var/tmp/exported_mycma01.28Oct2011-153404.tgz
into the CMA /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1 [yes/no] ? yes

Verifying data before importing. Please wait ...
Source management version detected:
R70
======================================================================
>>> Executing Source Version Upgrade Path Checker

======================================================================
>>> Executing Plug-in Mismatch Detector

======================================================================
>>> Executing License Upgrade Tool

======================================================================
>>> Executing Global Encryption Rule Detector

======================================================================
>>> Executing Global Policy on Source Database Detector

======================================================================
>>> Executing Multiple CMAs with the same ICA Keys Detector

======================================================================
>>> Executing Default Global Services Inconsistency Detector

======================================================================
>>> Executing Globally Used Gateways Customer Name Inconsistency Detector

======================================================================
>>> Executing Globally Used Gateways Names Inconsistency Detector

======================================================================
>>> Executing Firmware References Detector

======================================================================
>>> Executing SmartCenter Pre Upgrade Verifier

Pre-migrate verification ended successfully.
A log file was created: /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/log/pre_migrate.log

Proceeding with migration.

Starting the migration process
Source management version detected:
R70
Starting the migration process
Migrating files from /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/tmp//export_db to /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1 ...
Removing non-required log-related files
Merging cp.license files using 'cplic put' command:
/opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/conf/vip_index.conf.tmp will be saved as /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/conf/vip_index.conf.tmp.premigrate
Setting updated IP address...
Setting SIC in registry
Converting CA database format...
Setting FQDN...
Command is: /opt/CPmds-R70/customers/mycma01/CPshrd-R70/bin/cp_conf ca fqdn 10.10.10.13
Your previous fwopsec.conf file was copied to fwopsec.conf.user. If it contains manual adjustments, please update them in the new fwopsec.conf
         Attribute 'hosted_by_mds' repaired in object 'mycma01'.

Error:   Failed to update CMA database according to the installed Plug-ins.

The Migration is just partly ok, the fwm process cannot start:

[Expert@provider1]# mdsstat
+--------------------------------------------------------------------------------------+
|                              Processes status checking                               |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| Type| Name            | IP address      | FWM       | FWD      | CPD      | CPCA     |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| MDS |        -        | 10.10.10.15    | up 2718   | up 2717  | up 2716  | up 3165  |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| CMA |mycma01          | 10.10.10.13    | down      | up 4795  | up 4785  | up 4816  |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| Total customer add-ons checked: 1     0 up   1 down                                  |
| Tip: Run mdsstat -h for legend                                                       |
+--------------------------------------------------------------------------------------+
1.1.1 Problem 2. – fwm process cannot start

The fwm process does not want to start and I get the following message:

# mdsstat
+--------------------------------------------------------------------------------------+
|                              Processes status checking                               |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| Type| Name            | IP address      | FWM       | FWD      | CPD      | CPCA     |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| MDS |        -        | 10.10.10.15    | up 2818   | up 2817  | up 2816  | up 3361  |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| CMA |mycma01          | 10.10.10.33    | down      | up 12365 | up 12345 | up 12428 |
| CMA |mycma02          | 10.10.10.13    | up 2771   | up 2769  | up 2750  | up 3337  |
+-----+-----------------+-----------------+-----------+----------+----------+----------+
| Total customer add-ons checked: 2     1 up   1 down                                  |
| Tip: Run mdsstat -h for legend                                                       |
+--------------------------------------------------------------------------------------+
1.1.2 Trial2 – troubleshoot sofaware management process

Actually this log message I got only if I tried to migrate a CMA with Edge configurations but for CMAs without Edge Firewall configuration I haven’t seen such error messages (But another error message was there and the fwm process was down). The error message I see now in fwm.elg was the following:

15000: Can't contact database'

And I saw that the sms process could not start, the watchdog tried to start it without no success. Expected processes:

/bin/sh /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/bin/smsstart_wd
 \_ /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/bin/sms -confdir /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/conf/sofaware

But experienced:

/bin/sh /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/bin/smsstart_wd
 \_sleep 5

I googled first with the error message and found actually nothing useful, just an SK – sk33168 and an unresolved issue on cpug forum. I tried troubleshooting the sms process according to the Understanding_SofaWare_Management_Modules.pdf:

1. Edit $FWDIR/conf/sofaware/SWManagementServer.ini with vi editor

2. Change in line containing LogPolicy1 the value Info to Debug

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
LogPolicy1="console://localhost/severity=Info"
LogPolicy2="debugger://localhost/severity=Debug"
LogPolicy3="local://localhost/severity=Critical & origintype=SMS"

3. Restart the sms process

# smsstop && smsstart

4. Replicate the problem and watch for console output.

Result:

Does not give more info.

1.1.3 Trial3 – missing folders in target cma

The source CMA has more plugins/packages – or whatever it is called – installed as the target CMA or at least I see more folders. Source CMA:

[Expert@iumg028]# ls -lat /opt/CPmds-R70/customers/mycma01/ | wc -l
     16
[Expert@iumg028]# ls -la /opt/CPmds-R70/customers/mycma01/
total 60
drwxrwx---   15 root     root         4096 Dec  2  2010 .
drwxrwx---   17 root     root         4096 Oct 25 15:41 ..
drwxrwx---   10 root     root         4096 Dec  2  2010 CPCON62CMP-R70
drwxrwx---   10 root     root         4096 Dec  2  2010 CPCON66CMP-R70
drwxrwx---   10 root     root         4096 Dec  2  2010 CPEdgecmp-R70
drwxrwx---   10 root     root         4096 Feb 10  2011 CPNGXCMP-R70
drwxr-xr-x    2 root     root         4096 Dec  2  2010 CPPIeventiaR70-R70
drwxr-xr-x    2 root     root         4096 Dec  2  2010 CPPIprov-R70
drwxr-xr-x    2 root     root         4096 Dec  2  2010 CPPIsg80-R70
drwxr-xr-x    2 root     root         4096 Dec  2  2010 CPPIworkflowR70-R70
drwxr-xr-x    5 root     root         4096 Feb  8  2011 CPSG80CMP-R70
drwxrwx---   10 root     root         4096 Dec  2  2010 CPV40Cmp-R70
drwxrwx---    8 root     root         4096 Nov  2 10:33 CPshrd-R70
drwxrwx---    4 root     root         4096 Dec  2  2010 CPsuite-R70
drwxrwx---   10 root     root         4096 Dec  2  2010 CPvsxngxcmp-R70

Target CMA

[Expert@provider1]# ls -lat /opt/CPmds-R70/customers/mycma01/ | wc -l
     11
[Expert@provider1]# ls -la /opt/CPmds-R70/customers/mycma01/
total 40
drwxr-xr-x   10 root     root         4096 Nov  3 15:28 .
drwxrwx---    3 root     root         4096 Nov  3 15:28 ..
drwxr-xr-x   10 root     root         4096 Nov  3 15:28 CPCON62CMP-R70
drwxr-xr-x   10 root     root         4096 Nov  3 15:28 CPCON66CMP-R70
drwxr-xr-x   10 root     root         4096 Nov  3 15:28 CPEdgecmp-R70
drwxr-xr-x   10 root     root         4096 Nov  3 15:28 CPNGXCMP-R70
drwxr-xr-x   10 root     root         4096 Nov  3 15:28 CPV40Cmp-R70
drwxr-xr-x    7 root     root         4096 Nov  3 15:28 CPshrd-R70
drwxr-xr-x    4 root     root         4096 Nov  3 15:28 CPsuite-R70
drwxr-xr-x   10 root     root         4096 Nov  3 15:28 CPvsxngxcmp-R70

Missing folders:

drwxr-xr-x    2 root     root         4096 Dec  2  2010 CPPIeventiaR70-R70
drwxr-xr-x    2 root     root         4096 Dec  2  2010 CPPIprov-R70
drwxr-xr-x    2 root     root         4096 Dec  2  2010 CPPIsg80-R70
drwxr-xr-x    2 root     root         4096 Dec  2  2010 CPPIworkflowR70-R70
drwxr-xr-x    5 root     root         4096 Feb  8  2011 CPSG80CMP-R70

1.2 Trial4 – analyze the cma_migrate script

The cma_migrate script can be debugged with the bash -x /opt/CPmds-R70/scripts/cma_migrate command. “bash -x” displays commands and their arguments as they are executed. Another way would be to use trap. See link for more information about trap and script debugging in Linux: http://linuxtopia.org/online_books/advanced_bash_scripting_guide/debugging.html

Suspicious called scripts, commands:

/opt/CPmds-R70/system/pre_upgrade/mds_pre_upgrade.sh – This calls the so called PUs (Upgrade checking processes).

In the cma_migrate script (in pre_migrate_tests function) we can change the way the mds_pre_upgrade.sh will be called:

pre_migrate_tests()

        $RM $pulog
    fi
    LOG_FILE=$pulog
    export LOG_FILE
    bash -x ./mds_pre_upgrade.sh
    case $? in
        $PU_EXIT_VALUE_OK)
            EchoNLog "Pre-migrate verification ended successfully."

And the first error message comes at the PU00030:

...
+ /bin/echo -e '>>> Executing Plug-in Mismatch Detector'
>>> Executing Plug-in Mismatch Detector
++ /bin/basename /opt/CPmds-R70/system/pre_upgrade/PU00030
+ cur_puv_id=PU00030
+ cur_log_file=/opt/CPInstLog/pre_upgrade/pu.log.PU00030
+ '[' -f /opt/CPInstLog/pre_upgrade/pu.log.PU00030 ']'
+ env LOG_FILE=/opt/CPInstLog/pre_upgrade/pu.log.PU00030 /opt/CPmds-R70/system/pre_upgrade/PU00030/execute

Error: Some plug-ins exist only on the source database. Plug-In Mismatch.

Error: The source database and the target CMA do not share the same plug-ins
+ exitCode=20
+ '[' -f /opt/CPInstLog/pre_upgrade/pu.log.PU00030 ']'
+ /bin/cat /opt/CPInstLog/pre_upgrade/pu.log.PU00030
...

The Plug-in Mismatch Detector is here in execute file:

# cd /opt/CPmds-R70/system/pre_upgrade/PU00030/
# ls -alt
total 28
drwxr-x---   18 root     bin          4096 Oct 31 22:28 ..
-rwxrwx---    1 root     root            7 Jul 21 22:23 context
-rwxrwx---    1 root     root           25 Jul 21 22:23 title
drwxr-x---    2 root     bin          4096 Jul 19  2010 .
-rwxr-x---    1 root     bin          9325 Jul 19  2010 execute

It checks with the following command the installed products on source CMA and the target CMA:

# mdsenv < source CMA>
# cpprod_util CPPROD_GetInstalledPlugIns
PIeventiaR70
PIprovR70
PIsfw
PIworkflowR70
#

cpprod_util works with $CPDIR/registry/HKLM_registry.data file that contains the plugin list as well and it find some differencies in the number of plugins between the source and the target CMA. Lets see it if we find something with the following steps:

1. Delete and create the Customer and CMA again

2. Check the plugin list and compare it with the live system

Result: The number of plugins in a new CMA is always 0, but on the live system it is4.

Check what sees the /opt/CPmds-R70/system/pre_upgrade/PU00030/execute:

Add the followings Control Points to the script:

  #Remove Plug-ins that are already merged into the current version.
  MERGED_PLUGINS="PIvsx PIutm1 PIprov PIconR66-R65 PIconnectra"
  # TODO: PIvoip is yet to be merged
  #After FLO EA PIconnectra must also be added to the list.
  for extra_plugin in $MERGED_PLUGINS
  do
        $GREP -v $extra_plugin /tmp/orig_plugins.txt > /tmp/tmp_plugins.txt
        $CP /tmp/tmp_plugins.txt /tmp/orig_plugins.txt
  done

  NUM_TARGET_PLUGINS=`cat /tmp/target_plugins.txt | wc -l`
  NUM_SOURCE_PLUGINS=`cat /tmp/orig_plugins.txt | wc -l`

 echo "Here is my debug part 1."
 echo "target plugins"
 cat /tmp/target_plugins.txt
 echo "orig plugins"
 cat /tmp/orig_plugins.txt

  if [ $NUM_TARGET_PLUGINS -eq 0 -a $NUM_SOURCE_PLUGINS -eq 0 ] ; then
    ret_val=$res_ok
  else

        if [ $NUM_TARGET_PLUGINS -ne $NUM_SOURCE_PLUGINS ] ; then
            ret_val=$res_plugin_mismatch
        else

And in the output we see that the target CMA has no plugins but the source has.but if you create a CMA but do not start it then it will never show with the cpprod_util any plugin, it is maybe a BUG?

++ /bin/basename /opt/CPmds-R70/system/pre_upgrade/PU00030
+ cur_puv_id=PU00030
+ cur_log_file=/opt/CPInstLog/pre_upgrade/pu.log.PU00030
+ '[' -f /opt/CPInstLog/pre_upgrade/pu.log.PU00030 ']'
+ env LOG_FILE=/opt/CPInstLog/pre_upgrade/pu.log.PU00030 /opt/CPmds-R70/system/pre_upgrade/PU00030/execute
Here is my debug part 1.
target plugins
orig plugins
PIeventiaR70
PIsfw
PIworkflowR70

Error:   Some plug-ins exist only on the source database. Plug-In Mismatch.

Error:   The source database and the target CMA do not share the same plug-ins
+ exitCode=20

As a workaround disable the Plug-in Mismatch Detector and try it again:

# cd /opt/CPmds-R70/system/pre_upgrade/
# mv PU00030 /var/tmp
# cma_migrate /var/tmp/exported_mycma01.28Oct2011-153404.tgz /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1

Migration completed.

The Migration is ready! One more step and we are ready. Update in Provider1 GUI your Customer:

After activating R70.40 Provider1 restarts the Customer and sets the plugins. Some examples for the processes are here:
PIprovR70 plugin:

/opt/CPmds-R70/bin/fwm mds
  \_ /bin/sh /opt/CPmds-R70/scripts/plugin_cmd activate PIprovR70 /tmp/plugin_cmd.cmas.B768CF
      \_ /opt/CPPIprov-R70/bin/uacRunner -p PIprovR70 -install
          \_ /opt/CPmds-R70/customers/mycma02/CPsuite-R70/fw1/bin/cpdb plugin_install --src_type legacy --no-override --default_path pluginDefault

.

/opt/CPmds-R70/bin/fwm mds
  \_ /bin/sh /opt/CPmds-R70/scripts/plugin_cmd activate PIprovR70 /tmp/plugin_cmd.cmas.B768CF
      \_ /bin/sh /opt/CPmds-R70/scripts/plugin_cmd activate PIprovR70 /tmp/plugin_cmd.cmas.B7
          \_ /bin/echo -e /opt/CPmds-R70/lib:/opt/CPmds-R70/bin:/bin:/usr/bin:/opt:/usr/etc:/

PIsfw plugin:

/opt/CPmds-R70/bin/fwm mds
  \_ /bin/sh /opt/CPmds-R70/scripts/plugin_cmd activate PIsfw /tmp/plugin_cmd.cmas.B768CFAA-0
      \_ /opt/CPPIsg80-R70/bin/uacRunner -p PIsfw -install
          \_ /opt/CPmds-R70/customers/mycma02/CPshrd-R70/bin/CPRegSvr /f /opt/CPm

.

/opt/CPmds-R70/bin/fwm mds
  \_ /bin/sh /opt/CPmds-R70/scripts/plugin_cmd activate PIsfw /tmp/plugin_cmd.cmas.B768CFAA-0
      \_ /opt/CPPIsg80-R70/bin/uacRunner -p PIsfw -install
          \_ /opt/CPmds-R70/customers/mycma02/CPsuite-R70/fw1/bin/cpdb plugin_ins

PIworkflowR70 plugin:

/opt/CPmds-R70/bin/fwm mds
  \_ /bin/sh /opt/CPmds-R70/scripts/plugin_cmd activate PIworkflowR70 /tmp/plugin_cmd.cmas.B7
      \_ /opt/CPPIworkflowR70-R70/bin/uacRunner -p PIworkflowR70 -install
          \_ /opt/CPmds-R70/customers/mycma02/CPsuite-R70/fw1/bin/cpdb plugin_ins

2. Problem 3. – If you start the CMA before migration

# cma_migrate /var/tmp/exported_mycma02.31Oct2011-205249.tgz /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1

Are you sure you want to migrate the management at /var/tmp/exported_mycma02.31Oct2011-205249.tgz
into the CMA /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1 [yes/no] ? yes

Verifying data before importing. Please wait ...
Source management version detected:
R70
======================================================================
>>> Executing Source Version Upgrade Path Checker

======================================================================
>>> Executing Plug-in Mismatch Detector

======================================================================
>>> Executing License Upgrade Tool

======================================================================
>>> Executing Global Encryption Rule Detector

======================================================================
>>> Executing Global Policy on Source Database Detector

Warning: Global policy was detected on the source database:
         /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/tmp//export_db
         The CMA after migration will have global rules and/or objects, but
         there will be no indication that a global policy is assigned on it.
         After migration, in order to have a global policy assigned on this
         customer, use 'Assign Global Policy'.

Todo:    Stop the migration process and remove the global policy from the
         source database, or continue with the migration process.
----------------------------------------------------------------------
Global Policy on Source Database Detector completed with warnings

======================================================================
>>> Executing Multiple CMAs with the same ICA Keys Detector

======================================================================
>>> Executing Default Global Services Inconsistency Detector

======================================================================
>>> Executing Globally Used Gateways Customer Name Inconsistency Detector

======================================================================
>>> Executing Globally Used Gateways Names Inconsistency Detector

======================================================================
>>> Executing Firmware References Detector

======================================================================
>>> Executing SmartCenter Pre Upgrade Verifier

Data verification warnings were found.
Would you like to continue [yes/no] ? yes
A log file was created: /opt/CPmds-R70/customers/mycma01/CPsuite-R70/fw1/log/pre_migrate.log

Proceeding with migration.

Error:   The CMA has already been started once.
         The migrate operation can take place only on a CMA which
         has not been started yet.

Migration not completed.

That was  my first CMA migration, it can be that something is not clearly documented or done :-)

Advertisements