Upgrade

Upgrading to SVN MultiSite Plus 1.2
For Red Hat Enterprise Linux (RHEL 6) deployments that are moving from version 1.0 Build 5778 or earlier to the latest version (1.2): You should first complete the following procedure for an upgrade to version 1.0 Build 6383). Once you have verified the upgrade was successful, only then should you repeat the upgrade with the 1.2 files. If you have any questions or concerns contact WANdisco's support team.

Before you begin - IMPORTANT
Run these instructions on each node one at a time -- don't attempt to run them simultaneously. The process would still complete successfully, although you are likely to see pending transactions that won't ever complete.

Although you can update to the latest version of SVN MultiSite Plus by completing a fresh installation (using the installation guide), you would need to recreate your configuration in terms of node, repository and replication group settings. This upgrade procedure offers a method for upgrading while keeping your established setup.

Overview

There are a number of different upgrade procedures, you need to make sure that you use the one that matches your current installed version of SVN MultiSite Plus.

Before you upgrade

Recheck the deployment checklist.
Back up existing Apache config files - Once the upgrade is completed you should verify that it hasn't made Apache configuration changes that will adversely affect your operation.
You shouldn't make any system changes like adding new repositories until the upgrade has been completed.

Select an upgrade path:

If you are upgrading from

Upgrading to SVN MultiSite Plus 1.2 Build: 6903 to later
Upgrading to SVN MultiSite Plus build: 5435 to 5778 (prior to 6903)
Upgrading to SVN MultiSite Plus Build: 5041 or earlier

Which version am I running?
The version number of each SVN MultiSite Plus component is tracked. For distinguishing between product releases we use the svn-ms-replicator version. You can find it in the following places:

Listed at the bottom of the admin UI.
Listed at the bottom of the Settings tab of the admin UI (along with other component versions).
Without starting up and accessing the admin UI you can see the version in the following file:
```
/opt/wandisco/svn-multisite-plus/VERSION
```

Roll-back

In the unlikely even that the upgrade leaves you in a state where you need to return to your orginal setup, see the Roll-back procedure at the end of this document

Upgrades from build: 6903 and later

Open a terminal window and login to the server.

Get the latest installer file and make sure it is executable:
```
chmod a+x svn-multisite-plus.sh
    
```

Run the installer:

   Verifying archive integrity... All good.
Uncompressing WANdisco SVN MultiSite Plus........................
Running in non-interactive mode, installing with user 'wandisco' and group 'wandisco'.
Output will be logged to the daemon.info syslog facility
SVN MultiSite Plus Pre-Upgrade Backup Script

Please enter the username of an administrative user:

As SVN MultiSite Plus is already installed and running, the installer has determined that you wish to perform an upgrade instead of an initial installation.
Enter the SVN MultiSite Plus admin credentials - username then password.

Please enter password for the 'admin' administrative user:

Once authenticated as an admin user the process will provide instructions on what to do next, which will depend on whether this is the first node to be upgraded or a subsequent node.

      This process must be run on ALL nodes individually, in sequence. NOT AT THE SAME TIME If this
      is the first node we will use this node to co-ordinate the upgrade.
      Is this the first node? y/N: [N] > y

As this is the first upgrade, enter "y".

    Initiating synchronised stop...
Node has achieved synchronised stop
Initiating database dump...
Database dump complete
Stopping ui:.                                              [  OK  ]
Stopping replicator:.                                      [  OK  ]
Performing backup...
Backup has been stored in /opt/wandisco/svn-multisite-plus/var/backups

Backup process logged at /opt/wandisco/svn-multisite-plus/logs/backup.log

Repeat the procedure for all nodes, remembering to answer "N" to the question whether this is the first node you are upgrading. The subsequent upgrades will use your first upgraded node as a reference point for some settings.

Updates between SVN MultiSite version with different database structures

When updating to a version of SVN MultiSite Plus that contains a serialisation change, you will need to perform an additional transform step to modify the backup data before it's reimported using the restore procedure.

Extract that backup archive that was generated during the upgrade process . e.g.
```
cd /opt/wandisco/svn-multisite-plus/var/backups/
tar -xvf backup-NodeParis-20140123-154117.tar.gz
```
Once extracted (in situ) you will have the backup file available to transform. The file would be located here:
```
/opt/wandisco/svn-multisite-plus/var/backups/replicator/database/backup/
```

Use the transformer-tool.jar found here:

/opt/wandisco/svn-multisite-plus/replicator/transformer-tool.jar

You run it with the following command.

java -jar transformer-tool.jar <PATH TO BACKUP>

Note that the path to the backup is the path to the extracted backup file, e.g.

/opt/wandisco/svn-multisite-plus/var/backups/replicator/database/backup/20140114.154112.933_DConE_Backup

Once the transform has been completed run the restore tool

svn-ms-replicator-fsfsrestore.jar -c AppPropertiesFile -r <PATH TO BACKUP>

Repeat for each node

Now repeat the process for each of your other nodes in turn.

Restart

Before you begin - IMPORTANT
Restart nodes ONLY after all nodes have been successfully upgraded.

Restart each node . See Starting Up

If the synchronized stop failed in the previous step:

Ensure that no corresponding repositories are stuck in Local Read-only mode.
Ensure that there's at least one defined replication group on the nodes -- a requirement for a synchronized stop.
Reset all nodes to clean out any pending proposals then retrying the upgrade.

Upgrading from earlier versions

Upgrades from build: 5435 to 5778 (prior to 6903)

SSL Upgrades - IMPORTANT
SSL upgrades from build 5435 will fail with the error:
"Unable to determine application version numbers, curl returned 60", Issue NV-2273.
There is a simple workaround: Open the backup script at svn-multisite-plus/bin/backup Modify line 27 by adding a -k after the -f, i.e. from

alias curl='curl -f -u $USERNAME:$PASSWORD'

alias curl='curl -f -k -u $USERNAME:$PASSWORD'

This will work around the problem. The problem will be fixed in the next release.

Open a terminal window and login to the server.

Get the latest installer file and make sure it is executable:
```
chmod a+x svn-multisite-plus.sh
    
```

Run the installer:

    ./svn-multisite-plus.sh
Verifying archive integrity... All good.
Uncompressing WANdisco SVN MultiSite Plus.............
    ::   ::  ::     #     #   ##    ####  ######   #   #####   #####   #####
   :::: :::: :::    #     #  #  #  ##  ## #     #  #  #     # #     # #     #
  ::::::::::: :::   #  #  # #    # #    # #     #  #  #       #       #     #
 ::::::::::::: :::  # # # # #    # #    # #     #  #   #####  #       #     #
  ::::::::::: :::   # # # # #    # #    # #     #  #        # #       #     #
   :::: :::: :::    ##   ##  #  ## #    # #     #  #  #     # #     # #     #
    ::   ::  ::     #     #   ## # #    # ######   #   #####   #####   #####




Checking SVN
SVN version 1.7.9-2 is already installed, will not install SVN


Beginning WANdisco SVN MultiSite Plus Installation...
Attempting to run backup on existing install of SVN MultiSite Plus...

As SVN MultiSite Plus is already installed and running, the installer will determine that you wish to perform an upgrade instead of an initial installation:

SVN MultiSite Plus Pre-Upgrade Backup Script

Please enter password for the 'admin' administrative user:

Enter the SVN MultiSite Plus admin credentials:

Initiating synchronised stop...
Node has achieved synchronised stop
Initiating database dump...
Database dump complete
Stopping ui:.                                              [  OK  ]
Stopping replicator:.                                      [  OK  ]
Performing backup...
Backup has been stored in /opt/wandisco/svn-multisite-plus/var/backups

Starting ui:[  OK  ]
Starting replicator:[  OK  ]

So the installer brings your nodes to a stop, then backs up the application data (repository data is not touched).

If the synchronized stop failed in the previous step:

Ensure that no corresponding repositories are stuck in Local Read-only mode.
Ensure that there's at least one defined replication group on the nodes -- a requirement for a synchronized stop.
Reset all nodes to clean out any pending proposals then retrying the upgrade.

With the local replicator restarted, login to the admin UI, click on the Settings tab. You can confirm that SVN MultiSite Plus has been updated by looking at the Module versions.

When each node has been updated (one at a time), navigate to the Nodes tab on the admin UI and wake up the nodes. You can do this on each individual node in turn or you can use the Start All button - 14. Starting nodes.
Before you begin - IMPORTANT
Restart nodes ONLY after all nodes have been successfully upgraded.

Upgrading from Build: 5041 and earlier

Before you begin - IMPORTANT
If Subversion changes are in-flight when the upgrade process is run there is a good chance that the state of each node will not match when you restart, which will prevent replication. It is essential that Subversion write-activity is stopped on all sites, and that enough time is allowed for any in-flight transactions to be distributed to all sites.

Make all Subversion repositories READ-ONLY - There are a number of ways you can do this, If you use authz, create a lock-down version of the file to ensure that all write permissions are removed, or you could create a pre-commit hook that blocks all commits.

Wait - Allow enough time for in-flight transactions to complete on all sites.

Verify - Just before beginning the upgrade, check the "youngest" revisions in your repositories and compare between your sites to make sure that all replicas are in the same state.

Preparing for the upgrade

Apply this section to all sites before you continue to the upgrade:

tip Same version must run on all sites
You should complete the upgrade on all your sites before restarting replication.

As with your initial deployment, ensure that all software is running at the same version on all sites. Running with different software versions may result in sites behaving slightly differently which could result in loss of sync between Subversion repository replicas.

Open a shell onto the first node's server. Ensure that you log in with appropriate privileges.

Archive your SVN MultiSite Plus directory,in case of trouble during the upgrade.

With your current installation safely copied, we'll begin by creating a backup of the configuration data by invoking the following API call:

curl --user <username>:<password> -X POST http://[node_ip_address]:8082/dcone/backup

e.g.

curl --user admin:password -X POST http://172.16.2.41:8082/dcone/backup
    
cd /opt/wandisco/svn-multisite-plus/replicator/database/

After running the backup, verify and make a note of the name of the backup file for later -you will need to reference it during the upgrade. Backups are stored here:

[Install-Directory]/svn-multisite-plus/replicator/database/backup

The backup subdirectory is created the first time you run a backup. Each time you run a backup a corresponding date-time stamped subdirectory is created in the backup directory. A typical backup will have the following file structure.

    
drwxr-xr-x 3 root root 4096 Jul 24 10:53 application
drwxr-xr-x 3 root root 4096 Jul 25 10:33 backup <- backup folder created when backup is first run
   
    drwxr-xr-x 4 root root 4096 Jul 25 10:33 20130713.103306.674_DConE_Backup <- per-backup folder   
        
        drwxr-xr-x 2 root root 4096 Jul 25 10:33 instance
         
            -rw-r--r-- 1 root root   9900 Jul 25 10:33 application_resettable.db-backup.xml 
            -rw-r--r-- 1 root root 226838 Jul 25 10:33 DConE.application.db-backup.xml
            -rw-r--r-- 1 root root  84107 Jul 25 10:33 recovery_application.integration.db-backup.xml
            -rw-r--r-- 1 root root 222815 Jul 25 10:33 recovery_DConE.system.db-backup.xml
            -rw-r--r-- 1 root root  31341 Jul 25 10:33 recovery_DConE.topology.db-backup.xml
   
        drwxr-xr-x 7 root root 4096 Jul 25 10:33 subsystem
        
            drwxr-xr-x 2 root root 4096 Jul 25 10:33 application_resettable.db
            drwxr-xr-x 2 root root 4096 Jul 25 10:33 DConE.application.db
            drwxr-xr-x 2 root root 4096 Jul 25 10:33 recovery_application.integration.db
            drwxr-xr-x 2 root root 4096 Jul 25 10:33 recovery_DConE.system.db
            drwxr-xr-x 2 root root 4096 Jul 25 10:33 recovery_DConE.topology.db
    
drwxr-xr-x 2 root root 4096 Jul 25 10:23 DConE.application.db
drwxr-xr-x 5 root root 4096 Jul 24 10:57 recovery

Shut down SVN MultiSite Plus.
Read-only
Shutting down MultiSite will make all of its replicated repositories read-only. Depending on how disruptive this will be, it may be wise to warn your local Subversion users and complete the process at a time when the outage will be least disruptive.
To shut down, run the following script:
```
[Install-Directory]/svn-multisite-plus/bin/multisite_enterprise_shutdown.sh
```

Upgrade

Complete these steps on each node in turn:

Copy the latest SVN MultiSite Plus installer file to the same location as your backed up version and run it using the following command:

./multisite-plus.sh

This will kick off an installation which will first check to see if there is an existing installation, along with a database backup. Providing the above steps have been taken the process will look like this:

[root@localhost wandisco]# perl multisite-plus.sh
Verifying archive integrity... All good.
Uncompressing WANdisco MultiSite Plus...........
    ,             ..   .    .  `''` . ..`''   ...`    .   :,     ,;.     :;
   ::,  ::  ::     +   +'  '+ +++++++ ++++++, +   ,'  ; ;   :   ;   +  '`   +
  ::::,:::: :::    +'  ++  +;;+    ++ ++   '+ +     : ; +      '      .      :
 ::::::::::: :::   ;+ '++  + +'    `+ ++    + +     + ; '     .       +      +
,:::::::::::: :::   + +.+;++ +,     + ++    + +     ; ;  .++  :       +      '
 ::::::::::: :::    +++ .++, +'    `+ ++    + +     + ;     .`.       +      +
  ::::,:::: :::     .++  ++  '+    ++ ++    + +     : ;      ; '      ,      :
   ::,  ::  ::       +.  ++   +++++++ ++    + +   .'  ; +   ,  `;   '  +    +
    ,                ,   ``    .++` , ,,    , ,,,.    .  `;:     :',     ;;

Checking SVN
SVN version 1.7.9-2 is already installed, will not install SVN

Beginning WANdisco SVN MultiSite Plus Installation...

Upgrade complete, you may now start up the Web UI and Replicator
[root@localhost wandisco]#

The upgrade is now complete. You can use the regular startup command to start the Web UI and Replicator, however, you should complete the upgrade process on all of your other nodes before you do.

Restart

When all nodes are upgraded, complete the following step on all sites:

Restart SVN MultiSite Plus using svn-multisite service, located in the init.d folder. e.g.
```
[root@localhost init.d]#  ./svn-multisite start
```
It's worth doing some test commits to ensure that replication is working on all sites before allowing your Subversion users to continue their work. Remember to make your repositories writable again before announcing to your Subversion users that they can start committing again.

Roll-back

Follow this procedure if you nee to revert to your previous installation.

Get the installer for your previous installation.

Extract the RPM from our installer using the following command:

bash <installer> --noexec --target <target dir>

the installer's payload lives in:

/opt/wandisco/svn-multisite-plus/resources/<installer-rpm>

Stop all replicators. See Stopping nodes
Run:
```
rpm -e svn-multisite-plus
```
Run:
```
rpm -ivh  <installer-rpm>
```
Run the roll-back script located here:
```
./svn-multisite-plus/bin/rollback
```
Repeat the previous steps on all the other nodes before moving on to the next step

Restart the replicators. See Starting Nodes.

SVN MultiSite Plus
Last doc build: 17:59am - Tuesday 15th January 2014