Install

This guide runs through everything you need to know to get SVN MultSite Plus deployed. First we'll cover all the things that you need to have in place before you install. We'll then cover a standard installation and setup. Finally we'll look at some possible problems you might experience with some troubleshooting tips.

1. Before you deploy

Before installing SVN MultiSite Plus we need to ensure that we have sufficient hardware and all required software configured appropriately.

1.1 Skills requirement

This section details the knowledge and technical requirements for deployment and operation of SVN MultiSite Plus. You should ensure that each of these requirements is satisfied before you begin the deployment.

Technical Skill Requirements
System administration	Unix operating system installation Disk management Memory monitoring and management Command line administration and manually editing configuration files Service (init.d) configuration and management
Apache administration (if applicable)	Familiarity with Apache web server architecture Management of httpd.conf / Apache2 configuration file management settings WebDAV protocol User authentication options Log setup and viewing
Networking	IP Address assignation TCP/IP ports and Firewall setup
SVN and MultiSite Plus	Familiarity with SVN repository administration Familiarity withWANdisco's replication architecture Concept of Node types and replication groups Understanding of the installation procedures for your platform

If you're not confident about handling any of these tasks, you can request a supported installation from WANdisco.

A single administrator can manage all the systems running MultiSite Plus, although it's a good idea to have someone at each site who is familiar with the MultiSite Plus Basics.

1.2 Deployment overview

As with any software implementation you should deploy SVN MultiSite Plus following a well defined plan. We recommend that any deployment plan includes the following steps:

1. Pre-deployment Planning: Identifying the requirements, people and skills needed for deployment and operation. Also covers agreeing schedule and milestones, highlighting any assumptions, constraints, dependencies and risks to a successful deployment.

2. Deployment Preparation: Preparation and identification of server specifications, locations, node configuration, repository set-up, replication architecture, server and software configurations.

3. Testing Phase: Activities related to an initial installation and testing in a non production environment, executing test cases, and verifying deployment readiness.

4. Production Deployment: Activities related to the installation, configuration, testing, and deployment in the production environment.

5. Post-deployment Operations and Maintenance: Post-deployment activities including environment monitoring, system maintenance, training and technical support.

1.3 System requirements

This section covers everything you need to know if you are preparing existing servers for replication. You should view this information as a set of guidelines, not as a fixed set of requirements. There are a lot of factors at play and there's really no substitute to running your own performance tests during an evaluation period.

Hardware Recommendations

Hardware Sizing Guidelines
Size	#Users	Repository Size (Gigabytes)	CPU speed (Gigahertz)	#CPU	#Cores	RAM (Gigabytes)	HDD (Gigabytes)
Small	100	25	2	1	2-4	8-16	100
Medium	500	100	2	2	4	16-32	250
Large	1000	500	2.66	4	4	32-64	750
Very Large	5000	1000	2.66	4	4-6	128	1500

tip GB or GiB
We should clarify how we refer to measures of memory and data. SVN MultiSite Plus has now adopted the binary prefixes as provided by the International Electrotechnical Commission. We therefore use Mebibyte (1,048,576 bytes) instead of Megabytes (1,000,000 bytes) within our products. However, we'll still refer to Megabytes and Gigabytes where these are more commonly understood - such as in the above table.

If you're not familiar with the difference between the binary prefixs and the SI prefixes then we recommend that you give the following Wikipedia article a read - http://en.wikipedia.org/wiki/Mebibyte

Storage tips

Use separate physical disks for SVN and SVN MultiSite Plus. This ensures that heavy disk usage by either should not impact the other. If you are running with SAN storage we recommend using a fiber connection between the server and the SAN with a minimum dedicated bandwidth of 1GiB.

SVN repository storage requirement:
Plan your requirements. Clarify what version control changes may be on the horizon so that you can account for any sudden leaps in your repository storage requirement. Consider that it's usually a lot less costly to over-specify disk capacity than have to deal with running out of storage.
SVN MultiSite Plus storage requirement:
Although the storage requirements for the installed files is fairly modest (800 mebibytes), SVN MultiSite Plus will store data that has not yet been replicated to all other replication group members. Should a node be offline for an extended period this can result in the buildup of lots of temporary data.
How much storage does SVN MultiSite Plus need?
We provide a guideline for calculating for WANdisco's replication products: Hardware Sizing Guide

Use the fastest possible disks for storage, disk IO is the critical path for improving repository responsiveness.

The use of RAID-1 or RAID-2 solutions are recommended. You should not use RAID-0, the performance benefits are not worth the drop in resilience (and increased risk of data loss). Where performance is considered more important than resilience then RAID 0+1 can be used instead. This mirrors two or more striped segments, providing the high I/O performance of RAID-0 without the increased risk of failure.

Spinning vs Solid State
Solid state drives (SSDs) offer significant benefits for deployments that make big demands of disk I/O. SSDs are recommended if you have a large deployment or require extra capacity for future growth. However, if your concurrent SVN usage is not very high you may get acceptable performance from trustly old HDD technology.

Running in virtualization

Deploying on a virtual server platform provides lots of practical benefits. Costs, admin time, flexibility can all see big improvements when running services from a small number of specialist servers. However, there is no escaping the fact that virtualization doesn't suit every application. Dedicated servers give you confidence about what resources will be available - and whilst well designed virtual platforms can build in load balancing and failover, these are often bolt-ons that work against the whole drive to consolidate physical equipment and never really offer the seperation of services or militate the risks of a single point of failure.

Processor tips

SVN MultiSite Plus will run on a single 2GHz CPU, but for production you should run fast multi-core CPUs and scale the number of physical processors based on your peak concurrent usage.
You should aim to have no more than 15 concurrent SVN users per single-core CPU or 7 concurrent users per core with multi-core CPUs.
Example 1: A server with 4 physical single core processors is expected to support (15x1x4) = 60 concurrent users.
Example 2: A server with 4 physical processors, each being a quad core is expected to support (7x4x4) = 112 concurrent users.

Setup requirements

MultiSite Plus Servers must have:

This is a summary of the requirements, you'll need to run through the more detailed Installation Checklist.

the same operating system - including same architecture and patch versions.
Everything the same
The reason for keeping the setup of nodes identical is that subtle variations in software could result in non-deterministic behavior that might lead to a loss of sync.
Java & Python installed
a browser with network access to all servers
a command line compression utility
a unique license key file. (This is provided by WANdisco: you will need one for each node and you may need to provide the server IP addresses)

SVN installations:

We recommend that you install SVN during the installation of SVN MultiSite Plus. You can choose between version 1.7 or 1.8, both have the necessary WANdisco-modified FSFS libraries already included. Regardless of the version of SVN that you use, it must include:

matching file and directory level permissions on repositories

exactly matched contents of the svnroot directories between servers (including the repository UUID)
Specifically following this guide:

/conf

Strongly recommend that the contents match between replicas

/db

As this is where repository data is stored it is crucial that this is a perfect match between servers

hooks

Pre-commit Hooks
Wandisco's modified version of the FSFS libraries will intercept commits after any pre-commit hooks have run. This means that the pre-commit hook run on the initiating node (on the server, Apache, SVNserve, etc.) rather than in the replicator. Should a pre-commit hook fail then the server will return an error to the client before the FSFSWD intercept call. As a result, the replicator is never involved with failed pre-commit hooks (with the possible exception of protorev /abort notifications.

So if a commit (on the originating node) is delegated for replication any related pre-commit hook will already have succeeded.

Post-commit Hooks
The replicator completes the commit on the originating node by invoking a JNI function - a low level function that doesn't run any hooks. When the replicator returns the commit status to the originating repository FSFSWD a successful commit will cause the post-commit to run on the server.

The net effect is that pre- and post- hooks run in the server on the originating repository and they do not run at all for the replicated repositories. Although a replicator could explicitly invoke the hooks for the replicated repositories if required.

locks

Locks must be sync'ed between nodes. You can't afford for a commit to be rejected on one site that was allowed on all the others.

tip Requirement
If you run an existing installation of SVN instead of installing it during the setup of SVN MultiSite Plus, you need to make sure that it contains WANdisco's modified FSFS libraries.

You must run SVN and SVN MultiSite Plus on the same server.

** Alert! ** Alert
A repository can belong to only one replication group at a time.

2. Installation Checklist

Though you may have referred to the Installation Checklist prior to an evaluation of SVN MultiSite Plus we strongly recommend that you revisit the checklist and confirm that your system still meets all requirements.

System setup
Operating Systems	We've tested the following operating systems: Red Hat Linux Enterprise Server (32 or 64 bit): versions 5.x and 6.x High Level of confidence We've limited our testing to RHEL/CentOS so that we can provide a high level of confidence about operability, which would not be possible if we spread our testing across a larger number of distributions. Maximum confidence If you want the highest possible level of confidence you should match the version of RHEL used on your nodes with one of those that we use in our testing. These currently include 5.9, 6.1, 6.2, 6.3, 6.4 and 6.5. Running with 32-bit RHEL In order to run with 32-bit Red Hat (though 32-bit architecture is not recommended for production), you will need to edit multisite-plus/config/main.conf and change the maximum Java heap listed in the MSP_REP_MEM_HIGH parameter to be 2.5 Gigabytes or less otherwise the replicator will not start. MSP_UI_MEM_LOW=128 MSP_UI_MEM_HIGH=1024 MSP_REP_MEM_LOW=1024 MSP_REP_MEM_HIGH=2621 CentOS: versions 5.x and 6.x Debian: versions 6.x and 7.x SUSE Linux Enterprise: version 11. Go 64-bit While it's possible to run SVN MultiSite Plus on 32-bit architecture, this would impose serious limits on scalability. For this reason we strongly discourage deploying on anything less than a 64-bit Operating System unless your requirements are limited to a handful of sites and less than a hundred repositories.
SVN Server	Recommended Version: Installing the version of SVN that is bundled with SVN MultiSite Plus is the best option as this takes care of the requirement for running with WANdisco's customized FSFSWD libraries, it is also offers the benefit of being a version of SVN that have been extensively tested with MultiSite. Option Component Packages SVN MultiSite Plus installation checks for the presense of a number of option SVN components. These components, if found, are upgraded from a collection of packages that are bundled with SVN MultiSite Plus. However, if they are not already installed they will not be touched by the installer, if you need any of them you will need to install them manually. All SVN packages (including the optional packages) are located here: /opt/wandisco/svn-multisite-plus/resources/svn -rwxr-xr-x 1 root root 78400 Jun 17 16:20 mod_dav_svn-1.8.8-1.x86_64.rpm -rwxr-xr-x 1 root root 44276 Jun 17 16:20 serf-1.3.4-1.x86_64.rpm -rwxr-xr-x 1 root root 2267700 Jun 17 16:20 subversion-1.8.8-1.x86_64.rpm -rwxr-xr-x 1 root root 11352880 Jun 17 16:20 subversion-debuginfo-1.8.8-1.x86_64.rpm -rwxr-xr-x 1 root root 3715880 Jun 17 16:20 subversion-devel-1.8.8-1.x86_64.rpm -rwxr-xr-x 1 root root 24124 Jun 17 16:20 subversion-fsfswd-1.8.8-1.x86_64.rpm -rwxr-xr-x 1 root root 259648 Jun 17 16:20 subversion-javahl-1.8.8-1.x86_64.rpm -rwxr-xr-x 1 root root 968424 Jun 17 16:20 subversion-perl-1.8.8-1.x86_64.rpm -rwxr-xr-x 1 root root 690808 Jun 17 16:20 subversion-python-1.8.8-1.x86_64.rpm -rwxr-xr-x 1 root root 71288 Jun 17 16:20 subversion-tools-1.8.8-1.x86_64.rpm Repository Creation: If you are running with repositories that use the old Berkeley DB, make sure that all the repositories you intend to replicate are created or recreated as follows: svnadmin create Repo --fs-type fsfswd Then edit Repository_name/db/fsfs.conf (view an example fsfs.conf file) uncomment the line: #rep_port = 7777 Write access for system user The replicator user must have write permission for all repositories - as the replicator writes directly to the SVN repository. Manage repository file ownership if using SVN+SSH:// or file:// Accessing SVN repositories via Apache2+WEBDAV is simplified by the fact that all user access is handled via the same daemon user. SVN+SSH or file:// access is a little more tricky, there are potential permission problems when multiple users access the same repository. Run with the same system account When using SVN over SSH both processes should be run using the same system account as SVN MultiSite Plus. This account's .ssh/authorized_keys entry must provide the necessary access and specificy the appropriate account. However, when unifying control in this way you must lock down wider system access or SVN access will equate to full root access. Read more about controlling the invoked command. Tips: Simplify user management by putting SSH users into a single group, you can then ensure that the group has read-write permissions for the repositories. Make repositories wholly owned by the group. Ensure that the prevailing umask is set to provide suitable permissions (002 instead of the default 022). Use wrapper scripts for commands such as svn, svnadmin , svnserve etc. Note that with svn+ssh, an svnserve instance running as the user is spawned, so the wrapper will get called there too). Certified SVN Binaries are now available from WANdisco. Providing the latest builds, without the risks associated with Open Source distribution. Same location All replicas must be in the same location (same absolute path) and in exactly the same state before replication can start. Same UUID If you start with new repositories, don't create them individually at each site. This is because even though they may share the same repository data, each will have its own universally unique identifier (UUID) - unless the repositories have the same UUID they're not replicas. For more help, read the section about Setting up Repositories for Replication. Conversely, two different repositories must not share the same UUID - See UUID Warning. svnadmin pack support It's not currently possible to run the svnadmin pack command when running SVN MultiSite Plus. Support for this command is currently being added to FSFSWD and should be available in the near future. Linux Standard Base LSB provides developers with a degree of confidence about their applications being able to run on a range of distributions. The package is widely included by default, but not always. Run the following command to verify the version of LSB yours server is running: [root@redhat6 wandisco]# lsb_release -a LSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64: graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch Distributor ID: RedHatEnterpriseServer Description: Red Hat Enterprise Linux Server release 6.4 (Santiago) Release: 6.4 Codename: Santiago SVN MultiSite Plus's init.d scripts are dependent the the LSB package. Running the installer script, if the package isn't present it should be downloaded before the installation continues.
SVN client	Any that are compatible with local SVN servers.
Hooks	Hook scripts no longer need to be replicated on all repository replicas - See details
System memory	Minimum recommended: 8 Gigabytes RAM; 16 Gigabytes swapping container Memory requirements of DConE² replication Each state machine, or replicated object (repository/replication groupe, etc) needs about 1MB of system memory to run. So for small to moderate deployments the memory requirement of the replication system itself is quite modest. For very larger deployments where you are replicating hundreds or more repositories then you may need to consider the specific memory requirements of the DConE² replication engine.
Disk space	SVN: Match to projects and repositories. MultiSite Plus Transaction Journal: Equivalent of seven days of changes. Estimating your disk requirements can be very difficult and there's no perfect system for making an accurate estimation. Some organizations monitor their repository growth over a period of time and use an extraopolation as a guide. This method works best if your organization is unlikely to see the addition of large new projects that instantly introduce large amounts of extra repository data. you need to quantify some elements of your deployment: overall size of all of your SVN repositories. frequency of commits in your environment. types of files being modified - text,binaries (SVN clients only send deltas for text). number and size of files being changed. rate that new files are being added to the repository. Talk to those who know There is absolutely nothing like having a solid communications path between those managing the SVN system resources and those who manage the development project. Actually talking to the people who are planning upcoming SCM efforts is better than trusting an abstract system for measuring requirements.
Maximum User Processes and Open Files limits	Maximum User Processes and Open Files limits are low by default on some systems. It is possible to check their value with the `ulimit` or `limit` command: ulimit -u && ulimit -n -u The maximum number of processes available to a single user. -n The maximum number of open file descriptors. For optimal performance, we recommend both hard and soft limits values to be set to 64000 or more: RHEL6 and later: A file /etc/security/limits.d/90-nproc.conf explicitly overrides the settings in security.conf, i.e.: # Default limit for number of user's processes to prevent # accidental fork bombs. # See rhbz #432903 for reasoning. * soft nproc 1024 <- Increase this limit or ulimit -u will be reset to 1024
Journaling file system	Replicator logs should be on a journaling file system, for example, ext3 on Linux or VXFS from Veritas. Alert NFS must not be used with SVN MultiSite Plus- Why you shouldn't use NFS. ext4 can be used as your journaling file system, although it must be configured appropriately. See Using Ext4 filesystem for journaling . Avoiding Data Loss. We have an article in our Knowledge Base that looks at a number of implementation strategies that will militate against potential data loss as a result of power outages - Data Loss and Linux.
Java	Install JRE or JDK 7 Use Oracle Java Our development and testing is done using Oracle JDK 7 and JDK 6. While it may be possible to use other Java packages, we will not be able to support you unless you run with Oracle's package. 1. Install JDK/JRE 7 (from Oracle) and define the JAVA_HOME environment variable to point to the directory where the JDK/JRE is installed. 2. Add $JAVA_HOME/bin to the path and ensure that no other java (JDK or JRE) is on the path. $ which java /usr/bin/java $export JAVA_HOME="/usr" or /export/share/apps/jdk/1.6.0/bin/java $export JAVA_HOME="/export/share/apps/jdk/1.6.0" It is possible to run with the JRE package instead of the full JDK. You can check this by running java -server -version. If it generates a not found error, repeat Steps 1 and 2. If you find package management problems or conflicts with the JDK version you are downloading (for example, rpm download for Linux), you may want to use the self-extracting download file instead of the rpm (on Linux) package. The self-extracting download easily installs in any directory without any dependency checks.
Python	Install version 2.3 or later.
Browser Compatibility	Set up and configuration requires access through a browser, what follows is a basic (not exhaustive) guide to which browsers are known to work: Internet Explorer 10 Firefox 24 Google Chrome 30 Safari 7 Some earlier browser versions may work. However, we will not have completed thorough testing with older versions, we recommend that you keep browsers up-to-date. SVN MultiSite Plus is not compatible with either Internet Explorer 6 or 7 While we understand that some users are still tied to earlier versions of Internet Explorer, those earlier versions do not support modern web technologies and we are unable to support them.
Kerberos SSO	We support the implementation of Kerberos for single sign-on. By default Kerberos requires that stronger encryption algorithms be available than are currently provided by default in Java 6/Java 7. This is so that Oracle can avoid the complications that arise from countries that place import restrictions on encryption technology. The stronger encryption algorithms are available as an optional download where the user takes responsibility for compliance with the local laws. For Java 6: JCE Unlimited Strength Jurisdiction Policy For Java 7: JCE Unlimited Strength Jurisdiction Policy Once downloaded, extract the contents to (and overwrite the existing contents of) the Java security library directory on all nodes. e.g. $JAVA_HOME/lib/security/

Network settings
Reserved ports	There are a number of ports that will be reserved by MultiSite Plus. It's possible to change these ports after completing the installation - 6. Update a node's properties. The default values suggested during the installation are the following: Required Ports dcone.port= An integer between 1 - 65535 (Default: 6444) DConE port handles agreement traffic between sites content.server.port= An integer between 1 - 65535 (Default: 4321) The content server port is used for the replicator's payload data: repository changes etc. delegate.port= An integer between 1 - 65535 (Default: 7777) The delegate port is used by SVN to delegate write operations to the WANdisco Replicator (via the above content.server.port) jetty.http.port= An integer between 1 - 65535 (Default: 8082) The jetty port is used for the MultiSite Plus management interface. jetty.https.port An integer between 1 - 65535 (Default: 8082) The jetty port is used for the MultiSite Plus management interface when SSL encryption is enabled. Make each port different In contrast with earlier versions of SVN MultiSite Plus which used the same port for both the UI and replication traffic, SVN MultiSite Plus doesn't multiplex different traffic on a single port. You will need to assign a different port to each type of traffic.
Firewall or AV software	If you have a virus scanner running on the system housing your repositories and replicator you should: Ensure that you make frequent backups of your repository data If possible, configure your AV system to "Notify Only". Otherwise you should prepare for the possibility that a virus infection or for that matter a false-positive could result in potentially catastrophic corruption of either repository or system data. Whilst it's true that virus scanners don't normally work via port filtering, some AV In general, virus scanners don't filter ports: firewalls do that. However, some "Anti-Virus" products contain firewall-like filtering capabilities - if this is the case in your platform, you should make sure that you understand what impact it could have on your MultiSite deployment.
Full connectivity	MultiSite Plus requires full network connectivity between all nodes. Ensure that each node's server is able to communicate with all other servers that will host nodes in your MultiSite Plus installation.
VPN	Set up `IPsec tunnel`, and ensure WAN connectivity.
VPN persistent connections	Ensure that your VPN doesn't reset persistent connections for SVN MultiSite Plus.
Bandwidth	Put your WAN through realistic load testing before going into production. You can then identify and fix potential problems before they impact productivity.
DNS setup	Use IP addresses instead of DNS hostnames, this ensures that DNS problems won't hinder performance. If you are required to use hostnames, test your DNS servers performance and availability prior to going into production. It's possible to change these details after completing the installation - 6. Update a node's properties.
Monitoring	SVN MultiSite Plus provides a limited system for monitoring system disk space available. This monitor is intended only to provide a deployment with a last line of defence against running out of storage space. We recommend that you deploy a system-wide monitor that ensures that you quickly identify potential problems that could impact services. Monitor Recommendation Read our recommendations for system-wide monitoring tools
Load Balancing	The use of a correctly configured load balancer can greatly benefit performance in situations where there could be large numbers of concurrent SVN users. However, SVN MultiSite Plus requires that any load balancing solution has the following features: Stateless session persistence. Any potential SVN load-balancer needs the ability to handle stateless session persistence within its load balancing algorithm. This is because each Subversion commit needs to go to the same backend node in its entirety or the commit will fail. We achieve this by ensuring the client is bound to a particular back-end node in some way. Client's IP Address - Not always an option, but this IP-based persistence is easy to manage when the network is stable with static IPs. Cookie-based persistence - SVN command line clients can't read cookies so for a load balancer to use cookies for the binding they would need to be able to use sticky cookies that are not reliant on the client honoring them. Node health-checking Another vital requirement is the support for a health check mechanism - whereby the load-balancer makes periodic checks on the connected nodes to make sure that it isn't passing traffic to an off-line or overloaded server. Any prospective load-balancer should support HTTP status code (application-layer) checks. The load-balancer sends HTTP GET or HEAD requests to back-end nodes. Watching for 'unhealthy' response codes offers greater reliability and flexibilty than doing your checks belofe the network layer.

MultiSite Plus setup
Replication Configuration	Read our Replication Setup Guide for information on how to optimise your replication - Replication Setup.
Voters follow the sun	To ensure best performance, make sure that SVN MultiSite Plus can deliver the content of a commit to another local node. SVN MultiSite Plus normally requires that content reach at least one other node for data integrity purposes. As the content normally represents the bulk of the data in a commit, having a second local node available will improve performance. Furthermore, you may wish to use our scheduling system to modify the voter roles so a proposal may be accepted by local voter nodes during regular working hours. If you need more help with setting up the most efficient deployment please get in touch with our support team.
Disk space for recovery journal	Provision enough disk space for /opt/wandisco/multisite-plus/replicator/database to cover the expected number of commits for four hours of peak usage.
License Model	SVN MultiSite Plus is supplied through a licensing model based on the number of nodes and users. WANdisco generates a license file matched to your agreed usage model. Evaluation License To simplify the process of pre-deployment testing SVN MultiSite Plus is supplied with an evaluation license. This type of license imposes no restrictions on use but is time-limited to an agreed period. Production License Customers entering production need a production license file for each node. These license files are tied to the node's IP address. In the event that a node needs to be moved to a new server with a different IP address customers should contact WANdisco's support team and request that a new license be generated. Production licenses can be set to expire or they can be perpetual. Special Node Types SVN MultiSite Plus offers additional node types to provide limited sets of functionality: Passive Nodes (Learner only) - A passive node operates like a slave in a master-slave model of distribution. Changes to its repository replicas only occur through inbound proposals, it never generates any proposals itself. Voter-only nodes (Acceptor only) - A voter-only node does not contain repositories. It casts votes based only on the basis of replication history without knowing the actual contents of the proposal data. These limited-function nodes are licensed differently from active nodes. In short the IP addresses will be a fixed list but the node count and special node count may move between sets of nodes, as long as the number of each type of node is within the limit specified in the license. Speak to WANdisco's sales team for more details.

2.1 Migrating from WANdisco's SVN MultiSite 4.x

SVN MultiSite Plus uses a new version of WANdisco's DConE replication engine and has a different architecture compared with earlier versions of MultiSite. As a result there are some special considerations when migrating from SVN MultiSite 4.x

Byte-for-byte replicas

Repository replicas must be byte-for-byte mirrors of each other. This stringent requirement did not apply to SVN MultiSite 4.x - the previous tests for whether replicas are identical are not sufficient for FSFSWD replication (see 4.2 vs Plus, below). As a result you will need to recreate your replica repositories using a nominated master repository.

You'll need to identify which of your current replicas is to be the master repository.
You'll then remove / backup all other replicas.
Rsync from master to remote servers using checksumming *recursively (-r)*.

The exact process of copying repositories may need to be carefully thought out in order to be practical and achievable. Many production repositories will take a long time to checksum. If you are in any doubt about handling the process, talk to your WANdisco account manager.

tip 4.2 vs Plus
SVN MultiSite 4.2 replication is done using a proxy that sits between SVN and clients that replays commit operations from the users on the repository via Apache and so constructs a new transaction at every node. In contrast, SVN MultiSite Plus applies the same FSFS db/transactions at each node. This transaction is constructed based on the contents of the rev files - so with FSFSWD the repositories need to be identical at the revision (and revprop) file level.

Authentication /Apache

SVN MultiSite Plus opens up more options because MultiSite is no longer running as proxy and options that were previously not compatible with MultiSite now are compatible.

Other factors to consider

Access Control is no longer working as a proxy, all the traffic goes through Apache, everything at the repository level.
Access Control rules no longer supports regular expressions for defining repository paths and the concept of pre-replication authentication has been done away with.
Access Control still needs to be integrated with existing LDAP.
Lost the ability to specify rules using regular expressions - the concept of pre-replication authentication has been done away with.
Hook scripts - these no longer need to all run on all nodes. See Hook Scripts.

3. Installation

The installation guide runs through setting up SVN MultiSite Plus for the first time. If you are upgrading from an earlier version of SVN MultiSite Plus you should also follow this procedure - SVN MultiSite Plus is a completely new class of product so it's not possible to follow a shortcut upgrade procedure.

3.1 Installation overview

Before we begin, here is a recap of the installation process:

1. Revisit the Installation Checklist. Taking extra time making sure you have everything set up and ready may save you from running into problems during the installation process. The main boxes you need to tick are:

SVN authentication: SVN installed, and using authentication. If you require a SVN access control solution see our Access Control product.

JDK7: You need to have Oracle's Java JDK 7 installed.

Alert
Important: use Oracle JAVA JDK 7
All our development and testing is done using JDK 7. You will need to use JDK 7 to ensure compatibility with SVN MultiSite Plus. It may be possible to run SVN MultiSite Plus with other versions of Java, although we won't presently be able to support it.

Java Memory Settings: The Java process on which SVN MultiSite Plus runs is assigned a minimum and maximum amount of system memory, by default it will get 128 Megabytes at start up and 4 Gigabytes Maximum.

System resources: Ensure that your system meets the hardware recommendations.

2. Ensure that your repositories are copied into place on all nodes.

3. Download and copy the MultiSite Plus files into place.

4. Run the setup (as root user), then complete the installation from a web browser.

3.2 Before you start

Some things you should bear in mind before starting the installation:

Check through the Pre-installation Checklist
Back up Apache Config - As the installation could modify your Apache configuration we recommend that if you have existing config, you back it up before the installation and do a reconcillation once the installation has completed to check any changes are not going to adversely affect your operation.

3.3 Starting the installation

This procedure requires the entry of various settings into the terminal session during the installation. You can run the installation in a non-interactive mode which doesn't require user input (until then post-installation browser-based set up). To use the Non-interactive mode, see Non-interactive Installation

Run SVN MultiSite Plus installer as root
The installation will require full system access so you must run the installer as root or a user with equivalent permissions.

1. Extract the setup file.
2. Save the svn-multisite-plus.sh installer file to your Installation site.
3. Make the script executable, e.g. enter the command:
```
chmod a+x svn-multisite-plus.sh
```

4. Run the setup script.

** Alert! ** Running with Apache?
Run both SVN MultiSite Plus and Apache with the same user.

Back up any existing Apache configuration files
Once the installation is complete perform a verification that no changes have been made during the installation that will impact your operation.

[root@redhat6 wandisco]# chmod a+x multisite-plus.sh
[root@redhat6 wandisco]# ./svn-multisite-plus.sh
Verifying archive integrity... All good.
Uncompressing WANdisco SVN MultiSite Plus....................
    ::   ::  ::     #     #   ##    ####  ######   #   #####   #####   #####
   :::: :::: :::    #     #  #  #  ##  ## #     #  #  #     # #     # #     #
  ::::::::::: :::   #  #  # #    # #    # #     #  #  #       #       #     #
 ::::::::::::: :::  # # # # #    # #    # #     #  #   #####  #       #     #
  ::::::::::: :::   # # # # #    # #    # #     #  #        # #       #     #
   :::: :::: :::    ##   ##  #  ## #    # #     #  #  #     # #     # #     #
    ::   ::  ::     #     #   ## # #    # ######   #   #####   #####   #####

Welcome to the WANdisco SVN MultiSite Plus installation

Checking prerequisites:

Checking for perl: OK
Checking for svn: SVN MultiSite Plus requires a compatible version of SVN to be installed.

Install SVN? [Y] > Y
Installing SVN 1.8.3-1

5. On a clean system you will need to enter "Y" to install a compatible version of SVN onto your server. If you already have a suitable SVN installation in place you could select "N".
```
Install mod_dav_svn? (Y/n) Y

Stopping httpd: [  OK  ]
Starting httpd: [  OK  ]
OK
 
```
Select "Y".
6. The next test looks at the Java heap settings. It lists the maximum and minimum allocations for both the replicator component of SVN MultiSite Plus as well as the admin console UI:
```
INFO: Using the following Memory settings:

INFO: UI:         -Xms128m -Xmx1024m
INFO: Replicator: -Xms1024m -Xmx4096m

Do you want to use these settings for the installation? (Y/n) Y
  
```
Enter "Y" if these heap settings will suit the needs of your deployment, then enter your preferred values. If you have any doubts, discuss the heap requirements with WANdisco's support team before going into production.
7. You'll now be asked to enter a TCP port number for accessing the browser part of the installation process.
```
Which port should the MultiSite UI listen on? [8080]:
```
We use port 8080 in our documentation. If that port is okay, press Enter. Check with your network administrator about which ports are available. You can change the port during the next part of the installation.
8. The installer now checks to see which system user and system group should be used to run SVN MultiSite Plus.
Run SVN MultiSite Plus with the same user that runs Apache
When deploying SVN MultiSite Plus with Apache, ensure that they are both run by the same system user. Their operations are so entwined that attempting to run the services with separate users will introduce the risk of permission problems that would halt replication.
```
We strongly advise against running SVN MultiSite Plus as the root user.

Which user should SVN MultiSite Plus run as? wandisco
Do you want to continue? (Y/n)Y

Which group should SVN MultiSite Plus run as? wandisco
```
In this example we have system user and group set up for 'wandisco'. Also, take note of the above warning about not running SVN MultiSite Plus with a different user account.

9. The installer provides you with a summary of the settings you've so far provided:

MultiSite user:    wandisco
MultiSite group:   wandisco
MultiSite UI Port: 8080
MultiSite UI Minimum memory: 128
MultiSite UI Maximum memory: 1024
MultiSite Replicator Minimum memory: 1024
MultiSite Replicator Maximum memory: 4096

Do you want to continue with the installation? (Y/n) y

Enter "Y" unless you want to make changes to any of these choices.

10. Open a browser and go to the provided URL. If your server's DNS isn't running you can go to the next step at the following address:
```
Starting ui:[  OK  ]
..........

Please access the Web UI with a browser at the following address -
http://ip-10-0-100-152:8080/multisite-local
Installation Complete
```
e.g.
http://10.0.100.152:8080/multisite-local/
11.

Welcome to SVN MultiSite Plus.
You're about to run through the installation, which should only take a couple of minutes.

If you run into difficulties on the way, check our documentation or talk to our support team through the Customer Support Website.

Before you click Next, make sure you Read the Installation Checklist

Click Next to begin the installation.

12. The next (Terms & Conditions) screen contains the WANdisco Master Subscription Agreement.
To continue the installation click the I AGREE button.

13. On the next (License Upload) screen you are prompted to browse for your product license key file. Click on the + Browse button and locate your file. You will have been sent this by the WANdisco sales team, contact them if you have any problems locating or using your license file.

14. On the Administrator Setup screen you indicate whether this is the installation of the first node or a subsequent node.

This is the first node:
If this is the first node you'll be prompted to enter the username plus an associated password which you will use to login to SVN MultiSite Plus' UI.

Username

The administrator's username.

Password

The administrator's password.

Confirm Password

Enter your password again to confirm that it's been typed in correctly.

Full Name

Enter your full name.

Email address

Enter the email address that you wish to associate with your SVN MultiSite Plus admin account
This is the second or a subsequent node:
For later installations, you will, instead, be prompted for the users.properties
(default location: /opt/wandisco/svn-multisite-plus/replicator/properties/users.properties) file of the first node rather than risking a mismatch in admin account details between nodes - something that could prevent you from connecting the nodes together during induction.
Can I just enter the same details?
No, while you could enter exactly the same details for each node, encrypted password would not match. You've got to copy the users.properties file, there's no shortcut. In the event that this has been done, you can match up the neccessary details using the procedure for Matching a node's admin settings.

Important
If you are providing a users.properties file, take extra care to select the correct file. You won't be warned if the file is invalid and selecting the wrong file will prevent you from connecting the node to the replication network.

15. The last screen in the setup process covers Server Settings.
MS5 - Setup 06

Node Name: The default name for this node.
Temporary limitation
Node names can not contain spaces or ".".
Node IP/Host: The node's IP or hostname. If the server is multi-homed, you can select the IP to which you want SVN MultiSite Plus to be associated.
Replication Port: Select the port that will be used for agreement traffic between nodes.
Content Server Port: Select the port that will be used to transfer replicated content (repository changes). This is different from the port used by WANdisco's DConE² agreement engine.
Content Node Count: This setting gives you the ability to enforce a degree of resiliance. The value represents the number of nodes within a membership that must receive the content before a proposal is submitted for agreement. If the value is greater than the total learners in the current membership, then the value is adjusted to equal the total number of learners in the current membership. The proposing node is not considered in the calculation.
Minimum Content Nodes Required: Ticking this checkbox will enforce the Content Node Count as a prerequisite for replication.
REST API Port: The port to be used for SVN MultiSite Plus's REST-based API. (Default:8082)
REST API SSL Port: The port to be used for SVN MultiSite Plus's REST-based API when traffic is secured using SSL encryption.
REST API Using SSL: Check box for enabling the use of SSL for all API traffic.
SSL Trust Store: The location of your trust store file. Trust stores contain CA certifcates to trust. If your server's certificate is signed by a recognized Certification Authority (CA), the default trust store that ships with the JR will already trust it (because it already trusts trustworthy CAs), so you don't need to build your own, or to add anything to the one from the JRE.
SSL Trust Store Password: The password for your trust store.
SSL Key Store: The name of the keystore file. The keystore contains a public keys. for authorized users.
SSL Key Store Password: The password associated with the keystore.

** Alert! ** A word about trust stores and key stores
You might be familiar with the Public-key system that allows two parties to use encryption to keep their communication with each other private (incomprehensible to an intercepting third-party). The keystore is used to store the public and private keys that are used in this system. However, in isolation, the system remains susceptible to the hijacking of the public key file, where an end user may receive a fake public key and be unaware that it will enable communication with an impostor. Enter Certificate Authorities (CAs). These trusted third parties issue digital certificates that verify that a given public key matches with the expected owner. These digital certificates are kept in the trust store. An SSL implementation that uses both keystore and trust store files offers a more secure SSL solution.

If you need help getting your SSL keys set up, check out our guide in the Appendix - Setting up SSL

16. Click FINISH when you have entered everything. The installer will now complete the configuration. Once that's all done a START USING MULTISITE PLUS button will appear. Click the button to login for the first time.
MS5 - Setup end

17. Time to login. Enter the username and password during step 12. Then click FINISHED - LET'S GO!. MS5 - Setup ended - work begins

18. Next, you'll need to agreed to the WANdisco Subscription Agreement. Click I Agree to continue.

** Alert! ** Temporary duplication of license agreement
Currently the license agreement is presented twice - once during installation and then here, when the first end user logs in. In a future release we'll phase out is end-user agreement screen to remove the apparent duplication.

19. The first time you view the dashboard, it will contain mostly blank areas. You can view the reference section to learn what all the buttons and options mean. At this point you can set up some of your settings, such as SSL. However, we recommend holding off more advanced admin account management until you have completed induction.
MS5 - Node Induction begins

Non-interactive Installation

It's now possible to install SVN MultiSite Plus non-interactively. To do so the following environment variables must be set:

MSP_USER: The system user that run MultiSite Plus.
MSP_GROUP: The system group that MultiSite Plus will run within.
MSP_UI_PORT: The TCP port the browser UI will initially use - you'll be able to change this during the browser-based setup.
MSP_NO_SVN: Should you wish to exclude the SVN installation that is incorporated into the SVN MultiSite Plus installer you should use this variable. Note that if the server doesn't already have a compatible version of SVN installed, the SVN MultiSite Plus installation will not complete.
MSP_UI_MEM_LOW: The minimum amount of UI memory.
MSP_UI_MEM_HIGH: The maximum amount of UI memory.
MSP_REP_MEM_LOW: The minimum amount of Replicator memory.
MSP_REP_MEM_HIGH: The maximum amount of Replicator memory.

A scripted start to the installation requires the running of the following command:

MSP_USER=wandisco
MSP_GROUP=www-data
MSP_UI_PORT=9999
export MSP_USER MSP_GROUP MSP_UI_PORT
./svn-multisite-plus.sh

The installation will then run without user interaction. Once installation is completed, the browser-based UI will start. You'll then need to complete the node set up from step 10.

4. Repeat the installation process at all sites

Now repeat section 3 for each node that you wish to share your SVN repositories - note that we will take the configuration files from the first node and use them during the installation of all additional nodes to ensure that all nodes are started with the same administrator account - a requirement for a successful induction.

You may benefit from creating an image of your initial server, with the repositories in place and using this as a starting point on your other sites - this will help you ensure that your replicas are in exactly the same state.

** Alert! ** Same Location
All replicas must be in the same location (same absolute path) and in exactly the same state before replication can start.

** Alert! ** Same UUID
If you start with new repositories, don't create them individually at each node. This is because even though they may share the same repository data, each will have it's own universally unique identifier (UUID) - unless they have the same UUID they're not replicas.

Ensure that all nodes have matching configuration before completing the inductions

Copy configuration (e.g. admin account property file, SSL certs) to all other servers on which you intent to install SVN MultiSite Plus.
Run the installer on the servers nodes and continue to the the induction. Installer will let you select the copied-over admin property file instead of manually entering details for the admin account.
If you do not provide the admin account property file during installation, or the admin accounts use LDAP, or the admin accounts change before induction, then you have to use the regular export-import process.
If you have conflicts in the admin accounts then you need to delete or rename accounts on the to-be-inducted node to remove the conflicts.

5. Node Induction

After installing SVN MultiSite Plus at all sites, you'll need to make the sites aware of each other through the node induction process. There's a particular way that you need to run through this process so get another coffee and don't skip this section.

Overview

It's important that sites are connected together in a specific sequence. Run through the following steps to ensure that your sites are all able to talk to each other:

1. When SVN MultiSite Plus is installed on all your sites, select one node to be your Inductor. This node will accept requests for membership and share its existing membership information. It doesn't matter which node you select.

2. Login to this Inductor's admin console (http://<Inductor's IP>:8080/multisite-local/) and gather the following information, most is available from the Settings tab.

All your remaining sites are now classed as Inductees.

3. Select one of your remaining Inductee sites. Connect to its web admin console (http://<Inductee1:8080/multisite-local/) and go to the Nodes tab.

4. Click on the CONNECT TO NODE button and enter the details that you collected from your Inductor node.

Node Id *

The name of the inductor node - you can verify this from the NODE ID entry on the Inductor node's Settings tab (see step 2, above.)

Node Location Id *

The reference code that is used to define the inductor node's location -you can verify this from the NODE ID entry on the Inductor node's SETTINGS tab (see step 2, above).

Node IP Address *

The IP address of the inductor node server.

Node Port No *
The DConE Port number (6444 by default), defined on the inductor node's SETTINGS tab.

When these details are entered, click the SEND CONNECTION REQUEST button. The inductor node will accept the request and add the inductee to its membership. You will need to refresh your browser to see that this has happened..

5. Go back to step 3 and select one of your remaining inductees. Repeat this process until all the sites that you want to be included in the current membership have been connected to the inductor.

If Induction fails

If the induction process fails for whatever reason you may be left with the inductee in a pending state.

From the Nodes tab, review the state of your prospective node. During the induction process a prospect will display a Connectivity Status of "Pending Induction". The process should complete within a few seconds, providing that there isn't a network connection problem.

If the prospect appears to be stuck in the pending state then click the Cancel Induction link.

A growl message will confirm that the induction was cancelled successfully. Click the Reload button to clear the cancelled induction.
Repeat the induction procedure after confirming:
- You are entering the correct details for the inductee node.
- There isn't a network outage between nodes.
- There isn't a network configuration problem, such as a firewall blocking the necessary ports.
- There isn't an admin account mismatch between nodes - this occurs if you don't use the correct procedure for installing a second or subsequent node If the admin account doesn't match because nodes were not installed using the first node's user.properties file then you should follow Matching a node's admin settings.
- There isn't a product license problem. Should the license file clash between two nodes, or be missing from a node this could cause induction to fail. License problems are noted in the Application Logs.

Matching a node's admin settings

We ensure that all nodes start with a common admin account by importing the admin settings from the first installed node during the installation of all subsequent nodes. In the event that a node is accidently installed without this match you can use the following procedure to resync them. You'll need to follow this if you wish to induct the mismatched node into a replication network that includes the other nodes.

Log in to your first node, click on the Security and click Export Security Settings to perform a security (user) settings export.
Access the same node using a terminal window. Copy the exported settings file (/opt/wandisco/svn-multisite-plus/replicator/export/security-export.xml) to a location on the node that you fixing. e.g.
```
/opt/wandisco/svn-multisite-plus/replicator/import/security-export.xml
```
Log in to the admin UI of the node that you're fixing. Click on the Security tab then click the Import Secure Settings button.
Enter the path to the copied across security-export.xml file then click Check. You'll be presented with a Diff report that shows you what differences exist between the current user settings and those in the exported file. Click Import to overwrite the existing admin user settings with the correct user settings that will match those used in the other nodes.

Now that the admin user account details are matching again you'll be able to complete an induction of the corrected node into a replication network.

6. Create a Replication Group

SVN MultiSite Plus lets you share specific repositories between selected sites. This is done by creating Replication Groups that contain a list of sites and the specific repositories they will share.

This illustration shows a collection of four sites that are running two replication groups. Replication Group one replicates Repo1 across all four sites, whilst Replication Group 2 replicates repo2 across a subset of sites.

Follow this procedure to create a Replication Group. You can create as many replication groups as you like. However, each repository can only be part of one active replication group at a time.

1. Once you have sites defined, click on the REPLICATION GROUPS tab. Then click on the CREATE REPLICATION GROUP button.

Replication Group Validation
The admin UI won't let you create a replication group that doesn't meet the requirement set by DConE, for example, the proposed replication group must not have an even number of "acceptor" nodes (without also having a tie-breaker. When the selected member nodes don't make a valid replication group, the "Create Replication Group" button will be disabled (greyed out).

Create Replication Group

** Alert! ** Local node automatically made the first member
It's not possible to create a replication group remotely - the node on which you are creating the group must itself be an member. For this reason, when creating a replication group, the first node is added automatically.

2. Enter a name for your Replication Group in the Replication Group Name field. Then enter an existing Node name in the Add Sites field - any existing sites that match your entry will appear and can be selected with a click. Instead of typing in a name you can click on the drop-down button and choose from a list of existing sites (that are not already members of the new group).

You can select any number of available Sites. Those sites that you select will appear as clickable buttons in the Add Node field.

Enter a name and add some nodes.

3. New sites are added as Active Voters (denoted with "AV"). You can change the type of a node by clicking on its label. For an explanation of what each node type does, view the Reference Section - Node Types

Change node type
Once you have added all sites and configured their type, click CREATE REPLICATION GROUP to see a groups details.

4. Replication Groups that you create will be listed on the REPLICATION GROUPS tab.

Groups boxes, click QUICK VIEW view your options.

7. Add Repositories

Once you have added at least one Replication Group you will be able to add repositories to your node. Here's how:

Warning
A repository UUID is integral to the way SVN Multisite Plus tracks repositories. You must not introduce repositories that have duplicate UUIDs.
This means:

You cannot add two existing repositories with the same UUID.
You cannot use "svnadmin load --force-uuid" if the load will use a UUID that already exists on an SVN MultiSite Plus replicated repository.
You cannot use "svnadmin setuuid" on a repository, and use a UUID that already exists on an SVN MultiSite Plus replicated repository.

1. Click on the REPOSITORIES tab. Click on the ADD button.

Repositories > ADD
2. Enter the Repository's name, the file system path (full path to the repository) and use the drop-down to select the replication group. You can set the repository to be Read-only by ticking the Global Read-Only option. This option, if selected, means that the repository will not accept write activity from any node until the setting is cleared. You may find it useful to set this to verify successful replication activity before allowing end user access to the repository. You can deselect this later. Click ADD REPO.

Repositories > Enter details then click ADD REPO
3. Once added, a repository will appear in a list on the REPOSITORIES tab. The list provides the following details.

Repositories listed

Repo Name

The name you assign to the repository - this can be the same name that you give to the repository's folder on the file system, or it could be something else. There are no obvious limitations when naming a repository, although we recommend that you use a conservative naming scheme that avoids spaces and special characters etc.

FS Path

The file system path to the Repository.

Replication Group

The Replication Group in which the repository will be replicated.

Size

The file size of the repository. We use the binary prefix (KiB, MiB, GiB, etc) to denote that data is measured in blocks of 1024 bytes.

Youngest Rev

The youngest (latest) revision in the repository. Comparing the youngest revisions between replicas is a quick test that a repository is in the same state on all sites.

Transactions

The number of pending transactions associated with each repositorty.

Last Modified

The timestamp for the last revision, which provides a quick indicator for the last time a SVN user made a change.

Global RO

Checkbox that indicates whether the repository is globally Read-only, that is Read-only at all sites.

Local RO

Checkbox that indicates whether the repository is locally Read-only, that is Read-only to users at this node. The repository receives updates from the replicas on other sites, but never instigates changes itself.

Status

Indicates the replication status of each repository. Normally a repository status will be "Replicating".