4. Administrator's Guide

5.1 Running Git MultiSite

This guide runs through everything you need to know to get Git MultiSite deployed. First we'll cover all the things that you need to have in place before you install. We'll then cover a standard installation and setup. Finally we'll look at some possible problems you might experience with some troubleshooting tips.

Starting up

To start the Git MultiSite replicator, follow these steps:

1. Open a terminal window on the server and login with suitable file permissions.
2. Run the git-multisite service, located in the init.d folder:

   lrwxrwxrwx  1 root root    37 May  9 10:37 git-multisite -> /opt/git-multisite/bin/git-multisite
   

3. Run the start script:

   [root@localhost init.d]#  ./git-multisite start
   
   
   20130520-164811 (24088) [INFO]: Starting WANdisco MultiSite 
   20130520-164811 (24088) [INFO]: Started replicator (24100)
   20130520-164811 (24088) [INFO]: Started ui (24110)
   20130520-164811 (24088) [INFO]: Number of errors: 0
   20130520-164811 (24088) [INFO]: Number of warnings: 0
   

4. The two components of Git MultiSite; the replicator and the UI will start up. Read more about the git-multisite init.d script

The Dashboard

The dashboard is the first page you'll see when Git MultiSite starts. You can add widgets to your dashboard to display information about your repositories, such as size or latest activity, or about replicator tasks. We'll be adding more widgets in future releases to make your dashboard even more configurable.

Security

The security tab is used to manage admin accounts, either entered manually into Git MultiSite or managed through an LDAP authority. On the tab is an entry form for multiple admin accounts, along with LDAP Settings for binding MultiSite to one or more LDAP services.

Add User

Enter the details of an additional administrator who will be able to login to the Git MultiSite Admin UI. See Adding additional users for more information.

Add Authority

Enter the details of one or more LDAP authorities for managing administrator access. See Adding LDAP authorities for more information.

Disable Managed Users

This feature lets you block access to the Git MultiSite Admin UI by non-LDAP users. See Disabling (Internally) Managed Users below.

Export Security Settings

The data entered into the Securities tab can be backed up for later re-importing by clicking the Export Security Settings button. The data is stored in /opt/wandisco/git-multisite/replicator/export/security-export.xml which should be included in any backup procedures you are running. You will need access to the file from your desktop during a re-import.

Import Security Settings

Click the Import Security Settings button if you need to restore your Security settings, such as after a re-installation of Git MultiSite. The import will proceed providing that you can enter a file path to the security-export.xml file.

Reload

Click on the reload button to refresh the Admin UI screen, you will need to do this in order to view any changes that you make.

Admin Account Precedence

Git MultiSite uses the following order of precedence when checking for authentication of users:

• First: Internally managed users (if they are enabled - see Disable Managed Users)
• Second: Local LDAP authorities by order
• Third: Global LDAP authorities by order

This provider implementation tries to authenticate user credentials against either the list of internally managed users, or against any number of LDAP authorities, or both -- depending on how the administrator has configured the application.

When authenticating against LDAP authorities, each one is tried in sequence until one either grants access or they all deny access. In the event that they all deny access, only the error from the last authority tried will be returned.

Internally Managed Users

This table lists those admin users who have been entered through the Admin UI or imported using the Import Security Settings, along with the first admin account.

Admin Account #1

Note that the first admin account is the one set up during the installation of your first node. The credentials specified during this installation are stored to the users.properties file which is then used during the installation of all subsequent nodes.

Disable (Internally) Managed Users

Click the Disabled Managed Users button if you want to control access to Git MultiSite exclusively through LDAP. Once clicked, any Internally managed users will no longer be able to log into the Admin UI after they next log out. From that point only LDAP managed users will have access to the Git MultiSite Admin UI.

Admin Account #1 can be removed but the last admin account remaining on the system will not be deletable to ensure that it isn't possible for an administrator to be completely locked out of the admin UI.

/opt/wandisco/git-multisite/replicator

java -cp lib/*:./resetSecurity.jar com.wandisco.multisite.ui.local.util.security.ResetSecurity

Shutting down

To shutdown:

1. Open a terminal window on the server and login with suitable file permissions.
2. Run the git-multisite service, located in the init.d folder:

   lrwxrwxrwx  1 root root    37 May  9 10:37 git-multisite -> /opt/multisite-plus/bin/git-multisite

3. Run the stop script, i.e.:

   [wandisco@ip-10-0-100-7 bin]$  ./git-multisite stop
   
   20130520-165704 (24767) [INFO]: Stopping WANdisco MultiSite
   20130520-165704 (24767) [INFO]: Request received to shut down replicator
   20130520-165704 (24767) [INFO]: replicator processes ended
   20130520-165704 (24767) [INFO]: Request received to shut down ui
   20130520-165704 (24767) [INFO]: Sending signal 15 to watched ui process (attempt 1)...
   20130520-165707 (24767) [INFO]: Sending signal 15 to watched ui process (attempt 2)...
   20130520-165710 (24767) [INFO]: ui processes ended
   20130520-165710 (24767) [INFO]: Number of errors: 0
   20130520-165710 (24767) [INFO]: Number of warnings: 0
   

4. Both the replicator and the UI processes will shut down. Read more about the git-multisite init.d script

init.d script

The 'start-up' script for persistent running of Git MultiSite can be found in the /etc/init.d folder. Run the script with the help command to list the available commands:

[root@localhost init.d]# ./git-multisite help
Usage: git-multisite {start|stop|status|uistart|uistop|repstart|repstart}

  start      Start WANdisco Multisite service
  stop       Stop WANdisco Multisite service
  status     Display running WANdisco Multisite services
  uistart    Start WANdisco Multisite UI service
  uistop     Stop WANdisco Multisite UI service
  repstart   Start WANdisco Multisite Replicator service
  repstop    Stop WANdisco Multisite Replicator service    

Changing the admin console password

You can change Git MultiSite's login password at any time by following this procedure:

1. Login to the MultiSite admin console.
   
   

   Login.

2. Click on the Settings tab.
   
   

   Settings.

3. At the top of the settings screen is the password change form. Enter the current password, along with a new password.
   
   

   Changed password

4. Click the SAVE button store the new password. You can be sure that the new password has been accepted if you see a growl message appear on screen.
   
   

   Growl!

Updating your license.key file

Follow this procedure if you ever need to change your product license. You would need to do this if, for example, you needed to increase the number of subversion users or the number of replication nodes.

1. Login to your server's command line, navigate to the replicator directory (by default: /opt/git-multisite/replicator ) and rename the license.key to license.20130625.
   i.e.

   total 1120
   drwxr-xr-x 2 root root   4096 Jun 19 17:21 content
   drwxr-xr-x 5 root root   4096 Jun 19 17:21 database
   drwxrwxr-x 4 root root   4096 Jun 19 17:13 docs
   drwxrwxr-x 2 root root   4096 Jun 19 17:13 lib
   -rw-r--r-- 1 root root    256 Jun 19 17:18 license.key <-
   drwxrwxr-x 3 root root   4096 Jun 25 14:06 logs
   drwxrwxr-x 2 root root   4096 Jun 19 17:13 properties
   

2. Get your new license.key and drop it into the /opt/git-multisite/replicator directory.
3. Restart the replicator by running the SVN MultiSite script with the following argument:

   perl /etc/init.d/git-multisite repstart

   This will trigger a replicator restart, which will force Git MultiSite to pickup the new license file and apply any changes to permitted usage.

PANIC: License is invalid com.wandisco.fsfs.licensing.LicenseException: Failed to load filepath>

Application Properties file

The application.properties file contains configurable options for your Git MultiSite installation.

You should only make changes to this file if you are confident of the outcome, or have been instructed to do so by WANdisco's Support Team.

An example application.properties file is below:

#Mon Feb 03 16:55:07 GMT 2014
content.location=/opt/wandisco/git-multisite/replicator/content_delivery
application.hostname=10.4.4.64
beacon.period=1000L
jetty.https.port=8445
database.location=/opt/wandisco/git-multisite/replicator/database
content.min.learners.required=true
content.push.policy=Faster
deadlock.detect.interval=300000L
stats.collection.period=300
content.learners.count=1
content.transfer.md5sum=true
ssl.enabled=false
database.isPersistent=true
resourcemonitor.location=/opt/wandisco/git-multisite/replicator/properties
location.property.pending=true
content.server.port=4321
application.location=/opt/wandisco/git-multisite/replicator
ssl.debug=false
application.port=6444
prevayler.snapshot.interval=900000L
location.property.latitude=53.3833
jetty.http.port=8082
delegate.port=7777
membership.rotation.period=3600000
communication.hostName=10.4.4.64
node.id=Node1
location.property.longitude=-1.4667
gitms.local.jetty.port=9999

5.2 Managing Replication

This section covers the various functions that handle how repository data is replicated.

Adding a Node

To replicate Git repository data between sites, you first first tie the sites together, this process starts with adding (connecting) sites in a process we call induction.

1. Login to the Git MultiSite admin console of the new node that you are connecting to your existing servers.
2. Click on the Sites tab.
3. Click on the Connect to node button.
   

   Connect to Node

4. Enter the details of an existing node. You can get these details from the Settings tab of the existing node.
   

   Enter the details from an existing, connected node.

   
   

   Node Node ID

   This is the name that you gave the Node during installation. If you log into the node in question you can see the Node ID on the title of any screen that you view, it also appears in the logged in message: "Welcome to MultiSite, admin. You are connected to <NODE ID>"

   Node Location ID

   A unique string that that a node creates for use as in identifier. You can get this from the node's Settings tab:
   

   System Data table, found on the Settings tab.

   
   

   Node IP Address

   The IP Address of the node's server.

   Node Port No

   The TPC port that the node uses for DConE, which handles agreement traffic. The default is 6444 See Reserved Ports.

5. Click on the SEND CONNECTION REQUEST button. The new node will appear on the active list of Sites. Should a problem occur you may find that the new node gets stuck in a 'pending' state. If this happens click on the Cancel button.

Removing a Node

The method of removing a node depends on the current state of the node you want to remove:

• The node is not part of a replication group

If the node is not currently in a replication group you can remove it by browsing to the nodes tab, selecting the node and then the 'Remove Node' button.

• The node is part of a replication group

If the node is currently part of a replication group you'll need to remove it from the group before deleting it. You can do this either by browsing to the replication group and removing it, or by browsing to the node and selecting the 'Emergency Reconfiguration' button. Emergency reconfiguration will remove the node from any groups it is part of.

Adding a Replication Group

Use the procedure to add a new Replication Group. You need to add a new replication group when you need to replicate between a new combination of sites - i.e. sites that are not currently replicating in an existing group. If you are, instead, looking to replicate a new repository between existing sites, it's possible to add new a new reposity to those sites. In this case see Add a new repository.

1. Log in to the Git MultiSite browser-based user interface. Click on the REPLICATION GROUPS tab, then click on the CREATE REPLICATION GROUP button.
   

   Creating a replication group.

   
   
2. Enter a name for the group in the Group Name field, then click on the drop-down selector on the Add Sites field. Select the sites that you want to replicate between.
   

   replication group details.

   
   
   Replication Ground Rules
   - A node can belong to any number of replication groups
   - A repository can only be part of a single Active replication group at any particular time.
   - It's possible to change membership on the fly, moving a repository between replication groups with minimal fuss.
3. Ensure that member sites are each set as an appropriate type.
   

   Click on node labels to change their type.

   
   
   Quick guide to setting node types
   All sites are automatically added as the Active Voter type. To understand the differences between the different types of sites, read Guide to Node Types
4. Once all sites are in place and their settings adjusted to your needs, click CREATE REPLICATION GROUP.
   

   Create Replication Group.

   
   
5. Newly created replication groups will appear on the Replication Group tab, but only at the node that are themselves members of the new group.
   

   The new replication group now appears - if you are logged into one of its constituent sites.

   
   

Scheduling node changes - follow the sun

You can schedule the member sites of a replication group to change type according to when and where it is most beneficial to have active voters. To understand why you may want to change your sites, read about Node Types

Use the following API call

http://<ip>:8082/public-api/replicationgroup/{repgroupID}/schedule

http://10.0.100.135:8082/public-api/replicationgroup/97913c04-bbad-11e2-877a-028e03094f8d/schedule

To make Node N3 a tie-breaker 'T' FROM 10:00 - 16:00 (GMT) every day of the week with Node N1 as tie-breaker 'T' afterwards: (N.B Times are in GMT -4 hours so 14 = 10:00 GMT)

Example curl command:

Make a text file containing ReplicationgroupAPIDTO XML (as above) called schedule.xml

curl -u username:password -X PUT -d @schedule.xml http://[IP]:[PORT]/public-api/replicationgroup/97913c04-bbad-11e2-877a-028e03094f8d/schedule

sample 'schedule.xml' file

1. Login to a node, click on the REPLICATION GROUPS tab. Click on the QUICK VIEW link for the replication group that you wish to make a schedule.
   
   
2. The replication group's pop-up window will open, showing the member groups together, along with their current roles. Click the CONFIGURE button.
   

   Configure.

   
   
Membership views are what is scheduled not necessarily what is currently active The roles and membership displayed in the popup is based upon the agreed schedule, it's the setup that should be in place if everything is running smoothly. It is always possible that it doesn't accurately represent the state of the replication group, due to a delay in processing on a node, or if something has caused a process to hang. This should not be a cause for concern but it's important to be aware that the displayed membership is an approximation based on the information currently available to the local node.
3. The replication groups configuration screen will appear. You may notice that to the left a Role Schedule is noted. By default this will show as DISABLED. Click on the CONFIGURE SCHEDULE button, in the right-hand corner.
   
   
4. The Schedule screen will appear. The main feature of the screen is a table that lists all the sites in the replication group, set against a generic day (midnight to midnight) that is divided into hourly blocks. Hourly block is color-coded to indicate each site's type.
   In the image below NodeSanFrancisco is coded as turquoise (blue-green) which indicates that it is set as a tie-breaker. The hourly blocks associated with NodeChengdu and NodeAuckland are colored yellow, indicating that these sites are Active Voters. Finally, NodeParis is blank, indicating that it is Active.
   
   

   Vanilla Scheduling - no changes to type over time.

   
   
   
5. To make a change to the shedule, click on a block. It doesn't matter which block you select as the New Scheduled Configuration form will let you modify any hours for any available node.
   

   Frequency

   Select from the available frequency patterns: Daily, Weekly, Monday-Friday or Saturday to Sunday.

   From

   The starting hour for the new schedule, e.g. 00 for the start of the day.

   To

   The hour at which the scheduled changes end, e.g. 24 would effectively end the scheduled change at midnight.

   Sites list

   The member sites are listed, in graphical form, colour coded to their type.

6. Click on the node icon to change its type.

   In this example NodeSanFransisco is changed to an Active Voter, then NodeAuckland is changed into a Tie-breaker.
   

   
   

   Swapping roles.

   When all node changes have been made, click on the SAVE button to continue, or the CANCEL button if you change your mind.


7. The schedule view will now change to show the changes that you make. You must click the SAVE SCHEDULE button for the changes to be applied.
   
   

   Swapping roles.

   
   With all necessary changes made, you need to review the change to the schedule table and then click SAVE SCHEDULE button.

Hooks

There are two types of hooks which need to be taken into account for replication, pre- hooks and post- hooks.

When used in a Git MultiSite setup these hooks will only fire at the node which received the push request. A new hook specific to Git MultiSite, rp-post-receive will fire at all the other nodes in the replication group.

The rp-post-receive hook will receive the same stdin as the hook on the originating node, and can be found in the hooks directory.

5.3 Managing Repositories

Add Repositories

Once you have added at least one Replication Group you will be able to add repositories to your node. Here's how:

• 1. Click on the REPOSITORIES tab. Click on the ADD button.
  
  

  Repositories > ADD

• 2. Enter the Repository's name, the file system path (full path to the repository) and use the drop-down to select the replication group. You can set the repository to be Read-only by ticking the Global Ready-only You can deselect this later. Click ADD REPO.
  
  

  Repositories > Enter details then click ADD REPO

Repository stuck in Pending state


If a repository that you added gets stuck in the deploying state - you'll see this on the Dashboard, in the Replicator Tasks window - you can cancel the deployment and try adding the repository again. To cancel a deployment, go to the Replicator Tasks window and click on the Cancel Task link.

• 3. Once added, a repository will appear in a list on the REPOSITORIES tab. The list provides the following details.
  
  

  Repositories listed

  
  

  Name

  The name of the repository - this will be the same as the folder name in the Git directory.

  Path

  The full path to the Repository.

  Replication Group

  The Replication Group in which the repository may be replicated.

  Size

  The file size of the repository.

  Youngest Rev

  The youngest (latest) revision in the repository. Comparing the youngest revisions between replicas is a quick test that a repository is in the same state on all sites.

  Last Modified

  The timestamp for the last revision, which provides a quick indicator for the last time a Git user made a change.

  Global RO

  Checkbox that indicates whether the repository is globally Read-only, that is Read-only at all sites.

  Local RO

  Checkbox that indicates whether the repository is locally Read-only, that is Read-only to users at this node. The repository receives updates from the replicas on other sites, but never instigates changes itself.

Edit a repository

It's possible to Edit a repository properites after they have been set up in Git MultiSite. Follow this quick procedure.

1. Login to the admin console of one of your sites. The node will need to be the member of a replication group in which the repository is replicated, otherwise it won't appear on the tab. Click on the Repositories tab to see it.
   
   

   Login.

2. On the Repositories tab, click on the line that corresponds with the repository that you want to remove.
   
   

   Repositories.

3. Once a repository has been highlighted (Shown as a blue line), the EDIT button will become available. Click it.
   

   Edit.

   
   The repository edit window will open up.

   Local Read-only

   Change the Read-only setting, enable or disable the repository Local Read-only setting. When enabled, the repository will not be writable to local users. However, changes that are made on replica at other sites will be applied through inter-site replication.

   Global Read-only

   Change the Read-only setting, enable or disable the repository Global Read-only setting. When enabled, the repository will not be writable either locally or globally. This is used to lock a repository from any changes.

   Replication Group

   Use the drop-down selector to change the replication group to which the repository belongs.

Remove a repository

It's possible to remove repositories from Git MultiSite. Follow this quick procedure.

1. Login to the admin console of one of your sites. The site will need to be the member of a replication group in which the repository is replicated, otherwise it won't appear on the tab. Click on the Repositories tab to see it.
   
   

   Login.

2. On the Repositories tab, click on the line that corresponds with the repository that you want to remove.
   
   

   Repositories.

3. Once a repository has been highlighted (Shown as a blue line), the REMOVE button will become available. Click it.
   

   Remove.

   
   A dialog box will appear entitled "Remove repository from replication group". It will confirm that removing a repository from a replication group will stop any changes that are made to it from being replicated. However, no repository data is removed.

5.4 Back up Git MultiSite Data

It's possible to back up Git MultiSite's own database in case you need to quickly restore a node.

Create a backup of the current installation by invoking the following API call:

curl --user <username>:<password> -X POST http://[node_ip_address]:8082/dcone/backup

This will create a backup folder in [INSTALL-DIR]/git-multisite/replicator/db/backup/X.X.X_DConE_Backup directory.

Back up while shut down

java -cp ./fsfsrestore.jar com.wandisco.fsfs.backup.FsfsBackup -c ./properties/application.properties

Use this to back up the current state of all pervailers when Git MultiSite is shut down - you don't therefore need to start the replicator in order to create a backup of the database.

Back up selected prevailers

java -cp ./fsfsrestore.jar com.wandisco.fsfs.backup.FsfsClear -c ./properties/application.properties

This class clears selected prevaylers only, when the replicator is shut down. It does not clear all database instances, only those that are being restored during restore process.

5.5 Restore Git MultiSite Data

Use the following procedure to restore Git MultiSite settings after reinstalling and starting a node:

1. Shutdown the node
2. Run the following jar file:

   java -jar fsfsrestore.jar path/to/application.properties path/to/back-up-folder

   the first argument, path/to/application.properties is
   <INSTALL-DIR>/multisite-plus/replicator/properties/application.properties

   path/to/backup by default is
   <INSTALL-DIR>/multisite-plus/replicator/db/backup/

   FsfsBackup.class path/to/application.properties

3. Restart the node

5.6 Running Git MultiSite with Apache

Before you start:

• Ensure that GitMultiSite has been installed and configured for replication, using gitms as the user.
• Ensure that at least one repo has been replicated successfully and is in the directory /home/gitms/repos/ on each machine.

Install Apache

 yum install httpd mod_ssl

Next, ensure Apache will start on system restart

 chkconfig --add httpd

 service httpd start

Configure SeLinux

If SeLinux is running:

Enable access to home directories - ie /usr/home/gitms

setsebool -P httpd_enable_homedirs on 

Install semanage

usermod -a -G apache gitms

yum -y install policycoreutils-python 

Next, allow httpd read/write access to /home/gitms

chcon -R -t httpd_sys_rw_content_t /home/gitms

chcon -R -t httpd_sys_rw_content_t /opt/wandisco/git-multisite/replicator/content_delivery

Then we need to allow the update script to make a network connection to the Java service

setsebool -P httpd_can_network_connect on

setsebool -P git_system_enable_homedirs on

Configure IPTables

/etc/sysconfig/iptables should look like:

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [11:12222]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 6789 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 8082 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 8085 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 9001 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT
# Completed on Mon Jul 29 13:52:30 2013
Increased the OUTPUT ACCEPT from 2222 to 12222 (allows outgoing connections up to port 12222)
Allow incoming connections on 6789, 8082, 8085, 9001 and 443.

Create your HTTP password file

htpasswd -c /var/www/passwd username

Change the ownership of the passwd file:

chown apache:apache /var/www/passwd

Create git-http-backend Wrapper Script

The script executed by suexec must be under /var/www

mkdir -p /var/www/bin

Add the following to /var/www/bin/git-http-backend

#!/bin/bash
PATH_INFO=$SCRIPT_URL
GIT_PROJECT_ROOT=/home/gitms/repos
REMOTE_USER=$REDIRECT_REMOTE_USER
export GIT_HTTP_EXPORT_ALL=true
/usr/libexec/git-core/git-http-backend
chmod +x  /var/www/bin/git-http-backend 

The script and the directory it is in must be owned by the user who will be executing the script:

chown -R gitms:gitms /var/www/bin

You'll need to tell selinux that /var/www/bin has httpd exectuable scripts)

semanage fcontext -a -t httpd_sys_script_exec_t /var/www/bin  

restorecon /var/www/bin

Add Apache Config

Copy the following into /etc/httpd/conf.d/repo.conf

<VirtualHost *:80>
DocumentRoot /home/gitms/repos
ServerName git.example.com

<Directory "/home/gitms/repos">
Allow from All
Options +ExecCGI
AllowOverride All
</Directory>

<Location /repos>
AuthType Basic
AuthName "Private Git Access"
AuthUserFile "/var/www/passwd"
Require valid-user
</Location>

SuexecUserGroup gitms gitms
ScriptAlias /repos /var/www/bin/git-http-backend

Allow git pushes

Change to the directory your repo is located in

su - gitms -s /bin/bash
cd /home/gitms/repos/bar.git
git config http.receivepack true
git config core.sharedRepository group

This needs to be done in each repo and on all replicas.

Restart Apache and Test

service httpd restart
git clone http://192.168.122.1/repos/bar.git

HTTPS Support

Generate Certificates

You can use tools such as easy-rsa to generate certificates.

You need to have the Epel rep installed to use this:

wget http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
rpm -Uvh epel-release-6-8.noarch.rpm
yum install easy-rsa
cp -r /usr/share/easy-rsa/2.0 .
cd 2.0/
source vars
./clean-all
./build-ca
./build-key-server git-http1     (when prompted for the CommonName use the system IP address if the system does not have a register DNS name)
./build-key-server git-http2
./build-key-server git-http3

This will generate three server certs/keys as well as the ca cert/key in the ./keys directory.

Copy a cert and key into /etc/apache on each node.

cp ./keys/git-http1.crt /etc/httpd

cp ./keys/git-http1.key /etc/httpd

chown apache:apache /etc/httpd/git-http1.crt /etc/httpd/git-http1.key

Modify Apache Config to support SSL

Edit /etc/httpd/conf.d/repo.conf

<VirtualHost *:443>
DocumentRoot /home/gitms/repos
ServerName git.example.com
SSLEngine on
SSLCertificateFile /etc/httpd/git-http1.crt
SSLCertificateKeyFile /etc/httpd/git-http1.key

<Directory "/home/gitms/repos">
Allow from All
Options +ExecCGI
AllowOverride All
</Directory>

<Location /repos>
AuthType Basic
AuthName "Private Git Access"
AuthUserFile "/var/www/passwd"
Require valid-user
</Location>
 
SuexecUserGroup gitms gitms
ScriptAlias /repos /var/www/bin/git-http-backend
</VirtualHost>

Restart service:

service httpd restart

Add the CA certificate to client system:

The server certificate generated will not be recognized by your client. You can either turn of strict SSL checking:

git config --global http.sslVerify false

Or the CA certificate can be added to the client machine, this means adding the ./keys/ca.crt file to the set of CA certificates the client system accepts. Instructions for doing this can be found here: http://askubuntu.com/questions/73287/how-do-i-install-a-root-certificate

Test:

$ git push

WARNING: gnome-keyring:: couldn't connect to: /tmp/keyring-P25n4M/pkcs11: No such file or directory
Password for 'https://test@192.168.122.207': 
remote: GitMS - update replicated.
To https://test@192.168.122.207/repos/bar.git
 d434f14..8b7bab0 master -> master
$

5.7 Git MultiSite Authentication and Authorization

Overview

Git MultiSite authentication is performed by a third party service. Once the authentication is complete, details of the requested git operation and a username are passed back to Git MultiSite.

The operation and username are then checked internally by Git MultiSite to ensure sufficient permissions are available to perform the operation.

Authentication can use either:

• SSH
• HTTP with an authentication mechanism such as htpasswd or LDAP

5.7.1 Authentication

As mentioned already, authentication can be by either SSH directly or via an HTTP authentication mechanism. It is normal to only use one or the other, but it is possible to use both in parallel.

Here are examples of using either SSH or Apache for authentication:

Authentication through Apache

Apache authentication allows users to communicate via the HTTP(S) protocol. This is beneficial in environments with heavily restricted firewalls, as the usual ports 80 and 443 are used for communication.

Git has two HTTP-based protocols, namely 'git over HTTP' and 'Smart HTTP'. Git MultiSite only supports the 'Smart HTTP' protocol, due to improved functionality compared with the older 'git over HTTP' protocol, especially with regards to speed of operation.

Assumptions

The following assumptions are made:

• Git MultiSite is already set up and configured for replication
• Git MultiSite runs with the 'gitms' user and 'gitms' group
• The base directory for managed repositories is /home/gitms/repos/
• The apache2+suexec package is installed
• If an htpasswd file is required, it is stored in /var/www/passwd
• Apache runs as user 'apache'
• Apache configuration is in its default directory, /etc/httpd/conf.d/

Authentication providers

We'll show you how Apache can be configured to authenticate against either an internal file (htpasswd) or an LDAP directory service.

Authentication by htpasswd

This requires an htpasswd file to be created to store usernames.

Creating the htpasswd file

htpasswd -c /var/www/passwd <username>

Adding users to an existing file:

htpasswd /var/www/passwd <username>

Configuration for htpasswd

Add a file called repo.conf in the /etc/httpd/conf.d/ directory with the following contents:

<VirtualHost *:80>
# 80 is the port the webserver will bind to
DocumentRoot /home/gitms/repos
# The base directory for repositories managed by Git MultiSite
ServerName git.example.com
RewriteEngine On
RewriteCond %{REMOTE_USER} ^(.*)$
RewriteRule ^(.*)$ - [E=R_U:%1]
RequestHeader set X-Remote-User %{R_U}e
 
<Directory "/home/gitms/repos">                        
Allow from All
Options +ExecCGI
AllowOverride All
</Directory>
 
<Location /repos>
# This matches the location in the requesting url,
# for example, matches against request http://<ip>/repos/
AuthType Basic
AuthName "Private Git Access"
AuthUserFile "/var/www/passwd"
Require valid-user
</Location>
 
SuexecUserGroup gitms gitms
ScriptAlias /repos /var/www/bin/git-http-backend
# This script alias redirects matches made
# earlier to a script we will create later
</VirtualHost>

Authentication by LDAP

Apache can use an LDAP directory to authenticate against. Unlike htpasswd, this does not require the maintenance of a separate passwd file.

Add a file called repo.conf in the /etc/httpd/conf.d/ directory with the following contents:

<VirtualHost *:80>                                           
DocumentRoot /home/gitms/repos                             
ServerName git.example.com
RewriteEngine On
RewriteCond %{REMOTE_USER} ^(.*)$
RewriteRule ^(.*)$ - [E=R_U:%1]
RequestHeader set X-Remote-User %{R_U}e
 
<Directory "/home/gitms/repos">
Allow from All
Options +ExecCGI
AllowOverride All
</Directory>
 
 
<Location /repos>
AuthType Basic
AuthName "Git Repos"
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPURL "ldap://LDAP-IP:389/CN=CN-details,DC=DC-details,DC=DC-details?uid"
 
#If the LDAP directory requires a bind user and password:
AuthLDAPBindDN "CN=Administrator,CN=Users.DC=sr,DC=wandisco,DC=com"
AuthLDAPBindPassword password
 
Require valid-user
</Location>
 
SuexecUserGroup gitms gitms
ScriptAlias /repos /var/www/bin/git-http-backend   
</VirtualHost>

Configuration using HTTPS

You can set up Apache to use HTTPS rather than HTTP. This is preferred in Enterprise settings due to the security benefits.

Using HTTPS with htpasswd:

Add a file called repo.conf in the /etc/httpd/conf.d/ directory with the following contents:

<VirtualHost *:443>
DocumentRoot /home/gitms/repos
ServerName git.example.com
RewriteEngine On
RewriteCond %{REMOTE_USER} ^(.*)$
RewriteRule ^(.*)$ - [E=R_U:%1]
# The following two lines will redirect port 80 (HTTP) to 443 (HTTPS)
# if SSL/TLS is always required:
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
RequestHeader set X-Remote-User %{R_U}e
SSLEngine on
SSLCertificateFile /etc/httpd/git-http1.crt
SSLCertificateKeyFile /etc/httpd/git-http1.key
 
<Directory "/home/gitms/repos">
Allow from All
Options +ExecCGI
AllowOverride All
</Directory>
 
<Location /repos>
AuthType Basic
AuthName "Private Git Access"
AuthUserFile "/var/www/passwd"
Require valid-user
</Location>
 
SuexecUserGroup gitms gitms
ScriptAlias /repos /var/www/bin/git-http-backend
</VirtualHost>

Using HTTPS with LDAP

Add a file called repo.conf in the /etc/httpd/conf.d/ directory with the following contents:

<VirtualHost *:443>
DocumentRoot /home/gitms/repos
ServerName git.example.com
RewriteEngine On
RewriteCond %{REMOTE_USER} ^(.*)$
RewriteRule ^(.*)$ - [E=R_U:%1]
#  The following two lines will redirect port 80 (HTTP) to 443 (HTTPS)
# if SSL/TLS is always required:
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
RequestHeader set X-Remote-User %{R_U}e
SSLEngine on
SSLCertificateFile /etc/httpd/git-http1.crt
SSLCertificateKeyFile /etc/httpd/git-http1.key

<Directory "/home/gitms/repos">
Allow from All
Options +ExecCGI
AllowOverride All
</Directory>

<Location /repos>
AuthType Basic
AuthName "Git Repos"
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPURL "ldap://LDAP-IP:389/CN=CN-details,DC=DC-details,DC=DC-details?uid"

#If the LDAP directory requires a bind user and password:
AuthLDAPBindDN "CN=Administrator,CN=Users.DC=sr,DC=wandisco,DC=com"
AuthLDAPBindPassword password

Require valid-user
</Location>

SuexecUserGroup gitms gitms
ScriptAlias /repos /var/www/bin/git-http-backend
</VirtualHost>

Create git-http-backend script

Create a script called git-http-backend as follows:

#!/bin/bash
GIT_PROJECT_ROOT=/home/gitms/repos
# This value should be configured to match the base location of repos on disk
export GIT_BASEDIR=$GIT_PROJECT_ROOT
export GIT_HTTP_EXPORT_ALL=true
# Execute gitms_shell script
exec /opt/wandisco/git-multisite/bin/gitms_shell $REMOTE_USER

This script location should match where you set it to be called from in the repo.conf file on the ScriptAlias line - we have used /var/www/bin/git-http-backend. It is important to ensure that this script is executable, and the script and directory it is in are both owned by the "gitms" user - not "apache".

This script is run by suexec, and will therefore run as the user that owns it, which we require to be "gitms":

chmod +x /var/www/bin/git-http-backend
chown -R gitms:gitms /var/www/bin

Configuration of SELinux

If SELinux is being run, additional configuration is required, as follows:

#enable access to home directories - ie /home/gitms
setsebool -P httpd_enable_homedirs on
 
usermod -a -G apache gitms
 
#install semanage
yum -y install policycoreutils-python
 
#allow httpd read/write access to /home/gitms
chcon -R -t httpd_sys_rw_content_t /home/gitms
 
#allow httpd read/write access to /home/gitms
chcon -R -t httpd_sys_rw_content_t /opt/wandisco/git-multisite/replicator/content_delivery
 
#allow the update script make network connection to the Java service
setsebool -P httpd_can_network_connect on
 
setsebool -P git_system_enable_homedirs on
 
#configure selinux for suexec on git-http-backend
semanage fcontext -a -t httpd_sys_script_exec_t /var/www/bin
restorecon /var/www/bin

Final step

After you've finished updating your configuration, the last step is to restart apache:

service httpd restart

SSH Authentication

SSH authentication is available for Git and supported by Git MultiSite. It is simple to set up and is attached to a service which is often already enabled. Occasionally, firewall rules may block clients and is not as popular with Windows users.

Requirements

SSH authentication is done using the SSH daemon and public/private keypairs. Requirements are as follows:

• Git operations are run through a single shared user account
• The user account is the same as the account that runs the GitMS service - for example, "gitms"
• All authentication is done by public/private keypairs. No password or certificate based authentication
• The ssh daemon uses the authorized_keys file in "~/.ssh/" to do the public key lookup
• The shared user account must have a regular shell login in /etc/passwd - using for example, git-shell, will cause the command GitMS relies on to fail.

Authorized keys

Git MultiSite requires use of the 'command' keyword to be attached to each key in the authorized_keys file. This associates a username with the key used to login by using it as an argument to the gitms_shell script. By default, the script is found in /opt/wandisco/git-multisite/bin/gitms_shell, but this can vary between installations.

An example entry is as follows:

command="/opt/wandisco/git-multisite/bin/gitms_shell user1" ssh-rsa <SSH_KEY>

If your script is in a different location, just update this line accordingly.

5.7.2 Authorization

Git MultiSite configuration

Authorization isn't enabled by default. To change this, and any other authorization settings, edit the application.properties file. The location is /opt/wandisco/git-multisite/replicator/properties/ for default installations.

# enable/disable authz module
# Default:
#gitms.authz.enabled=false
gitms.authz.enabled=true
 
# Set the file location of the authz file
# Default: /opt/wandisco/git-multisite/replicator/properties/auth.authz
#gitms.authz.file=/opt/wandisco/git-multisite/replicator/properties/auth.authz
gitms.authz.file=<filepath>
 
# Set the default permissions policy to DENY or ACCEPT
# for requests without a specific rule
# Default:
#gitms.authz.policy=DENY;
gitms.authz.policy=<policy>
 
# Set the polling period to detect changes in the AuthZ file
# NOTE: The trailing L is important, as it indicates this
# number is a Long value
# Default:
#gitms.authz.poll.timer=50L
gitms.authz.poll.timer=<numberInMilliseconds>L

AuthZ File Format

The AuthZ format is similar to the formatting for WANdisco's SVN MultiSite products.

Begin with the following header:

# Git AuthZ Version:1.0

To define teams, add entries such as:

[groups]
team1 = user1, user2
team2 = user3, user4

To define rules against individual repositories, the repository path is used as a unique identifier, which is added in square brackets. For instance:

[/home/gitms/repos/Repo1.git]
user1 = R+W+
@team1 = R+W+C+D+

AuthZ rules are applied to users and teams using the following tokens:

• R(+/-) - Allow/Deny user read access
• W(+/-) - Allow/Deny user write access
• C(+/-) - Allow/Deny user create access
• D(+/-) - Allow/Deny user delete access

Note: In order to create a branch, a user must have both Create and Write access, i.e. C+W+

Rules can also be applied at branch level, as follows:

[/opt/gitRepos/Repo1.git:BRANCH/secure]
rick = W-

A complete example:

# Git AuthZ Version:1.0
[groups]
team1 = wayne, rick
team2 = wayne, allan
[/home/gitms/repos/Repo1.git]
rick = R+W+
@team2 = R+W+C+D+
[/home/gitms/repos/Repo1.git:BRANCH/secure]
rick = W-

AuthZ Rule Application

The AuthZ rules have 2 hierarchies which are considered in the evaluation of whether a user has a requested level of access. They are the Resource level hierarchy (Repo->Branch|Tag), and the user level hierarchy (User->Team). Rule conflicts within these hierarchies are resolved by picking the rule which is most granular. In our above example, "rick" would have Read and Write access to the whole repo, except for the "secure" branch, on which he wouldn't have write access. The process by which these rules are applied is as follows:

If a user called Tom is requesting Write access to branch master on repo Repo1.git, the AuthZ rule resolution goes like this:

1. Determine that the repo Repo1.git exists on the local node. If not, error out.
2. Lookup Tom's rules for branch master on Repo1.git
3. If rules exist for Tom which grant/deny the access he needs, apply them
4. If rules do not exist for Tom, check each of the teams Tom is a part of.
5. Tom may be a part of multiple teams which have conflicting rule permissions. One could grant, and one could deny access. In the case where the permissions conflict at the same point on the hierarchy, we always pick the most permissive rule.
6. If Tom's teams have no rules for the master branch either, we move up the Resource hierarchy and check the permissions assigned against Repo1.git.
7. If Tom has permissions assigned against Repo1.git, apply them.
8. If no relevant rules exist for Tom, check again for each of the teams he is a part of.
9. If no permissions exist at this point, apply the default policy permissions. (gitms.authz.policy)

Logging

The Git MultiSite replicator makes several log entries relating to AuthZ acitivities.

INFO: AuthZ: File change detected for <filename>

WARNING: AuthZ: Auth file invalid <errorMessage>

INFO: AuthZ: Time taken for parsing: <timeTaken>

DEBUG: AuthZ: Authorization disabled, accepting request

INFO: "AuthZ: Request [user: <username, repoPath: <repoPath>, ref:<refName>, accessRequested:<accessRequested>] received"

WARNING: AuthZ: There was an error with the Authorization setup, request declined

INFO: AuthZ: <ACCEPT/DENY> Permissions applied for [user: <username, repoPath: <repoPath>, ref:<refName>, accessRequested:<accessRequested>]

INFO: AuthZ: No permissions specified for [user: <username, repoPath: <repoPath>, ref:<refName>, accessRequested:<accessRequested>] using default policy: <ACCEPT/DENY>

AuthZ: Request received for non-existent user: <username> applying default policy: <ACCEPT/DENY>

Troubleshooting

Problem: Cannot clone via HTTP

Apache configuration can go wrong in a number of places. If you're trying to clone a repository and get an error before you're asked for user credentials, it's likely there's a problem with your apache configuration. Here are some steps to make sure that your setup is valid:

• Ensure that the server is running on the ports you expect: "netstat -tunldp | grep http"
• Ensure that the document root specified in repo.conf matches the base directory for the repositories on disk
• Ensure that the SuexecUserGroup setting matches the gitms user
• Ensure that apache is being run by the apache user
• Ensure that the git-http-backend script created in the instructions in /var/www/bin is executable, owned by the gitms user, and the directory is owned by the gitms user
• Ensure that the htpasswd file, if you're using one, is owned by the apache user and is readable.

More information about the apache failures can be found in the apache logs directory.

Problem: SSH setup isn't working - being asked for a password

The SSH daemon is quite strict about the permissions for its files and will check the following before allowing automatic login via a public/private key pair:

• The authorized_keys file must have 600 permissions
• The ~/.ssh directory must have 700 permissions
• The home directory of the user ("gitms" in our examples) must not allow write access to others.

Problem: Can't write to new empty repo

The first push to a new empty repo actually creates the master branch, so you will need C+ permissions, not just W+.

Problem: A user that should be denied access was allowed

The permissions hierarchy means that some permissions, specified at a more granular level, take precedence over others. Permissions specified at the branch level take precedence over those at the repo level, and user permissions take precedence over team permissions.

However, it is possible to have a conflict of permissions on the same level in the hierarchy - for example, a user can be part of two teams, one of whom has R-, and the other has R+.

In this case, Git MultiSite picks the most permissive permission available. So if you believe a user should be denied access to a resource that they are able to access, check their teams to ensure they're not in another team which grants them those permissions.

Copyright © 2010-2013 WANdisco plc.
All Rights Reserved

This product is protected by copyright and distributed under licenses restricting copying, distribution and decompilation.

Git MultiSite
Last doc build: 16:45 - Thursday 3rd July 2014