IBM Big Replicate - Fusion Installation

2.12 Installing IBM Big Replicate

This section runs through the installation of the companion installer that is specific to an IBM Big Replicate deployment.

Installing into a deployment that runs Hive - See WANdisco Hive Metastore plugin

Download the archive ibm-biginsights-installer-bundle.tar.gz from WANdisco's FD website, along with the appropriate WD Fusion installer file.
Extract the ibm-biginsights-installer-bundle.tar.gz into your node server. Copy the WD Fusion installer to the same location.
Navigate to the extracted files.
Copy the .swidtag file to <PRODUCT-LOCATION>/iso-swid/ i.e.:
```
[vagrant@dc00-vm1 ~]$ ls -la /opt/wandisco/fusion-ui-server/iso-swid/
total 4
-rw-r--r-- 1 1001 1001 682 Jun 16 10:33 ibm.com_IBM_Big_Replicate-2.0.0.swidtag
```
There can be only one (installer file):
Important: Ensure that only the one, relevant, fusion install .sh file in the IBM location into which you extracted the tarball. Currently, if there is more than one installer in the location, the IBM install script will use the first one that it finds - probably the oldest one. We'll address this issue in a future release.
Run the IBM Big Replicate installer:
```
./ibmBigInsightsInstallScript.sh
```
You can optionally pass an argument to the script which will set the location where that the IBM SLM tags will be written. E.g.
```
./ibmBigInsightsInstallScript.sh /opt/some/other/location
```
- If no argument is provided, the location defaults to /var/ibm/slmtags.
- The location will be created if it does not already exist.
IBM SLM Tagging
Read our guide on how to interpret IBM SLM tagging - SLM Tagging for IBM Big Replicate

Run through the installer:

[root@dc00-vm1 IBMTEST]# ll
total 703112
-rw-r--r-- 1 vagrant vagrant 718215802 Jun 16 10:28 fusion-cdh-nothive-33.sh
-rw-r--r-- 1 vagrant vagrant   1763050 Jun 16 10:26 ibm-biginsights-installer-bundle.tar.gz
-rwxr-xr-x 1 vagrant vagrant      1346 Jun 16 10:23 ibmBigInsightsInstallScript.sh
drwxr-xr-x 1 vagrant vagrant       154 Jun 16 10:18 lib
[root@dc00-vm1 IBMTEST]# ./ibmBigInsightsInstallScript.sh
 
LICENSE INFORMATION
 
The Programs listed below are licensed under the following
License Information terms and conditions in addition to the
Program license terms previously agreed to by Client and
IBM. If Client does not have previously agreed to license
terms in effect for the Program, the International Program
License Agreement (Z125-3301-14) applies.
 
Program Name: IBM Big Replicate 2.0
Program Number: 5737-A55
 
As described in the International Program License Agreement
("IPLA") and this License Information, IBM grants Licensee
a limited right to use the Program. This right is limited
 
Press Enter to continue viewing the license agreement, or
enter "1" to accept the agreement, "2" to decline it, "3"
to print it, or "99" to go back to the previous screen.

Enter "1" to continue.

The installer will first perform a health check and confirm that there is sufficient Java heap to support an installation.

Installing WD Fusion
Verifying archive integrity... All good.
Uncompressing WANdisco Fusion........................
 
    ::   ::  ::     #     #   ##    ####  ######   #   #####   #####   #####
   :::: :::: :::    #     #  #  #  ##  ## #     #  #  #     # #     # #     #
  ::::::::::: :::   #  #  # #    # #    # #     #  #  #       #       #     #
 ::::::::::::: :::  # # # # #    # #    # #     #  #   #####  #       #     #
  ::::::::::: :::   # # # # #    # #    # #     #  #        # #       #     #
   :::: :::: :::    ##   ##  #  ## #    # #     #  #  #     # #     # #     #
    ::   ::  ::     #     #   ## # #    # ######   #   #####   #####   #####
 
 
 
 
Welcome to the WANdisco Fusion installation
 
 
 
You are about to install WANdisco Fusion version 2.8-33
 
Do you want to continue with the installation? (Y/n)

Enter "Y" to continue.

The installer checks that both Perl and Java are installed on the system.

  Checking prerequisites:
 
Checking for perl: OK
Checking for java: OK

INFO: Using the following Memory settings for the WANdisco Fusion Admin UI process:
 
INFO: -Xms128m -Xmx512m
 
Do you want to use these settings for the installation? (Y/n)

Enter "Y" or "N" if you wish to set different Java heap settings.

The installer asks you to confirm which TCP port will be used for accessing the Fusion/Big Replicate web UI, the default is "8083".

Which port should the UI Server listen on?  [8083]:
Please specify the appropriate platform from the list below:

[0] ibm-4.0
[1] ibm-4.1
[2] ibm-4.2

Which fusion platform do you wish to use? 1
You chose ibm-4.1:2.7.1

Select from the available Hadoop packages.

Next, you set the system user, group for running the applicaton.

We strongly advise against running Fusion as the root user.
 
For default CDH setups, the user should be set to 'hdfs'. However, you should choose a user appropriate for running HDFS commands on your system.
 
Which user should Fusion run as? [hdfs]
Checking 'hdfs' ...
 ... 'hdfs' found.
 
Please choose an appropriate group for your system. By default CDH uses the 'hdfs' group.
 
Which group should Fusion run as? [hdfs]
Checking 'hdfs' ...
 ... 'hdfs' found.

You should press enter to go with the default "hdfs".

You will now be shown a summary of the settings that you have provided so far:

Installing with the following settings:
 
User and Group:                     hdfs:hadoop
Hostname:                           ip-10-11-0-40
Fusion Admin UI Listening on:       0.0.0.0:8083
Fusion Admin UI Minimum Memory:     128
Fusion Admin UI Maximum memory:     512
Platform:                           ibm-4.1 (2.7.1)
Fusion Server Hostname and Port:    ip-10-12-0-40:8082
Do you want to continue with the installation? (Y/n)

Enter "Y" unless you need to make changes to any of the settings.

The installation will now complete:

Adding the user hdfs to the hive group if the hive group is present.
Installing ibm-4.1 packages:
  fusion-hcfs-ibm-4.1-server-2.9_RC7.el6-1925.noarch.rpm ... Done
  fusion-hcfs-ibm-4.1-ihc-server-2.9_RC7.el6-1925.noarch.rpm ... Done
Installing plugin packages:
Installing fusion-ui-server package:
  fusion-ui-server-2.9-74.noarch.rpm ... Done

Once the installation has completed, you need to configure the WD Fusion server using the browser based UI.

Starting fusion-ui-server:                                 [  OK  ]
Checking if the GUI is listening on port 8083: .....Done
 
Please visit http://your.hostname.server.com:8083/ to complete installation of WANdisco Fusion
 
If 'your.hostname.server.com' is internal or not available from your browser, replace
this with an externally available address to access it.
 
Stopping fusion-ui-server:.                                [  OK  ]
Starting fusion-ui-server:                                 [  OK  ]

Open a browser and enter the provided URL, or IP address.

Follow the guide for running the configuration steps here - Configure WD Fusion through a browser

Known Issue:
If you are deploying IBM Big Replicate / WD Fusion into a BigInsights environment you must make the following changes to your Hive configuration. The changes should apply even if you are using regular Hive, rather than WANdisco's Hive Metastore replication plugin. These changes enable caching. Without the changes in place, Hive operation may be impacted by leaking file descriptors, resulting in a "Too many open files" error.

Set the following 2 hive configurations to true:

fs.hdfs.impl.disable.cache true
fs.file.impl.disable.cache true

fs.hdfs.impl.disable.cache: Disable HDFS filesystem cache. Default: "false".
fs.file.impl.disable.cache: Disable local filesystem cache. Default: "false".

These are both properties in the hive-site configuration file and can be changed and saved through the Ambari UI.