2.12 Installing IBM Big Replicate

This section runs through the installation of the companion installer that is specific to an IBM Big Replicate deployment.

Installing into a deployment that runs Hive - See WANdisco Hive Metastore plugin
  1. Download the archive ibm-biginsights-installer-bundle.tar.gz from WANdisco's FD website, along with the appropriate WD Fusion installer file.
  2. Extract the ibm-biginsights-installer-bundle.tar.gz into your node server. Copy the WD Fusion installer to the same location.
  3. Navigate to the extracted files.
  4. Copy the .swidtag file to <PRODUCT-LOCATION>/iso-swid/ i.e.:
    [vagrant@dc00-vm1 ~]$ ls -la /opt/wandisco/fusion-ui-server/iso-swid/
    total 4
    -rw-r--r-- 1 1001 1001 682 Jun 16 10:33 ibm.com_IBM_Big_Replicate-2.0.0.swidtag
    There can be only one (installer file):
    Important: Ensure that only the one, relevant, fusion install .sh file in the IBM location into which you extracted the tarball. Currently, if there is more than one installer in the location, the IBM install script will use the first one that it finds - probably the oldest one. We'll address this issue in a future release.
  5. Run the IBM Big Replicate installer:
    ./ibmBigInsightsInstallScript.sh
    You can optionally pass an argument to the script which will set the location where that the IBM SLM tags will be written. E.g.
    ./ibmBigInsightsInstallScript.sh /opt/some/other/location
    • If no argument is provided, the location defaults to /var/ibm/slmtags.
    • The location will be created if it does not already exist.
    IBM SLM Tagging
    Read our guide on how to interpret IBM SLM tagging - SLM Tagging for IBM Big Replicate
  6. Run through the installer:
    [root@dc00-vm1 IBMTEST]# ll
    total 703112
    -rw-r--r-- 1 vagrant vagrant 718215802 Jun 16 10:28 fusion-cdh-nothive-33.sh
    -rw-r--r-- 1 vagrant vagrant   1763050 Jun 16 10:26 ibm-biginsights-installer-bundle.tar.gz
    -rwxr-xr-x 1 vagrant vagrant      1346 Jun 16 10:23 ibmBigInsightsInstallScript.sh
    drwxr-xr-x 1 vagrant vagrant       154 Jun 16 10:18 lib
    [root@dc00-vm1 IBMTEST]# ./ibmBigInsightsInstallScript.sh
     
    LICENSE INFORMATION
     
    The Programs listed below are licensed under the following
    License Information terms and conditions in addition to the
    Program license terms previously agreed to by Client and
    IBM. If Client does not have previously agreed to license
    terms in effect for the Program, the International Program
    License Agreement (Z125-3301-14) applies.
     
    Program Name: IBM Big Replicate 2.0
    Program Number: 5737-A55
     
    As described in the International Program License Agreement
    ("IPLA") and this License Information, IBM grants Licensee
    a limited right to use the Program. This right is limited
     
    Press Enter to continue viewing the license agreement, or
    enter "1" to accept the agreement, "2" to decline it, "3"
    to print it, or "99" to go back to the previous screen.
    Enter "1" to continue.
  7. The installer will first perform a health check and confirm that there is sufficient Java heap to support an installation.
    Installing WD Fusion
    Verifying archive integrity... All good.
    Uncompressing WANdisco Fusion........................
     
        ::   ::  ::     #     #   ##    ####  ######   #   #####   #####   #####
       :::: :::: :::    #     #  #  #  ##  ## #     #  #  #     # #     # #     #
      ::::::::::: :::   #  #  # #    # #    # #     #  #  #       #       #     #
     ::::::::::::: :::  # # # # #    # #    # #     #  #   #####  #       #     #
      ::::::::::: :::   # # # # #    # #    # #     #  #        # #       #     #
       :::: :::: :::    ##   ##  #  ## #    # #     #  #  #     # #     # #     #
        ::   ::  ::     #     #   ## # #    # ######   #   #####   #####   #####
     
     
     
     
    Welcome to the WANdisco Fusion installation
     
     
     
    You are about to install WANdisco Fusion version 2.8-33
     
    Do you want to continue with the installation? (Y/n)
    Enter "Y" to continue.
  8. The installer checks that both Perl and Java are installed on the system.
      Checking prerequisites:
     
    Checking for perl: OK
    Checking for java: OK
    
    INFO: Using the following Memory settings for the WANdisco Fusion Admin UI process:
     
    INFO: -Xms128m -Xmx512m
     
    Do you want to use these settings for the installation? (Y/n)
    
    
    Enter "Y" or "N" if you wish to set different Java heap settings.
  9. The installer asks you to confirm which TCP port will be used for accessing the Fusion/Big Replicate web UI, the default is "8083".
    Which port should the UI Server listen on?  [8083]:
    Please specify the appropriate platform from the list below:
    
    [0] ibm-4.0
    [1] ibm-4.1
    [2] ibm-4.2
    
    Which fusion platform do you wish to use? 1
    You chose ibm-4.1:2.7.1
    
    Select from the available Hadoop packages.
  10. Next, you set the system user, group for running the applicaton.
    We strongly advise against running Fusion as the root user.
     
    For default CDH setups, the user should be set to 'hdfs'. However, you should choose a user appropriate for running HDFS commands on your system.
     
    Which user should Fusion run as? [hdfs]
    Checking 'hdfs' ...
     ... 'hdfs' found.
     
    Please choose an appropriate group for your system. By default CDH uses the 'hdfs' group.
     
    Which group should Fusion run as? [hdfs]
    Checking 'hdfs' ...
     ... 'hdfs' found.
    You should press enter to go with the default "hdfs".
  11. You will now be shown a summary of the settings that you have provided so far:
    Installing with the following settings:
     
    User and Group:                     hdfs:hadoop
    Hostname:                           ip-10-11-0-40
    Fusion Admin UI Listening on:       0.0.0.0:8083
    Fusion Admin UI Minimum Memory:     128
    Fusion Admin UI Maximum memory:     512
    Platform:                           ibm-4.1 (2.7.1)
    Fusion Server Hostname and Port:    ip-10-12-0-40:8082
    Do you want to continue with the installation? (Y/n)
    Enter "Y" unless you need to make changes to any of the settings.
  12. The installation will now complete:
    Adding the user hdfs to the hive group if the hive group is present.
    Installing ibm-4.1 packages:
      fusion-hcfs-ibm-4.1-server-2.9_RC7.el6-1925.noarch.rpm ... Done
      fusion-hcfs-ibm-4.1-ihc-server-2.9_RC7.el6-1925.noarch.rpm ... Done
    Installing plugin packages:
    Installing fusion-ui-server package:
      fusion-ui-server-2.9-74.noarch.rpm ... Done
    
  13. Once the installation has completed, you need to configure the WD Fusion server using the browser based UI.
    Starting fusion-ui-server:                                 [  OK  ]
    Checking if the GUI is listening on port 8083: .....Done
     
    Please visit http://your.hostname.server.com:8083/ to complete installation of WANdisco Fusion
     
    If 'your.hostname.server.com' is internal or not available from your browser, replace
    this with an externally available address to access it.
     
    Stopping fusion-ui-server:.                                [  OK  ]
    Starting fusion-ui-server:                                 [  OK  ]
    Open a browser and enter the provided URL, or IP address.
  14. Follow the guide for running the configuration steps here - Configure WD Fusion through a browser
Known Issue:
If you are deploying IBM Big Replicate / WD Fusion into a BigInsights environment you must make the following changes to your Hive configuration. The changes should apply even if you are using regular Hive, rather than WANdisco's Hive Metastore replication plugin. These changes enable caching. Without the changes in place, Hive operation may be impacted by leaking file descriptors, resulting in a "Too many open files" error.

Set the following 2 hive configurations to true:
fs.hdfs.impl.disable.cache true
fs.file.impl.disable.cache true
fs.hdfs.impl.disable.cache
Disable HDFS filesystem cache. Default: "false".
fs.file.impl.disable.cache
Disable local filesystem cache. Default: "false".
These are both properties in the hive-site configuration file and can be changed and saved through the Ambari UI.


Copyright © 2005-2017 WANdisco, Inc.
All rights reserved
This product is protected by copyright and distributed under licenses restricting copying, distribution and decompilation.

WANdisco WD Fusion UI version 2.9.4 Build 324
Last doc issue: 21 March 2017