Release Notes

WD Fusion 2.6 Build 595 - 18 September 2015

WD Fusion 2.6 brings a major enhancement that adds support for a mix of URI types. Now, WD Fusion will replicate over Hadoop's native "hdfs:///" URI, which ensures a wider support for Hadoop client applications, while still offering the WANdisco's native fusion:/// URI, where its use is beneficial.

Available Packages

WD Fusion currently supports the following versions of Hadoop:

• CDH 5.2.0
• CDH 5.3.0
• CDH 5.4.0
• HDP 2.1.0
• HDP 2.2.0
• HDP 2.3.0
• MAPR 4.0.1 ^*
• MAPR 4.0.2 ^*
• MAPR 4.1.0 ^*
• Pivotal HD 3.0.0 ^*

System Requirements

Before you install this software you must ensure that you meet the necessary system, software and hardware requirements for running WD Fusion. See a full checklist in our online user guide: docs.wandisco.com/bigdata/wdfusion/

Supported Hadoop Packages:

Certified Platforms / DBMS & DConE Support:

• HDP 2.1.2 - 2.3.X
• PHD 3.0
• CDH 5.15 5.4
• EMC Isilon 7.2
• MapR M5,7
• DConE 1.3
• MySQL (Hive MetaStore)

Client Applications Supported:

• Hive
• SparkSQL
• Impala* Some current limitations - contact WANdisco's solutions team
• HBase
• SQOOP
• Flume
• Kafka
• Storm

Installation

You can find detailed instructions on how to get up and running in our online user guide:
- docs.wandisco.com/bigdata/wdfusion/install.html#procedure

Upgrades from an earlier version:

It's essential that you remove previously installed versions of WD Fusion before you complete a new installation:
- docs.wandisco.com/bigdata/wdfusion/install.html#cleanup

Previous Release Notes:

You can view the release notes for previous releases in the release archive:
- docs.wandisco.com/bigdata/wdfusion/archive.html

New Features

• Updated WANdisco's DConE agreement engine to latest version (1.3.6). FUS-1014, FUS-779


• Added support for Open JDK 7. FUS-835


• It's now possible to set WD Fusion to use a mix of URIs and underlying File Systems. By default, the URI is hdfs:///, although this can be changed to the WD Fusion's native fusion:/// URI. The file system is set to HDFS by default, although it can be changed to operate on HCFS. You can read more in the Installation Guide - URI Selection
  FUI-1159, FUI-964


• It's now possible to ensure that file properties, particularly block size are preserved when replicated. FUS-723


• An authentication mechanism is now provided to secure the DConE REST API endpoint. FUS-145


• Added support for writing to WD Fusion's fusion:/// URI, through Impala. FUS-775


• WD Fusion's web UI now supports Kerberos authentication, so that a user will need to configure their browser to work with Kerberos HTTP SPNEGO. FUI-959, FUI-1162


• Added support for HDP 2.3.0 FUI-704, FUI-707


• Added support for Ubuntu (exclusively for versions 12.04TLS and 14.04LTS). FUI-919


• Improved formatting for the web UI's Settings screen. FUI-562


• The web UI now extends support to Pivotal HD 3.0.FUI-812


• The WD Fusion installer now supports for Ubuntu 12.04 and 14.04. FUI-919


• Added support for configurable consistency check frequency. The feature should be available in the web UI in the next release. FUS-585

Fixes

• System now correctly set to retry if MapReduce job is not assigned to a writer, instead of immediately failing the job. FUS-586


• FsStoreMap no longer persists writerMap. FUS-991


• Fixed a formatting issue that caused consistency checks run on newly created empty replicated folder to be stuck in "Pending Replication" state. These directories become stuck on the UI. FUS-846, FUI-406


• We now flush the contents of the buffer directly to the underlying stream, bypassing any checks for asynchronous hflush. FUS-794


• WD Fusion no longer introduces a dedicated system user "fusion" during installation. Instead, WD Fusion is run as the main Hadoop system user, usually "hdfs". This avoids potential errors where the standard "hdfs" user would need to impersonate a distinct WD Fusion system user. FUS-1012


• Fixed an issue that stopped WD Fusion server from resolving hostnames containing wildcard _HOST in Kerberos principal names. FUS-1083


• Fixed an apparent inconsistency between the installed client and what's made available on the Client download panel, during installation. FUS-788

Known Issues

• [Fixed in 2.6.9] We need to extend OutputStream instead of FSDataOutputStream to capture additional write methods for HFlush buffering. FUS-1789


• When running Hive on a Cloudera deployment, the Hivemetastore canary test reports as having "Bad health". FUS-1140


• There are known problems running Teragen and Terasort with FusionHdfs or FusionHcfs configurations. Some required directories are currently missing and will cause Terasort to hang. You can work around the problem by creating the following directories, then making sure that Yarn and MapR users are added and that they have access to the directories.

  sudo -u hdfs hadoop fs -mkdir /user/yarn
  sudo -u hdfs hadoop fs -chown yarn /user/yarn
  sudo -u hdfs hadoop fs -mkdir /user/mapred
  sudo -u hdfs hadoop fs -chown mapred /user/mapred

  FUS-1142, FUS-1026


• In deployments that mix kerberized and non-kerberized zones it's possible that permission errors will occur because the different zones don't share the same underlying system superusers. In this scenario you would need to ensure that the the superuser for each zone is created on the other zones. For example, if you connect a Zone that runs CDH, which has superuser 'hdfs" with a zone running MapR, which has superuser 'mapr', you would need to create the user 'hdfs' on the MapR zone and 'mapr' on the CDH zone. FUS-1149, FUS-1150