Connecting to an Hadoop external database in FDA requires certain additional configurations on the Adobe Campaign server. Learn how to install and configure the connector for HDInsight with this page.

Installing ODBC and HDInsight drivers

For Windows

  1. Install ODBC and Azure HD Insight drivers for Windows. Drivers can be downloaded here.

  2. Create the DSN (Data Source Name) by running the ODBC DataSource Adminstrator tool. A System DSN sample for Hive is provided for you to modify.

    Description: vorac (or any other name)
    Host: vorac.azurehdinsight.net
    Port: 443
    Database: sm_tst611 (or your database name)
    Mechanism: Azure HDInsight Service
    User/Password: admin/<your password here>
  3. Create the Hadoop external account, as detailed in this section.

For Linux

  1. Install unixodbc for Linux.

    apt-get install unixodbc
  2. Download and install ODBC drivers for Apache Hive from HortonWorks. Drivers can be downloaded here.

    dpkg -i hive-odbc-native_2.1.10.1014-2_amd64.deb
  3. Check ODBC files location.

    root@campadpac71:/tmp# odbcinst -j
    unixODBC 2.3.1
    DRIVERS............: /etc/odbcinst.ini
    SYSTEM DATA SOURCES: /etc/odbc.ini
    FILE DATA SOURCES..: /etc/ODBCDataSources
    USER DATA SOURCES..: /root/.odbc.ini
    SQLULEN Size.......: 8
    SQLLEN Size........: 8
    SQLSETPOSIROW Size.: 8
  4. Create the DSN (Data Source Name) and edit the odbc.ini file. Then, create a DSN for your Hive connection. 

    The UseNativeQuery parameter here is mandatory for both Windows and Linux. Campaign is Hive-aware and will not work correctly unless UseNativeQuery is set. Typically, the driver or Hive SQL Connector will rewrite queries and tamper the column ordering.

    Here is an example for HDInsight to setup a connection called "vorac":

    [ODBC Data Sources]
    vorac 
    
    [vorac]
    Driver=/usr/lib/hive/lib/native/Linux-amd64-64/libhortonworkshiveodbc64.so
    HOST=vorac.azurehdinsight.net
    PORT=443
    Schema=sm_tst611
    HiveServerType=2
    AuthMech=6
    UID=admin
    PWD=<your password here>
    HTTPPath=
    UseNativeQuery=1

    Note:

    The authentication setup depends on the Hive/Hadoop configuration. For instance, for HDInsight, use AuthMech=6 for user/password authentication, as described here.

  5. Export the variables.

    export ODBCINI=/etc/myodbc.ini
    export ODBCSYSINI=/etc/myodbcinst.ini
  6. Setup Hortonworks drivers via /usr/lib/hive/lib/native/Linux-amd64-64/hortonworks.hiveodbc.ini.

    You have to use UTF-16 to be able to connect with Campaign and unix-odbc (libodbcinst).

    [Driver]
    
    DriverManagerEncoding=UTF-16
    ErrorMessagesPath=/usr/lib/hive/lib/native/hiveodbc/ErrorMessages/
    LogLevel=0
    LogPath=/tmp/hive
    SwapFilePath=/tmp
    
    ODBCInstLib=libodbcinst.so
  7. You can now test your connection using isql.

    isql vorac
    isql vorac -v
  8. Create the Hadoop external account, as detailed in this section.

Create an external account

The Hive connector is setup using the OOB ODBC connection with the following parameters:

  • Account type: external database
  • Connection type: odbc
  • Server: name of the DSN. In the example above, that would be "vorac"
  • Account and password: user and password for authentication
  • Database: name of the database (if not specified in DSN). It can be left empty if specified in the DSN
  • Time zone: server time zone

Connection options

The following options can be set at the connection level for FDA connectors:

Option name
Option values
Comment
Hive options
bucketsNumber 20
Number of buckets when creating a clustered table
fileFormat
PARQUET
Default file format for work tables

Setting connection attributes

Connection-level attributes can be set here:

<postConnectSQL>
<![CDATA[
            create table if not exists a1 (
              i int
            );
            insert into a1(i) values ('42');
]]>
</postConnectSQL>