1. SnowMirror Requirements

SnowMirror is a Java based application, designed to mirror data from tables in Salesforce into your local database. This mirroring process is done using Salesforce’s SOAP API and Bulk API.

snowmirror diagram arch

What you need:
- Salesforce organization
- Salesforce User with sufficient Rights
- Server with one of supported operating systems - Supported platforms
- Database into which all mirrored data will be stored

1.1. Salesforce

For ideal communication between Salesforce and SnowMirror we recommend you to set up authentication credentials with sufficient rights.

1.1.1. User Access Rights

SnowMirror application communicates with Salesforce via Web Services. We are using API library provided by Salesforce itself. It contains Partner API for synchronous calls and Bulk API for asynchronous calls. This library needs Authentication credentials which can be set in Settings (see Salesforce settings).

1.2. Application Requirements

SnowMirror is a Java web application, meaning it runs centrally on a server, and users interact with it throughout web browser from any computer.

1.2.1. Hardware

SnowMirror runs on any reasonably fast PC. See the minimal recommended hardware configuration.

  • Dual core CPU (Intel or AMD)

  • 2GB RAM

  • 1GB HDD for installation

  • Additional 5GB for log files

These requirements expect that only SnowMirror runs on that machine. You will need to run separate database server for mirror database. Of course you may use your existing infrastructure for that database.

Estimating required storage space

SnowMirror keeps a log of all activities - the estimated storage required for these logs is around 80MB per year.

SnowMirror also creates separate activity log for each synchronization run. The storage required for these logs vary depending on how many synchronizations you have, how often you synchronize them and how many records are being synchronized. Naturally, the longer you use SnowMirror more space for logs is required. See the examples below.

Table 1. Examples of required storage for activity logs
Amount of data Estimated storage required per year

Big (~150,000 updated records) synchronization every day

~ 45 MB per year

Big (~150,000 updated records) synchronization every hour

~ 1 GB per year

Smaller (~10,000 updated records) synchronization every day

~ 3 MB per year

Smaller (~10,000 updated records) synchronization every day

~ 72 MB per year

Then the total storage required per year can be calculated as:

Storage per year = 80MB + (number of big daily synchronizations * 45)MB + (number of small daily synchronizations * 3)MB + (number of big hourly synchronizations * 1000)MB + (number of small hourly synchronizations * 72)MB

1.2.2. Java

Java is a piece of software that has to be installed on your computer. SnowMirror does not work without it. Currently, we support AdoptOpenJDK 11 LTS (HotSpot).

On Windows, an up-to-date version of Java is included in the installation package. On Linux or Mac, you have to install it yourself.

Java Is Provided Free Of Charge

Since Java 11, Oracle (the company that owns and develops Java) has changed licensing terms and support model. This topic is quite complex but the important message is that you don’t have to pay for Java.

There are several distributions of Java available:

  • Please, use Open JDK builds from AdoptOpenJDK. More specifically, use Open JDK 11 LTS - HotSpot.

  • Please, avoid releases of Oracle JDK. These are paid and come with Oracle’s commercial support.

1.2.3. Application server

SnowMirror runs on Apache Tomcat web server which is included in the installation package. We upgrade the Tomcat to the latest version on a regular basis.

1.2.4. Databases

SnowMirror uses two databases.

  • Configuration database - stores metadata, i.e. what to synchronize, when to synchronize, etc.

  • Mirror database - stores data mirrored from Salesforce

Configuration DB and mirror DB may use different database servers. However, the best practice is to have one database server containing two databases. For more information see Database Configuration section.

Table 2. Supported databases
Database Version Comments

Oracle

11g and newer

MySQL

5.5 and newer

Community version MariaDB is also supported

Microsoft SQL Server

2008 and newer

PostgreSQL

9.2 and newer

Snowflake

All versions

Snowflake can be used as a mirror database. It cannot be used as a configuration database.

H2

1.3.172

Embedded file-based database. Suitable only for evaluation.

1.2.5. Browser

SnowMirror runs on all modern web browsers such as Google Chrome, Mozilla Firefox and Internet Explorer 8 or higher.

1.2.6. Supported platforms

SnowMirror is operating system agnostic. It can run on any system which supports Java.

Supported platforms
  • Microsoft Windows XP 32/64 bit

  • Microsoft Windows 7 32/64 bit

  • Microsoft Windows 10 32/64 bit

  • Microsoft Windows Server 2003 32 bit

  • Microsoft Windows Server 2008 R2 64 bit

  • Microsoft Windows Server 2012 R2 64 bit

  • Microsoft Windows Server 2016

  • Linux

2. Installing SnowMirror

SnowMirror can be installed either using Windows Installer or manually from a ZIP package. Using the Windows Installer is the most easiest and straightforward way to install and configure SnowMirror. On Linux you have to do the installation and configuration manually.

2.1. Prerequisites

You need two databases on your database server - configuration database and mirror database. See Database Configuration chapter.

If you don’t have the databases yet, you can use H2 database which is embedded into SnowMirror. This will give you an opportunity to test SnowMirror even before your database administrators create the databases for you. Just note, that H2 is not suitable for production environment.

2.2. Windows Installer

2.2.1. Express installation

  1. Run the Installer - Go to a location to where you downloaded SnowMirror installation file and run it.

  2. First Step - Simply click next.

    winInstaller firstStep
  3. License Agreement - Please go through a license agreement and then confirm by clicking on "I agree".

    winInstaller licenseAgreement
  4. Selecting the action - Select "Express Install" option and click next.

    winInstaller choosingtheaction express
  5. Summary - Here you can see summary. It shows installation location, used ports windows service and such.

    winInstaller express summary
  6. Installing - Window with a progress bar.

    winInstaller installing
  7. Finish - Final screen. After clicking on "Finish", your default browser will be opened and application itself will load.

    winInstaller finish

2.2.2. Custom installation

  1. Run the Installer - Go to a location to where you downloaded SnowMirror installation file and run it.

  2. First Step - Simply click next.

    winInstaller firstStep
  3. License Agreement - Please go through a license agreement and then confirm by clicking on "I agree".

    winInstaller licenseAgreement
  4. Selecting the action - Select "Custom Install" option and click next.

    winInstaller choosingtheaction custom
  5. Selecting the Location - If you would like to install SnowMirror into a location different to default one, feel free to click "Browse" and select your desired location. Once you are done selecting a location, just click "Next".

    winInstaller selectingLocation
  6. Configuring Ports and Host - This screen is here for you to change host name and ports.

    winInstaller configPortsHosts
  7. Selecting the Database - Here you choose can which database you are going to use

    This step is about selecting configuration database. If you wish to read more about mirror db, please see mirror db section

    winInstaller selectdb1

    Now you can input information necessary to correctly set mirror database.

    • Database Type: Choose a database you are going to use

    • Server: Address of a server where the database is running

    • Port: A port on which the database is running

    • Database: Database name, SID etc.

    • Username: username of a database user through which the SnowMirror is going to access the database

    • Password: password for that user

      winInstaller selectdb2

      NOTE:
      Postgres, SQL Server - If you wish to use different schema than public, it is possible too. Just fill the field Schema with the name of your desired schema.

  8. Setting the Service - If you want to run SnowMirror as a service, leave the checkbox checked. Windows service is good if you want to run SnowMirror automatically after the operating system starts up. You can change the service name. This is useful if you need to install multiple instances of SnowMirror.
    You can also change the system account under which the service will be running, e.g. if you create a special account on server just for running SnowMirror.

    winInstaller settingService
  9. Summary - Here you can see summary. It shows installation location, used ports windows service and such.

    winInstaller custom summary
  10. Installing - Window with a progress bar.

    winInstaller installing
  11. Finish - Final screen. After clicking on "Finish", your default browser will be opened and application itself will load.

    winInstaller finish

2.3. Installation On Linux

  1. Create SnowMirror directory /opt/SnowMirror. You are free to choose a different directory.

    mkdir /opt/SnowMirror
  2. Download Java Java Runtime Environment (JRE) and SnowMirror and put them to the /opt/SnowMirror/tmp folder.
     
    JRE is available on https://www.java.com/en/download/manual.jsp page. For 64-bit system download "Linux x64" package, for 32-bit system download "Linux" package.
     
    SnowMirror ZIP package is available on our website.

    1. Unpack both JRE and SnowMirror:

      cd /opt/SnowMirror/tmp
      tar zxvf jre-<CurrentJreVersion>-linux-[i586|x64].tar.gz
      unzip snow-mirror-<CurrentSnowMirrorVersion>.zip
    2. Copy the content of JRE to /opt/SnowMirror/jre and the content of SnowMirror to /opt/SnowMirror.

      cp -r /opt/SnowMirror/tmp/jre<CurrentJreVersion>/. /opt/SnowMirror/jre
      cp -r /opt/SnowMirror/tmp/snow-mirror-<CurrentSnowMirrorVersion>/. /opt/SnowMirror
    3. Copy JCE Files. Passwords in configuration database are encrypted using AES 256. To make the encryption work you have to copy Java JCE files to your JRE.

      cp -r /opt/SnowMirror/snow-mirror/jce/1.8.0/* /opt/SnowMirror/jre/lib/security
    4. Remove temporary folder

      cd /opt/SnowMirror
      rm -rf /opt/SnowMirror/tmp
  3. Edit snowMirror.properties

    1. snowMirror.host property - by default, it is set to "localhost". That means, you can access SnowMirror only from the machine where you installed it. If you want to be able to acces SnowMirror from outside this machine set the property to its network name. See Application URL Configuration for details on how to change the host name.

    2. Provide a connection to your configuration database:

      Database Server config.db.type config.jdbc.url

      SQL Server

      mssql

      jdbc:jtds:sqlserver://<SERVER>:<PORT>/snowmirror;instance=<INSTANCE-NAME>

      Oracle

      oracle

      jdbc:oracle:thin:@<SERVER>:<PORT>:<SID>
      jdbc:oracle:thin:@<SERVER>:<PORT>/<SERVICE-NAME>

      MySQL

      mysql

      jdbc:mysql://<SERVER>:<PORT>/snowmirror

      PostgreSQL

      postgres

      jdbc:postgresql://<SERVER>:<PORT>/snowmirror

      H2 (Embedded)

      h2

      jdbc:h2:~/h2/snowmirror;MVCC=TRUE

      Also, provide username (config.jdbc.username) and password (config.jdbc.username).

  4. Start SnowMirror.

    /opt/SnowMirror/run.sh
  5. Open SnowMirror in your favorite web browser on the following URL: \http://<value of snowMirror.host property>:<value of snowMirror.port>.
     
    The default value is http://localhost:9191.

2.3.1. Service Installation

Optionally, you can install SnowMirror as a service. The service will automatically start when operating system starts.

  1. Open Terminal

  2. Copy a script responsible for starting and stopping SnowMirror init.d directory.

    sudo cp /opt/SnowMirror/bin/init.d/snowmirror.sh /etc/init.d/snowmirror
  3. Make the init script executable:

    sudo chmod a+x /etc/init.d/snowmirror
  4. Place symlinks in the run-level directories to start and stop this script automatically.

    1. For Debian based systems:

      update-rc.d snowmirror defaults
    2. For RedHat based systems:

      chkconfig --add snowmirror
      The snowmirror.sh script contains chkconfig settings

2.4. What Is Installed

We have described the installation process on both Windows and Linux server. Let’s have a look at files we installed.

File & Directory Description

/snowMirror.properties

Main configuration file. Contains a connection to the configuration database and SnowMirror base URL. If you want to know more about setting a different base url or changing port, please see chapter Deployment URL

/snowMirror-https.jks

Contains a certificate and a private key. Used when you configure SnowMirror to use HTTPS protocol.

/run.bat

Starts SnowMirror on Windows. Used only if a service is not installed.

/run.sh

Starts SnowMirror on Linux.

/shutdown.bat

Stops SnowMirror on Windows. Used only if a service is not installed.

/shutdown.sh

Stops SnowMirror on Linux.

/Uninstall.exe

Uninstalls SnowMirror from Windows.

/Uninstall.info

Stores data needed for uninstallation.

/bin

Contains Apache Tomcat web server.

/conf

Apache Tomcat configuration files.

/jre

Java runtime environment binaries.

/lib

Apache Tomcat classes and libraries.

/license

License files.

/logs

Folder used for both SnowMirror and Tomcat logs.

/logs/activityLogs

Folder used for synchronizations logs. Each synchronization run has its own log file. The files are located by this pattern: <synchronizationId>/<yyyy-MM>/<logId>.log.

/snow-mirror

Contains all SnowMirror specific application files like configurations, WAR file, documentation and data directory for embedded H2 database and a folder for mirrored attachments.

/snow-mirror/data

When embedded H2 database is used, this folder stores database files.

/snow-mirror/docs

Folder contains SnowMirror documentation.

/snow-mirror/jce

Contains Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files

/snow-mirror/conf

SnowMirror configuration files. Do not modify. If you need no make a change, modify the main configuration file /snowMirror.properties.

/snow-mirror/webapp

SnowMirror war archive.

/webapps

Used by Apache Tomcat.

/work

Used by Apache Tomcat.

If you want to make sure no directory is missing or you are not sure how SnowMirror installation should look like, please checkout the following directory structure. We omitted directories nested too deep and all files.

.
├── bin
│   ├── experimental
│   └── init.d
├── conf
│   └── Catalina
├── jre
│   ├── bin
│   ├── lib
│   ├── man
│   └── plugin
├── lib
├── logs
│   └── activityLogs
├── snow-mirror
│   ├── conf
│   ├── data
│   ├── docs
│   ├── license
│   ├── trusted-certificates
│   └── webapp
├── snow-mirror-themes
├── temp
│   └── attachment
├── webapps
│   └── ROOT
└── work
    └── Catalina

2.5. Uninstall SnowMirror

2.5.1. Windows

  1. Go to installation folder.

  2. Open Uninstall.exe and follow the instructions.

2.5.2. Linux

If you have installed SnowMirror manually from an archive file, just follow these steps.

  1. Stop SnowMirror

  2. Delete "/opt/SnowMirror" directory.

Uninstall Linux Service
  1. Open a Terminal.

  2. Log in as root user.

  3. Remove Service:

    1. For Debian based systems:

      update-rc.d -f snow-mirror remove
    2. For RedHat based systems:

      chkconfig --del snow-mirror

3. Running the Setup Wizard

When you first access SnowMirror you will be redirected to the configuration wizard. It will walk you through basic SnowMirror configuration. Once you finish the Configuration Wizard data will be stored to the database and SnowMirror will be immediately ready to use.

3.1. Walkthrough

3.1.1. License

On this page you can provide a license which you received to your mailbox.

wizz license

3.1.2. Salesforce

Enter information about Salesforce instance. You can test the configuration using Test Connection button.

Test Connection button checks:

  • Whether the user can authenticate

  • Whether the user is authorized to access all necessary data

If your do not know user’s security token, see Obtaining security token section.
wizz salesforce

3.1.3. Database

Specify a database where all the data mirrored from Salesforce will be stored. Currently, SnowMirror supports Oracle, SQL Server, PostgreSQL and MySQL/MariaDB. You can test the configuration using Test Connection button.

Test Connection button checks:

  • Whether the user can be authenticated on the given server

  • Whether the user has all necessary privileges

  • Whether the database uses appropriate encoding

wizz db

NOTE:
Postgres, SQL Server - If you wish to use different schema than public, it is possible too. Just fill the field Schema with the name of your desired schema.

3.1.4. Snowflake Specific Settings

  • Account Name - Specifies the name of your account. The name is the first part of the URL you use to access Snowflake: https://account-name.snowflakecomputing.com

3.1.5. SMTP Server

Application allows users to subscribe to synchronization notifications. To deliver email to these users the SMTP server must be configured. If you want to enable notifications please specify URL of SMTP server and credentials. You can test the configuration using Test Connection button.

wizz smtp

3.1.6. Security Realm

SnowMirror provides two authentication options.

  1. Internal Database - Users and their credentials are stored in configuration database. You manage the users, their credentials and permissions using Settings page.

  2. LDAP - Users and their credentials are stored outside SnowMirror in your company’s LDAP (most usually Active Directory). You manage the users, their credentials and permission in your LDAP.

Internal Database

If you choose an internal database as a security realm you have to create administrator account. You will use these credentials to login into SnowMirror. The account will have all permissions granted.

You can also choose timezone in which all dates will be presented to you. It is a setting of the user interface and does not affect how date are stored in database.

wizz admin
LDAP

If you choose LDAP as a security realm you have to:

  1. configure a connection to your LDAP server on this page

  2. create SnowMirror roles in your LDAP (see LDAP Groups chapter and LDAP Sample Configuration chapters).

wizz ldap

The meaning of all the field is explained in LDAP chapter.

Always use Test connection button to verify your configuration. If you enter an invalid connection information you lock yourself out and you will not be able to login and change settings. Look at the screenshot above. We clicked the "Test connection" button and we were able to login and the user has a role assigned to him.

3.1.7. Summary

Here you can see a summary of all settings. If you want to change the configuration you can do so by clicking on "Change *" in a particular section.

wizz summary

3.1.8. Finish

After clicking the "Finish" button SnowMirror stores and processes all the settings. The Login page shows up and SnowMirror is ready to use.

wizz final

4. Upgrading SnowMirror

4.1. Windows Installer

If you installed SnowMirror using Windows installer you can upgrade to a new version using a new version of Windows installer. Before opening the installer we recommend to stop SnowMirror. The installer will guide you through the upgrade process, see below.

  1. Download the latest SnowMirror Windows Installer and open it. Then simply click Next.

    winInstaller firstStep
  2. License Agreement - to continue with the installation you have to accept license agreement by clicking on "I agree".

    winInstaller licenseAgreement
  3. Selecting the action - select "Upgrade an existing SnowMirror installation" option. The installer automatically fills a folder with SnowMirror installation.

    In case you installed SnowMirror to a different location you have to find the folder manually. Click "…​" button and "Browse For Folder" window pops up. Navigate to SnowMirror installation folder and click Ok. Once you are back in the original window, just click Next.

    winInstaller choosingtheaction upg
  4. Summary & Upgrade - review the folder you chose and click Install. When you click Install the upgrade process begins and you can’t revert it back.

  5. Installing - wait for SnowMirror to upgrade. Progress bar shows you how much work has already been done and how much work is remaining.

    winInstaller installing
  6. Finish - when Final screen shows up it means SnowMirror has been upgraded. After clicking on "Finish", SnowMirror is opened in your default web browser.

    winInstaller finish

4.2. Manual Upgrade

These upgrade instructions are meant for those who installed SnowMirror manually, i.e. without Windows installer. If you installed SnowMirror using Windows installer you should upgrade it using the Windows installer.

In the following guide we use "<ZipPackage>" placeholder. You have to replace them with a real full path to SnowMirror ZIP archive file.

  1. Download the latest SnowMirror ZIP package.

  2. Stop SnowMirror

  3. Delete old files and folder:

    1. Delete folder "/opt/SnowMirror/bin"

    2. Delete folder "/opt/SnowMirror/lib"

    3. Delete all files and folders in "/opt/SnowMirror/conf" except for "server.xml" and "web.xml" files.

    4. Delete all files and folders in "/opt/SnowMirror/snow-mirror" except for "/opt/SnowMirror/snow-mirror/data" folder and its content.

    5. Delete folder "/opt/SnowMirror/work"

    6. Delete folder "/opt/SnowMirror/webapps"

    7. Delete all files in "/opt/SnowMirror" except for "snowMirror.properties"

  4. Copy new files and folders from ZIP package.

    1. Copy "<ZipPackage>/bin" to "/opt/SnowMirror/bin"

    2. Copy "<ZipPackage>/bin" to "/opt/SnowMirror/lib"

    3. Copy "<ZipPackage>/conf" to "/opt/SnowMirror/conf" except for "server.xml" and "web.xml" files.

    4. Copy "<ZipPackage>/snow-mirror" to "/opt/SnowMirror/snow-mirror"

    5. Copy all files from "<ZipPackage>" to "/opt/SnowMirror" (except for "snowMirror.properties)

  5. Open a command line and find out a version of Java:

    /opt/SnowMirror/jre/bin/java -version

    If it is lower than 1.8, upgrade it to the latest version (1.8.x). If you don’t have to upgrade then continue with the next step.

    1. Here’s how to upgrade. Download Java Java Runtime Environment (JRE) and put it to the /opt/SnowMirror/tmp folder.
       
      JRE is available on https://www.java.com/en/download/manual.jsp page. For 64-bit system download "Linux x64" package, for 32-bit system download "Linux" package.

    2. Unpack JRE:

      cd /opt/SnowMirror/tmp
      tar zxvf jre-<CurrentJreVersion>-linux-[i586|x64].tar.gz
    3. Remove old jre directory.

      rm -rf /opt/SnowMirror/jre
    4. Copy the content of JRE to /opt/SnowMirror/jre.

      cp -r /opt/SnowMirror/tmp/jre<CurrentJreVersion>/. /opt/SnowMirror/jre
    5. Copy JCE Files. Passwords in configuration database are encrypted using AES 256. To make the encryption work you have to copy Java JCE files to your JRE.

      cp -r /opt/SnowMirror/snow-mirror/jce/1.8.0/* /opt/SnowMirror/jre/lib/security
    6. Remove temporary folder

      cd /opt/SnowMirror
      rm -rf /opt/SnowMirror/tmp
  6. If you installed SnowMirror as a service:

    1. Copy a script responsible for starting and stopping SnowMirror init.d directory.

      sudo cp /opt/SnowMirror/bin/init.d/snowmirror.sh /etc/init.d/snowmirror
    2. Make the init script executable:

      sudo chmod a+x /etc/init.d/snowmirror
  7. Start SnowMirror

Are you upgrading from SnowMirror older than 3.6.0 and you have changed memory settings in run.sh? Please move your memory settings to snowMirror.properties file. By default, SnowMirror is allowed to use up to 2 gigabytes of memory.

5. Settings

5.1. Profile Settings

In Profile Settings page you can see your roles and permissions. You can choose a timezone in which SnowMirror will present dates and times. Note that this settings will affect only your user account.

You can also choose a new password by clicking Change Password button and following a subsequent form.

5.2. Notification Management

To track synchronizations, you can add notifications. SnowMirror provides notifications for user-specified synchronizations when their run fails or the resulting state changes. Each notification can have its own set of tracked synchronizations and recipient email address.

5.2.1. Configure SMTP Server

For SnowMirror to be able to send notifications SMTP server has to be configured. On the page Settings / Notifications Management / SMTP Server you can specify its URL, port and credentials and security option.

Security option defines if and how the communication with SMTP server is encrypted:

  • None - No encryption on the outgoing emails. They will be sent as plain as day between SnowMirror and the server.

  • STARTTLS - SnowMirror will ask the SMTP server if it’s it’s possible to send data using TLS. If so, it will use the secure method. If not, data will be sent unencrypted.

  • SSL - SnowMirror will send data to SMTP server using encrypted connection. Data is never send unencrypted. This is the most secure method.

settings smtp

At the bottom of the page there is a button called Test Connection. It validates the configuration and send testing email to the email address you provide it the dialog window.

settings smtp send mail

5.2.2. Create a New Notification

To create a new notification you need to input an e-mail address. You can also choose specific synchronization you want to be informed about. Then you will choose what events you want to be notified about.

  • Every state change - you’ll be notified only when the state of the synchronization changes

  • All failed, all warnings, first successful - you will be notified after every failed or warning run. When you fix the synchronization, you’ll be notified too.

  • All failed, first warning, first successful - you will be notified after every failed run. When the synchronization ends with warning or you fix it, you’ll be notified too.

  • Every synchronization run - notification will be sent after every synchronization run regardless its state

notifications

5.2.3. Delete Notification

All added notifications are in a notification table. To cancel sending of notifications you can delete them by selecting a checkbox next to desired email address and clicking on Delete button.

notification list

5.3. User Management

User Management page allows you to configure which users will be allowed to login and which permissions will be assigned to them.

There are two authentication options: internal database or LDAP.

This setting option is available only for users with 'Synchronization administrator' role. See User Roles.

5.3.1. Security Realms

  • Internal Database - Users and their credentials are stored in configuration database. You manage the users, their credentials and permissions using Settings page.

  • LDAP - Users and their credentials are stored outside SnowMirror in your company’s LDAP (most usually Active Directory). You manage the users, their credentials and permission in your LDAP.

  • SSO - Single Sing-On using SAML 2.0

5.3.2. Internal Database

Choose "Internal Database" in the Security Realm field. Then, don’t forget to create a user Create New User with administrator permissions before you logout. Otherwise, you’d lock yourself out and would not be able to login and change settings.

settings sec realm internal db

5.3.3. LDAP

LDAP configuration is a bit more complicated than Internal database so let’s look at all the attributes in detail.

ldap sec realm
Field Description

Server

URL of your LDAP server. For example: myLdapServer.snow-mirror.com

If your LDAP server uses a port other than default 389 (i.e. standard LDAP port) you can also append a port number. For example: myLdapServer.snow-mirror.com:1234

To connect to LDAP over SSL, specify it with the ldaps:// protocol. For example: ldaps://myLdapServer.snow-mirror.com

If your LDAP over SSL uses a port other than default 636 (i.e. standard LDAPS port) you can also append a port number. For example: ldaps://myLdapServer.snow-mirror.com:1234

Manager DN

If your LDAP server does not support anonymous binding (i.e. it does not allow you to send a query without authentication), then you have to provide Manager credentials. SnowMirror will authenticate using these credentials and will query for user and group data using this account.

DN identifies the Manager user. For example: CN=MyUser,CN=Users,DC=myDomain,DC=com

Manager Password

Password of the user from Manager DN field.

Root DN

LDAP is basically a tree. If empty we will search the whole three. To improve performance you can specify the base DN - i.e. subtree. Then we will search only in this subtree. For example: DN=myServer,DN=com

User Search Base

Where in LDAP tree to search for users.

If empty, SnowMirror will search the whole tree.

If not empty, SnowMirror will search only a subtree. It is relative to Root DN. For example: CN=Users,CN=Employees

User Search Filter

The query used for finding users by their username.

Defaults to "(sAMAccountName={0})" - i.e. user name is stored in "sAMAccountName" attribute.

If you want to allow access only to user in a group then type: "(&(sAMAccountName={0})(memberOf=CN=Your CN,OU=Your OU,DN=YourDomain,DN=com))" and change "memberOf" to match the full name of your desired group.

Group Search Base

Where in LDAP tree to search for group your users belong to.

If empty, SnowMirror will search the whole tree.

If not empty, SnowMirror will search only a subtree. It is relative to Root DN. For example: CN=SnowMirror,CN=Applications

Group Search Filter

Looks for LDAP groups to which a user belongs to.

Defaults to "(member={0})" - i.e. it looks for all groups which have a "member" attribute set to DN of the user.

Some LDAP implementations use "uniqueMember" attribute instead of "member" attribute.

Allowed placeholders:
{0} - gets replaced by DN of the user
{1} - gets replaced by the username of the user

Group Role Attribute

The ID of the attribute which contains the role name for a group. Default value: cn

After you fill in all the fields you should test your configuration using "Test Connection" button. This is really important because if the configuration is invalid you would lock yourself out and won’t be able to login to SnowMirror at all.

After Test connection popup window opens, provide credentials of the user you will use to login to SnowMirror. We will test if the user can login and what permissions are assigned to him.

ldap sec realm tester

If everything went fine you will see a green message. If something goes wrong we print an error message.

ldap sec realm tester ok
LDAP Groups

To be able to work with SnowMirror you have to create groups in your LDAP and assign user to them.

You have to create following groups. Names of the groups match the names of SnowMirror roles. Their meaning is explained in User Manual.

  1. SUPER_ADMINISTRATOR

  2. SYNCHRONIZATION_ADMINISTRATOR

  3. SYNCHRONIZATION_SUPERVISOR

  4. API_CALLER

  5. SYNCHRONIZATION_READER

Choosing Your Own Group Names

If you don’t want to create groups with these exact names, you can choose your own names.

Open "Settings → Advanced Settings" page and modify following properties so that their values match names of your LDAP groups:

  1. snowMirror.auth.ldap.role.mapping.SUPER_ADMINISTRATOR

  2. snowMirror.auth.ldap.role.mapping.SYNCHRONIZATION_ADMINISTRATOR

  3. snowMirror.auth.ldap.role.mapping.SYNCHRONIZATION_SUPERVISOR

  4. snowMirror.auth.ldap.role.mapping.API_CALLER

  5. snowMirror.auth.ldap.role.mapping.SYNCHRONIZATION_READER

Fallback Role

SnowMirror does not allow users to login if they have no role assigned. This behavior is configurable - if a user does not have any role, you can assign him a default role.

You can do that by putting SnowMirror role name into "snowMirror.auth.ldap.role.fallbackRoles" advanced property.

Sample LDAP Configuration

It does not matter where in LDAP tree you create the groups. See an example from our testing LDAP server:

ldap tree

We have two users in our company. We have created the groups inside "ou=SnowMirror,ou=Employees,ou=Company" subtree.

Let’s look at the detail of SYNCHRONIZATION_ADMINISTRATOR group. See the member attribute? In this example, John Smith was assigned to this group. When he logs is he is able to do all the action super administrators can do.

ldap role

5.3.4. Single Sing-On (SSO)

SnowMirror supports Single Sing-On over SAML 2.0 protocol and Web SSO profile.

Prerequisites
  1. You have your own Identity Provider (IdP) up and running.

  2. In case your IdP runs on HTTPS SnowMirror has to run on HTTPS too.

Dictionary
  • IdP: Identity Provider is an application which authenticates users

  • SP: Service Provider is an application which uses IdP to authenticate user. SnowMirror is a service provider.

  • Entity Id: Unique identification of an application in IdP and SP.

Setup
  1. IdP: Register SnowMirror as a new service provider. The service provider is sometimes referred to as a client. By default, SnowMirror’s Entity Id (sometimes referred to as "Client Id" or "Service Provider Name") is SnowMirror but you can choose whatever name you want.

  2. IdP: locate IDPSSODescriptor. The location is IdP-specific. See documentation of your IdP to learn where to find it.

    IDPSSODescriptor describes the IdP server:

    • URLs where to reach the IdP

    • Information wherether SnowMirror should sign requests

    • Public key of the IdP (which allows SnowMirror to verify IdP’s signatures)

      See sample IDPSSODescriptor. You cannot use this samle in your SnowMirror installation. Instead, you have to get your own from your IdP. The sample should give you an idea what kind of file SnowMirror needs.

  3. SnowMirror: Configure SnowMirror to use SSO. Go to Settings → User Management → Security Realm page and choose "SSO" as a "Security Realm"

    • Entity Id: You have registered SnowMirror in your IdP. Please use the same name here. Default value is SnowMirror.

    • IdP Metadata: Copy and paste the IDPSSODescriptor from step 2. Note that if you change a configuration in IdP the "IDPSSODescriptor" may change too. Always make sure SnowMirror uses up-to-date descriptor.

    • Signing key: A pair of public and private key.

      SnowMirror uses the private key to sign data. IdP uses the public key to verify the signature. IdP knows the public key because it is available on "/saml/metadata" page.

      By default, SnowMirror uses self-signed certificate but you can import and use your own certificate.

    • Encryption key: A pair of public and private key.

      IdP uses the public key to encrypt assertions (it knows the public key because it is available on "/saml/metadata" page). SnowMirror decrypts the assertions with the private key.

      By default, SnowMirror uses self-signed certificate but you can import and use your own certificate.

    • Roles Source: SnowMirror loads the roles from its own configuration database and ignores (if provided) the roles in IdP.

    • Require Local Account: If enabled, a user can login only if a matching account exists in SnowMirror’s configuration database. When installing SnowMirror, you setup an admin account stored in configuration database. Using that account you setup user accounts for SSO users and assign them roles before they login. When you properly configure SSO and setup admin account you can disable login using the initial admin account.

    • Internal Realm: When enabled you can login using credentials from internal database. We recommend you to have Internal Realm enabled until you successfully finished SSO configuration.

      • Enabled - besides SSO, you can login on /sideDoor page using credentials from the configuration database. This is especially useful when you’re testing SSO configuration. If the SSO configuration is invalid you can still login using the config. database credentials and you won’t get locked out.

      • Disabled - you can login only using SSO. If the SSO configuration is invalid you’re locked out.

    • Once the configuration is finished and saved, SSO endpoints are made public and metadata file is generated.

      security realm sso

      The configuration page in SnowMirror does not contain any "Test Connection" button. It may be convenient to be logged in in one browser using credentials from configuration database and test the configuration in another browser (not just a tab). This way you’re always logged in in one browser and you use the second one for testing.

  4. IdP: Finish the configuration you started in the first step. Use the information below.

Using Your Own Signing And Encryption Keys

To setup signing and encryption SnowMirror need a private key and public certificate. By default, SnowMirror contains self signed certificate. To use your own you have to import it into SnowMirror.

  1. Download and install Keystore Explorer {link-keystore-explorer}. Keystore Explorer is a 3rd party tool which allows you to import keys and certificates. You could use a command-line "keytool" which is bundled with SnowMirror but Keystore Explorer is simpler to use.

  2. Open snow-mirror/data/sso/samlKeystore.jks in KeyStore Explorer and import signing key/certificate pair (they’re bundled in one file). Then import encryption key/certificate pair. Use default password "changeit".

  3. Go to Settings → User Management → Security Realm and select the keys in "Signing Key" and "Encryption Key" fields

ADFS

If you use ADFS (Active Directory Federation Services) you have to configure SnowMirror and ADFS as follows:

  • SnowMirror: Go to Settings → Advanced Settings page. Find "snowMirror.auth.sso.webSSOProfileOptions.includeScoping" property and set it to false.

  • ADFS: Add NameID as "Claim rule name", choose "Active Directory" as Attribute store, choose "SAM-Account-Name" as LDAP Attribute and "Name ID" as "Outgoing claim type", finish the wizard and confirm the claim rules window, in ADFS 3.0 you might need to configure the Name ID as a Pass Through claim.

  • ADFS: Open service provider by double-clicking it, select tab Advanced and change "Secure hash algorithm" to SHA-1

How To Configure Cluster

This chapter is intended for owners of Cluster license. If you have a Cluster license you have multiple SnowMirror installations (nodes) using the same configuration database.

Following properties must be configured separately for each node: Entity Id, Encryption Key, Signing Key.

If you log into the first node and open Settings → User Management → Security Realm page you can configure Entity Id, Encryption Key, Signing Key of that node. If you log in to the second node you configure the second node.

Properties other than Entity Id, Encryption Key, Signing Key are shared between the nodes and it does not matter from which node you configure them.

You should restart SnowMirror after you make a change. That is necessary so that both nodes are properly initialized and aware of that change.

5.3.5. Create New User

If you use Internal database security realm you can create a new user. You have to provide its username, password, timezone and roles. See User Roles chapter so that you can decide which role you want to assign to the user. There is usually no need to add more than one role (except for the API Caller role).

new user

5.3.6. User Detail

On user detail page you can see user’s information summary - his or hers username, selected timezone, roles and permissions.

Permissions are here just for your information and can’t be altered. They are connected to user’s role not the user itself.
Edit User

On edit page you can edit username, selected timezone and user’s role (See User Roles). If you need to change user’s password see Change Password Section.

Activate/Deactivate

The user’s account can be deactivated by clicking on Deactivate button. After that the user is no longer able to log in. The account can be reactivated by clicking on Activate button.

Change Password

On this page you can set the password to any user. You can find it helpful when someone forgets the password and needs a new one.

5.4. General Settings

5.4.1. Cache Management

The application uses cache for retrieving Salesforce meta-data (what tables Salesforce works with, which columns they have etc.). Because of caching some data could be out of date. Typical example is list of Salesforce tables available to synchronize. Default reload interval of cache is 24 hour.

If you want to clear the cache manually go to Settings menu, choose General Settings and click on Clear All Caches button.

5.4.2. General

  • Use Temporary Tables - This settings applies only to Clean And Synchronize. If it is disabled SnowMirror cleans all data and then downloads them. Because of that you have to wait for you data to be downloaded to mirror database. If it is enabled SnowMirror downloads all data to temporary table. After synchronization finishes it removes old data and moves temporary table to your mirrored table. During Clean And Synchronize you have all your old data available in mirror database.

  • Column Name Suffix - Column names in Salesforce may end with a suffix. For example, custom columns end with "__c". There are two ways how to treat the suffixes.

    • Preserve - column names in your mirror database contain the suffix

    • Remove - column names in your mirror database do not contain the suffix. All such columns can be found on Edit Mapping page.

  • Records Delete - SnowMirror is able to find out when a record is deleted in Salesforce and then delete it in your mirror database. Records Delete allows you to choose how to delete the record.

    • Hard - record is irreversibly deleted from mirror database.

    • Soft - record is not deleted and "mirror_deleted_on" column is filled in. The column contains date and time when SnowMirror found out it was deleted.

      SnowMirror deletes all records (including soft-deleted records) in case:

      • Clean and synchronize is called

      • Truncate delete strategy is used

      • A new column is added using Auto Schema Update

      If you switch from Soft to Hard, "mirror_delete_on" column will be dropped during the next synchronization run.

  • Synchronize Archived - Salesforce archives records in some tables. Those records are hidden but are still available in Salesforce.

    • Yes - SnowMirror synchronizes archived records to the mirror database.

    • No - SnowMirror does not synchronize archived records. When Salesforce archives a record, SnowMirror deletes it from the mirror database.

      Warning: changing this option affects all synchronizations that may contain archived records (Announcement, ContentDocument, Event, CollaborationGroup, Pricebook2, Task). When you switch to Yes, you should Clean & Synchronize affected synchronizations. When you switch to No, you should run Differential synchronization or Clean & Synchronize on affected synchronizations.

  • Synchronize Deleted - Salesforce has a recycle bin which contains deleted records.

    • Yes - Clean synchronization synchronizes even records from recycle bin. This option is useful only if you use Soft deletes.

    • No - Clean synchronization does synchronize records from recycle bin.

      Warning: changing this option affects all synchronizations that may contain archived record. When you switch to Yes, you should Clean & Synchronize affected synchronizations. When you switch to No, you should run Differential synchronization or Clean & Synchronize on affected synchronizations.

general settings general

5.4.3. Activity Log Retention

Activity Log Retention removes old activity logs after the specified number of days (30 days by default). It removes activity logs from both configuration database and filesystem. Removing logs from filesystem saves you disc storage whereas removing logs from configuration database improves its performance. This is particularly true if you have a lot of synchronizations running very often.

If you are upgrading from SnowMirror < 3.2.0 this feature is disabled. On new installations it is enabled. You can change the settings on General Settings page.

  • Enabled - enable or disable activity log retention

  • Cleanup database - you may choose to remove the logs from configuration database or leave them there intact

  • Retention period - how many days you want to keep the activity logs intact

general settings activity log retention

5.4.4. Consistency Check

Allows you to configure consistency check. On fresh installations the consistency check is enabled by default. On existing installations it is disabled.

  • Consistency Check - enable or disable consistency check

  • Issue Warnings - If enabled SnowMirror will set status of a synchronization to Warning if it contains suspicious number of records in mirror database. If disabled a result of the consistency will be only written to the activity log and the synchronization will be marked as successful.

general settings consistency check

5.4.5. Skip Failing Records

If enabled SnowMirror will skip the records which it failed to insert or update. If that happens the synchronization will finish with a warning (you can configure notifications not to miss any warning).

If SnowMirror skipped a record you have to analyze the root cause of the issue and fix it. Then you download the missing records using Differential synchronization, Synchronize from or Clean and synchronize.

There is a limit of how many records will SnowMirror skip at maximum. Let’s say you have a table with hundreds of thousands records and SnowMirror fails to insert all of them. Then you want the synchronization to fail also because there must be a global issue (like failing database connection, permissions in Salesforce, etc.). On the other hand if just one record fails then you want to end up with a warning and fix the one records separately.

general settings skip failing records

5.4.6. Timezone Settings

By default SnowMirror stores data to mirror database in UTC. UTC is the most convenient and recommended way to store dates and times especially if your organization spans across several timezones. If you need to access and see date and time fields formatted in specific time zone, SnowMirror provides two ways to achieve that:

  • Mirror Data Time Zone - select a timezone of your choice. SnowMirror still receives dates and times from Salesforce in UTC but when storing them to database it converts them to the timezone of your choice.

  • Timezone Data Types - If you enable this option you will force SnowMirror to use following data types for storage:

    • Oracle - timestamp with local time zone

    • PostgreSQL - timestamp with time zone

    • MySQL - timestamp

    • Embedded H2 - datetime

    • SQL Server - datetimeoffset

      Internally dates and times are stored in UTC (MySQL, SQL Server, PostreSQL) or in database timezone (Oracle). Every time you request the data using SQL they are converted to user local session time zone you chose when creating a connection to mirror database. This does not work in SQL Server where you have to convert the dates from UTC to other time zones manually.

      Also note that you have to drop existing table for this option to work properly - SnowMirror needs to recreate the table with the proper data type.

general settings timezone settings

5.5. License

On this page you can see basic information about current license.

license detail

If you click to the Enter License button you can enter a new valid license key. This action overrides the old license key if any is present.

5.5.1. License key

License key is generated string (about 600 characters long) which includes information about SnowMirror application. If you don’t have any key yet please visit our website.

Salesforce

Enter information about Salesforce instance. You can test the configuration using Test Connection button.

Test Connection button checks:

  • Whether the user can authenticate

  • Whether the user is authorized to access all necessary data

If your do not know user’s security token, see Obtaining security token section.
settings salesforce

5.6. Mirror Database

Allows you to set information about the mirror database (the database where all data from Salesforce is stored). The common scenario is to use your own database server. Built in database is recommended for testing purposes only.

Test Connection button checks:

  • Whether the user can be authenticated on the given server

  • Whether the user has all necessary privileges

  • Whether the database uses appropriate encoding

settings database
If you want your Oracle or Postgres db to treat identifiers as case sensitive (using of double-quotes in sql statements), please make sure to have Quote Identifiers checkbox checked as seen on picture below.
settings database 2

NOTE:
Postgres, SQL Server - If you wish to use different schema than public, it is possible too. Just fill the field Schema with the name of your desired schema.

5.7. Import / Export

In this section you can export your synchronization settings or import your previously exported setting. Note that only settings is exported - the actual data in mirror database stay intact.

Example: You have testing and production environment. Now you want to move all synchronization settings from your testing environment to production environment. You export the settings on testing environment and then import on production.

In case imported file contains synchronizations which already exist SnowMirror will let you choose how to resolve the conflict:

  • Do not import duplicate synchronizations - this option won’t import synchronizations which already exist

  • Automatically rename duplicate synchronizations - this option will import all synchronizations but those with conflicted names will be renamed. For example: you try to import synchronization for table "task" but it already exists - SnowMirror will import the synchronization but its name and name of table in mirror database will be "task_1"

  • Update existing synchronizations - this option will overwrite (update) existing synchronizations.

5.8. Visual Theme

SnowMirror can be configured to use custom visual style such as colors and logo. You have two ways how to do that. You can either change it in application itself (/settings/theme) or you can do it manually in the installation folder.

5.8.1. In App Modification

Here you can see setting screen for visual themes. there are parameters that can be changed:

  • Theme color: You can change color of main navigation bar (right under the logo) and all table headers

  • Text Beside Logo: This option will help you to distinguish your SnowMirror instances by putting a text next to a logo (E.g.: development, test etc.). Are you can put here a short message to all SnowMirror users (Max 100 characters allowed)

  • Logo Image: You can change the main logo here (supported formats: jpg, png, gif), so you can unify SnowMirror with your corporate image.

visual themes gui

5.8.2. Manual Modification

This is done by using custom CSS stylesheet.
All themes are located under /snow-mirror-themes directory. Each theme consists of custom stylesheet and optional logo image (png, jpg or gif).

For example theme named myThemeName will have two files in /snow-mirror-themes folder:

  • myThemeName-style.css

  • myThemeName-logo.png

Now you have to tell SnowMirror which theme to use. This is done by setting snowMirror.theme.name property in /snowMirror.properties (or in application in Settings / Advanced Settings)

snowMirror.properties
## SnowMirror visual themes ##
snowMirror.theme.name = myThemeName

There are several themes available in SnowMirror distribution package. These can be used to differentiate between several environments (e.g. test, pre-live, …​) - they just add colored stripe with environment name to the top of the page.

5.9. Advanced Settings

5.10. Time Zones

Every SnowMirror user can set his or hers own time zone. This does not affect communication with Salesforce, or any of SnowMirror databases. The time zone setting is just for presenting data via SnowMirror gui. List of available time zones is configurable.

All data in mirror and configuration database are stored in UTC.

timezone

Communication with Salesforce happens in two time zones. While requesting data via web service is used UTC (both request and response). However while sending data via web service is used the time zone of Salesforce user. This time zone can be configured in Advanced Settings.

5.10.1. Changing Timezone

For changing timezone click on the planet icon next to timezone in the left bottom.

timezone edit

6. Salesforce Information

6.1. User rights

Here you can find steps to create user with sufficient rights for synchronizations. Login to your Salesforce instance using administrator account and follow these steps:

  1. Create Profile (Classic: Setup → Manage Users → Profiles, Lightning: Setup → Users → Profiles)

    1. New Profile

      1. Existing profile = Read Only

      2. Profile Name = <profile name>

      3. Create

    2. Edit <profile name>

      1. Check API Enabled (should be)

      2. In Standard Object Permissions check Read and View All by all tables that you want to synchronize by this user

      3. Save

  2. Create User (Classic: Setup → Manage Users → Users, Lightning: Setup → Users → Users)

    1. New User

      1. User Licence = Salesforce

      2. Profile = <profile name>

      3. Check Active (should be)

      4. Save

6.2. Obtaining security token

Login to your Salesforce instance and follow next steps:

6.2.1. Classic

  1. In the upper menu click on your name and My Settings.

  2. In the left menu click on Personal and Reset my security token.

  3. Click on button Reset my security token and you will have it in your mailbox.

6.2.2. Lightening

  1. In the the upper menu click on icon on the right and then choose "Settings".

  2. In the left menu navigate to "My Personal Information" → "Reset my security token".

  3. Click on Reset Security Token button and you will have it delivered in your mailbox.

6.3. Getting your Salesforce organization ID

Login to your Salesforce instance and follow next steps:

6.3.1. Classic

  1. In the upper menu click Setup.

  2. In the left menu click on Company Profile and Company Information.

  3. On displayed page you have your Salesforce.com Organization ID on the right side of the page

6.3.2. Lightening

  1. In the upper menu click the "Gear" icon and then choose "Setup"

  2. In the left menu navigate to "Company Settings" → "Company Information".

  3. You can find the "Salesforce.com Organization ID" on the right side of the page.

7. Advanced configuration

Most of the configuration settings can be done in SnowMirror on Settings page when the configuration choices are stored in a database. Yet some configuration have to be done using snowMirror.properties file.

snowMirror.properties

The file is located in <SnowMirrorInstallationFolder> folder and contains deployment URL settings and configuration database settings.

Settings page

These properties are stored in database in a table called config. You will set the most important settings through Configuration Wizard which pops up when SnowMirror starts for the first time. Later on if you need to change the properties you will do that on Settings page in SnowMirror.
Contains all other settings including mirror database, SMTP server, notifications, Salesforce, users, etc.

The snowMirror.properties file contains just a small subset of all configuration options. Yet the contents of the file is very important for SnowMirror to run properly. Following subsections explain the snowMirror.properties settings in detail.

7.1. Database Configuration

SnowMirror application uses two distinct databases.

  • snowmirror database - used for storing application synchronization settings, user settings, logs…​

  • mirror database - contains the actual tables synchronized from Salesforce.

This chapter deals with configuration database (snowmirror) only.
Both configuration and mirror databases have to use UTF-8 encoding. If they don’t SnowMirror may fail to run some synchronizations because it won’t be able to store some characters into the database.

Application supports variety of database systems which can be used for both snowmirror or mirror database. The actual databases are discussed in following subchapters.

  • snowmirror database configuration is located in snowMirror.properties file. If you want to use a database system other than H2, you can change it during the installation or later in the file manually. Example configuration can be found in the following sections.

  • mirror database is configured via Configuration Wizard. Later on you can change the settings directly in SnowMirror on the page Settings / Database

SnowMirror needs following database permissions:

  1. be able to connect and login to the database

  2. be able to create, alter and delete tables and views

  3. be able to create, alter and delete indexes on those tables

  4. be able to insert, update and delete data in those tables and views

7.1.1. H2

{{link-h2-web}[H2 Database Engine] is a lightweight open source Java SQL database. We have embedded H2 into SnowMirror.
The database is created automatically by SnowMirror based on connection url information from snowMirror.properties file.

Sample settings of configuration database in snowMirror.properties
# SnowMirror configuration database (H2)
config.db.type = h2
config.jdbc.url = jdbc:h2:~/h2/snowmirror;MVCC=TRUE
config.jdbc.username = sa
config.jdbc.password = sa

7.1.2. MySQL

When creating a new database please use utf8 encoding and utf8_unicode_ci collation.

Sample Create Database SQL
CREATE DATABASE snowmirror CHARACTER SET = utf8 COLLATE utf8_unicode_ci;
CREATE DATABASE mirror CHARACTER SET = utf8 COLLATE utf8_unicode_ci;
Sample settings of configuration database in snowMirror.properties
# SnowMirror configuration database (MySQL)
config.db.type = mysql
config.jdbc.url = jdbc:mysql://127.0.0.1:3306/snowmirror
config.jdbc.username = root
config.jdbc.password = password

7.1.3. SQL Server

When creating a new database please choose a collation that suits your needs the best.

Sample settings of configuration database in snowMirror.properties
# SnowMirror configuration database (SQL Server)
config.db.type = mssql
config.jdbc.url = jdbc:jtds:sqlserver://127.0.0.1/snowmirror;instance=SQLEXPRESS
config.jdbc.username = root
config.jdbc.password = password
config.jdbc.schema = dbo

7.1.4. Oracle

When creating a database please make sure to use AL32UTF8 character set.

Sample settings of configuration database in snowMirror.properties
# SnowMirror configuration database (Oracle)
config.db.type = oracle
config.jdbc.url = jdbc:oracle:thin:@127.0.0.1:1521:snowmirror
config.jdbc.username = root
config.jdbc.password = password

7.1.5. PostgreSQL

When creating a new database please use UTF8 encoding. Values for lc_collate and lc_ctype properties are provided automatically. If you need to change them please refer to the official PosgreSQL documentation.

Sample Create Database SQL
CREATE DATABASE snowmirror ENCODING 'UTF8';
CREATE DATABASE mirror ENCODING 'UTF8';
Sample settings of configuration database in snowMirror.properties
# SnowMirror configuration database (PostgreSQL)
config.db.type = postgres
config.jdbc.url = jdbc:postgresql://127.0.0.1:5432/snowmirror
config.jdbc.username = root
config.jdbc.password = password
config.jdbc.schema = public
If you are having performance issues please check out "mirror.springframework.ignoreParameterType" advanced property (Settings → Advanced Settings). If enabled we won’t try to retrieve columns meta-data for every column hence improving the speed of inserts/updates. We recommend having this property enabled for PostgreSQL.

7.2. Application Deployment URL

Deployment address depends is the address of the server where the application is deployed. The application needs to know this address to correctly serve its pages and content.

Provided configuration is used to construct absolute URLs for resources files like JavaScript and CSS. See how SnowMirror constructs URLs using these properties:

Constructed absolute URL
{snowMirror.scheme}://{snowMirror.host}:{snowMirror.port}{snowMirror.context}
Example of Application URL Configuration - excerpt from snowMirror.properties
### SnowMirror base URL ###
snowMirror.scheme = http
snowMirror.host = localhost
snowMirror.port = 9191
snowMirror.controlport = 8006
snowMirror.context =
Specifying a context

If you want SnowMirror to run not on http://localhost:9191 but on http://localhost:9191/snowMirror you have to change the snowMirror.context property. Note that when you specify a context it has to start with a backslash.

snowMirror.context = /snowMirror
Control Port

SnowMirror runs on Apache Tomcat server which uses a notion of a control port. The port is used to receive shutdown request. The default value is provided but if you change the port in Apache Tomcat configuration files you also have to change snowMirror.controlport property.

snowMirror.controlport = 8006

7.3. Security

7.4. SnowMirror Over HTTPS

By default, SnowMirror is accessible through HTTP protocol. That means the communication between the user and SnowMirror is not encrypted. This chapter explains how to enable HTTPS protocol.

  1. Stop SnowMirror

  2. Replace HTTP configuration files with HTTPS configuration files. Go to <SnowMirrorInstallationDirectory>/conf and rename:

    1. server.xml → server-http.xml

    2. server-https.xml → server.xml

    3. web.xml → web-http.xml

    4. web-https.xml → web.xml

  3. Open snowMirror.properties file and change "snowMirror.scheme" and "snowMirror.port" properties.

    snowMirror.properties excerpt
       snowMirror.scheme = https
       snowMirror.port=9443
  4. Now, SnowMirror is configured to use HTTPS. But you need to do one more step - you have to import your certificate and its private key to SnowMirror.

    The certificate and its private key are bundled in a single PFX file. You need to import it to snowMirror-https.jks file.
    If you are asked for a source keystore password, type your password. If you are asked for a destination keystore password, type "password". When asked if you trust the certificate, type "yes".

    Windows command line
    cd "C:\Program Files\SnowMirror"
    del snowMirror-https.jks
    jre\bin\keytool -importkeystore -srckeystore <path-to-your-pfx-file> -destkeystore snowMirror-https.jks -destkeypass password
    Linux command line
    cd /opt/SnowMirror
    rm snowMirror-https.jks
    jre/bin/keytool -importkeystore -srckeystore <path-to-your-pfx-file> -destkeystore snowMirror-https.jks -destkeypass password
    As you can see, we use a default password. You can choose whatever password you want. If you do so you also have to change it in "server.xml" file. Look for a "keystore" attribute and change its value to your password.
  5. Start SnowMirror

7.5. Security Properties

On Settings → Advanced Properties page we offer several security related properties. Usually, there is no need to change them. This chapter explains these properties.

Property Default Description

snowMirror.security.httpHeaders.useXFrameOptions

true

If enabled, pages contain "X-Frame-Options: SAMEORIGIN" HTTP header.

snowMirror.security.httpHeaders.useXXSSProtection

true

If enabled, pages contain "X-XSS-Protection: 1; mode=block" HTTP header.

snowMirror.security.httpHeaders.useXContentTypeOptions

true

If enabled, pages contain "X-Content-Type-Options: nosniff" HTTP header.

snowMirror.security.httpHeaders.strictTransportSecurity

false

If enabled, pages contain "Strict-Transport-Security: max-age=31536000 ; includeSubDomains" HTTP header.

snowMirror.security.showExceptionOnErrorPages

true

If enabled, the error page will display stack trace of an exception. Disable this property if your security policies do not allow to reveal such information.

snowMirror.security.showSystemInfoOnErrorPages

true

If enabled, the error page will display system information like SnowMirror version, database type and version, etc. Disable this property if your security policies do not allow to reveal such information.

snowMirror.security.showVersion

true

If enabled, pages contain SnowMirror version in a footer.

snowMirror.security.useChartsExternalLibrary

true

If enabled, SnowMirror draws charts using Google JS API. If disabled SnowMirror does not draw charts at all and does not communicate with that API at all.

7.6. Memory Settings

SnowMirror allows you to configure how much memory it can use. After the start, SnowMirror allocates 700MB of memory and is allowed to allocate up to 2GB of memory.

In some cases you may need to increase a memory allocated to SnowMirror: If you download a lot of big records at once or if you increased the number of synchronizations running in parallel (snowMirror.quartz.threadCount property).

7.6.1. Windows service

  1. Open cmd.exe from Windows Start menu as an Administrator

  2. Type: cd "C:/Program Files/SnowMirror/bin"
    (C:/Program Files/SnowMirror is default installation folder which is configurable during the installation)

  3. Open configuration window by typing: tomcat9w.exe //ES//SnowMirrorSF
    (SnowMirrorSF is a name of the service - if you changed the name during the installation, change it in this command)

    increase memory
  4. Open "Java" tab and increase the value in "Maximum memory pool" field. Click OK.

  5. Restart the service.

7.6.2. Linux

  1. Stop SnowMirror

  2. Edit file /opt/SnowMirror/snowMirror.properties

  3. Add or edit the following line. Specify the value in megabytes (e.g. 1500m) or gigabytes (e.g. 2G):

    snowMirror.memory.Xmx=2G
  4. Save the file

  5. Start SnowMirror

8. How To

This chapter contains a bunch of mutually unrelated "How To" guides which don’t fit into any other chapter.

8.1. Thread Dump

Thread dump captures the current state of SnowMirror and its threads. It allows us to understand what was going on inside SnowMirror at moment you captured it. That is useful in case SnowMirror gets stuck. You are not expected to understand thread dump output - that is job for our L3 support.

The next two chapters describe how to create a thread dump on Windows and Linux.

How To Create Thread Dump On Windows

Version: 4.3.0 and newer, OS: Windows

  1. Open Task Manager (Ctrl + Shift + Esc)

  2. Click "Details" tab

  3. Find "tomcat9.exe" and remember the value of "PID" column

  4. Open cmd.exe as an Administrator

  5. Execute: cd "C:\Program Files\SnowMirror"

  6. Execute: jre\bin\jstack.exe PID > logs\thread-dump.log

  7. Send us C:\Program Files\SnowMirror\logs\thread-dump.log file

thread dump windows
How To Create Thread Dump On Linux

Version: 4.3.0 and newer, OS: Linux

  1. Connect to your server.

  2. Find out PID of SnowMirror: ps aux | grep [/]opt/SnowMirror | awk '{print $2}'

  3. Create a thread dump file: /opt/SnowMirror/jre/bin/jstack PID > /opt/SnowMirror/logs/threaddump.log

  4. Send us /opt/SnowMirror/logs/thread-dump.log

thread dump linux

9. FAQ

  1. What is the default timezone?

    Application uses UTC as default timezone. Also the data stored in database are in UTC timezone.

  2. Can I delete old log files about synchronization runs?

    Yes, you can. They are located by snowMirror.activityLog.dir property (default value is logs/activityLogs). The files are located by this pattern: <synchronizationId>/<yyyy-MM>/<logId>.log. If you find the activityLogs directory too large you can delete files or whole directories to save some disk space.

  3. How do you protect my passwords?

    Every sensitive information such as password is encrypted with AES-256 cipher. Encryption is applied on data stored in database as well as data stored in configuration files.

10. Release Notes

Version Date Remarks

2.5.2

January 15, 2021

  • Snowflake data types - we have changed a mapping of Salesforce data types to Snowflake data types. If you use Snowflake database, please review the changes. If you create a new column, it will use the new mapping. Existing columns are not affected.

2.5.1

January 11, 2021

  • Fixed: some synchronizations failed with "unexpected end of file while reading quoted column beginning on line 2 and ending on line 2" error.

2.5.0

November 13, 2020

  • Snowflake support - you can store your data into Snowflake warehouse.

  • Authentication using SSO - SnowMirror can authenticate users using Single Sing-On over SAML 2.0 protocol and Web SSO profile.

  • Improved performance: SnowMirror downloads data from Salesforce and stores it to database in parallel. Data is stored to the database in so-called "Data Processing" threads.

    This may improved the total time needed to finish a synchronization especially if the connection to the database is slow.

    serial vs parallel processing
  • Java Upgrade - Upgraded Java to the latest version 11.0.9+11.

  • Apache Tomcat - Upgraded the embedded web-server Apache Tomcat to the latest version 9.0.39.

2.4.4

June 15, 2020

  • Signed Windows exe installer with a new code signing certificate.

2.4.3

June 6, 2020

  • Fixed: Broken link in documentation.

2.4.2

June 3, 2020

  • Fixed: Names of Lightening attachments did not contain an extension. For example a file named "Report.pdf" was synchronized as "Report".

    • To load the extensions for already synchronized files you have to re-synchronize from scratch (i.e. call Clean And Synchronize)

2.4.1

May 19, 2020

  • Retry mechanism - SnowMirror downloads data into temporary file before storing them to the database. If there was a network error during the download the synchronization failed. We have added a retry mechanism. In case of error, SnowMirror tries to download the file up to 5times before failing.

  • Fixed: Checkbox "Change Proxy Password" on Settings → Salesforce page was show even when it should not have been show.

  • Fixed: In some cases, SnowMirror failed to properly check if table exists. Only tables containing "_" were affected.

  • Fixed: Count Records button on Edit Setting page may have shown incorrect result.

    • Only users using the following General Settings were affected: Records Delete = Soft and Synchronize Archived = No and Synchronize Deleted = Yes

2.4.0

May 13, 2020

How to upgrade

  • We have upgraded the embedded web-server Apache Tomcat. Because of that, the name of the process has changed from "tomcat8" to "tomcat9". Some customers may have a rule in firewall (or some other security tool) which allows/prevents the process from running. If you have such a rule please change it so that the new name "tomcat9" is permitted.

  • Linux users must upgrade Java to the latest AdoptOpenJDK release as described below in the Change Log. On Windows, Java is updated automatically when SnowMirror is updated.

Changelog

  • Upgraded Java to 11.0.6+10 - SnowMirror is written in Java and needs Open JDK to run. Since Java 11, Oracle (the company that owns and develops Java) has changed licensing terms and support model. This topic is quite complex but the important message is that you don’t have to pay for Java.

    SnowMirror for Windows already contains free of charge Java. Linux users are advised to pay attention to which Java distribution they download. There are several distributions of Java available:

    • Please, use Open JDK builds from AdoptOpenJDK.

    • Please, avoid Oracle JDK. OracleJDK distribution is paid and comes with Oracle’s commercial support.

  • Apache Tomcat - Upgraded the embedded web-server Apache Tomcat to the latest version 9.0.35.

2.3.1

October 24, 2019

- Fixed: SnowMirror had trouble reading trial licenses downloaded and generated by our website.

2.3.0

August 12, 2019

- Added a support for archived records. Users can choose how to treat archived records - synchronize them (default) or ignore them. Users should clean and synchronize tables that support archiving (Announcement, ContentDocument, Event, CollaborationGroup, Pricebook2, Task).

2.2.3

August 6, 2019

- Fixed: Consistency check on synchronizations with Truncate delete strategy incorrectly assumed that nothing could have been deleted.

2.2.2

August 2, 2019

- Fixed: Soft deleted records were not updated before marked as soft deleted.
- Fixed: Soft deletes are picked up together with inserts and updates. If Soft delete is used, delete strategy may be set to None.
- Fixed: Incremental load could have missed updated records when Diff sync had run before it.

2.2.1

July 29, 2019

- Fixed: Failure to load meta-data of a single table caused failure of all the other synchronizations.

2.2.0

July 23, 2019

- Fixed: Audit Deleted strategy could have missed some records deleted in Salesforce. Therefore, mirror database contained more records than Salesforce. Use Differential synchronization to remove excessive records.

2.1.10

July 17, 2019

- Tables without "SystemModstamp" and "LastModifiedDate" columns always triggered clean synchronization. From now on, if none of the columns is present the "CreatedDate" column is used instead.
- Allowed to use Audit Delete Strategy on history tables.
- Fixed: In some cases, picklist used to be mapped to data types with max. length of 80 characters. From now on, the max. length has changed to 255 characters.

2.1.9

July 8, 2019

- Fixed: Picklist used to be mapped to data types with max. length of 80 characters. From now on, the max. length has changed to 255 characters.

2.1.8

July 2, 2019

- Fixed: Data type "time" was not properly handled which lead to a synchronization failure.

2.1.7

June 27, 2019

- Before storing data to the database, SnowMirror stores data from Salesforce into temporary file. That minimizes the time the connection is open and prevents network issues.

2.1.6

June 21, 2019

- Using "SystemModstamp" column instead of "LastModifiedDate" to synchronize data because it covers changes from automated processes.

2.1.5

June 13, 2019

- Fixed: Incremental synchronization failed when an "Id" column was renamed to "ID"
- Fixed: "jsonNot a valid enumeration for type: class com.sforce.soap.partner.SoapType" error. Fixed by updating "com.force.api" libraries to the latest versions (which support API v46).

2.1.4

June 7, 2019

- Introduced an advanced property "mirror.mssql.useUnicodeDataTypes" which allows you to use "nvarchar" data type instead of "varchar" on SQL Server.
- Introduced an advanced property "snowMirror.dataType.inheritPrecision" which allows you to use precision and scale of decimal type as it is defined in Salesforce.
- Introduced a new Auto Schema Update option - Enabled (no truncation). It allows you to add new columns without cleaning and synchronizing the whole table.
- Fixed: Synchronization failed due to error: null

2.1.3

May 29, 2019

- Added an option to use uppercase mirror table names. It can be configure on Settings → Advanced Settings page by setting "snowMirror.mirrorTable.namingStrategy" property.

2.1.2

March 27, 2019

- Soft Delete - besides "mirror_deleted_on", it is possible to add another column "mirror_deleted". It contains a flag which helps you to determine if a record was deleted or not. This is configurable through Advanced Properties.

2.1.1

March 27, 2019

- Fixed: Differential sync fail because it could not find mirror_deleted_on column.

2.1.0

March 27, 2019

- A new option in General Settings - ability to automatically remove suffixes (e.g. "__c") from column names in mirror database.
- Soft Delete - instead of deleting records, you can optionally mark them as deleted and keep them in mirror database.

2.0.2

February 26, 2019

- Security: Added ability to enforce strong password.
- Security: Changed pages where passwords are entered so that they adhere best practices.

2.0.0

August 15, 2017

- Upgraded and improved replication algorithm so that it can handle bigger tables.
- Besides replication we also support backups.

1.1.0

February 20, 2015

- Added password encryption - credentials to config database, mirror database and Salesforce are encrypted using AES 256.
- Improved UI - all necessary information is visible right away (e.g. status bars).
- Automatic detection of user’s timezone when creating a new user.
- Minor bug fixes for SQL Server and PostgreSQL databases.

1.0.0

January 27, 2015

Public release.