Many ServiceNow customers clone their production instances into their test or development environments. What to do if you are mirroring an instance that has been overridden by a clone? Please perform the following steps:
- Export all your synchronizations into an XML file
- Delete all synchronizations
- Import the XML file again
- Perform an initial synchronization
The reason to export and import the synchronizations is that SnowMirror stores references to ServiceNow meta-data. However the cloning procedure completely changes the meta-data so it is necessary to renew the references.
SnowMirror stores the data the same way as they are stored in the ServiceNow database. Not the way the data is displayed in the ServiceNow web application. The web user interface and even some exports (e.g. ODBC or exported XML) contain a display value of a referenced record (e.g. incident caller name). However SnowMirror downloads only a sys_id into the caller_id field. And again to obtain all the data you need please download two tables (incident and sys_user) and join them together.
The same applies for journal fields such as incident comments. These records are stored in the sys_journal_field table and the have to be joined with the incident table.
SnowMirror stores the data the same way as they are stored in the ServiceNow database. Not the way the data is displayed in the ServiceNow web application. So for example the journal fields are not stored directly in the original table (e.g. incident) but in a special system table called sys_journal_field. To view incidents together with comments you need to synchronize two tables (incident and sys_journal_field) and then perform a join to display the data together.
The same applies for referenced records. The web user interface and even some exports (e.g. ODBC or exported XML) contain a display value of a referenced record (e.g. incident caller name). However SnowMirror downloads only a sys_id into the caller_id field. And again to obtain all the data you need to download two tables (incident and sys_user) and join them together.
ServiceNow stores all the date and time related fields in the UTC timezone. What you see in the ServiceNow user interface is just a display setting of your user account, timezone configuration. SnowMirror stores the timestamps in a way it receives them from ServiceNow. So all timestamps are stored in the UTC timezone. You have to convert those values into your timezone if you need it for reporting purposes.
Why it is not a good idea to store the dates in your own timezone? What if you have users in several timezones? How about the DST?
SnowMirror itself is very fast and has been fine-tuned for performance. However the overall performance hugely depends on performance of the ServiceNow instance which varies a lot between ServcieNow customers. Using one of the fastest ServiceNow instances in the world SnowMirror was able to download a 500K+ table in about 5 minutes. However it is just an initial table load. All other runs are incremental load. Even on the biggest instances the update takes small number of minutes. Typical SnowMirror user synchronizes data once a day at night so the performance is not an issue.
By default SnowMirror downloads three tables in parallel. The number was carefully chosen according to the average instance performance. If you instance is faster you may want to increase this number (8 tables in parallel might be the maximum value making sense). Use the following properties
snowMirror.quartz.threadCount and serviceNow.httpMessageSender.max.connections.
Even in this case it is possible to configure SnowMirror successfully. It is a bit more complicated so we recommend not to configure it this way for evaluation purposes but as a step two. Follow a guide we have prepared for this scenario and use the Test Connection button on the Settings -> ServiceNow page which helps to identify missing permissions.
SnowMirror needs to access ServiceNow using an user account having access to web services API. We strongly recommend using admin rights for such a user. It is much more simple to configure SnowMirror this way. Remember, SnowMirror has to access not only the data but meta-data about tables and columns too. Normal user account usually do not have such a permission level anyway.
No. The Aggregate Web Services plug-in only adds one operation into direct web services. No existing operations are affected. None of our customers ever encountered an issue by activating the plug-in. However we do recommend to test SnowMirror and plug-in activation on non-production instances first.
No. The only mandatory requirement is to have a user account which has sufficient rights to access ServiceNow data and meta-data. However we recommend activating the Aggregate Web Services ServiceNow plug-in to improve the user experience, enable a few fancy features and a add a small performance boost.
Every sensitive information such as password is encrypted with AES-256 cipher. Encryption is applied on data stored in database as well as data stored in configuration files.
Yes. If you need high availability there is a possibility to install several SnowMirror nodes. Some of our customers even install one SnowMirror node into their DRC centers. SnowMirror architecture is cluster-ready and if one node goes down the other node is still synchronizing the data. We usually support these kind of requirements individually and we cooperate on the customer cluster design.
No. If you have several ServcieNow instances you have to install several SnowMirrors. The idea behind it is the same as for the MID servers.
Yes. And it is quite usual to run a mirror for a test ServiceNow instance and a development instance on one machine. It is necessary to configure a different ports and different Windows service name. Use Custom Installation if installing using the Windows installer. Similar approach applies to Unix systems.
SnowMirror in not resource intensive. It is a Java application running as a Windows service or a Unix daemon. It requires a server to be installed on. The server can be a small virtual or physical machine. The typical configuration is a dual core CPU server with 2 GB RAM and 5 GB HDD. We even recommend to install SnowMirror on the same machine as the ServiceNow MID server. You can save one server using this approach.
Read more about SnowMirror requirements in the admin guide:
Knowledge Base (8)
If your synchronization failed and you got an error similar to java.sql.BatchUpdateException: Data truncation, it can have two different root causes.
1) Something has changed on the ServiceNow side. Usually someone made a string field longer than before. For example your ServiceNow admin made incident short_description longer (max length) from 80 to 200. So it means the new incident subjects cannot fit into your varchar(80) in the mirror database. To fix this, either manually alter the mirror table (column) or re-create the whole synchronization and perform an initial data load.
2) ServiceNow does not always respect its own data constraints. E.g. try to synchronize the sys_perspective table and it fails by default because one column in this table is configured to have 40 chars as its max length however the longest string in the column has 125 characters by default. There is no other solution than to manually alter the mirror database table to reflect the real situation.
Auto Schema Update feature enables to modify the mirrored table automatically according to the changes on the ServiceNow side. If enabled all the table columns are being synchronized. If there is a new columns in ServcieNow, SnowMirror automatically adds it into the mirror table. The same applies for modified columns a removed ones.
ServiceNow enables to specify a parent for a table. It means all the columns in the parent table are used for the child table as well. For example the table task (parent) and incident (child). Many tables are derived from the task table. The whole CMDB is build on this inheritance design pattern. Etc.
SnowMirror enables two approaches when synchronizing inherited tables (e.g. incident)
1) Synchronize two tables. This is the way ServiceNow itself stores the data. You have to synchronize two tables (e.g. task and incident) and to obtain the incident record you have to join the tables using the sys_id primary key in both tables. The advantage of this approach is lower redundancy. If you are synchronizing several tables derived from the task table the situation is simpler to perform some type of queries.
2) Synchronize only one table with all parent inherited columns included. This is an approach that a typical user expects. To sync the data as it is displayed in the web application. Check the checkbox Include inherited columns in the synchronization settings to choose this approach.
To identify new or updated records in ServiceNow is a straightforward way. However to find the records deleted in a certain time window is not so simple. Most of the tables are audited so the default strategy search for deleted records in the system journals. You can choose different strategies if you need or if the table is not being audited. Truncate option means to perform Clean & Synchronize operation for every sync run (initial load every run). Diff is a good way to find the deleted records in a non-audited table and None ignores the deletes at all.
Please read more about the delete strategies in our user guide.
Please note that synchronization settings for Views do not contain the delete strategy choice at all. It always performs the Truncate option because there is no way how to achieve incremental updates.
SnowMirror does not access the database directly. ServiceNow does not allow neither customers not partners to access the underlying instance database. All the data is downloaded using the our-of-the-box API called Direct Web Services. Event the original ODBC driver uses this API which is the reason why the driver is so limited and slow.
SnowMirror distribution contains an optional ServiceNow dashboard application. If you have installed the update set into your ServiceNow instance enable this setting to start sending SnowMirror activity into the ServiceNow instance.
The Aggregate Web Services plug-in is a small plugin adding the aggregate method into all direct web services in ServiceNow. SnowMirror is using this new operation to count the records to insert or update before the actual download starts. By activating the plugin the progress bars work better, the count buttons enable you to count records in tables and SnowMirror is able to optimize the downloading algorithm a bit.
The main database is the mirror DB where the ServiceNow data is being stored. However SnowMirror requires to maintain its configuration in a second database. So called config DB. The config DB is configured during the installation phase and it is using H2 embedded database by default. The mirror DB can be configured in SnowMirror Settings page.
See a picture of the whole SnowMirror architecture.
No. You pay only for every SnowMirror connected to your production ServiceNow instance. Test, UAT, development or sandbox instances are for free.
I get this error while entering the license. Your ServiceNow is running at xxx.service-now.com but this license was issued for yyy.service-now.com only.
SnowMirror license is bound to a certain ServiceNow instance host name (e.g. yourcompany.service-now.com). So the ServiceNow host name you configure in Settings -> ServiceNow has to match the license host name. Please do not forget to restart SnowMirror after you change the instance in the Settings. In case of a trial license please feel free to generate yourself another trial license on our website if you need if for a different ServiceNow instance.