Introduction

In the latest version of RecordPoint Core we have introduced a new scaling architecture related to records submission from SharePoint that provides significant performance benefits to our customers and partners.

This document serves as a guide to assist system administrators in infrastructure planning and configuring their RecordPoint environment for maximum performance. It also assists Records Managers in planning and prioritising the processing requirements of their content sources.

Background

When a RecordPoint submission workflow is initiated from the Active site in SharePoint, an item is appended to a list called the RecordPointObjectQueue in that site collection for processing into RecordPoint. This list is better known as the Active site queue.

Historically the RecordPoint active queue service processes the queue items at a site collection level, with one Timer Job instance per RecordPoint Active web application. By default, this timer job was set to run every minute.

In RecordPoint Core version 3.8 and above we have changed the mechanism by which Active site items are processed from the Active site queue in SharePoint to provide a much more configurable and higher throughput solution.

The New Scaling Architecture

The previous architecture comprised of a single SharePoint Timer job per SharePoint Web Application to process all RecordPoint active site queues for submission into the RecordPoint Archive site. This was a multi-threaded process, but was limited in its ability to scale as it would run on just one server within a SharePoint farm.

We have changed this behaviour in version 3.8 to provide a single SharePoint Timer job for each site collection and its related Active site queue.

The new architecture provides the following benefits:

• ·     Easy scale out of Active site queue processing to multiple servers in the farm.
• ·     Provides customers with the ability to target specific queues for higher throughput by using the scheduling mechanism of the SharePoint timer jobs.
• ·     Enables much better use of resources in the farm.
• ·     Much higher overall throughput in the solution.

This architectural change is shown in the figures below:

In the previous architecture, RecordPoint scheduled each queue using custom logic and all code executed on a single server.	In the new architecture, the SharePoint Timer Service   executes a single timer job per queue and then automatically runs each of   these jobs on any SharePoint server in the farm.

Exploring the New Functionality

Following the deployment of RecordPoint Core 3.8 you will see that each site collection now has its own timer job called the RecordPoint Active Queue Timer Job- Site Name within the web application.

In versions prior to 3.8 there was a single RecordPoint Active Queue Timer Job in each web application.

As these are just SharePoint Timer Jobs, you are easily able to view and configure these jobs via Monitoring, Review Job Definitions in SharePoint Central Administration, as shown below.

Multiple RecordPoint active queue timer jobs for different sites running in a single web application.

With the new architecture it is also possible to change the scheduling information for each individual timer job and its related Active site. Manually configuring an individual timer job is achieved via the Central Admin web interface in SharePoint.

As you may have potentially hundreds or thousands of sites, we also provide a bulk editing tool for updating multiple timer job schedules in one operation.

For example, to meet your service requirements you may have a group of Active sites which you would like to process every two minutes and another group that you would like to process every ten minutes. For details see the section Tooling in this document.

Manually editing a RecordPoint active queue   timer Job in SharePoint.

Indicative Scaling

In this section we provide some indicative scaling scenarios based on some of the many different configurations that are possible with the new Active site queue architecture.

Each scenario demonstrates processing with varying numbers of Active sites (a.k.a. site collections in SharePoint). The sites are arranged into service groups with similar processing time requirements, based on business requirements.

Please note that these scenarios are indicative only. We encourage you to perform your own testing within your own environment and base your architecture on those observations and test results.

Scenario One – 500 sites, Low Processing Time Requirement

Scenario Two – 1000 sites, Medium Processing Requirement

Scenario Three – 2000 sites, Mixed Processing Requirement

Tooling

To assist the system administrator with the bulk configuration of the RecordPoint Active Queue Timer Jobs, we provide a set of PowerShell scripts that enable the export and import of the timer job configuration.

The relevant scripts are:

RecordPoint-Export-ActiveQueueTimerJobs.ps1

RecordPoint-Import-ActiveQueueTimerJobs.ps1

The scripts allow the system administrator to apply tiering of the processing schedules for large numbers of queues by providing for bulk export and import of the timer job configuration via CSV file.

Both scripts are included as part of the distribution of the product and be found within the Scripts directory.