Connector Configuration

In some scenarios, it can be useful to provide a system property or an environment variable within a configuration value. For example, you might want to distribute multiple connector instances over different containers and have certain configured parameters adapted according to specific system properties or environment variables, respectively. You can do this by providing a placeholder of the form ${my.system.property} within the value of some configuration option. The connector will resolve this property when reading the configuration, i.e., the placeholder will be replaced by the value of the corresponding system property / environment variable.

However, for security reasons this replacement is disabled by default. A malicious user could misuse this feature to obtain sensitive information about the connector environment such as the host’s operating system, the user under which the connector is running etc. You can mitigate this issue e.g. by restricting the access to the connector UI.

If you want to enable the resolution of system properties and environment variables, set the system property resolvePlaceholdersInPropertyValues to true. This can be done, for example, in the connector’s start script under the bin directory as follows.
In connector.bat (Windows):
set CONNECTOR_OPTS="-DresolvePlaceholdersInPropertyValues=true"
In connector (Linux/Unix):
CONNECTOR_OPTS='"-DresolvePlaceholdersInPropertyValues=true"'

Adobe Experience Manager Configuration

  1. Connection Config

  2. Content Traversal Config

  3. Page Content Retrieval Config

  4. Metadata Filter Config

Connection Config

Connection configuration for AEM server

Name Property Key Description

AEM URL

raytion.connector.agent.aem.connection.url

URL to Adobe AEM instance

Username

raytion.connector.agent.aem.connection.username

Username to login to Adobe AEM instance

Password

raytion.connector.agent.aem.connection.password

Password for login user

Workspace

raytion.connector.agent.aem.connection.workspace

Workspace inside the AEM to be crawled, e.g. "crx.default"

Content Traversal Config

Configuration options for content traversal

Name Property Key Description

Start Nodes

raytion.connector.agent.aem.content-traversal.startNodes

List of root nodes for content crawls

Type Filters

raytion.connector.agent.aem.content-traversal.typeFilters

List content types to exclude from indexing. Possible types are page, file and social. "file" filters both asset and file contents.

Content Max Size

raytion.connector.agent.aem.content-traversal.contentMaxSize

Maximum content size per item to crawl and send it to indexing

Click Url Prefix

raytion.connector.agent.aem.content-traversal.clickUrlPrefix

URL prefix to create clickable URLs

Sync Thumbnails

raytion.connector.agent.aem.content-traversal.syncThumbnails

Whether to index thumbnails

Page Content Retrieval Config

Configuration options for AEM page content retrieval

Name Property Key Description

Template Name

raytion.connector.agent.aem.page-content-retrieval.templateSelectors.template

Name of the template set in AEM site page

Selector

raytion.connector.agent.aem.page-content-retrieval.templateSelectors.selector

JQuery selector to get the content of the page

Default selector

raytion.connector.agent.aem.page-content-retrieval.defaultSelector

Default selector if no templates matched

Metadata Filter Config

Filter based on the metadata of an item. An item is indexed if it satisfies all filter rules

Name Property Key Description

Field Name

raytion.connector.agent.aem.metadata-filter.metadataFilters.field

Metadata field name

Filter Mode

raytion.connector.agent.aem.metadata-filter.metadataFilters.mode

Metadata filter mode - exclude or include

Field Value Patterns

raytion.connector.agent.aem.metadata-filter.metadataFilters.patterns

List of metadata field value patterns to consider to filter

Microsoft Azure Cognitive Search Configuration

Microsoft Azure Cognitive Search Instance Settings

Configuration Options to specify the service and index to feed documents to.

Name Property Key Description

Service Name

raytion.connector.backend.microsoft
.azure.cognitive.search.instance
.serviceName

Name of the target service in the Microsoft Azure Cognitive Search Portal.

Index ID

raytion.connector.backend.microsoft
.azure.cognitive.search.instance
.indexId

ID of the target index to feed to.

API Key

raytion.connector.backend.microsoft
.azure.cognitive.search.instance
.apiKey

Key for the API requests.

Microsoft Azure Cognitive Search Connection Settings

Configuration Options related to establish a connection and sending requests to Microsoft Azure Cognitive Search.

Name Property Key Description

Maximum Connections

raytion.connector.backend.microsoft
.azure.cognitive.search.connection
.maxConnections

Maximum number of open connections to the target instance.

Maximum Requests Per Second

raytion.connector.backend.microsoft
.azure.cognitive.search.connection
.maxRequestsPerSecond

Maximum number of requests per second to the target instance.

Request Timeout

raytion.connector.backend.microsoft
.azure.cognitive.search.connection
.requestTimeout

Timeout of requests to the target instance.

Socket Timeout

raytion.connector.backend.microsoft
.azure.cognitive.search.connection
.socketTimeout

Timeout of the socket connected to the target instance.

Use Proxy

raytion.connector.backend.microsoft
.azure.cognitive.search.connection
.useProxy

Use a proxy server to connect to Microsoft Azure Cognitive Search.

Microsoft Azure Cognitive Search Proxy Settings

Specify the proxy options for connecting to Microsoft Azure Cognitive Search.

Name Property Key Description

Endpoint

raytion.connector.backend.microsoft
.azure.cognitive.search.connection
.proxy.endpoint

The proxy endpoint including protocol, host and port.

General Configuration

Database Configuration

Name Property Key Description

Configuration Type

raytion.connector.db.config.type

Supported are PostgreSQL, MS SQL Server, and JDBC URL configuration.

PostgreSQL

Name Property Key Description

Host

raytion.connector.db.config.postgres.host

Domain name or IP address of the database server.

Port

raytion.connector.db.config.postgres.port

Specifies the port number PostgreSQL is listening on, default is 5432.

Database Name

raytion.connector.db.config.postgres.name

Name of the database.

Username

raytion.connector.db.config.postgres.username

Username to authenticate with. The regarding user has to have read and write permissions to the database.

Password

raytion.connector.db.config.postgres.password

Password of the configured database user.

Add Custom Parameter

raytion.connector.db.config.postgres.addParameters

Enables the configuration of additional parameters.

MS SQL Server

Name Property Key Description

Host

raytion.connector.db.config.mssql.host

Domain name or IP address of the database server. Instance to connect to on server can be specified by '‹server_name>|<instance_name>'.

Port

raytion.connector.db.config.mssql.port

Specifies the port number MS SQL Server is listening on, default is 1433.

Database Name

raytion.connector.db.config.mssql.name

Name of the database.

Username

raytion.connector.db.config.mssql.username

Username to authenticate with. The regarding user has to have read and write permissions to the database.

Password

raytion.connector.db.config.mssql.password

Password of the configured database user.

Add Custom Parameter

raytion.connector.db.config.mssql.addParameters

Enables the configuration of additional parameters.

JDBC URL

Name Property Key Description

URL

raytion.connector.db.config.jdbc.url

JDBC URL for the target database. Out of the box, the connector will use H2 file database. For productive usage, use PostgreSQL specifying the URL in format: jdbc:postgresql:<host>:<port>/<database>

Username

raytion.connector.db.config.jdbc.username

Database Username to read and write to database.

Password

raytion.connector.db.config.jdbc.password

Database Password for the specified user

Traversal Configuration

Name Property Key Description

Traversal History Length

raytion.connector.agent.traversal
.store.historyLength

Max. number of traversals to store in the history. Once the limit is exceeded, the connector will automatically remove oldest entries in the history. (default: 100)

Include Checksum

raytion.connector.agent.traversal
.pipeline.includePipelineChecksum

If enabled, any changes made to the pipeline e.g. configuration, the subsequent incremental run triggers a refeed of all items.

Change Processing Interval

raytion.connector.agent.traversal
.changeprocessing.interval

Interval between change processing traversals.

Resume on Start

raytion.connector.agent.traversal
.resume.resumeTraversalOnRestart

If enabled, any traversals in paused state are automatically resumed after the connector restart. Otherwise, the traversal remains in paused state.

Number of Traversal Workers

raytion.connector.agent.traversal
.workers.worker

Number of workers to execute the traversal in parallel. Increasing this value might improve the performance, but will footprint higher memory consumption. It is recommended to keep the default value. (default: 10)

Traversal Job Poll Interval

raytion.connector.agent.traversal
.workers.jobPollInterval

Interval between the workers to be triggered to fetch and process the next tasks. (default: 10ms)

Completion Timeout

raytion.connector.agent.traversal
.workers.completionTimeout

If the search engine indexes the items asynchronously, there might be some processing still in-flight during the completion process of a traversal. This value specifies the timeout value until all asynchronous callbacks are expected to return before completing the traversal. (default: 10m)

Executor Size

raytion.connector.agent.traversal
.execution.executorSize

The executor size restricts the max. number of concurrent running traversals.

Queue Size

raytion.connector.agent.traversal
.execution.queueSize

The queue size restricts the max. number of queued traversals. If the value is exceeded, the connector rejects further traversal requests until the queue size is below the configured size.

Traversal Jobs

Name Property Key Description

Job Timeout Check Frequency

raytion.connector.job-broker.heartbeatPeriod

Configures how often the connector checks for timed out jobs.

Job Timeout

raytion.connector.job-broker.heartbeatTimeout

The duration for which a job can stay idle before it is timed out.

Job Cache Size

raytion.connector.job-broker.jobPollCacheSize

Max. cache size of Jobs waiting for processing in memory. When cache is empty, next batch is fetched.

Security Configuration

Request Restriction Settings

Name Property Key Description

Accepted Host Names

raytion.connector.security
.requests.acceptedDomains

A list of domains (+ port) that are allowed as host names in the headers of HTTP requests to the connector. This means that you can access the connector only via a URL that employs one of the configured domains. Each entry must have the format domain:port. Examples:

  • localhost:16420

  • connector.company.com:16420

If no domains are configured (the default), then you can use any domain via which the connector host is reachable.

Principal Aliaser Configuration

Principal Aliasing is applied on user information as part of Content ACL processing during Content Synchronization and Principal processing during Principal Synchronization. It’s purpose is to map external source system user to the corresponding user in search engines domain. You can configure a list of aliasers in the connector which will be applied in sequence and in order on user ACEs and user principals. The Connector supports following custom aliasing mechanism.

Custom Aliaser Disabled

If the Custom Aliaser checkbox is not selected, the connector will process user information on ACE and user principals unchanged to Search Engine. If all relevant users in the source system can be found with the same identifier in the search engine, this setup is sufficient to reflect the same secure search experience in the search engine as defined by the policy in the source system. The connector uses this option as default to process user information.

Custom Aliaser Enabled

If custom aliasing is enable then there are four types of aliaser avaialble:

Simple XML Table Aliaser

Static mapping table which can be uploaded as XML file. The connector uses the uploaded file as lookup table to map a user in the source system to a user in the search engine. Users missing a record in the file will be dropped from the ACE and during Principal Synchronization. This option is only recommended for environment with a manageable amount of users as for each user the corresponding mapping entry needs to be specified in the file.

Name Description

XML Mapping File

Browse and upload or drag and drop.

Sample XML mapping file:

<?xml version="1.0" encoding="UTF-8"?>
<storeddata>
    <entry keyValue="user1">user1@raytion.com</entry>
    <entry keyValue="user2">user2@raytion.com</entry>
    <entry keyValue="user3">user3@raytion.com</entry>
</storeddata>
Regex Replacer Aliaser

Regex Replacer Aliaser computes aliases based on a regular expression. Principals that match the regular expression are replaced by the Substitution String.

Name Property Key Description

Pattern

raytion.connector.aliaser.aliasers[*]
.replacer.pattern

The regular expression to match, this is the part that will be replaced. If braces (…​) are used in the pattern then the matched value can be retrieved using $1

Substitute String

raytion.connector.aliaser.aliasers[*]
.replacer.substituteString

String to replace the matching part of the find string. Matched value is accessed by employing $1
Regex Extractor Aliaser

Regex Extractor Aliaser computes aliases based on a regular expression. Principals that match the regular expression are inserted into the Insert-Into String.

Name PropertyKey Description

Pattern

raytion.connector.aliaser.aliasers[*]
.extractor.pattern

The regular expression to match, this is the part that will be inserted into the new value. If braces (…​) are used in the pattern then the matched value can be retrieved using $$

Insert-Into String

raytion.connector.aliaser.aliasers[*]
.extractor.insertIntoString

String to replace the matching part of the pattern. Matched value is accessed by employing $$
LDAP Aliaser

Ldap Aliaser searches for an LDAP entry with the requested name in the input value and returns the specified output attribute.

Name Property Key Description

Host

raytion.connector.aliaser.aliasers[*]
.ldap.host

Fully Qualified Domain Name of an LDAP server

Port

raytion.connector.aliaser.aliasers[*]
.ldap.port

Port to use for LDAP connection, defaults are 389/636 or (recommended) 3268/3269 for simple/SSL

AccountDN

raytion.connector.aliaser.aliasers[*]
.ldap.bindAccountDN

AccountDN for bind to LDAP

Password

raytion.connector.aliaser.aliasers[*]
.ldap.password

Password part of credentials

Input Field

raytion.connector.aliaser.aliasers[*]
.ldap.inputField

The Active Directory attribute name for this equality filter

Search Root DN

raytion.connector.aliaser.aliasers[*]
.ldap.baseDN

Distinguished Name of the subtree which is searched. The smaller the subtree the better the performance but the higher the chance of encountering principals which are not part of this subtree

Output Field

raytion.connector.aliaser.aliasers[*]
.ldap.outputField

Attribute that should be returned in result entries