FormKiQ Enterprise Reference Guide

1. Introduction

FormKiQ Enterprise

FormKiQ Enterprise is made up of Enterprise Add-Ons that are available for FormKiQ Core, bringing enterprise-level features and functionality that may be required for some organizations.

NOTE: this is a preview of our in-progress documentation of these modules, as opposed to the full specification.

FormKiQ Enterprise Add-On Modules include:

✅ Document Tag Schemas Module

✅ Custom Domains Module

✅ Antivirus Protection Module

✅ Document Email Processing Module

✅ Document OCR Module

✅ Fulltext Search Module

✅ Network Share and Sync Module

✅ Intelligent Document Processing Module

✅ Full Encryption Module

✅ Document Archive and Lifecycle Module

✅ Single Sign-On and Custom JWT Authorizer Module

To learn more about FormKiQ Core, please visit https://github.com/formkiq/formkiq-core.

To learn more about FormKiQ Enterprise, please visit https://formkiq.com.

2. Tag Schema Module

✅ Enforce Required and Optional Tag Keys/Values

✅ Extend Search Functionality through Composite Keys for Tags

✅ Optionally Return Tags while using the POST /search API Endpoint

2.1. Overview

Tag Schema is a FormKiQ Enterprise Add-On Module (for the FormKiQ Core Headless Document Management System) that provides the enforcement of a tag schema on specific documents and enhances search functionality through composite keys for tags.

Use Cases:

  1. If you have multiple input methods, such as having more than one application using the API, you can ensure that your documents are properly and consistently tagged no matter the source

  2. If you expect your application(s) to search often using two tag keys, e.g., customerID and projectID, you can automate the creation and maintenance of composite keys to search in one request

With the Tag Schema Add-On Module, you can create different sets of required or optional keys, i.e., the Tag Schemas, and each schema can be associated to any number of documents. You can also specify both a default value and a list of allowed values for both required and optional keys.

Each tag schema can have up to 5 required and 10 optional keys, for a total of 15; you should consider prioritizing tags used for searches in cases where your document content may have more than five required key/value pairs, vs. attempting to store all document key/value pairs as tags. In cases where you expect to search for more than 15 keys, we recommend our Fulltext Search Module.

A tag schema can specify any of the following options as its rules:

  • only documents with the required tag keys can be accepted, with no optional keys allowed

  • only documents with the required tag keys can be accepted, with only the specified optional keys allowed

  • only documents with the required tag keys can be accepted, with any other keys being optional

  • documents have no required tag keys, but only the specified optional keys allowed

  • documents have no required or optional tag keys specified, and the schema is only used to create composite keys for searches (see below)

When a document is added with an associated tag schema, the document will be rejected if it does not conform to the rules of the schema.

When searching for documents, can you request that the tags in the schema are returned by using an optional boolean parameter, includeTagsFromSchema. Only documents with tag schemas will return their tags directly within search results. If a tag allows any number of optional tags (i.e., if allowAdditionalTags is not disabled in the schema), this optional parameter will only return the required tags. This is done to maintain a reasonable response size for these requests.

A tag schema also allows the creation of composite keys, i.e, compositeKeys, which allow for the searching of 2-3 tags at once. This allows FormKiQ to maintain its high scalability while remaining cost efficient.

There is no requirement that these composite keys be made up of required or specified optional tags, so you will want to ensure that you keep this in mind when creating any composite keys. Up to five separate composite keys can be created for a specific tag schema, while each composite key combines two or three tag keys. For more complex searches, we recommend our Fulltext Search Module.

NOTE: when using composite keys involving three tag keys, searching for these keys requires that all three tag keys are specified; if you require an additional search for only two of the three keys, you would need to create a separate composite key consisting of only those two tag keys.

2.2. API

The Tag Schema Module adds the following API endpoints:

  • GET /tagSchemas - Returns a list of TagSchemas

  • GET /tagSchemas/{TagSchemaId} - Returns a specific TagSchema

  • POST /tagSchemas - Adds a new TagSchema

NOTE: all FormKiQ API endpoints are case-sensitive

As this module interacts with the FormKiQ Core API Endpoints, a tagSchemaId will also be available on most document endpoints, such as POST /documents/.

2.2.1. POST /tagSchemas Example

The following POST /tagSchemas body creates two required fields, playerId and caseId. There is also an optional tag of category.

When the document is saved, a compositeKey is created automatically that will allow for a search using both playerId and caseId as search parameters.

{
  "schema": {
    "tags": {
      "compositeKeys": [
        {
          "key": [
            "playerId",
            "caseId"
          ]
        }
      ],
      "required": [
        {
          "key": "playerId"
        },
        {
          "key": "caseId"
        }
      ],
      "optional": [
        {
          "key": "category",
          "allowedValues": [
            "person",
            "city"
          ]
        }
      ],
      "allowAdditionalTags": false,
    }
  }
}

NOTE: as allowAdditionalTags is set to false in this example, any attempt to add additional tags with keys other than playerId, caseId, or category will cause the request to fail. As well, any category tag which has a value other than the ones specified, i.e., not person or city, will also be rejected. By default, this value is true, i.e., additional tags are allowed, if not specified in the schema.

2.2.2. Specifying Default Values

Default values can be set for both required and optional tags, and can include one or more values as the default (up to the maximum of five values per tag key, consistent with the multi-value tag specification in FormKiQ Core):

{
  "schema": {
    "tags": {
      "required": [
        {
          "key": "planId",
          "default": 1,
        },
        {
          "key": "userId"
        },
        {
          "key": "applicationId"
        }
      ],
      "optional": [
        {
          "key": "components",
          "allowedValues": [
            "idCheck",
            "creditCheck",
            "criticalIllnessInfo"
          ],
          "default": [
            "idCheck",
            "creditCheck"
          ]
        }
      ]
    }
  }
}

3. Custom Domains Module

✅ Customize your FormKiQ API and Console URLs with your domain

3.1. Overview

Custom Domains is a FormKiQ Enterprise Add-On Module (for the FormKiQ Core Headless Document Management System) that enables custom domains support for FormKiQ console, and API endpoints.

FormKiQ out of the box uses the AWS default domains for the Amazon CloudFront (XXXXXXX.cloudfront.net.) and Amazon API Gateway (https://XXXXXXX.execute-api.region.amazonaws.com)

The module uses Amazon Route 53 and AWS Certificate Manager to create domains and auto-renewing SSL certificates. Then adds the certificates and configures the domains to API Gateway and the FormKiQ console.

Use Cases:

  1. If you would like to share the FormKiQ API or FormKiQ Console internally to users, and would prefer that one of your domains or subdomains be used instead of the less-readable CloudFront and API Gateway domains

  2. If you are exposing any FormKiQ URLs externally and would prefer that external users remain unaware that you use AWS for your document management

4. Antivirus Module

✅ Automatically scan every newly added document

✅ Automatically tag document with scan status and timestamp

4.1. Overview

Antivirus Architecture

Antivirus is a module for the FormKiQ Headless Document Management System that provides document security by detecting trojans, viruses, malware & other malicious threats for every newly created document, using the ClamAV antivirus engine.

Using FormKiQ’s Document Events, each newly-created document is automatically added to a ClamAV Scanning SQS queue. An AWS Lambda function for ClamAV virus scanning pulls each document from the SQS queue and scans it for trojans, viruses, malware, and other malicious threats. The results from the scan are added as S3 Tags to the document, and are also attached to the document in DynamoDB. In addition, a message is posted to an ClamAV Scan Events SNS topic which allows other systems to be notified of the document’s scan status.

Use Cases:

  1. If you have received certain compliance goals that require antivirus scanning of all documents used by the organization, this module can meet those goals without any additional effort

  2. If you receive documents from external sources, whether through the FormKiQ API’s public endpoints or optional Enterprise Add-On Modules such as Document Email Processing, antivirus scanning will help to safeguard both your documents within FormKiQ and any documents you export from FormKiQ

5. Document OCR Module

✅ Use Amazon Textract to extract text and structured table or form data from documents

5.1. Overview

Document OCR is a FormKiQ Enterprise Add-On Module (for the FormKiQ Core Headless Document Management System) that provides optical character recognition (OCR) for the extraction of text and structured table or form data from documents.

The module uses Amazon Textract as its OCR engine. Amazon Textract uses advanced machine learning (ML) algorithms to identify, understand, and extract text and data from documents, including from forms and tables.

Use Case:

  1. If you would like to add metadata for document retrieval, this module can provide that metadata from scanned documents; you can then create tags (in the case of form or table data) or enable the entire OCR result to be searched using the Fulltext Search module

5.2. API

The Document OCR Module adds the following API endpoints:

NOTE: all FormKiQ API endpoints are case-sensitive

5.2.1. GET /documents/{documentId}/ocr

Description

Get Document optical character recognition (OCR) result; if exists. ONLY available with FormKiQ Enterprise

Parameters
Path Parameters
Name Description Required Default Pattern

documentId

Document Identifier

X

null
Query Parameters
Name Description Required Default Pattern

siteId

Site Identifier

-

null
Return Type

DocumentOcr

Content Type

  • application/json

Responses
Table 1. http response codes
Code Message Datatype

200

200 OK

DocumentOcr

5.2.2. POST /documents/{documentId}/ocr

Description

Document optical character recognition (OCR) request; extract text and data from a document. ONLY available with FormKiQ Enterprise

Parameters
Path Parameters
Name Description Required Default Pattern

documentId

Document Identifier

X

null
Body Parameter
Name Description Required Default Pattern

OcrBody

OcrBody

-
Query Parameters
Name Description Required Default Pattern

siteId

Site Identifier

-

null
Return Type

DocumentOcrResult

Content Type

  • application/json

Responses
Table 2. http response codes
Code Message Datatype

200

200 OK

DocumentOcrResult

5.2.3. DocumentOcr

Field Name Required Type Description Format

data

String

OCR text result

ocrEngine

String

The OCR technique used

ocrStatus

String

The status of the OCR request

contentType

String

Document Content-Type

isBase64

Boolean

Is Content Base64 encoded

userId

String

User who requested the OCR

documentId

UUID

Document Identifier

uuid

5.2.4. DocumentOcrResult

Field Name Required Type Description Format

message

String

OCR processing message

5.2.5. OcrBody

Field Name Required Type Description Format

parseTypes

List of [string]

OCR Parse types - TEXT, FORMS, TABLES

6. Full Encryption Module

Full Encryption

Full Encryption is a FormKiQ Enterprise Add-On Module (for the FormKiQ Core Headless Document Management System) that enables both in-transit and at-rest encryption for the AWS services used by FormKiQ.

The module uses the AWS Key Management Service (AWS KMS) service to create and manage cryptographic keys and control the use of the keys. AWS KMS integrates with all of the services used by FormKiQ:

As an option, FormKiQ provides AWS KMS integration with AWS CloudTrail, which delivers log files to your designated Amazon S3 bucket. By using CloudTrail, you can monitor and investigate how and when your KMS keys have been used and by which service or user. This integration if turned off by default, to avoid unexpected CloudTrail costs.

Use Case:

  1. If you have received certain compliance goals that require encryption of all documents used by the organization, this module can meet those goals without any additional effort

7. Single Sign-On and Custom JWT Authorizer Module

SAML Architecture

Single Sign-On and Custom JWT Authorizer is a FormKiQ Enterprise Add-On Module (for the FormKiQ Core Headless Document Management System) that enables the ability to customize the authorizer used for FormKiQ console, and API endpoints.

FormKiQ by default uses Amazon Cognito as the JWT authorizer.

Using this module the authorizer can be changed to another authorizer such as:

  • Another JWT authorizer such as Auth0

  • SAML identity providers through Security Assertion Markup Language 2.0 (SAML 2.0)

  • Public providers: FaceBook, Google, or Apple

  • OpenID Connect provider

Use Case:

  1. If you would like to maintain your ActiveDirectory or Google Apps login from your organization into the FormKiQ API or Console, this module allows that with minimal configuration

7.1. Integration Instructions - Microsoft Active Directory Federation Services

7.1.1. Retrieve Your Amazon Cognito Info

In order to configure your Windows Server AD FS, you will need to retrieve your FormKiQ Pool ID and Amazon Cognito Domain.

Go to Amazon Cognito in your AWS Management Console. Under "Settings", you can retrieve your User Pool ID:

Cognito User Pool Settings

You can then retrieve your Cognito Domain under "App Integration >> Domain Name":

Cognito User Pool Domain

7.1.2. Open AD FS Management in Windows Server Manager

NOTE: Ensure that

  • AD FS server(s) has https network access to amazoncognito.com

  • When using multiple AD FS servers, apply the following changes to all servers

Go to "Server Manager" and select "AD FS":

Windows Server Manager Dashboard

Choose the "Tools" menu, and select "AD FS Management":

Windows Server Manager - AD FS Management

7.1.3. Adding Relying Party Trust Management

Select "Relying Party Trusts" and choose the "Add Relying Party Trust Wizard". Choose the default "Claims aware" option:

Claims Aware Relying Party Trust

For "Select Data Source", choose the option "Enter data about the relying party manually":

Choosing the manual data source option

For "Specify Display Name", enter a descriptive name, such as "FormKiQ ADFS Login":

<em>Choosing a descriptive name</em>

For "Configure URL", choose "Enable support for the SAML 2.0 WebSSO protocol", and for the form field below, replace "yourDomainPrefix" with your Amazon Cognito User Pool’s Domain Prefix (retrieved as the first step), and replace region with the User Pool’s AWS Region (for example, "us-east-1"):

Configuring the URL

For "Configure Identifiers", you need to supply the "Relying party trust identifier". Enter "urn:amazon:cognito:sp:*yourUserPoolID*" as the URN, but with your Cognito User Pool (e.g., "us-east-1_g2zxiEbac"), instead of "yourUserPoolId":

Configuring the Trust Identifier

You can now click through the default options for the remainder of the Wizard and click "Finish".

7.1.4. Editing the Relying Party Trust Claim Issuance Policy

You should now see your new Relying Party Trust listed with the Display Name that you provided. You can select this trust:

Selecting Your Relying Party Trust

Choose "Edit Claim Issuance Policy":

Editing Your Claim Issuance Policy

You can now add a new Claim Issuance Policy Rule for the Windows account name:

Adding a new Claim Issuance Policy Rule for Windows account name

Next, create a rule for the email:

Adding a new Claim Issuance Policy Rule for Email

7.1.5. Signing in with AD FS

You can test out your AD sign in with the FormKiQ Console:

FormKiQ Console External Provider Login

You will then be redirected to an Active Directory server and prompted to choose your Corporate ID:

FormKiQ Console Corporate ID

You can then securely sign in with your Active Directory credentials:

Signing in Securely with AD

7.1.6. User Group Access

FormKiQ access is determined by the group(s) the user belongs to, see Multi-Tenant

The only difference is all user groups need to be prefixed with formkiq_.

At a minimum you need to create a group called formkiq_default which will give users read / write access to the default siteId.

8. Appendix

Please check out the website for further links, and to contact.

https://formkiq.com

https://github.com/formkiq/formkiq-core