Scribe Insight is designed to support the
effective deployment of a number of market-leading business
applications, including Microsoft Dynamics CRM, Microsoft Dynamics GP,
Microsoft Dynamics NAV, Salesforce, and SalesLogix.
Scribe Insight is the core technology that forms
the basis for the migration and integration solutions using a unique and
open template model that enables companies to quickly and efficiently
configure any data integration or migration to meet their specific
needs, all without having to write a single line of code. (See Figure 1.)
Because each customer configures their business
applications differently and has different requirements for how the
integration itself will work, Scribe Insight is designed to provide
highly functional integration solutions that meet the specific needs of
each customer.
Scribe Insight has five major design points:
No programming required:
The tool has a graphical user interface that enables business or data
analysts (the people who know the issue best) to design and deploy
sophisticated integration solutions.
A single point of management: Companies can support and maintain the integration solution after it has been deployed.
A consistent adapter model:
The core Insight design environment views all applications in the same
way, while presenting to the designer information about each application
that is important to the integration task.
Open connectivity:
Lets companies integrate these core business systems with the wide
variety of applications and data stores that are unique to their
business.
A template model:
Users can quickly assemble reusable integration components and
configure them for each deployment’s unique needs. After the initial
deployment, required changes in the integration can be accommodated with
a simple reconfiguration.
The core components of Scribe Insight are built
using the Microsoft Visual Studio development platform for the Windows
family of operating systems. The Scribe Server is the core engine that
provides connectivity to the various applications, databases, and
messaging systems within the integration environment. Communications
between the Scribe components and the applications being integrated is
provided using the appropriate technology. For example, Scribe adapters
to those applications that support web services, such as Salesforce or
Microsoft Dynamics CRM, use Simple Object Access Protocol (SOAP),
whereas other on-premise APIs are worked with using
Component/Distributed Component Object Model (COM\DCOM).
Figure 2 shows the topology of Scribe Insight components. These items represent the five major Scribe Insight components:
Scribe Server
Scribe Workbench
Scribe Console
Adapters
Templates
Scribe Insight is based on a loosely coupled, yet
tightly integrated architecture that is highly adaptable to each
customer’s unique and constantly changing business environment. For
example, each adapter communicates to the Scribe Server in precisely the
same way regardless of the application or database to which it is
connecting. This abstraction of the application or database details
provides for a highly productive design environment; once users learn to
use the Workbench, they can design integrations with a wide variety of
applications and data stores. This abstraction also means that templates
(representing specific
integration processes between applications or databases) are insulated
from most changes/updates to the application or database interface. The
same template that works with version x of an application will continue to work with version y, requiring no reconfiguration except to accommodate substantive changes in the schema or functionality of that application.
The Scribe Server
The Scribe Server is the core of Scribe
Insight–supported integration processes and facilitates the exchange of
data between two or more applications or databases. Because Scribe
Insight, in essence, brokers a conversation between these applications
and databases, it can support a highly heterogeneous server environment
of Windows, UNIX, Linux, on-demand applications, and so on. All that it
requires is a “connection” to these applications via a Windows client, a
non-platform-specific middleware protocol such as Open Database
Connectivity (ODBC), or via a Microsoft Message Queuing (MSMQ) message
queue.
Underlying the Scribe Server are a number of
Windows services designed to monitor and detect events, process
messages, raise alerts, and provide an access point for the Scribe
Console to the other services. The Scribe Server also includes its own
internal database that stores all execution and error logging, persisted
integration settings, cross-reference tables, and important integration
statistics. The Scribe internal database can be configured to support
the Microsoft SQL Server Express database (provided with Scribe Insight)
or the other Microsoft SQL Server editions.
The Scribe Workbench
The Scribe Workbench provides a rich graphical
environment where users design and configure the data mappings and
business rules that define the integration solution. All work completed
in the Scribe Workbench is “saved” in a lightweight file that is
referenced by the Scribe Server at runtime. This self-documenting,
metadata-driven model allows for easy debugging during the deployment
phase and rapid modification as the application environment or business
needs change.
The Scribe Workbench enables you to connect to
your applications, define a source result set, configure object-level
target processing, and then just point and click to modify or add data
mappings.
One of the key capabilities in the Scribe
Workbench is the ability to “normalize” source data on-the-fly as it
processes against the target application. In other words, single- or
multi-row “source” data can have multiple operations executed per row on
“target” data objects. These operations, referred to as steps,
can be conditionally executed based on user-defined logic, allowing
complex, transaction-enabled, multiple-object operations. (See Figure 3)
With the Scribe Workbench, designing complex data
transformations is a simple task. The Scribe Workbench provides more
than 150 Microsoft Excel-like functions for data translation, including
the following:
Parsing functions for names and addresses
Date and time conversions
String manipulation
Database and file lookups for processing synonym values
Logical if/then/else functions to support conditional processing
In the rare case where there is a need for data
transformation beyond what is included in the Scribe Workbench,
additional functions may be created using COM and simply added to the
function list.
The Scribe Workbench was designed to support many
advanced integration needs beyond data transformation and mapping and
includes these additional capabilities:
A Test window that shows the results of
processing test data without committing the data to the target system.
Users can view the results of data translations and object-level
transaction processing for easy and efficient debugging of integration
processes.
Built-in system key cross
reference and management, designed to dynamically maintain data
integrity between records across two or more loosely coupled
applications.
Built-in support for
foreign key value reference management, designed to dynamically maintain
data integrity between related records within an application.
Net
change tracking by updating or deleting successfully processed source
records or by comparing a source-side update stamp against a variable
last run date/time in the source query.
Conflict detection and resolution to support bidirectional data synchronization.
Formula-based lookups for “fuzzy” record matching logic.
Value cross-reference and lookup support.
Automatic data type mismatch resolution.
Transactional support for Header-Detail type data sets.
Configuration of target-side commit and rollback.
Rich error handling and logical flow control, including support for user-defined errors.
Rejected row logging to support automated repair and recovery processes.
The Scribe Console
The Scribe Console is a Microsoft Management
Console snap-in that provides the user interface to an array of powerful
features used to set up, organize, and manage key aspects of any number
of integration processes. The Scribe Console is the main user interface
to the capabilities underlying the Scribe Server.
The Scribe Console can be installed independently
from the Scribe Workbench, and can be configured to connect to the
Scribe Server using either COM/DCOM technology over a LAN or a provided
SOAP-based web service connection hosted by Microsoft Internet
Information Services (IIS).
The Scribe Console (shown in Figure 4)
provides a single point of management for a company’s various
integration points, organizing them as discrete units of work or
collaborations. Each collaboration is a series of related integration
processes and instructions for how and when these processes should be
automatically executed. Collaborations are organized in a graphical,
user-defined tree and can be managed as independent objects with their
own reporting, monitoring, and diagnostic functions. The Scribe Console
also provides easy access and control of all integration processes
running on the system, through controls implemented at the integration
server level.
At
the core of the Scribe Console are its sophisticated event management
capabilities. The Scribe Console allows each company to precisely define
the proper latency for each integration process, from scheduling batch
processes to run on a predefined time period, to establishing near
real-time polling intervals based on a file drop in a directory or the
results of a source-side query, to the real-time processing of messages
arriving in an in queue.
The Scribe Server is built with a modular,
multithreaded architecture that allows for scaling of integration
processes based on the available CPU processing strength. It also
features efficient connection sharing to maximize performance, where
possible.
Additional capabilities of the Scribe Console include the following:
Access to the files on the Scribe Server that may need to be moved, copied, renamed, or deleted.
Automated
system monitoring of business-level events or integration errors with
configurable alerts via email, page, and net send notification.
For
those data sources that do not have a built-in net change mechanism
(including event-based message publishing, time and date stamps for
updates, or other forms of update stamps), the Console provides a Query
Publisher that compares time-based “snapshots” of a source system and
publishes the differences as an XML message.
Settings
to launch an executable file to run before or after an integration
process. One example where this pre- or post-execution processing can be
useful is the ability to move files into an archive directory after the
process is executed.
Onscreen editable views of predefined queries that can be displayed in chart or list format.
User interface for MSMQ management providing message viewing, moving, copying, and deleting.
Review execution history of what processes succeeded or failed, including detailed error reporting.
Scribe Adapters
Scribe adapters enable Scribe Insight to communicate seamlessly to a variety of applications, databases, messages, and files.
Scribe adapters present rich levels of schema
information, which is available through the application interface (via a
“declare” type function in an application API or in a Web Service
Definition Language [WSDL] in the case of web services interfaces) to
the Scribe Server and Scribe Workbench. This schema information includes
object properties and interrelationships and detailed field attributes
such as labels, data types, lengths, restrictions, and default/pick-list
values. Combined with the rich features in the Scribe Workbench, this
information provides for unparalleled control over integration processes
and eliminates the “last-mile” coding required with other integration
tools.
Scribe adapters are classified in two ways: enterprise application adapters and connectivity adapters.
Enterprise Application Adapters
Enterprise application adapters are adapters
that have been designed and developed to optimize Insight for use with
Scribe-targeted CRM and ERP applications, including Microsoft Dynamics
CRM, Microsoft Dynamics GP, Microsoft Dynamics NAV, Microsoft Dynamics
AX, Salesforce, and SalesLogix. Scribe’s enterprise application adapters
are sold as add-ons to Scribe Insight. Key features of these
application adapters include the following:
The automation of common data loading
tasks such as assigning primary ID values, setting default values,
validating input data, and setting object relationships—all designed to
eliminate runtime errors and provide for greater data integrity.
Dynamic
discovery that presents the unique configuration of each application or
database instance to the Scribe Console and Scribe Workbench at runtime
and adjusts to changes in the application or database schema without
requiring recoding or recompiling.
The
seamless integration of application and database error messages to
provide detailed exception reporting and handling from the Scribe
Console’s single point of management.
Connectivity Adapters
Connectivity adapters are included in core
Scribe Insight product. They are designed to complement the enterprise
application adapters by providing a wide variety of integration options
to support connectivity to the varied applications and data stores
within each company’s computing environment. These connectivity adapters
enable Scribe Insight to communicate with applications and databases in
the following ways:
Direct communication with database
tables, views, and stored procedures through ODBC 3.0 or later and
natively to SQL Server. Scribe leverages all the filtering and querying
capability of these databases when employing this approach.
The exchange of flat files or XML documents via a directory or FTP/HTTP location.
The asynchronous exchange of XML messages via an industry-standard message queue, email, or integration broker.
SOAP Messages via web-based transport protocols, such as http/https.
A common use of Scribe’s connectivity adapters
is to support integration between the targeted applications served by
Scribe’s application adapters and a wide variety of other packaged
enterprise applications. These other enterprise applications include the
following:
ERP and CRM systems from SAP, Siebel, Oracle (Oracle, PeopleSoft, JD Edwards,) Sage (MAS 90/200/500,) Epicor, and so on
Packaged applications that serve a particular niche or vertical market
Custom in-house-developed systems
Scribe
provides a number of approaches to integrating with these applications,
depending on the business requirements and available technical
resources, including the following:
Directly to the database:
This is a simple, straightforward approach if you are migrating from an
application or your project is limited to a one-way feed of data from
that application. Scribe Insight provides a number of methods using this
approach to extract “net change” data from the application.
Via interface tables:
Many applications support a set of interface or staging tables that
provide for a safe way to integrate data into that application. After
data is passed into the interface tables, an application process is
initiated that validates the data and applies appropriate application
rules. With Scribe Insight, you can write to these tables and initiate
the application process automatically.
Via an XML/messaging interface:
Many enterprise applications provide an XML interface that is
incorporated into the workflow engine within the application. Using this
method, Scribe Insight can publish XML messages into a message queue
for real-time integration with the other application. Scribe Insight can
also receive XML transactions published by the application’s workflow
engine into a message queue in real time.
Via the application’s API:
Many applications expose a web services- or COM-based API where
transactions can be passed to the application. Data can also be queried
via this API. Out of the box, Scribe Insight cannot “natively” integrate
with this API; however, custom code can be written to convert these
calls into an intermediate format. This intermediate format can be an
XML message, a flat file, or a record in a database staging table.
Scribe Insight also includes connectivity
adapters for data migration from/to certain leading desktop
applications, including ACT!, GoldMine, and Microsoft Outlook/Exchange.
Scribe Templates
Scribe templates represent complete or partial
data integration or migration processes that have been developed using
Scribe Insight technology. Scribe provides a number of these templates
as free downloads from the Scribe Web Community to support the
successful deployment of Scribe Insight.
Templates consist of the building blocks of a
fully functional migration or integration solution as configured with
Scribe Insight, including the following:
Source-side “net change” processes and filtering
Event and process automation
Data mappings
Record matching for updates and duplicate avoidance
User/owner mappings
Field ownership and update rules
System key cross referencing and management
Connection validation and security
Data ownership and customizations
Application customizations
Transaction management
Commit and rollback settings
System monitors and alerts
Business monitors and alerts
There are two distinct styles of Scribe templates: solution templates and component templates.
Solution Templates
Solution templates represent a complete, fully
functional integration or migration solution between two applications.
Examples of these include migration solutions for ACT! into Microsoft
Dynamics CRM, SalesLogix, or Salesforce and “front to back-office”
integration solutions between Microsoft Dynamics GP and Microsoft
Dynamics CRM or Salesforce. Scribe’s unique template model provides
out-of-the-box functionality for these integration scenarios, built over
its industry-leading integration tool. Because most customers have
business needs unique to them, these standard templates can be quickly
extended and customized via the GUI-based mapping and development
environment.
The component architecture of these solution
templates also enables customers to implement templates in phases or
pick and choose the elements of the templates that they require. In the
front to back-office integration example, a customer might not want to
implement order integration initially (or in some cases never) but can
still synchronize customer activity (accounts, contacts, invoices)
between the two systems. This modularity enables customers to implement
an integration solution tailored to their exact needs.
Component Templates
Component templates are starting points for
common integration processes used by customers that are implementing an
integration solution for which Scribe has not developed a solution
template.
For example, Scribe provides templates
that integrate customers, products, orders, and invoices between a
Scribe-developed sample ERP system and Scribe’s targeted CRM
applications, including Microsoft Dynamics CRM, Salesforce, and
SalesLogix. A customer that is looking to integrate one of these
applications with their own ERP application can use the appropriate
component template as a significant starting point. Typically, these
component templates provide the bulk of the end solution, with the
remainder easily configurable with Scribe Insight.
