ATG Endeca Data Integration Diagram - v3.1.x

Integration Overview 

The basic workflow diagram to show integration of the ATG product catalog with Endeca.

As seen in the diagram above, the data integration supports the publication of Product or SKU and Category assets directly into Endeca Content Acquisition System (CAS) Record Stores. The developer can control the type and granularity of assets published via the Product Exporter and the Category Exporter.

The Product Exporter behavior is managed by the Product Indexing Configuration file, which specifies the granularity and inclusion of product or SKU assets.

The Category Exporter is managed by the Category Indexing Configuration, which specifies the export of the Category to Dimension.

The Schema Exporter, managed by the Schema Indexing Configuration, exports schema information to Endeca.

Operationally, the publication of ATG assets and creation of the corresponding Endeca index is controlled by the ATG Web Commerce Product Catalog Indexing Administrator (Indexing Admin). The Indexing Admin invokes the Endeca Application Controller (EAC), which controls the creation and publication of the Endeca index throughout your production or staging environment.

The EAC, via configurable Endeca Deployment Template modules, controls the orchestration of the Endeca index creation process. Further, the EAC orchestrates the publication of the built Endeca index to all configured production or staging servers and ensures that the Endeca MDEX is running.

Indexing Using ATG Dynamo Administration UI  

Navigate to the ProductCatalogSimpleIndexingAdmin component to invoke the Endeca baseline Indexing process or partial update where localhost and 8080 are the host and HTTP port of your production ATG instance (or publishing instance, if you are using publishing).



More details on Indexing:
http://ravihonakamble.blogspot.com/2015/06/how-to-read-data-from-endeca-cas-record.html
Posted by 1 comment:
Labels:

Endeca Forge process completed but fails at Dgidx process - Endeca Tips

Problem Statement:
Forge process completed but fails at Dgidx process

 $ ./baseline_update.sh
 BASELINE SCRIPT STARTED
[07.12.15 19:37:32] INFO: Released lock 'update_lock'.
 Copy crawl files to incoming and processing folder
[07.12.15 19:37:33] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[07.12.15 19:37:34] INFO: Definition has not changed.
[07.12.15 19:37:34] INFO: Starting baseline update script.
[07.12.15 19:37:34] INFO: Acquired lock 'update_lock'.
[07.12.15 19:37:34] INFO: [ITLHost] Starting shell utility 'cleanDir_processing'.
[07.12.15 19:37:35] INFO: [ITLHost] Starting shell utility 'move_-_to_processing'.
[07.12.15 19:37:37] INFO: [ITLHost] Starting copy utility 'fetch_config_to_input_for_forge_Forge'.
[07.12.15 19:37:38] INFO: [ITLHost] Starting backup utility 'backup_log_dir_for_component_ConfigurationGeneratorForge'.
[07.12.15 19:37:39] INFO: [ITLHost] Starting component 'ConfigurationGeneratorForge'.
[07.12.15 19:38:08] INFO: [ITLHost] Starting backup utility 'backup_log_dir_for_component_Forge'.
[07.12.15 19:38:10] INFO: [ITLHost] Starting component 'Forge'.
[07.12.15 19:38:24] INFO: [ITLHost] Starting backup utility 'backup_log_dir_for_component_Dgidx'.
[07.12.15 19:38:25] INFO: [ITLHost] Starting component 'Dgidx'.
[07.12.15 19:38:27] SEVERE: Batch component  'Dgidx' failed. Refer to component logs in /opt/app/endeca/apps/APP_NAME/./logs/dgidxs/Dgidx on host ITLHost.
Occurred while executing line 26 of valid BeanShell script:[[
23|        Forge.archiveLogDir();
24|        Forge.run();
25|        Dgidx.archiveLogDir();
26|        Dgidx.run();
27|
28|        // distributed index, update Dgraphs
29|        DistributeIndexAndApply.run();]][07.12.15 19:38:27] SEVERE: Caught an exception while invoking method 'run' on object 'BaselineUpdate'. Releasing locks.Caused by java.lang.reflect.InvocationTargetException
sun.reflect.NativeMethodAccessorImpl invoke0 - null
Caused by com.endeca.soleng.eac.toolkit.exception.AppControlException
com.endeca.soleng.eac.toolkit.script.Script runBeanShellScript - Error executing valid BeanShell script.
Caused by com.endeca.soleng.eac.toolkit.exception.EacComponentControlException
com.endeca.soleng.eac.toolkit.component.BatchComponent run - Batch component  'Dgidx' failed. Refer to component logs in /opt/app/endeca/apps/APP_NAME/./logs/dgidxs/Dgidx on host ITLHost.
[07.12.15 19:38:27] INFO: Released lock 'update_lock'.

Solution:
Look at below log location
/opt/app/endeca/apps/APP_NAME/logs/dgidxs/Dgidx/Dgidx.log
FATAL   07/13/15 00:38:26.899 UTC (1436747906899)       DGIDX   {dgidx,baseline}        ENE Indexer: Error processing records file.
WARN    07/13/15 00:38:26.899 UTC (1436747906899)       DGIDX   {dgidx,baseline}        Lexer/OLT log: level=-1: 2015/07/12 19:38:26 | INFO    | Disabling log callback

If you see above FATAL ERROR in Dgidx log that means forge process completed successfully but without producing output for dgidx process to index and hence dgidx process failed.

Refer below blog for more details:
http://ravihonakamble.blogspot.com/2015/06/how-to-read-data-from-endeca-cas-record.html
Posted by 1 comment:
Labels: ,

How to update Endeca configurations using runcommand? - Endeca Tips

If you update AppConfig.xml or any file in /opt/app/endeca/apps/APP_NAME/config/script folder you need to update your configuration for your changes to be applied.

/opt/app/endeca/apps/APP_NAME/control$>./runcommand.sh --update-definition
[07.12.15 19:14:43] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[07.12.15 19:14:44] INFO: Updating provisioning for component 'Dgidx'.
[07.12.15 19:14:44] INFO: Updating definition for component 'Dgidx'.
[07.12.15 19:14:45] INFO: Definition updated. 
Posted by No comments:
Labels: ,

Endeca Baseline Update Script - WARNING: Failed to obtain lock - Endeca Tips

If baseline update process is killed while running then it puts lock and it will not get released. Any subsequent baseline update process will fail.

Here is the message when you try to run baseline update when lock is acquired:
$ ./baseline_update.sh
 BASELINE SCRIPT STARTED
 Copy crawl files to incoming and processing folder
[07.12.15 18:47:26] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[07.12.15 18:47:27] INFO: Definition has not changed.
[07.12.15 18:47:27] INFO: Starting baseline update script.
[07.12.15 18:47:27] WARNING: Failed to obtain lock.
It took 0 minutes and 3seconds

Solution:
1) User can release lock manually using below OOTB script
 /opt/app/endeca/apps/APP_NAME/control>$ ./release_update_locks.sh
 
2) Add release_update_locks.sh script as part of baseline process. Every baseline update will remove the lock acquired and then get the new lock.
Ex:
WORKING_DIR=`dirname ${0} 2>/dev/null`
. "${WORKING_DIR}/../config/script/set_environment.sh"
. "${WORKING_DIR}/release_update_locks.sh"
"${WORKING_DIR}/runcommand.sh" BaselineUpdate run 2>&1
 
Posted by No comments:
Labels:

Endeca Platform Services - Documentation

Posted by No comments:
Labels:

Endeca Tools and Frameworks - Documentation

Posted by No comments:
Labels:

Endeca MDEX Engine - Documentation

Posted by No comments:
Labels:

Endeca CAS (Content Acquisition System) - Documentation

Content Acquisition System 3.0.1
http://docs.oracle.com/cd/E35649_01/index.html
 
Content Acquisition System 3.0.2
http://docs.oracle.com/cd/E28909_02/index.html

Content Acquisition System 3.1.1
http://docs.oracle.com/cd/E38681_01/index.html

Content Acquisition System 3.1.2.1
http://docs.oracle.com/cd/E48637_01/index.html

Content Acquisition System 11.0
http://docs.oracle.com/cd/E51274_01/index.html

Content Acquisition System 11.1
https://docs.oracle.com/cd/E55325_01/index.html

Posted by No comments:
Labels:

How to Backup/Restore Endeca CAS state? - Endeca CAS

Backup CAS State:
1) Stop the Endeca CAS Service.
2) Copy state directory to outside CAS installation folder
     /opt/app/endeca/CAS/workspace/state
3) Restart CAS  

Restore CAS State:
 1) Stop the Endeca CAS Service.
2) Copy the backed up state directory to below location
      /opt/app/endeca/CAS/workspace/state
3) Restart CAS
Posted by No comments:
Labels:

How to Backup/Restore Endeca Dimension value Id's mapping? - Endeca CAS

You can back up dimension value Id mappings to a CSV file using the CAS Server Command-line Utility. 
To back up dimension value Id mappings:
/opt/app/endeca/CAS/11.1.0/bin>./cas-cmd.sh exportDimensionValueIdMappings -m APPNAME-dimension-value-id-manager -f dimvalIdMgr.csv 

 To restore dimension value Id mappings:
/opt/app/endeca/CAS/11.1.0/bin>./cas-cmd.sh importDimensionValueIdMappings -m APPNAME-dimension-value-id-manager -f dimvalIdMgr.csv

 You can find dimension value Id manager from below location:

 /opt/app/endeca/apps/APPNAME/control:>vi initialize_RS_services.sh
DVAL_ID_MGR_NAME=APPNAME-dimension-value-id-manager
Posted by No comments:
Labels: ,

Assembler API - Oracle Endeca

Oracle Endeca Commerce 3.1 includes the Endeca Assembler, a new query API and framework that replaces the Java and .NET Content Assembler from previous releases.
Benefits are :
  • Supports multi-channel natively by serializing responses to multiple formats (JSON, XML and Java objects)
  • Centralizes application business logic in one place using cartridge handlers 
  • Supports multiple deployment modes as an in-process Java library or as a RESTful service
  • Provides out-of-the-box, extensible core cartridges for querying the MDEX Engine
  • Provides extensible architecture for querying other source systems, allowing business users to assemble the full digital experience using Experience Manager
  • Consolidates run-time back end queries when possible
  • Includes URL optimization for SEO
  • Returns view-ready response models
Posted by No comments:
Labels:

Endeca Deployment Template Module for Product Catalog Integration - Usage Guide

Posted by 1 comment:
Labels: ,

Oracle Endeca Commerce - Compatibility Matrix

Posted by No comments:
Labels:

Endeca integration API libraries in ATG DAF module - ATG Endeca Integration

ATG 10.1.1 include the following Endeca libraries.
/ATG10.1.1/DAF/Endeca/Assembler/lib/endeca_assembler-3.1.0.jar
/ATG10.1.1/DAF/Endeca/Assembler/lib/endeca_navigation-6.3.0.jar
/ATG10.1.1/DAF/Endeca/Index/lib/custom

ATG 10.1.2 includes
/ATG10.1.2/DAF/Endeca/Assembler/lib/endeca_assembler-3.1.1.jar
/ATG10.1.2/DAF/Endeca/Assembler/lib/endeca_navigation-6.4.0.jar
/ATG10.1.2/DAF/Endeca/Index/lib/custom
/ATG/ATG10.1.2/DAF/Endeca/Index/lib/custom311

ATG 10.2 includes 
/ATG10.2/DAF/Endeca/Assembler/lib/endeca_assembler-3.1.2.jar
/ATG10.2/DAF/Endeca/Assembler/lib/endeca_navigation-6.4.1.jar
/ATG10.2/DAF/Endeca/Index/lib/custom311
Posted by No comments:
Labels: ,

Avoiding CXF log entries in JBOSS log file during Endeca indexing - Endeca Indexing Issues

Need help to get clean JBOSS log during indexing and here is the change request:
Add the following to jboss-as\server\server-name\conf\jboss-log4j.xml: 

<category name="org.apache.cxf.interceptor.LoggingInInterceptor">
<priority value="WARN"/>
</category>
<category name="org.apache.cxf.interceptor.LoggingOutInterceptor"><priority value="WARN"/>
</category>


Posted by No comments:
Labels:

How to compare Authoring & already promoted Live content? - Endeca Experience Manager

Currently there is no tool in Endeca to compare content between Authoring & Live instance.
Here is the workaround options:
Download content from Authoring Instance:  
http://localhost:8006/ifcr/sites/CRS.all.zip

Download content from already promoted Live Instance:
http://localhost:8006/ifcr/sites-live/CRS.all.zip 
 OR   
opt/app/endeca/MDEX/6.5.1/bin>export_config.sh host port (Live Dgraph host and port values ) 
Posted by No comments:
Labels:
Subscribe to: Posts (Atom)