Oracle Commerce- Think Out of the box!!!

JBOSS console hangs while doing Endeca baseline Index from dynamo admin - ATG Endeca Integration

Problem Statement:
Getting below error message in JBOSS log and console hangs.

15:51:29,881 INFO [RecordStore] Outbound Message
---------------------------
ID: 1
Address: http://localhost:8500/CRS_en_schema/
Encoding: UTF-8
Content-Type: text/xml
Headers: {Accept=[application/fastinfoset, */*], Accept-Encoding=[gzip;q=1.0, identity; q=0.5, *;q=0], SOAPAction=[""]}
Payload: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><startTransaction xmlns="http://recordstore.itl.endeca.com/" xmlns:ns2="http://record.itl.endeca.com/"><tra
--------------------------------------
15:51:32,053 INFO [ReflectionServiceFactoryBean] Creating Service {http://recordstore.itl.endeca.com/}RecordStoreService from class com.endeca.itl.recordstore.RecordStore
15:51:32,649 INFO [DimensionDocumentSubmitter] Will attempt to contact record store "CRS_en_dimvals" on host "localhost" on port 8,500
15:51:32,655 INFO [RecordStore] Outbound Message
---------------------------
ID: 2
Address: http://localhost:8500/CRS_en_dimvals/
Encoding: UTF-8
Content-Type: text/xml
Headers: {Accept=[application/fastinfoset, */*], Accept-Encoding=[gzip;q=1.0, identity; q=0.5, *;q=0], SOAPAction=[""]}
Payload: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><startTransaction xmlns="http://recordstore.itl.endeca.com/" xmlns:ns2="http://record.itl.endeca.com/"><tra
--------------------------------------
15:51:36,588 INFO [RecordStore] Inbound Message
----------------------------
ID: 1
Response-Code: 200
Encoding: ISO-8859-1
Content-Type: application/fastinfoset
Headers: {Content-Length=[195], content-type=[application/fastinfoset], Date=[Mon, 27 Jul 2015 20:51:31 GMT], Server=[Jetty(6.1.26)]}
Payload: α ☺ 8╧♥soap(http://schemas.xmlsoap.org/soap/envelope/≡???Envelope???♥Body8╧☻ns2∟http://record.itl.endeca.com/═!http://recordstore.itl.endeca.com/≡=?↨startTransactionResponse=?♣return=?☺id
--------------------------------------
15:51:36,588 INFO [RecordStore] Inbound Message
----------------------------
ID: 2
Response-Code: 200
Encoding: ISO-8859-1
Content-Type: application/fastinfoset
Headers: {Content-Length=[195], content-type=[application/fastinfoset], Date=[Mon, 27 Jul 2015 20:51:32 GMT], Server=[Jetty(6.1.26)]}
Payload: α ☺ 8╧♥soap(http://schemas.xmlsoap.org/soap/envelope/≡???Envelope???♥Body8╧☻ns2∟http://record.itl.endeca.com/═!http://recordstore.itl.endeca.com/≡=?↨startTransactionResponse=?♣return=?☺id
--------------------------------------
15:51:37,204 INFO [ProductCatalogOutputConfig] Starting bulk load
15:51:37,440 INFO [RecordStore] Outbound Message
---------------------------
ID: 3
Address: http://localhost:8500/CRS_en_schema/
Content-Type: application/fastinfoset
Headers: {Accept=[application/fastinfoset, */*], Accept-Encoding=[gzip;q=1.0, identity; q=0.5, *;q=0], Content-Encoding=[gzip], SOAPAction=[""]}
propertyValues???♀propertyValue???♥name?soap.org/soap/envelope/????Envelope???♥Body8?☻ns2∟http://record.itl.endeca.com/?!http://recordstore.itl.endeca.com/?=?♂writeRecords=?♀transactionId=?☺id?7?=?
Endeca.Action????♦value?♥DELETE???↓attribute.property.data_type?        ?♥string?♂attribute.name?       product.url?♂attribute.type?    ?♣property?‼attribute.display_name?     ???♣?? ?♠item.type??
?       ?♂??    ?,Product -> Disallow as automated recommendation??♣?? ?↕product.nonreturnable??       ??      ?♀?
?       ?♂??    ?§Product -> Nonreturnable??♣?? ???     ?◄product.repositoryId??        ??↑attribute.search.searchable? ?♂??    ?
Product -> Id????       ?
product.brand???        ?◄?
product.language??      ??? ? ?¶?tiselect_type?       ?♣multi-or??    ?
?       ?♂??    ?►Product -> language??♣??      ?↕product.priceListPair??       ??      ?▬?
product.keywords??      ?☼?     ?♂??    ?↕Product -> Keyword(s)??♣??    ???     ?§product.adminDisplayName?☼?   ?♂??    ?▼Product -> Management Display Name????        ?↑??    ?♫product.startDate??

Solution:
Add the following to jboss-as\server\server-name\conf\jboss-log4j.xml:

<category name="org.apache.cxf.interceptor.LoggingInInterceptor"> <priority value="WARN"/> </category> <category name="org.apache.cxf.interceptor.LoggingOutInterceptor"><priority value="WARN"/> </category>

http://ravihonakamble.blogspot.com/2015/07/avoiding-cxf-log-entries-in-jboss-log.html

FileNotFound Error message in JBOSS log and site is not showing Endeca Experience Manager pages - Endeca Experience Manager

Problem Statement:

FileNotFound Error message in JBOSS log and web site is not showing Endeca Experience Manager pages

2015-07-25 12:51:40,317 INFO [nucleusNamespace.atg.endeca.assembler.AssemblerPipelineServlet] (ajp-0.0.0.0-10209-2) No country specific page found. Attempting to assemble non country specific page

2015-07-25 12:51:40,319 ERROR [nucleusNamespace.atg.endeca.assembler.AssemblerTools] (ajp-0.0.0.0-10209-2) A problem occured assembling requested content. Response from the assembler was {contentUri=/pages/browse/_/N-100972, @type=ContentInclude, @error=FileNotFound}

2015-07-25 12:51:40,319 WARN [nucleusNamespace.atg.endeca.assembler.AssemblerPipelineServlet] (ajp-0.0.0.0-10209-2) Country specific and non-country specific page could not be found. If the url attempting to be served is not meant to be an XM page, consider adding this path to ignore list to avoid unnecessary Endeca assembling. This request will now pass to next servlet in pipeline.

Solution:

Endeca Experience Manager content is not accessible from ATG store instances. ATG and Endeca talks over port 8007 for XM content.

Try below options:

1) Check if 8007 port is opened or not

2) Manually promote content

a) Go to control Directory of Endeca application

/opt/app/endeca/app/CRS/control

b) Run promote script ./promote_content.sh

How to print all Endeca components status? - Endeca EAC

Displaying the status of Endeca application components.

1) Go to Endeca APP control directory
Example: /opt/app/endeca/apps/CRS/control

2) Use runcommand to print status of all Endeca components
$ ./runcommand.sh --print-status
Status of components defined in AppConfig.xml.
AuthoringDgraph: Running
DgraphA1: Failed
LogServer: Running
WeeklyReportGenerator: NotRunning
DailyReportGenerator: NotRunning
DaySoFarReportGenerator: NotRunning
WeeklyHtmlReportGenerator: NotRunning
DailyHtmlReportGenerator: NotRunning
DaySoFarHtmlReportGenerator: NotRunning
ConfigurationGeneratorForge: NotRunning
Forge: NotRunning
PartialForge: NotRunning
Dgidx: NotRunning

How to list all record store instances created in CAS? - Endeca CAS

1) Go to CAS bin directory
Example: /opt/app/endeca/CAS/3.1.x/bin

2) Run below command
/opt/app/endeca/CAS/3.1.1/bin]
$ ./component-manager-cmd.sh list-components

NAME           TYPE              STATUS
CRS_en_dimvals        RecordStore        RUNNING
CRS_en_data RecordStore    RUNNING
CRS_en_prules           RecordStore        RUNNING
CRS_en_schema         RecordStore        RUNNING

How to create/delete record store instances manually? [ERROR Message - CRS-en-dimvals/?wsdl returned response code 404] - Endeca CAS

Useful when you get below error message due to absence of record store instances in CAS:
[ERROR Message - CRS-en-dimvals/?wsdl returned response code 404]
[ERROR Message - CRS-en-data/?wsdl returned response code 404]
[ERROR Message - CRS-en-schema/?wsdl returned response code 404]
[ERROR Message - CRS-en-prules/?wsdl returned response code 404]

1) Go to CAS bin directory
Example: /opt/app/endeca/CAS/3.1.x/bin

2) Use below command pattern to delete if any existing CAS record stores
Example: Delete 4 record stores created for ATG-Endeca integration

./component-manager-cmd.sh delete-component -n CRS_en_schema
./component-manager-cmd.sh delete-component -n CRS_en_dimvals
./component-manager-cmd.sh delete-component -n CRS_en_prules
./component-manager-cmd.sh delete-component -n CRS_en_data

3) Use below command pattern to create CAS record stores

Example:Create 4 record stores required for ATG-Endeca integration

./component-manager-cmd.sh create-component -t RecordStore -n CRS_en_schema
./component-manager-cmd.sh create-component -t RecordStore -n CRS_en_data
./component-manager-cmd.sh create-component -t RecordStore -n CRS_en_dimvals
./component-manager-cmd.sh create-component -t RecordStore -n CRS_en_prules

4) Use below command pattern to Read CAS record stores

recordstore-cmd read-baseline -a CRS_en_data -n 50000 -f CRS_en_data.xml
recordstore-cmd read-baseline -a CRS_en_schema -n 50000 -f CRS_en_schema.xml
recordstore-cmd read-baseline -a CRS_en_dimvals -n 500 -f CRS_en_dimvals.xml
recordstore-cmd read-baseline -a CRS_en_prules -n 50000 -f CRS_en_prules.xml

Oracle recommendations - Multi-site Operations setup

Here is the standard recommendations from Oracle on multi-site Operations setup:

SEO URL optimization, Endeca sitemap generation, SEO meta data best practices - Endeca SEO capabilities

Customers should use both Oracle Commerce Platform SEO and Guided Search and Experience Manager SEO capabilities as complements of each other in order to manage SEO for the entire site. There are multiple elements of SEO—URL optimization, site map generation, SEO meta data—that are outlined below.

URL Optimization
   1) Pages that have a MDEX navigation state should use the URL Optimization API included with the Assembler
   2) Use Oracle Commerce Platform URL Recoding for all other Oracle Commerce pages
Site Map Generation
   3) Customers should use components included in both Oracle Commerce Platform and Guided

Site Map Generation
Customers should use components included in both Oracle Commerce Platform and Guided Search for site map generation.
    1) Use the Guided Search Site Map Generator to generate the links to pages for navigation states that can include search terms or dimension intersections. This has a significant SEO benefit because the long-tail will be included in the site map, and natural search results can bring site visitors deep into the catalog (and closer to conversion).
    2) Use the Oracle Commerce Platform SitemapGeneratorService to generate the sitemap for all other Oracle Commerce pages

SEO Meta Data
   1) For pages managed in Experience Manager, set keywords in the pages and cartridges.
   2) For other pages, set up keywords in the BCC. With properly coded pages, the Oracle Commerce Platform will automatically add these keywords to pages.

Oracle Endeca UI design pattern Library - Search and Browse features

Link for Oracle Endeca UI design pattern Library for search and browse features:
http://www.oracle.com/webfolder/ux/applications/uxd/endeca/content/library/en/home/patterns.html

How to check the Endeca locks acquired and release them? - Endeca Scripts

1) Go to below directory
         /opt/app/endeca/PlatformServices/11.1.0/bin

2) To check all the flags acquired run below command
         ./eaccmd.sh list-flags --app CRS

Flag Examples: baseline_data_ready
                    update_lock

3) Depending on the output you can remove any flags acquired. Here is the command to remove flags one by one:
   ./eaccmd.sh remove-flag --app CRS--flag baseline_data_ready

4) you can remove all flags at once by using below command:
./eaccmd.sh remove-all-flags --app CRS

FYI - CRS is Endeca application name

Creating the Endeca EAC Application via Scripts - Deployment Template

How to create the Endeca EAC application?

On the machine where Oracle Commerce Guided Search with Experience Manager is installed, create the directory where the EAC application will reside

Example: /opt/app/endeca/apps
Copy the <ATG11dir>/CommerceReferenceStore/Store/Storefront/deploy directory to a temporary location on the machine where Oracle Commerce Guided Search with Experience Manager is installed.

Example: /opt/app/ATGtemp
Execute the deploy script.

Example:

/opt/app/endeca/ToolsAndFrameworks/11.1.0/deployment_template/bin >./deploy.sh --app /opt/app/ATGtemp/deploy/deploy.xml
Press Enter to continue with the install
Enter the following information for the script prompts:
- Enter N to install a base deployment.
- Enter CRS for the name of the application.
- Enter the full path to the Apps directory,
  
  Example: /opt/app/endeca/apps
- Enter the EAC port (default is 8888)
- Workbench port number [8006]
- Dgraph1 (Live Dgraph) port (default is 15000)
- Authoring Dgraph port (default is 15002)
- LogServer port (default is 15010)
- Enter the CAS install directory using UNIX-style forward slashes, even if on Windows.
  
  Example:/opt/app/endeca/CAS/11.1.0
- Enter the CAS version number (default is 11.1.0).
- Enter the CAS hostname (default is localhost).
- Enter the CAS port (default is 8500).
- Enter the language code (default is en).
- Enter the fully-qualified Workbench hostname (localhost)
- Production application server host name. This is the host where the ATGProduction server will be running. [localhost]
- Production application server HTTP port number.
  
  IMPORTANT: If you are following the instructions in this guide, these values are 7003 for WebLogic, 8080 for JBoss and Tomcat, and 9081 for WebSphere.
- Preview host name [localhost]
- Preview port number.
  
  If you are following the instructions in this guide, these values are 7003 for WebLogic, 8080 for JBoss and Tomcat, and 9081 for WebSphere.
- Context root of the preview application [crs]
- Host name of the user segment server. This is the server that will respond to Experience Manager requests for Business Control Center user segment data. Oracle recommends using the Content Adminstration server (also called the ATGPublishing server in this guide). [localhost]
- HTTP port of the user segment server.
  
  IMPORTANT: If you are following the instructions in this guide, these values are 7003 for WebLogic, 8080 for JBoss and Tomcat, and 9081 for WebSphere.
- Path for the Oracle Wallet jps-config.xml file
  
  Example: /opt/app/endeca/ToolsAndFrameworks/11.1.0/server/workspace/credential_store/jps-config.xml
- Application configuration archive path. This is the location where content promoted using the file-based method is stored,
  
  Example, /opt/app/endeca/apps/CRS/data/workbench/application_export_archive
You see a message that says “Application successfully deployed” when the deployment is finished.
Change directories to the EAC applications directory,

Example: /opt/app/endeca/apps

The CRS application directory is added as child of the /apps directory.

How to provision the EAC application?

In a UNIX shell, change directories to your EAC application’s /control directory,

Example: /opt/app/endeca/apps/CRS/control
Execute the initialize_services.sh script.

./initialize_services.sh

How to add Endeca Record Sort options in ATG 11.1? - Core Endeca

1) Go to /opt/app/endeca/apps/CRS/config/mdex/record_sort_config.xml and add the desired properties that can sort the result set.

Example:
<RECORD_SORT_CONFIG>
<RECORD_SORT NAME="product.displayName"/>
<RECORD_SORT NAME="product.salePrice"/>
<RECORD_SORT NAME="product.bvRatings"/>
</RECORD_SORT_CONFIG>

2) Run a baseline index to create the sorts in the MDEX.

How to Enable Endeca property for record filtering in ATG 11.1? - Core Endeca

1) Add the desired property in the /opt/app/endeca/CRS/config/index_config/index_config.json and add "isRecordFilterable" : true in the property.
For example:
"product.repositoryId" : {
          "propertyDataType" : "ALPHA",
          "mergeAction" : "UPDATE",
          "isRollupKey" : true,
          "jcr:primaryType" : "endeca:property",
          "isRecordFilterable" : true
        },

2) Once you update index-config.json make sure to set configurations at Endeca repository level.
Here is the command:
   /opt/app/endeca/apps/CRS/control :>
                          ./index_config_cmd.sh set-config -f ../config/index_config/index-config.json -o all
3) Run Endeca indexing

Endeca - Stop words

Endeca Stop words
Stop words are words that are ignored by the MDEX Engine when the words are part of a keyword search.

Where I can find OOTB Stop words?
Endeca 3.1.X
/opt/apps/endeca/apps/CRS/config/CRS.stop_words.xml
Endeca 11.X
/opt/app/endeca/apps/CRS/config/mdex/CRS.stop_words.xml

<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<!DOCTYPE STOP_WORDS SYSTEM "stop_words.dtd">
<STOP_WORDS>
<STOP_WORD>a</STOP_WORD>
<STOP_WORD>do</STOP_WORD>
<STOP_WORD>me</STOP_WORD>
<STOP_WORD>when</STOP_WORD>
<STOP_WORD>about</STOP_WORD>
<STOP_WORD>find</STOP_WORD>
<STOP_WORD>not</STOP_WORD>
<STOP_WORD>where</STOP_WORD>
<STOP_WORD>above</STOP_WORD>
<STOP_WORD>for</STOP_WORD>
<STOP_WORD>or</STOP_WORD>
<STOP_WORD>why</STOP_WORD>
<STOP_WORD>an</STOP_WORD>
<STOP_WORD>from</STOP_WORD>
<STOP_WORD>over</STOP_WORD>
<STOP_WORD>with</STOP_WORD>
<STOP_WORD>and</STOP_WORD>
<STOP_WORD>have</STOP_WORD>
<STOP_WORD>show</STOP_WORD>
<STOP_WORD>you</STOP_WORD>
<STOP_WORD>any</STOP_WORD>
<STOP_WORD>how</STOP_WORD>
<STOP_WORD>the</STOP_WORD>
<STOP_WORD>your</STOP_WORD>
<STOP_WORD>are</STOP_WORD>
<STOP_WORD>I</STOP_WORD>
<STOP_WORD>under</STOP_WORD>
<STOP_WORD>can</STOP_WORD>
<STOP_WORD>is</STOP_WORD>
<STOP_WORD>what</STOP_WORD>
</STOP_WORDS>

You can add application specific stop words in above mentioned file and run Endeca baseline update.

Assembler Code to fetch Endeca Experience Manager and Endeca GuidedSearch content - Endeca Assembler API

Code snippet to invoke Endeca Experience Manager Content:

<dsp:droplet name="/atg/endeca/assembler/droplet/InvokeAssembler">
            <dsp:param name="includePath" value="/pages/browse" />
            <dsp:oparam name="output">
               <dsp:getvalueof var="contentItem" vartype="com.endeca.infront.assembler.ContentItem"         param="contentItem" />
            </dsp:oparam>
</dsp:droplet>

Code snippet to invoke Endeca Guided Search content:

<dsp:droplet name="/atg/endeca/assembler/droplet/InvokeAssembler">
            <dsp:param name="includePath" value="/services/guidedsearch" />
            <dsp:oparam name="output">
               <dsp:getvalueof var="contentItem" vartype="com.endeca.infront.assembler.ContentItem" param="contentItem" />
            </dsp:oparam>
  </dsp:droplet>

Input Parameters

The InvokeAssembler servlet bean has two input parameters, includePath and contentCollection.

includePath

Use the includePath parameter for a page request.

For example, to assemble content for a browse page, specify /pages/browse for the includePath (passing in a /browse path will not match because it is missing the /pages prefix).

contentCollection

Use the contentCollection parameter for a content collection request.

For example, Search Box Auto Suggest Content. InvokeAssembler parses the contentCollection into a ContentSlotConfig component.

The includePath and contentCollection parameters are mutually exclusive but one of them must be passed when using the InvokeAssembler servlet bean.

ruleLimit

This optional parameter is used in conjunction with the contentCollection parameter to specify the number of items that should be returned from the specified content collection.

Output Parameters

The InvokeAssembler servlet bean has one output parameter, contentItem. This parameter contains the root ContentItem returned by the Assembler. If this content item is empty, the request was not an Assembler request.

ATG Endeca Data Integration Diagram - v3.1.x

Integration Overview

The basic workflow diagram to show integration of the ATG product catalog with Endeca.

As seen in the diagram above, the data integration supports the publication of Product or SKU and Category assets directly into Endeca Content Acquisition System (CAS) Record Stores. The developer can control the type and granularity of assets published via the Product Exporter and the Category Exporter.

The Product Exporter behavior is managed by the Product Indexing Configuration file, which specifies the granularity and inclusion of product or SKU assets.

The Category Exporter is managed by the Category Indexing Configuration, which specifies the export of the Category to Dimension.

The Schema Exporter, managed by the Schema Indexing Configuration, exports schema information to Endeca.

Operationally, the publication of ATG assets and creation of the corresponding Endeca index is controlled by the ATG Web Commerce Product Catalog Indexing Administrator (Indexing Admin). The Indexing Admin invokes the Endeca Application Controller (EAC), which controls the creation and publication of the Endeca index throughout your production or staging environment.

The EAC, via configurable Endeca Deployment Template modules, controls the orchestration of the Endeca index creation process. Further, the EAC orchestrates the publication of the built Endeca index to all configured production or staging servers and ensures that the Endeca MDEX is running.

Indexing Using ATG Dynamo Administration UI

Navigate to the ProductCatalogSimpleIndexingAdmin component to invoke the Endeca baseline Indexing process or partial update where localhost and 8080 are the host and HTTP port of your production ATG instance (or publishing instance, if you are using publishing).

More details on Indexing:
http://ravihonakamble.blogspot.com/2015/06/how-to-read-data-from-endeca-cas-record.html