How to Migrate Dimension Value IDs between environments - (Forgeless) - Endeca commerce 11.1

Move Dimension Value IDs from UAT to production:

1. Run the export command on the UAT environment:
/opt/app/endeca/CAS/11.1.0/bin>cas-cmd.sh exportDimensionValueIdMappings -m <APP_NAME>-dimension-value-id-manager -f UAT_DIMVALS.txt 

2. Move the file UAT_DIMVALS.txt, to the PRODUCTION environment

3. Run the import command on the PRODUCTION environment: 
/opt/app/endeca/CAS/11.1.0/bin>cas-cmd.sh importDimensionValueIdMappings –m <APP_NAME>-dimension-value-id-manager –f  UAT_DIMVALS.txt

     
      4. Move dvalid_mappings_archive from UAT to PRODUCTION 
       location:  <APP_NAME>/data/dvalid_mappings_archive

ATG-Endeca Site outage: Front End application goes down when baseline_update.bat|sh or promote_content.bat|sh script is triggered (Endeca v3.1.0, Endeca v3.1.1 and Endeca 3.1.2)

Problem Statement: 
ATG-Endeca Site goes down when baseline_update.bat|sh or promote_content.bat|sh script is triggered. This happens on applications which uses Endeca v3.1.0, Endeca v3.1.1 and Endeca 3.1.2 

Solution:
User may not see any error message when baseline_update.bat|sh or promote_content.bat|sh script is triggered but ERROR message will be logged in IFCR log file with failed rule which is created in Experience Manager with a specific location.

Ex: *ERROR*  POST /ifcr/system/endeca/mdexPublisher HTTP/1.1] com.endeca.ifcr.publish.impl.PublishService Unable to publish config documents com.endeca.ifcr.configService.ConfigServiceException: Config service error: Validation failed with 1 error [Unknown DVAL_ID 100972 specified for MERCH_RULE_TRIGGER in rule with id 21. Ignoring this MERCH_RULE element.]
Quick fix will be Correct all invalid rule(s) by modifying the problem rule(s) and set a valid location for each using Experience Manager and re-promote the content using promote_content.bat|sh script

Permanent fix: 
Apply below patch which is strongly recommended by ORACLE.
16761103: PUBLISH ACTION FOR EXPERIENCE MANAGER FAILS ON INVALID DVAL IDS
 OR
upgrading to MDEX 6.4.1.2 or later which contains these fixes 

This patch alters the MDEX behavior so that if an Experience Manager rule is ignored due to a missing dimension value used as the trigger, the MDEX publish action will still proceed with the valid rules and succeeds instead of failing.   After applying the patch, you will need to re-promote the content to the engines so that the MDEX's will end up with a good set of rules/content.

Endeca Record Aggregation - Rollup key and Benefits

Endeca Record Aggregation (Rollup) allows developers to do below two things:         
       1)  On Listing page, results can be rolled up based on a shared property or dimension value
       2)  On Product Details Page,  all aggregated records can be requested

Benefits of Record Aggregation (Rollup):
        1)  Avoid multiple records that are essentially same
              Ex: On search item with 5 sizes and 5 colors will return 25 results instead of 1 record
        2)  Organize a record list based on a common value
               Ex: A list of unique SKU’s grouped by Product
        3)  Avoid losing precision from consolidation
               Ex: Each unique SKU has a Unique inventory state 

How to define Rollup key?  
Refer -  http://ravihonakamble.blogspot.com/2015/07/oracle-commerce-11x-how-to-define.html

Endeca aggregated records in JSP reference application:

Recommended Installation order for Endeca components - Endeca Commerce

Recommended Installation order for Endeca components

Recommended order	Installer
1	MDEX Engine       Dgidx       Dgraphs
2	Platform Services Forge Endeca HTTP service (8888) Endeca Reference Application EAC Components Endeca Logging and Reporting CADK
3	Tools and Frameworks Endeca Tools Service (8006) Workbench Endeca Reference App Discover Electronics Ref. App Deployment Template Assembler API
4	Additional Installers      Developer Studio      Search Engine Optimization       CAS

JBOSS console hangs while doing Endeca baseline Index from dynamo admin - ATG Endeca Integration

Problem Statement:
Getting below error message in JBOSS log and console hangs.

15:51:29,881 INFO  [RecordStore] Outbound Message
---------------------------
ID: 1
Address: http://localhost:8500/CRS_en_schema/
Encoding: UTF-8
Content-Type: text/xml
Headers: {Accept=[application/fastinfoset, */*], Accept-Encoding=[gzip;q=1.0, identity; q=0.5, *;q=0], SOAPAction=[""]}
Payload: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><startTransaction xmlns="http://recordstore.itl.endeca.com/" xmlns:ns2="http://record.itl.endeca.com/"><tra
--------------------------------------
15:51:32,053 INFO  [ReflectionServiceFactoryBean] Creating Service {http://recordstore.itl.endeca.com/}RecordStoreService from class com.endeca.itl.recordstore.RecordStore
15:51:32,649 INFO  [DimensionDocumentSubmitter] Will attempt to contact record store "CRS_en_dimvals" on host "localhost" on port 8,500
15:51:32,655 INFO  [RecordStore] Outbound Message
---------------------------
ID: 2
Address: http://localhost:8500/CRS_en_dimvals/
Encoding: UTF-8
Content-Type: text/xml
Headers: {Accept=[application/fastinfoset, */*], Accept-Encoding=[gzip;q=1.0, identity; q=0.5, *;q=0], SOAPAction=[""]}
Payload: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><startTransaction xmlns="http://recordstore.itl.endeca.com/" xmlns:ns2="http://record.itl.endeca.com/"><tra
--------------------------------------
15:51:36,588 INFO  [RecordStore] Inbound Message
----------------------------
ID: 1
Response-Code: 200
Encoding: ISO-8859-1
Content-Type: application/fastinfoset
Headers: {Content-Length=[195], content-type=[application/fastinfoset], Date=[Mon, 27 Jul 2015 20:51:31 GMT], Server=[Jetty(6.1.26)]}
Payload: α  ☺ 8╧♥soap(http://schemas.xmlsoap.org/soap/envelope/≡???Envelope???♥Body8╧☻ns2∟http://record.itl.endeca.com/═!http://recordstore.itl.endeca.com/≡=?↨startTransactionResponse=?♣return=?☺id
--------------------------------------
15:51:36,588 INFO  [RecordStore] Inbound Message
----------------------------
ID: 2
Response-Code: 200
Encoding: ISO-8859-1
Content-Type: application/fastinfoset
Headers: {Content-Length=[195], content-type=[application/fastinfoset], Date=[Mon, 27 Jul 2015 20:51:32 GMT], Server=[Jetty(6.1.26)]}
Payload: α  ☺ 8╧♥soap(http://schemas.xmlsoap.org/soap/envelope/≡???Envelope???♥Body8╧☻ns2∟http://record.itl.endeca.com/═!http://recordstore.itl.endeca.com/≡=?↨startTransactionResponse=?♣return=?☺id
--------------------------------------
15:51:37,204 INFO  [ProductCatalogOutputConfig] Starting bulk load
15:51:37,440 INFO  [RecordStore] Outbound Message
---------------------------
ID: 3
Address: http://localhost:8500/CRS_en_schema/
Content-Type: application/fastinfoset
Headers: {Accept=[application/fastinfoset, */*], Accept-Encoding=[gzip;q=1.0, identity; q=0.5, *;q=0], Content-Encoding=[gzip], SOAPAction=[""]}
propertyValues???♀propertyValue???♥name?soap.org/soap/envelope/????Envelope???♥Body8?☻ns2∟http://record.itl.endeca.com/?!http://recordstore.itl.endeca.com/?=?♂writeRecords=?♀transactionId=?☺id?7?=?
Endeca.Action????♦value?♥DELETE???↓attribute.property.data_type?        ?♥string?♂attribute.name?       product.url?♂attribute.type?    ?♣property?‼attribute.display_name?     ???♣??  ?♠item.type??
?       ?♂??    ?,Product -> Disallow as automated recommendation??♣??  ?↕product.nonreturnable??       ??      ?♀?
?       ?♂??    ?§Product -> Nonreturnable??♣?? ???     ?◄product.repositoryId??        ??↑attribute.search.searchable? ?♂??    ?
Product -> Id????       ?
product.brand???        ?◄?
product.language??      ???  ?  ?¶?tiselect_type?       ?♣multi-or??    ?
?       ?♂??    ?►Product -> language??♣??      ?↕product.priceListPair??       ??      ?▬?
product.keywords??      ?☼?     ?♂??    ?↕Product -> Keyword(s)??♣??    ???     ?§product.adminDisplayName?☼?   ?♂??    ?▼Product -> Management Display Name????        ?↑??    ?♫product.startDate?? 

Solution:
Add the following to jboss-as\server\server-name\conf\jboss-log4j.xml: 

<category name="org.apache.cxf.interceptor.LoggingInInterceptor"> <priority value="WARN"/> </category> <category name="org.apache.cxf.interceptor.LoggingOutInterceptor"><priority value="WARN"/> </category>

http://ravihonakamble.blogspot.com/2015/07/avoiding-cxf-log-entries-in-jboss-log.html

FileNotFound Error message in JBOSS log and site is not showing Endeca Experience Manager pages - Endeca Experience Manager

Problem Statement:

FileNotFound Error message in JBOSS log and web site is not showing Endeca Experience Manager pages

2015-07-25 12:51:40,317 INFO  [nucleusNamespace.atg.endeca.assembler.AssemblerPipelineServlet] (ajp-0.0.0.0-10209-2) No country specific page found. Attempting to assemble non country specific page

2015-07-25 12:51:40,319 ERROR [nucleusNamespace.atg.endeca.assembler.AssemblerTools] (ajp-0.0.0.0-10209-2) A problem occured assembling requested content. Response from the assembler was {contentUri=/pages/browse/_/N-100972, @type=ContentInclude, @error=FileNotFound}

2015-07-25 12:51:40,319 WARN  [nucleusNamespace.atg.endeca.assembler.AssemblerPipelineServlet] (ajp-0.0.0.0-10209-2) Country specific and non-country specific page could not be found. If the url attempting to be served is not meant to be an XM page, consider adding this path to ignore list to avoid unnecessary Endeca assembling. This request will now pass to next servlet in pipeline.

Solution:

Endeca Experience Manager content is not accessible from ATG store instances. ATG and Endeca talks over port 8007 for XM content.

Try below options:

1) Check if 8007 port is opened or not

2) Manually promote content 

       a) Go to control Directory of Endeca application

              /opt/app/endeca/app/CRS/control

       b) Run promote script ./promote_content.sh

How to print all Endeca components status? - Endeca EAC

Displaying the status of Endeca application components.

1) Go to Endeca APP control directory
Example: /opt/app/endeca/apps/CRS/control

2)  Use runcommand to print status of all Endeca components 
$ ./runcommand.sh --print-status
Status of components defined in AppConfig.xml.
  AuthoringDgraph: Running
  DgraphA1: Failed
  LogServer: Running
  WeeklyReportGenerator: NotRunning
  DailyReportGenerator: NotRunning
  DaySoFarReportGenerator: NotRunning
  WeeklyHtmlReportGenerator: NotRunning
  DailyHtmlReportGenerator: NotRunning
  DaySoFarHtmlReportGenerator: NotRunning
  ConfigurationGeneratorForge: NotRunning
  Forge: NotRunning
  PartialForge: NotRunning
  Dgidx: NotRunning

How to list all record store instances created in CAS? - Endeca CAS

1) Go to CAS bin directory
Example: /opt/app/endeca/CAS/3.1.x/bin

2) Run below command
/opt/app/endeca/CAS/3.1.1/bin]
$ ./component-manager-cmd.sh list-components
 
NAME                         TYPE                  STATUS
CRS_en_dimvals        RecordStore        RUNNING
CRS_en_data              RecordStore        RUNNING
CRS_en_prules           RecordStore        RUNNING
CRS_en_schema         RecordStore        RUNNING

How to create/delete record store instances manually? [ERROR Message - CRS-en-dimvals/?wsdl returned response code 404] - Endeca CAS

Useful when you get below error message due to absence of record store instances in CAS:
 [ERROR Message - CRS-en-dimvals/?wsdl returned response code 404]
 [ERROR Message - CRS-en-data/?wsdl returned response code 404]
 [ERROR Message - CRS-en-schema/?wsdl returned response code 404]
 [ERROR Message - CRS-en-prules/?wsdl returned response code 404]  

1) Go to CAS bin directory
Example: /opt/app/endeca/CAS/3.1.x/bin

2) Use below command pattern to delete if any existing CAS record stores
Example: Delete 4 record stores created for ATG-Endeca integration 

./component-manager-cmd.sh delete-component -n CRS_en_schema
./component-manager-cmd.sh delete-component -n CRS_en_dimvals
./component-manager-cmd.sh delete-component -n CRS_en_prules
./component-manager-cmd.sh delete-component -n CRS_en_data

3) Use below command pattern to create CAS record stores

Example:Create 4 record stores required for ATG-Endeca integration

./component-manager-cmd.sh create-component -t RecordStore -n CRS_en_schema
./component-manager-cmd.sh create-component -t RecordStore -n CRS_en_data
./component-manager-cmd.sh create-component -t RecordStore -n CRS_en_dimvals
./component-manager-cmd.sh create-component -t RecordStore -n CRS_en_prules

4) Use below command pattern to Read CAS record stores

recordstore-cmd read-baseline -a CRS_en_data -n 50000 -f CRS_en_data.xml
recordstore-cmd read-baseline -a CRS_en_schema -n 50000 -f CRS_en_schema.xml
recordstore-cmd read-baseline -a CRS_en_dimvals -n 500 -f CRS_en_dimvals.xml
recordstore-cmd read-baseline -a CRS_en_prules -n 50000 -f CRS_en_prules.xml

Oracle recommendations - Multi-site Operations setup

Here is the standard recommendations from Oracle on multi-site Operations setup:

SEO URL optimization, Endeca sitemap generation, SEO meta data best practices - Endeca SEO capabilities

Customers should use both Oracle Commerce Platform SEO and Guided Search and Experience Manager SEO capabilities as complements of each other in order to manage SEO for the entire site. There are multiple elements of SEO—URL optimization, site map generation, SEO meta data—that are outlined below.

URL Optimization
   1) Pages that have a MDEX navigation state should use the URL Optimization API included with the Assembler
   2) Use Oracle Commerce Platform URL Recoding for all other Oracle Commerce pages
Site Map Generation
   3) Customers should use components included in both Oracle Commerce Platform and Guided  

Site Map Generation
Customers should use components included in both Oracle Commerce Platform and Guided Search for site map generation.
    1) Use the Guided Search Site Map Generator to generate the links to pages for navigation states that can include search terms or dimension intersections. This has a significant SEO benefit because the long-tail will be included in the site map, and natural search results can bring site visitors deep into the catalog (and closer to conversion).
    2) Use the Oracle Commerce Platform SitemapGeneratorService to generate the sitemap for all other Oracle Commerce pages

SEO Meta Data
   1) For pages managed in Experience Manager, set keywords in the pages and cartridges.
   2) For other pages, set up keywords in the BCC. With properly coded pages, the Oracle Commerce Platform will automatically add these keywords to pages.

Oracle Endeca UI design pattern Library - Search and Browse features

Link for Oracle Endeca UI design pattern Library for search and browse features:
http://www.oracle.com/webfolder/ux/applications/uxd/endeca/content/library/en/home/patterns.html

How to check the Endeca locks acquired and release them? - Endeca Scripts

1) Go to below directory
         /opt/app/endeca/PlatformServices/11.1.0/bin

2) To check all the flags acquired run below command
         ./eaccmd.sh list-flags --app CRS
         
Flag Examples: baseline_data_ready
                            update_lock

3) Depending on the output you can remove any flags acquired. Here is the command to remove flags one by one:
           ./eaccmd.sh remove-flag --app CRS--flag baseline_data_ready

4) you can remove all flags at once by using below command:
            ./eaccmd.sh remove-all-flags --app CRS

FYI - CRS is Endeca application name

Creating the Endeca EAC Application via Scripts - Deployment Template

How to create the Endeca EAC application?

1. On the machine where Oracle Commerce Guided Search with Experience Manager is installed, create the directory where the EAC application will reside
   Example: /opt/app/endeca/apps 
   
   
2. Copy the <ATG11dir>/CommerceReferenceStore/Store/Storefront/deploy directory to a temporary location on the machine where Oracle Commerce Guided Search with Experience Manager is installed.
   Example: /opt/app/ATGtemp
   
   
3. Execute the deploy script. 
   Example:
   /opt/app/endeca/ToolsAndFrameworks/11.1.0/deployment_template/bin >./deploy.sh --app /opt/app/ATGtemp/deploy/deploy.xml
   
   
4. Press Enter to continue with the install
5. Enter the following information for the script prompts:
   • Enter N to install a base deployment.
     
     
   • Enter CRS for the name of the application.
     
     
   • Enter the full path to the Apps directory, 
     Example: /opt/app/endeca/apps 
   • Enter the EAC port (default is 8888)
     
     
   • Workbench port number [8006]
     
     
   • Dgraph1 (Live Dgraph) port (default is 15000)
     
     
   • Authoring Dgraph port (default is 15002)
     
     
   • LogServer port (default is 15010)
     
     
   • Enter the CAS install directory using UNIX-style forward slashes, even if on Windows. 
     Example:/opt/app/endeca/CAS/11.1.0
     
     
   • Enter the CAS version number (default is 11.1.0).
     
     
   • Enter the CAS hostname (default is localhost).
     
     
   • Enter the CAS port (default is 8500).
     
     
   • Enter the language code (default is en).
     
     
   • Enter the fully-qualified Workbench hostname (localhost)
     
     
   • Production application server host name. This is the host where the ATGProduction server will be running. [localhost]
     
     
   • Production application server HTTP port number. 
     IMPORTANT: If you are following the instructions in this guide, these values are 7003 for WebLogic, 8080 for JBoss and Tomcat, and 9081 for WebSphere.
     
     
   • Preview host name [localhost]
     
     
   • Preview port number.
     
     
     If you are following the instructions in this guide, these values are 7003 for WebLogic, 8080 for JBoss and Tomcat, and 9081 for WebSphere.
     
     
   • Context root of the preview application [crs]
     
     
   • Host name of the user segment server. This is the server that will respond to Experience Manager requests for Business Control Center user segment data. Oracle recommends using the Content Adminstration server (also called the ATGPublishing server in this guide). [localhost]
     
     
   • HTTP port of the user segment server.  
     IMPORTANT: If you are following the instructions in this guide, these values are 7003 for WebLogic, 8080 for JBoss and Tomcat, and 9081 for WebSphere. 
     
     
   • Path for the Oracle Wallet jps-config.xml file
     Example:  /opt/app/endeca/ToolsAndFrameworks/11.1.0/server/workspace/credential_store/jps-config.xml
   • Application configuration archive path. This is the location where content promoted using the file-based method is stored,
     Example, /opt/app/endeca/apps/CRS/data/workbench/application_export_archive 
   You see a message that says “Application successfully deployed” when the deployment is finished. 
   
   
6. Change directories to the EAC applications directory, 
   Example: /opt/app/endeca/apps 
   The CRS application directory is added as child of the /apps directory.

How to provision the EAC application?

1. In a UNIX shell, change directories to your EAC application’s /control directory,
   Example: /opt/app/endeca/apps/CRS/control
   

2. Execute the initialize_services.sh script.
   ./initialize_services.sh