How to change admin password in Endeca (Workbench, CAS crawl, Endeca Configuration Repository etc...)? - Endeca Workbench version 11.1

If the admin password is still known, but you want to change it to new value - these are the instructions from the guide(s):

1. Login to Workbench as the admin (by default, the username would be admin, the password would also be admin). Change it to the new password.

- This will also change the password for the Endeca Configuration Repository.

2)  Run below command
Location: /opt/app/endeca/ToolsAndFrameworks/11.1.0/credential_store/bin> ./managed_credentials.sh add --user admin --key ifcr
key in the NewPassword and Repeat. 

3)  Stop the Endeca Tools Service
   Update the credentials in webstudio file
   /opt/app/endeca/ToolsAndFrameworks/11.1.0/server/workspace/conf/webstudio.properties 
   Look for the ifcr.password property 
   Start the Endeca Tools Service 

4) Update password in CAS crawl

/opt/app/endeca/apps/CRS/config/cas/last-mile-crawl.xml

5) Delete existing crawl file and re-create a new crawl file
Location: /opt/app/endeca/CAS/11.1.0/bin
./cas-cmd.sh deleteCrawl -id CRS-last-mile-crawl
./cas-cmd.sh createCrawls -f /opt/app/endeca/apps/CRS/config/cas/last-mile-crawl.xml

6) In the ATG Index Server go to below component and update new password:
/dyn/admin/atg/endeca/index/ConfigImportDocumentSubmitter

workbenchUser	admin
workbenchPassword	admin123

7) Run Baseline_update from ATG dyn/admin

/atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin/

IFCR PromotionSite Error retrieving content bytes and Site loses all Endeca content - Endeca 3.1.x

Problem: IFCR Throws Below ERRORS and Site loses all Endeca content
++++++++++++++++++++
26.11.2015 01:15:35.779 *ERROR* [New I/O server worker #1-24] com.endeca.ifcr.publish.impl.content.server.PromotionSite Error retrieving content bytes. javax.jcr.RepositoryException: /51/49/9fec56a9409dbc581ca02b80efec/%7bhttp%3a%2f%2fwww.jcp.org%2fjcr%2f1.0%7ddata.0.bin: the specified resource does not exist
at org.apache.jackrabbit.core.value.BLOBInResource.getStream(BLOBInResource.java:111)
at org.apache.jackrabbit.core.value.BLOBInResource.copy(BLOBInResource.java:100)
...
...
Caused by: org.apache.jackrabbit.core.fs.FileSystemException: /opt/app/endeca/ToolsAndFrameworks/3.1.1/server/workspace/state/sling/_ifcr/jackrabbit/workspaces/default/blobs/51/49/9fec56a9409dbc581ca02b80efec/%7bhttp%3a%2f%2fwww.jcp.org%2fjcr%2f1.0%7ddata.0.bin does not denote an existing file
26.11.2015 01:15:46.646 Channel.socket.nio.NioWorker.cleanUpWriteBuffer(NioWorker.java:639)
at org.jboss.net*ERROR* [New I/O server worker #1-26] com.endeca.ifcr.publish.impl.content.server.ContentServer Exception caught in channel handling. java.nio.channels.ClosedChannelException
++++++++++++++++++++
Solution:
Workaround Solution:
Try to execute multiple times promote content script
/opt/app/endeca/apps/APP_NAME/control> promote_content.sh| promote_content.bat

It could be due to:  
1) Increased number of application servers 
2) Increased number of Workbench clients
3) Larger promote_content snapshots
4) Endeca Resource Resolver issue

You can ask for a patch to fix it.

Promote content from Workbench 3.1.x to a Dgraph breaks the site - Endeca Site Down

Problem Statement:
When you run promote_content.bat|sh script site goes down immediately

Check IFCR Log :
Path: /opt/app/endeca/ToolsAndFrameworks\3.1.x/server/workspace/logs/ifcr.log
 
18.08.2015 19:49:38.641 *ERROR* [172.25.74.131 [1439945378629] GET /ifcr/sites/CRS/thesaurus.html HTTP/1.1] org.apache.sling.engine.impl.SlingRequestProcessorImpl service: Uncaught SlingException org.apache.sling.api.SlingException:
  ...
Caused by: java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - For input string: "en"
    at java.net.URLDecoder.decode(URLDecoder.java:173)
    at java.net.URLDecoder.decode(URLDecoder.java:83)
    at org.apache.jsp.apps.endeca.thesaurus.html_jsp._jspService(html_jsp.java:118)
    at org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:419)
    ... 61 more

Solution:
Using a non-xml 1.0 safe character in a thesaurus entry causes the thesaurus content publish step within the promote_content process to fail. The rest of the content may succeed to be published, or the publish may fail entirely.

Obtain the following patch for Tools And Frameworks 3.1.x from support.oracle.com:

16863449: XM3.1.1: MDEX 6.4 INVALID XML CHARACTERS USED IN THESAURUS CAUSE PUBLISH FAILURE

ATG-Endeca Site outage: Front End application goes down when baseline_update.bat|sh or promote_content.bat|sh script is triggered (Endeca v3.1.0, Endeca v3.1.1 and Endeca 3.1.2)

Problem Statement: 
ATG-Endeca Site goes down when baseline_update.bat|sh or promote_content.bat|sh script is triggered. This happens on applications which uses Endeca v3.1.0, Endeca v3.1.1 and Endeca 3.1.2 

Solution:
User may not see any error message when baseline_update.bat|sh or promote_content.bat|sh script is triggered but ERROR message will be logged in IFCR log file with failed rule which is created in Experience Manager with a specific location.

Ex: *ERROR*  POST /ifcr/system/endeca/mdexPublisher HTTP/1.1] com.endeca.ifcr.publish.impl.PublishService Unable to publish config documents com.endeca.ifcr.configService.ConfigServiceException: Config service error: Validation failed with 1 error [Unknown DVAL_ID 100972 specified for MERCH_RULE_TRIGGER in rule with id 21. Ignoring this MERCH_RULE element.]
Quick fix will be Correct all invalid rule(s) by modifying the problem rule(s) and set a valid location for each using Experience Manager and re-promote the content using promote_content.bat|sh script

Permanent fix: 
Apply below patch which is strongly recommended by ORACLE.
16761103: PUBLISH ACTION FOR EXPERIENCE MANAGER FAILS ON INVALID DVAL IDS
 OR
upgrading to MDEX 6.4.1.2 or later which contains these fixes 

This patch alters the MDEX behavior so that if an Experience Manager rule is ignored due to a missing dimension value used as the trigger, the MDEX publish action will still proceed with the valid rules and succeeds instead of failing.   After applying the patch, you will need to re-promote the content to the engines so that the MDEX's will end up with a good set of rules/content.

JBOSS console hangs while doing Endeca baseline Index from dynamo admin - ATG Endeca Integration

Problem Statement:
Getting below error message in JBOSS log and console hangs.

15:51:29,881 INFO  [RecordStore] Outbound Message
---------------------------
ID: 1
Address: http://localhost:8500/CRS_en_schema/
Encoding: UTF-8
Content-Type: text/xml
Headers: {Accept=[application/fastinfoset, */*], Accept-Encoding=[gzip;q=1.0, identity; q=0.5, *;q=0], SOAPAction=[""]}
Payload: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><startTransaction xmlns="http://recordstore.itl.endeca.com/" xmlns:ns2="http://record.itl.endeca.com/"><tra
--------------------------------------
15:51:32,053 INFO  [ReflectionServiceFactoryBean] Creating Service {http://recordstore.itl.endeca.com/}RecordStoreService from class com.endeca.itl.recordstore.RecordStore
15:51:32,649 INFO  [DimensionDocumentSubmitter] Will attempt to contact record store "CRS_en_dimvals" on host "localhost" on port 8,500
15:51:32,655 INFO  [RecordStore] Outbound Message
---------------------------
ID: 2
Address: http://localhost:8500/CRS_en_dimvals/
Encoding: UTF-8
Content-Type: text/xml
Headers: {Accept=[application/fastinfoset, */*], Accept-Encoding=[gzip;q=1.0, identity; q=0.5, *;q=0], SOAPAction=[""]}
Payload: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><startTransaction xmlns="http://recordstore.itl.endeca.com/" xmlns:ns2="http://record.itl.endeca.com/"><tra
--------------------------------------
15:51:36,588 INFO  [RecordStore] Inbound Message
----------------------------
ID: 1
Response-Code: 200
Encoding: ISO-8859-1
Content-Type: application/fastinfoset
Headers: {Content-Length=[195], content-type=[application/fastinfoset], Date=[Mon, 27 Jul 2015 20:51:31 GMT], Server=[Jetty(6.1.26)]}
Payload: α  ☺ 8╧♥soap(http://schemas.xmlsoap.org/soap/envelope/≡???Envelope???♥Body8╧☻ns2∟http://record.itl.endeca.com/═!http://recordstore.itl.endeca.com/≡=?↨startTransactionResponse=?♣return=?☺id
--------------------------------------
15:51:36,588 INFO  [RecordStore] Inbound Message
----------------------------
ID: 2
Response-Code: 200
Encoding: ISO-8859-1
Content-Type: application/fastinfoset
Headers: {Content-Length=[195], content-type=[application/fastinfoset], Date=[Mon, 27 Jul 2015 20:51:32 GMT], Server=[Jetty(6.1.26)]}
Payload: α  ☺ 8╧♥soap(http://schemas.xmlsoap.org/soap/envelope/≡???Envelope???♥Body8╧☻ns2∟http://record.itl.endeca.com/═!http://recordstore.itl.endeca.com/≡=?↨startTransactionResponse=?♣return=?☺id
--------------------------------------
15:51:37,204 INFO  [ProductCatalogOutputConfig] Starting bulk load
15:51:37,440 INFO  [RecordStore] Outbound Message
---------------------------
ID: 3
Address: http://localhost:8500/CRS_en_schema/
Content-Type: application/fastinfoset
Headers: {Accept=[application/fastinfoset, */*], Accept-Encoding=[gzip;q=1.0, identity; q=0.5, *;q=0], Content-Encoding=[gzip], SOAPAction=[""]}
propertyValues???♀propertyValue???♥name?soap.org/soap/envelope/????Envelope???♥Body8?☻ns2∟http://record.itl.endeca.com/?!http://recordstore.itl.endeca.com/?=?♂writeRecords=?♀transactionId=?☺id?7?=?
Endeca.Action????♦value?♥DELETE???↓attribute.property.data_type?        ?♥string?♂attribute.name?       product.url?♂attribute.type?    ?♣property?‼attribute.display_name?     ???♣??  ?♠item.type??
?       ?♂??    ?,Product -> Disallow as automated recommendation??♣??  ?↕product.nonreturnable??       ??      ?♀?
?       ?♂??    ?§Product -> Nonreturnable??♣?? ???     ?◄product.repositoryId??        ??↑attribute.search.searchable? ?♂??    ?
Product -> Id????       ?
product.brand???        ?◄?
product.language??      ???  ?  ?¶?tiselect_type?       ?♣multi-or??    ?
?       ?♂??    ?►Product -> language??♣??      ?↕product.priceListPair??       ??      ?▬?
product.keywords??      ?☼?     ?♂??    ?↕Product -> Keyword(s)??♣??    ???     ?§product.adminDisplayName?☼?   ?♂??    ?▼Product -> Management Display Name????        ?↑??    ?♫product.startDate?? 

Solution:
Add the following to jboss-as\server\server-name\conf\jboss-log4j.xml: 

<category name="org.apache.cxf.interceptor.LoggingInInterceptor"> <priority value="WARN"/> </category> <category name="org.apache.cxf.interceptor.LoggingOutInterceptor"><priority value="WARN"/> </category>

http://ravihonakamble.blogspot.com/2015/07/avoiding-cxf-log-entries-in-jboss-log.html

FileNotFound Error message in JBOSS log and site is not showing Endeca Experience Manager pages - Endeca Experience Manager

Problem Statement:

FileNotFound Error message in JBOSS log and web site is not showing Endeca Experience Manager pages

2015-07-25 12:51:40,317 INFO  [nucleusNamespace.atg.endeca.assembler.AssemblerPipelineServlet] (ajp-0.0.0.0-10209-2) No country specific page found. Attempting to assemble non country specific page

2015-07-25 12:51:40,319 ERROR [nucleusNamespace.atg.endeca.assembler.AssemblerTools] (ajp-0.0.0.0-10209-2) A problem occured assembling requested content. Response from the assembler was {contentUri=/pages/browse/_/N-100972, @type=ContentInclude, @error=FileNotFound}

2015-07-25 12:51:40,319 WARN  [nucleusNamespace.atg.endeca.assembler.AssemblerPipelineServlet] (ajp-0.0.0.0-10209-2) Country specific and non-country specific page could not be found. If the url attempting to be served is not meant to be an XM page, consider adding this path to ignore list to avoid unnecessary Endeca assembling. This request will now pass to next servlet in pipeline.

Solution:

Endeca Experience Manager content is not accessible from ATG store instances. ATG and Endeca talks over port 8007 for XM content.

Try below options:

1) Check if 8007 port is opened or not

2) Manually promote content 

       a) Go to control Directory of Endeca application

              /opt/app/endeca/app/CRS/control

       b) Run promote script ./promote_content.sh

How to Enable Endeca property for record filtering in ATG 11.1? - Core Endeca

1) Add the desired property in the /opt/app/endeca/CRS/config/index_config/index_config.json and add "isRecordFilterable" : true in the property.
For example:       
"product.repositoryId" : {
          "propertyDataType" : "ALPHA",
          "mergeAction" : "UPDATE",
          "isRollupKey" : true,
          "jcr:primaryType" : "endeca:property",
          "isRecordFilterable" : true
        },

2) Once you update index-config.json make sure to set configurations at Endeca repository level.
    Here is the command:
   /opt/app/endeca/apps/CRS/control :>
                          ./index_config_cmd.sh set-config -f ../config/index_config/index-config.json -o all
3) Run Endeca indexing

Assembler Code to fetch Endeca Experience Manager and Endeca GuidedSearch content - Endeca Assembler API

Code snippet to invoke Endeca Experience Manager Content:

<dsp:droplet name="/atg/endeca/assembler/droplet/InvokeAssembler">
            <dsp:param name="includePath" value="/pages/browse" />
            <dsp:oparam name="output">
               <dsp:getvalueof var="contentItem" vartype="com.endeca.infront.assembler.ContentItem"         param="contentItem" />
            </dsp:oparam>
</dsp:droplet>

Code snippet to invoke Endeca Guided Search content:

 <dsp:droplet name="/atg/endeca/assembler/droplet/InvokeAssembler">
            <dsp:param name="includePath" value="/services/guidedsearch" />
            <dsp:oparam name="output">
               <dsp:getvalueof var="contentItem" vartype="com.endeca.infront.assembler.ContentItem" param="contentItem" />
            </dsp:oparam>
  </dsp:droplet>

Input Parameters

The InvokeAssembler servlet bean has two input parameters, includePath and contentCollection.

includePath

Use the includePath parameter for a page request.

For example, to assemble content for a browse page, specify /pages/browse for the includePath (passing in a /browse path will not match because it is missing the /pages prefix).

contentCollection

Use the contentCollection parameter for a content collection request.

For example,  Search Box Auto Suggest Content. InvokeAssembler parses the contentCollection into a ContentSlotConfig component.

The includePath and contentCollection parameters are mutually exclusive but one of them must be passed when using the InvokeAssembler servlet bean.

ruleLimit

This optional parameter is used in conjunction with the contentCollection parameter to specify the number of items that should be returned from the specified content collection.

Output Parameters

The InvokeAssembler servlet bean has one output parameter, contentItem. This parameter contains the root ContentItem returned by the Assembler. If this content item is empty, the request was not an Assembler request.

ATG Endeca Data Integration Diagram - v3.1.x

Integration Overview 

The basic workflow diagram to show integration of the ATG product catalog with Endeca.

As seen in the diagram above, the data integration supports the publication of Product or SKU and Category assets directly into Endeca Content Acquisition System (CAS) Record Stores. The developer can control the type and granularity of assets published via the Product Exporter and the Category Exporter.

The Product Exporter behavior is managed by the Product Indexing Configuration file, which specifies the granularity and inclusion of product or SKU assets.

The Category Exporter is managed by the Category Indexing Configuration, which specifies the export of the Category to Dimension.

The Schema Exporter, managed by the Schema Indexing Configuration, exports schema information to Endeca.

Operationally, the publication of ATG assets and creation of the corresponding Endeca index is controlled by the ATG Web Commerce Product Catalog Indexing Administrator (Indexing Admin). The Indexing Admin invokes the Endeca Application Controller (EAC), which controls the creation and publication of the Endeca index throughout your production or staging environment.

The EAC, via configurable Endeca Deployment Template modules, controls the orchestration of the Endeca index creation process. Further, the EAC orchestrates the publication of the built Endeca index to all configured production or staging servers and ensures that the Endeca MDEX is running.

Indexing Using ATG Dynamo Administration UI  

Navigate to the ProductCatalogSimpleIndexingAdmin component to invoke the Endeca baseline Indexing process or partial update where localhost and 8080 are the host and HTTP port of your production ATG instance (or publishing instance, if you are using publishing).

More details on Indexing:
http://ravihonakamble.blogspot.com/2015/06/how-to-read-data-from-endeca-cas-record.html