How to test Endeca response content returned on the browser? - Endeca

There are two formats to render Endeca response content and that is XML or JSON.

Here is the URL structure:

For XML response content use parameter format=xml and for JSON content use format=json

http://HOST:PORT/browse/sweaters/women/_/N-27bh?format=xml

http://HOST:PORT/browse/sweaters/women/_/N-27bh?format=json

Endeca CAS | Multiple Record store merge - Oracle Commerce 11.1

Multiple record store could be used to merge data. This merging would be work as switch join.  Find out the steps below to use multiple record store :-

 1. Open <Endeca App Path>\APPNAME\config\cas\last-mile-crawl.xml file and add all record stores. Find out sample below. Multiple record store is highlighted below. We can add any number of record stores to merge.
                                <sourceConfig>
                                                <moduleId>
                                                                <id>com.endeca.cas.source.RecordStoreMerger</id>
                                                </moduleId>
                                                <moduleProperties>
                                                                <moduleProperty>
                                                                                <key>dataRecordStores</key>
                                                                                <value>APPNAME-data</value>
                                                                                <value>APPNAME-web-crawl</value>
                                                                </moduleProperty>
                                                                <moduleProperty>
                                                                                <key>dimensionValueRecordStores</key>
                                                                                <value>APPNAME-dimvals</value>
                                                                </moduleProperty>
                                                </moduleProperties>
                                                <excludeFilters />
                                                <includeFilters />
                                </sourceConfig>
  2. Run following command to update configuration in CAS
a.       <Installpath>\CAS\11.1.0\bin >cas-cmd.bat updateCrawls -f <Endeca App Path>\APPNAME\config\cas\last-mile-crawl.xml
  3.  Run Baseline.

Note :- Endeca CAS uses record.id as unique identifier. We can define our own. Suppose two record appears in record store A and B with same record id . CAS would discard one of the record.

Endeca Select feature aka set Selection - Controlling Endeca Record Values

The Select feature allows you to select specific keys (Endeca properties and/or dimensions) from the data so that only a subset of values will be transferred for Endeca records in a query result set.

This functionality prevents the transferring of unneeded properties and dimension values when they will not be used by the front-end Web application.

It therefore makes the application more efficient because the unneeded data does not take up network bandwidth and memory on the application server.

There are two possible scenarios:
1)  If you are making independent Endeca queries then you can use below API:

// Create a query
ENEQuery usq = new UrlENEQuery(request.getQueryString(),"UTF-8");
// Create an empty selection list
FieldList fieldList = new FieldList();
// Add an Endeca property to the list
fieldList .addField("P_DisplayName");
// Add an Endeca dimension to the list
fieldList .addField("P_RepositroyId");
// Add the selection list to the query
usq.setSelection(fieldList );
// Make the MDEX Engine query
ENEQueryResults qr = nec.query(usq);

2)If you are using ATG-Endeca integration then use below component and add only required fields.
 /atg/endeca/assembler/cartridge/handler/config/ResultsListConfig.properties
 fieldNames=\
  P_DisplayName,\
  P_RepositroyId


Endeca Derived properties to get Min and Max price for Sale/Clearance/List/MSRP prices - Endeca Features

A derived property is a property that is calculated by applying a function to properties or Dimension values from each member record of an aggregated record. Derived properties are created by Forge, based on the configuration settings in the Derived_props.xml file. After a derived property is created, the resultant derived property is assigned to the aggregated record.

Problem Statement:
Consider one product with 5 SKU's and each SKU has sale,List,clearance and MSRP price tagged and requirement is to traverse through all the prices tagged on SKU's and get the MIN and MAX for sale, Clearance, List  and MSRP.


Solution:
1) Change Derived_props.xml file in pipeline

/opt/app/endeca/apps/CRS/config/pipeline/Derived_props.xml
<?xml version="1.0" encoding="UTF-8" standalone="no" ?> <!DOCTYPE DERIVED_PROPS SYSTEM "derived_props.dtd"> <DERIVED_PROPS>
  <DERIVED_PROP DERIVE_FROM="sku.listPrice" FCN="MAX" NAME="P_SKU_ListPrice_Max"/>
  <DERIVED_PROP DERIVE_FROM="sku.listPrice" FCN="MIN" NAME="P_SKU_ListPrice_Min"/>
  <DERIVED_PROP DERIVE_FROM="sku.salePrice" FCN="MAX" NAME="P_SKU_SalePrice_Max"/>
  <DERIVED_PROP DERIVE_FROM="sku.salePrice" FCN="MIN" NAME="P_SKU_SalePrice_Min"/>
  <DERIVED_PROP DERIVE_FROM="sku.MSRP" FCN="MAX" NAME="P_SKU_MSRP_Max"/>
  <DERIVED_PROP DERIVE_FROM="sku.MSRP" FCN="MIN" NAME="P_SKU_MSRP_Min"/>
  <DERIVED_PROP DERIVE_FROM="sku.clearencePrice" FCN="MAX" NAME="P_SKU_ClearencePrice_Max"/>
  <DERIVED_PROP DERIVE_FROM="sku.clearencePrice" FCN="MIN" NAME="P_SKU_ClearencePrice_Min"/>
</DERIVED_PROPS>

2) Run baseline index
3) Check reference application for Derived properties
Endeca reference Application:

8056678  (5 Records)
DERIVED PROPERTIES:
P_SKU_ListPrice_Max:  39.990000
P_SKU_ListPrice_Min:  38.990000
P_SKU_SalePrice_Max:  39.990000
P_SKU_SalePrice_Min:  38.990000
P_SKU_MSRP_Max:  55.000000
P_SKU_MSRP_Min:  54.000000
P_SKU_ClearencePrice_Max: 
P_SKU_ClearencePrice_Min: 
REPRESENTATIVE REC PROPERTIES:
allAncestors.repositoryId:  cat130021
allAncestors.repositoryId:  cat130052
allAncestors.repositoryId:  cat130155
allAncestors.repositoryId:  rootCategory
FYI - Derived properties can be only created on Aggregated records.
Valid functions are MIN, MAX, AVG, or SUM

Endeca concurrent write exception - atg.repository.search.indexing.IndexingException: com.endeca.itl.recordstore.ConcurrentWriteException???

Problem:You will see below exception in Jboss log
ERROR [ProductCatalogSimpleIndexingAdmin]
atg.repository.search.indexing.IndexingException: com.endeca.itl.recordstore.ConcurrentWriteException: Write in progress with generation 40
at atg.endeca.index.RecordStoreDocumentSubmitterSessionImpl.beginSession(RecordStoreDocumentSubmitterSessionImpl.java:225)
at atg.endeca.index.AbstractRecordStoreAggregateSession.getSessionForKey(AbstractRecordStoreAggregateSession.java:300)
at atg.endeca.index.AbstractRecordStoreAggregateSession.getSessionForContext(AbstractRecordStoreAggregateSession.java:249)
at atg.endeca.index.AbstractRecordStoreAggregateSession.getSessionForCurrentContext(AbstractRecordStoreAggregateSession.java:238)
at atg.endeca.index.AbstractRecordStoreAggregateSession.submitRecord(AbstractRecordStoreAggregateSession.java:334)
at atg.endeca.index.LocaledExporter.exportRecords(LocaledExporter.java:295)
at atg.endeca.index.LocaledExporter.export(LocaledExporter.java:258)
at atg.endeca.index.LocaledExporter.export(LocaledExporter.java:208)
at atg.endeca.index.LocaledExporter.performBaselineUpdate(LocaledExporter.java:345)
at atg.endeca.index.admin.IndexingTask.doTask(IndexingTask.java:401)
at atg.endeca.index.admin.IndexingTask.performTask(IndexingTask.java:359)
at atg.endeca.index.admin.IndexingPhase.performPhaseTasks(IndexingPhase.java:265)
at atg.endeca.index.admin.IndexingPhase.performPhaseTasksSerially(IndexingPhase.java:242)
at atg.endeca.index.admin.IndexingJob.performJobSerially(IndexingJob.java:264)
at atg.endeca.index.admin.SimpleIndexingAdmin.indexBaseline(SimpleIndexingAdmin.java:657)
at atg.endeca.index.admin.SimpleIndexingAdmin.indexBaseline(SimpleIndexingAdmin.java:636)
at atg.endeca.index.admin.SimpleIndexingAdmin$2.run(SimpleIndexingAdmin.java:1008)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.endeca.itl.recordstore.ConcurrentWriteException: Write in progress with generation 40 at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:141) at $Proxy154.startTransaction(Unknown Source) at atg.endeca.index.RecordStoreDocumentSubmitterSessionImpl.beginSession(RecordStoreDocumentSubmitterSessionImpl.java:217)

Solution:
If you stop Endeca indexing from ATG(/atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin/) before it completes then it will acquire lock and then subsequent baseline updates will fail with above exception.

Solution#1: Restarting Endeca CAS which will remove the lock acquired - Temporary fix

Solution #2:  Make sure you restart CAS after every baseline update to avoid this issue plus some other issues related to CAS

Add /control/initialize_RS_services.sh script at the end of  /control/baseline_update.sh script - It will delete all the records stores and create new one each time and also helps to improve some level of performance on CAS side

Solution #3: Set resetActive flag to true on following components to rollback any previously active transactions
/dyn/admin/nucleus/atg/endeca/index/DataDocumentSubmitter/?propertyName=resetActive

/dyn/admin/nucleus/atg/endeca/index/DimensionDocumentSubmitter/?propertyName=resetActive
/dyn/admin/nucleus/atg/endeca/index/SchemaDocumentSubmitter /?propertyName=resetActive

How to check endeca process(MDEX, Platformservices, MDEX, CAS etc..) running on the box? - Endeca Process

Run below command:
ps -ef | grep endeca
You should see all below process running else need to restart manually.
 1) MDEX(Dgraphs)  Ex: /opt/app/endeca/MDEX/6.5.1/bin/dgraph
2) Platform Services Ex: /opt/app/endeca/PlatformServices/11.1.0/j2sdk/bin/java
3) Tools and Framework Ex: /opt/app/endeca/ToolsAndFrameworks/11.1.0/server/j2sdk/bin/java

How to check CAS is running?
ps -ef | grep cas  
You should see process with /bin/bash ./cas-service.sh  if not then restart CAS.

How to read data from Endeca CAS record store instance, Run ATG CMS and Endeca baseline update through Dynamo admin? - ATG Endeca Integration Issues

If you are not sure whether the ATG records are pushed at Endeca CAS level or not, then read the record store from the CAS by running below script.

/opt/app/endeca/CAS/version/bin>./recordstore-cmd.sh read-baseline -a <appname_en_data> -f <new-file-path>.xml

Ex. - /opt/app/endeca/CAS/version/bin>./recordstore-cmd.sh read-baseline -a APPNAME_en_data -f ../../APPNAME_en_data.xml

It will help to cross verify if any record is not showing in Endeca but merchandized in BCC and deployed. Sometime CMS(Catalog Maintenance Service) fails and does not pass latest records to Endeca. Try manually running CMS and Endeca Indexing.

Running ATG CMS(Catalog Maintenance Service):

http://localhost:port/dyn/admin/atg/commerce/admin/en/maintenance/startService.jhtml?process=BasicMaintProcess

Running Endeca Baseline update:
http://localhost:port/dyn/admin/nucleus//atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin/