Saturday 22 August 2015

Solr settings and full indexing (Attribute task update)

SOLR is mainly used for :
  • Indexing
  • Querying
   SOLR is a product of apache and having the inbuilt server named jeety.
Mainly the SOLR has the data types(normal java)that has to be mapped in the solr manner where these are called as dynamics fields(SOLR specific).
       Data Types = dynamic fields
Example : <field name ="catentry_id" type="string" indexed="true" stored="true" required = "true"                         multiValued=£false"/>
Here in the above example :
  • type is the solr datatype.
  • name is the WCS datatype (Field name of table).
  • indexed must be true (used for indexing in solr search).
  • stored  mainly used to store the indexed field.
     If we want to customize by adding any table data that need to be indexed then we need to update the schema.xml.
      In the schema.xml we will find the file names like stopwords.txt and protwords.txt which files are responsible for the words that can be skipped during the search filtering process.
      Steps to create the SOLR setup:
  1.       setupSearchIndex.bat :
           <toolkit-home>\components\foundation\subcomponents\search\bin\setupSearchIndex.bat
Syntax :
       D:\IBM\WCDE_ENT70\bin>setupSearchIndex.bat -masterCatalogId <mastercatalog_id>
     Actually the search folder gets created in the home directory which contains the following files mentioned below:
  •     solrconfig.xml (requestHandlers, search components).
  •      wc-data-config.xml ( Mainly for establishing the server connections, Solr Data Import Handler configuration for DB config).
  •     Schema.xml (mainly used for mapping the tables and fields that has to be indexed by solr)
          Make sure the server is in up and running when running this setupSearchIndex.bat
        
         2.di-preProcess.bat :
          <toolkit-home> \components\foundation\bin\di-preprocess.bat
       Syntax :
           D:\IBM\WCDE_ENT70\bin>di-preprocess.bat D:\IBM\WCDE_ENT70\search\pre-   processConfig\MC_<mastercat_id>\Cloudscape -localename en_US
       Actually this is used to push the data into the temporary table after the index has took place.
       3.di-buildIndex.bat :
        <toolkit-home> \components\foundation\bin\di-buildIndex.bat
          D:\IBM\WCDE_ENT70\bin>di-buildindex.bat -masterCatalogId <mastercat_id> -localename en_US
      Actually this step is the final batch that we run during the whole solr indexing process.    Especially this is mainly meant to get the indexing completed.
     The following url is used for testing the solr indexing is done properly or not is :
      Main files to be remembered for SOLR update are below mentioned:
   ·  D:\IBM\WCDE_ENT70\search\solr\home\MC_10351\en_US\CatalogEntry\conf\solrconfig.xml
   ·  D:\IBM\WCDE_ENT70\search\solr\home\MC_10351\en_US\CatalogEntry\conf\wc-data-   config.xml
   ·  D:\IBM\WCDE_ENT70\search\solr\home\MC_10351\en_US\CatalogEntry\conf\schema.xml
   ·  D:\IBM\WCDE_ENT70\search\pre-processConfig\MC_10351\Cloudscape\ALL XML files related to indexing of solr
   ·  The whole files (esply the xml) and folders (search present in toolkit home location) that are created during the process of setupSearchIndex.bat running are already present and in the  sample folder present in the below mentioned path
      D:\IBM\WCDE_ENT70\components\foundation\samples 
Task related to attribute dictionary model
    1.     Start the management center
    2.     Login to CMC and select the store(AURORA) from the drop down.
    3.     Select the catalog and select the attribute dictionary
Create a new attribute under this section and give the required values.
NOTE: Here keep in mind while creating the attribute dictionary attribute always select the attribute dictionary attribute with the predefined value.
         Under this step 3 we need fill the attribute values information with any value (for    EX: sequence can be n value and value section with any text value).
           Remaining fields can be left free.
          Once done select save and close.
       SQL select * from attr where identifier='<Name given in the step 3 for attribute >';
       SQL select * from attrval where attr_id= <select the attr_id which we got from the above sql>;
    4.     Now select a product to associate the create attribute ( step 3 ) for a specific product     for getting the catentryattr table updated with the associated attribute values.
       SQL : select * from catentryattr where attr_id=<select the attr_id which we got from the above sql>;
     Here from the above query we can see that the catentryattr table is being updated with  the associated attribute that we created in the step 3 above.
    5.     Now let’s check the solr part once the above steps are completed successfully, for   this we need to run the preprocess to get the temporary table updated with the above created table
     For the checking of above step lets verify the date that’s being updated in the attribute  related temporary date given in the OOB solr indexing process.( i.e. TI_ATTR_0_#lang_tag# where the lang id has to be mentioned appropriate and this table name we can get it from the preprocess xml files that are created during the setupSearchIndex.bat file running that is the search folder content (wc-dataimport-preprocess-attribute.xml) where we can see the temporary tables creation queries.)
     NOTE : while running this running this preprocess.bat keep the server in stop mode.
    6.     Now if the preprocess is completed with no errors then check the TI_ATTR_0_#lang_tag# table whether the date has been updated with the appropriate   value that was given for temporary table update.
       SQL : select * from TI_ATTR_0_1;
     You will find the updated data with field being updated with the name like
         'ads_f10501: <Value given in step3>||ads_f10501_ntk_cs:<Value given in step3>'
    7.     With this above correct value update in the temporary table we can conclude  preprocess has been done correctly.
    8.     Now the final step is buildindex in this we need to go for full build so that the data is    pushed to solr search engine so that we can retrieve when we search in the site. J J
    9.     Once the di-buildIndex.bat step is completed with no error codes. Then we need check the indexed data by running the below url in the browser.
    We can find the data indexed properly with details as expected with this the solr indexing for attribute dictionary task has been updated J J


2 comments:

  1. Hey keep posting such good and meaningful articles.

    ReplyDelete
  2. It was very useful for me. Keep sharing such ideas in the future as well. This was actually what I was looking for, and I am glad to came here! Thanks for sharing the such information with us.

    ReplyDelete

Solr settings and full indexing (Attribute task update)

SOLR is mainly used for : Indexing Querying    SOLR is a product of apache and having the inbuilt server named jeety. Mainly the ...