Efficient Product Importer in Sitecore – Part 2

In previous part of the tutorial we created basic product importer which periodically checks products from external Product Information Management (PIM) system and stores them as Sitecore items. The implementation could be further improved by the usage of Sitecore item buckets and search indexes which improves the performance for large number of data.

To better reflects the structure of products from PIM, we use custom Sitecore item bucket, with following structure of product repository:

  • Active products
    • A (all products with name starting with A)
    • B (all products with name starting with B)
  • Inactive products
    • A (all products with name starting with A)
    • B (all products with name starting with B)

Extend Data Model

We assume that product status (active vs inactive) is a flag imported from PIM system. Let’s start with extending our data model, created in previous part, by adding new checkbox field for the status:

We add corresponding property to our model class:

Change Product Repository to Item Bucket

Now let’s select our “Product” template’s standard value item and make it “bucketable”, by checking following fields:

Setting “Bucketable” means that all “Product” items can be stored in item bucket. We checked also “Lock child relationship” which we’ll be use later.

By default Sitecore stores items in buckets using structure based on item’s create date. We want to change this, so it will mirror the structure of our products from PIM system. This can be done by creating custom bucketing rule action item under /sitecore/system/Settings/Rules/Definitions/Elements/Bucketing node. We fill “Text” field with content editor’s friendly message and “Type” field, where we point to our custom class in our assembly.

Now we need to create custom RuleAction class. The class must have “Apply” method which defines each level of item bucket structure. For first level we based on “Active” field. Second bucket level is generated using first letter of product name:

To join the rule action with our “Product” items we need to edit /sitecore/system/Settings/Buckets/Item Bucket Settings node. We select rule “where the new bucketable item is based on the … template”, choose “Product” template and assign the action created in previous step:

Now we can transform “Product Repository” into item bucket. This is done in “Configure” tab by clicking “Bucket” button, while the repository item is selected in the content editor tree. If we already have some products in the repository, we may need to click “Sync” button, so our bucket will have proper structure.

Change Product Importer to Use Buckets

Assuming we will have lot of products, we want also to improve import mechanism. We can do it by replacing existing products search done directly on database, with search indexes. We take the index of bucket’s item, build predicate and perform the search.

We also need a method, which will synchronize the bucket, so every new item will be correctly placed in our item bucket custom structure:

Now let’s use those methods in our importer code and replace searching for existing products with “SearchBucket” method and add “SyncBucket” at the end of the importing. Mind that we do this only if there are any new or modified products.

We need also modify Map method a little bit, to include product status flag:

After the scheduled task completed we should see our products in new bucket structure:

In previous step, we checked “Lock child relationship” field in “Product” template’s standard values. This allows us to create subitems under the items which are “bucketable”, so for example our products could have variants:

Mind that you don’t need to check “Bucketable”, or “Lock child relationship” fields in subitems.