Auto Tagging Spidering Wrong Information or Tags


If you're leveraging Sailthru's Auto Tagging functionality there may be times where you find that it has spidered incorrect information or tags that actually belong to a different article or web page. This happens because the feature attempts to determine information such as the title and description of the page itself and ignores any Sailthru meta tags you may have defined. There isn't currently an option to allow a combination of user-defined metatags + auto-tagged tags - it's one or the other.


Content API

The content API can be used to update any incorrect information you have identified on an item on a case by case basis. For example, if you've noticed that an item has been spidered with an incorrect title you could update the item with a POST call to the content API:

    "id": "",
    "title" : "The correct title",
"spider" : 0   

Note that it's important to pass "spider" : 0 so that the item is not automatically spidered again after the update as this would undo the correction being made.

Any of the optional parameters available via the POST mode of the content API can be updated in this way. This workaround is good for any one-off or small amounts of affected articles due to the manual intervention required.

Spider Extra Meta Tags

It's possible to leverage the Spider Extra Meta Tags functionality of the Personalization Engine in conjunction with Auto-Tagging to allow meta-tags that would normally only be spidered with Auto-Tagging disabled, to be spidered under the "vars" section of the item. This functionality would normally be used to spider additional custom tags which the Personalization Engine does not spider however, in this case, it can be used to spider the standard Sailthru tags which should be already set up on the item.

For example, if an item has been spidered by Auto-Tagging with incorrect information, it's still possible to pull the correct values from the standard Sailthru meta tags:


These tag names should be added to the Spider Extra Meta Tags field on the Spider Settings page:


And once the Personalization Engine spiders the item, these values become available under the vars section of the item which can be accessed from any Sailthru feed the item appears in:


These can then be accessed in a template or campaign using Zephyr in the same way described under Section 4 of the Extra Meta Tags article:


This workaround works well for ongoing issues with incorrect information being spidered or a large number of affected pages due to automatic handling and spidered of the custom tags.