The keyword delimiter of an attribute is used to split the value of that attribute into keywords or 'tokens'. It is used both for "Keyword search links on view" and when tokenising the attribute value before adding it to the search index. If it is not set then values are tokenised on word boundaries when they are added to the search index. Setting a keyword delimiter can make it possible to search for things that it would otherwise be impossible to search for. For example "2009-2010" would usually be tokenised as two words, "2009" and "2010" which would make it impossible to search for "2009" without also finding results for "2009-2010". Setting the delimiter to "," would mean that searches for an attribute value of "2009" (including quotes) would not include results where the attribute value was "2009-2010".
If you require an attribute's value to be used for running searches from an item's details page, you need to decide whether the whole value, or parts of the value should be used as search 'tokens'. If you wish the entire value to be used as a single search term, no delimiter is required and this field can remain empty. If you require the value to be broken up into separate search terms, the delimiter will tell the system where one term finishes and another starts.
The attribute's value will be split around matches of a 'regular expression' which this field contains. This could be a single character such as a comma or space, or a much more complicated expression. If you are familiar with regular expressions then you just need to be aware that the syntax should conform to Java's java.uti.regex.Pattern class's implementation.
If you are not comfortable with regular expressions and wish to make use of this feature, it is safest to contact Bright Interactive for assistance.