Evaluating heuristics
Always stay up-to-date on your heuristics
We constantly analyze how well your heuristics are doing, no matter what type they are. Once you execute a heuristic - and there is some manually labeled data we can use for evaluation - you will find a statistic like this at the bottom of your heuristics page:
It shows you per label the relevant data for you to know. The values have the following meaning:
est. precision
=true positives / (true positives + false positives)
for the reference data you labeled.est. recall
(only for extraction tasks) =true positives / (true positives + false negatives)
for the reference data you labeled.coverage
: how many records does this heuristic generally hit?hits
(only for extraction tasks): how many spans are hit by this heuristic?conflicts
: on how many records (or spans) does this heuristic create conflicting expressions to other heuristics? (E.g. heuristic A says record 1 is "positive", while heuristic B says it is "negative").overlaps
: on how many records (or spans) does this heuristic create overlapping expressions to other heuristics? (E.g. heuristic A says record 1 is "positive", and so does also heuristic B).
You can also find the precision and coverage for each heuristic on the heuristics overview page.
Updated 3 months ago