Evaluating heuristics

Always stay up-to-date on your heuristics

We constantly analyze how well your heuristics are doing, no matter what type they are. Once you execute a heuristic - and there is some manually labeled data we can use for evaluation - you will find a statistic like this at the bottom of your heuristics page:

28802880

It shows you per label the relevant data for you to know. The values have the following meaning:

  • est. precision = true positives / (true positives + false positives) for the reference data you labeled.
  • est. recall (only for extraction tasks) = true positives / (true positives + false negatives) for the reference data you labeled.
  • coverage: how many records does this heuristic generally hit?
  • hits (only for extraction tasks): how many spans are hit by this heuristic?
  • conflicts: on how many records (or spans) does this heuristic create conflicting expressions to other heuristics? (E.g. heuristic A says record 1 is "positive", while heuristic B says it is "negative").
  • overlaps: on how many records (or spans) does this heuristic create overlapping expressions to other heuristics? (E.g. heuristic A says record 1 is "positive", and so does also heuristic B).

You can also find the precision and coverage for each heuristic on the heuristics overview page.

28802880

Did this page help you?