Evaluating answer engines — a half-day test that beats reading price tables

Three lines

• Human verification of citations assumed → Perplexity; weighting Japanese-source retrieval → Felo; wanting a deliverable at the end → Genspark
• Choosing on price will mislead you — pricing changes often and second-hand figures are unreliable
• The fastest decision is twenty of your own work questions sent to all three, scoring citation quality

1. What makes an answer engine different

Conventional search returns links and you read what you clicked. A chat model answers from learned knowledge, where provenance is vague. An answer engine sits between: it retrieves pages and summarises while showing them as the basis. Answer and evidence arrive together, which means it takes over the reading rather than the thinking.

Which also means that if the pages it retrieved are off-target, the answer is off-target. The quality of an answer engine is decided by what it chose as evidence, not by how well it writes. Evaluating on that basis makes the differences between services obvious.

2. Where the three stand

Perplexity

The service that popularised the category. Answers carry numbered citations, so tracing which sentence came from which page is straightforward. It is dependable on English-language material, which suits research against international primary sources. Questions in other languages work, but retrieval can skew English, so domestic subject matter needs the verification described below.

Felo

Built in Japan with Japanese-language use as the default path. It retrieves Japanese sources for Japanese queries and can assemble results into slides and documents. On domestic topics the set of sources it reaches for differs noticeably — putting the same question to both makes that visible immediately.

Genspark

Leans toward assembling a deliverable from what it found rather than stopping at the answer. That suits a flow from research to document. Processing is tied to credit consumption, so heavy use needs managing; for high-frequency fact checks the other two can be easier to live with.

Note: all three revise pricing and plan names frequently, and published comparisons disagree on the figures. That is why no prices appear here — check each vendor's page on the day you decide.

3. Five axes that matter at work

4. The half-day procedure

• 1. Write twenty questions you actually asked at work in the last month
• 2. Put all twenty to each service, unchanged
• 3. For each answer, open the citations and mark whether the page supports the claim
• 4. Count, per service, how many of the twenty had valid citations
• 5. Note separately how many citations were roundups rather than primary sources

Step five is the one that separates them. A service that answers fluently from secondary sources looks good and fails on anything time-sensitive. Counting primary-source citations predicts how much verification work you inherit.

Frequently asked questions