Testing Hive

A prompt for AI agents · Scope as of 5 October 2026

You are testing Hive, an MCP server that picks a tool for a request, predicts how it will fail, calls it, and verifies the result. Test it only on what it covers today, described below, and judge each part separately.

What Hive is

Hive is a router over other people's MCP tools, not a tool collection of its own. It indexes connectors from a public registry of MCP connectors, and for each request it does four things.

1. Selects the best tool among the tools it can call, or stops and says why when none fits.
2. Predicts the call's chance of success and its most likely failure modes, before the call.
3. Calls the tool with the arguments you fill in. Hive never writes arguments itself.
4. Verifies the result against the request, on a satisfaction scale from 0 to 3 with a failure mode, and records it so later predictions improve.

Hive is in early testing. Your job is to find where these four steps go wrong, on requests that Hive's connectors can actually serve.

What Hive covers today

Hive can only call tools on connectors that need no sign-in. Today that is about 350 tools on 42 connectors, out of 639 tools on 60 connectors indexed.

Hive cannot call Gmail, Google Calendar, Slack, GitHub, Notion, or any service behind a personal account or API key. Do not test it with requests that need one. When Hive stops on such a request and names tools that need authentication, it is working as designed: record it as out of scope, not as a failure.

Test with requests in these areas, which the callable connectors serve:

Some connectors charge per call through x402 or keep state for a sign-up, such as AI-Mail, Genesis402, MyPenny and Steledger. Their calls may fail for payment or account reasons that Hive does not control.

Requests known to work

• "If I invest $10,000 at 7% a year for 20 years, what does it grow to?"
• "I have $18,000 saved and spend $3,200 a month. How many months will my savings last?"
• "Which vendor makes the network card with MAC address 00:1A:2B:3C:4D:5E?"
• "Run an AI search readiness audit on https://example.com"
• "Find Pakistani court judgments about bail in narcotics cases"
• "Check whether the MCP server at https://gate.horizonshield.dev/mcp is conformant"

Write your own requests too, in plain words, the way a user would ask. Mix easy ones with near misses, such as a request with a required value left out.

The three tools

search_tool(request)

Pass the user's request in their words. It returns one of two shapes.

• A pick: tool (its tool_id, description, connector, bucket, similarity and reliability), risk, input_schema, request_fit (how the request lines up with the input the tool needs), profile (the prediction), up to four alternatives, the mentions its description names, routing_notes, first_steps, hints and timings_ms.
• A stop: tool is null and stopped_because says why, such as "the tools that fit best need authentication". Treat a stop as an answer, not an error.

The profile holds basis (where the numbers come from: the tool's own calls, its bucket's calls, or nothing yet), success_probability, reliability, expected_satisfaction_score from 0 to 3, and up to three likely_failure_modes with probabilities.

select_tool(tool_id)

Looks up one tool, for example an alternative from a search. It returns the tool's details, risk and input schema, its profile across all recorded calls, its mentions, the routing_edges its description states (each with the exact excerpt), the other tools on its connector as siblings, and hints. A tool that cannot be called says why in excluded_because.

call_tool(tool_id, request, arguments, confirm_destructive=false)

Calls the tool and verifies the result. Pass the same request you searched with. It returns result (the tool's own output), completed, tool_reported_error, transport_error, prepared_by_search, next_steps, hints, and an outcome:

How to judge Hive fairly

Separate Hive's mistakes from everything else. Each step can fail on its own, and a connector's bug is not a routing bug.

Hive sometimes returns the closest callable tool for a request it cannot serve. For example, "send an email to [email protected]" picks AI-Mail's wait_for_email, which reads an inbox instead of sending. Report that as a selection mistake. Do not then call the tool and also count its result against Hive.

Most profiles have few recorded calls, so basis is often bucket or empty and the probabilities sit near their starting values. A wrong prediction on little evidence is expected; note the basis when you report one.

Tips

• Read hints first. They name required inputs, confirmations and better alternatives.
• Never invent arguments. Fill them only from the request or an earlier result. If a required value is missing, ask, or record that Hive flagged it.
• Search, then call within 10 minutes with the same request and tool. The call then reuses the search's prediction, so the outcome is judged against what you saw.
• Use select_tool to compare an alternative or a mentioned tool before you call it. A tool id is namespace/slug/name.
• Do not retry failed calls blindly. Hive never retries a call, since a retry could repeat a change.
• Keep destructive calls out of tests unless the user agreed to them.

What to report

For each request you tested, report one line with these parts:

1. The request, word for word.
2. The tool you expected, or "none callable".
3. Hive's pick or stop, with the tool_id.
4. The outcome's failure_mode and satisfaction_score, if you called it.
5. Your judgment from the table above, and one sentence of evidence.

Finish with the number of requests in each judgment, and the mistakes you would fix first.