Open Hex's agent to one data domain first, and widen access only after it answers that domain's test questions correctly. Hex's documentation and the Chime and LangChain rollouts support this order.
When you need this
- The data team spends most of its week on ad hoc questions from the business.
- Leaders want business users asking data questions in Slack or in their AI assistant.
- A pilot of Threads produced answers people questioned, and the rollout stalled.
Evaluation criteria
Setup: scope what the agent sees
Which connections, schemas, and tables should the agent search?
Look forWorkspace data connections with schema filtering, descriptions on tables and columns, a scheduled schema refresh, and a default connection set so the agent searches it first.
Setup: curate one domain
Which tables and projects answer this domain's questions, and who maintains them?
Look forEndorsed data and projects, workspace rules, and a semantic model authored in Hex or synced from dbt MetricFlow, Cube, or Snowflake Semantic Views. Admins and Managers own these. Name one owner per domain.
Rollout: Endorsed Mode first
Will business users see only endorsed assets?
Look forExplorers kept in Endorsed Mode, the default, so Threads answers only from endorsed data and projects. Turn it off for a team only after its domain passes evals.
Governance: gate on evals
What must the agent answer correctly before a domain launches?
Look forAn eval suite per domain, written in YAML, with test questions and rubrics that grade the final answer and the tables and guides used. Run it from the CLI and schedule it after launch. Chime launched a domain only when Hex's agent passed all three levels of its eval set.
Review practice: watch where it fails
Who reads the warnings and topics each week?
Look forA weekly pass through Context Studio's topics, warnings, and user feedback, with Suggestions accepted or rejected by a Manager. LangChain's analytics engineers moved from building reports to curating context this way.
Cost control: limits before scale
What stops one team's questions from draining the credit pool?
Look forCredit limits by user, group, and workspace default, plus an auto top-up spend limit, set before Slack and MCP access widen the audience.
Questions to ask vendors
- Which data domain has the most repeated questions and the cleanest models to start with?
- Who owns each domain's context once it launches, and how often do they review it?
- Which connections are sensitive, and should they be reachable from Slack and MCP at all?
- What share of answers must pass before a domain opens to the whole company?
Red flags
- Threads opened to everyone with Endorsed Mode off and nothing endorsed.
- No eval suite, so the first sign of a wrong metric is a leader quoting it.
- Metric definitions confined to dashboards, absent from a semantic model or guide.
- Slack and MCP access enabled before sensitive connections are marked.
Products in this piece
Sources
- Official documentation
- 10
- Public source
- 5
- Hex docs: Data connections introduction
- Hex docs: Introduction to semantic modeling
- Hex docs: Threads
- Hex docs: Context Studio
- Hex docs: Evals
- Hex docs: Agent observability
- Hex docs: Credits
- Hex docs: Sensitive data connections for external integrations
- Hex customer story: Chime (Aug 4, 2026)
- Hex customer story: LangChain (Jun 25, 2026)
- LangChain: How we built LangChain's agent-first data stack (Jul 27, 2026)
- Hacker News comment from a Hex workspace admin on context feedback (May 13, 2026)
- Reddit post from an analyst using Claude and Hex on a semantic layer (Aug 9, 2026)
- G2 review from a head of data on Hex's agents and Explorer seat price (Nov 24, 2025)
- GitHub issue: Sentry's Slack agent waiting on Hex Threads (Sep 25, 2026)