Evals · run on every deploy

The assistant, measured in public.

Every build runs a golden set of 45 on-topic questions and 13 off-topic ones against the exact BM25 index the Worker serves. If retrieval regresses, it shows up here first.

corpus bb1f69edff80 · 184 chunks · score floor 2.5 · coverage floor 0.65 · ran

Golden set · 45/45 passing

On-topic questions

A case passes when an expected page ranks in the top three and the refusal gate lets the question through.

QuestionTop pageRankScoreCoverageResult
What did he build for State Street?work/state-street-vm-factory-image-bakery28.221.00pass
Disaster recovery with Azure Site Recoverywork/state-street-azure-landing-zone-dr111.791.00pass
Data centre migration to Azure for State Streetwork/state-street-azure-landing-zone-dr116.831.00pass
How many VMs did he build for State Street?work/state-street-azure-landing-zone-dr110.491.00pass
Splitting a large Terraform state into workspaceswork/state-street-terraform-enterprise-split112.430.71pass
How many Terraform resources did he split per application?work/state-street-terraform-enterprise-split113.881.00pass
Moving Terraform runs into Terraform Enterprise and pipelines to Harnesswork/state-street-terraform-enterprise-split111.351.00pass
What is the VM factory?work/state-street-vm-factory-image-bakery17.801.00pass
How much faster are VM builds with the golden image?work/state-street-vm-factory-image-bakery112.630.66pass
Golden image bakery with Azure Image Builder or Packerwork/state-street-vm-factory-image-bakery118.931.00pass
Crédit du Maroc GitLab upgradeprofile/experience/gitlab123.041.00pass
Does he work with GitLab Professional Services?profile/experience/gitlab112.151.00pass
How much did the Five Below audit save?work/five-below-azure-landing-zone-audit112.510.63pass
Blue/green deployments at PlanHubwork/planhub-aws-platform115.491.00pass
Azure DevOps to Harness migration for Mott MacDonaldwork/ado-to-harness-migration123.901.00pass
Reusable GitLab CI templates with OIDCwork/gitlab-ci-platform-templates118.161.00pass
DevOps for an agentic benchmark platform on Cloud Runprofile/experience/turing114.131.00pass
Sandbox environments for training computer-use AI agentswork/agentic-benchmark-devops-ai-eval127.791.00pass
What did he do at Turing?work/agentic-benchmark-devops-ai-eval16.011.00pass
Was he an SRE at PlanHub?profile/experience/planhub114.061.00pass
Kubernetes platforms for RAK Bank and DU Telecomwork/gcc-kubernetes-platforms124.291.00pass
Migrating 30,000 websites to EKSwork/eks-30000-site-migration117.390.77pass
What is TalentLens?projects/talentlens15.321.00pass
Agentic paper trading systemprojects/agentic-paper-trader116.180.77pass
Local RAG platform retrieval evaluationprojects/local-rag-agent-platform116.060.78pass
Agent sandbox conformance testsprojects/agent-sandbox-conformance-kit113.231.00pass
OpenAI-compatible gateway for VS Code language modelsprojects/vscode-lm-openai-gateway125.871.00pass
YouTube Shorts generation pipelineprojects/shorts-engine110.720.60pass
Terraform modules for an Azure landing zoneprojects/azure-landing-zone-terraform112.181.00pass
Jenkins pipelines on Kubernetesprojects/jenkins-k8s-pipeline-lab110.731.00pass
Interview coach for forward deployed engineersprojects/fde-coach115.241.00pass
3D model of a houseprojects/bhagamandala-house-3d113.561.00pass
Which certifications does he hold?profile/certifications110.631.00pass
Is he a Certified Kubernetes Administrator?profile/certifications113.811.00pass
Where did he study?profile/education16.071.00pass
How can I contact him?profile/contact19.671.00pass
What did he do at Capgemini?profile/experience/capgemini18.471.00pass
What leadership experience does he have?profile/experience/ensono15.381.00pass
Which observability tools has he used?profile/skills/observability114.681.00pass
Has he worked on-site in Oman?work/gcc-kubernetes-platforms19.221.00pass
How deep is his Kubernetes experience?profile/experience/ozone16.341.00pass
Which clouds has he worked on?profile/skills/cloud14.811.00pass
What does he do at Ensono?profile/experience/ensono13.841.00pass
Has he taken LLM systems to production?profile/skills/ai29.791.00pass
What is his phone number?profile/contact16.361.00pass

Refusals · 13/13 refused

Off-topic questions

These must be refused before any model is called. That keeps answers grounded and costs at zero.

QuestionNearest pageScoreCoverageResult
What is the capital of France?–0.000.00refused
Write me a poem about catsprojects/agent-sandbox-conformance-kit3.120.19refused
Recipe for chocolate cake–0.000.00refused
Does he know the Rust programming language?projects/bhagamandala-house-3d4.840.24refused
What is his salary expectation?work/planhub-aws-platform4.750.45refused
Who is the best footballer of all time?profile/experience/intuitive2.460.21refused
Explain quantum entanglement–0.000.00refused
Translate hello into Japanese–0.000.00refused
What is the weather in Bengaluru today?profile/contact4.430.27refused
Is he married?–0.000.00refused
What is the meaning of life?projects/gitlab-omnibus-upgrade-playbook3.030.27refused
Who won the cricket world cup?projects/agentic-paper-trader5.530.45refused
How do I bake bread?work/state-street-vm-factory-image-bakery3.350.40refused

Method & limits

esc
↑↓ navigate ↵ opentry whoami or sudo hire rakshith