Dev - Clerky

API Tests

AI Model Registry

Test and monitor all available AI models across providers. Working models can be used in user preferences.

Provider	Model	Cost (Input) ⇅	Cost (Output) ⇅	Status	Action
Click "Refresh Registry" to load models

Usage Analytics

View AI usage statistics from Firestore. Data updates as Clerky uses AI models.

Total Requests

Total Tokens

Total Cost

Avg Latency

Usage by Provider

No data yet

Usage Over Time

Recent AI Interactions

Time	Provider	Model	Tokens	Cost	Latency	Status
Click "Refresh Analytics" to load data

AI Model Health Check

Test all 5 AI models used for chunk distribution. Cost: < $0.0001 (fraction of a cent).

Status	Model	Response Time	Response

Maintenance

Edit practice point criteria

Tighten the APPLIES TO description of one practice point to fix false-positives. Find the guideline ID and practice point name in the feedback record.

Target Population Generator

Generate a concise targetPopulation field for each guideline — used by the relevance assessor to avoid misclassifying guidelines based on title alone.

Clinical Issues Synchronization

Scan all guidelines against the clinical issues list. If a guideline's main topic is not covered, suggest a new issue. Run this periodically to keep the issue taxonomy up to date.

Processing... 0%

Reingest Single Guideline to Vector DB

Re-ingest a specific guideline to Pinecone vector database. Useful for guidelines that were processed but not ingested due to errors.

Re-extract PDF Content

Re-extract verbatim text from PDFs using pdf-parse and update Firestore content field. This fixes issues where stored content doesn't match the actual PDF text.

Regenerate Auditable Elements

Regenerate auditable elements for guidelines using AI. Now supports filtering and batch selection. Focuses on actionable practice points with algorithmic logic.

	Title	Org	Year	Date Extracted
Click "Refresh List" to load guidelines

Endpoint Performance Monitor

View API endpoint performance stats. Data shows the last 500 requests.

Method	Endpoint	Count	Avg	Min	Max	Errors
Click "Refresh Timings" to load data

Time	Method	Endpoint	Status	Duration
Click "Refresh Timings" to load data

Edit Guideline Display Names

ID	Title	Display Name	Organisation	Actions

Firestore Data Viewer

Browse Firestore collections and inspect document data. Select fields to show/hide and sort by values.

Collection

Field Visibility

Guideline Discovery

Prompt Evolution System

Run test scenarios through the Analyse workflow, evaluate results against guidelines, and generate prompt improvements. Using multiple scenarios helps ensure changes are generalizable rather than overfitting to specific cases.

Select Clinical Scenarios (3 recommended)

✓ = has transcript, ○ = needs generation. Hold Ctrl/Cmd to select multiple. Guidelines are auto-matched using the same process as Analyse.

Processing... 0%

Avg Recall

Coverage of applicable recommendations

Avg Precision

Appropriateness of suggestions

Avg Latency

Scenarios

Redundancy Rate

Lower is better

False Negative Rate

Lower is better

Applicability Rate

Higher is better

Per-Scenario Results

Suggested Prompt Improvements

Evolvable Prompt Evolution

Evolve the Completeness Check and Dynamic Advice prompts using fake clinical scenarios. The system runs scenarios, judges output quality, generates an improved prompt, and stores it as a new version you can review and activate.

Prompt to Evolve

Scenarios

Processing... 0%

Avg Recall

Avg Precision

Avg Latency

Scenarios

Per-Scenario Results

Improvements

Prompt Version History

Click "Load Versions" to see version history

Parallel:

Danger Zone

	Time	Endpoint	Provider	Tokens	Latency	Status
Click Refresh to load logs

Select a log entry to view details

Click on a row in the table above to view the full prompt and response.

Algorithms

Use Discovery to open guidelines in the preview panel and generate or validate algorithms.

Clinical Conditions & Transcripts Manager

Manage clinical conditions and their test transcripts. Add or remove conditions individually without affecting existing transcripts.

➕ Add New Condition

Condition Name

Condition	Category	Transcript	Preview	Actions
Click "Load Conditions" to view all clinical conditions

Edit Transcript

Condition ID

Transcript Content

User Feedback

Filter:

Date	User	Feedback	Context	Status	Action

Improvements & Audit Trail

Track all improvements made in response to user feedback, including implementation, testing, and resolution status.