Promptfoo

ab is the Apache HTTP benchmarking tool. Single-threaded load generation. Quick baseline measurements. Built into Apache. Suitable for API endpoints.

JMeter simulates load from multiple concurrent threads. Test APIs, web apps, FTP servers. Assertions validate responses. Results graphs show throughput and latency. Open-source standard. Easy to scrip…

Artillery is a Node.js framework for load testing. Define scenarios in YAML or code. Ramp-up and hold phases. Performance assertions. Integration with cloud platforms. Growing popularity.

Soasta CloudTest (part of Micro Focus) provides integrated testing. Load testing and functional testing combined. Cross-browser compatibility. Geo-distributed load generation. Enterprise platform.

Datadog creates synthetic tests for APIs and web applications. Record user journeys. Continuous validation from multiple regions. Integrates with Datadog monitoring. Unified observability platform.

Fivetran automates data movement from 500+ source systems (Salesforce, Marketo, production DBs) into cloud warehouses. Connectors auto-detect schema changes and replay late-arriving data without rebui…

AI evaluation platform for monitoring LLM hallucinations, quality, and performance in production applications.

Testing and observability platform for AI pipelines with regression detection and evaluation management.

Open source testing framework for ML models with automated vulnerability detection and quality assurance.

Great Expectations is an open Python library for data quality testing and documentation. Write expectations declaratively (expect table to have 1M rows, column X in range 0-100). GX automatically test…

Testing and evaluation platform for voice AI agents with automated call simulations and quality metrics.

k6 is a developer-centric load testing tool. Write tests in JavaScript. CLI runs tests locally or in cloud. Thresholds define pass/fail. Integration with CI/CD. Founded by Load Impact. Growing adoptio…

LevelValidator Pro automatically checks game levels for design issues and bugs. The system detects soft-locks, unreachable areas, performance problems, and gaps. Reports include visual maps showing pr…

Locust writes load tests in Python. Spawn thousands of concurrent users. Requests library for HTTP. Real-time web UI shows metrics. Distributed load generation. Simple and powerful.

New Relic simulates user interactions from 100+ locations. Monitors critical flows (login, checkout, search). Alerts on performance degradation. Integration with APM data. Cloud-based monitoring.

AI agent that automatically writes and maintains end-to-end tests by discovering user flows, generating Playwright tests, and updating them whenever the UI changes—no test code to write or maintain ma…

AI evaluation platform for testing LLM outputs against hallucination, toxicity, and compliance criteria.

ProtoTest is an automated testing framework for game logic and mechanics. Developers write test cases in visual scripting and run across all platforms. The tool detects crashes, physics errors, input …

QAConnect Pro distributes game testing tasks to remote QA contractors worldwide. Studio managers define test cases and recruit specialized testers across regions. Real-time bug reporting includes vide…

Open source framework for evaluating RAG pipeline quality with metrics for faithfulness and relevance.

Tabular is an Apache Iceberg company founded by the original Iceberg committers from Netflix. It provides a fully managed, serverless Iceberg environment with automatic optimization and time-travel re…

Taurus is an open load testing automation tool. Abstraction for multiple engines (JMeter, Gatling, Selenium). Simple YAML syntax. CI/CD integration. Reduces scripting overhead.

Vegeta is a versatile HTTP load testing tool in Go. Pipe attack output through reporters. Real-time metrics. Histogram plots. Powerful Unix pipeline style.

wrk is a modern HTTP load testing tool written in C. Lightweight and fast. Lua scripting for custom requests. 50K+ requests per second per core. Ideal for API testing.