Back to Case Studies
Arro Labs × College recruiting & events platform
Manual QA Test Case Documentation AI-Augmented Testing Regression Testing

From Minimal QA to a 3,000-Case Foundation, Powered by AI

3,000+

Test Cases Documented

5

Releases Supported

6+

Month Engagement

About Arro Labs

Arro Labs is a software development partner that builds and supports digital products for clients across industries. When one of their edtech clients, a college recruiting and events platform, needed a structured QA layer to keep up with their development pace and seasonal release cadence, Arro Labs brought QLTY Automation in as their dedicated QA partner.

"QLTY Automation is one of the best QA partners we've had the pleasure of working with. They bring a wealth of knowledge on QA best practices, are organized and thoughtful, and understand when to push and when to follow. They are an asset to any team, not only making your product stronger but also improving the people they work with."

Tess Rethore, Head of Product, Arro Labs

Arro Labs

The Challenge

Going into the engagement, the client had minimal QA support in place, mostly developer self-testing and ad-hoc verification. A major feature was being prepared for their spring college fair season, the platform's highest-stakes period of the year. Any quality issue during that window would directly affect the recruiters and institutions counting on the product to capture leads in the field.

The goal: build a durable QA foundation in time for the season, then extend it into something the internal team could rely on long after the engagement ended.

The Approach

Rather than treating manual QA as a slow, headcount-heavy bottleneck, QLTY Automation took an AI-augmented approach from day one, with Claude woven into every stage of the workflow. The result is human judgment where it matters most, and AI handling the repetitive, structured work that traditionally consumes a QA engineer's time.

Over the course of the engagement, the focus was on:

  • Establishing a structured test case library covering the full surface area of the web application
  • Filing and triaging bugs with detailed reproduction steps, severity assessment, and proper context for the dev team
  • Running regression cycles before each release to catch unintended impact from new changes
  • Operating inside the team's existing tooling: TestRail for test management, Jira for bug tracking, Figma for specs, and Slack for fast feedback loops

Why Manual QA, Not Automation?

QLTY Automation builds test automation frameworks. So why manual QA here? Three reasons:

  • Speed of coverage. With spring college fair season approaching, the client needed broad coverage fast. Building automation first requires a stable understanding of what to test, and that understanding didn't yet exist in a structured form.
  • You can't automate what isn't documented. Before you can write reliable automated tests, you have to know what "correct behavior" looks like. The 3,000+ test cases we built now serve exactly that purpose: a structured baseline of what the application is supposed to do. Many are strong candidates for future automation.
  • The product was actively evolving. Automating flows that are still being iterated burns time twice, once to write the script, again every time the underlying UI changes. AI-augmented manual QA flexes with change in a way scripted tests can't.

Manual QA was the right starting point. Automation comes next, when the rules of the game stabilize.

AI at Every Stage of the QA Workflow

What sets this engagement apart is how deeply AI was integrated into the day-to-day. This wasn't a one-off experiment, it was a 5–10x speedup on certain tasks that fundamentally changed how the work got done.

Specifically:

  • Test case generation: Drafting structured test cases from Figma designs and product specifications, then refining with human judgment. What previously took hours per feature now takes minutes.
  • Bug reports: Detailed reports with reproduction steps, severity, expected vs. actual behavior, and contextual analysis, drafted with AI and finalized by humans.
  • Regression scoping: Analyzing change sets between releases to surface the right scope of regression to run, instead of running everything blindly or skipping critical paths.
  • Automated verification of complex processes: Multi-step business flows that would normally require dedicated manual time were verified automatically with AI agents, freeing human attention for edge cases and judgment calls.
  • Documentation and summaries: Test plans, release summaries, and weekly status reports drafted in seconds, refined for accuracy.

The outcome: manual QA at the speed of automation, without losing the human judgment that catches what scripts can't.

The Results

3,000+ test cases organized into 104 structured sections

A searchable, navigable test library in TestRail covering the full surface area of the application, ready for the team to maintain and extend

5 production releases supported and counting

Every release goes through a rigorous regression cycle before reaching production users

A QA process the team can trust

Standardized bug reports, scoped regression cycles, and clear documentation that the internal team can take over at any time

5–10x speedup on AI-accelerated tasks

Test case generation, bug reporting, and complex process verification all dramatically faster than traditional manual QA

Ready to Build Quality Without the Bottleneck?

Whether you need automation, manual QA, or AI-augmented testing, we'll help you build a foundation your team can rely on.

Get in Touch