Did someone test this code?

Westworld is back with a new season and what everyone is wondering is...did someone test the code for these hosts?! Here at Mavens, we recognize the importance of a thoroughly vetted application and we dislike everything about the tagline "chaos takes control," so our testing methodology leaves nothing to chance. We live by these five mantras when it comes to testing:

  1. Have a comprehensive plan
  2. Document solid requirements
  3. Use an intuitive tool
  4. Write simple scripts
  5. Do exploratory testing

At Mavens, our testing methodology leaves nothing to chance.

1. Have a comprehensive plan
Improvisation might be the second tier in Arnold's pyramid of consciousness but it's a big no-no when it comes to testing. You need a plan and you need it promptly. Your plan should outline the following:

  • Objective What is the purpose of testing? Typically, the goal is to provide assurance that the system functions in accordance with the requirements. A well-defined objective can help guide your approach.
  • Approach How will testing be conducted? This should include details on the test environment, format, timeline, tools, and processes. The approach should comprise the bulk of your plan.
  • Scope What types of tests will you perform? The scope should describe the extent of your testing and call out any exclusions. Evaluate the need for path testing, boundary testing, exploratory testing, regression testing, and etc.
  • Deliverables What are the agreed upon outputs? Deliverables can include a testing plan, test scripts, a test summary, status reports, and a traceability matrix to demonstrate where a particular requirement was tested.
  • Roles and responsibilities Who will be involved in testing? Documenting the test plan, authoring test scrips, completing test executions, reporting on progress, triaging defects; there are plenty of responsibilities to go around and they need an owner.

2. Document solid requirements
Requirements define your solution but they are also the basis for testing. If they are poorly written, dark and difficult times lie ahead, and testing is at risk. Writing good requirements is an art and a subject that deserves its own blog post, but the key takeaways are that you should have clear standards for the structure and format of your requirements and they should be written with testing in mind.

Requirements define your solution but they are also the basis for testing. If they are poorly written, dark and difficult times lie ahead, and testing is at risk.

3. Use an intuitive tool
A testing tool that is simple to use and conforms to your business processes is a must. Easy enough, right? Our research of test management tools led us to believe otherwise, so we took the DIY approach (sacrificing our technical architect's sanity in the process) and we love the results! Whatever direction you choose to take, ensure that the tool: 

  • Is the singular repository or interfaces with the tool where you manage your requirements
  • Supports your script review and approval process
  • Is easy to use and does not distract the tester from testing
  • Provides efficiency around capturing, routing, and resolving defects
  • Reduces the manual work involved in producing deliverables such as the test results, defect logs, and traceability matrices

4. Write simple scripts
Testers have to run hundreds of scripts so take pity on them and aim for simplicity! Your testers should be focused on the system they are examining, not trying to decipher the hidden meaning of your scripts. Make sure your scripts are:

  • Uniform in structure
  • Manageable length
  • Easy to follow
  • Unambiguous
  • Consistent with terminology

What exactly is exploratory testing? We like to think of it as an open challenge to break the system.

5. Do exploratory testing
No matter how many scripts you write and how much test coverage you have for a requirement, you can't account for every possible system scenario. This is where exploratory testing comes into play and, trust us, there is no substitute for it. So what exactly is exploratory testing? We like to think of it as an open challenge to break the system. It takes patience and creativity and while it may seem less "official," our observation has been that it yields a higher percentage of defects when compared to scripted testing. 

So will following these tips help prevent AI takeover? Nah, that's imminent. They will, however, save you the embarrassment of letting those stubborn defects make it into production. 

Desy Velcheva

Author Bio

Desy Velcheva is a Mavens Solution Architect based in Chicago. She has over 7 years of technology experience and has delivered solutions in HLS functional areas such as pharmaceutical sales, medical device sales, veterinary sales, patient support, clinical trial management, and medical information. Over the past year and a half, Desy has acquired expertise in software testing and has been deeply involved in defining Mavens' testing methodology.