Effective Test Case Writing in Natural Language Processing (NLP)

Essentials Of A Good Test Case

A well-written test case should:

• Be easy to understand and execute
• Have a specific objective
• Have clear requirements
• Have clear pass/fail criteria
• Be repeatable
• Verify each action performed during the test case

Best Practices For Creating Effective Test Cases

Good test cases are concise, targeted and easy to execute and maintain. They make the testing process more effective by saving time and effort. Here are the top 10 best practices to follow while creating test cases.

1. Plan and organize your testing before you begin.
   • Create a folder structure organized by feature area. 
   • Plan out your tags for use in creating Orchestrations, for example tags such as "Regression", "Smoke", "Integration"
2. Make regression test cases small and specific. This gives you faster feedback since the test takes less time to execute, as well as being clearer when there is a defect in the application.
3. Functionize has a great feature that lets you update similar steps across test cases, don't worry about over engineering tests with reusable steps (Page Objects) until after you've created the test. 
4. Don't assert large areas of the website, verify each section individually. For example, instead of asserting the content of a div that contains an entire menu, verify each menu item individually. This helps our ML engine better 'self-heal', and it lets you identify changes in the menu immediately when there is a change or defect.
5. Don't forget about instructions when you're using Architect. You have the option to add instructions, in addition to the actions recorded, to the test. This helps you plan, structure, and organize your test. For example, you may create an instruction called "Create Account" and then step through the account creation process. It's easy to convert the entire instruction into a reusable Page Object if you create these as you go.
6. If you are using NLP to create test cases, or are importing test cases, treat the steps like you are writing them for a machine. Keep the instructions very simple and single-purposed.
   • Good Examples:
     • Enter "Hello World" into the Search Box
     • Click "Search"
     • Verify "Search" text appears on the button next to the text "Cancel"
   • Bad Examples:
     • Click on the 4th chair in the search results
     • Click the yellow button (if it's not the only yellow button, or it's not very close to pure yellow)
     • Search google and verify result for "Puppy Picture" 
7. Design tests so that you can run them in parallel, as much as possible. This will get you faster results when you want to execute your entire regression suite.
8. Data driven tests are extremely powerful - think about where you can randomize data versus using known structured data.
9. Orchestrations can help you manage integration testing across multiple projects. Consider using API and Database tests for complete End to End integrated tests.
10. Don't forget about security testing! Create folders or use tags for different types of security testing at the application level.