OCTA is a tool for test case documents classification, which aims to improve its quality using Machine Learning and NLP
Technologies: Python - NLP (NLTK) - ML (SVM) - SKLearn - Flask
In computing, oracle (test) is a mechanism for determining whether a test has passed or failed. The use of oracles involves comparing the outputs of the system under test for a test-case input. Determining the correct output for a given input is known as the test oracle problem.
Oracles can be categorized in:
Specified - Derived - Implicit - Human as you can check here
One of the problems is related a oracles in test case documents. In a determined context, testers were writting test case documents kind confused and this was causing some derived oracle problems, so I decided to built an classifier, using the company's project dataset to help testers write better test case documents, reducing the oracle problems and making easier to verify its behavior
I made a tool called OCTA for test case documents classification. I used controlled language (PENG) concepts to standard the text, creating a catalog of good practices of how a test case document should be written in terms of oracle. And also I read a lot of oracle problem acticles to get knowledge of what scientists were doing to contribute to this subject.
Here I'm using sklearn, nltk, np and so on. The method to process the sentences was TF-IDF.
And then I pre processed texts from two datasets, the oracles and non oracles, and after that I split the data into 75% train and 25% test.
After all I decided to use the Support Vector Machine (SVM) classifier because of the (not so long) size of the dataset. I also tried other 3 classifiers but SVM gave me the best results.
My model was ready but I had to put on the web using flask and after that I created a catalog using my model's intelligence to guide new testers on a test case document development. Unfortunately I cannot share the entire project because a company sponsored me, but I can share the structure I used to built the entire application, which is here.