This semester has allowed me to learn a lot more about different kinds of testing; unit testing in particular via the Cxxtest suite. Before this semester, most of the tests that we ran on programs were given to us, so we weren't able to see the level of consideration that went into writing tests. Though implementing unit tests takes more time to develop adequate tests up front, it saves loads of time in the long run.
Unit testing helps you to identify where your problem stems from via isolation. In this class, since we have split up WTF (the name for our project) into smaller subprojects, we are able to write tests in the Cxxtest suite for the individual functionalities of each of those pieces. Doing this grants us reusable tests translate into long-term tests that help with the integration process. For Iteration 2 of our project, where we had to link the parser to our previously written scanner, it was helpful to be able to run our scanner unit tests every now and then and make sure that the changes we implemented were not affecting the expected functionality. I am sure that as the semester progresses, these unit tests that we have already developed will prove vital in ensuring that there is no error trickle down between the various pieces of the translator.
In order to help with the process of integration, it would be nice to learn about how we can implement automated tests in our project. I know that automated runs are generally reserved for large projects where you have many users making modifications, however it would be nice to be able to see how this process works in a partner setting, since it is most likely going to be something that we are going to be dealing with in a professional environment in the near future. The general concept behind doing automated, scheduled runs is that after you make changes, you want to run tests to ensure that your program is still doing what you expect. I doubt that this would be a very complicated task to implement, so hopefully we will get some exposure in the remaining semester.
Learning about the various ways to implement tests has convinced me that unit testing is a very important process, despite the time you have to spend up front. Unfortunately, writing tests can sometimes be a bit of a hassle, especially using the Cxxtest suite.
One downfall to the testing process is that Cxxtest in itself was somewhat awkward to get acquainted with right away. I know that I struggled with understanding exactly what the various assert() functions that you could call were doing and how to go about writing tests that would actually test what you wanted tested. After spending some time looking through the various tests that the Cxxtest incorporate as well as understanding how Cxxtest works in general, I now feel adequately comfortable in writing valid unit tests. Knowing what your program should do and why it is important for you to test will help make your life a lot easier in regards to writing different sorts of tests in your career.
Sometimes it is difficult to define the functionalities that you want to ensure are working. If you are doing a large project that has a lot of functions in it, you most likely do not want to write unit tests for each of those functions. Using Iteration 1 of WTF as an example, there were a lot of things that we didn't test for in regards to functions that the scanner called. We did not make an explicit test for determining whether regular expressions were being generated correctly, but this is because it is assumed that they are working in tests that we actually implemented. If you make tests that cover various run types for your program, you will most likely be writing adequate tests.
Overall, testing frameworks are important tools to help you when working on projects that have high modularity, which is what most projects become as size increases. It allows you to be able to distinguish problems before they escalate into tedious problems, which is something that all of us want in the field that we are working in.
I can relate to what you mean by cxxTest being difficult to get acquainted with right away. It took me a while too, but my partner did a good job explaining it. I am also intrigued by the idea of automating unit tests. It can be used for quickly testing each component's functionality in any stage of the development.
Ajay Jashnani
Well, I wish I would have had a nice partner who explains things to me, anyway I only can say that I was just able to run the three tests from the lab because my partner was so in a hurry to get it done, so I could not appreciate the lab testing assignment in its entire context.
I believe I forgot to put my name in my comment. Here, my name is Javier Ayala.
I agree with you that it is great to finally have the tests on our hands instead of being provided to us. It is a great way for us to understand exactly how to test parts of our code, and how to find those key tests that will break it. Automated tests seem like something that would be hard for us at our current knowledge, though it is something that I would love to start learning because it is a critical part of real-life coding. CxxTest is confusing, but that isn't necessarily bad. It forces you to spend time learning how it works, but after you get acquainted with it, you have also learned how to use cxxTest as well as possible. It is worth the extra time that one has to put in to learn it. Great post, and it is a good introduction to the world of unit testing.
Kevin ZDon
I agree that cxxTest was awkward to get acquainted with. I didn't have so much trouble with the "assert" statements, but I was very confused about how things were to be distributed between the header files and source files. I liked your question about automation. I guess that including the tests in your makefile is one form of automation, but I wonder if there are other options.
Jonathan Bassen