Unit testing emails
I have not yet done any research on existing libraries/frameworks that solve this problem, but I think it has to be either of two solutions:
- Have the test code poll a dedicated email box, matching emails with testcases and determining whether they are correct
- Launch a mock email server from your code, modify the test configuration in such a way that the mails are sent to that mail server and immediately check back whether the mails are correct.
If you already have an project layout and architecture that allows dependency injection and a seperate configuration for tests, then the second seems the obvious way to go: much more flexible and contained. Unfortunately, the project for which I would like to implement this does not have anything of the sort and adding that will not be easy, which make me prefer the first solution.
There is another thing to consider: library support. As I said, I have not yet searched, but my guess is that the second solution is the one for which there will be several solutions readily available, because it is the 'proper' solution. Moreover, I doubt whether I will find any existing solution that makes implementing the first solution easier.
I think I'll just go with the 'proper' solution .
you're testing the application end-to-end.
Proper unit-tests would be:
- test whether the mailing subsystem code can send large amounts of e-mails correctly
e.g. send 1million e-mails and verify that 1million e-mails arrive correctly.
The easy way to perform an end-to-end mail test is to embed a unique increasing sequence number in each mail.
- test whether all code-paths produce the correct mails
this can even occur offline
both of these tests are far easier to perform statically than what you were describing.
Any mature language allows you to turn on debug flags that may let your functions behave different in that stage.
Anyway, you could try this approach for the testing if
* the correct amount of e-mails are generated (test 1)
* all the code path are followed, all distinct e-mails are generated correctly (test 2)
* send alle e-mails to a dedicated mailbox and read the number of messages (manually)
added + is that the mail proces is decoupled, meaning that people do not have to wait on 'your confirmation code email has been send' like web pages (PHP)
This should be an integration test. Read purpose:
Also because your unit test will emulate the real conditions the way the program is running.
True, I tend to misuse the term 'unit test', but you have to agree that there isn't a fine line between unit tests and integration tests. This test does not test the complete application, as that also involves parsing a file, adding required information from the database to the information parsed from a file entry and logic deciding which mail to send. In that respect, it isn't what I would generally call an integration test, because it does not integrate functionally seperate parts.
This test will test "given a certain method call with appropriate parameters, find the required email template, construct the email and send it". This is because that is the smallest 'unit' that can feasibly be tested without refactoring the application. Of course, it is an integration test of several classes, but in my book, a 'unit test' is not necessarily a test of a single class (we are talking Java here).
I must add: in situations like these, I would not write a seperate test checking that the correct template was fetched anyway, because that is already tested by test described above and is a type of failure than can be clearly distinguished. Writing a seperate test for a seperate 'unit' only makes sense when a higher-level test would show errors with multiple possible causes.
Motto: when in doubt, write an extra test, but be sane about the amount of tests you write.
[Comment edited on Tuesday 24 June 2008 11:20]
Typically, component[unit] testing occurs with access to the code being tested and with the support of the development environment, such as a unit test framework or debugging tool, and, in practice, usually involves the programmer who wrote the code. Defects are typically fixed as soon as they are found, without formally recording incidents \[reference:
ISTQB foundation level sylabus].
Integration testing tests interfaces between components, interactions with different parts of a system, such as the operating system, file system, hardware, or interfaces between systems. There may be more than one level of integration testing and it may be carried out on test objects of varying size. For example:
1. Component integration testing tests the interactions between software components and is doneafter component testing;
2. System integration testing tests the interactions between different systems and may be done after system testing. In this case, the developing organization may control only one side of the interface, so changes may be destabilizing. Business processes implemented as workflows may involve a series of systems. Cross-platform issues may be significant.
\[reference: ISTQB foundation level sylabus].
In my opinion you're still unit testing. So are ther codecompilers or parsers that can help? as for the options, I would go for the second. the fake mail server can than also be used in other tests like integration, system, systemintegration testing. Although not an easy task. It's still my choise to add a separate environment for testing
@RoadRunner84: popups or other solutions requiring human interaction are not an option due to the number of different emails that need to be verified. Writing to a logfile requires the same kind of dependency injection as a mock mailserver and would probably require more work on my part to retrieve the emails. A mock mailserver already recognizes where a mail start and ends; I would have to parse a log file, introducing extra possibilities for bugs.
@ikke007: if the mail generation and sending were sufficiently decoupled, testing what was generated without actually sending the mails would be preferable. As it stands however, that may involve more work than finding a ready-made mock mail server .
Comments are closed