Non-functional Testing Results
From Evolution
The Non-functional Testing Results would be recorded for the following 3 categories:
- Mailer
- Calendar
- Address Book
Contents |
Mailer
Scenario 1
The account type used for test execution was IMAP on a test IMAP server.
This test scenario consisted of the following 6 major operations in the evolution mailer:
- Send Mail [With atleast 1 attachment]
- Read Mail
- Delete Mail
- Copy Mail
- Move Mail
- Sort Mails
An input file with the number of times each of these above operations are to be performed is provided as an input to the test script. The order in which these operations are performed is randomized inside the test script.
Number of times the above mailer operations are performed were: Read Mail = 10 Send Mail = 20 Delete Mails = 20 Move mails = 10 Copy Mails = 25 Sort mails = 5
The statistics captured during the execution of the test script are mainly the Operation, Time, CPU Utilization and Memory usage (Resident) of the evolution process.
The tables shown below summarize the results
send_email
This graph shows the memory usage of the "evolution" client process when it is sending emails (with attachments)Operation | Time | CPU Utilization (%) | Resident Memory (in MB) | Additional Information | |
---|---|---|---|---|---|
send_email | 15:40:15 | 8.0 | 34 | Entered send_email routine | |
send_email | 15:43:29 | 0.0 | 40 | After typing the text in the compose message | |
send_email | 15:43:37 | 11.9 | 37 | During verification of sent email | |
send_email | 15:43:40 | 0.0 | 37 | After sending and verifying the email | |
send_email | 15:49:16 | 4.0 | 38 | The above comments repeat for the remaining entries in send_email table as 4 readings correspond to 1 sent email | |
send_email | 15:50:40 | 0.0 | 42 | ||
send_email | 15:50:47 | 27.1 | 39 | ||
send_email | 15:50:50 | 0.0 | 39 | ||
send_email | 15:56:54 | 0.0 | 40 | ||
send_email | 16:00:06 | 0.0 | 44 | ||
send_email | 16:00:14 | 2.0 | 42 | ||
send_email | 16:00:17 | 0.0 | 42 | ||
send_email | 16:00:53 | 0.0 | 42 | ||
send_email | 16:02:18 | 0.0 | 46 | ||
send_email | 16:02:26 | 0.0 | 42 | ||
send_email | 16:02:29 | 0.0 | 42 | ||
send_email | 16:04:18 | 0.0 | 42 | ||
send_email | 16:05:43 | 0.0 | 46 | ||
send_email | 16:05:51 | 41.7 | 42 | ||
send_email | 16:05:54 | 0.0 | 42 | ||
send_email | 16:06:18 | 0.0 | 42 | ||
send_email | 16:09:31 | 0.0 | 46 | ||
send_email | 16:09:39 | 0.0 | 43 | ||
send_email | 16:09:41 | 0.0 | 43 | ||
send_email | 16:11:54 | 0.0 | 43 | ||
send_email | 16:15:12 | 0.0 | 46 | ||
send_email | 16:15:20 | 9.9 | 43 | ||
send_email | 16:15:23 | 0.0 | 43 | ||
send_email | 16:15:47 | 0.0 | 43 | ||
send_email | 16:19:11 | 0.0 | 47 | ||
send_email | 16:19:19 | 11.9 | 44 | ||
send_email | 16:19:21 | 0.0 | 44 | ||
send_email | 16:20:28 | 0.0 | 45 | ||
send_email | 16:21:46 | 0.0 | 46 | ||
send_email | 16:21:54 | 0.0 | 45 | ||
send_email | 16:21:56 | 0.0 | 45 | ||
send_email | 16:22:23 | 0.0 | 45 | ||
send_email | 16:25:37 | 0.0 | 50 | ||
send_email | 16:25:45 | 0.0 | 46 | ||
send_email | 16:25:48 | 0.0 | 46 | ||
send_email | 16:25:54 | 0.0 | 46 | ||
send_email | 16:29:04 | 0.0 | 50 | ||
send_email | 16:29:12 | 0.0 | 46 | ||
send_email | 16:29:15 | 0.0 | 46 | ||
send_email | 16:29:21 | 0.0 | 46 | ||
send_email | 16:30:47 | 0.0 | 50 | ||
send_email | 16:30:55 | 33.9 | 46 | ||
send_email | 16:30:58 | 0.0 | 46 | ||
send_email | 16:31:04 | 0.0 | 46 | ||
send_email | 16:33:44 | 0.0 | 50 | ||
send_email | 16:33:52 | 0.0 | 47 | ||
send_email | 16:33:55 | 0.0 | 47 | ||
send_email | 16:34:01 | 0.0 | 46 | ||
send_email | 16:35:32 | 0.0 | 49 | ||
send_email | 16:35:40 | 0.0 | 49 | ||
send_email | 16:35:43 | 0.0 | 49 | ||
send_email | 16:37:19 | 0.0 | 46 | ||
send_email | 16:40:31 | 2.0 | 50 | ||
send_email | 16:40:39 | 0.0 | 46 | ||
send_email | 16:40:42 | 0.0 | 46 | ||
send_email | 16:40:48 | 0.0 | 46 | ||
send_email | 16:42:12 | 0.0 | 50 | ||
send_email | 16:42:19 | 30.2 | 46 | ||
send_email | 16:42:22 | 0.0 | 46 | ||
send_email | 16:42:49 | 0.0 | 46 | ||
send_email | 16:46:04 | 0.0 | 50 | ||
send_email | 16:46:12 | 11.9 | 46 | ||
send_email | 16:46:15 | 0.0 | 46 | ||
send_email | 16:47:02 | 0.0 | 46 | ||
send_email | 16:50:14 | 0.0 | 50 | ||
send_email | 16:50:22 | 8.0 | 46 | ||
send_email | 16:50:24 | 0.0 | 46 | ||
send_email | 16:51:12 | 0.0 | 47 | ||
send_email | 16:52:34 | 0.0 | 51 | ||
send_email | 16:52:42 | 42.0 | 47 | ||
send_email | 16:52:45 | 0.0 | 47 | ||
send_email | 16:53:57 | 0.0 | 47 | ||
send_email | 16:55:25 | 0.0 | 51 | ||
send_email | 16:55:32 | 45.7 | 48 | ||
send_email | 16:55:35 | 0.0 | 48 |
read_email
This graph depicts the memory usage of the "evolution" client process when it is executing read_email.Operation | Time | CPU Utilization (%) | Resident Memory (in MB) | Additional Information |
---|---|---|---|---|
read_email | 15:39:28 | 0.0 | 34 | |
read_email | 15:43:46 | 0.0 | 37 | |
read_email | 15:44:33 | 0.0 | 37 | |
read_email | 15:45:50 | 0.0 | 38 | |
read_email | 15:55:14 | 0.0 | 40 | |
read_email | 15:56:24 | 0.0 | 40 | |
read_email | 16:00:23 | 0.0 | 42 | |
read_email | 16:02:35 | 0.0 | 42 | |
read_email | 16:03:05 | 0.0 | 42 | |
read_email | 16:09:47 | 0.0 | 43 |
copy_email
Operation | Time | CPU Utilization (%) | Resident Memory (in MB) | Additional Information |
---|---|---|---|---|
copy_email | 15:37:45 | 0.0 | 33 | |
copy_email | 15:39:58 | 0.0 | 34 | |
copy_email | 15:44:15 | 0.0 | 37 | |
copy_email | 15:45:32 | 0.0 | 38 | |
copy_email | 15:52:08 | 0.0 | 39 | |
copy_email | 15:53:05 | 18.1 | 37 | |
copy_email | 15:53:32 | 0.0 | 39 | |
copy_email | 15:54:36 | 0.0 | 39 | |
copy_email | 15:54:55 | 0.0 | 40 | |
copy_email | 16:03:34 | 0.0 | 42 | |
copy_email | 16:06:00 | 0.0 | 42 | |
copy_email | 16:10:18 | 0.0 | 43 | |
copy_email | 16:10:36 | 0.0 | 43 | |
copy_email | 16:19:48 | 0.0 | 44 | |
copy_email | 16:35:49 | 2.0 | 46 | |
copy_email | 16:36:06 | 10.0 | 46 | |
copy_email | 16:36:33 | 4.0 | 46 | |
copy_email | 16:37:01 | 0.0 | 46 | |
copy_email | 16:50:30 | 0.0 | 46 | |
copy_email | 16:52:51 | 0.0 | 47 | |
copy_email | 16:53:08 | 0.0 | 47 | |
copy_email | 16:55:41 | 0.0 | 48 | |
copy_email | 16:56:04 | 4.0 | 48 | |
copy_email | 16:56:29 | 0.0 | 48 | |
copy_email | 16:56:46 | 14.0 | 48 |
move_email
Operation | Time | CPU Utilization (%) | Resident Memory (in MB) | Additional Information |
---|---|---|---|---|
move_email | 15:37:26 | 0.0 | 33 | |
move_email | 15:48:01 | 0.0 | 38 | |
move_email | 15:48:18 | 4.0 | 38 | |
move_email | 15:50:56 | 0.0 | 39 | |
move_email | 15:52:28 | 0.0 | 39 | |
move_email | 15:52:46 | 0.0 | 39 | |
move_email | 15:56:06 | 0.0 | 40 | |
move_email | 16:03:52 | 10.0 | 42 | |
move_email | 16:11:36 | 0.0 | 43 | |
move_email | 16:15:29 | 0.0 | 43 |
sort_email
Operation | Time | CPU Utilization (%) | Resident Memory (in MB) | Additional Information |
---|---|---|---|---|
sort_email | 15:45:02 | 0.0 | 37 | |
sort_email | 15:46:41 | 0.0 | 38 | |
sort_email | 15:47:31 | 0.0 | 38 | |
sort_email | 15:48:46 | 2.0 | 38 | |
sort_email | 15:51:38 | 41.9 | 39 |
Scenario 2
This test scenario consisted only the send email operation continuously with an IMAP account and observe the memory usage of evolution client. Only plain text emails were sent in this scenario and at specific counts of sent emails the memory usage of evolution was noted.
The following 2 graphs summarizes the outcome.
This graph shows the memory consumption of other evolution processes i.e. evolution-data-server, evolution-exchange-storage and evolution-alarm-notify.
Observations/Concerns
I have made following observations after executing the above mentioned test scenarios:
PS: Some of these might qualify as bugs and can be filed in evolution bugzilla Need Comments
- The memory usage of evolution process grows continuously over a period of time.
- The CPU utilization is constant most of the times at 0.0 but in between some of the operations (sort, copy, send and move emails) it shoots up once. The maximum CPU utilization seen in this test execution is 47%. As the number of operations and the time period of execution increase this value is seen to be touching 99.9% at one point of time.
- As the number of operations increase the Memory usage keeps on growing and doesn't come down even if we are not using the evolution client to perform any mailer operation.
- After finishing the "copying and moving mail(s) to a folder" operation in the IMAP account, the evolution client immediately starts fetching the summary information for new messages. Because of this, there is a substantial delay in displaying the next mail in the list.