Non-functional Testing Results

From Evolution

The Non-functional Testing Results would be recorded for the following 3 categories:

  1. Mailer
  2. Calendar
  3. Address Book


Contents

[edit]

Mailer

[edit]

Scenario 1

The account type used for test execution was IMAP on a test IMAP server.

This test scenario consisted of the following 6 major operations in the evolution mailer:

  • Send Mail [With atleast 1 attachment]
  • Read Mail
  • Delete Mail
  • Copy Mail
  • Move Mail
  • Sort Mails

An input file with the number of times each of these above operations are to be performed is provided as an input to the test script. The order in which these operations are performed is randomized inside the test script.

Number of times the above mailer operations are performed were: Read Mail = 10 Send Mail = 20 Delete Mails = 20 Move mails = 10 Copy Mails = 25 Sort mails = 5

The statistics captured during the execution of the test script are mainly the Operation, Time, CPU Utilization and Memory usage (Resident) of the evolution process.

The tables shown below summarize the results

[edit]

send_email

memory usage of "evolution" process when running send_email
Enlarge
memory usage of "evolution" process when running send_email
This graph shows the memory usage of the "evolution" client process when it is sending emails (with attachments)
Operation Time CPU Utilization (%) Resident Memory (in MB) Additional Information
send_email 15:40:15 8.0 34 Entered send_email routine
send_email 15:43:29 0.0 40 After typing the text in the compose message
send_email 15:43:37 11.9 37 During verification of sent email
send_email 15:43:40 0.0 37 After sending and verifying the email
send_email 15:49:16 4.0 38 The above comments repeat for the remaining entries in send_email table as 4 readings correspond to 1 sent email
send_email 15:50:40 0.0 42
send_email 15:50:47 27.1 39
send_email 15:50:50 0.0 39
send_email 15:56:54 0.0 40
send_email 16:00:06 0.0 44
send_email 16:00:14 2.0 42
send_email 16:00:17 0.0 42
send_email 16:00:53 0.0 42
send_email 16:02:18 0.0 46
send_email 16:02:26 0.0 42
send_email 16:02:29 0.0 42
send_email 16:04:18 0.0 42
send_email 16:05:43 0.0 46
send_email 16:05:51 41.7 42
send_email 16:05:54 0.0 42
send_email 16:06:18 0.0 42
send_email 16:09:31 0.0 46
send_email 16:09:39 0.0 43
send_email 16:09:41 0.0 43
send_email 16:11:54 0.0 43
send_email 16:15:12 0.0 46
send_email 16:15:20 9.9 43
send_email 16:15:23 0.0 43
send_email 16:15:47 0.0 43
send_email 16:19:11 0.0 47
send_email 16:19:19 11.9 44
send_email 16:19:21 0.0 44
send_email 16:20:28 0.0 45
send_email 16:21:46 0.0 46
send_email 16:21:54 0.0 45
send_email 16:21:56 0.0 45
send_email 16:22:23 0.0 45
send_email 16:25:37 0.0 50
send_email 16:25:45 0.0 46
send_email 16:25:48 0.0 46
send_email 16:25:54 0.0 46
send_email 16:29:04 0.0 50
send_email 16:29:12 0.0 46
send_email 16:29:15 0.0 46
send_email 16:29:21 0.0 46
send_email 16:30:47 0.0 50
send_email 16:30:55 33.9 46
send_email 16:30:58 0.0 46
send_email 16:31:04 0.0 46
send_email 16:33:44 0.0 50
send_email 16:33:52 0.0 47
send_email 16:33:55 0.0 47
send_email 16:34:01 0.0 46
send_email 16:35:32 0.0 49
send_email 16:35:40 0.0 49
send_email 16:35:43 0.0 49
send_email 16:37:19 0.0 46
send_email 16:40:31 2.0 50
send_email 16:40:39 0.0 46
send_email 16:40:42 0.0 46
send_email 16:40:48 0.0 46
send_email 16:42:12 0.0 50
send_email 16:42:19 30.2 46
send_email 16:42:22 0.0 46
send_email 16:42:49 0.0 46
send_email 16:46:04 0.0 50
send_email 16:46:12 11.9 46
send_email 16:46:15 0.0 46
send_email 16:47:02 0.0 46
send_email 16:50:14 0.0 50
send_email 16:50:22 8.0 46
send_email 16:50:24 0.0 46
send_email 16:51:12 0.0 47
send_email 16:52:34 0.0 51
send_email 16:52:42 42.0 47
send_email 16:52:45 0.0 47
send_email 16:53:57 0.0 47
send_email 16:55:25 0.0 51
send_email 16:55:32 45.7 48
send_email 16:55:35 0.0 48
[edit]

read_email

evolution memory usage when reading emails
Enlarge
evolution memory usage when reading emails
This graph depicts the memory usage of the "evolution" client process when it is executing read_email.
Operation Time CPU Utilization (%) Resident Memory (in MB) Additional Information
read_email 15:39:28 0.0 34
read_email 15:43:46 0.0 37
read_email 15:44:33 0.0 37
read_email 15:45:50 0.0 38
read_email 15:55:14 0.0 40
read_email 15:56:24 0.0 40
read_email 16:00:23 0.0 42
read_email 16:02:35 0.0 42
read_email 16:03:05 0.0 42
read_email 16:09:47 0.0 43
[edit]

copy_email

evolution memory usage when it is copying mails from one folder to another
Enlarge
evolution memory usage when it is copying mails from one folder to another

Operation Time CPU Utilization (%) Resident Memory (in MB) Additional Information
copy_email 15:37:45 0.0 33
copy_email 15:39:58 0.0 34
copy_email 15:44:15 0.0 37
copy_email 15:45:32 0.0 38
copy_email 15:52:08 0.0 39
copy_email 15:53:05 18.1 37
copy_email 15:53:32 0.0 39
copy_email 15:54:36 0.0 39
copy_email 15:54:55 0.0 40
copy_email 16:03:34 0.0 42
copy_email 16:06:00 0.0 42
copy_email 16:10:18 0.0 43
copy_email 16:10:36 0.0 43
copy_email 16:19:48 0.0 44
copy_email 16:35:49 2.0 46
copy_email 16:36:06 10.0 46
copy_email 16:36:33 4.0 46
copy_email 16:37:01 0.0 46
copy_email 16:50:30 0.0 46
copy_email 16:52:51 0.0 47
copy_email 16:53:08 0.0 47
copy_email 16:55:41 0.0 48
copy_email 16:56:04 4.0 48
copy_email 16:56:29 0.0 48
copy_email 16:56:46 14.0 48
[edit]

move_email

evolution memory usage when it is moving mails
Enlarge
evolution memory usage when it is moving mails

Operation Time CPU Utilization (%) Resident Memory (in MB) Additional Information
move_email 15:37:26 0.0 33
move_email 15:48:01 0.0 38
move_email 15:48:18 4.0 38
move_email 15:50:56 0.0 39
move_email 15:52:28 0.0 39
move_email 15:52:46 0.0 39
move_email 15:56:06 0.0 40
move_email 16:03:52 10.0 42
move_email 16:11:36 0.0 43
move_email 16:15:29 0.0 43
[edit]

sort_email

evolution memory usage when sorting emails
Enlarge
evolution memory usage when sorting emails

Operation Time CPU Utilization (%) Resident Memory (in MB) Additional Information
sort_email 15:45:02 0.0 37
sort_email 15:46:41 0.0 38
sort_email 15:47:31 0.0 38
sort_email 15:48:46 2.0 38
sort_email 15:51:38 41.9 39
[edit]

Scenario 2

This test scenario consisted only the send email operation continuously with an IMAP account and observe the memory usage of evolution client. Only plain text emails were sent in this scenario and at specific counts of sent emails the memory usage of evolution was noted.

The following 2 graphs summarizes the outcome.


"evolution" Memory usage
Enlarge
"evolution" Memory usage
This graph shows the memory consumption by the main evolution client process.
 Other evolution processes Memory usage
Enlarge
Other evolution processes Memory usage
This graph shows the memory consumption of other evolution processes i.e. evolution-data-server, evolution-exchange-storage and evolution-alarm-notify.
[edit]

Observations/Concerns

I have made following observations after executing the above mentioned test scenarios:

PS: Some of these might qualify as bugs and can be filed in evolution bugzilla Need Comments

  1. The memory usage of evolution process grows continuously over a period of time.
  2. The CPU utilization is constant most of the times at 0.0 but in between some of the operations (sort, copy, send and move emails) it shoots up once. The maximum CPU utilization seen in this test execution is 47%. As the number of operations and the time period of execution increase this value is seen to be touching 99.9% at one point of time.
  3. As the number of operations increase the Memory usage keeps on growing and doesn't come down even if we are not using the evolution client to perform any mailer operation.
  4. After finishing the "copying and moving mail(s) to a folder" operation in the IMAP account, the evolution client immediately starts fetching the summary information for new messages. Because of this, there is a substantial delay in displaying the next mail in the list.
[edit]

Calendar

[edit]

Address Book