Testing, installation, and verification procedures and data for EasyCount Senate software
Dear Australian Electoral Commission,
I request under the Freedom of Information Act 1982 that you provide the following documentation regarding the "EasyCount Senate (ECS)" software ("the software"):
- Manual and automated software testing procedures for verifying that the software is operating correctly, including manual test procedures, unit tests, integration tests, functional tests, end-to-end tests, table-driven tests, smoke tests, fuzz tests, installation and compatibility tests, regression tests, acceptance tests, performance tests and benchmarks, security tests, usability tests, network and connectivity tests, and concurrency tests;
- Data sets used in the software testing procedures listed above, including real or simulated sets of votes, and real or simulated preference distributions;
- Installation, maintenance, and troubleshooting instructions, including the specifications of computer hardware, network configuration, operating system and runtime environment configuration required for the testing procedures, and,
- User manuals, guides, usage instructions, and parameter specifications for use of the software.
The documents listed above are substantially different in nature to documents that are the subject of other published FOI applications currently known to me. I am not affiliated with any other FOI applicant. This request is substantially different for the following reasons. First and foremost, this is not a request for the EasyCount Senate software itself. I am certain that the above-listed documentation regarding testing procedures cannot possibly be a "trade secret," even those components expressed in the form of source code, as adequate testing procedures should not reveal or rely upon details of the methods by which the software's results are determined---the testing procedures, when used, would at most establish a lower bound on the trustworthiness of the software. Furthermore, automated test procedures in the form of source code (such as unit tests) are, conventionally, logically separated from the software that is the subject of the testing procedures. Therefore the above-listed documents can be provided easily, electronically, separately from and without risk of revealing any "trade secret", and with minimal diversion of resources.
This request should not be considered "joined" to other pending applications, for the above reasons. I also request that access fees and charges for this request be waived as a matter of public interest. There is clear and increasing public interest in the EasyCount Senate software as evidenced by attention in the media. Release of the documents will promote public trust in the AEC's systems.
Yours faithfully,
Josh Deprez
Thank you for contacting us.
This is an automatic response from the Australian Electoral Commission to confirm we have received your email.
For more information on enrolling to vote, federal elections or the AEC, visit www.aec.gov.au.
Please do not respond to this email.
UNCLASSIFIED
Dear Mr Deprez
I refer to your email of 17 July 2014 11:36 PM in which you make your FOI
Request in respect of this matter. I enclose a scanned letter to you dated
21 July 2014 from Paul Pirani, Chief Legal Officer acknowledging receipt
of your FOI Request and indicating how it will be processed.
Regards
Owen Jones
Owen Jones | Senior Lawyer
Legal Services Section | Legal, Parliamentary and Procurement Branch
Australian Electoral Commission
T: (02) 6271 4528 | F: (02) 6293 7657
--------------------------------------------------------------------------
[1]Australian Electoral Commission logo [2]Australian Electoral
Commission
This email may contain legal advice that is subject to legal professional
privilege. Care should be taken to avoid unintended waiver of that
privilege. The Australian Electoral Commission’s Chief Legal Officer
should be consulted prior to any decision to disclose the existence or
content of any advice contained in this email to a third party.
--------------------------------------------------------------------------
-----Original Message-----
From: Josh Deprez [[3]mailto:[FOI #714 email]]
Sent: Thursday, 17 July 2014 11:36 PM
To: INFO
Subject: Freedom of Information request - Testing, installation, and
verification procedures and data for EasyCount Senate software
Dear Australian Electoral Commission,
I request under the Freedom of Information Act 1982 that you provide the
following documentation regarding the "EasyCount Senate (ECS)" software
("the software"):
- Manual and automated software testing procedures for verifying that the
software is operating correctly, including manual test procedures, unit
tests, integration tests, functional tests, end-to-end tests, table-driven
tests, smoke tests, fuzz tests, installation and compatibility tests,
regression tests, acceptance tests, performance tests and benchmarks,
security tests, usability tests, network and connectivity tests, and
concurrency tests;
- Data sets used in the software testing procedures listed above,
including real or simulated sets of votes, and real or simulated
preference distributions;
- Installation, maintenance, and troubleshooting instructions, including
the specifications of computer hardware, network configuration, operating
system and runtime environment configuration required for the testing
procedures, and,
- User manuals, guides, usage instructions, and parameter specifications
for use of the software.
The documents listed above are substantially different in nature to
documents that are the subject of other published FOI applications
currently known to me. I am not affiliated with any other FOI applicant.
This request is substantially different for the following reasons. First
and foremost, this is not a request for the EasyCount Senate software
itself. I am certain that the above-listed documentation regarding testing
procedures cannot possibly be a "trade secret," even those components
expressed in the form of source code, as adequate testing procedures
should not reveal or rely upon details of the methods by which the
software's results are determined---the testing procedures, when used,
would at most establish a lower bound on the trustworthiness of the
software. Furthermore, automated test procedures in the form of source
code (such as unit tests) are, conventionally, logically separated from
the software that is the subject of the testing procedures. Therefore the
ab ove-listed documents can be provided easily, electronically,
separately from and without risk of revealing any "trade secret", and with
minimal diversion of resources.
This request should not be considered "joined" to other pending
applications, for the above reasons. I also request that access fees and
charges for this request be waived as a matter of public interest. There
is clear and increasing public interest in the EasyCount Senate software
as evidenced by attention in the media. Release of the documents will
promote public trust in the AEC's systems.
Yours faithfully,
Josh Deprez
-------------------------------------------------------------------
Please use this email address for all replies to this request:
[4][FOI #714 email]
Is [5][AEC request email] the wrong address for Freedom of Information
requests to Australian Electoral Commission? If so, please contact us
using this form:
[6]https://www.righttoknow.org.au/change_re...
Write your response as plain text. Only send PDF documents as a last
resort. Government guidelines make it clear that PDF is not an acceptable
format for you to use in the delivery of government information.
[7]https://www.righttoknow.org.au/help/offi...
This request is being made by an individual using the Right to Know
website. The unique email address provided by the service for this request
satisfies s.15(2)(c) of the Freedom of Information Act.
Disclaimer: This message and any reply that you make will be published on
the internet. Our privacy and copyright policies:
[8]https://www.righttoknow.org.au/help/offi...
If you find this service useful as an FOI officer, please ask your web
manager to link to us from your organisation's FOI page.
-------------------------------------------------------------------
UNCLASSIFIED
UNCLASSIFIED
DISCLAIMER:
If you have received this transmission in error please notify us
immediately by return email and delete all copies. If this email or any
attachments have been sent to you in error, that error does not constitute
waiver of any confidentiality, privilege or copyright in respect of
information in the email or attachments.
References
Visible links
1. http://emailfooter.aec.gov.au/email/
2. http://emailfooter.aec.gov.au/email-promo/
3. mailto:[FOI #714 email]
4. mailto:[FOI #714 email]
5. mailto:[AEC request email]
6. https://www.righttoknow.org.au/change_re...
7. https://www.righttoknow.org.au/help/offi...
8. https://www.righttoknow.org.au/help/offi...
UNCLASSIFIED
Dear Mr Deprez
I refer to your email of 17 July 2014 11:36 PM in which you make your FOI
Request in respect of this matter. I enclose a scanned letter to you dated
21 July 2014 from Paul Pirani, Chief Legal Officer acknowledging receipt
of your FOI Request and indicating how it will be processed.
This supersedes my email to you of 21 July 2014 10:16 AM which forwarded
only the first page of Mr Pirani’s letter. I apologise for the omission
Regards
Owen Jones
Owen Jones | Senior Lawyer
Legal Services Section | Legal, Parliamentary and Procurement Branch
Australian Electoral Commission
T: (02) 6271 4528 | F: (02) 6293 7657
--------------------------------------------------------------------------
[1]Australian Electoral Commission logo [2]Australian Electoral
Commission
This email may contain legal advice that is subject to legal professional
privilege. Care should be taken to avoid unintended waiver of that
privilege. The Australian Electoral Commission’s Chief Legal Officer
should be consulted prior to any decision to disclose the existence or
content of any advice contained in this email to a third party.
--------------------------------------------------------------------------
-----Original Message-----
From: Josh Deprez [[3]mailto:[FOI #714 email]]
Sent: Thursday, 17 July 2014 11:36 PM
To: INFO
Subject: Freedom of Information request - Testing, installation, and
verification procedures and data for EasyCount Senate software
Dear Australian Electoral Commission,
I request under the Freedom of Information Act 1982 that you provide the
following documentation regarding the "EasyCount Senate (ECS)" software
("the software"):
- Manual and automated software testing procedures for verifying that the
software is operating correctly, including manual test procedures, unit
tests, integration tests, functional tests, end-to-end tests, table-driven
tests, smoke tests, fuzz tests, installation and compatibility tests,
regression tests, acceptance tests, performance tests and benchmarks,
security tests, usability tests, network and connectivity tests, and
concurrency tests;
- Data sets used in the software testing procedures listed above,
including real or simulated sets of votes, and real or simulated
preference distributions;
- Installation, maintenance, and troubleshooting instructions, including
the specifications of computer hardware, network configuration, operating
system and runtime environment configuration required for the testing
procedures, and,
- User manuals, guides, usage instructions, and parameter specifications
for use of the software.
The documents listed above are substantially different in nature to
documents that are the subject of other published FOI applications
currently known to me. I am not affiliated with any other FOI applicant.
This request is substantially different for the following reasons. First
and foremost, this is not a request for the EasyCount Senate software
itself. I am certain that the above-listed documentation regarding testing
procedures cannot possibly be a "trade secret," even those components
expressed in the form of source code, as adequate testing procedures
should not reveal or rely upon details of the methods by which the
software's results are determined---the testing procedures, when used,
would at most establish a lower bound on the trustworthiness of the
software. Furthermore, automated test procedures in the form of source
code (such as unit tests) are, conventionally, logically separated from
the software that is the subject of the testing procedures. Therefore the
ab ove-listed documents can be provided easily, electronically,
separately from and without risk of revealing any "trade secret", and with
minimal diversion of resources.
This request should not be considered "joined" to other pending
applications, for the above reasons. I also request that access fees and
charges for this request be waived as a matter of public interest. There
is clear and increasing public interest in the EasyCount Senate software
as evidenced by attention in the media. Release of the documents will
promote public trust in the AEC's systems.
Yours faithfully,
Josh Deprez
-------------------------------------------------------------------
Please use this email address for all replies to this request:
[4][FOI #714 email]
Is [5][AEC request email] the wrong address for Freedom of Information
requests to Australian Electoral Commission? If so, please contact us
using this form:
[6]https://www.righttoknow.org.au/change_re...
Write your response as plain text. Only send PDF documents as a last
resort. Government guidelines make it clear that PDF is not an acceptable
format for you to use in the delivery of government information.
[7]https://www.righttoknow.org.au/help/offi...
This request is being made by an individual using the Right to Know
website. The unique email address provided by the service for this request
satisfies s.15(2)(c) of the Freedom of Information Act.
Disclaimer: This message and any reply that you make will be published on
the internet. Our privacy and copyright policies:
[8]https://www.righttoknow.org.au/help/offi...
If you find this service useful as an FOI officer, please ask your web
manager to link to us from your organisation's FOI page.
-------------------------------------------------------------------
UNCLASSIFIED
UNCLASSIFIED
UNCLASSIFIED
DISCLAIMER:
If you have received this transmission in error please notify us
immediately by return email and delete all copies. If this email or any
attachments have been sent to you in error, that error does not constitute
waiver of any confidentiality, privilege or copyright in respect of
information in the email or attachments.
References
Visible links
1. http://emailfooter.aec.gov.au/email/
2. http://emailfooter.aec.gov.au/email-promo/
3. mailto:[FOI #714 email]
4. mailto:[FOI #714 email]
5. mailto:[AEC request email]
6. https://www.righttoknow.org.au/change_re...
7. https://www.righttoknow.org.au/help/offi...
8. https://www.righttoknow.org.au/help/offi...
Luke John left an annotation ()
The following Q and As were received whilst I was a scrutineer.
Q. Who has conducted the testing of EasyCount?
A. IV&V, based in Sydney.
Q. Specifically what they were testing for? Ideally this would also include a copy of all test results, and the final certification.
A. IV&V certified that the ECS program correctly applies the Senate scrutiny rules contained in Part XVIII of the Commonwealth Electoral Act 1918.
It's important to note that they are referring to multiple separate software as EasyCount as well as the complete system. So in that response they are saying that only a very limited portion of the EasyCount code was tested.
When asked in person at the central scrutiny station as to what auditing had occurred of the below the line input software, the answer was none.
Dear Mr Jones,
I am writing to enquire about the status of FOI Request No. LS5101 (your reference), as the letter dated 21 July 2014 from Mr Pirani indicated that a decision should have been expected by 18 August 2014.
Thank you for your assistance in this matter.
Kind regards,
Dr Josh Deprez
Dear Australian Electoral Commission,
Please pass this on to the person who conducts Freedom of Information reviews.
I am writing to request an internal review of Australian Electoral Commission's handling of my FOI request 'Testing, installation, and verification procedures and data for EasyCount Senate software'.
It is disappointing that there appears to have been no decision made on this FOI Request (your reference LS5101). The request was submitted on 17 July 2014 and acknowledged in the letter dated 21 July 2014 from Mr Pirani. The acknowledgement indicated that a decision should have been expected by 18 August 2014, yet as of 15 September 2014 no word of a decision has been received.
A full history of the FOI request and all correspondence is available on the Internet at this address: https://www.righttoknow.org.au/request/t...
Yours faithfully,
Dr Josh Deprez
Thank you for contacting us.
This is an automatic response from the Australian Electoral Commission to confirm we have received your email.
For more information on enrolling to vote, federal elections or the AEC, visit www.aec.gov.au.
Please do not respond to this email.
UNCLASSIFIED
Dear Mr Deprez
I refer to our FOI Request No. LS5101. I enclose a scanned letter to you
dated 21 October 2014 from Paul Pirani notifying you that a practical
refusal reason exists in relation to your FOI Request and consulting you
about how you wish to address this matter.
Regards
Owen Jones
Owen Jones | Senior Lawyer
Legal Services Section | Legal, Parliamentary and Procurement Branch
Australian Electoral Commission
T: (02) 6271 4528 | F: (02) 6293 7657
--------------------------------------------------------------------------
[1]Australian Electoral Commission logo [2]Australian Electoral
Commission
This email may contain legal advice that is subject to legal professional
privilege. Care should be taken to avoid unintended waiver of that
privilege. The Australian Electoral Commission’s Chief Legal Officer
should be consulted prior to any decision to disclose the existence or
content of any advice contained in this email to a third party.
--------------------------------------------------------------------------
UNCLASSIFIED
DISCLAIMER:
If you have received this transmission in error please notify us
immediately by return email and delete all copies. If this email or any
attachments have been sent to you in error, that error does not constitute
waiver of any confidentiality, privilege or copyright in respect of
information in the email or attachments.
References
Visible links
1. http://emailfooter.aec.gov.au/email/
2. http://emailfooter.aec.gov.au/email-promo/
Alex Sadleir left an annotation ()
These documents should exist based on this submission of the AEC to the 2003 Joint Standing Committee on Electoral Matters http://www.aph.gov.au/parliamentary_busi...
"8.9 [...] EasyCount is being developed to meet the Standard ISO 9126 'Software engineering - Product quality'. This is a rigorous Quality Assurance standard. Also, the AEC plans to have Easycount independently verified, as well as have code available for inspection by stakeholders.
8.10 The AEC will conduct extensive in house testing of the application prior to the engagement of an appropriately qualified external and independent testing authority.
[...]
8.12 In the interests of transparency, and because there are no security implications [For security reasons, Easycount will operate on stand alone machines, which will prevent hacking.], the code will be available for review."