Testing, installation, and verification procedures and data for EasyCount Senate software

Josh Deprez made this Freedom of Information request to Australian Electoral Commission

The request was refused by Australian Electoral Commission.

From: Josh Deprez

Delivered

Dear Australian Electoral Commission,

I request under the Freedom of Information Act 1982 that you provide the following documentation regarding the "EasyCount Senate (ECS)" software ("the software"):

- Manual and automated software testing procedures for verifying that the software is operating correctly, including manual test procedures, unit tests, integration tests, functional tests, end-to-end tests, table-driven tests, smoke tests, fuzz tests, installation and compatibility tests, regression tests, acceptance tests, performance tests and benchmarks, security tests, usability tests, network and connectivity tests, and concurrency tests;
- Data sets used in the software testing procedures listed above, including real or simulated sets of votes, and real or simulated preference distributions;
- Installation, maintenance, and troubleshooting instructions, including the specifications of computer hardware, network configuration, operating system and runtime environment configuration required for the testing procedures, and,
- User manuals, guides, usage instructions, and parameter specifications for use of the software.

The documents listed above are substantially different in nature to documents that are the subject of other published FOI applications currently known to me. I am not affiliated with any other FOI applicant. This request is substantially different for the following reasons. First and foremost, this is not a request for the EasyCount Senate software itself. I am certain that the above-listed documentation regarding testing procedures cannot possibly be a "trade secret," even those components expressed in the form of source code, as adequate testing procedures should not reveal or rely upon details of the methods by which the software's results are determined---the testing procedures, when used, would at most establish a lower bound on the trustworthiness of the software. Furthermore, automated test procedures in the form of source code (such as unit tests) are, conventionally, logically separated from the software that is the subject of the testing procedures. Therefore the above-listed documents can be provided easily, electronically, separately from and without risk of revealing any "trade secret", and with minimal diversion of resources.

This request should not be considered "joined" to other pending applications, for the above reasons. I also request that access fees and charges for this request be waived as a matter of public interest. There is clear and increasing public interest in the EasyCount Senate software as evidenced by attention in the media. Release of the documents will promote public trust in the AEC's systems.

Yours faithfully,

Josh Deprez

Link to this

Australian Electoral Commission

Thank you for contacting us.

This is an automatic response from the Australian Electoral Commission to confirm we have received your email.

For more information on enrolling to vote, federal elections or the AEC, visit www.aec.gov.au.

Please do not respond to this email.

Link to this

Alex Sadleir left an annotation ()

These documents should exist based on this submission of the AEC to the 2003 Joint Standing Committee on Electoral Matters http://www.aph.gov.au/parliamentary_busi...

"8.9 [...] EasyCount is being developed to meet the Standard ISO 9126 'Software engineering - Product quality'. This is a rigorous Quality Assurance standard. Also, the AEC plans to have Easycount independently verified, as well as have code available for inspection by stakeholders.
8.10 The AEC will conduct extensive in house testing of the application prior to the engagement of an appropriately qualified external and independent testing authority.
[...]
8.12 In the interests of transparency, and because there are no security implications [For security reasons, Easycount will operate on stand alone machines, which will prevent hacking.], the code will be available for review."

Link to this

From: Owen Jones
Australian Electoral Commission


Attachment image001.gif
4K Download

Attachment image005.gif
0K Download

Attachment image002.png
19K Download

Attachment image004.png
0K Download

Attachment LS5101 Letter to J Deprez acknowledging 20140721.pdf
305K Download View as HTML


UNCLASSIFIED

Dear Mr Deprez

I refer to your email of 17 July 2014 11:36 PM in which you make your FOI
Request in respect of this matter. I enclose a scanned letter to you dated
21 July 2014 from Paul Pirani, Chief Legal Officer acknowledging receipt
of your FOI Request and indicating how it will be processed.

Regards

Owen Jones

Owen Jones | Senior Lawyer

Legal Services Section | Legal, Parliamentary and Procurement Branch

Australian Electoral Commission

T: (02) 6271 4528 | F: (02) 6293 7657

--------------------------------------------------------------------------

[1]Australian Electoral Commission logo [2]Australian Electoral
Commission

This email may contain legal advice that is subject to legal professional
privilege. Care should be taken to avoid unintended waiver of that
privilege. The Australian Electoral Commission’s Chief Legal Officer
should be consulted prior to any decision to disclose the existence or
content of any advice contained in this email to a third party.

--------------------------------------------------------------------------

 

 

show quoted sections

Link to this

From: Owen Jones
Australian Electoral Commission


Attachment image001.gif
4K Download

Attachment image005.gif
0K Download

Attachment image003.png
19K Download

Attachment image006.png
0K Download

Attachment LS5101 Letter to J Deprez acknowledging 20140721.pdf
599K Download View as HTML


UNCLASSIFIED

Dear Mr Deprez

I refer to your email of 17 July 2014 11:36 PM in which you make your FOI
Request in respect of this matter. I enclose a scanned letter to you dated
21 July 2014 from Paul Pirani, Chief Legal Officer acknowledging receipt
of your FOI Request and indicating how it will be processed.

This supersedes my email to you of 21 July 2014 10:16 AM which forwarded
only the first page of Mr Pirani’s letter. I apologise for the omission

Regards

Owen Jones

Owen Jones | Senior Lawyer

Legal Services Section | Legal, Parliamentary and Procurement Branch

Australian Electoral Commission

T: (02) 6271 4528 | F: (02) 6293 7657

--------------------------------------------------------------------------

[1]Australian Electoral Commission logo [2]Australian Electoral
Commission

This email may contain legal advice that is subject to legal professional
privilege. Care should be taken to avoid unintended waiver of that
privilege. The Australian Electoral Commission’s Chief Legal Officer
should be consulted prior to any decision to disclose the existence or
content of any advice contained in this email to a third party.

--------------------------------------------------------------------------

 

 

show quoted sections

Link to this

Luke John left an annotation ()

The following Q and As were received whilst I was a scrutineer.

Q. Who has conducted the testing of EasyCount?
A. IV&V, based in Sydney.

Q. Specifically what they were testing for? Ideally this would also include a copy of all test results, and the final certification.
A. IV&V certified that the ECS program correctly applies the Senate scrutiny rules contained in Part XVIII of the Commonwealth Electoral Act 1918.

It's important to note that they are referring to multiple separate software as EasyCount as well as the complete system. So in that response they are saying that only a very limited portion of the EasyCount code was tested.

When asked in person at the central scrutiny station as to what auditing had occurred of the below the line input software, the answer was none.

Link to this

From: Josh Deprez

Delivered

Dear Mr Jones,

I am writing to enquire about the status of FOI Request No. LS5101 (your reference), as the letter dated 21 July 2014 from Mr Pirani indicated that a decision should have been expected by 18 August 2014.

Thank you for your assistance in this matter.

Kind regards,

Dr Josh Deprez

Link to this

From: Josh Deprez

Delivered

Dear Australian Electoral Commission,

Please pass this on to the person who conducts Freedom of Information reviews.

I am writing to request an internal review of Australian Electoral Commission's handling of my FOI request 'Testing, installation, and verification procedures and data for EasyCount Senate software'.

It is disappointing that there appears to have been no decision made on this FOI Request (your reference LS5101). The request was submitted on 17 July 2014 and acknowledged in the letter dated 21 July 2014 from Mr Pirani. The acknowledgement indicated that a decision should have been expected by 18 August 2014, yet as of 15 September 2014 no word of a decision has been received.

A full history of the FOI request and all correspondence is available on the Internet at this address: https://www.righttoknow.org.au/request/t...

Yours faithfully,

Dr Josh Deprez

Link to this

Australian Electoral Commission

Thank you for contacting us.

This is an automatic response from the Australian Electoral Commission to confirm we have received your email.

For more information on enrolling to vote, federal elections or the AEC, visit www.aec.gov.au.

Please do not respond to this email.

Link to this

From: Owen Jones
Australian Electoral Commission


Attachment image001.gif
4K Download

Attachment image003.png
19K Download

Attachment image005.gif
0K Download

Attachment image006.png
0K Download

Attachment LS5101 Letter to J Deprez practical refusal consultation 20141021.pdf
713K Download View as HTML


UNCLASSIFIED

Dear Mr Deprez

I refer to  our FOI Request No. LS5101. I enclose a scanned letter to you
dated 21 October 2014 from Paul Pirani notifying you that a practical
refusal reason exists in relation to your FOI Request and consulting you
about how you wish to address this matter.

Regards

Owen Jones

Owen Jones | Senior Lawyer

Legal Services Section | Legal, Parliamentary and Procurement Branch

Australian Electoral Commission

T: (02) 6271 4528 | F: (02) 6293 7657

--------------------------------------------------------------------------

[1]Australian Electoral Commission logo [2]Australian Electoral
Commission

This email may contain legal advice that is subject to legal professional
privilege. Care should be taken to avoid unintended waiver of that
privilege. The Australian Electoral Commission’s Chief Legal Officer
should be consulted prior to any decision to disclose the existence or
content of any advice contained in this email to a third party.

show quoted sections

Link to this

Daniel O'Connor left an annotation ()

Well that's not an internal review!

Link to this

Things to do with this request

Anyone:
Australian Electoral Commission only: