164:
intrusion. If ballots have not been stored successfully, advocates of risk-limiting audits say there should be a re-vote, or no result should be declared, which usually requires a re-vote, or results can be declared if "the number of questionable or missing audit records is small enough that they cannot alter the outcome of the contest." However, if storage or records are flawed, laws may require initial results to be accepted without audit. To provide an alternative to a re-vote, seven
Florida counties back up the paper ballots by copying them the day after they are voted, with machines independent of election machines. While any copy can have flaws, comparing cast vote records to these independent backup copies would give an alternative to re-voting or skipping the audit when storage is not trustworthy. Florida does not hand-check this backup, which would be required by a risk-limiting audit. Instead Florida machine-audits 100% of votes and contests. They have found discrepancies of 1-2 ballots from official machines. Maryland has a less safe alternate approach. Maryland's election machines create and store ballot images during the election, separate from the cast vote records. Most election machines do so. Maryland compares cast vote records to these ballot images from the same election machines. Unlike Florida, this approach is not an independent backup or check. A hack or bug in the election machine can alter, skip, or double-count both image and cast vote record simultaneously. Maryland's semi-independent checking is better than no checking, since it has found and resolved discrepancies, such as folded ballots leaving fold lines on the images, which computers interpreted as write-in votes; sensor flaws which left lines on the images, interpreted as
28:
include: the sample needs to be a large fraction of all ballots to minimize the chance of missing mistakes, if any contest is close; and it is hard to check computer totals publicly, except by releasing computer records to the public. If examining sampled ballots shows flaws in ballot storage, the usual approach cannot recover correct results, and researchers recommend a re-vote if the number of ballots held in flawed storage is enough to change winners. An alternative to re-votes is to create and verify backups of the paper ballots soon after they are voted, so there is an alternative to flawed storage of the original ballots.
181:
66:
and allowing no more than 10% of any mistaken outcomes to go undetected, method 1, ballot comparison, on average, needs 80 ballots, method 2, ballot polling, needs 700 ballots, and method 3, batch comparison, needs 13,000 ballots (in 26 batches). The methods are usually used to check computer counts, but methods 2 and 3 can also be used to check accuracy when the original results were hand-counted. The steps in each type of risk-limiting audit are:
214:
47:
297:
tallied the cast vote records correctly. Like any computer step this independent tally is subject to hacks and bugs, especially when voting rules are complex, such as variations in the number of candidates from different districts to vote for. The reason for the re-tabulation step is that independently programming a different kind of machine provides an independent check on official election machines.
156:
a random sample of contests, allowing a known risk that erroneous winners will take office; or a non-random sample, so no statistical confidence is available on the non-audited contests. Based on a formula, a sample size is determined for each contest being audited. The size of the sample depends primarily on the margin of victory in the targeted contest.
288:
interpretations into an online software tool, which is supposed to handle the comparison to the voting system interpretation, report discrepancies, and tell staff whether to sample further. It is also hard to prepare the list of ballots to sample from (ballot manifest) without using information from the election system.
259:
Method 1 requires the ballots to be kept in strict order so one can compare the computer interpretations of sampled ballots with those exact physical ballots. If the correct ballots are present, but out of order, method 2 can be used. Maryland, like other states, randomizes the order of paper ballots
198:
below) for a contest with a 2% margin of victory. It would need 3,000 ballots (50-100 staff hours in the city or county) for a 0.1% margin of victory. Method 2 or 3, ballot polling or batch comparison, would need a full hand count of the 4,000 ballots (70-130 staff hours). Margins under 0.1% occur in
155:
The process starts by selecting a "risk limit", such as 9% in
Colorado, meaning that if there are any erroneous winners in the initial results, the audit will catch at least 91% of them and let up to 9% stay undetected and take office. Another initial step is to decide whether to audit: all contests;
341:
require some form of results audit. Typically, these states prescribe audits that check only a small flat percentage, such as 1%, of voting machines. As a result, few jurisdictions have samples large or timely enough to detect and correct tabulation errors before election results are declared final.
296:
Method 1, ballot comparison, requires a second step, besides checking the sample of ballots: 100% of the computer interpretations of ballots ("cast vote records") need to be re-tabulated by computers independent of the original election computers. This re-tabulation checks whether election computers
255:
Experienced testers can usually bypass all physical security systems. Security equipment is vulnerable before and after delivery. Insider threats and the difficulty of following all security procedures are usually under-appreciated, and most organizations do not want to learn their vulnerabilities.
251:
No US state has adequate laws on physical security of the ballots. Security recommendations for elections include: starting audits as soon as possible after the election, regulating access to ballots and equipment, having risks identified by people other than those who design or manage the storage,
225:
An alternative to large samples is to audit an affordable sample size, and let the risk limit vary instead of the sample size. For a fixed sample, closer margins of victory would have more risk of letting erroneous winners take office, but any substantial sample would still have a known substantial
86:
Election results provide total for each batch of ballots (e.g. precinct); in a random sample of batches humans hand-count all ballots; for 100% of batches humans check by manual addition or independent computer if the election's initial summation of batches was correct; humans report any difference
39:
assume that all ballots, contests or machines were handled the same way, in which case spot checks could suffice. The sample sizes are designed to have a high chance of catching even a brief period when a scratch or fleck of paper blocks one sensor of one scanner, or a bug or hack switches votes in
385:
SEPARATION OF RESPONSIBILITIES: Neither the policy and regulation setting for the audit, nor the authority to judge whether an audit has satisfied those regulations, shall be solely in the hands of any entity directly involved with the tabulation of the ballots or the examination of ballots during
206:
Large numbers of contests on a ballot raise the chances that these small margins and large samples will occur in a jurisdiction, which is why no place does risk-limiting audits on all contests, leaving most local government races unaudited, though millions of dollars are at stake in local spending
131:
Hand-checking ballots (method 1) identifies bugs and hacks in how election computers interpret each ballot, so computer processing can be improved for future elections. Hand-counting ballots (methods 2 and 3) bypasses bugs and hacks in computer counts, so it does not identify exactly what mistakes
65:
There are three general types of risk-limiting audits. Depending on the circumstances of the election and the auditing method, different numbers of ballots need to be hand-checked. For example, in a jurisdiction with 64,000 ballots tabulated in batches of 500 ballots each, an 8% margin of victory,
27:
Advantages of an RLA include: samples can be small and inexpensive if the margin of victory is large; there are options for the public to watch and verify each step; and errors found in any step lead to corrective actions, including larger samples, up to a 100% hand count if needed. Disadvantages
221:
When
Maryland evaluated audit methods, it noted that local boards of elections could not budget, or plan staffing, for risk-limiting audits, since the sample "is highly dependent on the margin of victory in any given audited contest... A very close margin of victory could... require days of staff
73:
Election computers provide their interpretation of each ballot ("cast vote record"); humans check computers' "cast vote records" against stored physical ballots in a random sample of ballots; an independent computer tabulates all "cast vote records" independently of earlier tabulations to get new
300:
While all methods require physical security on the paper ballots, method 1 also requires enough security on the cast vote records so no one can change them. This can be accomplished by computer-calculating, storing and comparing a hash code for each file of cast vote records: (a) right after the
171:
When an audit produces the same result as initial election results, the outcome is confirmed, subject to the risk limit, and the audit is complete. If the audit sample shows enough discrepancies to call the outcome into question, a larger sample is selected and counted. This process can continue
163:
When storage is opened, records are checked to see if each sampled precinct still has the same number of ballots recorded during the election, if correct numbers appear on seals, if machines or containers have been tampered with in any way, and/or other methods to check if ballots have avoided
345:
In 2017, Colorado became the first state to implement ballot comparison audits, auditing one contest, not randomly chosen, in each of 50 of its 64 counties, several days after the election. Following the 2018 General
Election, Colorado will conduct audits in the 62 of its 64 counties that use
287:
All methods are designed to be independent of the election software, to ensure that an undetected error in the election software can be found by the audit. The audit in practice is dependent on its own software, separate from the election system. Election staff examine ballots and enter staff
159:
A random starting point (seed) is chosen by combining information from multiple independent people, to create a series of random numbers identifying specific ballots to pull from storage, such as the 23rd, 189th, 338th, 480th ballots in precinct 1, and other random numbers in other precincts.
378:
EXAMINATION OF VOTER-VERIFIABLE PAPER BALLOTS: Audits require human examination of voter-marked paper ballots – the ground truth of the election. Voter-marked paper ballots may be marked by hand or by ballot marking device. Audits cannot rely on scanned images or machine
304:
Colorado says it has a system to do the independent count of cast vote records, but it is not yet publicly documented, so the chance of bugs or hacks affecting this independent computer at the
Secretary of State's office along with one or more of the election machines is unknown.
132:
were made. Independently totaling cast vote records (method 1) or batch totals (method 3) identifies bugs and hacks in how election computers calculate totals. Method 2 does not need this independent totaling step, since it has a large enough sample to identify winners directly.
20:(RLA) is a post-election tabulation auditing procedure which can limit the risk that the reported outcome in an election contest is incorrect. It generally involves (1) storing voter-verified paper ballots securely until they can be checked, and (2) manually examining a
274:
All the methods, when done for a state-wide election, involve manual work throughout the state, wherever ballots are stored, so the public and candidates need observers at every location to be sure procedures are followed. However, in
Colorado and most states the law
2045:
1454:
229:
The sample sizes presented will be enough to confirm a result, subject to the risk limit, when the apparent winner is the actual winner. If the sample does not confirm the win, more ballots are sampled, up to a 100% hand count to confirm a different winner.
95:
Procedure to re-count all paper ballots more accurately if errors are detected. This is usually planned as a 100% manual count, but could involve fixing or replacing erroneous computers, doing a new computer count, and auditing that, until an audit shows no
322:
Cost depends on pay levels and staff time needed, recognizing that staff generally work in teams of two or three (one to read and one or two to record votes). Teams of four, with two to read and two to record are more secure and would increase costs.
326:
Each minute per vote checked means 25 cents per vote at $ 15/hour, or $ 250 per thousand votes. Checking random ballots can take more time: pulling individual ballots from boxes and returning them to the same spot. It is relevant to methods 1 and 2.
409:
By selecting samples of varying sizes dictated by statistical risk, risk-limiting audits eliminate the need to count all the ballots to obtain a rapid test of the outcome (that, is, who won?), while providing some level of statistical confidence.
135:
Colorado uses method 1 in most counties, and method 2 in a few counties which use election machines which do not record and store "cast vote records". Colorado uses no audit method in two counties which hand-count ballots in the first place.
392:
COMPREHENSIVENESS: All jurisdictions and all validly cast ballots, including absentee, mail-in and accepted provisional ballots, must be taken into account. No contest should be excluded a priori from auditing, although some contests may be
405:"Commission endorses both risk-limiting audits that ensure the correct winner has been determined according to a sample of votes cast, and performance audits that evaluate whether the voting technology performs as promised and expected."
751:
207:
and land use decisions. Colorado picks contests with wider margins to avoid large samples. California's rules for 2019–2021 require any RLA to audit all contests, and no election offices have chosen to use RLAs under these rules.
349:
Rhode Island passed legislation requiring that state's Board of
Elections to implement risk-limiting audits beginning in 2018. Individual jurisdictions elsewhere may be using the method on the local election clerks' initiative.
35:, the goal is to identify not only intentional alterations of ballots and tallies, but also bugs in election machines, such as software errors, scanners with blocked sensors or scanners skipping some ballots. The approach does
114:
Doing the manual check immediately when the sample is selected; if insiders have altered computer files, they could use any delay to change sampled ballots to match the erroneous computer files, thus hiding the
193:
Sample sizes rise rapidly for narrow margins of victory, with all methods. In a small city or county, with 4,000 ballots, method 1, ballot comparison, would need 300 ballots (300–600 minutes, as discussed in
108:
Selecting a random sample after initial results are public, because telling hackers in advance which contests and ballots will be in the sample, lets them freely hack other contests and ballots.
1403:
210:
The power of the sample also depends on staff expanding the audit after any discrepancy, rather than dismissing it as a clerical error, or re-scanning problematic ballots to fix just them.
102:
Clarifying which contests were audited and which were not, or auditing all contests or a large enough random sample of contests so the chance of missing erroneous results is acceptably low.
121:
Having enough independent participants select different digits of the random number seed, so no one can control the seed and hence the random number series which selects the random sample.
226:
chance of catching errors. Election managers would announce the level of confidence provided by the sample, and would have procedures to follow up if the sample finds one or more errors.
802:
Lindeman, Mark (executive editor), Jennie
Bretschneider, Sean Flaherty, Susannah Goodman, Mark Halvorson, Roger Johnston, Ronald L. Rivest, Pam Smith, Philip B. Stark (October 1, 2012).
308:
California's process for risk-limiting audits omits the step of independent totals. When it did a pilot, independent totals were calculated by a student on a university computer.
382:
TRANSPARENCY: Elections belong to the public. The public must be able to observe the audit and verify that it has been conducted correctly, without interfering with the process.
2154:
Trautgott, Michael W.; Conrad, Frederick G. (2012). "Confidence in the
Electoral System: Why We Do Auditing". In Alvarex, R. Michael; Atkeson, Lonna Rae; Hall, Thad E. (eds.).
1747:
1623:
2144:, Pamela Smith, Verified Voting Foundation; Michelle Mulder, Rutgers School of Law-Newark; Susannah Goodman, Common Cause Education Fund, 2012. Accessed May 12, 2017
1584:
Experts recommend letting different party members ride in the ballot transport vehicle, letting other observers accompany the vehicle, and using guards when needed:
2003:
1375:
727:
1310:
1210:
200:
420:
Professor
Phillip Stark of the University of California at Berkeley has posted tools for the conduct of risk-limiting audits on the university's website.
124:
Having the public see all steps, including the content of ballots and computer records while officials examine them, to know they are counted accurately.
80:
Humans count a random sample of ballots; humans report any difference between manual percentage for the sample and computer percentage for the election.
32:
2126:
2031:
1880:
1793:
1733:
1264:
1042:
845:
687:
863:
1148:
Benaloh, Josh; Jones, Douglas; Lazarus, Eric; Lindeman, Mark; Stark, Philip B. (2011). "SOBA: Secrecy-preserving Observable Ballot-level Audit".
260:
and cast vote records to protect ballot secrecy, so method 1 cannot be done there, since paper ballots and cast vote records cannot be compared.
889:
2094:"Post-Election Auditing: Effects of Procedure and Ballot Type on Manual Counting Accuracy, Efficiency, and Auditor Satisfaction and Confidence"
1690:
1560:
1826:
1170:
546:. In Bernhard, Matthew; Bracciali, Andrea; Camp, L. Jean; Matsuo, Shin'ichiro; Maurushat, Alana; Rønne, Peter B.; Sala, Massimiliano (eds.).
389:
BALLOT PROTECTION: All the ballots being tabulated and audited must be verifiably protected from loss, substitution, alteration or addition.
1346:
252:
using background checks and tamper-evident seals. However seals on plastic surfaces can typically be removed and reapplied without damage.
1852:
1236:
2231:
1948:
973:
1347:""Exhibit B. Pages 20-23 in Lamone, Linda H. (December 22, 2016). "Joint Chairman's Report on the 2016 Post-Election Tabulation Audit"
2140:
1658:
Electronic voting : second International Joint Conference, E-Vote-ID 2017, Bregenz, Austria, October 24-27, 2017, proceedings
2336:
2206:
1184:
2275:, Final Report to the United States Election Assistance Commission, California Secretary of State 2013. Accessed May 12, 2017
2163:
573:
2072:"Post-Election Risk-Limiting Audit Pilot Program 2011-2013 Final Report to the United States Election Assistance Commission"
244:
Ballots are at risk when being transported from drop boxes and polling places to central locations, and may be protected by
118:
Having enough security on the ballots during transportation and storage, so neither insiders nor outsiders can change them.
1705:
480:
2271:
374:
and several election integrity groups endorsed all three methods of risk-limited audits. Their first five criteria are:
1533:
1013:
Proceedings of the USENIX Electronic Voting Technology Workshop/Workshop on Trustworthy Elections (EVT/WOTE), Aug. 2010
398:
943:
1808:
1666:
2258:
The American Voting Experience: Report and Recommendations of the Presidential Commission on Election Administration
2071:
1704:
Garland, Lynn, Mark Lindeman, Neal McBurnett, Jennifer Morrell, Marian Schneider, Stephanie Singer (December 2018).
338:
276:
359:
1973:
915:"Report on Election Auditing by the Election Audits Task Force of the League of Women Voters of the United States"
1185:"2015 North Carolina General Statutes: Principles and rules for counting official ballots. § 163-182.1 - 1(b)(1)"
1021:
550:. Lecture Notes in Computer Science. Vol. 12063. Cham: Springer International Publishing. pp. 319–336.
172:
until the sample confirms the original winner, or a different winner is determined by hand-counting all ballots.
2257:
628:"An Introduction to Risk-Limiting Audits and Evidence-Based Elections Prepared for the Little Hoover Commission"
449:
414:
598:
128:
The last three items are hard in one-party states, where all participants may be swayed by the ruling party.
239:
2326:
1776:
2180:
2316:
363:
43:
Comparisons can be done ballot-by-ballot or precinct-by-precinct, though the latter is more expensive.
1005:
803:
245:
105:
Auditing a large enough random sample of ballots so the chance of missing mistakes is acceptably low.
50:
Sample sizes depend on: winning margin, number of ballots voted, confidence level, and type of audit.
1117:
1097:
2331:
658:
Benaloh, Josh, Douglas Jones, Eric L. Lazarus, Mark Lindeman, Philip B. Stark (September 9, 2011).
317:
2181:"Colorado's first-of-its-kind election audit is complete, with all participating counties passing"
1481:
659:
1974:"Colorado Secretary of State Election Rules [8 CCR 1505-1], Rule 25. Post-election audit"
1429:
543:
1809:"Vulnerability & Security Assessment Report Election Systems &Software's Unity 3.4.1.0"
1112:
838:
521:
2120:
1036:
21:
417:
initiated grants for pilot projects to test and demonstrate the method in actual elections.
217:
When sample sizes are limited by budget, they still have some likelihood of catching errors.
2285:
1923:
627:
301:
election, (b) when independent tabulation is done, and (c) when ballot comparison is done.
40:
one precinct or one contest, if these problems affect enough ballots to change the result.
1586:
706:
8:
444:
2093:
2002:
Lindeman, Mark, Ronald L. Rivest, Philip B. Stark and Neal McBurnett (January 3, 2018).
2321:
1684:
1149:
1130:
579:
551:
346:
automated vote counting equipment (the two remaining counties hand count the ballots).
2232:"Passing Your Vote Through Security: The Rise of Risk Limiting Audits in Rhode Island"
2159:
2025:
1874:
1787:
1727:
1672:
1662:
1656:
1630:
1258:
1073:
826:
681:
583:
569:
1134:
2108:
1897:
1122:
1063:
561:
168:; and double-feeds where two ballots overlap in the scanner, and one is uncounted.
1604:
2156:
Confirming Elections: Creating Confidence and Integrity Through Election Auditing
914:
565:
439:
429:
371:
60:
24:
of the paper ballots until enough evidence is gathered to meet the risk limit.
1814:– via Freeman, Craft, MacGregor Group for California Secretary of State.
1676:
1068:
1051:
864:"Saving throw: securing democracy with stats, spreadsheets, and 10-sided dice"
99:
Auditing all types of ballots, including military, absentee, provisional, etc.
2310:
1774:
1634:
1077:
111:
Selecting the random sample before results are final, so errors can be fixed.
2141:
Counting Votes 2012: A State by State Look at Voting Technology Preparedness
1561:"Vaccine Shipments Present a Security Challenge Worthy of a James Bond Film"
1211:"Leon County among Florida precincts to implement Clear Audit for elections"
1280:"Ballot Images – A new way to verify that results are true"
1279:
367:
213:
74:
totals; humans report any differences in interpretations and total tallies.
2112:
180:
1126:
184:
Colorado elections in 2017, sample sizes needed for risk-limiting audits
269:
46:
1557:
Truckers have developed high security systems for valuable shipments.
974:"Target contests for risk-limiting audits per Election Rule 25.2.2(i)"
434:
1507:
1020:
Lindeman, Mark, Philip B. Stark, Vincent S. Yates (August 6, 2012).
544:"Sets of Half-Average Nulls Generate Risk-Limiting Audits: SHANGRLA"
1706:"Principles and Best Practices for Post-Election Tabulation Audits"
556:
481:"Principles and Best Practices for Post-Election Tabulation Audits"
248:
tracking, guards, security systems, and/or a convoy of the public.
165:
2092:
Goggin, Stephen N., Michael D. Byrne, and Juan E. Gilbert (2012).
1655:
Benaloh, Public Evidence from Secret Ballots; et al. (2017).
1154:
379:
interpretations of the ballots to accurately reflect voter intent.
1775:
Coherent Cyber, Freeman, Craft McGregor Group (August 28, 2017).
1308:
660:"SOBA: Secrecy-preserving Observable Ballot-level Audit (slides)"
2004:"Comments re statistics of auditing the 2018 Colorado elections"
1860:
Bowen center for Public Affairs and Colorado Secretary of State
1022:"BRAVO: Ballot-polling Risk-limiting Audits to Verify Outcomes"
222:
work, possibly compromising the local certification deadline."
1006:"Super-Simple Simultaneous Single-Ballot Risk-Limiting Audits"
531:– via University of California at Berkeley, pages 1, 3.
599:"The Humboldt County Election Transparency Project and TEVS"
1147:
971:
2272:
Post-Election Risk-Limiting Audit Pilot Program 2011-2013
1853:"Colorado Risk-Limiting Audit:Conception to Application"
1605:"South Africa: Most Mpumalanga Ballot Papers Recovered"
1171:"Records: Too many votes in 37% of Detroit's precincts"
700:
Examples of blocked or faulty scanner sensors include:
143:
to determine if votes were tabulated accurately, not a
1782:. p. 9 – via California Secretary of State.
1309:
Maryland State Board of Elections (October 21, 2016).
1234:
2207:"Audited contests in each county for 2017 RLA (XLSX)"
1311:"Post-Election Tabulation Audit Pilot Program Report"
2286:"Tools for Comparison Risk-Limiting Election Audits"
1777:"Security Test Report ES&S Electionware 5.2.1.0"
1534:"Security and ballot drop boxes: How it works in WA"
2211:
www.sos.state.co.us/pubs/elections/auditCenter.html
948:
www.sos.state.co.us/pubs/elections/auditCenter.html
1602:Ballots have fallen from trucks during transport:
1430:"Land Use Decisions and Decision Making Processes"
399:Presidential Commission on Election Administration
240:Election audit § Ballot transport and storage
804:"Risk-Limiting Post-Election Audits: Why and How"
653:
651:
147:, to determine if good procedures were followed.
2308:
1851:Lovato, Jerome, Danny Casias, and Jessi Romero.
2153:
1827:"Physical Security Threats and Vulnerabilities"
1587:"Securing Ballot Boxes During Transportation —"
750:Ryan, Tom and Benny White (November 30, 2016).
233:
1383:National Institute of Standards and Technology
967:
965:
735:National Institute of Standards and Technology
707:"Hart InterCivic Optical-Scan Has a Weak Spot"
648:
1091:
1089:
861:
522:"Gentle Introduction to Risk-limiting Audits"
2125:: CS1 maint: multiple names: authors list (
2030:: CS1 maint: multiple names: authors list (
1879:: CS1 maint: multiple names: authors list (
1792:: CS1 maint: multiple names: authors list (
1732:: CS1 maint: multiple names: authors list (
1263:: CS1 maint: multiple names: authors list (
1235:Dailey, Aikens, Maddox (November 21, 2016).
1141:
1052:"Sequential Tests of Statistical Hypotheses"
1041:: CS1 maint: multiple names: authors list (
972:Colorado Secretary of State (July 5, 2018).
686:: CS1 maint: multiple names: authors list (
596:
279:any of the audit work to be done in public.
1949:"2 California Code of Regulations 20123(d)"
1892:
1890:
1768:
1304:
1302:
1300:
962:
87:between original tallies and audit tallies.
1689:: CS1 maint: location missing publisher (
1237:"Voting System Post-Election Audit Report"
1086:
318:Vote counting § Cost of manual counts
1578:
1153:
1116:
1067:
1000:Formulae are presented and explained in:
857:
855:
844:CS1 maint: multiple names: authors list (
555:
2038:
2001:
1995:
1887:
1806:
1800:
1740:
1624:"A Ballad of the Ballot Follies in D.C."
1447:
1297:
1019:
994:
801:
606:Report for Elections Advisory Commission
548:Financial Cryptography and Data Security
515:
513:
511:
509:
507:
505:
475:
473:
471:
469:
467:
465:
401:recommended the methods in broad terms:
212:
179:
45:
1824:
1713:American Statistical Association, et al
1703:
1654:
1558:
1551:
1479:
1376:"2016 Post Election Audits in Maryland"
1367:
1340:
1338:
1336:
1334:
1162:
1096:Stark, PB and DA Wagner (May 8, 2012).
887:
881:
728:"2016 Post Election Audits in Maryland"
657:
54:
2309:
2224:
2204:
2198:
2091:
2085:
1966:
1844:
1818:
1531:
1525:
1499:
1482:"Electronic voting and partial audits"
1480:Mercuri, Rebecca (February 21, 2008).
1373:
1277:
1271:
1208:
852:
752:"Transcript of Email on Ballot Images"
725:
704:
282:
2064:
1941:
1916:
1648:
1396:
1374:Walker, Natasha (February 16, 2017).
1168:
1095:
1056:The Annals of Mathematical Statistics
1003:
797:
726:Walker, Natasha (February 16, 2017).
694:
625:
619:
597:Trachtenberg, Mitch (July 29, 2013).
541:
519:
502:
462:
291:
2178:
1807:Stauffer, Jacob (November 4, 2016).
1621:
1508:"The Four Fatal Flaws of RLA Audits"
1473:
1344:
1331:
1228:
1049:
811:University of California at Berkeley
795:
793:
791:
789:
787:
785:
783:
781:
779:
777:
749:
263:
1622:King, Colbert (September 9, 2006).
1532:Santos, Melissa (August 19, 2020).
1202:
936:
330:
201:one in sixty to one in 460 contests
13:
2205:Casias, Danny (November 8, 2017).
2172:
1850:
1697:
1661:. Cham, Switzerland. p. 122.
1422:
635:University of California, Berkeley
590:
14:
2348:
2179:Paul, Jesse (November 22, 2017).
1825:Seivold, Garett (April 2, 2018).
1755:US Election Assistance Commission
1559:Buckley, Thomas (March 2, 2021).
1354:Maryland State Board of Elections
1345:Vora, Poorvi (November 6, 2016).
1169:Kurth, Joel (December 13, 2016).
774:
150:
1924:"Risk-Limiting Audit (RLA) FAQs"
1505:
1278:Harris, Bev (November 2, 2016).
1004:Stark, Philip (August 1, 2010).
888:Volkosh, Dan (January 4, 2018).
520:Stark, Philip (March 16, 2012).
360:American Statistical Association
2278:
2264:
2250:
2147:
2133:
1209:Stofan, Jake (April 25, 2018).
907:
862:Cyrus Farivar (July 25, 2012).
626:Stark, Philip (July 26, 2018).
353:
2337:Elections in the United States
2050:California Code of Regulations
1748:"Chapter 3. Physical Security"
1459:California Code of Regulations
1404:"State and Local Expenditures"
535:
450:List of close election results
415:Election Assistance Commission
188:
1:
1953:California Secretary of State
705:Gideon, John (July 5, 2005).
455:
566:10.1007/978-3-030-54455-3_23
234:Ballot transport and storage
7:
2011:Colorado Secretary of State
1981:Colorado Secretary of State
894:Colorado Secretary of State
423:
139:Risk-limiting audits are a
10:
2353:
1098:"Evidence-Based Elections"
364:Brennan Center for Justice
315:
267:
237:
58:
1105:IEEE Security and Privacy
1050:Wald, A. (June 1, 1945).
542:Stark, Philip B. (2020).
529:IEEE Security and Privacy
195:
175:
2079:California Sec. of State
1182:and North Carolina law:
337:As of early 2017, about
1831:losspreventionmedia.com
1069:10.1214/aoms/1177731118
944:"Round #1 State Report"
890:"Voting Systems Public"
311:
2158:. Palgrave MacMillan.
1318:elections.maryland.gov
922:League of Women Voters
218:
185:
91:All methods require:
51:
2290:www.stat.berkeley.edu
2113:10.1089/elj.2010.0098
413:In 2011, the federal
268:Further information:
216:
183:
59:Further information:
49:
2101:Election Law Journal
1434:University of Oregon
1244:Leon County, Florida
759:Pima County, Arizona
55:Categories of audits
2327:Statistical methods
1928:www.sos.state.co.us
1904:. February 10, 2017
1512:Citizens' Oversight
1127:10.1109/MSP.2012.62
950:. November 20, 2017
711:www.votersunite.org
445:Electoral integrity
283:Software dependence
18:risk-limiting audit
2238:. January 29, 2018
2236:State of Elections
1898:"State Audit Laws"
1284:BlackBoxVoting.org
837:has generic name (
488:ElectionAudits.org
292:Independent totals
219:
186:
71:Ballot comparison.
52:
22:statistical sample
2317:Types of auditing
2165:978-0-230-34121-0
2052:. August 21, 2020
1983:. August 23, 2018
1461:. August 21, 2020
575:978-3-030-54455-3
264:Public monitoring
84:Batch comparison.
2344:
2301:
2300:
2298:
2296:
2282:
2276:
2268:
2262:
2254:
2248:
2247:
2245:
2243:
2228:
2222:
2221:
2219:
2217:
2202:
2196:
2195:
2193:
2191:
2176:
2170:
2169:
2151:
2145:
2137:
2131:
2130:
2124:
2116:
2098:
2089:
2083:
2082:
2081:. July 31, 2014.
2076:
2068:
2062:
2061:
2059:
2057:
2042:
2036:
2035:
2029:
2021:
2019:
2017:
2008:
1999:
1993:
1992:
1990:
1988:
1978:
1970:
1964:
1963:
1961:
1959:
1945:
1939:
1938:
1936:
1934:
1920:
1914:
1913:
1911:
1909:
1894:
1885:
1884:
1878:
1870:
1868:
1866:
1857:
1848:
1842:
1841:
1839:
1837:
1822:
1816:
1815:
1813:
1804:
1798:
1797:
1791:
1783:
1781:
1772:
1766:
1765:
1763:
1761:
1752:
1744:
1738:
1737:
1731:
1723:
1721:
1719:
1710:
1701:
1695:
1694:
1688:
1680:
1652:
1646:
1645:
1643:
1641:
1620:
1618:
1616:
1611:. April 22, 2009
1601:
1599:
1597:
1582:
1576:
1575:
1573:
1571:
1555:
1549:
1548:
1546:
1544:
1529:
1523:
1522:
1520:
1518:
1503:
1497:
1496:
1494:
1492:
1477:
1471:
1470:
1468:
1466:
1451:
1445:
1444:
1442:
1440:
1426:
1420:
1419:
1417:
1415:
1400:
1394:
1393:
1391:
1389:
1380:
1371:
1365:
1364:
1362:
1360:
1351:
1342:
1329:
1328:
1326:
1324:
1315:
1306:
1295:
1294:
1292:
1290:
1275:
1269:
1268:
1262:
1254:
1252:
1250:
1241:
1232:
1226:
1225:
1223:
1221:
1206:
1200:
1199:
1197:
1195:
1181:
1179:
1177:
1166:
1160:
1159:
1157:
1145:
1139:
1138:
1120:
1102:
1093:
1084:
1081:
1071:
1046:
1040:
1032:
1026:
1016:
1010:
998:
992:
991:
989:
987:
978:
969:
960:
959:
957:
955:
940:
934:
933:
931:
929:
919:
911:
905:
904:
902:
900:
885:
879:
878:
876:
874:
859:
850:
849:
842:
836:
832:
830:
822:
820:
818:
813:. pp. 3, 16
808:
799:
772:
769:
767:
765:
756:
745:
743:
741:
732:
721:
719:
717:
698:
692:
691:
685:
677:
675:
673:
664:
655:
646:
645:
643:
641:
632:
623:
617:
616:
614:
612:
603:
594:
588:
587:
559:
539:
533:
532:
526:
517:
500:
499:
497:
495:
485:
477:
331:State variations
277:does not require
2352:
2351:
2347:
2346:
2345:
2343:
2342:
2341:
2332:Electoral fraud
2307:
2306:
2305:
2304:
2294:
2292:
2284:
2283:
2279:
2269:
2265:
2255:
2251:
2241:
2239:
2230:
2229:
2225:
2215:
2213:
2203:
2199:
2189:
2187:
2185:The Denver Post
2177:
2173:
2166:
2152:
2148:
2138:
2134:
2118:
2117:
2096:
2090:
2086:
2074:
2070:
2069:
2065:
2055:
2053:
2044:
2043:
2039:
2023:
2022:
2015:
2013:
2006:
2000:
1996:
1986:
1984:
1976:
1972:
1971:
1967:
1957:
1955:
1947:
1946:
1942:
1932:
1930:
1922:
1921:
1917:
1907:
1905:
1902:Verified Voting
1896:
1895:
1888:
1872:
1871:
1864:
1862:
1855:
1849:
1845:
1835:
1833:
1823:
1819:
1811:
1805:
1801:
1785:
1784:
1779:
1773:
1769:
1759:
1757:
1750:
1746:
1745:
1741:
1725:
1724:
1717:
1715:
1708:
1702:
1698:
1682:
1681:
1669:
1653:
1649:
1639:
1637:
1627:Washington Post
1614:
1612:
1603:
1595:
1593:
1585:
1583:
1579:
1569:
1567:
1556:
1552:
1542:
1540:
1530:
1526:
1516:
1514:
1504:
1500:
1490:
1488:
1478:
1474:
1464:
1462:
1455:"2 CCR § 20114"
1453:
1452:
1448:
1438:
1436:
1428:
1427:
1423:
1413:
1411:
1410:. March 5, 2015
1408:Urban Institute
1402:
1401:
1397:
1387:
1385:
1378:
1372:
1368:
1358:
1356:
1349:
1343:
1332:
1322:
1320:
1313:
1307:
1298:
1288:
1286:
1276:
1272:
1256:
1255:
1248:
1246:
1239:
1233:
1229:
1219:
1217:
1207:
1203:
1193:
1191:
1183:
1175:
1173:
1167:
1163:
1146:
1142:
1118:10.1.1.299.8807
1100:
1094:
1087:
1034:
1033:
1024:
1008:
999:
995:
985:
983:
981:sos.state.co.us
976:
970:
963:
953:
951:
942:
941:
937:
927:
925:
917:
913:
912:
908:
898:
896:
886:
882:
872:
870:
860:
853:
843:
834:
833:
824:
823:
816:
814:
806:
800:
775:
763:
761:
754:
739:
737:
730:
715:
713:
699:
695:
679:
678:
671:
669:
667:USENIX EVT/WOTE
662:
656:
649:
639:
637:
630:
624:
620:
610:
608:
601:
595:
591:
576:
540:
536:
524:
518:
503:
493:
491:
483:
479:
478:
463:
458:
440:Electoral fraud
430:Election audits
426:
356:
339:half the states
333:
320:
314:
294:
285:
272:
266:
242:
236:
191:
178:
153:
78:Ballot polling.
63:
57:
33:election audits
12:
11:
5:
2350:
2340:
2339:
2334:
2329:
2324:
2319:
2303:
2302:
2277:
2263:
2261:, January 2014
2249:
2223:
2197:
2171:
2164:
2146:
2132:
2084:
2063:
2037:
1994:
1965:
1940:
1915:
1886:
1843:
1817:
1799:
1767:
1739:
1696:
1667:
1647:
1591:aceproject.org
1577:
1550:
1524:
1498:
1472:
1446:
1421:
1395:
1366:
1330:
1296:
1270:
1227:
1201:
1161:
1140:
1085:
1083:
1082:
1062:(2): 117–186.
1047:
1017:
993:
961:
935:
906:
880:
851:
773:
771:
770:
746:
722:
693:
647:
618:
589:
574:
534:
501:
460:
459:
457:
454:
453:
452:
447:
442:
437:
432:
425:
422:
407:
406:
395:
394:
390:
387:
383:
380:
372:Public Citizen
355:
352:
332:
329:
316:Main article:
313:
310:
293:
290:
284:
281:
265:
262:
238:Main article:
235:
232:
190:
187:
177:
174:
152:
151:Implementation
149:
126:
125:
122:
119:
116:
112:
109:
106:
103:
100:
97:
89:
88:
81:
75:
61:Election audit
56:
53:
31:As with other
9:
6:
4:
3:
2:
2349:
2338:
2335:
2333:
2330:
2328:
2325:
2323:
2320:
2318:
2315:
2314:
2312:
2291:
2287:
2281:
2274:
2273:
2267:
2260:
2259:
2253:
2237:
2233:
2227:
2212:
2208:
2201:
2186:
2182:
2175:
2167:
2161:
2157:
2150:
2143:
2142:
2136:
2128:
2122:
2114:
2110:
2106:
2102:
2095:
2088:
2080:
2073:
2067:
2051:
2047:
2046:"2 CCR 20125"
2041:
2033:
2027:
2012:
2005:
1998:
1982:
1975:
1969:
1954:
1950:
1944:
1929:
1925:
1919:
1903:
1899:
1893:
1891:
1882:
1876:
1861:
1854:
1847:
1832:
1828:
1821:
1810:
1803:
1795:
1789:
1778:
1771:
1756:
1749:
1743:
1735:
1729:
1714:
1707:
1700:
1692:
1686:
1678:
1674:
1670:
1668:9783319686875
1664:
1660:
1659:
1651:
1636:
1632:
1628:
1625:
1610:
1606:
1592:
1588:
1581:
1566:
1565:Bloomberg.com
1562:
1554:
1539:
1535:
1528:
1513:
1509:
1502:
1487:
1483:
1476:
1460:
1456:
1450:
1435:
1431:
1425:
1409:
1405:
1399:
1384:
1377:
1370:
1355:
1348:
1341:
1339:
1337:
1335:
1319:
1312:
1305:
1303:
1301:
1285:
1281:
1274:
1266:
1260:
1245:
1238:
1231:
1216:
1212:
1205:
1190:
1186:
1172:
1165:
1156:
1151:
1144:
1136:
1132:
1128:
1124:
1119:
1114:
1110:
1106:
1099:
1092:
1090:
1079:
1075:
1070:
1065:
1061:
1057:
1053:
1048:
1044:
1038:
1030:
1023:
1018:
1014:
1007:
1002:
1001:
997:
982:
975:
968:
966:
949:
945:
939:
923:
916:
910:
895:
891:
884:
869:
865:
858:
856:
847:
840:
828:
812:
805:
798:
796:
794:
792:
790:
788:
786:
784:
782:
780:
778:
760:
753:
747:
736:
729:
723:
712:
708:
702:
701:
697:
689:
683:
668:
661:
654:
652:
636:
629:
622:
607:
600:
593:
585:
581:
577:
571:
567:
563:
558:
553:
549:
545:
538:
530:
523:
516:
514:
512:
510:
508:
506:
489:
482:
476:
474:
472:
470:
468:
466:
461:
451:
448:
446:
443:
441:
438:
436:
433:
431:
428:
427:
421:
418:
416:
411:
404:
403:
402:
400:
397:In 2014, the
391:
388:
384:
381:
377:
376:
375:
373:
369:
365:
361:
351:
347:
343:
340:
335:
328:
324:
319:
309:
306:
302:
298:
289:
280:
278:
271:
261:
257:
253:
249:
247:
241:
231:
227:
223:
215:
211:
208:
204:
202:
197:
182:
173:
169:
167:
161:
157:
148:
146:
145:process audit
142:
141:results audit
137:
133:
129:
123:
120:
117:
113:
110:
107:
104:
101:
98:
94:
93:
92:
85:
82:
79:
76:
72:
69:
68:
67:
62:
48:
44:
41:
38:
34:
29:
25:
23:
19:
2293:. Retrieved
2289:
2280:
2270:
2266:
2256:
2252:
2240:. Retrieved
2235:
2226:
2214:. Retrieved
2210:
2200:
2188:. Retrieved
2184:
2174:
2155:
2149:
2139:
2135:
2121:cite journal
2104:
2100:
2087:
2078:
2066:
2056:September 3,
2054:. Retrieved
2049:
2040:
2014:. Retrieved
2010:
1997:
1985:. Retrieved
1980:
1968:
1958:February 11,
1956:. Retrieved
1952:
1943:
1933:February 11,
1931:. Retrieved
1927:
1918:
1906:. Retrieved
1901:
1863:. Retrieved
1859:
1846:
1836:November 17,
1834:. Retrieved
1830:
1820:
1802:
1770:
1760:November 17,
1758:. Retrieved
1754:
1742:
1718:November 17,
1716:. Retrieved
1712:
1699:
1657:
1650:
1638:. Retrieved
1626:
1613:. Retrieved
1608:
1594:. Retrieved
1590:
1580:
1568:. Retrieved
1564:
1553:
1541:. Retrieved
1537:
1527:
1517:September 3,
1515:. Retrieved
1511:
1501:
1491:September 3,
1489:. Retrieved
1485:
1475:
1465:September 3,
1463:. Retrieved
1458:
1449:
1437:. Retrieved
1433:
1424:
1412:. Retrieved
1407:
1398:
1386:. Retrieved
1382:
1369:
1357:. Retrieved
1353:
1321:. Retrieved
1317:
1287:. Retrieved
1283:
1273:
1247:. Retrieved
1243:
1230:
1218:. Retrieved
1214:
1204:
1192:. Retrieved
1188:
1174:. Retrieved
1164:
1143:
1111:(5): 33–41.
1108:
1104:
1059:
1055:
1037:cite journal
1028:
1012:
996:
984:. Retrieved
980:
952:. Retrieved
947:
938:
926:. Retrieved
921:
909:
897:. Retrieved
893:
883:
871:. Retrieved
868:Ars Technica
867:
835:|first=
815:. Retrieved
810:
762:. Retrieved
758:
738:. Retrieved
734:
714:. Retrieved
710:
703:Washington:
696:
670:. Retrieved
666:
638:. Retrieved
634:
621:
609:. Retrieved
605:
592:
547:
537:
528:
492:. Retrieved
487:
419:
412:
408:
396:
393:prioritized.
368:Common Cause
358:In 2018 the
357:
354:Endorsements
348:
344:
336:
334:
325:
321:
307:
303:
299:
295:
286:
273:
258:
254:
250:
243:
228:
224:
220:
209:
205:
192:
170:
162:
158:
154:
144:
140:
138:
134:
130:
127:
90:
83:
77:
70:
64:
42:
36:
30:
26:
17:
15:
1506:Lutz, Ray.
189:Sample size
2311:Categories
1987:August 12,
1677:1006721597
1640:August 15,
1615:August 15,
1596:August 12,
1439:August 17,
1414:August 17,
1388:August 12,
1359:August 12,
1289:August 14,
1249:August 17,
1220:August 17,
1194:August 17,
1189:Justia Law
1176:August 17,
764:August 12,
748:Maryland:
740:August 12,
724:Maryland:
716:August 12,
672:August 14,
640:August 16,
557:1911.10035
494:August 12,
456:References
386:the audit.
270:Scrutineer
2322:Elections
2216:April 15,
2190:April 10,
2107:: 36–51.
1685:cite book
1635:0190-8286
1609:AllAfrica
1155:1105.5803
1113:CiteSeerX
1078:0003-4851
584:208248011
435:Elections
166:overvotes
2242:June 30,
2026:cite web
2016:April 9,
1908:April 2,
1875:cite web
1865:April 5,
1788:cite web
1728:cite web
1570:March 3,
1543:March 3,
1538:Crosscut
1323:June 29,
1259:cite web
1135:11776857
1029:Evt/Wote
986:June 29,
954:April 9,
928:April 7,
899:April 9,
873:July 27,
827:cite web
817:April 9,
682:cite web
611:April 5,
424:See also
96:problem.
2295:May 17,
115:errors.
2162:
1675:
1665:
1633:
1133:
1115:
1076:
924:. 2009
582:
572:
490:. 2018
176:Issues
2097:(PDF)
2075:(PDF)
2007:(PDF)
1977:(PDF)
1856:(PDF)
1812:(PDF)
1780:(PDF)
1751:(PDF)
1709:(PDF)
1379:(PDF)
1350:(PDF)
1314:(PDF)
1240:(PDF)
1150:arXiv
1131:S2CID
1101:(PDF)
1025:(PDF)
1009:(PDF)
977:(PDF)
918:(PDF)
807:(PDF)
755:(PDF)
731:(PDF)
663:(PDF)
631:(PDF)
602:(PDF)
580:S2CID
552:arXiv
525:(PDF)
484:(PDF)
2297:2017
2244:2019
2218:2018
2192:2018
2160:ISBN
2127:link
2058:2020
2032:link
2018:2018
1989:2019
1960:2022
1935:2022
1910:2018
1881:link
1867:2018
1838:2020
1794:link
1762:2020
1734:link
1720:2020
1691:link
1673:OCLC
1663:ISBN
1642:2019
1631:ISSN
1617:2019
1598:2019
1572:2021
1545:2021
1519:2020
1493:2020
1486:CNET
1467:2020
1441:2019
1416:2019
1390:2019
1361:2019
1325:2019
1291:2019
1265:link
1251:2019
1222:2019
1215:WCTV
1196:2019
1178:2019
1074:ISSN
1043:link
988:2019
956:2018
930:2017
901:2018
875:2012
846:link
839:help
819:2018
766:2019
742:2019
718:2019
688:link
674:2019
642:2019
613:2018
570:ISBN
496:2019
312:Cost
196:Cost
2109:doi
1123:doi
1064:doi
562:doi
246:GPS
37:not
2313::
2288:.
2234:.
2209:.
2183:.
2123:}}
2119:{{
2105:11
2103:.
2099:.
2077:.
2048:.
2028:}}
2024:{{
2009:.
1979:.
1951:.
1926:.
1900:.
1889:^
1877:}}
1873:{{
1858:.
1829:.
1790:}}
1786:{{
1753:.
1730:}}
1726:{{
1711:.
1687:}}
1683:{{
1671:.
1629:.
1607:.
1589:.
1563:.
1536:.
1510:.
1484:.
1457:.
1432:.
1406:.
1381:.
1352:.
1333:^
1316:.
1299:^
1282:.
1261:}}
1257:{{
1242:.
1213:.
1187:.
1129:.
1121:.
1109:10
1107:.
1103:.
1088:^
1072:.
1060:16
1058:.
1054:.
1039:}}
1035:{{
1027:.
1011:.
979:.
964:^
946:.
920:.
892:.
866:.
854:^
831::
829:}}
825:{{
809:.
776:^
757:.
733:.
709:.
684:}}
680:{{
665:.
650:^
633:.
604:.
578:.
568:.
560:.
527:.
504:^
486:.
464:^
370:,
366:,
362:,
203:.
16:A
2299:.
2246:.
2220:.
2194:.
2168:.
2129:)
2115:.
2111::
2060:.
2034:)
2020:.
1991:.
1962:.
1937:.
1912:.
1883:)
1869:.
1840:.
1796:)
1764:.
1736:)
1722:.
1693:)
1679:.
1644:.
1619:.
1600:.
1574:.
1547:.
1521:.
1495:.
1469:.
1443:.
1418:.
1392:.
1363:.
1327:.
1293:.
1267:)
1253:.
1224:.
1198:.
1180:.
1158:.
1152::
1137:.
1125::
1080:.
1066::
1045:)
1031:.
1015:.
990:.
958:.
932:.
903:.
877:.
848:)
841:)
821:.
768:.
744:.
720:.
690:)
676:.
644:.
615:.
586:.
564::
554::
498:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.
