r2b2.audit

Abstract module defining an Audit framework.

Module Contents

class r2b2.audit.PairwiseAudit(sub_contest: r2b2.contest.PairwiseContest)

Store audit information for pairwise comparison.

The PairwiseAudit class hold the audit data for a single pair of candidates.

Variables

• sub_contest (PairwiseContest) – Pairwise contest data for relevant pair of candidates.

• min_sample_size (int) – The smallest valid sample size. The minimum round size where kmin <= round

• risk_schedule (List[float]) – Schedule of risk assocaited with each previous round. Corresponds to tail of null distribution.

• stopping_prob_schedule (List[float]) – Schedule of stopping probabilities associated with each previous round. Corresponds to tail of reported tally distribution.

• pvalue_schedule (List[float]) – Schedule of pvalues associated with each previous round. Corresponds to ratio of risk and stopping probability.

• distribution_null (Dict[str, List[float]]) – Current distribution associated with a tied election for each pairwise subcontest.

• distribution_reported_tally (Dict[str, List[float]]) – Current distribution associated with reported tally for each pairwise subcontest.

• min_winner_ballots (List[int]) – List of stopping sizes (kmin values) for each round.

• stopped (bool) – Indicates if pairwise audit has stopped.

__repr__(self)

Return repr(self).

__str__(self)

Return str(self).

get_pair_str(self)

Get winner-loser pair as string used as dictionary key in Audit.

class r2b2.audit.Audit(alpha: float, beta: float, max_fraction_to_draw: float, replacement: bool, contest: r2b2.contest.Contest)

Bases: abc.ABC

Abstract Base Class to define a general Audit object type.

The Audit class is an abstract base class which defines the general structure and properties of a risk-limiting audit. Individual RLAs are subclasses of the Audit class.

Variables

• alpha (float) – Risk limit. Alpha represents the chance that given an incorrectly called election, the audit will fail to go to a full recount.

• beta (float) – the worst case chance of causing an unnecessary full recount. For many RLAs, beta will simply be set to 0 and will not appear to be a parameter.

• max_fraction_to_draw (float) – The maximum total number of ballots auditors are willing to draw during the course of the audit.

• replacement (bool) – Indicates if the audit sampling should be done with (true) or without (false) replacement.

• rounds (List[int]) – List of round sizes (i.e. sample sizes).

• sample_ballots (Dict[str, List[int]]) – Dictionary mapping candidates to sample counts drawn throughout audit. Sample counts are cumulative.

• pvalue_schedule (List[float]) – Schedule of pvalues for overall audit in each round. In each round, the overall pvalue is the maximum pvalue of all subaudits.

• contest (Contest) – Contest on which to run the audit.

• sub_audits (Dict[str, PairwiseAudit]) – Dict of PairwiseAudits wthin audit indexed by loser.

• stopped (bool) – Indicates if the audit has stopped (i.e. met the risk limit).

Create an instance of an Audit.

Note

This should only be called when initializing a subclass as the Audit class is an abstract class.

__repr__(self)

String representation of Audit class.

Note

Can (and perhaps should) be overwritten in subclass.

__str__(self)

Human readable string (i.e. printable) representation of Audit class.

Note

Can (and perhaps should) be overwritten in subclass.

current_dist_null(self)

Update distribution_null for each sub audit for current round.

_current_dist_null_pairwise(self, sub_audit: PairwiseAudit, bulk_use_round_size=False)

Update distribution_null for a single PairwiseAudit

Parameters

• sub_audit (PairwiseAudit) – Pairwise subaudit for which to update distribution.

• bulk_use_round_size (bool) – Optional argument used by bulk methods. Since the bulk methods do not sample ballots for candidates during the rounds, this flag simply uses the round schedule as the round draw (instead of the pairwise round draw) for updating the distribution. Default is False.

current_dist_reported(self)

Update distribution_reported_tally for each subaudit for current round.

_current_dist_reported_pairwise(self, sub_audit: PairwiseAudit, bulk_use_round_size=False)

Update dist_reported for a single PairwiseAudit.

Parameters

• sub_audit (PairwiseAudit) – Pairwise subaudit for which to update distriution.

• bulk_use_round_size (bool) – Optional argument used by bulk methods. Since the bulk methods do not sample ballots for candidates during the rounds, this flag simply uses the round schedule as the round draw (instead of the pairwise round draw) for updating the distribution. Default is False.

truncate_dist_null(self)

Update risk schedule and truncate null distribution for each subaudit.

_truncate_dist_null_pairwise(self, pair: str)

Update risk schedule and truncate null distribution for a single subaudit.

Parameters

pair (str) – Dictionary key for subaudit (within the audit’s subaudits) to truncate distribution and update risk schedule.

truncate_dist_reported(self)

Update stopping prob schedule and truncate reported distribution for each subaudit.

_truncate_dist_reported_pairwise(self, pair)

Update stopping prob schedule and truncate reported distribution for a single subaudit.

Parameters

pair (str) – Dictionary key for subaudit (within the audit’s subaudits) to truncate distribution and update stopping prob schedule.

__get_interval(self, dist: List[float])

Get relevant interval [l, u] of given distribution.

Find levels l and u such that cdf(l) < tolerance and 1 - cdf(u) < tolerance. The purpose of this is to improve efficiency in the current_dist_* functions for audits without replacement where almost all of the hypergeometric distribution falls in a fraction of its range, i.e. between l and u.

Note

cdf() in this context does not require cdf(infinity) = 1, although the distribution should sum very closely to 1.

asn(self, pair: str)

Compute ASN as described in BRAVO paper for pair of candidates.

Given the fractional margin for the reported winner and the risk limit (alpha) produce the average number of ballots sampled during the audit.

Parameters

pair (str) – Dictionary key referencing pairwise audit in audit’s subaudits.

Returns

int – ASN value.

execute_round(self, sample_size: int, sample: dict, verbose: bool = False) → bool

Execute a single, non-interactive audit round.

Executes 1 round of the audit, given its current state. If the audit is stopped, its state will not be modified.

Warning: This method assumes the audit is in the correct state to be executed. If multiple

executions of a full audit will be run the same audit object, make sure to call reset on the audit object between full executions.

Parameters

• sample_size (int) – Total ballots sampled by the end of this round (cumulative).

• sample (dict) – Sample counts for each candidate by the end of this round (cumulative).

Returns

bool – True if the audit met its stopping condition by this round.

run(self, verbose: bool = False)

Begin interactive audit execution.

Begins the interactive version of the audit. While computations for different audits will vary, the process for executing each one is the same. This provides a process for selecting a sample size, determining if the ballots found for the reported winner in that sample size meet the stopping condition(s), and if not continuing with the audit. As the audit proceeds, data including round sizes, ballots for the winner in each round size, and per round risk and stopping probability are stored.

_reset(self)

Reset attributes modified during run().

abstract get_min_sample_size(self, sub_audit: PairwiseAudit)

Get the minimum valid sample size in a sub audit

Parameters

sub_audit (PairwiseAudit) – Get minimum sample size for this sub_audit.

abstract next_sample_size(self, *args, **kwargs)

Generate estimates of possible next sample sizes.

Note: To be used during live/interactive audit execution.

stopping_condition(self, verbose: bool = False) → bool

Determine if the audits stopping condition has been met.

The audit stopping condition is met if and only if each pairwise stopping condition is met.

abstract stopping_condition_pairwise(self, pair: str, verbose: bool = False) → bool

Determine if pairwise subcontest meets stopping condition.

Parameters

pair (str) – Dictionary key referencing pairwise audit in audit’s sub_audits.

Returns

bool – If the pairwise subaudit has stopped.

next_min_winner_ballots(self, verbose: bool = False)

Compute next stopping size of given (actual) sample sizes for all subaudits.

abstract next_min_winner_ballots_pairwise(self, sub_audit: PairwiseAudit) → int

Compute next stopping size of given (actual) sample size for a subaudit.

Parameters

sub_audit (PairwiseAudit) – Compute next stopping size for this subaudit.

Note: To be used during live/interactive audit execution.

abstract compute_min_winner_ballots(self, sub_audit: PairwiseAudit, progress: bool = False, *args, **kwargs)

Compute the stopping size(s) for any number of sample sizes for a given subaudit.

abstract compute_all_min_winner_ballots(self, sub_audit: PairwiseAudit, progress: bool = False, *args, **kwargs)

Compute all stopping sizes from the minimum sample size on for a given subaudit.

abstract compute_risk(self, sub_audit: PairwiseAudit, *args, **kwargs)

Compute the current risk level of a given subaudit.

Returns

Current risk level of the audit (as defined per audit implementation).

abstract get_risk_level(self, *args, **kwargs)

Find the risk level of the audit, that is, the smallest risk limit for which the audit as it has panned out would have already stopped.

Returns

float – Risk level of the realization of the audit.