Skip to content

Environments

icgym.envs

Reinforcement-learning environments for the batch request game.

  • :class:BatchRequestParallelEnv - PettingZoo ParallelEnv: every applicant submits a request each batch; an episode repeats the batch on the same region for rounds batches.
  • :class:SingleApplicantEnv - Gymnasium Env: one learner against rivals that follow fixed policies (for example the equilibrium, or best responses).
  • :class:LocationParallelEnv - PettingZoo ParallelEnv: applicants choose placements; the request stage is played at its unique equilibrium.
  • :class:DeviceParallelEnv - PettingZoo ParallelEnv: requests plus on-site generation, storage, or non-firm service.

Actions are requests as a fraction of the applicant's hosting capacity times request_cap (so 1.0 means request_cap * h_i). Rewards are payoffs divided by V_i h_i unless normalize_rewards=False. The info dict carries the theory's ground truth (equilibrium requests, exploitability).

BatchRequestParallelEnv

BatchRequestParallelEnv(game: BatchGame, rounds: int = 1, request_cap: float = 2.0, observation: str = 'private', normalize_rewards: bool = True, compute_exploitability: bool = False)

Bases: ParallelEnv

Repeated batch request game for several learning applicants.

Observation of applicant i (all scaled by h_i), for observation='private': its last request, its last served load, and the fraction of the episode elapsed. With 'public', every applicant's last request and served load are appended.

observation_space

observation_space(agent)

PettingZoo API: the observation space of agent.

action_space

action_space(agent)

PettingZoo API: the action space of agent.

reset

reset(seed=None, options=None)

PettingZoo API: start an episode; returns observations and infos.

requests_from_actions

requests_from_actions(actions) -> np.ndarray

Requests in MW from the agents' actions.

step

step(actions)

PettingZoo API: play one batch; returns observations, rewards, terminations, truncations, infos.

SingleApplicantEnv

SingleApplicantEnv(game: BatchGame, learner: int = 0, rival_policy: Policy = equilibrium_policy, rounds: int = 1, request_cap: float = 2.0, normalize_rewards: bool = True)

Bases: Env if gymnasium is not None else object

One learner (applicant learner) against rivals with fixed policies.

Each step is one batch: the rivals choose their requests from the last profile, the learner's action sets its request, and the learner's reward is its payoff.

reset

reset(seed=None, options=None)

Gymnasium API: start an episode.

step

step(action)

Gymnasium API: play one batch against the rivals.

LocationParallelEnv

LocationParallelEnv(game, rounds: int = 1, normalize_rewards: bool = True, compute_exploitability: bool = True)

Bases: ParallelEnv

Repeated location game (PettingZoo ParallelEnv) on a :class:icgym.location.LocationGame.

Each step every applicant picks a placement from its menu (Discrete); the request stage is solved at its unique equilibrium, so applicant i earns margin_i * w_i(sigma), divided by game.scale unless normalize_rewards=False. The observation is the one-hot vector of every applicant's last placement (zeros before the first step), each applicant's last share over game.scale, and the fraction of the episode elapsed. info carries the shares, whether the profile is a pure equilibrium, and each applicant's gain from its best move (compute_exploitability=True).

profile property

profile

The current placement profile, or None before the first step.

observation_space

observation_space(agent)

PettingZoo API: the observation space of agent.

action_space

action_space(agent)

PettingZoo API: the action space of agent.

reset

reset(seed=None, options=None)

PettingZoo API: start an episode; returns observations and infos.

step

step(actions)

PettingZoo API: play one batch; returns observations, rewards, terminations, truncations, infos.

DeviceParallelEnv

DeviceParallelEnv(game, rounds: int = 1, request_cap: float = 2.0, observation: str = 'private', normalize_rewards: bool = True, compute_exploitability: bool = False)

Bases: ParallelEnv

Repeated two-part game (PettingZoo ParallelEnv): requests plus devices or non-firm service.

Works with :class:icgym.devices.GenerationGame (request, generator or battery) and :class:icgym.devices.NonFirmGame (firm and non-firm requests, or a request and a curtailable fraction). Actions are in [0, 1]^2, mapped by game.strategy_from_action; rewards are payoffs divided by V * hosting_capacity(i). Observation: the applicant's last strategy and served load over hosting_capacity(i), every applicant's with observation='public', and the fraction of the episode elapsed. info carries the strategies, served loads, the theory's prediction, and (compute_exploitability=True) each applicant's gain from deviating.

observation_space

observation_space(agent)

PettingZoo API: the observation space of agent.

action_space

action_space(agent)

PettingZoo API: the action space of agent.

reset

reset(seed=None, options=None)

PettingZoo API: start an episode; returns observations and infos.

step

step(actions)

PettingZoo API: play one batch; returns observations, rewards, terminations, truncations, infos.

equilibrium_policy

equilibrium_policy(i: int, last_requests: ndarray, game: BatchGame) -> float

Request the predicted equilibrium (falls back to the last request).

best_response_policy

best_response_policy(i: int, last_requests: ndarray, game: BatchGame) -> float

Best-respond to the other applicants' last requests.

icgym.vector

Batched allocation and a vectorized environment, for fast learning.

:func:allocate_batch applies a rule to E request profiles at once (the uniform and proportional rules are vectorized in numpy, with the minimum allocation; other rules and implicit regions fall back to a loop). :class:VectorBatchRequestEnv steps E independent copies of the batch request game in one call, with numpy arrays in and out:

env = VectorBatchRequestEnv(game, num_envs=256)
obs = env.reset(seed=0)                  # (E, n, obs_dim)
obs, rew, done, info = env.step(acts)    # acts (E, n) in [0, 1]; rew (E, n)

It has the same action map, rewards, and observations as :class:icgym.envs.BatchRequestParallelEnv, and needs neither gymnasium nor pettingzoo.

VectorBatchRequestEnv

VectorBatchRequestEnv(game: BatchGame, num_envs: int = 64, rounds: int = 1, request_cap: float = 2.0, observation: str = 'private', normalize_rewards: bool = True)

num_envs copies of :class:icgym.envs.BatchRequestParallelEnv, stepped together.

Each copy runs rounds batches and then resets itself; done[e] marks the step that ended copy e's episode, and the returned observation is then the first observation of its next episode.

reset

reset(seed=None)

Reset every copy; returns observations of shape (E, n, obs_dim).

requests_from_actions

requests_from_actions(actions) -> np.ndarray

Requests from actions in [0, 1], shape (E, n).

step

step(actions)

Step every copy; returns (obs, rewards, done, info).

water_fill_batch

water_fill_batch(region: Region, requests, weights=None, active=None) -> np.ndarray

:func:icgym.water_fill on each row of requests (shape (E, n)).

allocate_batch

allocate_batch(rule, region, requests) -> np.ndarray

rule.allocate on each row of requests (shape (E, n)).

payoffs_batch

payoffs_batch(game: BatchGame, requests) -> np.ndarray

Payoffs of E request profiles, shape (E, n).

icgym.sequential

Sequential batches: a queue of applicants studied batch after batch.

Real interconnection queues run one batch after another. Here a network is a :class:icgym.Region over buses (column j is one MW of new load at bus j). Before each batch:

  • new applicants arrive (an :class:ArrivalProcess): each has a bus, a target size D (the MW it can use), and a value per MW served;
  • the residual region is what the network can still host after the loads connected in earlier batches and any background load growth.

In the batch, every queued applicant requests capacity; the operator's rule serves it on the residual region; served MW connect for good. An applicant whose request was not met in full stays in the queue for its remaining need, paying a waiting cost, until it is connected in full or gives up after max_tries batches. The default game keeps each applicant at its arrival bus.

Ground truth per batch: with a per-MW commitment below every value, the uniform rule's unique equilibrium of the batch is the water-filling of the residual region capped at each applicant's remaining need (paper: max-min fair equilibrium, with requests bounded by need). :meth:SequentialBatchGame.myopic_equilibrium returns it, so policies can be scored batch by batch.

Applicant dataclass

Applicant(ident: int, bus: int, need: float, value: float, arrived: int, connected: float = 0.0, tries: int = 0, paid: float = 0.0)

One queued applicant: its bus, remaining need (MW), value per MW, and history.

ArrivalProcess dataclass

ArrivalProcess(menu: list, rate: float = 2.0, size: tuple = (20.0, 80.0), value: tuple = (1.0, 1.0))

Random arrivals: Poisson(rate) applicants per batch at buses of menu.

Sizes are drawn uniformly from size (MW) and values from value (per MW, in the same units as the fee).

draw

draw(rng, batch: int, start_id: int) -> list[Applicant]

New applicants for batch.

BatchRecord dataclass

BatchRecord(batch: int, applicants: list, requests: ndarray, served: ndarray, payoffs: ndarray, equilibrium: ndarray, residual_capacity: float)

What happened in one batch.

SequentialBatchGame dataclass

SequentialBatchGame(region: Region, arrivals: ArrivalProcess, batches: int = 8, rule: AllocationRule = UniformRule(), fee: Fee = (lambda: PerMWCommitment(0.4))(), growth: ndarray | None = None, wait_cost: float = 0.0, max_tries: int = 4, initial: list = list())

A queue of applicants on a bus region, studied in batches consecutive batches.

growth is background load growth per batch, in MW per limit (subtracted from the residual capacity), or None. wait_cost is charged per MW of remaining need for each batch an applicant waits.

reset

reset(seed=None) -> None

Start a new queue: nothing connected, the initial applicants queued.

residual_b

residual_b() -> np.ndarray

Capacity left on each limit after connected loads and growth.

batch_region

batch_region(applicants=None) -> Region

The residual region seen by the queued applicants (column i is applicant i).

myopic_equilibrium

myopic_equilibrium() -> np.ndarray

Requests at the batch's equilibrium: water-filling capped at each applicant's need.

step

step(requests) -> BatchRecord

Study one batch with these requests (one per queued applicant); returns its record.

simulate

simulate(policy=None, seed=None) -> dict

Run every batch; policy(game) -> requests (default: the myopic equilibrium).

Returns summary metrics and the per-batch history.

summary

summary() -> dict

Connected MW, MW requested but not served, waits, and the deviation from the batch equilibria.

SequentialBatchEnv

SequentialBatchEnv(game: SequentialBatchGame, max_applicants: int = 64, request_cap: float = 2.0)

Bases: ParallelEnv

Sequential batches as a PettingZoo ParallelEnv with arriving and leaving agents.

Agent applicant_k is the k-th applicant to arrive (up to max_applicants); it is in env.agents while it is queued. Action: its request as a fraction of request_cap times its remaining need. Observation (6 numbers): remaining need and connected MW over the original size, the residual hosting capacity at its bus over its need, its value over the commitment, batches waited over max_tries, and the episode elapsed. Reward: its payoff in the batch over value * size. info carries the batch equilibrium request for the agent.

observation_space

observation_space(agent)

PettingZoo API: the observation space of agent.

action_space

action_space(agent)

PettingZoo API: the action space of agent.

reset

reset(seed=None, options=None)

PettingZoo API: start a new queue.

step

step(actions)

PettingZoo API: study one batch; returns observations, rewards, terminations, truncations, infos.

icgym.rule_design

The operator as a learning agent: choose the rule, its minimum allocation, and the commitment.

Applicants respond strategically, so a rule's parameters have to be judged at the applicants' equilibrium, not at their truthful requests. These environments let an operator (a single RL agent, a bandit, or a search) choose them and be scored by an objective of its choice:

  • :class:OperatorEnv (Gymnasium): one batch per episode on a network drawn from a scenario distribution;
  • :class:SequentialOperatorEnv (Gymnasium): a queue of batches (:class:icgym.sequential.SequentialBatchGame), parameters chosen before each batch.

Applicants answer with :func:equilibrium_response: the theory's equilibrium where it is known (uniform rule with a per-MW commitment: the max-min fair point capped at each applicant's need; uniform rule with a flat fee: an end of the band [w, w/beta]), and best-response dynamics otherwise.

Objective dataclass

Objective(waste: float = 0.5, unfairness: float = 0.0)

Operator's reward: efficiency - waste * wasted_share - unfairness * (1 - jain).

  • efficiency: total served over the planner's optimum;
  • wasted_share: MW requested but not served, over MW requested;
  • jain: Jain's index of the served shares of hosting capacity (1 = equal).

OperatorEnv

OperatorEnv(scenario=None, rules=RULE_NAMES, objective: Objective | None = None, kappa_max: float = 1.0, max_applicants: int = 6, max_limits: int = 6, flat_fee_end: str = 'high')

Bases: Env if gymnasium is not None else object

One batch per episode: the operator picks (rule, beta, kappa), applicants respond at equilibrium.

Action a in [0, 1]^3: a[0] picks the rule from rules (equal bins), a[1] is the minimum allocation beta, a[2] * kappa_max is the commitment per MW (0 is a flat fee). Observation: the number of applicants and of limits over their maxima, the sorted hosting capacities over the largest (padded with zeros), the price-of-anarchy bound over 10, and the mean value over kappa_max. Reward: objective(metrics); info has the metrics, the requests and served loads, and the theory's reference (the uniform rule with a commitment serves 1 / PoA of the planner, with nothing wasted).

reset

reset(seed=None, options=None)

Gymnasium API: draw a network and the applicants' values.

step

step(action)

Gymnasium API: apply the operator's choice to the batch and score the equilibrium.

SequentialOperatorEnv

SequentialOperatorEnv(game, rules=RULE_NAMES, kappa_max: float = 1.0, waste: float = 0.5, max_queue: int = 20)

Bases: Env if gymnasium is not None else object

The operator sets (rule, beta, kappa) before each batch of a queue.

Built on :class:icgym.sequential.SequentialBatchGame; applicants answer each batch at the equilibrium of that batch (capped at their remaining needs). Observation: queue length over max_queue, remaining capacity over the initial, total remaining need over the initial capacity, and the episode elapsed. Reward per batch: MW connected over the initial capacity, minus waste times the MW requested but not served over the initial capacity.

reset

reset(seed=None, options=None)

Gymnasium API: start a new queue.

step

step(action)

Gymnasium API: study the next batch under the operator's choice.

make_rule

make_rule(name: str, beta: float)

Rule by name: 'uniform', 'proportional' (cut by loading), or 'utilitarian'.

equilibrium_response

equilibrium_response(region: Region, rule, fee, value, need=None, flat_fee_end: str = 'low', max_rounds: int = 15, grid: int = 80) -> np.ndarray

Applicants' requests at equilibrium on region under rule and fee.

need caps the MW each applicant can use (None: unbounded). The uniform rule with a per-MW commitment below every value gives the max-min fair point capped at the needs; with a flat fee the band's low end w or high end w / beta (flat_fee_end). Other cases use best-response dynamics from the capped max-min fair point. Applicants whose value is at most the commitment request nothing.

batch_metrics

batch_metrics(region: Region, requests, served) -> dict

Efficiency, wasted share, and Jain's fairness index of one batch.