Environments¶
icgym.envs ¶
Reinforcement-learning environments for the batch request game.
- :class:
BatchRequestParallelEnv- PettingZooParallelEnv: every applicant submits a request each batch; an episode repeats the batch on the same region forroundsbatches. - :class:
SingleApplicantEnv- GymnasiumEnv: one learner against rivals that follow fixed policies (for example the equilibrium, or best responses). - :class:
LocationParallelEnv- PettingZooParallelEnv: applicants choose placements; the request stage is played at its unique equilibrium. - :class:
DeviceParallelEnv- PettingZooParallelEnv: requests plus on-site generation, storage, or non-firm service.
Actions are requests as a fraction of the applicant's hosting capacity times
request_cap (so 1.0 means request_cap * h_i). Rewards are payoffs
divided by V_i h_i unless normalize_rewards=False. The info dict
carries the theory's ground truth (equilibrium requests, exploitability).
BatchRequestParallelEnv ¶
BatchRequestParallelEnv(game: BatchGame, rounds: int = 1, request_cap: float = 2.0, observation: str = 'private', normalize_rewards: bool = True, compute_exploitability: bool = False)
Bases: ParallelEnv
Repeated batch request game for several learning applicants.
Observation of applicant i (all scaled by h_i), for
observation='private': its last request, its last served load, and the
fraction of the episode elapsed. With 'public', every applicant's last
request and served load are appended.
reset ¶
reset(seed=None, options=None)
PettingZoo API: start an episode; returns observations and infos.
requests_from_actions ¶
requests_from_actions(actions) -> np.ndarray
Requests in MW from the agents' actions.
step ¶
step(actions)
PettingZoo API: play one batch; returns observations, rewards, terminations, truncations, infos.
SingleApplicantEnv ¶
SingleApplicantEnv(game: BatchGame, learner: int = 0, rival_policy: Policy = equilibrium_policy, rounds: int = 1, request_cap: float = 2.0, normalize_rewards: bool = True)
Bases: Env if gymnasium is not None else object
One learner (applicant learner) against rivals with fixed policies.
Each step is one batch: the rivals choose their requests from the last profile, the learner's action sets its request, and the learner's reward is its payoff.
LocationParallelEnv ¶
LocationParallelEnv(game, rounds: int = 1, normalize_rewards: bool = True, compute_exploitability: bool = True)
Bases: ParallelEnv
Repeated location game (PettingZoo ParallelEnv) on a :class:icgym.location.LocationGame.
Each step every applicant picks a placement from its menu (Discrete);
the request stage is solved at its unique equilibrium, so applicant i
earns margin_i * w_i(sigma), divided by game.scale unless
normalize_rewards=False. The observation is the one-hot vector of every
applicant's last placement (zeros before the first step), each applicant's
last share over game.scale, and the fraction of the episode elapsed.
info carries the shares, whether the profile is a pure equilibrium,
and each applicant's gain from its best move (compute_exploitability=True).
DeviceParallelEnv ¶
DeviceParallelEnv(game, rounds: int = 1, request_cap: float = 2.0, observation: str = 'private', normalize_rewards: bool = True, compute_exploitability: bool = False)
Bases: ParallelEnv
Repeated two-part game (PettingZoo ParallelEnv): requests plus devices or non-firm service.
Works with :class:icgym.devices.GenerationGame (request, generator or
battery) and :class:icgym.devices.NonFirmGame (firm and non-firm
requests, or a request and a curtailable fraction). Actions are in
[0, 1]^2, mapped by game.strategy_from_action; rewards are payoffs
divided by V * hosting_capacity(i). Observation: the applicant's last strategy
and served load over hosting_capacity(i), every applicant's with
observation='public', and the fraction of the episode elapsed. info
carries the strategies, served loads, the theory's prediction, and
(compute_exploitability=True) each applicant's gain from deviating.
equilibrium_policy ¶
equilibrium_policy(i: int, last_requests: ndarray, game: BatchGame) -> float
Request the predicted equilibrium (falls back to the last request).
best_response_policy ¶
best_response_policy(i: int, last_requests: ndarray, game: BatchGame) -> float
Best-respond to the other applicants' last requests.
icgym.vector ¶
Batched allocation and a vectorized environment, for fast learning.
:func:allocate_batch applies a rule to E request profiles at once (the
uniform and proportional rules are vectorized in numpy, with the minimum
allocation; other rules and implicit regions fall back to a loop).
:class:VectorBatchRequestEnv steps E independent copies of the batch
request game in one call, with numpy arrays in and out:
env = VectorBatchRequestEnv(game, num_envs=256)
obs = env.reset(seed=0) # (E, n, obs_dim)
obs, rew, done, info = env.step(acts) # acts (E, n) in [0, 1]; rew (E, n)
It has the same action map, rewards, and observations as
:class:icgym.envs.BatchRequestParallelEnv, and needs neither gymnasium nor
pettingzoo.
VectorBatchRequestEnv ¶
VectorBatchRequestEnv(game: BatchGame, num_envs: int = 64, rounds: int = 1, request_cap: float = 2.0, observation: str = 'private', normalize_rewards: bool = True)
num_envs copies of :class:icgym.envs.BatchRequestParallelEnv, stepped together.
Each copy runs rounds batches and then resets itself; done[e] marks
the step that ended copy e's episode, and the returned observation is
then the first observation of its next episode.
water_fill_batch ¶
water_fill_batch(region: Region, requests, weights=None, active=None) -> np.ndarray
:func:icgym.water_fill on each row of requests (shape (E, n)).
allocate_batch ¶
allocate_batch(rule, region, requests) -> np.ndarray
rule.allocate on each row of requests (shape (E, n)).
payoffs_batch ¶
payoffs_batch(game: BatchGame, requests) -> np.ndarray
Payoffs of E request profiles, shape (E, n).
icgym.sequential ¶
Sequential batches: a queue of applicants studied batch after batch.
Real interconnection queues run one batch after another. Here a network is a
:class:icgym.Region over buses (column j is one MW of new load at bus
j). Before each batch:
- new applicants arrive (an :class:
ArrivalProcess): each has a bus, a target sizeD(the MW it can use), and a value per MW served; - the residual region is what the network can still host after the loads connected in earlier batches and any background load growth.
In the batch, every queued applicant requests capacity; the operator's rule
serves it on the residual region; served MW connect for good. An applicant
whose request was not met in full stays in the queue for its remaining need,
paying a waiting cost, until it is connected in full or gives up after
max_tries batches. The default game keeps each applicant at its arrival bus.
Ground truth per batch: with a per-MW commitment below every value, the uniform
rule's unique equilibrium of the batch is the water-filling of the residual
region capped at each applicant's remaining need (paper: max-min fair
equilibrium, with requests bounded by need). :meth:SequentialBatchGame.myopic_equilibrium
returns it, so policies can be scored batch by batch.
Applicant
dataclass
¶
Applicant(ident: int, bus: int, need: float, value: float, arrived: int, connected: float = 0.0, tries: int = 0, paid: float = 0.0)
One queued applicant: its bus, remaining need (MW), value per MW, and history.
ArrivalProcess
dataclass
¶
ArrivalProcess(menu: list, rate: float = 2.0, size: tuple = (20.0, 80.0), value: tuple = (1.0, 1.0))
Random arrivals: Poisson(rate) applicants per batch at buses of menu.
Sizes are drawn uniformly from size (MW) and values from value
(per MW, in the same units as the fee).
BatchRecord
dataclass
¶
BatchRecord(batch: int, applicants: list, requests: ndarray, served: ndarray, payoffs: ndarray, equilibrium: ndarray, residual_capacity: float)
What happened in one batch.
SequentialBatchGame
dataclass
¶
SequentialBatchGame(region: Region, arrivals: ArrivalProcess, batches: int = 8, rule: AllocationRule = UniformRule(), fee: Fee = (lambda: PerMWCommitment(0.4))(), growth: ndarray | None = None, wait_cost: float = 0.0, max_tries: int = 4, initial: list = list())
A queue of applicants on a bus region, studied in batches consecutive batches.
growth is background load growth per batch, in MW per limit (subtracted
from the residual capacity), or None. wait_cost is charged per MW of
remaining need for each batch an applicant waits.
reset ¶
reset(seed=None) -> None
Start a new queue: nothing connected, the initial applicants queued.
residual_b ¶
residual_b() -> np.ndarray
Capacity left on each limit after connected loads and growth.
batch_region ¶
batch_region(applicants=None) -> Region
The residual region seen by the queued applicants (column i is applicant i).
myopic_equilibrium ¶
myopic_equilibrium() -> np.ndarray
Requests at the batch's equilibrium: water-filling capped at each applicant's need.
step ¶
step(requests) -> BatchRecord
Study one batch with these requests (one per queued applicant); returns its record.
simulate ¶
simulate(policy=None, seed=None) -> dict
Run every batch; policy(game) -> requests (default: the myopic equilibrium).
Returns summary metrics and the per-batch history.
summary ¶
summary() -> dict
Connected MW, MW requested but not served, waits, and the deviation from the batch equilibria.
SequentialBatchEnv ¶
SequentialBatchEnv(game: SequentialBatchGame, max_applicants: int = 64, request_cap: float = 2.0)
Bases: ParallelEnv
Sequential batches as a PettingZoo ParallelEnv with arriving and leaving agents.
Agent applicant_k is the k-th applicant to arrive (up to
max_applicants); it is in env.agents while it is queued. Action: its
request as a fraction of request_cap times its remaining need.
Observation (6 numbers): remaining need and connected MW over the original
size, the residual hosting capacity at its bus over its need, its value over
the commitment, batches waited over max_tries, and the episode elapsed.
Reward: its payoff in the batch over value * size. info carries the
batch equilibrium request for the agent.
step ¶
step(actions)
PettingZoo API: study one batch; returns observations, rewards, terminations, truncations, infos.
icgym.rule_design ¶
The operator as a learning agent: choose the rule, its minimum allocation, and the commitment.
Applicants respond strategically, so a rule's parameters have to be judged at the applicants' equilibrium, not at their truthful requests. These environments let an operator (a single RL agent, a bandit, or a search) choose them and be scored by an objective of its choice:
- :class:
OperatorEnv(Gymnasium): one batch per episode on a network drawn from a scenario distribution; - :class:
SequentialOperatorEnv(Gymnasium): a queue of batches (:class:icgym.sequential.SequentialBatchGame), parameters chosen before each batch.
Applicants answer with :func:equilibrium_response: the theory's equilibrium
where it is known (uniform rule with a per-MW commitment: the max-min fair point
capped at each applicant's need; uniform rule with a flat fee: an end of the
band [w, w/beta]), and best-response dynamics otherwise.
Objective
dataclass
¶
Objective(waste: float = 0.5, unfairness: float = 0.0)
Operator's reward: efficiency - waste * wasted_share - unfairness * (1 - jain).
- efficiency: total served over the planner's optimum;
- wasted_share: MW requested but not served, over MW requested;
- jain: Jain's index of the served shares of hosting capacity (1 = equal).
OperatorEnv ¶
OperatorEnv(scenario=None, rules=RULE_NAMES, objective: Objective | None = None, kappa_max: float = 1.0, max_applicants: int = 6, max_limits: int = 6, flat_fee_end: str = 'high')
Bases: Env if gymnasium is not None else object
One batch per episode: the operator picks (rule, beta, kappa), applicants respond at equilibrium.
Action a in [0, 1]^3: a[0] picks the rule from rules (equal
bins), a[1] is the minimum allocation beta, a[2] * kappa_max is
the commitment per MW (0 is a flat fee). Observation: the number of
applicants and of limits over their maxima, the sorted hosting capacities
over the largest (padded with zeros), the price-of-anarchy bound over 10,
and the mean value over kappa_max. Reward: objective(metrics);
info has the metrics, the requests and served loads, and the theory's
reference (the uniform rule with a commitment serves 1 / PoA of the
planner, with nothing wasted).
SequentialOperatorEnv ¶
SequentialOperatorEnv(game, rules=RULE_NAMES, kappa_max: float = 1.0, waste: float = 0.5, max_queue: int = 20)
Bases: Env if gymnasium is not None else object
The operator sets (rule, beta, kappa) before each batch of a queue.
Built on :class:icgym.sequential.SequentialBatchGame; applicants answer
each batch at the equilibrium of that batch (capped at their remaining
needs). Observation: queue length over max_queue, remaining capacity
over the initial, total remaining need over the initial capacity, and the
episode elapsed. Reward per batch: MW connected over the initial capacity,
minus waste times the MW requested but not served over the initial
capacity.
make_rule ¶
make_rule(name: str, beta: float)
Rule by name: 'uniform', 'proportional' (cut by loading), or 'utilitarian'.
equilibrium_response ¶
equilibrium_response(region: Region, rule, fee, value, need=None, flat_fee_end: str = 'low', max_rounds: int = 15, grid: int = 80) -> np.ndarray
Applicants' requests at equilibrium on region under rule and fee.
need caps the MW each applicant can use (None: unbounded). The
uniform rule with a per-MW commitment below every value gives the max-min
fair point capped at the needs; with a flat fee the band's low end w or
high end w / beta (flat_fee_end). Other cases use best-response
dynamics from the capped max-min fair point. Applicants whose value is at
most the commitment request nothing.
batch_metrics ¶
batch_metrics(region: Region, requests, served) -> dict
Efficiency, wasted share, and Jain's fairness index of one batch.